Clarification on Bro and Suricata logging to Kafka

Hello,

I would like to understand the Bro and Suricata logging to Kafka better if possible. Specifically I am a bit confused as to why logs for Bro and Suricata are written to disk and to Kafka. Is there a specific reason to write to disk and Kafka or is it possible to turn off the logging to disk and just have the logs written only to Kafka?

Thank you,

Clayton

For production sensors, I actually recommend you disable ASCII logs to disk with Bro. We actually include a script to do that, but lots of people like those logs, so we leave them on by default (for now).

You can disable them by adding the following to /usr/share/bro/site/local.bro.

@load scripts/rock/frameworks/logging/disable-ascii

As for Suricata, it’s trickier because Suricata can’t natively write to Kafka. We have some experimental code that will allow us to disable writing to disk, but it hasn’t been tested at scale yet. In the meantime, we have to rely on Filebeat to pick that up and ship to Kafka. As soon as that’s rock solid, we’ll ship that.

Awesome, thank you for the quick reply, I’ll be sure to add disable-ascii to my site imports!