Hey folks, how are you generating alerts with your osquery logs? Would it be useful to have this functionality directly in Fleet?

➕ on this. We have a lambda function that pulls the results out of ECS cloudwatch logs and sends them to Streamalert, but would love to have that integration, or even to be able to set up alerts on

missing

or

offline

hosts

Currently our logs are sent to our central logging platform via GCP PubSub where we have some alerts. We’re looking at building out a more flexible alerting pipeline by adding additional subscribers to the PubSub topic, but if that was built into Fleet that would save us a lot of time 🙂

Just being able to have more ways to get the results from fleet to somewhere else would be choice.

Yes, I think that would he helpful to have. We'll be using Splunk to generate our alerts, but having the flexibility to do that also from Fleet would be helpful, especially if we decide to somehow limit what we send to Splunk due to cost.

Thanks for the responses! Where would you want alerts to go? Email? Webhook? Others?

Both of those, along with a way to template the alert message. The webhook should also support headers, so authorization tokens, etc. can be included

➕ to the above.

If you’re feeling really nice, pre-defined webhooks with the required fields clearly laid out for Slack, Teams, and others are nice as it makes it harder to mess up the configuration 🙂 But of course, if there’s a generic webhook feature, the community can pretty easily add those as needed

This is great, please keep the ideas coming.

In our environment we collect logs via fleet which are sent over to elasticsearch via filebeat and from there we generate alerts on it using elastalert

@zwass If you’re up for it, the alerting engine could use OPA’s Rego language (https://www.openpolicyagent.org/docs/latest/policy-language/). I’ve used it pretty extensively now for writing Kubernetes policies and while the language has a small learning curve, it is a real pleasure to use for making decisions on structured data. It wouldn’t be hard to adapt for this case, and OPA has some nice features such as built-in ability to make HTTP requests to remote endpoints and/or sync other data into its cache to be used in the decision making, allowing for much more complex policies (aka alerts in this context) outside of relying solely on the data returned from osquery on the endpoint

+ 1 to OPA

At a glance Rego looks hard to learn for someone used to writing SQL statements. I had been looking at https://github.com/antonmedv/expr which seems much more intuitive.

@zwass We send the osquery logs to elasticsearch, and then use the Sigma toolkit to generate an ElastAlert rule (https://github.com/Neo23x0/sigma)