Hey folks, how are you generating alerts with your osquery logs? Would it be useful to have this functionality directly in Fleet?
10/07/2020, 12:05 PM
➕ on this. We have a lambda function that pulls the results out of ECS cloudwatch logs and sends them to Streamalert, but would love to have that integration, or even to be able to set up alerts on
10/07/2020, 3:14 PM
Currently our logs are sent to our central logging platform via GCP PubSub where we have some alerts. We’re looking at building out a more flexible alerting pipeline by adding additional subscribers to the PubSub topic, but if that was built into Fleet that would save us a lot of time 🙂
Just being able to have more ways to get the results from fleet to somewhere else would be choice.
10/08/2020, 8:53 PM
Yes, I think that would he helpful to have. We'll be using Splunk to generate our alerts, but having the flexibility to do that also from Fleet would be helpful, especially if we decide to somehow limit what we send to Splunk due to cost.
10/08/2020, 9:41 PM
Thanks for the responses! Where would you want alerts to go? Email? Webhook? Others?
10/08/2020, 9:52 PM
Both of those, along with a way to template the alert message. The webhook should also support headers, so authorization tokens, etc. can be included
If you’re feeling really nice, pre-defined webhooks with the required fields clearly laid out for Slack, Teams, and others are nice as it makes it harder to mess up the configuration 🙂 But of course, if there’s a generic webhook feature, the community can pretty easily add those as needed
10/08/2020, 10:10 PM
This is great, please keep the ideas coming.
10/08/2020, 10:17 PM
In our environment we collect logs via fleet which are sent over to elasticsearch via filebeat and from there we generate alerts on it using elastalert
For alerts we use Slack Webhooks
10/08/2020, 11:18 PM
@zwass If you’re up for it, the alerting engine could use OPA’s Rego language (https://www.openpolicyagent.org/docs/latest/policy-language/). I’ve used it pretty extensively now for writing Kubernetes policies and while the language has a small learning curve, it is a real pleasure to use for making decisions on structured data. It wouldn’t be hard to adapt for this case, and OPA has some nice features such as built-in ability to make HTTP requests to remote endpoints and/or sync other data into its cache to be used in the decision making, allowing for much more complex policies (aka alerts in this context) outside of relying solely on the data returned from osquery on the endpoint