• we use benchmarks to create artificial constant rate of process events per second • we monitor CPU, RAM, & diskIO • we monitor the osquery logs to see when event buffers overflow one finding i wanted to share - we set

snapshot=true

on our eventing queries and measured a 5X increase in the events per second throughput of

process_events

before we saw events buffers overflow

Do your queries on

process_events

look similar to

select * from process_events

or are they more complex?

they are more complex. we join on stuff like process table & process_containers table

ah, cool, I asked because I think there's some under the hood logic to detect simple queries against

_events

tables and turn on

snapshot=True

for you (I am going from memory though)

we have about the same performance whether we join to the tables or not

this is worth investigating, thanks for sharing the analysis!

One thing that this brings up for me... If someone has

events_optimize

on, then we can probably skip the diffing step on a differential query to an event-based table... the results should all be "added".

@zwass - kinda naively assumed that was how things were working. we had

removed=true

on our event queries and had

events_optimize=true

. it surprised me how much of a difference

snapshot=true

made for the audit data.

The major difference in perf I would anticipate for snapshot vs. differential would be the diffing step. Both generate the results in approximately the same way. Someone would have to have special-cased the diffing for event-based tables -- I'm not sure that has been done.

my read says it hasn’t been

Seems ripe for doing!

there is code related to

events_optimize

that introspects the query on a specific event table. potentially a cheap spot to simply flip the

snapshot

option on for the

ScheduledQuery

It's an interesting idea. snapshot vs. diff queries have different log schema so that could be an issue.

a doc update recommending

snapshot

would even go a long way