what is the implication of setting

removed: false

in a query? I'm having some difficulty understanding what the expected behavior should be.

removed false is only relevant for differential queries, it causes only "added" rows to be emitted to the log and will not report rows that are removed. This is useful when you don't really care about tracking when existing rows are removed from the result set when the queries is run over time.

what is an example of a differential query?

yeah

so if I had that query it would essentially just tell me when new processes are created?

right, and if you have removed: false you would not see when they were stopped

and if I didn't have the query I'd see both

added

and

removed

in the results of the query that would indicate when a process is started or shut down?

right

Awesome, that's incredibly helpful, thank you!

are you the same Zach Zeid that worked at Mandiant/FEYE? If so, it's Jason Meller, not sure if you remember me.

oh snap! what's up!

Haha, I was like, no way two people have an awesome double Z name like that.

We're a rare kind for sure 😄

Anyway, the osquery docs are actually super good at explaining this stuff. https://osquery.readthedocs.io/en/latest/deployment/logging/ is worth a read

That's what I was reading, I wanted to make sure I was understanding it correctly. Right now, we're doing snapshots, but want to move towards a more "continuous" monitoring with osquery.

What a lot of people do is they continue to use snapshots (they just run them much less frequently) but also use diffs as well for the same queries on a more frequent schedule. That way when you emit results to wherever (splunk, ELK) you can still see the full results in the log output every 4 or 6 hours, if you want to be sure of the current state, but the diff covers you for incremental changes in-between those snapshots.

if I don't include either

removed: false

or

snapshot: true

do they default to specific values?

yes, diff is the default, and by default it will show both add/remove events

Excellent, that's good to know, thanks!