https://github.com/osquery/osquery logo
Title
z

Zach Zeid

05/20/2020, 12:30 PM
what is the implication of setting
removed: false
in a query? I'm having some difficulty understanding what the expected behavior should be.
so my understanding is that if I have
removed: false
as a configuration item in a query, then it'll only send a single line at a time whenever there is a change that is captured by the query?
t

terracatta

05/20/2020, 1:28 PM
removed false is only relevant for differential queries, it causes only "added" rows to be emitted to the log and will not report rows that are removed. This is useful when you don't really care about tracking when existing rows are removed from the result set when the queries is run over time.
z

Zach Zeid

05/20/2020, 1:29 PM
what is an example of a differential query?
something like
select * from process;
?
t

terracatta

05/20/2020, 1:34 PM
yeah
any query can be run in differential mode
z

Zach Zeid

05/20/2020, 1:35 PM
so if I had that query it would essentially just tell me when new processes are created?
t

terracatta

05/20/2020, 1:35 PM
right, and if you have removed: false you would not see when they were stopped
this is useful if you only need the output to see if certain programs are running, but don't care if certain programs are NOT running
z

Zach Zeid

05/20/2020, 1:36 PM
and if I didn't have the query I'd see both
added
and
removed
in the results of the query that would indicate when a process is started or shut down?
t

terracatta

05/20/2020, 1:37 PM
right
z

Zach Zeid

05/20/2020, 1:37 PM
Awesome, that's incredibly helpful, thank you!
t

terracatta

05/20/2020, 1:37 PM
are you the same Zach Zeid that worked at Mandiant/FEYE? If so, it's Jason Meller, not sure if you remember me.
z

Zach Zeid

05/20/2020, 1:37 PM
oh snap! what's up!
t

terracatta

05/20/2020, 1:38 PM
Haha, I was like, no way two people have an awesome double Z name like that.
z

Zach Zeid

05/20/2020, 1:39 PM
We're a rare kind for sure 😄
t

terracatta

05/20/2020, 1:40 PM
Anyway, the osquery docs are actually super good at explaining this stuff. https://osquery.readthedocs.io/en/latest/deployment/logging/ is worth a read
z

Zach Zeid

05/20/2020, 1:41 PM
That's what I was reading, I wanted to make sure I was understanding it correctly. Right now, we're doing snapshots, but want to move towards a more "continuous" monitoring with osquery.
t

terracatta

05/20/2020, 1:43 PM
What a lot of people do is they continue to use snapshots (they just run them much less frequently) but also use diffs as well for the same queries on a more frequent schedule. That way when you emit results to wherever (splunk, ELK) you can still see the full results in the log output every 4 or 6 hours, if you want to be sure of the current state, but the diff covers you for incremental changes in-between those snapshots.
z

Zach Zeid

05/20/2020, 1:45 PM
if I don't include either
removed: false
or
snapshot: true
do they default to specific values?
t

terracatta

05/20/2020, 1:45 PM
yes, diff is the default, and by default it will show both add/remove events
z

Zach Zeid

05/20/2020, 1:50 PM
Excellent, that's good to know, thanks!