Title
#general
a

Austin Burnett

11/15/2019, 4:05 PM
for process events - if you have a scheduled query that runs more frequently than daily - is it generally suggested to set your
--events_expiry
to the frequency of that query (or
1
- but in this case we might have multiple queries pulling from
process_events
)?
packetzero

packetzero

11/15/2019, 4:39 PM
yes, you can have multiple queries pulling from same events table. They don't have to be the same interval, but the current code doesn't handle it perfectly, so I recommend using same interval. The recommendation for the events_expiry=1 is for performance reasons. The events are cached to disk as they come in, then read when queries run. If you only want your queries to read them once, then you ideally want the cache to clear after they run. The events_expiry=1 will wait until all queries for that table have run before clearing. It keeps a count of number of queries depending on table.
a

Austin Burnett

11/15/2019, 4:50 PM
thanks! we were seeing a problem with disk usage being abnormally high after enabling process events but hadn't set expiry - so I think we were always holding on to a day's worth of events, so I was looking at the best setting for multiple queries but it looks like we'll just be able to use events_expiry=1
packetzero

packetzero

11/15/2019, 5:09 PM
Chris long had some good details on event tables and expirations in his audit talk a while back:

https://youtu.be/AIO7mgVt4O8?t=925

a

Austin Burnett

11/19/2019, 7:04 PM
thanks for this video - lots of good stuff in here! would you be able to link me to the code that handles the events_expiry stuff? I'm wondering specifically in reference to
events_expiry=1 will wait until all queries for that table have run before clearing
- if we have one query that say runs every 10 minutes, and one that runs once a day - will the event expiry essentially be a day in that case?
packetzero

packetzero

11/20/2019, 1:00 AM
It’s supposed to function like that.1. expireRecords() checks for executedAllQueries() https://github.com/osquery/osquery/blob/93ceef7e5f1c7e17be63e4d7a58c641bc8031474/osquery/events/events.cpp#L235 2. executedAllQueries() checks to see if queries_.size() is equal to query_count_ , which is set during config update. https://github.com/osquery/osquery/blob/93ceef7e5f1c7e17be63e4d7a58c641bc8031474/osquery/events/events.cpp#L383 3. When query is executed, inserts the query name into queries_ map : https://github.com/osquery/osquery/blob/93ceef7e5f1c7e17be63e4d7a58c641bc8031474/osquery/events/events.cpp#L135