Title
#general
p

Prakhar

07/20/2018, 7:44 AM
Can we query *_events table more than once for same set of data ? Are events deleted from table as soon as they are first queried or they do persist in table as per --events-expiry flag value ?
v

Vibhor

07/20/2018, 2:26 PM
If you have configured osquery to send the events through the backend, then it will retain the events till it sends the events to a backend, like Uptycs (osquery sends events to Uptycs as soon as it occurs, doesn’t wait for scheduled query packs to read the events). As soon as you perform SELECT on events *_events table, osquery will flush all the events. About --events-expiry, flag is only for keeping the events if you haven’t query the events table. However, osquery may remove the events; if osquery memory is full and new events come, then based on LIFO, it may remove the events. In the case of Uptycs, osquery sends the events directly to Uptycs as it occurrs in near real time.
m

Mustafa

07/20/2018, 2:43 PM
are you sure that osquery will flush the events after they queried ?
v

Vibhor

07/20/2018, 3:20 PM
@Mustafa I re-tested this and I can confirm that SELECT on *_events table in interactive mode flush all the data
3:21 PM
Please let me know if you have different observations
3:23 PM
We can re-check the version and validate behavior one more time
r

Rich5

07/20/2018, 4:25 PM
I thought only expired events were flushed at query time So you could query an events table as many times as you want and get the same results (plus new entries) as long as the entries are not expired. Is that not right?
m

Mustafa

07/20/2018, 4:26 PM
Thanks vibhor. What do you mean ... Interactive mode
Mike Myers

Mike Myers

07/20/2018, 9:51 PM
I think he meant osqueryi, which is the interactive mode of osquery (as opposed to osqueryd)
m

Mustafa

07/21/2018, 1:52 AM
Hmm but is it important that osqueryi flush all the events after they have been queried? Isn't it important that osqueryd behaviour for rockdb?
clippy

clippy

07/23/2018, 4:57 AM
Just scrolling through and catching up and saw this thread. The above answer is, I believe, incorrect. In osqueryi (what you guys are calling interactive modde) the osqueryi process is detached from the persistent database, meaning that only events that occur while the interactive shell is open will be recorded and be visible. Things work differently with osqueryd, which will store things in the database. Complicating maters further is that that --events-expiry flag only flushes events on SELECT, IF events-expiry=1. For all other values, events will persist for a longer period of time. This also depends on what your events-max is set to, as events will be pushed off teh stack if events_max is reached. Events which have been SELECTED at least once are available to be expired, (think of them as being marked as "read") but are not necessarily flushed
4:58 AM
This is a rather complicated topic though, TBH. Suggested watching

https://www.youtube.com/watch?v=AIO7mgVt4O8

r

Rich5

07/23/2018, 11:11 AM
@clippy This is how I understood the events tables to work too. Thanks for explaining
v

Vibhor

07/23/2018, 2:14 PM
@clippy Thanks for sharing the Video. Interestingly, my environment shows other wise. It may be a bug or something else. I will re-check.