Hi everyone! When working with *_events tables in osquery, should I always select everything from those tables to flush events from RocksDB? Otherwise /var/osquery/osquery.db directory will grow in size, which as I understand affect the osquery performance. I transport osquery events with syslog. The problem is when select * I lose some data because of suppressed messages by systemd therefore I’d like to change select * to get only required data. But i don’t want my RocksDB grow in size and never drops old events from *_events tables. Correct me I’m wrong because I doubt that osquery works as I understand it.

Osquery has the ability to configure expiration of events.

--events_optimize=true

 apply optimizations when `SELECT`ing from events-based tables, enabled by default. - Events you select with your slimed down query will be. deleted.

--events_expiry

the lifetime of buffered events in seconds with a default value of 86000. - This option should expire ALL events after the threshold is met

--events_max

the maximum number of events to store in the buffer before expiring them with a default value of 1000 - So only 1000 events will be stored in RocksDB.

@CptOfEvilMinions thanks you, what if set events_expiry option to 1. In documentation I see that events are going to be deleted only by one select query against such data.What if I never select those events? They will be stored eternally?

Without testing, I’m not sure which of the options above would take precedence. I would ASSUME that if you set

--events_expiry=1

and set

--events_max=1,000

this combination would keep the RocksDB from eternally growing. A quick test case would be to enable process monitoring with

--events_expiry=1

and set

--events_max=10

and then do:

SELECT * FROM process_events

@CptOfEvilMinions thx! Will try to inspect