Title
#general
Macear

Macear

07/10/2020, 11:07 AM
Hi everyone! When working with *_events tables in osquery, should I always select everything from those tables to flush events from RocksDB? Otherwise /var/osquery/osquery.db directory will grow in size, which as I understand affect the osquery performance. I transport osquery events with syslog. The problem is when select * I lose some data because of suppressed messages by systemd therefore I’d like to change select * to get only required data. But i don’t want my RocksDB grow in size and never drops old events from *_events tables. Correct me I’m wrong because I doubt that osquery works as I understand it.
CptOfEvilMinions

CptOfEvilMinions

07/10/2020, 5:28 PM
Osquery has the ability to configure expiration of events.
--events_optimize=true
 apply optimizations when SELECTing from events-based tables, enabled by default. - Events you select with your slimed down query will be. deleted.
--events_expiry
the lifetime of buffered events in seconds with a default value of 86000. - This option should expire ALL events after the threshold is met
--events_max
the maximum number of events to store in the buffer before expiring them with a default value of 1000 - So only 1000 events will be stored in RocksDB.
Macear

Macear

07/10/2020, 6:37 PM
@CptOfEvilMinions thanks you, what if set events_expiry option to 1. In documentation I see that events are going to be deleted only by one select query against such data.What if I never select those events? They will be stored eternally?
CptOfEvilMinions

CptOfEvilMinions

07/10/2020, 7:18 PM
Without testing, I’m not sure which of the options above would take precedence. I would ASSUME that if you set
--events_expiry=1
and set
--events_max=1,000
this combination would keep the RocksDB from eternally growing. A quick test case would be to enable process monitoring with
--events_expiry=1
and set
--events_max=10
and then do:
SELECT * FROM process_events
Macear

Macear

07/11/2020, 7:40 AM
@CptOfEvilMinions thx! Will try to inspect