Title
#general
Mystery Incorporated

Mystery Incorporated

10/28/2020, 6:31 PM
Expiring events for subscriber: windows_events (overflowed limit 50000)
zwass

zwass

10/28/2020, 6:34 PM
Looks like you are perhaps not actually selecting against the
windows_events
table? That would clear out the buffered events into logs which could be shipped off elsewhere.
Mystery Incorporated

Mystery Incorporated

10/28/2020, 6:58 PM
My query is select * from windows_events
6:58 PM
But seems to not be emptying the table
a

alessandrogario

10/28/2020, 6:59 PM
Ah I see, so you are using events; what's your --events_expiry set at?
Mystery Incorporated

Mystery Incorporated

10/28/2020, 7:00 PM
It seems I have not set an expirey which is likely mu problem
a

alessandrogario

10/28/2020, 7:01 PM
It shouldn't be a problem though, event optimization (default enabled) will not return the same data twice, so it won't log duplicates
Mystery Incorporated

Mystery Incorporated

10/28/2020, 7:01 PM
or actually would it be a problem if i'm fetching the events every 10 seconds
a

alessandrogario

10/28/2020, 7:01 PM
That message is not an error, just an information
7:02 PM
if you wish to use the same table from different queries, it's not ideal to expire everything on the first query
Mystery Incorporated

Mystery Incorporated

10/28/2020, 7:02 PM
Ah yes I see because data will be gone for query 2
7:03 PM
bit somehow I am ending up with 50000 entries in the table
7:04 PM
does osqueryi and osqueryd share same db?
a

alessandrogario

10/28/2020, 7:04 PM
that is normal, without expiration you will keep getting new events added at the end, and expiration will remove them from the start of the queue
7:05 PM
once the max amount of events (--events_max), it will remove events on one side as new ones are added to the other side
7:06 PM
when using scheduled queries + osqueryd, event optimization will make sure that only new events are added (i.e. you will only log new rows added since the last query)
Mystery Incorporated

Mystery Incorporated

10/28/2020, 7:06 PM
so to alleviate that warning I should set max to say 49000 then?
a

alessandrogario

10/28/2020, 7:06 PM
osqueryi and osqueryd are separate
7:06 PM
it's not a warning, it's just an informational message
7:06 PM
everything is working as intended
Mystery Incorporated

Mystery Incorporated

10/28/2020, 7:07 PM
overflowed is usually bad isn't it?
a

alessandrogario

10/28/2020, 7:07 PM
if you do not wish to query the same table from multiple queries, you can set the events_expiry so that the first query will remove them
7:07 PM
no it's just telling you that it's performing the cleanup, nothing bad is happening
Mystery Incorporated

Mystery Incorporated

10/28/2020, 7:07 PM
oh rightio
7:07 PM
it's cleaning up at 50000