I am also experiencing the race condition mentioned on git where filtered events get collected. Is there a way to reduce this, at times the amount of logs collected causes the extension to crash. And it seems even after allowing time for the race condition to correct I see events that should have been filtered being collected by the extension.
i am using extension version 3.5.1
05/18/2022, 5:14 AM
please share the filter applied and details of event that didnt get filtered. thanks
Almost every time the query runs I get process that I have excluded such as splunk, datadog, and amazon ssm agent
05/19/2022, 1:58 AM
thanks. we'll check the query and filters.
could it be possible that the events from processes splunk, datadog, and amazon ssm agent were generated before event filters got into action? And later you are seeing those same set of events again and again in your query?
05/19/2022, 1:40 PM
yes, it continues to show filtered results each 60 second cycle. This also seems to be causing CPU exhaustion and I start seeing warnings that I am above the low watermark and the extension stops producing records.
when these events are collected without a filter it can be 10's of thousands of results
05/19/2022, 1:52 PM
if any query is run repeatedly on EIQ agent evented tables such as win_socket_events or win_process_events, the results fetched each time would have the complete set of filtered events based on the query. so you may see same set of event data again and again, it doesnt do any diff from previous results and doesnt return any delta of events in next query run.
05/19/2022, 2:12 PM
I understand that and it is not just the repeted events it is the fact that they should be filtered and not showing up. Most of the results should be excluded due to filtering. I understand the race condition stated in the docs but I see this behavior after startup and continues during subsequent queries
05/19/2022, 2:31 PM
filtering applies in the kernel component of the extension when extension is started and prevents the events being logged to the extension's event viewer channel. but due to race in applying filters, if some events have already logged to the event viewer by the kernel component, the queries on evented tables will read those event records from the event viewer channel and will keep sending them in subsequent queries also. hope it clarifies.