https://github.com/osquery/osquery logo
d

defensivedepth

11/29/2022, 6:42 PM
In relation to ETW events, recent windows process PR. Does ETW have any concept of filtering? Is there any possibility of defining a capture filter ie
exclude processes that look like svchost.exe -k
@Marcos Oviedo et al.
m

Marcos Oviedo

11/29/2022, 6:48 PM
Yes, there could be filtering happening on the event properties (i.e. process_name in this case). The ETW tracing engine (POTE) could support that filtering in its post-processing stage. @zwass and I briefly talked about the idea of having filtering support but didn't decide on how to move this fwd. Should filtering be configured through a file-based configuration (similar to an FIM-like configuration) ?
d

defensivedepth

11/29/2022, 7:08 PM
Ya, @zwass and I have talked about this a few times over the years... I've even tried auto-generating a query to filter against the current non-evented win process events table - uses SwiftOnSecurity sysmon filter as a source - https://github.com/defensivedepth/osquery-filters I think any kind of functionality that we can build into it that allows some type of filtering like sysmon.
Probably lends itself to file-based config
z

zwass

11/29/2022, 7:09 PM
cc @alessandrogario have you thought about filtering on ingestion for the BPF tables you are working on?
a

alessandrogario

11/29/2022, 7:11 PM
Would this be pre- or post-capture filtering in the context of BPF?
z

zwass

11/29/2022, 7:12 PM
I suppose this would be pre-capture. I wonder if we could find a good approach to unifying this kind of configuration across tables/platforms.
a

alessandrogario

11/29/2022, 7:15 PM
We are kind of limited on what we can do in BPF land. With the current core BPF tables we are generating the probe logic at runtime (based on the syscall prototypes/tracepoint descriptors on the system). With the new PR, the logic is written in a .c file that gets embedded within osquery and is read on startup.
A way to mix the two could be to add back a code generator that based on the filter decides to call or skip the second, .c BPF probe
But depending on what the filter looks like it may be hard as we have several limitations when we are inside BPF
One possible approach is to split the generic configuration in two parts, pre- and post-capture to make up for the technical limitations. This is assuming the filter only works on the fields that are returned in the table. If filtering requires extra data from other places, then it gets a little harder
(other systems could "lower" that configuration entirely in the pre-capture if possible)
We are kind of at disadvantage on Linux though, modern deployments work with containers and so osquery gets flooded with events if we are not careful
I think I'd like to start with a restricted but useful set of event fields and expand from there once we can verify that users can find it usable on their endpoints
m

Marcos Oviedo

11/29/2022, 8:19 PM
| I've even tried auto-generating a query to filter against the current non-evented win process events table @defensivedepth that looks really cool! Did you manage to translate all the sysmon filtering primitives to SQL? We can either go this way and provide a tool to convert sysmon configuration into SQL sentences so they can be used on osquery, OR we can think of supporting a common filtering language through configuration. I explored the idea of building an open-source drop-in replacement of sysmon in the past (sysmonx), where I implemented all of the sysmon filtering primitives. Most of these primitives were already provided by boost (see here and here for examples), so the mimicking sysmon filtering shouldn't be complicated. Another filtering language to consider would be the Event Query Language (EQL) from Elastic. This filtering/threat-hunting language gained a lot of adoption in the security space as it was designed for security use cases. EQL syntax reference is here. One significant difference between these two filtering languages is that Sysmon filtering allows you to filter atomic events, and EQL allows you to express relationships between events. This is key if you want to capture suspicious behaviors rather than isolated events. An example of suspicious behavior will be winword.exe launching cmd.exe, and this cmd.exe instance is later running whoami.exe. If we see that filtering through configuration is something we would like to support, I think it will be important to have a standard filtering mechanism that can be used across all of the evented tables.
d

defensivedepth

11/29/2022, 9:23 PM
No, I didnt get them all done, but was still able to reduce rows returned by 30%+ just with known good stuff.
Yes, I think a standard filtering mechanism across all evented tables would be key. That was one of the primary reasons why Sysmon gained traction so quickly - the ability to use a standard filter at the capture point, across multiple data sources.
m

Marcos Oviedo

11/29/2022, 11:06 PM
I agree. We can think of handling this as a separate feature. We can start by implementing a sysmon-like filtering approach and keep it simple for the time being.
d

defensivedepth

12/02/2022, 7:29 PM
Would love to be involved in discussions around this if it moves forward
m

Marcos Oviedo

12/02/2022, 7:35 PM
Definitively! Something that would be good to define at this point will be the new evented tables to add on top of POTE. This framework will simplify the process of adding new ETW-based Osquery Event Publishers, so having a list of areas we might want to provide visibility into would be great to prioritize this work For example, from the top of my head, the new event tables I'd love to see going live are: • DNS monitoring • DLL Loading • Network Connections • Registry Monitoring • File Monitoring • Code Integrity
we should track this somewhere, along with the target usecases and the ETW provider source for these events
d

defensivedepth

12/02/2022, 8:28 PM
Drop it in the Blueprint issue?
or somewhere else
m

Marcos Oviedo

12/02/2022, 9:36 PM
good point, I'm going to create a different issue to capture and prioritize these tables
@defensivedepth done! I've just created this issue for tracking purposes - https://github.com/osquery/osquery/issues/7836
2 Views