Also, would the experimental bpf based implementation help in this regard so that we can rule out most of noise in the kernel space and not let it enter the full blown processing within osquery. I would be really interested in that.
PS: I am not using the managed audit config and letting osquery manage the audit config as we kind of need some globbing patterns in the file patterns.
I got really interested into the bpf stuff when I came across it and then saw this PR and got even more excited about it 😄 .
Was there a significant resource utilization improvement while using bpf with the process_events table.. ? if yes, then that might be a good route, atleast for the future.
10/06/2020, 7:19 PM
You can try out the packages i uploaded in the #ebpf channel!
i will update them with the next release 4.5.1
But back to memory usage, depends on the hardware
we have to use per-cpu maps to store data
it's 4 buffer storages (to store strings like paths or buffers like sockaddr_in) of 4096 (configurable) slots of 4k bytes each (not configurable) for each cpu core
plus the perf_events ring buffer of 2^12 (configurable) bytes, also per cpu core
there is also an additional small map for each syscall we trace but the size is really tiny
so if you have a monster machine with 64 core then it's going to use quite a lot of memory
main issue is that with bpf you can't block or pause the execution forever, so the best approach is to just save data without having to rely on locks
so we basically use per-cpu maps that basically save data like this: map[cpu_index][key] = value