hi team I am trying to use osquery to get continuously bpf s

Question

hi team I am trying to use osquery to get continuously bpf socket events I am using osqueryi since it is already installed as part of our agent My program starts osqueryi and every minute sends query via stdin and grabs the result via stdout However I noticed the memory consumption of osqueryi starts at 200M and goes up very quickly even if I configure events max = 2000 and events expiry = 120 Can you help me understand why is that Also if there is a better way to get events from bpf socket events continuously with low impact on the hosting machine I would like to learn about that Thanks a lot

clong · Answer

< Tal Kapon> any reason you wouldn t just use osqueryd

clong · Answer

osqueryi is geared more towards someone at the keyboard writing an interactive query and isn t really designed to be queried on a regular basis

Tal Kapon · Answer

thanks the reason I used osqueryi is that it s already installed with our agent Also it s more complicated to create logging plugin than simply read stdout Still I can use osqueryd if there is a significant difference Can you elaborate how come the memory footprint is so large given events max = 2000 and events expiry = 120 I would like to learn and understand better

alessandrogario · Answer

< Tal Kapon> How does memory usage increase is it after BPF is initialized You can pass the ` verbose` option to know when the probes are installed

alessandrogario · Answer

You can estimate how much memory should be in use scroll down to Memory usage in the following link <https osquery readthedocs io en stable deployment process auditing linux process and socket auditing using bpf>

alessandrogario · Answer

The short version is ```buffer storage bytes = memory pool count bpf buffer storage size 4096 possible cpu count + perf bytes = 2 ^ bpf perf event array exp online cpu count``` With memory pool count=6 bpf buffer storage size set through command line options and possible cpu count online cpu count based on hardware

alessandrogario · Answer

You can see those two numbers here ```possible cpu count sys devices system cpu possible online cpu count sys devices system cpu online``` If you are running on VMware Fusion possible cpu count is set to 128 unless you disable CPU hotswapping in the vmx file

Tal Kapon · Answer

thanks will try to follow those calculations the cpu count based on your directions is 0 1