hi team,
I am trying to use osquery to get contin...
# ebpft
Tal Kapon
07/17/2021, 12:37 PMhi team,
I am trying to use osquery to get continuously bpf_socket_events. I am using osqueryi since it is already installed as part of our agent. My program starts osqueryi, and every minute sends query via stdin and grabs the result via stdout. However, I noticed the memory consumption of osqueryi starts at 200M and goes up very quickly even if I configure events_max = 2000 and events_expiry = 120.
Can you help me understand why is that?
Also, if there is a better way to get events from bpf_socket_events continuously with low impact on the hosting machine, I would like to learn about that.
Thanks a lot
c
clong
07/19/2021, 3:22 PM@Tal Kapon any reason you wouldn’t just use osqueryd?
clong
07/19/2021, 3:22 PMosqueryi is geared more towards someone at the keyboard writing an interactive query and isn’t really designed to be queried on a regular basis
t
Tal Kapon
07/19/2021, 3:39 PMthanks. the reason I used osqueryi is that it’s already installed with our agent. Also, it’s more complicated to create logging plugin than simply read stdout.
Still, I can use osqueryd if there is a significant difference. Can you elaborate how come the memory footprint is so large given events_max = 2000 and events_expiry = 120? I would like to learn and understand better.
a
alessandrogario
07/19/2021, 4:45 PM
@Tal Kapon How does memory usage increase? is it after BPF is initialized? You can pass the
--verbosealessandrogario
07/19/2021, 4:46 PM
You can estimate how much memory should be in use; scroll down to 'Memory usage' in the following link: https://osquery.readthedocs.io/en/stable/deployment/process-auditing/#linux-process-and-socket-auditing-using-bpf
alessandrogario
07/19/2021, 4:47 PM
The short version is
Copy code
buffer_storage_bytes = memory_pool_count * (bpf_buffer_storage_size * 4096) * possible_cpu_count
+
perf_bytes = (2 ^ bpf_perf_event_array_exp) * online_cpu_countalessandrogario
07/19/2021, 4:49 PM
You can see those two numbers here:
Copy code
possible_cpu_count: /sys/devices/system/cpu/possible
online_cpu_count: /sys/devices/system/cpu/onlinet
Tal Kapon
07/19/2021, 5:57 PMthanks. will try to follow those calculations. the cpu count based on your directions is 0-1.
115 Views