Stefano Bonicatti
06/15/2022, 9:15 AMsystem_controls
to see if unprivileged BPF is enabled.zwass
07/25/2022, 6:29 PMzwass
07/26/2022, 6:15 PMbpf_(process|socket)_events
the global pid or in-container pid?zwass
08/09/2022, 5:03 PMalessandrogario [9:52 AM]
The procfs-based approach can be a really useful addition to the processes table; it can't be used for events if the containers or the processes are shortlived. in that case, we have to update ebpfpub. To do that, we just have to
detect if btf can be constructed
pass a constructed btf object to FunctionTracer (osquery/events/linux/bpf/bpfeventpublisher.cpp), ideally only for fork/vfork/clone so that we don't add 64 bytes of overhead to all the probes
update the sytem state tracker (osquery/events/linux/bpf/systemstatetracker.h) to propagate the cgroup names
alessandrogario [9:52 AM]
I think having both approaches would be really cool
alessandrogario [9:55 AM]
I've merged an example
Initializing btf: <https://github.com/trailofbits/ebpfpub/blob/main/examples/execsnoop/src/main.cpp#L167>
Reading the cgroup names: <https://github.com/trailofbits/ebpfpub/blob/main/examples/execsnoop/src/main.cpp#L79>
alessandrogario
08/09/2022, 5:04 PMArtemis Tosini
08/09/2022, 5:10 PMalessandrogario
08/09/2022, 5:18 PMArtemis Tosini
08/11/2022, 4:20 PMArtemis Tosini
08/11/2022, 4:22 PMalessandrogario
08/11/2022, 5:06 PMArtemis Tosini
08/11/2022, 7:48 PMalessandrogario
08/17/2022, 10:51 AMalessandrogario
08/17/2022, 10:51 AM/**
* Copyright (c) 2014-present, The osquery authors
*
* This source code is licensed as defined by the LICENSE file found in the
* root directory of this source tree.
*
* SPDX-License-Identifier: (Apache-2.0 OR GPL-2.0-only)
*/
#include <string_view>
#include <vector>
#include <osquery/logger/logger.h>
#include <sqlite3.h>
namespace osquery {
namespace {
const std::vector<std::string> kPrefixList{
"libpod-conmon-",
"libpod-",
"docker-",
};
const std::string kSeparatorList{"./"};
void cgroupContainerName(sqlite3_context* context,
int argc,
sqlite3_value** argv) {
auto buffer_ptr = reinterpret_cast<const char*>(sqlite3_value_text(argv[0]));
if (buffer_ptr == nullptr) {
sqlite3_result_null(context);
return;
}
auto buffer_size = std::strlen(buffer_ptr);
std::string_view cgroup_path(buffer_ptr, buffer_size);
for (const auto& prefix : kPrefixList) {
auto start_index = cgroup_path.find(prefix);
if (start_index == std::string::npos) {
continue;
}
start_index += prefix.size();
auto end_index = cgroup_path.find_first_of(kSeparatorList, start_index);
auto substr_size = (end_index != std::string::npos)
? end_index - start_index
: std::string::npos;
auto container_name = cgroup_path.substr(start_index, substr_size);
sqlite3_result_text(context,
container_name.data(),
container_name.size(),
SQLITE_TRANSIENT);
return;
}
sqlite3_result_null(context);
}
} // namespace
void registerCgroupUtilsExtensions(sqlite3* db) {
sqlite3_create_function(db,
"cgroup_container_name",
-1,
SQLITE_UTF8 | SQLITE_DETERMINISTIC,
nullptr,
cgroupContainerName,
nullptr,
nullptr);
}
} // namespace osquery
alessandrogario
08/17/2022, 10:52 AMosqueryd -S 'SELECT name, cgroup_path, cgroup_container_name(cgroup_path) AS container FROM processes WHERE container <> "";'
+--------+--------------------------------------------------------------------------------------------------------+------------------------------------------------------------------+
| name | cgroup_path | container |
+--------+--------------------------------------------------------------------------------------------------------+------------------------------------------------------------------+
| conmon | /machine.slice/libpod-conmon-b8c8b2f3d8dcffed531ea0557e59e740264a1b2a3d6b76fd86b2c258309e3254.scope | b8c8b2f3d8dcffed531ea0557e59e740264a1b2a3d6b76fd86b2c258309e3254 |
| bash | /machine.slice/libpod-b8c8b2f3d8dcffed531ea0557e59e740264a1b2a3d6b76fd86b2c258309e3254.scope/container | b8c8b2f3d8dcffed531ea0557e59e740264a1b2a3d6b76fd86b2c258309e3254 |
+--------+--------------------------------------------------------------------------------------------------------+------------------------------------------------------------------+
alessandrogario
08/17/2022, 10:53 AMseph
08/17/2022, 12:24 PMREGEX_MATCH
but cgroup_container_name
seems less fiddly 🙂alessandrogario
08/17/2022, 4:17 PMalessandrogario
09/27/2022, 1:34 AMzwass
09/28/2022, 10:10 PMalessandrogario
09/29/2022, 3:04 PMag4ve
09/29/2022, 8:35 PMalessandrogario
10/01/2022, 3:56 PMalessandrogario
10/04/2022, 8:33 AMArtemis Tosini
10/14/2022, 6:58 PMLili
11/25/2022, 9:15 AMorbit[24125]: I1125 11:47:48.859995 24164 rocksdb.cpp:67] RocksDB: [ERROR] [table/block_based/block_based_table_reader.cc:1090] Encountered error while reading data from properties block IO error: While pread offset 4655206 len 40: /opt/orbit/osquery.db/004706.sst: Is a directory
orbit[24125]: I1125 11:47:48.955194 24164 rocksdb.cpp:67] RocksDB: [ERROR] [db/db_impl/db_impl_compaction_flush.cc:2624] Waiting after background flush error: IO error: While pread offset 4655206 len 40: /opt/orbit/osquery.db/004706.sst: Is a directoryAccumulated background error counts: 1
orbit[24125]: E1125 11:49:16.361975 25035 shutdown.cpp:79] Error adding new results to database for query pack/Global/bpf_socket_events (test)-1: IOError: Is a directory
2.
orbit[13777]: I1125 11:43:09.381443 13824 rocksdb.cpp:67] RocksDB: [WARN] [db/error_handler.cc:387] Background IO error IO error: While appending to file: /opt/orbit/osquery.db/005943.sst: Bad file descriptor
orbit[13777]: I1125 11:43:09.381582 13824 rocksdb.cpp:67] RocksDB: [ERROR] [db/db_impl/db_impl_compaction_flush.cc:2624] Waiting after background flush error: IO error: While appending to file: /opt/orbit/osquery.db/005943.sst: Bad file descriptorAccumulated background error counts: 1
orbit[13777]: E1125 11:44:52.375362 19350 shutdown.cpp:79] Error adding new results to database for query pack/Global/ebpf proccess events (test): IOError: Bad file descriptor
After that agent is restarted.
Why this errors occurs? Maybe I can tune some flags for exclude this error?
Query:
1.
SELECT uid, gid, local_address, local_port, path, pid, remote_address, remote_port, ntime, duration, family, exit_code FROM bpf_socket_events WHERE path NOT IN ('/usr/bin/hostname') AND remote_address NOT IN ('127.0.0.1', '169.254.169.254', '', '0000:0000:0000:0000:0000:0000:0000:0001', '::1', '0000:0000:0000:0000:0000:ffff:7f00:0001', 'unknown', '0.0.0.0', '0000:0000:0000:0000:0000:0000:0000:0000');
2.
SELECT cmdline, ntime, cwd, gid, uid, parent, path, pid FROM bpf_process_events;
Flags used:
table_delay: 200
augeas_lenses: /usr/share/augeas/lenses/dist
logger_plugin: tls
disable_events: false
watchdog_delay: 120
buffered_log_max: 3000000
disable_watchdog: false
enable_bpf_events: true
logger_min_stderr: 1
logger_tls_period: 1
logger_tls_max_lines: 8192
watchdog_memory_limit: 2048
bpf_buffer_storage_size: 4096
Fleet version: 4.23.0
Agent version: 1.3.0
Osquery version: 5.5.1
VM has 8 GB RAM, 4 CPU.Andrea
01/17/2023, 5:22 PMebpfpub
. is that correct or I missed it ?Sunil Kahalekar
02/27/2023, 6:55 AMNick Cheng
04/14/2023, 12:51 AMNick Cheng
04/14/2023, 12:59 AMI0414 00:54:17.598778 839266 bpfeventpublisher.cpp:254] Failed to load the BPF probe for syscall fork: Failed to open the tracepoint descriptor file: /sys/kernel/debug/tracing/events/syscalls/sys_enter_fork/id. This syscall may not be available on this system, continuing despite the error
I0414 00:54:17.599380 839266 bpfeventpublisher.cpp:254] Failed to load the BPF probe for syscall vfork: Failed to open the tracepoint descriptor file: /sys/kernel/debug/tracing/events/syscalls/sys_enter_vfork/id. This syscall may not be available on this system, continuing despite the error
I0414 00:54:17.655769 839266 bpfeventpublisher.cpp:267] Initialized BPF probe for syscall clone (33)
I0414 00:54:17.715793 839266 bpfeventpublisher.cpp:254] Failed to load the BPF probe for syscall close: Module verification failed: Stored value type does not match pointer operand type!
store i32 %29, i64* %30
i64Stored value type does not match pointer operand type!
store i8 %32, i64* %33
i64
I0414 00:54:17.716641 839266 eventfactory.cpp:156] Event publisher not enabled: BPFEventPublisher: Failed to create the function tracer: Module verification failed: Stored value type does not match pointer operand type!
store i32 %29, i64* %30
i64Stored value type does not match pointer operand type!
store i8 %32, i64* %33
i64
and I checked /sys/kernel/debug/tracing/events/syscalls/ has many sys_enter_*, but not _fork or _vfork.Andrea
04/21/2023, 9:59 AMkprobe
. I don’t think we can trace a returning function with the kretprobe
though. Is that correct or am I missing something ? my use case would be tracing tcp_v4_connect
the same way bcc/tools/tcpconnect
does, which looks better for tracing local/destination address+port than tracing connect() bind() accept() separately (again..unless I missed something 🙂 ). Any pointer would be appreciated! thanks