Stryker0x
04/02/2023, 7:58 PMosqueryi --verbose
and check the .tables
, I see that the tables are present and that the extension is loaded successfully. I can also run the sample select * from foobar
and get results successfully.
However, when I check the sudo cat /var/log/osquery/osqueryd.results.log
it remains empty. When I check the INFO logs, I see errors about the table foobar
not found.
E0402 19:35:09.994508 124231680 scheduler.cpp:128] Error executing scheduled query my_extension_query: no such table: foobar
What's even more strange is that if I query a didn't non-custom table, "SELECT hostname FROM system_info;"
, I can get the results from the table and it populates the results log.
Again, the goal is to simply get the result log populated with the results from the scheduled query. Does any know know why the scheduled query can not find the table or if there's an additional step that I'm missing?/var/log/osquery.conf
{
"options": {
"disable_events": false
},
"schedule": {
"my_extension_query": {
"query": "SELECT * FROM foobar;",
//"query": "SELECT * FROM my_process_events;",
//"query": "SELECT hostname FROM system_info;",
"interval": 20
}
}
}
seph
04/03/2023, 1:15 AMStryker0x
04/03/2023, 12:31 PMif __name__ == "__main__":
osquery.start_extension(name="my_process_ext", version="1.0.0")
I've added the path of these extensions that ive tried all to /var/osquery/extensions.load
In /var/log/osquery.flags
I've added:
--extensions_autoload=/var/osquery/extensions.load
--verbose
When i start the osquery service i've just been running
sudo osqueryctl start
or restart.seph
04/03/2023, 12:54 PM/var/log/osquery.flags
seems odd, that should be in etc?extensions_autoload
is the right option here.Stryker0x
04/03/2023, 12:57 PMseph
04/03/2023, 12:57 PMStryker0x
04/03/2023, 12:57 PM/var/osquery
seph
04/03/2023, 12:58 PMosqueryd
I usually try to do things in the following order:
1. Is my config and flags syntactically correct? You can point osqueryi
at them
2. Does osqueryd log any errors or warnings?
3. Connect directly to the running osqueryd
(Either via the distributed read TLS interface, or using osqueryi --connect <socket path>
and debug.
For (3) look at things like:
• select * from osquery_flags
to see if it picked up the flags you’re setting
• select * from osquery_extensions
• select * from osquery_registry where owner_uuid != 0;
to see what extensions didStryker0x
04/03/2023, 1:01 PMUsing a virtual database. Need help, type '.help'
[*]osquery> select * from osquery_flags
[*] ...> ;
Error: Extension call failed: connect() failed: No such file or directory
[*]osquery> select * from osquery_flags;
.tables
seph
04/03/2023, 1:02 PM.tables
doen’t use the socket. It doesn’t count$osqueryi --connect /tmp/no/socket/here
Connected to extension socket /tmp/no/socket/here for debugging
Using a virtual database. Need help, type '.help'
[*]osquery> select * from osquery_flags;
Error: Extension call failed: connect() failed: No such file or directory
Stryker0x
04/03/2023, 1:11 PM<socket path>
in your example?seph
04/03/2023, 1:15 PMps
to see how osquery invoked your extensionStryker0x
04/03/2023, 1:17 PM| 0 |
| events_max | uint64 | Maximum number of event batches per type to buffer | 50000 | 50000
| 0 |
| events_optimize | bool | Optimize subscriber select queries (scheduler only) | true | true
| 0 |
| experiment_list | string | Comma-separated list of experiments to enable | |
| 0 |
| extension | string | Path to a single extension to autoload | |
| 1 |
| extensions_autoload | string | Optional path to a list of autoloaded & managed extensions | /var/osquery/extensions.load | /var/osquery/extensions.
load | 0 |
| extensions_default_index | bool | Enable INDEX on all extension table columns (default true) | true | true
| 0 |
| extensions_interval | string | Seconds delay between connectivity checks | 3 | 3
| 0 |
seph
04/03/2023, 1:19 PMselect * from osquery_extensions
say?Stryker0x
04/03/2023, 1:19 PM+-----------------------------------------+--------+-----------------------------------------------------------------------------------------------------------------------------+-----------------------------------------+-------------------------------------------------------------------------+------------+
osquery> select * from osquery_extensions;
+-------+----------------------+---------+-------------+-------------------------------------------------------------------------------+-----------+
| uuid | name | version | sdk_version | path | type |
+-------+----------------------+---------+-------------+-------------------------------------------------------------------------------+-----------+
| 0 | core | 5.8.2 | 0.0.0 | /var/folders/2g/jjj_bgln5qn8c6t1szz_fmmc0000gn/T/osquery-24593/shell.em | core |
| 52910 | my_awesome_extension | 1.0.0 | 1.8.0 | /var/folders/2g/jjj_bgln5qn8c6t1szz_fmmc0000gn/T/osquery-24593/shell.em.52910 | extension |
| 56368 | my_process_ext | 1.0.0 | 1.8.0 | /var/folders/2g/jjj_bgln5qn8c6t1szz_fmmc0000gn/T/osquery-24593/shell.em.56368 | extension |
+-------+----------------------+---------+-------------+-------------------------------------------------------------------------------+-----------+
osquery>
seph
04/03/2023, 1:19 PMosquery>
is an osqueryi prompt, not connecting to osquerydStryker0x
04/03/2023, 1:20 PMosquery> select * from osquery_registry where owner_uuid != 0;
+----------+-------------------+------------+----------+--------+
| registry | name | owner_uuid | internal | active |
+----------+-------------------+------------+----------+--------+
| table | foobar | 52910 | 0 | 1 |
| table | my_process_events | 56368 | 0 | 1 |
+----------+-------------------+------------+----------+--------+
osquery>
seph
04/03/2023, 1:21 PM--connect
to talk to the osqueryd socket. Or you can use the TLS interface.Stryker0x
04/03/2023, 1:22 PMls -la `which osqueryi`
lrwxr-xr-x 1 root wheel 52 Apr 1 09:06 /usr/local/bin/osqueryi -> /opt/osquery/lib/osquery.app/Contents/MacOS/osqueryd
s -el |grep osquery
0 76223 1 4004 0 20 10 35142996 10452 - SNs 0 ?? 0:00.25 /opt/osquery/lib/osquery.app/Contents/MacOS/osqueryd --flagfile=/private/var/osquery/osquery.flags
0 76224 76223 4044 0 20 10 35165200 14800 - SN 0 ?? 0:00.10 /opt/osquery/lib/osquery.app/Contents/MacOS/osqueryd
501 76603 17604 4006 0 31 0 34271284 968 - R+ 0 ttys007 0:00.00 grep osq
seph
04/03/2023, 1:26 PMStryker0x
04/03/2023, 1:39 PM77351 ttys006 0:00.08 /usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/Resources/Python.app/Contents/MacOS/Python /Users/ec2-user/osqdev/my_ext/demo.ext --socket /Users/ec2-user/.osquery/shell.em --timeout 3 --interval 3
.connect /Users/ec2-user/.osquery/shell.em
seph
04/03/2023, 1:40 PMec2-user
user?Stryker0x
04/03/2023, 1:41 PMseph
04/03/2023, 1:41 PMStryker0x
04/03/2023, 1:42 PMps -el |grep osquery
0 77158 1 4004 0 20 10 35283284 10520 - SNs 0 ?? 0:00.40 /opt/osquery/lib/osquery.app/Contents/MacOS/osqueryd --flagfile=/private/var/osquery/osquery.flags
0 77159 77158 4004 0 20 10 35157016 14932 - SN 0 ?? 0:00.13 /opt/osquery/lib/osquery.app/Contents/MacOS/osqueryd
seph
04/03/2023, 1:44 PMgrep osquery
isn’t going to find your extension.Stryker0x
04/03/2023, 1:47 PMseph
04/03/2023, 2:30 PMStryker0x
04/03/2023, 2:32 PMsudo osqueryctl stop
/opt/osquery/lib/osquery.app/Contents/MacOS/osqueryd --flagfile=/private/var/osquery/osquery.flags
seph
04/03/2023, 2:36 PMosqueryctl
, or even the osquery packages, so this starts being outside what I know.
You could reboot, and see if things all shake out. Or try to read through how it works.
osqueryctl
is mostly a wrapper that’s going to use the underlying macOS launchd stuffStryker0x
04/03/2023, 2:37 PMsudo /opt/osquery/lib/osquery.app/Contents/MacOS/osqueryd --flagfile=/private/var/osquery/osquery.flags
I0403 14:28:39.296025 455534080 init.cpp:413] osquery initialized [version=5.8.2]
I0403 14:28:39.298817 455534080 extensions.cpp:438] Found autoloadable extension: /Users/ec2-user/osqdev/my_ext/demo.ext
I0403 14:28:39.298899 455534080 extensions.cpp:438] Found autoloadable extension: /Users/ec2-user/osqdev/my_ext/my_extension.ext
I0403 14:28:39.298928 455534080 dispatcher.cpp:78] Adding new service: WatcherRunner (0x60000372c1f8) to thread: 0x7000023d2000 (0x600000a222a0) in process 80596
I0403 14:28:39.299501 37560320 watcher.cpp:674] osqueryd watcher (80596) executing worker (80597)
I0403 14:28:39.299984 37560320 watcher.cpp:726] Created and monitoring extension child (80598): /Users/ec2-user/osqdev/my_ext/demo.ext
I0403 14:28:39.300451 37560320 watcher.cpp:726] Created and monitoring extension child (80599): /Users/ec2-user/osqdev/my_ext/my_extension.ext
I0403 14:28:39.315002 70604288 init.cpp:410] osquery worker initialized [watcher=80596]
I0403 14:28:39.316332 70604288 dispatcher.cpp:78] Adding new service: WatcherWatcherRunner (0x6000029fc018) to thread: 0x7000063fc000 (0x6000010f4000) in process 80597
I0403 14:28:39.316404 70604288 rocksdb.cpp:90] Opening RocksDB handle: /var/osquery/osquery.db
I0403 14:28:39.342285 70604288 dispatcher.cpp:78] Adding new service: ExtensionWatcher (0x7fbc047048b8) to thread: 0x700006791000 (0x6000010f4090) in process 80597
I0403 14:28:39.342326 70604288 dispatcher.cpp:78] Adding new service: ExtensionRunnerCore (0x7fbc04705888) to thread: 0x700006814000 (0x6000010f4030) in process 80597
I0403 14:28:39.342355 70604288 auto_constructed_tables.cpp:99] Removing stale ATC entries
I0403 14:28:39.342363 109133824 interface.cpp:299] Extension manager service starting: /var/osquery/osquery.em
I0403 14:28:39.342998 70604288 eventfactory.cpp:156] Event publisher not enabled: endpointsecurity: EndpointSecurity is disabled via configuration
I0403 14:28:39.344350 70604288 eventfactory.cpp:156] Event publisher not enabled: endpointsecurity_fim: EndpointSecurity is disabled via configuration
I0403 14:28:39.344372 70604288 eventfactory.cpp:156] Event publisher not enabled: openbsm: Publisher disabled via configuration
I0403 14:28:39.344384 70604288 eventfactory.cpp:156] Event publisher not enabled: scnetwork: Publisher not used
I0403 14:28:39.344395 70604288 eventfactory.cpp:156] Event publisher not enabled: event_tapping: Publisher disabled via configuration
I0403 14:28:39.344743 70604288 eventsubscriberplugin.cpp:492] Found 24 events for subscriber diskarbitration.disk_events
I0403 14:28:39.345078 70604288 events.cpp:70] Skipping subscriber: socket_events: Subscriber disabled via configuration
I0403 14:28:39.361346 70604288 loader.cpp:45] No experiments selected
I0403 14:28:39.361413 70604288 main.cpp:104] Not starting the distributed query service: Distributed query service not enabled.
I0403 14:28:39.361420 109670400 eventfactory.cpp:390] Starting event publisher run loop: diskarbitration
I0403 14:28:39.361438 110206976 eventfactory.cpp:390] Starting event publisher run loop: fsevents
I0403 14:28:39.361439 110743552 eventfactory.cpp:390] Starting event publisher run loop: iokit
I0403 14:28:39.361454 70604288 dispatcher.cpp:78] Adding new service: SchedulerRunner (0x600002be00f8) to thread: 0x700006a20000 (0x6000010fc1c0) in process 80597
I0403 14:28:39.361470 111280128 scheduler.cpp:120] Executing scheduled query my_extension_query: SELECT * FROM foobar;
I0403 14:28:39.361505 110206976 eventfactory.cpp:410] Event publisher fsevents run loop terminated for reason: Publisher disabled via configuration
E0403 14:28:39.361850 111280128 scheduler.cpp:128] Error executing scheduled query my_extension_query: no such table: foobar
I0403 14:28:39.367128 111816704 interface.cpp:137] Registering extension (my_awesome_extension, 23266, version=1.0.0, sdk=3.0.7)
I0403 14:28:39.367218 112353280 interface.cpp:137] Registering extension (my_process_ext, 34893, version=1.0.0, sdk=3.0.7)
I0403 14:28:39.367337 111816704 registry_factory.cpp:114] Extension 23266 registered table plugin foobar
I0403 14:28:39.367471 112353280 registry_factory.cpp:114] Extension 34893 registered table plugin my_process_events
I0403 14:29:00.520484 111280128 scheduler.cpp:120] Executing scheduled query my_extension_query: SELECT * FROM foobar;
I0403 14:29:00.521996 111280128 query.cpp:128] Scheduled query has been updated: my_extension_query
I0403 14:29:00.522194 111280128 scheduler.cpp:201] Found results for query: my_extension_query
I0403 14:29:21.842377 111280128 scheduler.cpp:120] Executing scheduled query my_extension_query: SELECT * FROM foobar;
I0403 14:29:42.984807 111280128 scheduler.cpp:120] Executing scheduled query my_extension_query: SELECT * FROM foobar;
I0403 14:30:04.148504 111280128 scheduler.cpp:120] Executing scheduled query my_extension_query: SELECT * FROM foobar;
I0403 14:30:25.334856 111280128 scheduler.cpp:120] Executing scheduled query my_extension_query: SELECT * FROM foobar;
I0403 14:30:46.533720 111280128 scheduler.cpp:120] Executing scheduled query my_extension_query: SELECT * FROM foobar;
I0403 14:31:07.696177 111280128 scheduler.cpp:120] Executing scheduled query my_extension_query: SELECT * FROM foobar;
I0403 14:31:28.890303 111280128 scheduler.cpp:120] Executing scheduled query my_extension_query: SELECT * FROM foobar;
I0403 14:31:50.084290 111280128 scheduler.cpp:120] Executing scheduled query my_extension_query: SELECT * FROM foobar;
I0403 14:32:11.279191 111280128 scheduler.cpp:120] Executing scheduled query my_extension_query: SELECT * FROM foobar;
I0403 14:32:32.456142 111280128 scheduler.cpp:120] Executing scheduled query my_extension_query: SELECT * FROM foobar;
I0403 14:32:53.595430 111280128 scheduler.cpp:120] Executing scheduled query my_extension_query: SELECT * FROM foobar;
I0403 14:33:14.786928 111280128 scheduler.cpp:120] Executing scheduled query my_extension_query: SELECT * FROM foobar;
I0403 14:33:35.946554 111280128 scheduler.cpp:120] Executing scheduled query my_extension_query: SELECT * FROM foobar;
I0403 14:33:57.050361 111280128 scheduler.cpp:120] Executing scheduled query my_extension_query: SELECT * FROM foobar;
seph
04/03/2023, 2:38 PMStryker0x
04/03/2023, 2:40 PMDaniel Bretón Suárez
04/04/2023, 1:51 PMosqueryi --extension ./external/external_extension_hello.ext
To run the extension detached
Run the core
sudo osqueryd --flagfile /etc/osquery/osquery.flags --extensions_socket ~/.osquery/shell.em --verbose 2>&1 | tee osquery-verbose.out
And then, run the extension in a different terminal
sudo ./your_extension.ext --socket ~/.osquery/shell.em
Or use gdb (or any other debugger tool), doesn't work on windows:
sudo gdb --args ./your_extension.ext --socket ~/.osquery/shell.em
Stryker0x
04/04/2023, 3:44 PMseph
04/04/2023, 3:49 PMStryker0x
04/04/2023, 3:50 PM