wave Hello I m trying to get the sample

Question

wave Hello I m trying to get the sample <https github com osquery osquery python|foobar> extension to load and log data to the results log via a scheduled query When I run `osqueryi verbose` and check the ` tables` I see that the tables are present and that the extension is loaded successfully I can also run the sample `select from foobar` and get results successfully However when I check the `sudo cat var log osquery osqueryd results log` it remains empty When I check the INFO logs I see errors about the table `foobar` not found ```E0402 19 35 09 994508 124231680 scheduler cpp 128 Error executing scheduled query my extension query no such table foobar``` What s even more strange is that if I query a didn t non custom table ` SELECT hostname FROM system info ` I can get the results from the table and it populates the results log Again the goal is to simply get the result log populated with the results from the scheduled query Does any know know why the scheduled query can not find the table or if there s an additional step that I m missing

Stryker0x · Answer

my ` var log osquery conf` ``` options disable events false schedule my extension query query SELECT FROM foobar query SELECT FROM my process events query SELECT hostname FROM system info interval 20 ```

Stryker0x · Answer

I check the extensions suffix to match ext checked the directory and extensions file owner permissions checked that the python code runs as root

seph · Answer

osqueryi and osqueryd are not really connected How are you starting this extension and is it connected to osqueryd

Stryker0x · Answer

wave hello < seph> In each python extension i have something similar to this ```if name == main osquery start extension name= my process ext version= 1 0 0 ``` I ve added the path of these extensions that ive tried all to ` var osquery extensions load` In ` var log osquery flags` I ve added ``` extensions autoload= var osquery extensions load verbose``` When i start the osquery service i ve just been running `sudo osqueryctl start` or restart

Stryker0x · Answer

How can i check that its connected to osqueryd and am i missing a step to connect otherwise

seph · Answer

` var log osquery flags` seems odd that should be in etc

seph · Answer

`extensions autoload` is the right option here

Stryker0x · Answer

its on macos so i thought it had to be in var where everything else is

Stryker0x · Answer

sorry not log

seph · Answer

logs is for logs I forget what the path is

Stryker0x · Answer

its in ` var osquery`

seph · Answer

When I m debugging `osqueryd` I usually try to do things in the following order 1 Is my config and flags syntactically correct You can point `osqueryi` at them 2 Does osqueryd log any errors or warnings 3 Connect directly to the running `osqueryd` Either via the distributed read TLS interface or using `osqueryi connect lt socket path gt ` and debug For 3 look at things like `select from osquery flags` to see if it picked up the flags you re setting `select from osquery extensions` `select from osquery registry where owner uuid = 0 ` to see what extensions did

Stryker0x · Answer

```Using a virtual database Need help type help osquery gt select from osquery flags gt Error Extension call failed connect failed No such file or directory osquery gt select from osquery flags ```

Stryker0x · Answer

that seems odd because i can see the table when i run ` tables`

seph · Answer

That snippet is small That looks like a osqueryi pointed at a socket but there s nothing on the other end

seph · Answer

` tables` doen t use the socket It doesn t count

seph · Answer

```$osqueryi connect tmp no socket here Connected to extension socket tmp no socket here for debugging Using a virtual database Need help type help osquery gt select from osquery flags Error Extension call failed connect failed No such file or directory```

Stryker0x · Answer

got it what should i point it at for the ` lt socket path gt ` in your example

seph · Answer

Uh that depends on how it s running You d want to check the docs for what the default is Or use `ps` to see how osquery invoked your extension

Stryker0x · Answer

okay got it

Stryker0x · Answer

so running the first command is working now

Stryker0x · Answer

seph · Answer

What s `select from osquery extensions` say

Stryker0x · Answer

seph · Answer

`osquery gt ` is an osqueryi prompt not connecting to osqueryd

Stryker0x · Answer

Stryker0x · Answer

got it hmm

seph · Answer

Invoking and using osqueryi outside osqueryd works but won t really debug osqueryd As said you can use it s ` connect` to talk to the osqueryd socket Or you can use the TLS interface

Stryker0x · Answer

gotcha looking up the right command to connect

Stryker0x · Answer

so according to ls and ps ``` ls la `which osqueryi` lrwxr xr x 1 root wheel 52 Apr 1 09 06 usr local bin osqueryi gt opt osquery lib osquery app Contents MacOS osqueryd``` ```s el |grep osquery 0 76223 1 4004 0 20 10 35142996 10452 SNs 0 0 00 25 opt osquery lib osquery app Contents MacOS osqueryd flagfile= private var osquery osquery flags 0 76224 76223 4044 0 20 10 35165200 14800 SN 0 0 00 10 opt osquery lib osquery app Contents MacOS osqueryd 501 76603 17604 4006 0 31 0 34271284 968 R+ 0 ttys007 0 00 00 grep osq```

Stryker0x · Answer

based on the <https osquery readthedocs io en latest deployment debugging inspecting daemon state using the shell|docs >it looks like i can use osqueryi to connect

seph · Answer

If you want to know what socket osqueryd is using you d need to use ps to look at the extension not osqueryd And yes you can use osqueryi to connect to osqueryd but you need to find the socket it s using first

Stryker0x · Answer

from ps ```77351 ttys006 0 00 08 usr local Cellar python 3 11 3 11 2 1 Frameworks Python framework Versions 3 11 Resources Python app Contents MacOS Python Users ec2 user osqdev my ext demo ext socket Users ec2 user osquery shell em timeout 3 interval 3```

Stryker0x · Answer

which i think is the same socket i tried i basically did osqueryi then ` connect Users ec2 user osquery shell em`

seph · Answer

Okay that should work

seph · Answer

Also that s a weird path for a root osquery Is this running as root or as the `ec2 user` user

Stryker0x · Answer

root

seph · Answer

That seems a little weird I wonder if that s from the osqueryi

Stryker0x · Answer

that might be the case beacuse when i exited i now only see ```ps el |grep osquery 0 77158 1 4004 0 20 10 35283284 10520 SNs 0 0 00 40 opt osquery lib osquery app Contents MacOS osqueryd flagfile= private var osquery osquery flags 0 77159 77158 4004 0 20 10 35157016 14932 SN 0 0 00 13 opt osquery lib osquery app Contents MacOS osqueryd```

seph · Answer

`grep osquery` isn t going to find your extension

Stryker0x · Answer

i dont even see the extension running anymore based on the process output

Stryker0x · Answer

okay so the two extensions dont appear until AFTER ive started osqueryi in a different shell

seph · Answer

From that it souhds like osqueryd isn t starting your extension osqueryi will but that s not really helpful

seph · Answer

I don t know if your oqueryd is reading a different flags file or if there s some error somewhere

seph · Answer

the system logs for it might help

Stryker0x · Answer

so i just stropped the osquery service with `sudo osqueryctl stop`

Stryker0x · Answer

then ran osqueryd ` opt osquery lib osquery app Contents MacOS osqueryd flagfile= private var osquery osquery flags`

Stryker0x · Answer

and at least the errors disappeared

Stryker0x · Answer

and it appears that i do get at least a couple results log now

Stryker0x · Answer

so perhaps the service isnt correct somehow or osqueryctl isnt doing something right

seph · Answer

I don t use `osqueryctl` or even the osquery packages so this starts being outside what I know You could reboot and see if things all shake out Or try to read through how it works `osqueryctl` is mostly a wrapper that s going to use the underlying macOS launchd stuff

Stryker0x · Answer

right i think thats the source of my issues things seem okay when running the osqueryd process directly ```sudo opt osquery lib osquery app Contents MacOS osqueryd flagfile= private var osquery osquery flags I0403 14 28 39 296025 455534080 init cpp 413 osquery initialized version=5 8 2 I0403 14 28 39 298817 455534080 extensions cpp 438 Found autoloadable extension Users ec2 user osqdev my ext demo ext I0403 14 28 39 298899 455534080 extensions cpp 438 Found autoloadable extension Users ec2 user osqdev my ext my extension ext I0403 14 28 39 298928 455534080 dispatcher cpp 78 Adding new service WatcherRunner 0x60000372c1f8 to thread 0x7000023d2000 0x600000a222a0 in process 80596 I0403 14 28 39 299501 37560320 watcher cpp 674 osqueryd watcher 80596 executing worker 80597 I0403 14 28 39 299984 37560320 watcher cpp 726 Created and monitoring extension child 80598 Users ec2 user osqdev my ext demo ext I0403 14 28 39 300451 37560320 watcher cpp 726 Created and monitoring extension child 80599 Users ec2 user osqdev my ext my extension ext I0403 14 28 39 315002 70604288 init cpp 410 osquery worker initialized watcher=80596 I0403 14 28 39 316332 70604288 dispatcher cpp 78 Adding new service WatcherWatcherRunner 0x6000029fc018 to thread 0x7000063fc000 0x6000010f4000 in process 80597 I0403 14 28 39 316404 70604288 rocksdb cpp 90 Opening RocksDB handle var osquery osquery db I0403 14 28 39 342285 70604288 dispatcher cpp 78 Adding new service ExtensionWatcher 0x7fbc047048b8 to thread 0x700006791000 0x6000010f4090 in process 80597 I0403 14 28 39 342326 70604288 dispatcher cpp 78 Adding new service ExtensionRunnerCore 0x7fbc04705888 to thread 0x700006814000 0x6000010f4030 in process 80597 I0403 14 28 39 342355 70604288 auto constructed tables cpp 99 Removing stale ATC entries I0403 14 28 39 342363 109133824 interface cpp 299 Extension manager service starting var osquery osquery em I0403 14 28 39 342998 70604288 eventfactory cpp 156 Event publisher not enabled endpointsecurity EndpointSecurity is disabled via configuration I0403 14 28 39 344350 70604288 eventfactory cpp 156 Event publisher not enabled endpointsecurity fim EndpointSecurity is disabled via configuration I0403 14 28 39 344372 70604288 eventfactory cpp 156 Event publisher not enabled openbsm Publisher disabled via configuration I0403 14 28 39 344384 70604288 eventfactory cpp 156 Event publisher not enabled scnetwork Publisher not used I0403 14 28 39 344395 70604288 eventfactory cpp 156 Event publisher not enabled event tapping Publisher disabled via configuration I0403 14 28 39 344743 70604288 eventsubscriberplugin cpp 492 Found 24 events for subscriber diskarbitration disk events I0403 14 28 39 345078 70604288 events cpp 70 Skipping subscriber socket events Subscriber disabled via configuration I0403 14 28 39 361346 70604288 loader cpp 45 No experiments selected I0403 14 28 39 361413 70604288 main cpp 104 Not starting the distributed query service Distributed query service not enabled I0403 14 28 39 361420 109670400 eventfactory cpp 390 Starting event publisher run loop diskarbitration I0403 14 28 39 361438 110206976 eventfactory cpp 390 Starting event publisher run loop fsevents I0403 14 28 39 361439 110743552 eventfactory cpp 390 Starting event publisher run loop iokit I0403 14 28 39 361454 70604288 dispatcher cpp 78 Adding new service SchedulerRunner 0x600002be00f8 to thread 0x700006a20000 0x6000010fc1c0 in process 80597 I0403 14 28 39 361470 111280128 scheduler cpp 120 Executing scheduled query my extension query SELECT FROM foobar I0403 14 28 39 361505 110206976 eventfactory cpp 410 Event publisher fsevents run loop terminated for reason Publisher disabled via configuration E0403 14 28 39 361850 111280128 scheduler cpp 128 Error executing scheduled query my extension query no such table foobar I0403 14 28 39 367128 111816704 interface cpp 137 Registering extension my awesome extension 23266 version=1 0 0 sdk=3 0 7 I0403 14 28 39 367218 112353280 interface cpp 137 Registering extension my process ext 34893 version=1 0 0 sdk=3 0 7 I0403 14 28 39 367337 111816704 registry factory cpp 114 Extension 23266 registered table plugin foobar I0403 14 28 39 367471 112353280 registry factory cpp 114 Extension 34893 registered table plugin my process events I0403 14 29 00 520484 111280128 scheduler cpp 120 Executing scheduled query my extension query SELECT FROM foobar I0403 14 29 00 521996 111280128 query cpp 128 Scheduled query has been updated my extension query I0403 14 29 00 522194 111280128 scheduler cpp 201 Found results for query my extension query I0403 14 29 21 842377 111280128 scheduler cpp 120 Executing scheduled query my extension query SELECT FROM foobar I0403 14 29 42 984807 111280128 scheduler cpp 120 Executing scheduled query my extension query SELECT FROM foobar I0403 14 30 04 148504 111280128 scheduler cpp 120 Executing scheduled query my extension query SELECT FROM foobar I0403 14 30 25 334856 111280128 scheduler cpp 120 Executing scheduled query my extension query SELECT FROM foobar I0403 14 30 46 533720 111280128 scheduler cpp 120 Executing scheduled query my extension query SELECT FROM foobar I0403 14 31 07 696177 111280128 scheduler cpp 120 Executing scheduled query my extension query SELECT FROM foobar I0403 14 31 28 890303 111280128 scheduler cpp 120 Executing scheduled query my extension query SELECT FROM foobar I0403 14 31 50 084290 111280128 scheduler cpp 120 Executing scheduled query my extension query SELECT FROM foobar I0403 14 32 11 279191 111280128 scheduler cpp 120 Executing scheduled query my extension query SELECT FROM foobar I0403 14 32 32 456142 111280128 scheduler cpp 120 Executing scheduled query my extension query SELECT FROM foobar I0403 14 32 53 595430 111280128 scheduler cpp 120 Executing scheduled query my extension query SELECT FROM foobar I0403 14 33 14 786928 111280128 scheduler cpp 120 Executing scheduled query my extension query SELECT FROM foobar I0403 14 33 35 946554 111280128 scheduler cpp 120 Executing scheduled query my extension query SELECT FROM foobar I0403 14 33 57 050361 111280128 scheduler cpp 120 Executing scheduled query my extension query SELECT FROM foobar ```

Stryker0x · Answer

how do you start it

seph · Answer

i work for Kolide and we have our own packaging It s not really germane here

Stryker0x · Answer

got it well thanks for all the pointers and questions

Daniel Bretón Suárez · Answer

I might be late but I hope these commands help you To run osquery shell with the extension ```osqueryi extension external external extension hello ext``` To run the extension detached Run the core ```sudo osqueryd flagfile etc osquery osquery flags extensions socket ~ osquery shell em verbose 2 gt amp 1 | tee osquery verbose out``` And then run the extension in a different terminal ```sudo your extension ext socket ~ osquery shell em``` Or use gdb or any other debugger tool doesn t work on windows ```sudo gdb args your extension ext socket ~ osquery shell em```

Daniel Bretón Suárez · Answer

I did not test those commands on MacOS so you may need to change some paths

Stryker0x · Answer

So i think the issue was that the osqueryctl service was not picking up the extensions somehow

Stryker0x · Answer

i just ran osqueryd directly with the right flags and things worked as expected

seph · Answer

Osqueryctl shouldn t do anything for extensions It just restarts osqueryd

Stryker0x · Answer

but it does setup the launch daemon which i thought had keys for the flags file

Stryker0x · Answer

which had the path to the extensions file