Osquery uses basic SQL commands to leverage a relational data-model to describe a device.

osquery

Speaking of #osquery-go-distributed-read-plugin-implementation-questions :face_with_rolling_eyes:,  I am deploying osquery with Kolide on my team's endpoints (mostly windows machines). I am trying to perform distributed queries (from Kolide) - but can't receive any answer. Looking at the status logs - I see that for a long while the queried computers did not post to distributed/read, it is as if the distributed plugin crashed. Did anyone ever noticed such a problem? Yesterday it also happened, and only after few hours the osquery agent started posting to distributed/read again.

I had this same problem when using the latest version of osquery (2.11) with Kolide. I ended up compiling their binary dist which included it's own version of osquery for launcher (2.9) and this resolved my problem. I did clear out the state directories as well.

<https://github.com/kolide/launcher/blob/master/docs/launcher.md#generating-relocatable-binary-bundles>

and that creates a version of osquery that will work with launcher

Thanks,my problem is that i am using osquery on windows, launcher won't work for me

Enrollment is not mandatory for logger plugins. You can authenticate however you like.

I am currently trying to use osquery to produce data into Kafka as a producer and I get an error 'could not autoload extensions: failed reading...'

Hi <@USBUNP03T>, I have a few questions, hopefully I can help.

I assume you are running into a problem using the Kafka logger plugin? What exactly does your configuration look like and what steps have you followed thus far?

The log message you are referencing: "Could not autoload extensions" is a verbose message, not an error. If you are not intending to use extensions (you most likely are not) then it can be safely ignored.

It most likely looks similar to:

```I0114 17:48:18.285985 186105280 extensions.cpp:337] Could not autoload extensions: Failed reading: /root/fake```
If you look the initial `I` means this message is informational, if you see a log message that starts with an `E` that is an error.

It might help if you shared all of the log messages you see.

Hi theopolis, I was trying to use the Kafka extension on windows but it seems from some research, the extension does not yet exist

The configuration I had worked on MacOS but on Windows, lacked the extension. Thanks for the information regarding the logs

Hey there! Has anybody run into the osqueryd worker no longer sending results via TLS a few minutes after the watchdog blacklists one of them? I get a "scheduled query may have failed," then ~10 minutes later, osquery stops attempting to hit any endpoint except /distributed/read.

I think this is probably a problem with my osquery worker, not the tls plugin – but I figured I would start here.

Is it possible that every query is blacklisted? Can you run a live query and see what you get from the `osquery_schedule` table?

<@U0JFM04MS> I believe only one query is blacklisted (I get one log line saying scheduled query may have failed, and I know why it's taking so long) but live queries are also hanging, which is unfortunate. I'm able to replicate locally though, so I'm going to try to see if I can look through rocksdb to find the set of blacklisted queries

osquery.conf

This is my config. plz tell me if there is something wrong here...

This config is not valid json (missing commas in the schedule section)

hello,does anybody tell me that the kafka_producer can be used to windows?

I think it does as of osquery 4.3.0 <https://github.com/osquery/osquery/pull/6095>

image.png

thank your reply.
Now I am using osquery4.3.0. but I can't find any parameter related kafka.

It seems not ready in 4.3.0.
If you have further information about the kafka_producer on windows, please let me know.
Thanks again

Yes, you are right. I misunderstood the release notes and pull request.

It’s mainly a performance issue, the extensions API requires a bit of serialization and deserialization and the rate of publishing events can be intense.

<@U09M563C7> looking at some of the extensions, events are modeled as regular tables. So they are taking the performance hit anyway if the query is scheduled frequently. Would avoiding JSON and using thrift union, etc help with performance issues?

Things like DNS query or TLS SNI sniffing. Both of which I’ve as patches to OSQuery that have rightly been rejected because doing packet parsing in a non-memory safe language unless you really know what you’re doing is a bad idea…
These both generating lots of rows though so would be a good fit for an events table

This is the patch from the PR that has been closed right?  I don't think it's a problem with the language itself, there are several production quality tools out there using libpcap.

There are some requirements that must be met regardless of how much safety the language provides:
1. Fuzzing on the packet parsers
2. Extensive unit testing
3. Separating privileges in a good way. This can be easily done the way other tools based on pcap are doing
The PR was closed for two reasons: privacy concerns (which may be negotiable with a blueprint issue), and the lack of the above 3 requirements (mandatory, biggest issue)

Yep correct. Completely agree it’s do-able (and I’m excited for the eBPF PR!) but, as you say, requires a lot of extra stuff to be confident it’s safe.

That’s mainly why I was interested if this was possible as a plugin i.e. because writing something as Go plugin means you can be a bit laxer (crashes aren’t a security risk and wouldn’t take down the whole of osqueryd with it)