Osquery uses basic SQL commands to leverage a relational data-model to describe a device.

osquery

`map&lt;string, Object&gt;` where `Object` is a `union` should work in thrift AFAIK. Overhead for serialization/de-serialization via domain sockets should be negligible compared to JSON.

We have been pushing events in near real-time for ~4 years. OSS PR <https://github.com/osquery/osquery/pull/3482> and others didn't go anywhere. Our Osquery at most have a latency of 4 seconds on events. Here is a customer from 2020 @scale conference who measured events -&gt; alerts in ~1.7 seconds <https://www.youtube.com/watch?v=-9BfXMYn0wk&amp;t=1210> The more I explore extensions, these limitations are biting me. Happy to contribute, if there is consensus.

Regarding events -- did you ever do an analysis of the perf cost of the rocksdb roundtrip? I'd be curious to see what that looked like.

We probably did some perf testing in lab few years back. Will see if I can find the details. We haven't been writing event to DB for few years now. Helps with perf and resource utilization. We use it as fallback when remote TLS is not reachable.

I think there’s a good case for sending events directly to the logger if that’s what you are advocating for. I think we need to win over Seph’s strong opinions but my feedback/requirement is that the original PR is hacky and we need to spend more time on the interface.

ToB and Teddy have both done rework on rocksDB. Alessandro says it’s a lot faster now.

My general sentiment is that I’m not sure if this belongs in osquery. At least today, I feel less strident than some days, but I think osquery fufills a different niche than the Elastic Beats tools. And what you generally describe feels like trying to push beats into osquery, and I don’t really understand why.

So even before the interfaces, I feel like we should settle on the why of it.

Thinking a bit — I could imagine a streaming sql (&lt;cough&gt; CEP) version of osquery. But that feels like a big project.