I'd be fine with writing an events buffer for osqu...
# extensions
I'd be fine with writing an events buffer for osquery-go with a backing store if we could figure out how to resolve the events_optimize issue.
💯 1
is… in many ways analogous to kafka or redis consumer groups? Now I’m giggling about creating a redis stream backed event store
means each query only sees each event once (the first time the query is run after the event is generated).
Yes. If you imagined events on a log pipeline, each query is a consumer group. The underlying pipeline keeps track of that dispatch.
It’s the core innovation in kafka, And something all streaming systems do now,
👍 1
Without having written any code… I can imagine storing an event stream, seems pretty straight forward, Borrow redis trick for naming them with timestamps. Track a pointer to the last seen item in the stream.
But I kinda wish there was a pure go library for this
Yeah 100% we can do that but we need a way to get the name of the executing query or some equivalent to the extension so that it can do the optimization.
I've been looking at https://github.com/dgraph-io/badger as a local store for Go lately.
Do we need the name, or can you just hash the query? Slightly different semantics
I bet the extension doesn’t get
which might break things
I’ve really enjoyed github.com/dgraph-io/ristretto for a project. But it’s a bit simpler
Oh, we probably don’t get the query eithger
Yeah I think we don't get the query