https://github.com/osquery/osquery logo
Join Slack
Powered by
Playing with the code, it seems hard to use as a l...
# goquery
s
Playing with the code, it seems hard to use as a library? So if I want to use it against some internal server, I need to make a hard fork. At least, trying to use a bunch of the integration seems hard since it’s based on a global 
api
object, and not something that’s passed around.
a
Yes, integration would require adding a module in 
api
that returned a custom struct that implements the 
GoQueryAPI
interface. The 
api
module imported by the commands is just a shim for the calls into that struct. Maybe we can implement an option if the server calls could be standardized such that the only integration would be defining the url of each action endpoint (login basic auth/saml, connect, schedule)
s
I was looking to do something else. I wrote a suitable api object, it's a simple enough interface. But i was expecting to be able to write my own main that instantiated it and then handed it to the goquery code.
If it’s interesting, I can try to mock up what I mean
a
Yes, it's not too obvious but you would implement another case statement in driver.go and then run main with --config pointing to the config json which states which api driver to use. It is a little burried so we can work on the implementation or write up some clear docs, let me know :)
s
Yes. I forked goquery, and wrote an API for an internal service. It’s kinda fun.
But that’s code I’m never going to PR back. Partly because it’s an internal service, and partly because its messy.
I’d suggest that an interface that allows me to write my own main, and treat goquery as a library would be powerful. Super psuedo code:
Copy code
func main() {
  intApi := myApi.New(hostname)
  p := prompt.New(intApi)
  p.Run()
}
vs the model today where the API is a global and I can’t really keep track
I guess I’m sorta suggestion the entire 
api/driver.go
stuff move to main.
I’m not sure if this sort of project architecture conversation is welcome.
a
Ah I see your idea, have goquery as a dependency so it can be worked on independently of internal integrations
s
Yeah… I find it’s a really natural way to use go interfaces/
a
I think it's an interesting approach. @obelisk what do you think ?
s
I can try to find some time to PR some of this, if that’s interesting to y’all.
There’s a handful of other things I was noticing too. Command line flag parsing jumped out at me. Etc.
o
So my long term plan was to figure out a better way to plug and play APIs, that way you can easily separate you backend communication code while maintaining core goquery
Sort of like osquery extensions
Using goquery as an API was never something I considered because it’s inherently an interactive tool, if you want to use apis, I figured you would just use the backend apis directly
s
To be honest, do you need that? I’d generally expect that if I need a custom backend, I build my own. Making a hot pluggable thing seems like a lot of overhead for likely a small number of use cases
I don’t know what the right words are… I’d rather built my own main, to use my custom weird plugins, than build a hot pluggable interface for go.
o
For everyone who has a custom backend, that means they can keep using the shell that we all help build together easily while keeping the code separate and for more common backends like osctrl, fleet, zentral, we can give them even better support all in one. At least that was my idea
s
They’re going to have to build a plugin. Which implies they have a working go toolchain. May as well build a whole goquery. (I have some bias here)
I think it’s likely a much smaller use case than osquery extensions. So using some heavy thing like thrift feels like a lot of extra dev overhead
I’m partly coming from really liking the compile type safety in go. A lot of what I do is in my little monorepo, with a lot of type assurance. Plugins get away from that.
o
The idea wasn’t to use thrift, but if you have a custom backend, you’ll need to build it again anyway and you’re probably comfortable with coding. My current setup for our internal backend is another folder called private and we compile it in directly
Maybe osquery modules is a better model to follow
s
mmmm, so you’re thinking a model might be to have a local fork where one maintains some extra code in private?
Personally I find that hard — git gets in the way. Either I have a hard fork, or I’m fighting with submodules.
o
I like the model of osquery extensions that people have to do things specific for their infra so I wanted to try and give people that power
s
My guess is do something easy and see what use cases emerge
o
I grant right now the system isn’t great :p
s
Oh, to be clear. I wrote and have running a fork of goquery against some internal infra. So some 🍺 all around
🎉 1
o
I don’t like sub modules and don’t really want to use thrift with separate processes
Go has a plugins system that I’m eyeing
Maybe that in the future
Early days :p
s
Right. So what does that leave? there are go plugins, which feel unloved. There’s a hashicorp rpc thing, which is probably weird
I can probably try PR some changes if you want to see what I’m thinking about.
(Since I don’t expect you to promise to accept said PR, I don’t promise to actually do that work)
o
Sounds good :) I don’t think I fully understand so I’ll be excited to see the code if you do open it
Thanks!
s
Cool
14 Views
Open in Slack
PreviousNext
Powered by