Title
#goquery
s

seph

11/07/2019, 4:50 AM
Playing with the code, it seems hard to use as a library? So if I want to use it against some internal server, I need to make a hard fork. At least, trying to use a bunch of the integration seems hard since it’s based on a global
api
object, and not something that’s passed around.
Andrew Guthrie

Andrew Guthrie

11/07/2019, 6:36 AM
Yes, integration would require adding a module in
api
that returned a custom struct that implements the
GoQueryAPI
interface. The
api
module imported by the commands is just a shim for the calls into that struct. Maybe we can implement an option if the server calls could be standardized such that the only integration would be defining the url of each action endpoint (login basic auth/saml, connect, schedule)
s

seph

11/07/2019, 11:51 AM
I was looking to do something else. I wrote a suitable api object, it's a simple enough interface. But i was expecting to be able to write my own main that instantiated it and then handed it to the goquery code.
2:19 PM
If it’s interesting, I can try to mock up what I mean
Andrew Guthrie

Andrew Guthrie

11/07/2019, 7:54 PM
Yes, it's not too obvious but you would implement another case statement in driver.go and then run main with --config pointing to the config json which states which api driver to use. It is a little burried so we can work on the implementation or write up some clear docs, let me know 😃
s

seph

11/07/2019, 8:00 PM
Yes. I forked goquery, and wrote an API for an internal service. It’s kinda fun.
8:00 PM
But that’s code I’m never going to PR back. Partly because it’s an internal service, and partly because its messy.
8:03 PM
I’d suggest that an interface that allows me to write my own main, and treat goquery as a library would be powerful. Super psuedo code:
func main() {
  intApi := myApi.New(hostname)
  p := prompt.New(intApi)
  p.Run()
}
vs the model today where the API is a global and I can’t really keep track
8:03 PM
I guess I’m sorta suggestion the entire
api/driver.go
stuff move to main.
8:03 PM
I’m not sure if this sort of project architecture conversation is welcome.
Andrew Guthrie

Andrew Guthrie

11/07/2019, 8:42 PM
Ah I see your idea, have goquery as a dependency so it can be worked on independently of internal integrations
s

seph

11/07/2019, 8:42 PM
Yeah… I find it’s a really natural way to use go interfaces/
Andrew Guthrie

Andrew Guthrie

11/07/2019, 8:43 PM
I think it's an interesting approach. @obelisk what do you think ?
s

seph

11/07/2019, 8:46 PM
I can try to find some time to PR some of this, if that’s interesting to y’all.
8:47 PM
There’s a handful of other things I was noticing too. Command line flag parsing jumped out at me. Etc.
obelisk

obelisk

11/07/2019, 8:47 PM
So my long term plan was to figure out a better way to plug and play APIs, that way you can easily separate you backend communication code while maintaining core goquery
8:47 PM
Sort of like osquery extensions
8:48 PM
Using goquery as an API was never something I considered because it’s inherently an interactive tool, if you want to use apis, I figured you would just use the backend apis directly
s

seph

11/07/2019, 8:49 PM
To be honest, do you need that? I’d generally expect that if I need a custom backend, I build my own. Making a hot pluggable thing seems like a lot of overhead for likely a small number of use cases
8:50 PM
I don’t know what the right words are… I’d rather built my own main, to use my custom weird plugins, than build a hot pluggable interface for go.
obelisk

obelisk

11/07/2019, 8:51 PM
For everyone who has a custom backend, that means they can keep using the shell that we all help build together easily while keeping the code separate and for more common backends like osctrl, fleet, zentral, we can give them even better support all in one. At least that was my idea
s

seph

11/07/2019, 8:51 PM
They’re going to have to build a plugin. Which implies they have a working go toolchain. May as well build a whole goquery. (I have some bias here)
8:52 PM
I think it’s likely a much smaller use case than osquery extensions. So using some heavy thing like thrift feels like a lot of extra dev overhead
8:55 PM
I’m partly coming from really liking the compile type safety in go. A lot of what I do is in my little monorepo, with a lot of type assurance. Plugins get away from that.
obelisk

obelisk

11/07/2019, 8:55 PM
The idea wasn’t to use thrift, but if you have a custom backend, you’ll need to build it again anyway and you’re probably comfortable with coding. My current setup for our internal backend is another folder called private and we compile it in directly
8:56 PM
Maybe osquery modules is a better model to follow
s

seph

11/07/2019, 8:56 PM
mmmm, so you’re thinking a model might be to have a local fork where one maintains some extra code in private?
8:56 PM
Personally I find that hard — git gets in the way. Either I have a hard fork, or I’m fighting with submodules.
obelisk

obelisk

11/07/2019, 8:57 PM
I like the model of osquery extensions that people have to do things specific for their infra so I wanted to try and give people that power
s

seph

11/07/2019, 8:57 PM
My guess is do something easy and see what use cases emerge
obelisk

obelisk

11/07/2019, 8:57 PM
I grant right now the system isn’t great 😛
s

seph

11/07/2019, 8:58 PM
Oh, to be clear. I wrote and have running a fork of goquery against some internal infra. So some 🍺 all around
obelisk

obelisk

11/07/2019, 8:58 PM
I don’t like sub modules and don’t really want to use thrift with separate processes
8:58 PM
Go has a plugins system that I’m eyeing
8:58 PM
Maybe that in the future
8:58 PM
Early days 😛
s

seph

11/07/2019, 8:59 PM
Right. So what does that leave? there are go plugins, which feel unloved. There’s a hashicorp rpc thing, which is probably weird
8:59 PM
I can probably try PR some changes if you want to see what I’m thinking about.
9:00 PM
(Since I don’t expect you to promise to accept said PR, I don’t promise to actually do that work)
obelisk

obelisk

11/07/2019, 9:01 PM
Sounds good 😃 I don’t think I fully understand so I’ll be excited to see the code if you do open it
9:01 PM
Thanks!
s

seph

11/07/2019, 9:02 PM
Cool