https://github.com/osquery/osquery logo
Join Slack
Powered by
I think there's a bug with using an extension's ta...
# general
h
I think there's a bug with using an extension's table as a discovery query for a pack. Details in thread.
It's a python extension, and here's the lines from 
sudo osqueryi --verbose
that reference it:
Copy code
I1101 01:06:38.005260 15364 watcher.cpp:732] Created and monitoring extension child (15369): /usr/lib/osquery/bookings_meminfo.ext
W1101 01:06:38.069183 15363 packs.cpp:326] Discovery query failed (select valueKB from bookings_meminfo where key='Hugepagesize';): no such table: bookings_meminfo
I1101 01:06:38.081598 15374 interface.cpp:137] Registering extension (meminfo, 18355, version=1.0.0, sdk=1.8.0)
I1101 01:06:38.082294 15374 registry_factory.cpp:107] Extension 18355 registered table plugin bookings_meminfo
This one's kind of a nonsense discovery query, but we have a more complex one that's actually causing trouble.
I get no such problem when I use one of the core tables in this manner.
Also, once the prompt comes up for me, I can easily do a
Copy code
select * from bookings_meminfo;
and results work fine.
Ideas?
s
Hrm. I wonder if the discovery query is triggering before the table is loaded? (IIRC those might run and get cached) Does it change if you use the osquery flag 
--extensions_require
so osquery waits for this extension?
h
@seph, I think I tried that before, but will test it today and let you know
Oh, and that's definitely my read on things from the --verbose events order above...
@seph Hmm, confusing. I'm assuming that it wants the extension's filename (without path or the 
.ext
at the end). That works for one of the extensions, to launch osqueryi with that 
--extensions_require
parameter. But if I change the discovery query to another custom extension (we have 5 presently), and attempt require the other one (or both, comma-delimited) it doesn't work.
s
I would expect extensions_require to be looking for the name the extension registers itself with. Which is unrelated to the file name
h
Hmm, seems experimentally like that's not the case. Most of ours line up 1-to-1, but not one of them. That one is the unique one that seems to work at suppressing the error.
Oof, I need to go read the documentation and see what's intended for each. These things don't seem like they line up:
Copy code
bookings_meminfo.ext
14:        return "bookings_meminfo"
59:    osquery.start_extension(name="meminfo", version="1.0.0")

sysconfig_bookings.ext
19:        return "bookings_puppet"
91:    osquery.start_extension(name="sysconfig_bookings", version="1.0.2")

bookings_scsi_devices.ext
14:        return "bookings_scsi_devices"
71:    osquery.start_extension(name="scsi_info", version="1.0.0")

bookings_last.ext
18:        return "bookings_last"
62:    osquery.start_extension(name="bookings_last", version="1.0.1")

bookings_lsblk.ext
14:        return "bookings_lsblk"
47:    osquery.start_extension(name="bookings_lsblk", version="1.0.0")
Yeah, that may be my root cause. Will get those all ship-shape and see if this problem goes away.
Thanks!!
@seph are there any interdependencies between file name, table name, and extension name (the one referenced by start_extension)?
s
Nope! I haven’t read the code, so it’s a bit 🤷 but I’m not aware of anything. I will say that Kolide’s Launcher is pretty weird — it manages itself, and registers 2 extensions with names that have nothing to do with the file name. And osquery doesn’t know the file name anyhow. So I think you have a lot of flexibility
h
Thanks so much; I think in covering that factoid, we've stumbled across a solid lead in trying to reduce some errors in our setup. 🙂
s
Happy to help!
one other thing to note, is that it’s a bit of a toss up whether having 1 extension or many is “better” Combining into a single extension means fewer processes running, for example
h
Oh, that's an interesting idea. We've got 5 because they're going after completely different things. Is your thought process that we could just mash the results together into a virtual table?
s
No. Just register 5 tables in one extension
h
Oh, hmm. Have to think about that, especially regarding if one of them "dies" somehow, would the watchdog restart the whole set of them?
s
I have no idea how the watchdog and extensions intersect.
h
Well, that's a cool idea. I'll backlog a ticket to play with that.
s
Somewhat there’s a question here aboiut what’s easiest to manage. From lots of perspectives: • software dev • extension distribution • runtime stability • runtime performance Probably no really simple answer. Though at least a few years ago we all biased towards single extension, multiple things inside it
(The SDKs are based around it)
h
Oh, wow. I've been oblivious to that.
s
Always something new to learn 🙂
h
(that's standing in for amen)
Any examples of 2+ TablePlugin within one file? I'm looking at the example, with one in it, and my assumptions are: 1. gather all the unique 
import
lines to the top 2. have an 
@osquery.register_plugin
macro at the top of each class definition 3. have a start_extension() call for each of them down in the 
name == main
section
s
Sounds like python? I don’t really use python. but I suspect it’s calling 
REGISTER_EXTERNAL
several times.
Oh, which is probably 
@osquery.register_plugin
in your comment.
I’d expect onle a single 
start_extension()
Basically, you create an extension. Then you register plugins to the extension. Then you start the extension
h
Oh, yup. Oh. Here I was thinking that the name was the table name, but you're right. Probably just the one. Not
Copy code
osquery.start_extension(name="bookings_puppet", version="1.0.3")
osquery.start_extension(name="bookings_last", version="1.0.1")
osquery.start_extension(name="bookings_lsblk", version="1.0.0")
osquery.start_extension(name="bookings_meminfo", version="1.0.1")
[...]
Thus the extension name is distinct from the table names
s
Yeah. not that. 🙂
h
Okay, thanks! Helped to have a fresh set of eyes!
s
name is the extension name, version is the extension version.
You can poke around some of the osquery tables that expose this. 
osquery_extensions
and 
osquery_registry
h
Okay, will do. I strongly suspect we're going to get rid of a couple classes of errors by doing this. 🤞
3 Views
Open in Slack
PreviousNext
Powered by