https://github.com/osquery/osquery logo
Title
t

Terje Kvernes

11/29/2022, 12:22 PM
We have some metadata on hosts that we would like to have available to join into some searches. This is metadata like purchase orders, location, and similar. This data is not located on each host, but in a central system that we can extract data from. How would I go about making this data available in fleet? I am happy to write an extension or script table generation as needed, as long as I can find an outline of how to (correctly?) solve the issue. 🙂
b

Benjamin Edwards

11/29/2022, 1:33 PM
To me this sounds like the job for an ETL data pipeline. Scheduled query results -> fleet backend -> message broker(Kafka/Kinesis) -> some application logic for joining data from central database based on host identifiers etc. -> datalake/seim My only worry about making the data fetching happen on the host that’s queried, through extension or otherwise, is that you could easily DDOS said central repository if you live query thousands of hosts at once. Maybe that isn’t a concern in your deployment. An alternative might be to if the central repository of data is accessible via API you could use curl table to fetch the data for each host, but you’d have to figure out a way to identify each host in said API layer. Custom extension would also be possible if that makes sense for your deployment. Happy to discuss ideas further.
k

Keith Swagler

11/29/2022, 4:58 PM
It sounds like you should probably go the other way around and have Fleet feed things to an Asset Management application
t

Terje Kvernes

11/29/2022, 6:07 PM
The “problem” is that the fleet interface is so nice to search from, relative to the Asset Management solution we (kinda) have, so I see our users vastly prefer to sit in fleet and ask things like “what research groups do the machines running older OSen belong to”. I’m kinda torn really, because asset management is also not one thing for us, we have at least three systems that own different bits of the metadata for the system, and for all of them, the UI pales relative to fleet. I suppose the answer here is to create a new portal to all three solutions, plus fleet, XKCD style:

https://imgs.xkcd.com/comics/standards_2x.png

j

Jason

11/30/2022, 1:50 AM
Not a bad suggestion for the fleet folks to have an api specifically for feeding custom data into the fleet database to be viewed in the UI. This stuff gets really messy really fast though unless it’s really basic basic like “ custom field 1” “custom field 2” etc that are fixed
It still “feels” wrong