Hi Everyone, I'm trying to write my own query pack...
# fleet
j
Hi Everyone, I'm trying to write my own query pack to import into FleetDM. I've found a few files in the wild, but they seem to follow different syntax/schemas. Can anyone point me to a documentation for the schema used to parse query packs? I'm all out of my Google-Fu. EDIT: If there's no "real" documentation (as in "website"), I'll also go with a GitHub link or something along those lines.
k
Hi @Jörg Sachse! While the Pack functionality is still present in Fleet, it is going to be deprecated in the future to keep things streamlined since there's significant overlap with scheduled queries. There isn't a current hard end date Packs at the moment, but I would encourage you to use the scheduling functionality instead. That being said, you can find a historical version of the docs here.
m
Hi @Jörg Sachse and @Kathy Satterlee! Some background: Packs will always be supported in Fleet. As originally envisioned by Zach Wasserman and team when creating osquery, packs are a way to import and export queries into (and out of!) any platform that speaks osquery, whether that's Fleet, SecurityOnion, an EDR, or even Rapid7. Queries should be portable to minimize lock-in to particular tools. The "Packs" section of the UI that existed in kolide/fleet c. 2017 was using an overloaded definition of the word "packs". It existed as an early attempt to get to the right way of segmenting and targeting particular formations of hosts that share certain characteristics. This came with some difficulties with debugging and collaboration, since it could be hard to tell which queries were running on which hosts. It also made it harder to understand what performance impact running all those queries might be causing. But the kicker was when Fleet started adding long-requested improvements (e.g. health checks for conditional access and posture reporting, role-based access control for particular groups of hosts and queries, remote-control agent configuration, remote-control osquery extension deployment, software inventory, browser plugin inventory, NVD+CVSS+EPSS+CISA KEV integration for vulns). It became clear to Zach and I that Fleet needed a better way to organize hosts, controls, reports, and configuration that wasn't tied exclusively to data collection in Splunk. It was time to learn from the original design and come up with a way to semantically group hosts, and then apply query packs to those. The first step towards that was to add a simpler way to schedule queries, and tuck away the legacy feature called "Packs", so that "packs" refer to what they were originally: a portable way to import and export queries. PS. Recently, as part of solving something bigger, Fleet began the process of simplifying that further into just "Queries", that can either be applied to your whole fleet, or to a particular formation of hosts. (Will be compatible with query packs and backwards-compatible with automatic migration, whether or not you're using the Teams feature in Fleet Premium)
j
Hi @Kathy Satterlee and @mikermcneil, thank you so much for your elaborate answers and also for the insight into the larger picture and development context. I have looked into the documenation you provided and into Scheduled Queries in general. Now, what I'm trying to do is: 1. Manage all my queries in YAML files in a local GitLab repository. 2. Deploy them to the FleetDM server whenever there are any changes. 3. (Here's the critical point) Send their scheduling along with the queries, so I don't have to configure the scheduling manually for each query individually in FleetDM's UI. Packs provided just that using something like
interval: 3600
, but I don't see a way to accomplish the same thing with Scheduled Queries. I even tried to schedule some queries manually in the UI and then exported all configuration I could get a hold of using
fleetctl
hoping that I could get an example from there, but I wasn't lucky. Is there a way to control Scheduled Queries just using plain YAML files? I'm still in my test system and I have some 100 queries already, so scheduling them individually in production would be a real pain. If there's no way to do that, I'll probably go with packs anyway, but as that's not where the development effort is going right now, I'd prefer and try to use FleetDM as intended. :-) Thanks again!
Oh yeah, if I understand the release notes on GitHub correctly and I didn't overlook anything in FleetDM itself, it seems like the "packs" feature has been completely removed from the UI and is only available via
fleetctl
anymore, is that correct? Also, if I import a pack with intervals for the queries set, are these converted into Scheduled Queries?
k
That's correct. Packs are still available in the UI if you go directly to
/packs/manage
, but the button has been removed. We're working on expanding the
fleetctl
functionality to allow you to schedule queries using YAML files. It definitely sounds like for you, sticking with Packs in the short term is the way to go so you can get that automation flow going until that functionality gets added!
m
^@Rachael Shaw re: YAML
+ @Mo Zhu too re: the changes y'all were working on