nyanshak03/08/2021, 10:18 PM
block in this doc: https://github.com/fleetdm/fleet/blob/master/docs/1-Using-Fleet/2-fleetctl-CLI.md. Is that new? I've been using
(or windows, etc). Are there docs on what type of things can be in the support block? Looks like I can see
. Might make sense to support some sort of per-query targeting (perhaps in the
Noah Talerman03/09/2021, 10:45 PM
nyanshak03/09/2021, 10:45 PM
Noah Talerman03/09/2021, 10:45 PM
is targeting always done at the pack levelYes, targeting is always done at the pack level.
block isn’t supported.
block but the option was never actually implemented. That being said, I’d like to understand what you’re trying to accomplish by proposing the ability to target at the query level.
nyanshak03/09/2021, 10:54 PM
. In this pack, I have a query called
. In osquery 4.8.0 (🤞), there will be a new table called
that collects process events from endpoint security framework. If I want to switch to
, I need two things: 1. macOS is updated everywhere to 10.15+ 2. macOS osquery clients are at 4.8.0+ (or whatever the version where this gets released). Logically, this query still belongs in
pack, but I don't want to run both queries on
hosts, just the query that's supported.
Noah Talerman03/09/2021, 11:02 PM
nyanshak03/09/2021, 11:03 PM
Noah Talerman03/09/2021, 11:17 PM
you’ll need to move the original query into a new packWhy is this? Would you need to move the original query into a new pack because now the pack only targets a subset of the original set of hosts? In your example you can’t just leave
as is because it no longer targets all
machines (instead a subset of them)
nyanshak03/09/2021, 11:19 PM
Imagine this is the original pack and it's targeted to macOS hosts (all macOS hosts)
--- pack definition queries: - queryA - queryB - query...Etcetera - process_events
Noah Talerman03/09/2021, 11:27 PM
nyanshak03/09/2021, 11:27 PM
Noah Talerman03/09/2021, 11:33 PM
nyanshak03/09/2021, 11:42 PM
This isn't exactly what I do but it's similar and helpful for illustrating. In this example, the queries for a given pack are stored in the directory for a pack, so it's easy to reference the list of queries for a pack. --- In the 'just create a new identical pack' scenario, you can no longer manage the queries the same way, because applying the queries under one pack will overwrite the query definitions in another pack, and also you now have to maintain two copies of the pack. You could just keep a comment in pack_b that says "hey, this query is defined in pack_a". Basically, organizationally it could be problematic. --- And also, if I have another query that needs targeting specifically, now instead of two packs, I need at minimum four packs. Next split results in eight, then sixteen, etc. Using packs to get around query targeting will create an exponential growth in number of packs used.
pack_a/ queries.yml pack.yml pack_b/ queries.yml pack.yml
Noah Talerman03/09/2021, 11:46 PM
you now have to maintain two copies of the packRight and as you said this definitely doesn’t scale as you try to add more targeting by query.
I’m not sure what the right answer is exactly, but it’s been a persistent frustration.Totally understand. And my questioning / lack of understanding doesn’t help on the frustration front. Your explanations/examples are immense for my understanding. I too want to reach a good answer for this frustration. The method for arranging configurations you explained is very interesting. It seems logical to tie a specific pack to its set of queries. At a high level the issue we’ve been discussing seems to stem from new information (es_process_events_table) being only available for a specific criteria of hosts. The flexibility necessary to acquire this new info (new query) while still acquiring the rest of the info (the other queries) isn’t supported well by the pack level targeting. The new information is tied to a specific query. ^This is kind of a jumble and I’m typing as I think. Generally, I’m now curious if there are other ways to support the query level of flexibility without having targets at the query level.
Logically, this query still belongs inHi @nyanshak I’m revisiting this discussion. Why is it undesirable to run both queries onpack, but I don’t want to run both queries on
macOS monitoringhosts, just the query that’s supported.
hosts? Apologies if the answer to this question was already discussed.
nyanshak03/23/2021, 3:54 PM
Noah Talerman03/23/2021, 3:59 PM
we’d prefer one source over the other if it’s supported on that hostGot it. Am I right when thinking that the ultimate motivation for this is similar to the performance/cost motivation discussed in this thread?
nyanshak03/23/2021, 4:03 PM
Noah Talerman03/23/2021, 5:24 PM
nyanshak03/23/2021, 5:26 PM
table, which shows stats for each scheduled query execution. We then make dashboards based on this and use it to help tune poorly-performing queries.
Noah Talerman03/23/2021, 5:49 PM
nyanshak03/23/2021, 5:53 PM
Noah Talerman03/23/2021, 6:02 PM
nyanshak03/23/2021, 6:03 PM
Noah Talerman03/23/2021, 6:17 PM
table) allow you to answer this 2nd question: • What’s the measurable amount we’ve reduced osquery overhead on a host? Is this the answer to the 2nd question even valuable?
nyanshak03/23/2021, 6:21 PM
What’s the measurable amount we’ve reduced osquery overhead on a host?In our scenarios, we're not looking at an individual host except in the problematic case. And the measure we'd use would be graphing osquery resource usage on the host over some time period to collect cpu / memory stats, for example, and comparing with one version of the query vs another version