Channels
android_tests
apple-silicon
arm-architecture
auditing-warroom
aws
carving
code-review
community-feeds
core
darkbytes
doorman
ebpf
eclecticiq-polylogyx-extension
extensions
file-carving
fim
fleet
fleet-dev
fleetosquery
foundation
fuzzing
general
golang
goquery
infrastructure
jobs
kolide
linen-dev
linux
macos
officehours
osctrl
plugins
process-auditing
querycon
queryhub
random
selfgroup
sql
tls
uptycs
vendor-feeds
website
windows
zeek
zentral
zercurity
Title
n
nyanshak
12/08/2019, 7:15 PM@Seren / @seph re: disabling distributed queries
... moving to thread so it's easier to follow this conversation ...
Isn't that sort of possible through options? e.g., in fleet, you could set
disable_distributedtrueThere are still some drawbacks, since I think you can't exactly disable someone from running them, but they just won't get picked up by the host, I think?
s
seph
12/09/2019, 2:43 AMAs an osquery option? I don’t know if fleet ever uses distributed queries for internal purposes
s
Seren
12/09/2019, 3:07 AMI can reconfirm, but if I run osqueryd (not launcher) with distributed queries disabled, the host information doesn’t seem to get completely populated. I’ll check on Monday if running launcher with distributed queries disabled causes problems (I’ve been doing testing mostly by running osqueryd directly rather than launcher so I need to make sure I’m remembering correctly). With osqueryd there were 3 fields in the hosts table which weren’t being populated.
n
nyanshak
12/09/2019, 3:43 PM🤷 Fleet does use distributed queries for internal purposes. You can ""safely"" ** disable distributed queries via osquery options / set them in fleet with a few caveats...
1. You won't get all the host info
2. Queries entered into distributed query panel still seem to get saved for the host, because I don't think fleet really tracks this. So if you re-enable distributed queries and you've run queries against all hosts, then you'll see those hosts get and push data for the queries....
3. The UI / support for disabling is not that great, and it still will totally look like you can run distributed queries against hosts even though they have it disabled. See: https://github.com/kolide/fleet/issues/2140
z
zwass
12/09/2019, 5:31 PMThe above understanding looks correct to me. There isn't (currently) a clean way to disable distributed queries while maintaining consistency in the UI.
s
seph
12/09/2019, 5:41 PMI’m curious about the use case. Why disable them?
s
Seren
12/09/2019, 5:49 PMThanks for the info. I'm ok with any UI issues. The only thing that is a blocker is that query-packs aren't being sent to the to the hosts. There were 3 fields in the hosts table that weren't being populated correctly (primary_ip_id, config_tls_refresh, uptime), but I've updated osquery to 4.11 so I want to reconfirm that everything's the same now when a) using launcher and b) having started with a fresh Fleet db.
z
zwass
12/09/2019, 5:52 PMThe host details and labels are both populated by distributed queries, so what you can do will be quite limited. For a hack, you could go into the code and disable the endpoint that actually creates a query. I think this would allow the details and labels to populate while preventing a user from creating a new query.
s
Seren
12/09/2019, 5:55 PM@seph We're rolling out osquery for servers and workstations. Servers are pretty straight-forward and distributed-queries are great. Workstations are a bit more sensitive since in our org is small everyone is used to freedom and privacy (users are technical and have admin on their machines).
The compromise to running osquery on the workstations is that we only run pre-approved query-packs and disable the ability to randomly snoop on what's going on (nevermind that we could schedule snooping queries every minute, we're trying to keep everyone calm and happy).
s
seph
12/09/2019, 5:59 PMI was wondering about that underlying intent — It’s a weird balance. Y’all are aware that you can just schedule things, but it’s seen as a higher bar than the LQ side.
It’s hard to find the right balance there.
s
Seren
12/09/2019, 5:59 PM@zwass Interesting idea. I may have to do that, though at that point users are trusting we're running a modified server, so just a promise not to run distributed queries would probably be sufficient. I was hoping there could be something client-side that would make it obvious distributed queries were disabled, but that may not be possible.
It's ok in this case for the UI (labels and status) to be borked, as long as the query packs are sent to the hosts and the logs can be received from clients. I'll get back in a minute and see if query-pack assignment still doesn't work
s
seph
12/09/2019, 5:59 PMWe’ve done some work around that in our SaaS (which is fairly different than fleet). Blacklists might also get you somewhere
z
zwass
12/09/2019, 6:01 PMI think query pack assignment will no longer work because it uses labels and they will not be populated.
s
Seren
12/09/2019, 6:01 PMInteresting, I'll have to look at blacklists, though the users would still have to trust that we're not running the queries, no?
Btw, what is "LQ"?
z
zwass
12/09/2019, 6:02 PMLQ - Live Query
s
Seren
12/09/2019, 6:03 PMAhh, thanks. Yeah, that seemed to be the behavior I saw, though it seemed like labels were being populated:
This was with osquery 4.02:
With osqueryd (LQ disabled):
osqueryd --host_identifier=$(scutil --get ComputerName) --verbose --allow_unsafe --tls_hostname=<http://fleet.globus.org:443|fleet.globus.org:443> --tls_server_certs=/etc/osquery/globus_osquery.crt --enroll_secret_path=/etc/osquery/globus_osquery_secret --enroll_tls_endpoint=/api/v1/osquery/enroll --config_plugin=tls --config_tls_endpoint=/api/v1/osquery/config --config_refresh=60 --disable_distributed=false --logger_plugin=tls --logger_tls_endpoint=/api/v1/osquery/log --logger_tls_period=60 --distributed_tls_read_endpoint=/api/v1/osquery/distributed/read --distributed_tls_write_endpoint=/api/v1/osquery/distributed/writeMySQL [kolide]> select id, platform, primary_ip_id, config_tls_refresh, uptime from hosts;
+----+----------+---------------+--------------------+--------+
| id | platform | primary_ip_id | config_tls_refresh | uptime |
+----+----------+---------------+--------------------+--------+
| 11 | darwin | NULL | 0 | 0 |
+----+----------+---------------+--------------------+--------+
1 row in set (0.00 sec)build/launcher --hostname <http://testfleet.globus.org:443|testfleet.globus.org:443> --enroll_secret 'xEXAMPLExEXAMPLE' --root_pem=/etc/osquery/globus_osquery.crt --osqueryd_path=/usr/local/bin/osqueryd --root_directory ~/.osquery --debugMySQL [kolide]> select id, platform, primary_ip_id, config_tls_refresh, uptime from hosts;
+----+----------+---------------+--------------------+-----------------+
| id | platform | primary_ip_id | config_tls_refresh | uptime |
+----+----------+---------------+--------------------+-----------------+
| 12 | darwin | 109 | 300 | 917500000000000 |
+----+----------+---------------+--------------------+-----------------+Ok, retested now. Running launcher as well as osqueryd failed to produce hosts that can be assigned packs (as zwass predicted). The osqueryd host at least seems to have some fields populated, but the launcher host hangs (at least in the UI):
The db reflects this:
| id | osquery_host_id | created_at | updated_at | deleted_at | deleted | detail_update_time | node_key | host_name | uuid | platform | osquery_version | os_version | build | platform_like | code_name | uptime | physical_memory | cpu_type | cpu_subtype | cpu_brand | cpu_physical_cores | cpu_logical_cores | hardware_vendor | hardware_model | hardware_version | hardware_serial | computer_name | primary_ip_id | seen_time | distributed_interval | logger_tls_period | config_tls_refresh |
| 25 | ip-192-168-86-26.ec2.internal | 2019-12-09 18:13:22 | 2019-12-09 18:17:24 | NULL | 0 | 1970-01-02 00:00:00 | 1k17AggNWIxQgBTvfRyBs0mLYQHKHsgo | ip-192-168-86-26.ec2.internal | A61DC7D2-0C6D-56B9-92A5-REDACTED | darwin | 4.1.1 | Mac OS X 10.14.6 | 18G1012 | darwin | | 0 | 17179869184 | x86_64h | Intel x86-64h Haswell | Intel(R) Core(TM) i7-6567U CPU @ 3.30GHz | 2 | 4 | Apple Inc. | MacBookPro13,2 | 1.0 | redacted-serial | mylaptopname-current | NULL | 2019-12-09 18:17:25 | 10 | 10 | 0 |
| 26 | d3bc7341-f65e-4df5-a9a6-f26c7fe42c1e | 2019-12-09 18:13:47 | 2019-12-09 18:14:47 | NULL | 0 | 1970-01-02 00:00:00 | Ux8TA4D2yF9bnNcNFIzCOFTZg2KI0uwx | | | | | | | | | 0 | 0 | | | | 0 | 0 | | | | | | NULL | 2019-12-09 18:14:48 | 10 | 10 | 0 |(sorry for the text flood in a thread column)
Anyway, it seems like trying to disable LQ for the client is a no-go, so I'll either need to distribute query-packs and configs, and use Fleet (at least for workstations) as a centralized log collector, or see if promising not to use LQ is sufficient.
z
zwass
12/09/2019, 6:24 PMYeah, using Fleet just to receive logs from such hosts seems like the best solution here IMO. This also makes it easier for users to see the full osquery configuration (which would be more tricky to do if the query packs come from Fleet).
s
Seren
12/09/2019, 6:26 PMThanks for your (as seph's) help!