hey :wave: I have fleet setup with my hosts report...
# kolide
hey 👋 I have fleet setup with my hosts reporting to it. I'm trying to get the firehose output to work correctly. So far I get the status log in there, but not the results. They are both set to save to the same firehose stream, could that be the issue? I have a pack enabled with
select * from time
running every 30s but I don't see any results from that. I can run manual queries and see the result in the UI fine, but nothing in firehose. When I do that I see
Executing distributed query: kolide_distributed_query_45: SELECT * FROM time
sent to firehose but no actual result. Here are the fleet env vars I'm using (excluding mysql and redis). I did a manual write to the stream with the same creds with awscli successfully (The role has permission to describe and put record batch).
Copy code
I'm using firehose with elasticsearch and s3 backup, and there is no failed delivery files in s3. What is the best way to debug this issue? I don't see any errors in the fleet output itself. Thanks!
sorry had posted in general by mistake
First, live query results will never go to the logger. You'll only see them through the UI/fleetctl.
My guess is that pack is not targeted appropriately to the host. Can you take a look at how it is targeted?
Also, take a look at the logs on the host to see whether it is every running the scheduled query.
the pack is targeted to all hosts
let me check the logs on a host
the only thing I see in there is the distributed query read and log.
Copy code
--tls-hostname XXX
my flags
do I need a special one to enable packs?
oh, I think I just realised, I need the launcher for the packs right? the osqueryd doesn't have the ability to fetch new config right?
You don't need Launcher. The osquery_status logs should at least show you if it is connecting to Fleet and if it is executing the scheduled queries
I do see
Jul 03 21:17:46 ip-172-23-6-159 osquery.sh[1019]: I0703 21:17:46.221714  1263 tls.cpp:240] TLS/HTTPS POST request to URI:
for distributed read and log
no other log
Then it is not a target for that pack. You're looking for logs like
{"hostIdentifier":"FB2C3611-EA6B-C10B-3611-3C4CF096B325","calendarTime":"Wed Jul  3 21:17:52 2019 UTC","unixTime":"1562188672","severity":"0","filename":"scheduler.cpp","line":"100","message":"Executing scheduled query pack/events/user_events: SELECT * FROM user_events \nWHERE time < DATE('now') - 300","version":"3.3.2","decorations":{"host_uuid":"FB2C3611-EA6B-C10B-3611-3C4CF096B325","hostname":"REDACTED.local"}}
(we log in JSON, so your formatting will be a bit different)
right sorry I do have those
they are forwarded to firehose
OK, so it's working?
things like opening rocksdb etc
but nothing about scheduled
OK, then that host isn't targeted for the pack or the pack contains no queries.
This pack looks targeted correctly
Can you run osqueryd on the host with —verbose —tls_dump and see what config it receives?
The issue was I didn't have config-refresh set on osqueryd, I do get my results now. Thanks @zwass @sundsta 🙂
👍 1