Hi all I am seeing majority of our hosts going offline in fl osquery #fleet

Hi all! I am seeing majority of our hosts going of...

Rajesh Kumar

07/26/2023, 7:53 PM

Hi all! I am seeing majority of our hosts going offline in fleet. Any leads to troubleshoot these? Also whats the location of file to check environment variables set at the moment? Thanks in advance 🙂

Kathy Satterlee

07/26/2023, 7:56 PM

The easiest way to check the current config is with

fleetctl get config --include-server-config

. For the offline hosts, are you seeing any errors in the Fleet logs? If not, do you have access to the osquery logs on any of the offline hosts? Those will likely give us some more information if there are communication issues.

frederick ferby

07/26/2023, 8:08 PM

Thanks @Kathy Satterlee, Let me check if I can pull this up.

Rajesh Kumar

07/26/2023, 8:20 PM

We could pull the config. Is there any specific parameter you would like us to check the values in the config file?

Rajesh Kumar

07/26/2023, 8:30 PM

@Kathy Satterlee I have shared logs of one of such offline machine.

Kathy Satterlee

07/26/2023, 8:38 PM

Sorry, should have been clearer there 🙂 The osquery status logs are what we're looking for.

frederick ferby

07/26/2023, 9:00 PM

Ok we didn’t set the logging pipeline to another destination yet however looking locally at /tmp/osquery_status we didn’t see anything.

Kathy Satterlee

07/27/2023, 3:49 PM

The default location for those is

/var/log/osquery/

if you're using osquery's filesystem plugin.

frederick ferby

07/27/2023, 5:38 PM

Gotcha. Does this path change using the default setting for fleet charts?

frederick ferby

07/27/2023, 5:55 PM

Looks like /logs/ is the path using fleet charts. We were checking the local path for orbit of a few computers. https://fleetdm.com/docs/using-fleet/fleetd#troubleshooting Realized - We saw the errors below. I see these agents are actually connected and online with our fleetdm panel. Is this indicative of anything? in this path /private/var/log/orbit/orbit.stderr.log 2023-07-27T133808-04:00 INF periodic check of token failed, initiating rotation error=“missing or invalid license” 2023-07-27T133822-04:00 INF calling flags update 2023-07-27T133852-04:00 INF calling flags update 2023-07-27T133922-04:00 INF calling flags update 2023-07-27T133952-04:00 INF calling flags update 2023-07-27T134022-04:00 INF calling flags update 2023-07-27T134052-04:00 INF calling flags update 2023-07-27T134122-04:00 INF calling flags update 2023-07-27T134152-04:00 INF calling flags update 2023-07-27T134222-04:00 INF calling flags update 2023-07-27T134252-04:00 INF calling flags update 2023-07-27T134308-04:00 INF periodic check of token failed, initiating rotation error=“missing or invalid license” 2023-07-27T134322-04:00 INF calling flags update 2023-07-27T134352-04:00 INF calling flags update 2023-07-27T134422-04:00 INF calling flags update 2023-07-27T134452-04:00 INF calling flags update 2023-07-27T134522-04:00 INF calling flags update

3 Views

Open in Slack

Previous Next