Where are Fleet servers expected to store their connection i

Question

Where are Fleet servers expected to store their connection information MySQL or Redis

sundsta · Answer

The distributed live queries state lives in redis

BJF · Answer

Thank you How would one query that

sundsta · Answer

You would use a Redis client there are a few The connection settings would be the same as what you have configured Fleet with but you would of course need to be on a network that could access the Redis instance

BJF · Answer

I have access to redis sli on the redis server itself What I would like to check initially would be the connection status of osquery clients

BJF · Answer

CLIENT LIST does list the connections from both Fleet servers

sundsta · Answer

The osquery agents talk to Fleet not redis They would never be listed there

BJF · Answer

What I am trying to resolve is why osquery agents connected to Fleet server 2 appear offline to Fleet server 1 but not the other way around

BJF · Answer

Thank you Would I find connection information in MySQL If so how might I locate that information

sundsta · Answer

I don t think active connections are stored in MySQL but there is a `last seen` column I would guess that is what is used to show in the UI if a host is online or not but I m not certain

BJF · Answer

Interesting I will take a look into that Thanks

BJF · Answer

In MySQL I found something interesting the seen time field in the hosts table does not get updated when osquery clients are not logged into fleet server 2 although they are updated when connected to fleet server 2 I suspect some permissions issue is in place

sundsta · Answer

Whoops yeah the column is `seen time` not `last seen` But it sounds like you re on the right track

BJF · Answer

slightly smiling face

BJF · Answer

Interesting only one fleet server at a time can write to kolide hosts seen time

sundsta · Answer

That seems odd and unexpected I d be interested what the logs say if you turn on debug logging

BJF · Answer

When Fleet server 1 is up only it can write Fleet server 2 can only write when Fleet server 1 is down

BJF · Answer

I will look into that

BJF · Answer

The storage method is InnoDB Is that relevant

BJF · Answer

I found root cause of the issue the time of the two Fleet servers were not synchronized

BJF · Answer

Once I ensured that chronyd was functioning properly the issue was resolved One server s time was two minutes behind the other so client connection times to it always appeared out of date Synchronizing the time resolved this