Channels
android_tests
apple-silicon
arm-architecture
auditing-warroom
aws
carving
code-review
community-feeds
core
darkbytes
doorman
ebpf
eclecticiq-polylogyx-extension
extensions
file-carving
fim
fleet
fleet-dev
fleetosquery
foundation
fuzzing
general
golang
goquery
infrastructure
jobs
kolide
linen-dev
linux
macos
officehours
osctrl
plugins
process-auditing
querycon
queryhub
random
selfgroup
sql
tls
uptycs
vendor-feeds
website
windows
zeek
zentral
zercurity
Title
o
Ojas
01/31/2023, 12:08 PMHey Team,
I am seeing 0 online devices whereas everything was fine until few hours ago. I dont see any err in logs or any error that its not able to communicate with fleet server.
Also for alot of devices it still shows last fetched a minute ago and thats keeps updating. Not able to run any query though
r
Rachel Perkins
01/31/2023, 3:37 PMThe second part is interesting, so for some specific hosts, you can see a recent
updated_atofflineo
Ojas
01/31/2023, 4:32 PM@Rachel Perkins nope refetch dosent work nor does the query
dose this log mean that netskope is not allowing the connection? it says provider rejected
k
Kathy Satterlee
01/31/2023, 5:34 PMIs Netskope a recent addition to your environment? If you're using a proxy server, do you see any logs there that indicate that connections are being refused and why?
o
Ojas
02/01/2023, 4:24 AMnah! netkope is old one @Kathy Satterlee
its been there for many months and was there when we deployed fleet.
@Kathy Satterlee @Rachel Perkins this is becoming a p1 issue for us as our crucial part of vuln management is dependent on fleet side. Any thing i should do to debug this?
k
Kathy Satterlee
02/01/2023, 3:18 PMI know you said you didn’t see any error in the logs. Which logs have you checked? There could be useful information in the logs for your Load balancer, Fleet server or the host’s osquery logs.
o
Ojas
02/01/2023, 4:43 PMi am checking the logs on host machine:
service is running and everything seems fine, i dont see any error logs in the machine
k
Kathy Satterlee
02/01/2023, 5:10 PMOkay. Anything happening in the Fleet logs to either indicate an error in that end or confirm that traffic is making it through to Fleet?
o
Ojas
02/01/2023, 7:13 PM@Kathy Satterlee which logs do i check like the path of the fleet logs?
i’ll have to login to fleet server for this right?
FYI i am running it on ECS so not sure how to login
k
Kathy Satterlee
02/01/2023, 7:17 PMThe Fleet logs will likely be in Cloudwatch
o
Ojas
02/02/2023, 10:10 AMHey @Kathy Satterlee i got these logs and seems to be auth issues. missing node keys & no matching secret found.
@Kathy Satterlee any idea on how to fix this? I dont see any other error logs in there
k
Kathy Satterlee
02/08/2023, 4:27 PMSo sorry, @Ojas. I missed that update!!
It sounds like all of those errors are related to one host, with an invalid enroll secret. Just to be sure though, what are you using as your identifier?
o
Ojas
02/08/2023, 6:52 PMi had this error on all the hosts @Kathy Satterlee. All hosts stopped reporting back all of sudden. I updated fleet to latest and then i can see most hosts online now.
But still i see many hosts which i know are running but on fleet they are offline.
i dont see any identifiers in agent options so it should be whatever default value is
k
Kathy Satterlee
02/08/2023, 7:03 PMYes, I just wanted to make sure that the errors in the logs were all related to the same host. If you were using something that could be duplicated as the identifier, that might explain the other issue. If not, the errors in the Fleet log are likely unrelated. Can you run
fleetctl get config --include-server-configo
Ojas
02/09/2023, 11:09 AMI am working on updating the whole agents and things after that i’ll fetch this and share with you 🙂