Title
#fleet
j

jimmy

05/15/2022, 7:04 PM
hi, im using fleet 3.5.1 and i have 3 fleet servers, and for some reason my load balancer gives the queries only to the two first servers, and i looked at yaml file of the fleet service that doesn't get queries and he is the same like the other fleet service that getting queries. and when I use fleetctl the third fleet works. do you have any solution?
Lucas Rodriguez

Lucas Rodriguez

05/16/2022, 2:27 PM
Hi @jimmy! Fleet is unaware of how it get requests or if there are other Fleet instances running as part of the deployment. So, this is most likely a configuration/network issue between the load balancer and the hosts running your Fleet instances. Here's a guide on how to deploy Fleet: https://fleetdm.com/docs/deploying/server-installation#deploying-the-load-balancer (with a section on deploying a load balancer)
2:28 PM
If possible, we advise users to upgrade to Fleet's latest version,
4.14.0
(there have been several performance improvements since
3.5.1
).
2:32 PM
and when I use fleetctl the third fleet works. do you have any solution?
1.
fleetctl
--> 3rd Fleet Server 2.
fleetctl
--> load balancer --> 3rd Fleet Server If you are doing (1) then I'd suggest you try with (2) to test that the request is really going through the load balancer.
j

jimmy

05/16/2022, 6:32 PM
hi @Lucas Rodriguez the section of the load balancer is only with kubectl, and my server doesn't have kubernetes. and its
fleetctl
--> 3rd Fleet Server
6:33 PM
@Lucas Rodriguez "If you are doing (1) then I'd suggest you try with (2) to test that the request is really going through the load balancer." how can i check this
6:33 PM
@Lucas Rodriguez and thanks for the help 🙂
Lucas Rodriguez

Lucas Rodriguez

05/16/2022, 6:35 PM
how can i check this
So if I understood correctly, your devices will connect to Fleet through the load balancer (via some defined URL). Have
fleetctl
use the same URL as the osquery hosts. To be sure that you are connecting to Fleet the same way as your devices.
j

jimmy

05/16/2022, 6:41 PM
@Lucas Rodriguez I don't think that the fleetctl go threw the load balancer
6:42 PM
@Lucas Rodriguez but how can i check this to be sure?
Lucas Rodriguez

Lucas Rodriguez

05/16/2022, 6:49 PM
fleetctl config get address
will return the URL it's using to connect to Fleet. Such URL should match the one your devices use to connect to Fleet. E.g. in my development setup:
$ fleetctl config get address
  default.address => <https://localhost:8080>
6:50 PM
but how can i check this to be sure?
A way to check this would be to run the third Fleet server in debug mode (
FLEET_LOGGING_DEBUG
var or
--logging_debug
cli argument) and see if it's getting any requests (Fleet logs one line per request when
--logging_debug
is set)
7:31 PM
j

jimmy

05/17/2022, 8:13 AM
@Lucas Rodriguez thanks for the help, "fleetctl config get address" i did this command and my server gave me wrong ip of the load balancer.
8:13 AM
@Lucas Rodriguez so what should i do now?
Lucas Rodriguez

Lucas Rodriguez

05/17/2022, 11:50 AM
You can configure
fleetctl
address with:
fleetctl config set --address https://$LOAD_BALANCER_IP:$LOAD_BALANCER_PORT
j

jimmy

05/17/2022, 7:24 PM
i did this but sadly it still doesn't get queries from the load balancer
7:25 PM
should i maybe restart the load balancer after that?
Lucas Rodriguez

Lucas Rodriguez

05/17/2022, 7:26 PM
i did this but sadly it still doesn't get queries from the load balancer
What do you mean by this?
7:27 PM
fleetctl config set --address https://$LOAD_BALANCER_IP:$LOAD_BALANCER_PORT
This command is only configuring
fleetctl
to connect to Fleet via the load balancer.
j

jimmy

05/17/2022, 8:29 PM
"hi, im using fleet 3.5.1 and i have 3 fleet servers, and for some reason my load balancer gives the queries only to the two first servers, and i looked at yaml file of the fleet service that doesn't get queries and he is the same like the other fleet service that getting queries. and when I use fleetctl the third fleet works. do you have any solution?" I mean that I still have that problem
Lucas Rodriguez

Lucas Rodriguez

05/17/2022, 9:41 PM
OK, we are just trying to connect via
fleetctl
via the load balancer to be able to troubleshoot.
9:42 PM
https://osquery.slack.com/archives/C01DXJL16D8/p1652729790761829?thread_ts=1652726641.378669&amp;cid=C01DXJL16D8 this should also help troubleshooting. Basically have your load balancer use Fleet's healthz endpoint.