Title
#fleet
j

JojoD

03/29/2022, 5:24 PM
Hi there! I've run the deployment of a new version of fleet, everything went smoothly, however the tools is not allowing the login, in the logs I see
authentication error: find host: timestamp: 2022-03-29T17:04:30Z: missing destination name public_ip in *fleet.Host
I've remotely connected to the DB, but all the information is still there, any thoughts on this?
Kathy Satterlee

Kathy Satterlee

03/29/2022, 5:31 PM
Hi, @JojoD! Just to make sure we're on the same page, was this an upgrade to an existing Fleet instance, or a fresh install?
j

JojoD

03/29/2022, 5:31 PM
it was an existing instance
Kathy Satterlee

Kathy Satterlee

03/29/2022, 5:33 PM
Gotcha. And did you run
fleet prepare db
before serving the updated version?
j

JojoD

03/29/2022, 5:39 PM
yep, migrations were completed
5:39 PM
I run it twice, the second time it returned the message that there was nothing to do because the migration was complete
Kathy Satterlee

Kathy Satterlee

03/29/2022, 5:40 PM
Thanks for the additional context. I'll do a bit of research and get back to you shortly.
j

JojoD

03/29/2022, 5:41 PM
thank you, @Kathy Satterlee
Kathy Satterlee

Kathy Satterlee

03/29/2022, 5:41 PM
Might be relevant... how are you logging in? Through the UI, or using
fleetctl
?
j

JojoD

03/29/2022, 5:41 PM
through the UI
Kathy Satterlee

Kathy Satterlee

03/29/2022, 5:42 PM
Are you seeing errors when you actually log in, or just through the logs?
j

JojoD

03/29/2022, 5:43 PM
just through logs, in the UI it shows "Authentication failed"
Kathy Satterlee

Kathy Satterlee

03/29/2022, 5:43 PM
Thanks again! I'm sure I'll have more questions.
j

JojoD

03/29/2022, 5:46 PM
no worries
5:46 PM
thanks for the support 👌
zwass

zwass

03/29/2022, 5:47 PM
It sounds like you might be running a different version of Fleet when running the migrations vs. when running the server.
j

JojoD

03/29/2022, 5:49 PM
oh I think I used different fleet installations for making the migration
5:50 PM
no, actually I've used the same docker image 🤔
Kathy Satterlee

Kathy Satterlee

03/29/2022, 5:54 PM
Can you double check the Fleet version with
fleetctl --version
?
zwass

zwass

03/29/2022, 5:55 PM
^ That will be the
fleetctl
version. Maybe double check what you get from the
/version
API endpoint on Fleet?
Kathy Satterlee

Kathy Satterlee

03/29/2022, 5:55 PM
Thanks, @zwass
j

JojoD

03/29/2022, 5:56 PM
can that be checked using the cli?
Kathy Satterlee

Kathy Satterlee

03/29/2022, 6:04 PM
Did you upgrade
fleetctl
to the newest version along with Fleet?
6:06 PM
And you can hit the
/version
endpoint through the REST API without Auth
<https://fleet.example.com/version>
j

JojoD

03/29/2022, 6:08 PM
• thank you, so the
fleetctl
shows
fleetctl - version 4.8.0
and the other shows
version: "4.10.0",
zwass

zwass

03/29/2022, 6:13 PM
4.10 is not right. It sounds like you've run migrations for 4.12 but your server is running 4.10.
j

JojoD

03/29/2022, 6:19 PM
how can I know what version the migration was run for? also, the last version I've uploaded to the server was the 4.10.0 🤔
zwass

zwass

03/29/2022, 6:22 PM
What command are you using to run the migration? Can you replace the
prepare db
part with
version --full
?
j

JojoD

03/29/2022, 6:24 PM
this is how I run it
6:25 PM
like running
fleet version --full
?
Kathy Satterlee

Kathy Satterlee

03/29/2022, 6:25 PM
What do you get if you run
/usr/bin/fleet version --full
from the same location?
j

JojoD

03/29/2022, 6:27 PM
so it seems it does has the
4.12.0
😮
Kathy Satterlee

Kathy Satterlee

03/29/2022, 6:28 PM
There's the culprit!
j

JojoD

03/29/2022, 6:28 PM
gosh, so recaping, fleet and fleetctl should both be in the same version to run successfully
Kathy Satterlee

Kathy Satterlee

03/29/2022, 6:29 PM
Ideally, but the real culprit here is that the version of Fleet that's served and the version that was used for database migrations don't match up.
j

JojoD

03/29/2022, 6:30 PM
is there a way to know what version will run the migration?
Kathy Satterlee

Kathy Satterlee

03/29/2022, 6:30 PM
You can double check the version of Fleet installed with the
/usr/bin/fleet version --full
command before running the migrations.
j

JojoD

03/29/2022, 6:31 PM
got it
6:31 PM
thanks a lot for the help, @Kathy Satterlee and @zwass
Kathy Satterlee

Kathy Satterlee

03/29/2022, 6:31 PM
Do you still have the terminal window where you ran
fleet serve
after the upgrade open?
j

JojoD

03/29/2022, 6:32 PM
not really, I've used the cli of the docker image used to deploy the new version, is not exactly the same server
7:56 PM
hello again 😅 ... I've deployed the new version, but the behavior remains, any additional ideas? 🤔
zwass

zwass

03/29/2022, 8:06 PM
Can you check the
/version
endpoint again?
8:08 PM
It sounds like you still aren't running 4.12 on your server.
j

JojoD

03/29/2022, 8:10 PM
yup, first thing I did, the version now looks up to date
zwass

zwass

03/29/2022, 8:11 PM
What do you mean at the top when you say "the tools is not allowing the login"?
j

JojoD

03/29/2022, 8:13 PM
when headed to the UI, we get this message when trying to access with email + pass
8:14 PM
additionally we have SSO enabled, if we try this option, it returns 404
8:15 PM
one of my team members asures that the error should be in the DB, but I see all the data there, so I doubt the DB it's the problem
zwass

zwass

03/29/2022, 8:15 PM
What do you see in the Fleet server logs just after attempting this?
j

JojoD

03/29/2022, 8:19 PM
in the logs, it actually seems to be receiving data from the endpoints
8:33 PM
gosh, the problem ended up being the IdP callback URL, I've updated this URL to update it from
/api/v1/kolide/sso/callback
to
/api/v1/fleet/sso/callback
, everything is working now 🙌
8:34 PM
again thanks @zwass and @Kathy Satterlee for all the help! 🤝
Kathy Satterlee

Kathy Satterlee

03/29/2022, 8:37 PM
That's awesome! Way to stick with it, @JojoD 🙂