Channels
#fleet
Title
# fleet
p
pvirani
02/01/2022, 12:32 AMHey friends! Is there no way to get device enrolment to work with OAuth on at the moment? (assuming that's why my enrolment is failing going by the following error message)
Copy code
Could not read SMBIOS memory
I0201 00:31:42.133729 1860 tls.cpp:255] TLS/HTTPS POST request to URI: <https://fleetdm.segment.build/api/v1/osquery/enroll>
I0201 00:31:42.152062 1860 http_client.cpp:420] HTTP(S) request re-directed to: <https://segment.okta.com/oauth2/v1/authorize?client_id=**********************************>
^CW0201 00:31:42.745014 1860 tls_enroll.cpp:101] Failed enrollment request to <https://fleetdm.segment.build/api/v1/osquery/enroll> (Cannot parse JSON: Invalid value. Offset: 0) retrying...z
zwass
02/01/2022, 12:38 AM
Hmmm, sounds like you have an oauth proxy? You probably need to add exceptions for the
/api/v1/osquery/*🙏🏽 1
💯 1
If that's not possible, you can expose only those API endpoints on a different hostname (filtering routes via nginx or your load balancer).
💯 1
p
pvirani
02/01/2022, 12:40 AMI think the exceptions route should work. I didn't even know that was an option with Oauth setups. TIL!
🍻 1
I removed SAML and OIDC from my Stage deployment and the enrolment seemed to have worked briefly but now I have this very weird issue where the hosts tab is completely blank and nothing loads in the activity section. Screenshots below. Queries weren't loading for a while either but they've started loading now somehow. (have already tried logging out, back in etc basic troubleshooting methods)
I'm trying to debug it on my own but somehow my App logs aren't showing up via kubectl either so there's that I'm trying to solve first but also posting here just in case you or anyone else in the community has seen a similar behaviour before.
Queries page has stopped loading again too! Screenshot below
z
zwass
02/01/2022, 7:57 PM
Can you open up the network inspector and see what's happening with the requests?
p
pvirani
02/01/2022, 10:12 PM Copy code
ts=2022-02-01T21:44:22.46995579Z mysql="could not connect to db: dial tcp 10.80.113.69:3306: connect: connection timed out, sleeping 0s"
ts=2022-02-01T21:46:33.541954613Z mysql="could not connect to db: dial tcp 10.80.113.69:3306: connect: connection timed out, sleeping 1s"
ts=2022-02-01T21:48:44.613944779Z mysql="could not connect to db: dial tcp 10.80.113.69:3306: connect: connection timed out, sleeping 2s"
ts=2022-02-01T21:50:57.733936566Z mysql="could not connect to db: dial tcp 10.80.113.69:3306: connect: connection timed out, sleeping 3s"
ts=2022-02-01T21:53:10.853938337Z mysql="could not connect to db: dial tcp 10.80.113.69:3306: connect: connection timed out, sleeping 4s"
ts=2022-02-01T21:55:26.021983497Z mysql="could not connect to db: dial tcp 10.80.113.69:3306: connect: connection timed out, sleeping 5s"
ts=2022-02-01T21:57:41.193944455Z mysql="could not connect to db: dial tcp 10.80.113.69:3306: connect: connection timed out, sleeping 6s"
ts=2022-02-01T21:59:58.405983358Z mysql="could not connect to db: dial tcp 10.80.113.69:3306: connect: connection timed out, sleeping 7s"
ts=2022-02-01T22:02:15.621947566Z mysql="could not connect to db: dial tcp 10.80.113.69:3306: connect: connection timed out, sleeping 8s"
ts=2022-02-01T22:04:32.837935262Z mysql="could not connect to db: dial tcp 10.80.113.69:3306: connect: connection timed out, sleeping 9s"
ts=2022-02-01T22:06:52.102010455Z mysql="could not connect to db: dial tcp 10.80.113.69:3306: connect: connection timed out, sleeping 10s"
ts=2022-02-01T22:09:11.366299187Z mysql="could not connect to db: dial tcp 10.80.113.69:3306: connect: connection timed out, sleeping 11s"
ts=2022-02-01T22:11:32.677947186Z mysql="could not connect to db: dial tcp 10.80.113.69:3306: connect: connection timed out, sleeping 12s"coz this was ofcourse never a problem when the app was deployed. I can try redeploying it if needed but that wouldn't be ideal
why would an attempt to enroll a host break mysql 🤔
z
zwass
02/01/2022, 10:32 PM
Seems unlikely that it would have caused that...? Never know though
p
pvirani
02/01/2022, 10:34 PMhumm if it was broken from the get to, Fleet wouldn't even have deployed right? I remember not being able to start the app due to failing SQL connection and the app didn't start till I fixed that
anyways trying to upgrade to the latest version now and seeing if that deploys ok somehow or if it complains about connecting with the mysql DB 🤞🏽
z
zwass
02/01/2022, 10:42 PM
If you were able to get past the login screen in the past then you definitely had MySQL working. The server won't start up without connection to MySQL, but it will stay up if it was previously connected and keep trying to connect.
p
pvirani
02/01/2022, 11:52 PMAll upgraded, found the root cause and fixed. TL;DR it wasn't fleet it was Segment's Terraform modules. womp womp!
When I removed the OIDC settings it overwrote my RDS security group (SG) settings. Even though there was a policy defined my TF file to allow 3306 connections between my EKS pool SG and RDS SG it somehow didn't apply them. That broke the App's RDS connection with MySQL and the rest of it as we know it. A good exercise in trying to upgrade the App version though! This was my first upgrade after setting it up 🙂
z
zwass
02/01/2022, 11:54 PM
Oh glad you worked it out! Thank you for sharing 🙂
🎉 1
✍🏽 1
🙏🏽 1
p
pvirani
02/02/2022, 12:00 AMAs always, thanks for patiently replying to my debugging rants and being an awesome collaborator 🙏🏽
🍻 2
2 Views