Hi. I can't seem to get orbit running on some of m...
# fleet
f
Hi. I can't seem to get orbit running on some of my containers (proxmox container os):
Copy code
ERR run orbit failed error="found a new hardware uuid, restarting"
i tried removing orbit and reinstalling it but i can't get past that point. all my containers (that used to work) show as offline fleet 4.53.1 orbit 1.27.0 osquery 5.12.1
i think this stopped working around the time i updated to fleet 4.50.x
this is the last time some of my nodes reported in:
Copy code
2024-06-24T23:51:49+02:00 INF Upgrading fleetd from 1.26.0 to 1.27.0
l
Hi Florian! Let me check with the team. This was a recent change in fleetd. Q: Is it possible that your proxmox containers are not reporting the hardware UUID reliably? (fleetd/osquery https://github.com/osquery/osquery/blob/b46d96957fba2ceb14a61f9dfb3076482350a754/osquery/core/system.cpp#L149 uses that file to load the hardware UUID)
f
Hi Lucas, let me take a quick look
these containers don't seem to have access to that path
Copy code
# cat /sys/class/dmi/id/product_uuid
cat: /sys/class/dmi/id/product_uuid: Permission denied
l
No luck with
sudo cat
?
Oh you are root, nvm
f
yes, i executed as root - no luck
i have one other machine that is showing the same symptoms but is dedicated hardware instead - as root i am able to access that path and it has a uuid value that kind of looks right
the dedicated server has this exact value in
/sys/class/dmi/id/product_uuid
https://github.com/osquery/osquery/blob/b46d96957fba2ceb14a61f9dfb3076482350a754/osquery/core/system.cpp#L93
l
Ah, it's ignored.
Hi @Florian! We've created #20168 to track this issue. In the meantime: • Is there a workaround in proxmox to have it define a UUID? • As a workaround you could downgrade to fleetd 1.26.0 by generating new packages with
--orbit-channel 1.26.0 --desktop-channel 1.26.0
(though you will need to change them back to
stable
again as soon as we release the fix for #20168)
f
Thank you Lucas. Unfortunately there doesn’t seem to be a universal way. Under the hood these containers are incus/lxc containers and apparently the part of sysfs isn’t namespaced https://forum.proxmox.com/threads/how-to-get-an-unique-machine-id-for-lxc.70363/
Personally I think one could derive a system id off of MAC addresses as a fallback. I am sure that has its own downsides