My upgrade to 4.3.0 hasn’t gone well, it appears t...
# fleet
r
My upgrade to 4.3.0 hasn’t gone well, it appears to be stuck on a migration, MySQL process list shows it “sending data” for the following query:
DELETE FROM software WHERE NOT EXISTS (select 1 from host_software hs where hs.software_id=software.id)
t
have you checked that all the fleet instances have stopped running?
r
they were still running, but I haven’t been stopping them for previous migrations, is that something I should start doing? it does make rolling deployments more difficult.
In the end I just truncated all the software tables, there were ~ four of them. Given they are rebuilt by the system itself anyway as hosts “check in” I didn’t think they were too important, and once that was done, the migrations were able to complete and Fleet started working again.
It was a bit of a scary release though, all the others have been super smooth so I had gotten used to just running them.
t
I haven’t been stopping them for previous migrations, is that something I should start doing?
yes, always. Otherwise locking is very likely to happen in an active system
It was a bit of a scary release though, all the others have been super smooth so I had gotten used to just running them.
I hear you. I'm sorry it was that way. Migrations were never intended to run while fleet is running, though, so the fact that you were able to so far was a lucky shot.
r
hahaha fair enough 😂
I’ll make an enhancement to our automation to stop the service when migrations are needed. Is there some mechanism I can use to detect when a migration should run, or just set it up to trigger if the package version ever changes?
t
the simplest is to trigger with every update. It's a quick noop if there's nothing
r
ok no worries then 🙂
thanks for your help, appreciated!
👍 1