I saw in a github issue this might be caused by the network_interfaces inserts.. is that something i can disable in fleet?

Not yet, but very soon we have some major performance improvements that will be released.

Good to hear improvements are on the way! Is there a suggested temporary fix or workaround for the current situation? I’m thinking about creating python script to kill hanging sql queries for now as a temporary solution

Increase

--detail_update_interval

will reduce the load on the network_interfaces queries.

Hmm looks like i still have hanging queries even after this change, mainly: select distinct from distributed_query_campaigns. Causing updates to hang

I don't think that's likely to help, but could be worth a try. What is the

distributed_interval

you are configuring for your hosts?

I have gradually increased it, from 600, now to 1800 (on fleet options side)

Labels do use distributed, so you can't do that and still use labels. But if you would be okay with disabling live queries, maybe just set an distributed_interval of 24 hours so you get daily label updates?

Ok i will try that, Wouldnt the problem reappear after 24 hours when they will check for distributed queries again?

I see it as a throughput issue... If hosts are checking in for an expensive query every 24 hours instead of every 30 minutes you are running ~50x fewer expensive queries. You're less likely to overload the mysql server that way.

Thanks, in meantime i have also enabled slow query logging, which will at least show the full query instead of prepared statement. Will see how it goes

Likely. All of these issues are interconnected and addressed with the new changes.