I am using 3.11 fleet deployed in AWS and when I updated the task parameter and updated the task, the new task gave me me below error and aborted.
ts=2021-07-29T13:02:43.909676523Z mysql="could not connect to db: Error 1040: Too many connections, sleeping 13s (for a number of time with increasing seconds)
Error initializing datastore: Error 1040: Too many connections
After few failed retries, I opted for rollback and those tasks as well emit the exact messages. Any idea what could lead to this? The fleet currently has ~300 online hosts, ~2900 new hosts and ~150K offline hosts(mostly residues of a scale test script). Any idea how I could get this working back?
Also an unrelated question, does the fleet supports bulk "delete hosts" in any way?
07/29/2021, 2:06 PM
thats an error returned by MySQL. Might indicate that there is another host running fleet server that is already using all the connections to MySQL. It might also be a good idea to check the available disk space on the host running MySQL. Running out of space would block INSERTS, causing connection count to grow as well.
07/29/2021, 7:25 PM
Hola, you can config
to expire old offline hosts in either
or the UI. The UI to delete multiple hosts is being built as we speak!