:thread: The second issue is with the `CleanupExpi...
# fleet
a
🧵 The second issue is with the
CleanupExpiredHosts
cron job (the
expired_hosts
job within
cleanups_then_aggregation
). Our setup: • ~9M hosts in Fleet, only ~100k actively checking in • Host expiry is enabled, but stale hosts aren't being removed Error:
get host details for expired hosts: select hosts by id: Error 1390 (HY000): Prepared statement contains too many placeholders
Root cause: Three code paths pass the full list of expired host IDs into unbounded
IN (?)
clauses, which blows past MySQL's ~65,535 placeholder limit: •
ListHostsLiteByIDs
•
DELETE FROM host_seen_times
•
CleanupHostActivities
Related issues: • This looks related to github.com/fleetdm/fleet/issues/31157 which was reported by another customer and closed. The symptoms are the same — hosts not being removed after hitting the expiry window — but at our scale the prepared statement limit is the specific failure mode. • Also wanted to ask about github.com/fleetdm/fleet/issues/45947 — is there a timeline or release planned for a fix? Our fix: I've wrapped the three unbounded
IN
clauses with
BatchProcessSimple
. github.com/adityaoza1901/…/cleanup-expired-hosts-batching
s
Hello, github.com/fleetdm/fleet/issues/45947 should be part of 4.89.0. You can see this on the right-hand side of the page for the issue.
@Aditya Oza Have you tested that branch with the fix and confirmed that it works?
u
If you have, please open a pull request detailing what you ran into and how you resolved it. Our team would love to review it and merge it if it fixes a bug.
@Aditya Oza Just wanted to check in on this again. I figured you'd like credit for the work for this, so please submit a PR when you can!
a
@Steven Palmesano I was on a vacation. Just got back. Let me get back to you on this later this week. For now, I have another issue I'd like to report. Will do so in a new thread.
u
Thanks. I'll open the bug report for this today. When you submit the PR, the template will have a spot for you to enter the issue number that it resolves, which will connect it to the bug report.
s