Hey everyone, I have been experiencing recurring peaks on my CPU that is hosted on AWS. I'm unsure if I am overlooking something, but these peaks occur daily. I would like to know if there are any solutions other than increasing capacity, as this incurs high costs. Currently, I am using 95 hosts on my fleet. I appreciate any insights or suggestions you can provide. Thank you in advance."

Based on the timing, that looks like you're seeing spikes for vulnerability processing. You can try changing the periodicity to confirm. How are things specced out in AWS?

We run multiple instances and see this hourly on one of them as well. It hasn't caused us any issues, but we also have 2 other instances handling requests during that time.

Thanks you for your input! After reviewing and making adjustments to the periodicity, I'm currently trying to understand the underlying factors causing these peaks. Furthermore, I would like to determine whether these peaks should be a cause for concern in terms of the stability of the Fleet or if it has effects on our hosts.

That sounds like exactly what I would expect. Vuln processing is the most resource intensive thing Fleet does, and one instance handles that processing at a time. For up to 1000 hosts, that instance should have around 512 CPU units and 4GB of ram available. There are a couple of options that would help from a cost-saving perspective if you have multiple Fleet instances running and are seeing performance issues (detailed writeup here), but are generally recommended for larger deployments: • You can set Fleet up to always use one particular instance for vuln processing and give that one instance a little more processing power and memory. You'd have a slightly higher cost for that one instance, but could keep others trim. ◦ To accomplish this, you'd set

vulnerabilities.current_instance_checks

to

no

on the instances that are not processing vuln data and

yes

on the (beefier) instance that is. • You can run vuln processing on demand using an instance that only runs temporarily. ◦ First, you'd turn the schedule off on all Fleet instances with

vulnerabilities.disable_schedule

◦ Then, You can set up an instance that spins up periodically and runs vuln processing. https://fleetdm.com/docs/deploying/configuration#current-instance-checks https://fleetdm.com/docs/deploying/configuration#disable-schedule

Thanks for your insights on this! I'm just wondering do these peaks degrade the service of Fleet o affect in any way our hosts? not sure if I should be concern. These peaks are currently taking 30 minutes every each hour, that's Or how can I monitor this on Fleet. @Benjamin Edwards @Kathy Satterlee

You might see sluggishness in the UI and potentially some slow performance in live queries. Other than that, osquery is pretty resilient and data should flow through eventually. As long as you aren’t seeing crashes or a large uptick in Fleet errors, you could let it ride. That being said, the process will take a lot less time with more resources dedicated.