Title
#general
d

danielpops

04/26/2018, 7:55 AM
I'm not sure if this an issue that belongs on github or more of a question, so I'm starting here. I'm seeing this issue where over time, osquery will start to utilize a lot of CPU. We have observed that when this happens, it is typically after osquery has been running for several months. We also observe that when this happens, there are 50k+ sst files under /var/osquery/osquery.db! Couple of questions:- Is it normal to have so many files under there? - Is it safe to purge old files based on mtime? or atime? - Has anyone else observed so many files and/or a high spike in CPU after running for several months? osquery info:
osquery_version = 2.8.0
     os_version = 14.04, Trusty Tahr
       os_build =
    os_platform = ubuntu
some bash showing the files and that they're all accessed within the last few days (this is on a host that is not yet observing the high CPU, but sure has a lot of sst files): https://pastebin.com/pvaxwdPm
clong

clong

04/26/2018, 2:53 PM
2.8 is pretty old! I think this was a known issue that got resolved in later versions
d

danielpops

04/26/2018, 5:22 PM
@clong ++ thanks! Do you have a pointer to an issue or PR that describes the issue/fix?
5:24 PM
I did see some issues about "too many open files", and noticed that the max_open_files parameter was added in version 2.8.0 https://github.com/facebook/osquery/commit/16b40138feb0c4d2f8509bf80227f5fe4de77cd5
clong

clong

04/26/2018, 10:27 PM
i dont unfortunately — just anecdotal experience. im not 100% that will resolve the issue, but i seem to recall having similar issues in earlier versions that got cleared up with newer releases