Title
#fleet
r

Rafa

08/03/2022, 9:10 PM
Hi people! We are currently running fleet with more then 3k hosts and growing but we find something odd. Even though software_inventory is set to true it seems that the vulnerabilities are not being checked. And when I look at the logs I see:
cron=vulnerabilities cron=vulnerabilities databases-path=/tmp/vulndbs
but I manually entered in this directory in each node and it is empty. This folder has read and write permissions. Is there another env that I have to set? Version 4.18
Kathy Satterlee

Kathy Satterlee

08/03/2022, 9:13 PM
Hey, @Rafa. That database path would be on the server side. Are you seeing any errors about vulnerabilities, or do you just have none showing up in the API/UI?
r

Rafa

08/03/2022, 9:14 PM
Hello! I am seeing some vulnerabilities on the API/UI but I think should be more (because we run a lot of old systems). And the folder where it should download the db is empty...
9:15 PM
And there are only software vulns, nothing about OS vulns
Kathy Satterlee

Kathy Satterlee

08/03/2022, 9:21 PM
The folder existing is a good sign, but I would definitely expect there to be some data there especially since you are seeing some vulnerabilities. That info has to be coming from somewhere, unless it just got cleared from
/tmp
.
9:21 PM
You'd see some errors in the logs if the cron job was failing. Can you try setting a new database path to see what happens?
r

Rafa

08/03/2022, 9:24 PM
Done!
9:24 PM
level=error ts=2022-08-03T21:13:00.733144068Z component=crons cron=cleanups cron=leader err="sending statistics" details="error posting to <https://fleetdm.com/api/v1/webhooks/receive-usage-analytics>: 400. "
8/3/2022 6:13:17 PM level=error ts=2022-08-03T21:13:17.696003379Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: Карта: fts5: syntax error near \"\""
8/3/2022 6:13:17 PM level=error ts=2022-08-03T21:13:17.696117432Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: . .: fts5: syntax error near \"\""
8/3/2022 6:13:17 PM level=error ts=2022-08-03T21:13:17.696203752Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: . . .: fts5: syntax error near \"\""
8/3/2022 6:13:17 PM level=error ts=2022-08-03T21:13:17.696288523Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: Неведомый Космос: fts5: syntax error near \"\""
8/3/2022 6:13:17 PM level=error ts=2022-08-03T21:13:17.696368434Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: Рай: fts5: syntax error near \"\""
Kathy Satterlee

Kathy Satterlee

08/03/2022, 9:37 PM
Weird question maybe, but how much RAM is Fleet working with?
r

Rafa

08/03/2022, 9:37 PM
2 gb
Kathy Satterlee

Kathy Satterlee

08/03/2022, 9:39 PM
That could be the culprit. Vulnerability scanning needs 4GB and we start seeing weird errors like this when memory starts to get eaten.
r

Rafa

08/03/2022, 9:39 PM
hmmm I will change this now. one minute
Kathy Satterlee

Kathy Satterlee

08/03/2022, 9:39 PM
🤞
r

Rafa

08/03/2022, 9:46 PM
I noticed something odd:
level=debug ts=2022-08-03T21:44:00.440381147Z component=crons cron=vulnerabilities cron=vulnerabilities msg="Not the leader. Skipping..."
8/3/2022 6:44:00 PM
I am running with one node only. It should be the leader right?
Benjamin Edwards

Benjamin Edwards

08/03/2022, 9:54 PM
The lock, database row, from the last vuln processing attempt might still be live. It will eventually expire and the next iteration should acquire leader status.
r

Rafa

08/03/2022, 9:55 PM
Cool! I will wait and let you know. Thanks a lot for the attention and for this amazing software!
Benjamin Edwards

Benjamin Edwards

08/03/2022, 9:57 PM
Also how are you running fleet? Container?
r

Rafa

08/03/2022, 9:57 PM
Yeap!
9:57 PM
First 3 pods, but I reduced to 1 just to solve this leader problem
Benjamin Edwards

Benjamin Edwards

08/03/2022, 9:58 PM
Ok just checking thanks
r

Rafa

08/04/2022, 1:54 PM
Hello!
1:54 PM
How are you?
1:55 PM
Looking at the deploy I saw that it downloaded the vulns:
1:55 PM
And in the logs I see: fleet-85f4d45756-cprgm fleet level=debug ts=2022-08-04T13:21:50.549502862Z component=crons cron=vulnerabilities cron=vulnerabilities cpe-processing-3=start fleet-85f4d45756-cprgm fleet level=debug ts=2022-08-04T13:21:50.54967825Z component=crons cron=vulnerabilities cron=vulnerabilities cpe-processing-1=start fleet-85f4d45756-cprgm fleet level=debug ts=2022-08-04T13:21:50.549706165Z component=crons cron=vulnerabilities cron=vulnerabilities cpe-processing-0=start fleet-85f4d45756-cprgm fleet level=debug ts=2022-08-04T13:21:50.549715001Z component=crons cron=vulnerabilities cron=vulnerabilities cpe-processing-2=start fleet-85f4d45756-cprgm fleet level=debug ts=2022-08-04T13:21:50.678175379Z component=crons cron=vulnerabilities cron=vulnerabilities pushingcpes=done fleet-85f4d45756-cprgm fleet level=debug ts=2022-08-04T13:21:50.67854018Z component=crons cron=vulnerabilities cron=vulnerabilities cpe-processing-1=done fleet-85f4d45756-cprgm fleet level=debug ts=2022-08-04T13:21:50.678925991Z component=crons cron=vulnerabilities cron=vulnerabilities cpe-processing-0=done fleet-85f4d45756-cprgm fleet level=debug ts=2022-08-04T13:21:50.679066333Z component=crons cron=vulnerabilities cron=vulnerabilities cpe-processing-2=done fleet-85f4d45756-cprgm fleet level=debug ts=2022-08-04T13:21:50.680472866Z component=crons cron=vulnerabilities cron=vulnerabilities cpe-processing-3=done
Kathy Satterlee

Kathy Satterlee

08/04/2022, 2:56 PM
Looking good!
2:56 PM
How's the software looking now?
2:57 PM
And doing well this morning, how's your day going?
r

Rafa

08/04/2022, 3:06 PM
Good! I am fine too 😃
3:13 PM
About the software, it still seems to do not have scanned all vulnerabilities. Is there a way to run the scan in my host added to fleet with fleetctl? I already downloaded the vulns with
fleetctl vulnerability-data-stream --dir .
5:09 PM
The odd part is that I am still getting these errors: fleet-6ff77cf554-4nfvp fleet level=error ts=2022-08-04T17:04:44.892674563Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: Карта: fts5: syntax error near """ fleet-6ff77cf554-4nfvp fleet level=error ts=2022-08-04T17:04:44.892780759Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: . .: fts5: syntax error near """ fleet-6ff77cf554-4nfvp fleet level=error ts=2022-08-04T17:04:44.892888206Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: . . .: fts5: syntax error near """ fleet-6ff77cf554-4nfvp fleet level=error ts=2022-08-04T17:04:44.893001757Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: Неведомый Космос: fts5: syntax error near """ fleet-6ff77cf554-4nfvp fleet level=error ts=2022-08-04T17:04:44.893094859Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: Рай: fts5: syntax error near """
Benjamin Edwards

Benjamin Edwards

08/04/2022, 5:27 PM
are you running the fleet containers from dockerhub or did you build your own?
r

Rafa

08/04/2022, 6:45 PM
From dockerhub
Michal Nicpon

Michal Nicpon

08/04/2022, 10:01 PM
I was able to reproduce the error and created a github issue https://github.com/fleetdm/fleet/issues/7067 Note that this only affects software with non-ascii names.
r

Rafa

08/04/2022, 10:37 PM
Thanks! But @Michal Nicpon would this break the vulnerabilities scan as a hole or only for that software?
Michal Nicpon

Michal Nicpon

08/04/2022, 10:38 PM
Only for that software. We should probably change this to a warning
r

Rafa

08/04/2022, 10:39 PM
Great! Thanks!
4:16 PM
Hi!
4:18 PM
How are you? Sorry for the delay but I was trying to find evidence of the problem.
4:18 PM
So
4:18 PM
This is my host
4:19 PM
0 vulnerable software
4:19 PM
But after running a scan in my machine:
4:20 PM
And looking inside the folder with the downloaded database by fleet:
4:22 PM
And
CVE-2022-1652
is in the
fleet_oval_ubuntu_2004-2022_08_05.json
.
4:22 PM
So this vulnerability should be in the board, right?
Kathy Satterlee

Kathy Satterlee

08/05/2022, 4:26 PM
I believe that's expected behavior since it's an OS vulnerability rather than installed software, but I'm going to double check just to make sure.
4:34 PM
And verified. Fleet doesn't scan for OS vulnerabilities and is only looking at additional software at this point.
4:36 PM
You could set up a policy to check if the OS is up to date. It's generally a good way to make sure things are as secure as possible 😃
r

Rafa

08/05/2022, 4:40 PM
Cool! doing that now! thanks a lot!