Channels
android_tests
apple-silicon
arm-architecture
auditing-warroom
aws
carving
code-review
community-feeds
core
darkbytes
doorman
ebpf
eclecticiq-polylogyx-extension
extensions
file-carving
fim
fleet
fleet-dev
fleetosquery
foundation
fuzzing
general
golang
goquery
infrastructure
jobs
kolide
linen-dev
linux
macos
officehours
osctrl
plugins
process-auditing
querycon
queryhub
random
selfgroup
sql
tls
uptycs
vendor-feeds
website
windows
zeek
zentral
zercurity
Title
r
Rafa
08/03/2022, 9:10 PMHi people! We are currently running fleet with more then 3k hosts and growing but we find something odd. Even though software_inventory is set to true it seems that the vulnerabilities are not being checked. And when I look at the logs I see:
cron=vulnerabilities cron=vulnerabilities databases-path=/tmp/vulndbsk
Kathy Satterlee
08/03/2022, 9:13 PMHey, @Rafa. That database path would be on the server side. Are you seeing any errors about vulnerabilities, or do you just have none showing up in the API/UI?
r
Rafa
08/03/2022, 9:14 PMHello! I am seeing some vulnerabilities on the API/UI but I think should be more (because we run a lot of old systems). And the folder where it should download the db is empty...
And there are only software vulns, nothing about OS vulns
k
Kathy Satterlee
08/03/2022, 9:21 PMThe folder existing is a good sign, but I would definitely expect there to be some data there especially since you are seeing some vulnerabilities. That info has to be coming from somewhere, unless it just got cleared from
/tmpYou'd see some errors in the logs if the cron job was failing. Can you try setting a new database path to see what happens?
r
Rafa
08/03/2022, 9:24 PMDone!
level=error ts=2022-08-03T21:13:00.733144068Z component=crons cron=cleanups cron=leader err="sending statistics" details="error posting to <https://fleetdm.com/api/v1/webhooks/receive-usage-analytics>: 400. "
8/3/2022 6:13:17 PM level=error ts=2022-08-03T21:13:17.696003379Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: Карта: fts5: syntax error near \"\""
8/3/2022 6:13:17 PM level=error ts=2022-08-03T21:13:17.696117432Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: . .: fts5: syntax error near \"\""
8/3/2022 6:13:17 PM level=error ts=2022-08-03T21:13:17.696203752Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: . . .: fts5: syntax error near \"\""
8/3/2022 6:13:17 PM level=error ts=2022-08-03T21:13:17.696288523Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: Неведомый Космос: fts5: syntax error near \"\""
8/3/2022 6:13:17 PM level=error ts=2022-08-03T21:13:17.696368434Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: Рай: fts5: syntax error near \"\""k
Kathy Satterlee
08/03/2022, 9:37 PMWeird question maybe, but how much RAM is Fleet working with?
r
Rafa
08/03/2022, 9:37 PM2 gb
k
Kathy Satterlee
08/03/2022, 9:39 PMThat could be the culprit. Vulnerability scanning needs 4GB and we start seeing weird errors like this when memory starts to get eaten.
r
Rafa
08/03/2022, 9:39 PMhmmm I will change this now. one minute
k
Kathy Satterlee
08/03/2022, 9:39 PM🤞
r
Rafa
08/03/2022, 9:46 PMI noticed something odd:
level=debug ts=2022-08-03T21:44:00.440381147Z component=crons cron=vulnerabilities cron=vulnerabilities msg="Not the leader. Skipping..."
8/3/2022 6:44:00 PMb
Benjamin Edwards
08/03/2022, 9:54 PMThe lock, database row, from the last vuln processing attempt might still be live. It will eventually expire and the next iteration should acquire leader status.
r
Rafa
08/03/2022, 9:55 PMCool! I will wait and let you know. Thanks a lot for the attention and for this amazing software!
b
Benjamin Edwards
08/03/2022, 9:57 PMAlso how are you running fleet? Container?
r
Rafa
08/03/2022, 9:57 PMYeap!
First 3 pods, but I reduced to 1 just to solve this leader problem
b
Benjamin Edwards
08/03/2022, 9:58 PMOk just checking thanks
r
Rafa
08/04/2022, 1:54 PMHello!
How are you?
Looking at the deploy I saw that it downloaded the vulns:
And in the logs I see:
fleet-85f4d45756-cprgm fleet level=debug ts=2022-08-04T13:21:50.549502862Z component=crons cron=vulnerabilities cron=vulnerabilities cpe-processing-3=start
fleet-85f4d45756-cprgm fleet level=debug ts=2022-08-04T13:21:50.54967825Z component=crons cron=vulnerabilities cron=vulnerabilities cpe-processing-1=start
fleet-85f4d45756-cprgm fleet level=debug ts=2022-08-04T13:21:50.549706165Z component=crons cron=vulnerabilities cron=vulnerabilities cpe-processing-0=start
fleet-85f4d45756-cprgm fleet level=debug ts=2022-08-04T13:21:50.549715001Z component=crons cron=vulnerabilities cron=vulnerabilities cpe-processing-2=start
fleet-85f4d45756-cprgm fleet level=debug ts=2022-08-04T13:21:50.678175379Z component=crons cron=vulnerabilities cron=vulnerabilities pushingcpes=done
fleet-85f4d45756-cprgm fleet level=debug ts=2022-08-04T13:21:50.67854018Z component=crons cron=vulnerabilities cron=vulnerabilities cpe-processing-1=done
fleet-85f4d45756-cprgm fleet level=debug ts=2022-08-04T13:21:50.678925991Z component=crons cron=vulnerabilities cron=vulnerabilities cpe-processing-0=done
fleet-85f4d45756-cprgm fleet level=debug ts=2022-08-04T13:21:50.679066333Z component=crons cron=vulnerabilities cron=vulnerabilities cpe-processing-2=done
fleet-85f4d45756-cprgm fleet level=debug ts=2022-08-04T13:21:50.680472866Z component=crons cron=vulnerabilities cron=vulnerabilities cpe-processing-3=done
k
Kathy Satterlee
08/04/2022, 2:56 PMLooking good!
How's the software looking now?
And doing well this morning, how's your day going?
r
Rafa
08/04/2022, 3:06 PMGood! I am fine too 😃
About the software, it still seems to do not have scanned all vulnerabilities. Is there a way to run the scan in my host added to fleet with fleetctl? I already downloaded the vulns with
fleetctl vulnerability-data-stream --dir .The odd part is that I am still getting these errors:
fleet-6ff77cf554-4nfvp fleet level=error ts=2022-08-04T17:04:44.892674563Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: Карта: fts5: syntax error near \"\""
fleet-6ff77cf554-4nfvp fleet level=error ts=2022-08-04T17:04:44.892780759Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: . .: fts5: syntax error near \"\""
fleet-6ff77cf554-4nfvp fleet level=error ts=2022-08-04T17:04:44.892888206Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: . . .: fts5: syntax error near \"\""
fleet-6ff77cf554-4nfvp fleet level=error ts=2022-08-04T17:04:44.893001757Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: Неведомый Космос: fts5: syntax error near \"\""
fleet-6ff77cf554-4nfvp fleet level=error ts=2022-08-04T17:04:44.893094859Z component=crons cron=vulnerabilities cron=vulnerabilities software->cpe="error translating to CPE, skipping..." err="getting cpes for: Рай: fts5: syntax error near \"\""
b
Benjamin Edwards
08/04/2022, 5:27 PMare you running the fleet containers from dockerhub or did you build your own?
r
Rafa
08/04/2022, 6:45 PMFrom dockerhub
m
Michal Nicpon
08/04/2022, 10:01 PMI was able to reproduce the error and created a github issue https://github.com/fleetdm/fleet/issues/7067
Note that this only affects software with non-ascii names.
r
Rafa
08/04/2022, 10:37 PMThanks! But @Michal Nicpon would this break the vulnerabilities scan as a hole or only for that software?
m
Michal Nicpon
08/04/2022, 10:38 PMOnly for that software. We should probably change this to a warning
r
Rafa
08/04/2022, 10:39 PMGreat! Thanks!
Hi!
How are you? Sorry for the delay but I was trying to find evidence of the problem.
So
This is my host
0 vulnerable software
But after running a scan in my machine:
And looking inside the folder with the downloaded database by fleet:
And
CVE-2022-1652fleet_oval_ubuntu_2004-2022_08_05.jsonSo this vulnerability should be in the board, right?
k
Kathy Satterlee
08/05/2022, 4:26 PMI believe that's expected behavior since it's an OS vulnerability rather than installed software, but I'm going to double check just to make sure.
And verified. Fleet doesn't scan for OS vulnerabilities and is only looking at additional software at this point.
You could set up a policy to check if the OS is up to date. It's generally a good way to make sure things are as secure as possible :)
r
Rafa
08/05/2022, 4:40 PMCool! doing that now! thanks a lot!