https://github.com/osquery/osquery logo
Title
s

Stijn Pieters

08/31/2021, 2:29 PM
Hey, I'm trying to get a list of vulnerabilities for the software installed on my hosts. For Windows hosts I get a proper software list on the "Host" page, but for Linux hosts I'm getting a user list instead. I also haven't seen any vulnerabilities in the software list, but maybe I'm just not that vulnerable :^) I set the required environment variable and configured a path to write the vulnerability DB to. Did I miss something?
t

Tomas Touceda

08/31/2021, 2:34 PM
hi Stijn, few questions: • what version of fleet are you running? • is the databases path created and writable in the fleet instance? • do you have any files in the databases path?
s

Stijn Pieters

08/31/2021, 2:43 PM
• I'm running Fleet 4.2.3 • For my tests I'm running fleet as root (not yet in a service) • The database path is filled with .json.gz and .meta files named nvdcve-1.1...
t

Tomas Touceda

08/31/2021, 2:44 PM
ok, that's good. Do you have logs of the fleet server that you can share to see if any issues appear there?
s

Stijn Pieters

08/31/2021, 3:01 PM
The only error I can find seems to be in my mysql db. I'm getting a "Got an error reading communication packets"
I tried running with cmdline parameters instead of the yaml config file, which seems to have solved the problem. I think the issue might have been with the password parameter in my yaml file
👀 1
A new "issue" is now that in my startup logs, it says "vulnerability scanning": "not configured"
r

Rachel Perkins

08/31/2021, 6:02 PM
Hmm, with that last error, it only renders if
if appConfig.VulnerabilitySettings.DatabasesPath == "" &&
config.Vulnerabilities.DatabasesPath == "" {
<http://level.Info|level.Info>(logger).Log("vulnerability scanning", "not configured")
return
}
t

Tomas Touceda

08/31/2021, 6:04 PM
yeah, the database path is not being set. Could you share how you're running it exactly?
s

Stijn Pieters

09/01/2021, 8:33 AM
Thanks for those last 2 comments. It seems like more settings are not propagating well through my yaml config. My database path was in there, the path was populated with files but have never been used. I started by running "fleet serve --config fleet.yaml", which gave me mysql and vulnerability problems. Now I'm trying to run using cmdline parameters only. The STDOUT log "vulnerability scanning: not configured" didn't appear. Updating this comment if it works
I still am not seeing any vulnerability information. Adding my running options for reference.
fleet serve --mysql_address=127.0.0.1:3306 --mysql_database=fleet --mysql_username=fleet --mysql_password=xxxxxxxxx --redis_address=127.0.0.1:6379 --server_cert=/path/here --server_key=/path/here --logging_json --vulnerabilities_periodicity 10m --vulnerabilities_current_instance_checks=yes --vulnerabilities_databases_path=/var/fleet/vulndb/
Update: Through the API I see that practically no CPEs have been generated
t

Tomas Touceda

09/02/2021, 1:16 PM
through what API are you seeing that?
s

Stijn Pieters

09/02/2021, 2:20 PM
https://myfleetinstan.ce:8080/api/v1/fleet/hosts/3 Following this API call I see stuff like in the image. Most of them return "generated_cpe": "". An extremely small amount actually match against the CPE database now though
t

Tomas Touceda

09/02/2021, 2:28 PM
could you share debug logs for fleet serve? it's hard to debug without seeing them
s

Stijn Pieters

09/02/2021, 3:17 PM
I'm going to be honest, I didn't even have debugging on (because I didn't read the docs properly) Here's an outtake since the restart after enabling debugging. Blurred out stuff are just IPs. At this point I think it's working as intended, just not matching a lot because of weird software name formats retrieved by osquery
t

Tomas Touceda

09/02/2021, 3:40 PM
all looks good there, could you tell me what's the output of the following sql queries:
select count(*) software;
select * from software_cpe;
?
s

Stijn Pieters

09/02/2021, 3:53 PM
Seems to be the list of actual matched vulnerabilities I saw in the GUI
In hindsight I think you probably meant "from software;"
t

Tomas Touceda

09/02/2021, 3:56 PM
right, so that's from all the software you got the ones that fleet has detected as potentially vulnerable. It all looks good
it might be the case that there's some cases we are missing
in order to detect those, we would have to check one by one by hand
s

Stijn Pieters

09/02/2021, 3:57 PM
yeah there really isn't an easy way to properly match software names against the CPEs I think. Even installing software with vulns didnt get detected because they had a tiny irregularity in their names
t

Tomas Touceda

09/02/2021, 3:58 PM
well, if you have examples of where we can do better, please do create a ticket for it
the translation, as you mention, is the trickiest, but we want to improve it
s

Stijn Pieters

09/02/2021, 4:04 PM
We were thinking about parsing software names by removing all kinds of irregularities like "(x64)", "(Build xxxx)", "EN-US" and all that good stuff. Still very static, but better than nothing. I'll create a ticket when I have something more concrete
t

Tomas Touceda

09/02/2021, 4:26 PM
great, thank you!