I am looking to grab the vulnerability data (NIST feeds and the NVD DB) and host them on an internal server. I see that the NIST data needs to be the JSON files from /feeds/json/cve/1.1/ for the JSON files, however I am not sure what files should be pulled from https://github.com/fleetdm/nvd/releases. Is Fleet expecting the sqlite.gz, the zip file or the tar.gz? Or is it looking for the decompressed DB itself?

https://fleetdm.com/docs/deploying/configuration#cpe-database-url might be helpful.

It expects a file in the same format as can be found in https://github.com/fleetdm/nvd/releases.

Assuming that means the sqlite.gz file

That is what I wasn’t sure of, as the files that are there are all 3 types i listed, with none specifically the zip file.

there's

fleetctl vulnerability-data-stream

that is meant to download everything needed, then you can configure it to skip the data sync, if you place those files in the databases path, it'll pick them up from there

yeah its not super clear. The zip and tar.gz files are the source code of this repo.

Interesting. I’ll have to play with it - I want to build an automated process and am looking to make sure I understand all of the pieces.

this is meant to fit a CI/CD pipeline, in case you want to run the files through other checks before moving them to a server

My goal is to have the pulling of the DB and the NIST data into an artifactory repo, then have my fleet servers pull from there.

yes, and the CVE data streams, everything that fleet does when left to its own devices for vulnerability processing

Cool. That gives me a much clearer idea of what I need. Thank you!

you'll lose the differential downloading that you get when you let fleet download things, but it's not a ton of data, so I would bet it's fine

Eh, I am not worried about that.

@koo can you please help turn this discussion into Fleet documentation?

As part of the documentation, can you add the specific links that need to be mirrored? I know you outline the NIST ones, however without the url for the NVD DB, it will take some digging to figure out exactly what needs to be hosted/mirrored if fleetctl is not being invoked.

what do you think would be needed from

fleetctl vulnerability-data-stream

to cover that need?

I guess what url is being used to pull down the NVD db. I don’t want to use wireshark and see what is being pulled today, and then have it break when the next version is released. I need to let the artifactory admins know what files/urls need to be synchronized.

so what I'm hearing is: the fleetctl command that downloads the data is not going to be used, you need to know what URLs to pull from. Is this understanding correct?

That is correct.

good to know

I can use fleetctl if I was spinning up a server, however I am trying to integrate with an existing cache/proxy service we have.

yeah, we could look into adding that