#1076 Add opt-in vulnerability monitoring (backend) Pull request opened by javuto Add opt-in vulnerability monitoring (backend) First part of fleet vulnerability monitoring: it reports which nodes run packages affected by known CVEs. This PR is the backend only. The UI, alert/posture integration and Windows/macOS app matching follow in separate PRs. Problem osctrl already collects package inventory for posture, but nothing correlates it with vulnerability data. The posture README states as much: there is "no vulnerability-feed correlation." Operators have to export inventory to a separate scanner to answer "which hosts are exposed to CVE-X?". What changed Everything sits behind a new

--vuln-enabled

flag (

SERVICE_VULN_ENABLED

), off by default, like the console and file explorer. With it off: no routes, no tables, no downloads, and

osctrl:vuln:*

query results are discarded. Inventory (osctrl-tls) • New

pkg/vulns

stores each node's installed packages from daily

osctrl:vuln:

scheduled queries. Snapshot mode only: a differential result is never treated as a full inventory. • Each category's snapshot fully replaces the previous one, keeping one row per package in

node_software

. • A snapshot over 50,000 rows is rejected, not truncated, because a truncated list would read as "not vulnerable." • Each node's OS (

os_version

) is recorded in

vuln_node_state

. It decides which advisory set a node's distro packages are matched against. • Inventory profiles for Linux, macOS and Windows use the existing posture profile shape and are served at

GET /api/v1/vulnerabilities/profiles

. They are not applied automatically. Advisory data and matching (osctrl-api) • A background worker downloads OSV advisories and the CISA KEV catalog. It fetches only the OSV ecosystems the fleet actually reports, unless

--vuln-ecosystems

sets an explicit list. • First and later syncs: • The first sync of an ecosystem imports its

all.zip

. • Later syncs fetch only the records listed as changed in

modified_id.csv

. • Withdrawn advisories are removed. • Version comparison uses

<http://github.com/google/osv-scalibr/semantic|github.com/google/osv-scalibr/semantic>

, OSV's own dpkg/rpm/PEP 440/semver comparators. It's stdlib-only, so it pulls in no transitive dependencies. • Severity is scored locally from the CVSS v3/v4 vectors in each record (`github.com/pandatix/go-cvss`). Records without a vector, common for Debian and Ubuntu, are

unknown

, not low. • Matching rules: • Debian and Ubuntu are matched by source package, so

libssl3

matches

openssl

advisories. • RPM versions include the epoch. • PyPI names are PEP 503-normalized. • Not assessed, never "clean": an unrecognized distro, a version with no digit, or an ecosystem with no feed loaded. Each node and environment reports the count. • Findings lifecycle: • A finding that no longer matches gets `resolved_at`; it isn't deleted. • A finding that matches again is reopened with its original

first_seen

. • Resolved findings are purged after

--vuln-retention-days

. • Coordination is in SQL. A lease row keeps one API replica working at a time. "Needs re-matching" is

matched_at < inventory_at

, and "sync now" is a column. The feature needs no Redis. • Housekeeping: rows of deleted nodes are swept hourly, because node deletion (API and CLI) doesn't know about this feature. REST API (registered only when enabled) | Route | Access | | --------------------------------------------------- | --------------------------------------------------------------------- | | GET /api/v1/vulnerabilities/{env}/findings | environment permission (same check as node/posture reads) | | GET /api/v1/vulnerabilities/{env}/summary | environment permission | | GET /api/v1/vulnerabilities/{env}/advisories/{id} | environment permission; lists affected nodes in that environment only | | GET /api/v1/nodes/{env}/node/{uuid}/vulnerabilities | environment permission; the node must belong to {env} | | GET /api/v1/vulnerabilities/feeds | super admin | | POST /api/v1/vulnerabilities/feeds/sync | super admin, CSRF, audited; queues a sync and never downloads inline | | GET /api/v1/vulnerabilities/profiles | authenticated | •

GET /api/v1/features

gains

vulnerabilities

. • Responses carry

loaded

(no OSV feed synced yet) and

stale

(no successful sync for 3× the interval), so an empty result isn't mistaken for "no vulnerabilities." New flags (sync settings are osctrl-api only): •

--vuln-osv-url

•

--vuln-kev-url

•

--vuln-ecosystems

•

--vuln-sync-hours

(default 6) •

--vuln-retention-days

(default 90) •

--vuln-max-download-mb

(default 2048) Security and operational impact • Inventory never leaves the deployment. The only outbound traffic is feed downloads, to URLs the operator sets (`http`/`https` only). Air-gapped sites point both URLs at an internal mirror. • Downloads are size-capped, including responses without

Content-Length

. Archives are parsed in memory and never extracted, so

../

entry names are inert. • A failed sync keeps existing data and the cursor where it was, and retries after 15 minutes. A KEV feed with no entries is rejected instead of clearing every flag. • Advisory text is external input. It's returned as data; the coming UI must render it as plain text. • Schema: nine new tables, created only when the flag is on. No existing table changes. • Rollout order between API and TLS doesn't matter. An ecosystem syncs within ~15 minutes of the first inventory that needs it. Older processes ignore the new query prefix and tables. • Log sinks: inventory results also flow to configured log sinks, like any scheduled query. • Dependencies:

go.mod

adds

osv-scalibr

and

go-cvss

(plus two indirect entries from

go mod tidy

).

go.sum

also loses 48 stale lines from the same tidy. Known limitations • No UI yet. The frontend, alert rule type, posture score rule and health counters come in follow-up PRs. • Not assessed: Alpine (osquery has no

apk_packages

table), CentOS and SUSE, Ubuntu Pro/FIPS and RHEL EUS streams, and the Windows and macOS OSes and apps. NVD/CPE matching for apps is a later PR. • Re-matching is fleet-wide. Any advisory change re-matches every node: one indexed lookup per node and category. A

ponytail:

comment in

worker.go

marks the targeted alternative. Validation •

go vet ./...

,

go build ./...

, gofmt clean. • `go test ./... -count=1`: *all 55 packages pass*;

go test ./pkg/vulns -race

passes. •

make openapi

regenerated, and

make openapi-check

exits 0 (

osctrl-api.yaml

+423 lines). •

golangci-lint

on the changed packages reports one gocritic

exitAfterDefer

warning in

cmd/api/main.go

. It already exists on

develop

(at the OIDC

log.Fatal

); this change only moves where it's first reported. • Mutation-checked (each test fails when its guard is removed): • source-package matching; • the per-environment permission check, which fails the findings, advisory and node tests; • inventory query columns against the bundled osquery 5.23.1 schema. • Version tests cover real-world formats: • Debian epoch and tilde; • RPM epoch, release and module streams; • PEP 440 and semver pre-releases; • `last_affected`; • re-introduced ranges. The test cases were checked against the library during planning. Record fixtures are modeled on live OSV Red Hat and Ubuntu records. • Not run: a manual smoke test against the local Docker stack (needs Redis), and an upgrade on Postgres/MySQL; the tests use SQLite. jmpsec/osctrl