#998 Log sink categories: route different data classes to different sinks Pull request opened by javuto Log sink categories: route different data classes to different sinks Problem Every configured log sink received every type of osquery data — status logs, scheduled-query results, on-demand query results, and carve lifecycle events. Operators could not say "send status logs to Splunk but keep results in the DB" or "route carve metadata to Kafka only." Adding a second sink meant both got the full firehose. Change Added per-sink category routing so operators choose which data classes each sink receives. Five categories are supported: | Category | Data | | ---------- | ----------------------------------------------------- | | status | osquery daemon status logs | | result | Scheduled-query result logs | | query | On-demand distributed-query results | | carve.meta | Carve lifecycle metadata (scheduled, init, completed) | | carve.data | Carve block metadata (block ID, session ID, size) | Backend: •

pkg/logsinks/categories.go

(new): category constants,

MatchesCategory

,

ValidateCategories

,

NormalizeCategories

(collapses explicit "all five" to empty), `DecodeCategories`/`encodeCategories` helpers • `pkg/logsinks/logsinks.go`: Added

Categories

text column to

LogSink

model (JSON-encoded array; empty = all). `Create`/`Update` accept and validate categories.

CloneEnvironment

copies them.

BuildExporters

passes them to

ExporterEntry

• `pkg/logging/exporter.go`: Added

Categories

to

ExporterEntry

.

MultiExporter

stores a parallel categories slice and skips exporters whose categories don't match the incoming

logType

in

Export

(nil/empty = all, backwards-compatible) • `cmd/tls/handlers/carves.go`: `ProcessCarveWrite`/`ProcessCarveInit`/`ProcessCarveBlock` now emit

carve.meta

and

carve.data

events via

LoggerTLS.LogWithEnv

so external sinks can track carve lifecycle. Raw block data is NOT sent through the fan-out — it stays in the DB/S3 via

pkg/carves

• `pkg/types/types.go`: Added

Categories

to

LogSinkCreateRequest

,

LogSinkUpdateRequest

,

LogSinkTypeSpec

• `cmd/api/handlers/log_sinks.go`: Added

Categories

to `logSinkDTO`; create/update handlers pass categories through; types endpoint advertises

AllCategories

Frontend: • `frontend/src/api/log-sinks.ts`: Added

categories

to

LogSink

, create/update request types, and

LogSinkTypeSpec

• `frontend/src/features/log-sinks/LogSinksPage.tsx`: Categories checkbox section in the sink editor (All checkbox + individual category toggles). New Destination column in the table showing color-coded category badges per sink — "all" badge when unscoped, individual badges when categories are explicitly set Migration:

Categories

is a new nullable text column added by

AutoMigrate

. Existing rows get

""

= all categories — no behavior change. No YAML changes; seeded sinks get empty categories. Rollback by clearing the column. Validation •

go test ./pkg/logsinks/...

—

TestMatchesCategory*

,

TestValidateCategories

,

TestNormalizeCategories

,

TestEncodeDecodeCategories

,

TestCreateWithCategories

,

TestUpdateWithCategories

,

TestCreateRejectsInvalidCategory

,

TestClonePreservesCategories

•

go test ./pkg/logging/...

—

TestMultiExporterCategoryFiltering

verifies status-only, all, and query+result sinks receive the correct logTypes •

go test ./pkg/... ./cmd/...

— full Go test suite passes (no regressions) •

npm run check

— TypeScript typecheck passes •

npm run test

— 292 frontend tests pass (no regressions) jmpsec/osctrl