#1006 Alerting system Stage 2: ingest hook — async dispatch pipeline in `osctrl-tls` Pull request opened by javuto Alerting system Stage 2: ingest hook — async dispatch pipeline in osctrl-tls Problem Stage 1 shipped the rule-matching engine (

pkg/alerts

) and the

alertsEnabled

service flag, but nothing evaluated rules: the ingest path never consulted the snapshot and no pipeline existed between a match and a recorded alert. This PR wires the matcher into osctrl-tls's log ingest with a performance-first, non-blocking design. Change Dispatch pipeline — new

pkg/alerts/worker.go

• Buffered queue (8192 slots) drained by 2 worker goroutines;

Enqueue

uses select-default so a full queue drops with a counter instead of ever blocking the ingest goroutine (same contract as the existing activity writer) • Per-hit pipeline: cooldown claim (Redis

SetNX

) → sink dispatch → history write; a failed dispatch releases the claim so the next hit retries within the window instead of being swallowed by the cooldown •

WorkerMetrics

as plain atomics (matched / dropped / dispatched / collapsed / failed) — no locks on the hot path •

NewSyncWorker

test mode dispatches inline for sleep-free assertions Ingest adapter — new

pkg/alerts/ingest.go

•

IngestMatcher

implements the hook interface over snapshot + worker; all methods nil-receiver-safe, so feature-off is a single nil comparison per hook site Hooks —

pkg/logging

• `logging.go`:

AlertMatcher

interface (result/status/query-log methods) +

LoggerTLS.Alerts

field • `process.go`: hooked

ProcessLogs

— result batches are handed over already decoded (zero extra parse); status batches get a dedicated full-schema decode inside the hook only when a matcher exists;

ProcessLogQueryResult

calls the matcher once per answered query with node env context • Nil matcher leaves

ProcessLogs

behavior byte-for-byte unchanged Env scoping —

pkg/alerts/matcher.go

•

Match*

now take `envID`/`environment`: global rules (EnvironmentID 0) apply everywhere, scoped rules only to their environment; hits carry the env name for history rows and rendered payloads Wiring —

cmd/tls/main.go

• Constructs manager + snapshot + Redis state + worker when

--alerts-enabled

is set; all stay nil otherwise (hooks no-op, no cost) • Matcher attached to `loggerTLS.Alerts`; 5-minute

watchAlertRules

ticker reloads the rule snapshot so DB/API edits propagate without restart (Stage 4 will add a service-command trigger for instant refresh) •

stopAlerts

drains the dispatch queue on shutdown/restart Metrics — new

pkg/alerts/metrics.go

• Prometheus custom collector exporting `osctrl_alerts_{matched,dropped,dispatched,collapsed,failed,queue_depth}`; a collector reads the atomics on scrape, so the dispatch path never touches the registry Performance (priority) • Enqueue is O(1), allocation-free, and non-blocking — verified by a test that fills the queue behind a blocked sink and asserts drop-not-block • Matcher benchmarks unchanged from Stage 1: 3.1 ms for 20 rules × 500-row batch, 2.2 ns / 0 allocs with no rules (feature off or empty rule set) • Status-log decode for matching only happens when a matcher is attached; result-log matching reuses the batch

LogHandler

already decoded • Cooldown claims, history writes, and all I/O run on worker goroutines only Validation •

go build ./...

,

go vet

,

gofmt

— clean •

golangci-lint run ./pkg/alerts/... ./pkg/logging/... ./cmd/tls/...

— 0 issues • New tests, all passing: • worker: dispatch→history round-trip, cooldown collapse + different-detail independence, sink-failure (no history, claim released), full-queue drop,

Close

drains queued hits, nil-worker no-op, adapter end-to-end, env-scoping (global vs per-env rules) • `pkg/logging/alerts_hook_test.go`: result hook receives decoded batch with env context, status hook receives full-schema decode, nil matcher is a pinned no-op, query hook fires per answered query • pre-existing `ProcessLogs`/`ProcessLogQueryResult` tests unchanged and green • Binary smoke test:

osctrl-tls --help

shows the flag; full build runs Roadmap position Stage 2 of 7 (

alerts-ingest-hook

). Matches now flow from osquery log ingest through rule evaluation to cooldown-gated history rows — observable end-to-end via

alert_history

and Prometheus. Next: Stage 3 (

alerts-channels

) implements email/webhook notification sinks behind the existing

DispatchSink

interface, followed by Stage 4 (management API) which also adds the service-command trigger for instant snapshot refresh. jmpsec/osctrl