<#1063 Added a generic HTTP logsink> Pull request ...
# osctrl
g
#1063 Added a generic HTTP logsink Pull request opened by javuto Overview Adds a new generic HTTP log sink (
http
) so osquery logs can be forwarded to any HTTP/HTTPS endpoint with configurable method, headers, format, and metadata wrapping. This is the twelfth sink type, joining Splunk, Graylog, Kafka, Kinesis, S3, Elastic, Logstash, DB, file, stdout, and none. Configuration The sink exposes seven fields through the existing dynamic form schema (no new frontend component needed): | Field | Type | Default | Description | | --------------- | -------------------- | ------------------- | ------------------------------------------------------------------------ | | url | string (required) | — | HTTP/HTTPS endpoint | | method | select | POST | HTTP method (POST or PUT) | | format | select | json | json (forward as-is), ndjson (one object per line), raw (bytes verbatim) | | contentType | string | derived from format | Overrides Content-Type header | | headers | string (JSON object) | — | Custom HTTP headers, e.g. {"Authorization":"Bearer xyz"} | | includeMetadata | boolean | false | Wraps each event with environment, uuid, log_type, timestamp | | timeoutSeconds | integer | 30 | Per-request timeout | Backend changes Config (
pkg/config
)
•
loggers.go
— new
HTTPLogger
struct with URL, method, headers (
map[string]string
), format, content type, timeout, and metadata toggle. •
types.go
—
LoggingHTTP = "http"
constant;
HTTP *HTTPLogger
field added to
YAMLConfigurationLogger
. •
validation.go
—
LoggingHTTP
registered as a valid logging type. Exporter (
pkg/logging
)
•
http.go
(new) —
LoggerHTTP
with a pooled
http.Client
, configurable timeout, and three encoding modes: • JSON — forwards the raw payload with
application/json
content type. • NDJSON — splits array payloads into newline-delimited JSON objects. • Raw — sends bytes verbatim with the configured content type. • When
includeMetadata
is true, each event is wrapped in an
httpEnvelope
adding environment, UUID, log type, and Unix timestamp. • Non-2xx responses are logged at warn level; errors do not stall the ingestion path. •
exporter_adapters.go
— `Name`/`IsEnabled`/`Export` methods so
LoggerHTTP
satisfies the
DataExporter
interface. •
exporter_factory.go
—
config.LoggingHTTP
case in
CreateExporter
. Registry (
pkg/logsinks
)
•
logsinks.go
— new
config.LoggingHTTP
entry in
Registry
with field specs for the dynamic form, a
Build
func, and a custom
decodeHTTPConfig
decoder that handles
headers
arriving as either a JSON object (API/YAML) or a JSON string (frontend single-line input). Seed path wired in
configRowForType
. •
logsinks_test.go
—
TestRegistryCoversAllYAMLTypes
updated to include `http`; added tests for registry presence, header decoding (object + string shapes), empty/null config, exporter build, and seed round-trip. Frontend changes •
LogSinksPage.tsx
— added an icon (arrow-up/upload pictogram) for the
http
sink type in
SINK_TYPE_ICONS
. Tests •
pkg/logging/http_test.go
(new) — 12 tests: defaults, nil config, JSON/NDJSON/raw formats, metadata envelopes (JSON + NDJSON), custom headers, PUT method, non-2xx response handling,
Export
adapter,
Close
idempotency. •
pkg/logsinks/logsinks_test.go
— 6 new tests: registry entry, header decoding variants, empty config, exporter build, seed round-trip. •
pkg/config/validation_test.go
—
TestValidateTLSConfigValuesAcceptsHTTP
. Validation •
go build ./...
— pass •
go test ./pkg/logging/... ./pkg/logsinks/... ./pkg/config/...
— pass •
golangci-lint run
— 0 issues •
npm run check
(TypeScript) — pass •
npm test
(415 frontend tests) — pass •
make openapi-check
— no changes needed (sink types are served dynamically from the registry) jmpsec/osctrl