#644 Consolidate Configuration into a Single YAML File with Environment Variable Support Issue created by zhuoyuan-liu The current osctrl configuration system uses multiple separate JSON files for different components: •

db.json

for database configuration •

redis.json

for Redis cache •

service.json

for service-specific configuration •

jwt.json

for JWT authentication • And other component-specific configuration files This multi-file approach creates several challenges: 1. Increased complexity in configuration management 2. Difficult to understand relationships between configuration parameters 3. Multiple files to manage during deployment 4. Less visibility into the overall system configuration ## Proposed Changes We should consolidate all configuration into a single YAML file with the following improvements: 1. Single Configuration File: Replace multiple JSON configuration files with a single YAML file 2. Hierarchical Structure: Organize configuration in logical sections (services, database, cache, authentication) 3. Environment Variable Support: Use environment variables for sensitive information like database passwords 4. Better Documentation: Include comments directly in the YAML file to document options ## Benefits 1. Simplified Management: Single source of truth for configuration 2. Better Readability: YAML format with comments makes configuration more understandable 3. Cloud-Native Friendly: Better alignment with Kubernetes ConfigMap patterns 4. Reduced Complexity: Simplified configuration loading logic ## Example Structure The single YAML configuration file could look like: # osctrl main configuration file # Common settings common: log_level: info log_format: console # Database configuration database: host: localhost port: 5432 name: osctrl username: postgres # Password comes from environment variable OSCTRL_DB_PASSWORD password: ${OSCTRL_DB_PASSWORD} max_connections: 100 conn_retry: 5 # Redis configuration redis: host: localhost port: 6379 # Password comes from environment variable OSCTRL_REDIS_PASSWORD password: ${OSCTRL_REDIS_PASSWORD} conn_retry: 5 # Service-specific configuration services: tls: listener: 0.0.0.0 port: 9000 host: https://tls.osctrl.domain.com auth: none logger: stdout # Support multiple exporter exporter: stdout: enabled: true s3: enabled: true bucket: osctrl-carves region: us-west-2 # Credentials from environment variables access_key: ${OSCTRL_S3_ACCESS_KEY} secret_access_key: ${OSCTRL_S3_SECRET_ACCESS_KEY} The struct may look like: // OsctrlConfig represents the complete configuration for osctrl type OsctrlConfig struct { // Database configuration (currently in db.json) DB backend.JSONConfigurationDB

yaml:"db"

// Redis configuration (currently in redis.json) Redis cache.JSONConfigurationRedis

yaml:"redis"

// Services configuration (replaces service.json for all services) Services struct { TLS config.JSONConfigurationService

yaml:"tls"

Admin config.JSONConfigurationService

yaml:"admin"

API config.JSONConfigurationService

yaml:"api"

}

yaml:"services"

// and more } jmpsec/osctrl