Hello! For the life of me, I can't get pubsub queues to work. This is my config:

mysql:
  address: SNIP
  database: SNIP
  username: SNIP
  password: SNIP
redis:
  address: 127.0.0.1:6379
server:
  tls: false
logging:
  json: false
  debug: true
  #tracing_enabled: true
  #tracing_type: opentelemetry
pubsub:
  project: my-project
  result_topic: my-result-topic
  status_topic: my-status-topic
osquery:
  osquery_status_log_plugin: pubsub
  osquery_result_log_plugin: pubsub

Fleet is running in a VM within GCP. ADC definitely works (

gcloud pubsub topics publish my-result-topic --message "Hello world!"

) works from the host with the default ADC creds from the attached Service account, but I don't see anything in the fleet logs about pubsub. I'm not running Fleet Premium on this server if that makes a difference.

Hey @Jacob Burley, Here are some env vars setup for another client to use GCP as example

FLEET_PUBSUB_PROJECT             = "pe-insights-private"
    FLEET_PUBSUB_RESULT_TOPIC        = "fleet-macos-data"
    FLEET_OSQUERY_RESULT_LOG_PLUGIN  = "pubsub"

Can check on mappings here https://fleetdm.com/docs/configuration/fleet-server-configuration Looks like you might have wired up correctly from quick glance. Would you have have "Automations" turned on for the scheduled queries?

Hey @Grant Bilstad! Automations are definitely turned on. I have prod and dev fleet servers (this is in dev), and in prod the filesystem logs defo work with automations enabled. In dev, I have a test query run every 5 mins with automations enabled, and nothing ends up in pubsub seemingly 😅 I’ll try environment variables in the systemd file tomorrow

Awesome, sounds good and let us know how it goesfleet

Are you seeing traffic coming in on the logging endpoint?

/api/v1/osquery/log

you should see the following log line (at info level) at fleet server startup if its configured properly:

GCP PubSub writer configured

then if ADC credentials are incorrect and fleet was getting an error writing to pubsub topic, you'd get error logs in the server logs

@Kathy Satterlee indeed i do, every 5 minutes from my home IP. lines up with my automation query