Hi team! In fleet 4.6.0 there was an update "Updat...
# fleet
m
Hi team! In fleet 4.6.0 there was an update "Update the JSON payload for the host status webhook by renaming the "message" property to "text" so that the payload can be received and displayed in Slack." and after that update I get big amount of errors from osqueryd "Error sending results to logger: Cannot parse JSON: Invalid value. Offset: 0". Did anyone get this problem and how can I fix this? I use logger-plugin TLS and before updating from 4.5.0 there was none of these errors.
I found out that POST requests are trying to be sent to /api/osquery/log but in flagfile the path is /api/v1/osquery/log - why the path changes by itself?😄
l
Hi @Mikhail! Couple of questions: 1. Any reason not to update to a more recent version of Fleet? (WE strongly advice this as there have been many bug fixes, features, and performance improvements since 4.6.0). 2. Which
flagfile
are you referring to? Are you assembling `flagfile`s manually for hosts running osquery? 3. Can you share the logs that show the error? Also what's your Agent Settings/Options in the Fleet UI (particularly the field
logger_tls_endpoint
).
m
@Lucas Rodriguez, hello! 1. I updated in several steps - to 4.6.0, then to 4.15.0 and to 4.25.0, and the problem already started after the first update. 2. I use flagfile osquery.flags for hosts with osquery installed on them instead of osquery.conf - to be honest I don't think that it makes difference, but maybe I'm wrong. 3. Sure, I can also show whole flagfile:
Copy code
--enroll_secret_path=C:\Program Files\osquery\enroll_secret
--tls_hostname=my.server:8081
--pidfile=C:\Program Files\osquery\osqueryd.pidfile
--host_identifier=uuid
--enroll_tls_endpoint=/api/v1/osquery/enroll
--config_plugin=tls
--config_tls_endpoint=/api/v1/osquery/config
--config_refresh=60
--disable_distributed=false
--disable_events=true
--disable_extensions=true
--disable_tables=curl
--distributed_plugin=tls
--distributed_interval=60
--distributed_tls_max_attempts=3
--distributed_tls_read_endpoint=/api/v1/osquery/distributed/read
--distributed_tls_write_endpoint=/api/v1/osquery/distributed/write
--logger_plugin=tls
--logger_tls_endpoint=/api/v1/osquery/log
--logger_tls_period=10
--watchdog_memory_limit=400
--watchdog_utilization_limit=150
and log snippet after queries distributing say this:
Copy code
I1229 16:05:17.613806 17656 tls.cpp:255] TLS/HTTPS POST request to URI: <https://my.server:8081/api/osquery/log>
I1229 16:05:17.936024 17656 buffered.cpp:90] Error sending status to logger: Cannot parse JSON: Invalid value. Offset: 0
l
Great, thanks for the detailed info, can you share Fleet's agent settings also? From the browser UI: Settings -> Agent options -> Look for
logger_tls_endpoint
.
m
Here:
Copy code
Message:
config:
  options:
    logger_plugin: tls
    disable_tables: windows_events
    pack_delimiter: /
    logger_tls_period: 10
    distributed_plugin: tls
    disable_distributed: false
    logger_tls_endpoint: /api/osquery/log
    distributed_interval: 10
    schedule_splay_percent: 10
    logger_snapshot_event_type: true
    distributed_tls_max_attempts: 3
    distributed_tls_read_endpoint: /api/v1/osquery/distributed/read
    distributed_tls_write_endpoint: /api/v1/osquery/distributed/write
  decorators:
    load:
      - SELECT uuid AS host_uuid FROM system_info;
      - SELECT hostname AS hostname FROM system_info;
  file_paths:
    efi:
      - /System/Library/CoreServices/boot.efi
    binaries:
      - /usr/bin/%%
      - /usr/sbin/%%
      - /bin/%%
      - /sbin/%%
      - /usr/local/bin/%%
      - /usr/local/sbin/%%
      - /opt/bin/%%
      - /opt/sbin/%%
    configuration:
      - /etc/%%
command_line_flags: {} # requires Fleet's osquery installer
Actually, due to errors info, logs come to /api/v1/osquery/log, but fleet config uses /api/osquery/log - it's very strange. Maybe config changed after server update somehow or it's impossible?
l
Maybe config changed after server update somehow or it's impossible?
Maybe? Am not sure either.
logger_tls_endpoint
Amend, save settings and hosts should eventually start using the correct path. Makes sense?
m
Trying to correct, but getting error
Copy code
Could not update settings. unsupported key provided: "logger_plugin"
Trying to make new config.yaml, but getting error
Copy code
Error: no spec field on "" document
r
we added validation of the config in recent versions, please try removing
logger_plugin
from the settings as it's not a valid key anymore and it's safe to remove
m
@Lucas Rodriguez @roberto thank you very much, the problem disappeared!