Hi! I keep hitting the following error: ```"level=...
# fleet
s
Hi! I keep hitting the following error:
Copy code
"level=error ts=2023-01-17T20:52:34.232573858Z component=http method=POST uri=/api/v1/osquery/distributed/write took=419.259117ms ip_addr=74.122.186.196 x_for_ip_addr="74.122.186.196, 35.201.103.38" ingestion-err="ingesting query users: update host users: insert users: Error 1390: Prepared statement contains too many placeholders" err="error in query ingestion"
"
Have seen a mention here in #kolide: https://osquery.slack.com/archives/C1XCLA5DZ/p1531260258000023 Currently running
fleetdm/fleet:v4.25.0
with ~900 hosts
z
Have you double-checked you ran all the DB migrations.
s
Just doubled checked and ran the migrations again
Copy code
~/Development/infosec-fleet-kubernetes master kubectl logs job.batch/fleet-prepare-db -c fleet -n fleet
Migrations already completed. Nothing to do.
Following up on this issue, as this might be red herring. The real problem im having is that after upgrading osquery from 5.2.2 to 5.7 our pubsub topic stopped receiving logs. However all the agents were online and queryable through the UI after the update to 5.7. Reverting to 5.2.2 did not fix the issue either
k
Can you verify the logging configuration on your agents by running the following query through Fleet:
Copy code
SELECT name, value FROM osquery_flags WHERE name = "logger_plugin" ;
s
image.png
should this be tls?
Looks like it is reverting for filesystem for all of our linux hosts
Thanks! Ill read into this
Looks like we are using
config_plugin
to configure tls?
Copy code
--logger_stderr=false
--logger_min_status=1
--enroll_tls_endpoint=/api/v1/osquery/enroll
--config_plugin=tls
--config_tls_endpoint=/api/v1/osquery/config
--config_refresh=1800
--watchdog_level=0
--watchdog_memory_limit=1024
--watchdog_delay=3600
Looks like it has changed to
--logger_plugin=name
Where can I find deprecations like this? I cant seem to find it in the release notes?
k
You'll need to set that in your osquery flags. I don't believe that the flag itself has changed, but osquery itself has gotten a bit stricter about how and where command line flags can be set. It sounds like you may have previously set that in your Fleet agent options. is that right?
s
Correct, I did notice that last night when pulling down my config
Copy code
~/Development/infosec-fleet-kubernetes/staging master * sudo fleetctl apply -f fleet_org.yml
Warning: Version mismatch.
Client Version:   4.24.0
Server Version:  4.25.0
Error: applying fleet config: PATCH /api/latest/fleet/config received status 400 Bad Request: unsupported key provided: "logger_mode"
~/Development/infosec-fleet-kubernetes/staging master * sudo fleetctl apply -f fleet_org.yml
Warning: Version mismatch.
Client Version:   4.24.0
Server Version:  4.25.0
Error: applying fleet config: PATCH /api/latest/fleet/config received status 400 Bad Request: unsupported key provided: "logger_mode"
~/Development/infosec-fleet-kubernetes/staging master * V                                   
~/Development/infosec-fleet-kubernetes/staging master * vi fleet_org.yml                    
~/Development/infosec-fleet-kubernetes/staging master * sudo fleetctl apply -f fleet_org.yml
Warning: Version mismatch.
Client Version:   4.24.0
Server Version:  4.25.0
Error: applying fleet config: PATCH /api/latest/fleet/config received status 400 Bad Request: unsupported key provided: "logger_plugin"
~/Development/infosec-fleet-kubernetes/staging master * vi fleet_org.yml                    
~/Development/infosec-fleet-kubernetes/staging master * sudo fleetctl apply -f fleet_org.yml
Warning: Version mismatch.
Client Version:   4.24.0
Server Version:  4.25.0
Error: applying fleet config: PATCH /api/latest/fleet/config received status 400 Bad Request: unsupported key provided: "utc"
Had to delete these keys
logger_mode
,
logger_plugin
,
utc
k
s
Thanks Ill definitely review