Title
#fleet
Jocelyn Bothe

Jocelyn Bothe

04/20/2021, 7:32 PM
is there a way to tell it to dump old records?
zwass

zwass

04/20/2021, 8:30 PM
Is this running a standard version of Fleet? Which version? I am looking at the code and can't understand how it would try to send more than 500 records in a request. Kinesis logging was a community contribution so I'm not super familiar with it but we would like to fix the bug if we can figure out what's going on.
8:31 PM
To clear out the old records you'd pass
--buffered_log_max=1
to osquery, which would cause it to drop all but the newest log after the next logging attempt.
Jocelyn Bothe

Jocelyn Bothe

04/20/2021, 8:53 PM
we're running the latest release of Fleet
8:57 PM
we're running on 2500 hosts, we don't have an easy way to change osquery flags on the fly
8:58 PM
[root@osquery-service-vab183 ouser]# /usr/bin/fleet version
fleet version 3.10.0
zwass

zwass

04/20/2021, 9:51 PM
Another idea could be switching Fleet's logging plugin to
filesystem
or
stdout
and waiting until osquery is done dumping the logs. Then switch back to Kinesis.
9:51 PM
Any idea how it got into this state?
9:52 PM
Advantage to the above is you could potentially save the logs and feed them back into Kinesis if you need them to get back into your logging pipeline.
Jocelyn Bothe

Jocelyn Bothe

04/21/2021, 2:07 PM
cool, switching the logger to filesystem was going to be my next attempt 🙂
5:13 PM
that seems to have worked to get rid of the backlog