Generally this would be the responsibility of whatever is ingesting the logs.
Many deployments have osquery push the logs off local disk directly, bypassing this completely.
10/26/2020, 10:58 PM
Thanks yes I was thinking msybe filebeat or logstash or something as i want to push to elastic but was thinking what would happen in a situation where the device was offline for a prolonged time and couldn't send logs back to elastic?
10/26/2020, 11:52 PM
If it helps:
We use kinesis for this the logs and we observed that given an impossibility of publishing (for example wrong creds or absence of internet), it caches the results until it can publish again.
When back online all of the results come at once, but got reports of osquery taking higher resources than expected in the devices.
Also I really recommend the Kinesis + Lambda for processing the logs. It scales as hell and it's pretty cheap. We're processing 16M records per day at a cost of ~85 USD a month (only counting computing power, not storage services).