I read the osquery doc, but I am not sure my interpretation is correct. I am trying to find: 1. what the maximum size of a log file is before it gets shipped to the logger destination 2. what is the maximum number of records a log file can hold before shipped to the logger destination

Hi @Vinny, logs sent to a destination are buffered in the osquerydb, not saved to a file. You can modify the maximum buffer size, as well as the max number of logs to be sent at once and max log size to be sent in the osquery configuration. You can read more about how to set this here. f you're using filesystem logging in addition to TLS logging, You can use the

--logger_rotate=true

to tell the filesystem plugin to rotate logs based on size.

--logger_rotate_size=<number of bytes>

and

--logger_rotate_max_files=25

will let you specify the size or number of files to trigger rotation. You can read more about that here. Hope this helps!

I am sending them to AWS Firehose, and when they get to S3 they seem to be a file. Is the agent sending them in a file or is Firehose compacting them into files and saving in S3 as such?

Should be that: Agent sends logs to Fleet as JSON -> Fleet sends logs to Firehouse as JSON -> Firehouse batches logs together and saves them to files in S3

Thank you!