Has anyone seen an issue where windows just stops sending osQ results to AWS Kinesis from some machines? I seem to have at least 2 machines in this state currently. Any pointers would be appreciated

some possible reasons: 1. the AWS token expires and agent is not successful in refreshing. 2. The machines are generating a ton of results, and sending is slow enough that you don't see any recent results. There are several performance issues with the buffered logger that the kinesis logger depends on.such as: https://github.com/facebook/osquery/issues/5494 https://github.com/facebook/osquery/issues/5495

Thanks @packetzero - W.r.t. 1: The same token is used on all clients, so we have over 300 other clients successfully using the token, so this is ruled out. The github issues would make sense if I was getting some results in (albeit late), but the two machines I mention above, we literally aren’t seeing anything… I’m wondering if this is due to another issue you may have contributed on: https://github.com/facebook/osquery/issues/5554

Look at the log files, are there a ton of them with timestamps close together? This will indicate watchdog killing agent frequently due to watchdog settings and a demanding schedule.

We pull osquery logs to kinesis with fluentd. May be an option if you don't find resolution.