We think we found the problem, but we’re not sure why. Switching from kinesis data stream to kinesis firehose brought us down from a duplicate entry count of over 100% to about 2% which is likely from overlapping queries.