Hi all, i have this issue (https://github.com/osquery/osquery/issues/7079) where osquery is generating a big amount of IOPS, and i am trying to figure out how to improve the situation. This boxes are windows boxes with a load of events of around 100 events/sec. We had the

logger_tls_max_lines

increased from 1024 to 4096 which seems to help improve the situation, as it avoids the a growing pattern of the DB over time (seems to be cleaned up every hour), however there are still a lot of .LOG files created (it goes up to 300 files). I have been considering the idea of completely disabling the use of WAL in RocksDB, which suppose a drastical drop in IOPS. I can see in the code that there is a disableWAL in case of events, but it does not seem to be working, as LOG files build up even when this box is only looking at windows events.

// Events should be fast, and do not need to force syncs.
  auto options = rocksdb::WriteOptions();
  if (kEvents == domain) {
    options.disableWAL = true;
  } else {
    options.sync = true;
  }

I just disabled WAL in every case, and then i can get rid of the LOG files and see a big drop in IOPS. I do not have a big reason to not disable WAL as it seems that we may only lose some in-memory data in case of fatal crash, but i am pretty new to this so, does somebody know any side-effects that i am missing?

Let’s brainstorm on the GH issue