Title
#windows
thor

thor

07/19/2018, 4:04 PM
None of the information you get from osquery is coming from a cached place, with the exception of event based tables. As such there should never be a need to interact with the rocksdb backing store. You can use the
--database_dump
flag if you'd like to investigate the contents, but there's not need to insert/modify data that's in the DB already
b

Bit_by_bit

07/19/2018, 4:19 PM
I see..I thought Rocksdb would be used for more. Then I guess the same would also apply to memory maps and processes as well. What about logging information then? Where (which db) would that come from? Also, as I mentioned before, is none of my stuff logged due to me being unable to set 'disable_logger = 'false' in osquery.conf?
4:21 PM
Also, how is the differential calculated by Rocksdb if none of the previous info is stored? Sorry for asking stupid questions..I am just a bit confused and trying to learn šŸ˜…
4:21 PM
@thor how is logging done in osquery by default then if the disable_logger option is by default set to true and rocksdb is used only for differential queries? Are queries from osqueryi not logged and only ones written in osquery.conf for osqueryd are logged?
thor

thor

07/20/2018, 4:26 PM
Hey sorry I didn't see these šŸ™‚ ā€¢ I really don't have much experience with the
disable_logger
flag, so you might need to do some digging. I've also not really heard of folks using this, what about just using
logger_plugin=stdout
? What are you trying to accomplish by disabling the logger? ā€¢ You're right, the rocksdb can also be used for log caching, and every period of the logger buffer that's specified this cache will be flushed to whatever logger plugin is being used. But again, this isn't something we typically encourage folks to interact with. ā€¢ Queries from osqueryi are just returned directly to the shell.
osqueryi
was really intended originally to be used more for query debugging and shell experience. It's not really intended for getting you information in a periodic manner, that's what
osqueryd
is for. When you launch into
osqueryi
there's a number of plugins and features that aren't turned on, as osquery is just an interactive shell that operates with the osquery registry to allow you to call different tables. You can turn on a bunch of the features using the CLI flags, but again this wont properly emulate how the daemon operates. For a lot of these questions it might be useful to know more about what you're trying to accomplish. Are you hoping to achieve something specific? Or are you just exploring the code base and the features?
4:26 PM
Ah one more point -- The differentials are computed by, as you might be expecting šŸ™‚, storing the results of the previous query in the rocksdb, and when the next query interval hits we do a "diff" on the serialized data blobs, and the difference is what gets cached before eventually being logged.
b

Bit_by_bit

07/20/2018, 4:55 PM
@thor I am trying to figure out how data is stored and retrieved with osquery and what the differences are between osqueryd and osqueryi. (I think osqueryd code runs on the config file while the shell is interactive) There is also code using sqlite, ephemeral, glog, rocksdb and just database.cpp and I am pretty sure from lookin online that it is all event based. Just trying to fiure out how these work together and make a mental flowchart diagram of sorts...
4:58 PM
And disable_logging=false means that I will be enabling logging right?
thor

thor

07/20/2018, 5:32 PM
Yeah
disable_logging=false
should enable logging, but honestly I just wouldn't touch that flag at all. For the record, osqueryi and osqueryd are the same binary, there's just a slightly different code path taken when the binary name is osqueryi.exe, which is to drop you to an interactive sqlite shell. Both osqueryi and osqueryd generate data in the same way, but osqueryd gets it's query information from the configuration whereas osqueryi gets its query information from the shell. Further, osqueryd has a scheduler, where as osqueryi doesn't active this (I think, I could be wrong ^.^) Hope that helps!
b

Bit_by_bit

07/20/2018, 5:48 PM
Yup! This all makes sense when looked at the code