i have quick question does anyone have a query to get a full

Question

i have quick question does anyone have a query to get a full file listing of a system using osquery i tried the one mentioned at <https blog kolide com the file table in osquery is amazing 99db0f52a066> but it does not seem to be returning all the files i may be doing something wrong i also tried searching slack but did not get any hits thanks

seph · Answer

gt does anyone have a query to get a full file listing of a system using osquery Generally speaking this is a pretty hard thing I don t know how to do it short of a really intense disk crawl I think that s true for all platform and software really

puffycid · Answer

darn thats kind of major limitation gap from a forensic perspective is it a design decision or something that just hasnt been added because no one has volunteered to implement it there are probably some technical challenges to implementing it but generally speaking i think most forensic live response software can pull a full file listing i realize osquery is used for alot of other reasons use cases and a full file listing may not be as important for other use cases

seph · Answer

Technically speaking I m not aware of any tool that does this without crawling the filesystem Sometimes in off hours Usually at high performance cost

seph · Answer

I m not sure it s the kind of thing we d want directly supported It feels a bit weird privacy wise and I think an implementation wouldn t fit very well

puffycid · Answer

yes getting a full file listing requires crawling the full file system sweat smile there are several forensic tools that can do this in addition to hashing md5 all files and other stuff this is a standard practice when doing a full IR investigation vs triaging for linux and mac a full disk crawl file listing is often a requirement if a full live response investigation is being done for windows parsing the MFT file may be enough to get a full file listing contains metadata for each file directory on disk can u clarify the privacy question thought its just metadata of all the files ex filename directory timestamps attributes a full file listing would only be needed when doing an investigation maybe hunting too with limits ex all files that have the extension dmp or all zip files greater than 10GB in size potential data staging

seph · Answer

My gut sense is that there are two areas to work through I don t know the answer to either of them this is a pretty big proposal

seph · Answer

First There s the technical implementation I don t think this can be done well on demand There s just too much data I think to do this right would require something that crawled in the background and maintained state Analogous to spotlight or other security tools with this kind of functionality It would be a big thing to add to osquery

seph · Answer

Second Privacy I m not sure where this falls

seph · Answer

I m hesitant to try to wrap my head around either of those separately They d both need to be answered

seph · Answer

OTOH if there are reasonable other tools that do this is there an extension way to get at this

puffycid · Answer

Understood I may try playing around with some ideas I have Though I m sure others have tried them smile This sounds like it could be a good office hours discussion I can t attend this weeks office hours But the next one on sept 14 I should be able to attend o o When I reference other tools I was talking more along the lines of other EDR IR forensic applications No of which would work well as an extension I think Unless the extension shells out to run the application

seph · Answer

You could also open a ticket and try to get conversation going

seph · Answer

Ignoring the privacy side for a moment I see 2 ways this could go

seph · Answer

One is adding robustness to the file table to allow an intense on demand crawl

seph · Answer

The other is building a subsystem to try doing this more asyncronously and more gently I think that pattern is common in EDR tools