Channels
#general
Title
p
puffycid
08/28/2021, 3:07 AMi have quick question
does anyone have a query to get a full file listing of a system using osquery?
i tried the one mentioned at https://blog.kolide.com/the-file-table-in-osquery-is-amazing-99db0f52a066
but it does not seem to be returning all the files (i may be doing something wrong)
i also tried searching slack but did not get any hits
thanks
s
seph
08/28/2021, 1:47 PM
does anyone have a query to get a full file listing of a system using osqueryGenerally speaking this is a pretty hard thing. I don’t know how to do it, short of a really intense disk crawl. I think that’s true for all platform and software, really.
😢 1
p
puffycid
08/28/2021, 2:09 PMdarn thats kind of major limitation/gap from a forensic perspective
is it a design decision or something that just hasnt been added because no one has volunteered to implement it?
there are probably some technical challenges to implementing it, but generally speaking i think most forensic/live response software can pull a full file listing
(i realize osquery is used for alot of other reasons/use cases, and a full file listing may not be as important for other use cases)
s
seph
08/29/2021, 12:29 AM
Technically speaking, I’m not aware of any tool that does this without crawling the filesystem. Sometimes in off hours. Usually at high performance cost.
I’m not sure it’s the kind of thing we’d want directly supported. It feels a bit weird privacy wise, and I think an implementation wouldn’t fit very well.
p
puffycid
08/29/2021, 1:22 AMyes, getting a full file listing requires crawling the full file system? 😅
there are several forensic tools that can do this (in addition to hashing (md5) all files, and other stuff), this is a standard practice when doing a full IR investigation (vs triaging)
for linux and mac a full disk crawl/file listing is often a requirement if a full live response investigation is being done
for windows parsing the MFT file may be enough to get a full file listing (contains metadata for each file/directory on disk)
can u clarify the privacy question/thought?
its just metadata of all the files (ex: filename, directory, timestamps, attributes?)
a full file listing would only be needed when doing an investigation, (maybe hunting too, with limits ex: all files that have the extension ".dmp" or all zip files greater than 10GB in size (potential data staging))
s
seph
08/31/2021, 2:19 AM
My gut sense is that there are two areas to work through. I don’t know the answer to either of them. this is a pretty big proposal.
First. There’s the technical implementation. I don’t think this can be done well on-demand. There’s just too much data. I think to do this right would require something that crawled in the background and maintained state.Analogous to spotlight, or other security tools with this kind of functionality.
It would be a big thing to add to osquery.
👍 1
Second. Privacy. I’m not sure where this falls.
I’m hesitant to try to wrap my head around either of those separately, They’d both need to be answered.
OTOH. if there are reasonable other tools that do this, is there an extension way to get at this?
p
puffycid
08/31/2021, 5:54 AMUnderstood, I may try playing around with some ideas I have
Though I'm sure others have tried them 😄
This sounds like it could be a good office hours discussion?
I can't attend this weeks office hours
But the next one on sept 14? I should be able to attend \o.o/
When I reference other tools I was talking more along the lines of other EDR/IR/forensic applications
No of which would work well as an extension I think? (Unless the extension shells out to run the application)
s
seph
08/31/2021, 3:16 PM
You could also open a ticket, and try to get conversation going.
👍 1
Ignoring the privacy side for a moment… I see 2 ways this could go.
One, is adding robustness to the file table to allow an intense on-demand crawl.
The other is building a subsystem to try doing this more asyncronously and more gently. (I think that pattern is common in EDR tools.)
😮 1
5 Views