https://github.com/osquery/osquery logo
Join Slack
Powered by
(I should clarify that I thought that `/%%` would ...
# general
z
(I should clarify that I thought that 
/%%
would recurse down from the root directory, but the top query is only return 3 results for some reason, which is not what I thought it would do)
m
Hey Zander, this isn't exactly an answer, but in the mean time this made me think this is something that might be valuable in Fleet (if so, feel free to add context: https://github.com/fleetdm/fleet/issues/1862)
FWIW I get 20 results from both queries
image.png
I see it going recursively deep:
👍 1
p
I also thought 
/%%
would recursively go through the full file system but it usually went only 2-3 directories but I may have been doing something wrong as it looks like mikermcneil got some accurate? results Also just to addon to the fleet issue/feature/idea In addition to listing recently modified files, listing all created files in a specified time range could be very powerful/valuable Ex scenario: attacker logs onto system via RDP, session lasts 4 hours. Run query to list all files created during that 4 hour period. Same idea with modified, accessed, etc
m
Actually... I bumped up my 
LIMIT
to 200 to verify, and I'm seeing what looks like a max depth of 3 directories deep too
👍 1
@puffycid got it, thanks! I'll add that feedback to the issue
👍 1
This behavior is unexpected to me too-- if intentional, I think it deserves to be mentioned in the table documentation: https://osquery.io/schema/4.9.0/#file (I'll make a note to discuss at office hours if no one comes up with a solution before then)
s
Generally speaking, this kind of thing is going to crawl disk. And probably not work well.
Doing this well would require something with file events. I don't know if you could do it wholly inside osquery or if you need the server to coordinate.
m
Seph do you know if the cap on depth is intentional?
s
I don't. I'd have to look at the source code. Offhand, it seems like a reasonable limit for resource utilization. Crawling disk is a common way to lock machines.
ty 1
I guess we have other limits exposed in various ways? This could be too? But I've never seen a fa crawl be a good idea. So I think it'd be better to pursue this a different way.
p
Just to add my 2 cents about some of the reasons u may want a full file listing (or at least scan the whole disk for a single system): • Look for all files created in a specified time range • Count number of files encrypted by ransomware • Hunt for webshells (would also require reading/regex file contents to be effective, whole different discussion :)) • Data staging for exfil (ex: look for all files greater than 10GB) • Scan for indicators of compromise (ex: does the filename mimkatz.exe exist anywhere on disk) Some of the examples r applicable to one system others would be multiple systems Ex: query all servers for websells/all files with the extension php Vs Ex: query all files created during the rdp session on serverA If others have other ideas and thoughts on how to get this data without a full file listing I would be very interested 😀
👍 2
Also just to add the stuff above is stuff I do at work So pulling a full file listing is definitely a common/normal thing to do for incident reaponse(IR)/forensics And since osquery can be used for IR/forensics (along with other stuff) It would be nice to have this capability if an implementation was made (that works well) IMO
z
Agreed that this would be useful. In my case, I was doing IR on machine specifically walled off and snapshotted for this purpose, so the resource use wasn’t an issue. In any case, It would be nice to be able to (as a user) make that determination myself (by tuning some value). What I’m trying to do is improve our IR processes and put them as much in osquery as possible. To perform this task another way, I’d use 
find
fully knowing it could lock the machine:
Copy code
find src/ -type f -print0 | xargs -0 stat --format '%Y :%y %n' | sort -nr | cut -d: -f2- | head -n 20
👍 2
m
goal: • is there a bug here? • if not, can we document the limitation it in the table docs on osquery.io/schema ? Notes: • could be due to 50MB filesystem read limit (that is configurable) -- could try configuring that to rule out it's not this • another potential culprit: https://github.com/osquery/osquery/blob/40bbe86f91fcf8387cb249594127e2e30b7c2013/osquery/filesystem/filesystem.cpp#L52 • Stefano: on linux, we use the glob function, not sure if it's necessarily the fastest way to crawl the fs. On windows, it is slower. If interested in crawling the whole filesystem, it's bound to be slow, especially on Windows. If we are interested in solving this, we might consider giving a second look to Windows to improve performance. • Seph: Could be another type of limit we aren't thinking of right now as well. • mikermcneil: I'll make a bug issue. @Zander Mackie @puffycid In the meantime, it's potentially worth experimenting with the config for the filesystem read limit to see if it fixes this-- if so, that could prove that the filesystem read limit is the problem. See recording of office hours for more context.
👍 1
z
Left a comment to this effect in the issue, but this does not appear to be fixed by adjusting 
read_max
.
(at least testing on macOS)
s
read_max
should be about max file size.
z
could be due to 50MB filesystem read limit (that is configurable) -- could try configuring that to rule out it’s not this
Was this referring to 
read_max
or something else? I was a bit confused because the docs agreed with what @seph pointed out.
s
I wonder if osquery has full disk access permissions
s
I thnk the read_max was a speculation.
z
Ah gotcha.
Open in Slack
PreviousNext
Powered by