https://github.com/osquery/osquery logo
Title
d

Dougr

12/07/2018, 4:08 PM
We keep hitting a wall with FIM. If we monitor directories we end up having to create too many system specific excludes as there is usually a noisy file in the directories that is application specific. If we monitor specific files directly with inotify, when vi or something else changes it, you now no longer have a file monitor on it. Would it be feasible to add some type of setting to the inotify method to set a new watch on a file after x amount of time, so it can catch repeated changes to a file?
p

packetzero

12/07/2018, 4:14 PM
yes, and doing MD5,SHA1,SHA256 for every CREATE|UPDATE of a file is a killer.
d

Dougr

12/07/2018, 4:18 PM
Generally we are trying to set specific watches on files that should not be changed often. We just have no way of restarting Osq after each event, so after a single vi write, we have lost fim.
p

packetzero

12/07/2018, 4:20 PM
what do you mean by "need to restart osq after each event"?
d

Dougr

12/07/2018, 4:20 PM
If you monitor say, /etc/hosts with no wildcards using inotify.
After one change with vi, due to the way it edits files, you will get an event for that change, and no more after unless osq is restarted.
p

packetzero

12/07/2018, 4:22 PM
That sounds like a bug in either osquery FIM or inotify. You should get an event for every update of a monitored file.
d

Dougr

12/07/2018, 4:23 PM
Its no longer the same inode, its not a bug, its because vi creates a new file instead of in place edit.
p

packetzero

12/07/2018, 4:23 PM
often you get double events for each change...
d

Dougr

12/07/2018, 4:23 PM
With nano, it edits in place and you will not see that effect.
p

packetzero

12/07/2018, 4:23 PM
And FIM is relying on an open file handle... so it's a bug in FIM
it should periodically re-assess the config and open handles
d

Dougr

12/07/2018, 4:24 PM
Thats basically what I am looking to have happen, but currently it does not.
p

packetzero

12/07/2018, 4:25 PM
Also an issue if you want to monitor /etc/apache/file, but /etc/apache did not exist at osquery startup... won't get watched.
d

Dougr

12/07/2018, 4:25 PM
thats correct, if you watch using wildcards, it does see the new files
but then you end up with a ton of exclusions depending on what you ended up roping in
p

packetzero

12/07/2018, 4:27 PM
If you have time, please file an issue on FIM, this is probably not being considered.
d

Dougr

12/07/2018, 4:27 PM
I did a year or so ago, ended up adding a note to the docs so others know, but considered expected.
I was hoping to get some discussion around solutions as I don't want to file a repeat issue.
p

packetzero

12/07/2018, 4:30 PM
Is the issue you filed closed?
d

Dougr

12/07/2018, 4:31 PM
Yes
p

packetzero

12/07/2018, 4:39 PM
As a security solution, I don't see documenting as a sufficient resolution. Seems like an easy osquery FIM bypass by editing /etc/hosts by adding a space character, then editing again with malicious intent. I will take a closer look.
👍 1
The solution seems somewhat simple: catch IN_IGNORED event (already being done) and re-add watch. https://stackoverflow.com/questions/13409843/inotify-vim-modification
d

Dougr

12/07/2018, 5:01 PM
There would have to be some consideration for the timing on the new watch. The new file could not get written for quite some time.
p

packetzero

12/07/2018, 5:02 PM
yes
I tried this on Ubuntu 18 and seems to work just fine in 3.3.1, even though vim is doing the create/write instead of update.
Works in Ubuntu 16 as well, both with a 4.15 kernel variation. What linux and kernel version were you running?
d

Dougr

12/10/2018, 3:05 PM
CentOS 6.10 kernel 2.6.32-754
Also CentOS 7.5 kernel 3.10.0-862