https://github.com/osquery/osquery logo
Join Slack
Powered by
Im seeing duplicate data in the `shimcache` table ...
# windows
d
Im seeing duplicate data in the 
shimcache
table - is this expected?
m
Probably not expected. Is it all the time, or on certain hosts? Could the shims actually be installed twice?
d
Looking into it...
Ok so to clarify - the 
appcompat_shims
table looks for app compatibility shims. The 
shimcache
table uses a byproduct of the appcompat shim functionality, which is a listing of recently executed apps I've tested the 
shimcache
table on multiple W10 systems and I am seeing the same duplicate results. Looking at the 
shimcache
code, it looks like it is pulling from multiple places in the registry & not de-duping the resulting data: https://github.com/osquery/osquery/blob/master/osquery/tables/system/windows/shimcache.cpp#L35 In fact, 
CurrentControlSet
is a pointer to the currently used 
ControlSet001
So I'm thinking this wildcard that is in use 
%ControlSet%
is causing the duplicates.
s
I don’t know enough windows… • Do you think the wild card is wrong? • Should we dedup? • If the registry path is different, should we expose that?
d
Ive been testing out some changes. I would have expected that changing 
%ControlSet%
to 
ControlSet%
would only match: 
HKEY_LOCAL_MACHINE\\SYSTEM\\ControlSet001\\Control\\Session Manager\\AppCompatCache
but it appears it is still matching that and this one: 
HKEY_LOCAL_MACHINE\\SYSTEM\\CurrentControlSet\\Control\\Session Manager\\AppCompatCache
What am I missing?
s
First — I don’t know what correct is. Do you want to omit CurrentControlSet, but get all the ControlSet XXX ones?
Second… It’s going to match the full key. So you need the wildcard for 
HKEY_LOCAL_MACHINE\\SYSTEM\\
I’d have to test with code, but I’d expect you to be able to match the 
\
there. Something like 
%\\
. But I’m not totally sure how escaping it would work. (
\\
or 
\\\\
? I dunno)
I’m also not sure if you’re in sql wildcards or regex. I assume sql? In sqlite, 
_
matches a single character, while 
%
matches any range of characters. This can be used to more narrowly create a filter
d
I want to omit CurrentControlSet, but get all the ControlSet XXX ones. In the shimcache code, expandRegistryGlobs is being called (https://github.com/osquery/osquery/blob/master/osquery/tables/system/windows/shimcache.cpp#L204) expandRegistryGlobs looks like it is using sql wildcards.
s
Hrm. So you changed https://github.com/osquery/osquery/blob/master/osquery/tables/system/windows/shimcache.cpp#L35 from 
"HKEY_LOCAL_MACHINE\\SYSTEM\\%ControlSet%\\Control\\Session Manager\\AppCompatCache"
to 
"HKEY_LOCAL_MACHINE\\SYSTEM\\ControlSet%\\Control\\Session Manager\\AppCompatCache"
And still got 
CurrentControlSet
?
d
Correct.
s
That doesn’t match the sql wildcard. I’m not sure where it would be doing that
d
Howerver, there may be something else at play
When I query the registry table for ControlSet001, ProcMon is showing that it is actually pulling from CurrentControlSet. Again, CurrentControlSet maps to whatever ControlSet is active (ControlSet001, ControlSet002 etc).
Wondering if Windows is intercepting the request for 001 and just mapping it under CurrentControlSet.
If that makes any kind of sense
s
Yeah, I’m wondering about that too. But I’d expect sqlite to filter it out.
I think there’s something here I’m missing
d
Another view, this time of querying the shimcache table, with the current code. Again, looks like it is pulling twice from CurrentControlSet
m
Heya, sorry I am catching up again with this thread. Windows' Registry does have the concept of the Current and then the 001, 002 etc control sets as alternating backups for redundancy. This is by design, and yes, one of them maps to Current.
CurrentControlSet
is a symbolic link to either `ControlSet001`or 
ControlSet002
and Windows alternates which one of those it is. The other key is a backup for a "Last Known Good" feature.
If osquery is producing two identical results that is not helpful, obviously. Maybe it should only look at 
CurrentControlSet
.
I think this is enough to open an issue on the osquery repo, as a 
bug
and then the desired/expected behavior. One of us can at least confirm.
s
Does it make sense to pull everything and add a column to display whether it’s in the current control set?
z
@Marcos Oviedo any thoughts on this?
m
@seph I don't think it does unless there is a forensics use-case of trying to detect some difference between the Current and the Last version
m
Hey guys, joining late to the thread. @Mike Myers is correct. 
CurrentControlSet
is a symbolic link to 
ControlSet001
or 
ControlSet002
. I couldn't find an easy way to show that through PowerShell so I ended up relying on TotalRegistry tool to show that registry symbolic link (screenshot below). There is also this old link with more information on this. I think the code should just check 
CurrentControlSet
as this subkey reflects the current state of the system
d
To clarify - the shimcache process exec logs are only written to the registry when the system is being shutdown/rebooted. So when you query the table right now, you are actually seeing the logs up to 
now minus uptime
I think pulling just from 
CurrentControlSet
is a good place to land - if that is agreeable, I can submit a PR for that. We should also notate that in the schema notes.
m
change looks good! I've just made a comment on a minor styling issue that needs to be fixed to get the CI tests to pass
d
Fixed
m
I've just approved it 😄
d
Thanks!
6 Views
Open in Slack
PreviousNext
Powered by