https://github.com/osquery/osquery logo
Join Slack
Powered by
Has anyone has success in getting include and/or e...
# eclecticiq-polylogyx-extension
l
Has anyone has success in getting include and/or exclude filters to work. No matter what I try I can't seem to filter out anything with exclude or only collect specific things with include. Any help would be appreciated
@OpenPlgx any thoughts
o
Hi @lvferdi, Seeing the messages now. Are you having problems only with socket event filters or filters in general? How about default filters..are they working fine?
l
I am not able to get any of the filtering to apply to any of the queries.
The only error I have states 
no event_filter found
o
are you running osqueryi/osqueryd? How are the configs refreshed on the agent?
What is your osquery management infrastructure?
l
This is running on a test machine in AWS. It is running using osqueryd. I have the options set in the 
osquery.conf
file and the queries in a query pack
o
did you restart the osqueryd after making changes in the osquery.conf?
l
yes, several times. And to confirm it was reading the config I made json errors and upon restart it would advise of the errors in the logs
o
ok
l
If it helps I am on osquery version 4.9.0
o
and the extension is loaded, right?
l
correct as I get results of the osquery query but none of the filtering of the extension
I can see the table and query 
select * from win_socket_events;
and receive the results, but I don't see the filtering being applied by the extension.
o
hmm..weird...can you export the following registry key into a text file and share? HKLM\System\CurrentControlSet\Services\vast
l
one moment
o
In the meantime, let me try to repro locally and give you a set of config that works (with a test tool)
Also, is this a regression? Did you try them with the earlier extension and found it to be working then?
l
no this is my first attempt with the extension. I tried the earlier version unsuccessfully but after 2 days the new version was released.
apologies, that has a larger config running on it, which includes socket and process events. I can put the single query single filter config on the machine and export again
o
That will help
These filters seem empty
Can you values for "include"/"exclude" keys under each event category?
l
sorry had to run the kids to school. I'll look now
They all say 
value not set
here are the options I have set in my osquery.conf file and the single query and filter I have set in my pack.
Copy code
{
"utc": "true",
"custom_plgx_EnableSSL": "true",
"custom_plgx_EnableAmsiStreamEventData": "true"
},



{
  "version": "1.0",
  "queries": {
    "win_process_events": {
      "query": "select wpe.action, wpe.eid, wpe.pid, wpe.path, wpe.process_guid, wpe.cmdline, wpe.parent_pid, wpe.parent_path, wpe.parent_process_guid, wpe.owner_uid, wpe.utc_time as time, h.md5, h.sha1, h.sha256 from win_process_events wpe JOIN win_hash h USING (path) where action='PROC_CREATE';",
      "interval": 30,
      "platform": "windows",
      "version": "2.9.0",
      "description": "Windows Process Events",
      "value": "Process Events"
    }
  },
  "plgx_event_filters": {
    "win_process_events": {
      "parent_path": {
        "exclude": {
          "values": [
            "C:\\Program Files\\osquery\\extensions\\plgx_win_extension.ext.exe",
            "C:\\Program Files\\SplunkForwarder\\bin\\splunkd.exe",
            "C:\\Windows\\System32\\svchost.exe"
          ]
        }
      }
    }
  }
}
o
Can you share your osquery.flags file?
l
Copy code
--config_plugin=filesystem
--config_path=C:\Program Files\osquery\osquery.conf
--enable_monitor=false
--events_expiry=3600
--events_max=2500
--logger_plugin=filesystem
--logger_path=C:\Program Files\osquery\log
--logger_event_type=true
--database_path=C:\Program Files\osquery\osquery.db
--pidfile=C:\Program Files\osquery\osquery.pid
--disable_watchdog=true
--watchdog_level=-1
--disable_events=false
--enable_windows_events_publisher=true
--enable_windows_events_subscriber=true
--enable_powershell_events_subscriber=true
--enable_powershell_events_publisher=true
--enable_ntfs_event_publisher=true
--enable_ntfs_event_subscriber=true
--windows_event_channels=Microsoft-Windows-Powershell/Operational,Microsoft-Windows-AppLocker/EXE and DLL,Microsoft-Windows-AppLocker/MSI and Script,Microsoft-Windows-AppLocker/Packaged app-Deployment,Microsoft-Windows-AppLocker/Packaged app-Execution
--allow_unsafe
--extensions_autoload=C:\Program Files\osquery\extensions.load
--extensions_interval=10
--extensions_timeout=90
--database_dump=false
--schedule_default_interval=30
--pack_refresh_interval=30
--extensions_require=plgx_win_extension
h
i have just installed osquery 4.9.0 and used these files alongwith latest released extension. the osquery.conf includes the event filters published in https://github.com/polylogyx/osq-ext-bin/blob/master/osquery.conf Stopped osqueryd service and ran the following on command prompt: 
C:\Program Files\osquery>osqueryd\osqueryd.exe -S --flagfile osquery.flags
Filters got applied in registry and events are coming in table queries. can you try these and let us know it worked just to move a step further that osquery+extension is working in your environment.
osquery.confosquery.flagsextensions.load
registry should show the following if filters are successfully applied
and 
exclude
and 
include
keys should have registry value having entries of respective filters specified in conf file
l
ok the config is running and I see the same reg entries as you have above but my keys are empty
I have data and here is what I see when running osqueryd in the foreground
but it still seems to ignore the filter. I added the parent_path for the plgx extension but I am still seeing it logged as a parent path
I am going to let it run for 30 minutes. thank you for the help so far. very appreciated
h
filters will be effective only if they are added in registry successfully. a sample of filters in my registry is as shown in the image. can you attach and share the osquery.conf file itself that you are using? if the osquery.conf has 
"plgx_event_filters":
field, you should not see 
No event_filter found
message in foreground.
l
my filters are in a pack in a separate directory. my osquery.conf contains my pack section with a path to the events pack which contain the queries and filters for the extension. Do the filters HAVE to be in the osquery.conf file or can they reside in a pack.
I will share the conf file shortly
h
filters have to be in osquery.conf only
o
@lvferdi, as @himanshu mentioned can you try with filters in osquery.conf file?
l
I will try them this morning (sorry for the delay, the weekend and life got in the way)
here is my new osquery.conf file
Untitled.txt
running it now and will advise
I now see this filter in the 
win_proc_events
registry entry but socket events are not being filtered. I will check for process events
I had an extra set of parens in my events.conf file. Although it was valid json it made it so the extension was unable to see the filters past the first one (processes). I now see all of the registry entries from the above screenshot with filters in each registry entry. Waiting to see if the socket filter is function correctly now.
o
Great. FWIW, socket filters are not stored in the registry, so if you don't see them, thats fine. Do let know if it all worked out fine
l
so far so good. just playing with other filters now. The only ones giving me issues are ssl_events, but I'm not super worried about those. So far so good with the new config and filtering. Amazing product and something osquery has needed for some time.
❤️ 1
thanks for the help and extension
h
great to know
12 Views
Open in Slack
PreviousNext
Powered by