https://github.com/osquery/osquery logo
Powered by
#eclecticiq-polylogyx-extension
Title
# eclecticiq-polylogyx-extension
l

lvferdi

11/03/2021, 7:23 PM
Has anyone has success in getting include and/or exclude filters to work. No matter what I try I can't seem to filter out anything with exclude or only collect specific things with include. Any help would be appreciated
@OpenPlgx any thoughts
o

OpenPlgx

11/04/2021, 12:01 PM
Hi @lvferdi, Seeing the messages now. Are you having problems only with socket event filters or filters in general? How about default filters..are they working fine?
l

lvferdi

11/04/2021, 12:01 PM
I am not able to get any of the filtering to apply to any of the queries.
The only error I have states 
no event_filter found
o

OpenPlgx

11/04/2021, 12:03 PM
are you running osqueryi/osqueryd? How are the configs refreshed on the agent?
What is your osquery management infrastructure?
l

lvferdi

11/04/2021, 12:05 PM
This is running on a test machine in AWS. It is running using osqueryd. I have the options set in the 
osquery.conf
file and the queries in a query pack
o

OpenPlgx

11/04/2021, 12:07 PM
did you restart the osqueryd after making changes in the osquery.conf?
l

lvferdi

11/04/2021, 12:08 PM
yes, several times. And to confirm it was reading the config I made json errors and upon restart it would advise of the errors in the logs
o

OpenPlgx

11/04/2021, 12:08 PM
ok
l

lvferdi

11/04/2021, 12:08 PM
If it helps I am on osquery version 4.9.0
o

OpenPlgx

11/04/2021, 12:08 PM
and the extension is loaded, right?
l

lvferdi

11/04/2021, 12:09 PM
correct as I get results of the osquery query but none of the filtering of the extension
I can see the table and query 
select * from win_socket_events;
and receive the results, but I don't see the filtering being applied by the extension.
o

OpenPlgx

11/04/2021, 12:10 PM
hmm..weird...can you export the following registry key into a text file and share? HKLM\System\CurrentControlSet\Services\vast
l

lvferdi

11/04/2021, 12:12 PM
one moment
o

OpenPlgx

11/04/2021, 12:13 PM
In the meantime, let me try to repro locally and give you a set of config that works (with a test tool)
Also, is this a regression? Did you try them with the earlier extension and found it to be working then?
l

lvferdi

11/04/2021, 12:20 PM
no this is my first attempt with the extension. I tried the earlier version unsuccessfully but after 2 days the new version was released.
apologies, that has a larger config running on it, which includes socket and process events. I can put the single query single filter config on the machine and export again
o

OpenPlgx

11/04/2021, 12:33 PM
That will help
These filters seem empty
Can you values for "include"/"exclude" keys under each event category?
l

lvferdi

11/04/2021, 1:29 PM
sorry had to run the kids to school. I'll look now
They all say 
value not set
here are the options I have set in my osquery.conf file and the single query and filter I have set in my pack.
Copy code
{
"utc": "true",
"custom_plgx_EnableSSL": "true",
"custom_plgx_EnableAmsiStreamEventData": "true"
},



{
  "version": "1.0",
  "queries": {
    "win_process_events": {
      "query": "select wpe.action, wpe.eid, wpe.pid, wpe.path, wpe.process_guid, wpe.cmdline, wpe.parent_pid, wpe.parent_path, wpe.parent_process_guid, wpe.owner_uid, wpe.utc_time as time, h.md5, h.sha1, h.sha256 from win_process_events wpe JOIN win_hash h USING (path) where action='PROC_CREATE';",
      "interval": 30,
      "platform": "windows",
      "version": "2.9.0",
      "description": "Windows Process Events",
      "value": "Process Events"
    }
  },
  "plgx_event_filters": {
    "win_process_events": {
      "parent_path": {
        "exclude": {
          "values": [
            "C:\\Program Files\\osquery\\extensions\\plgx_win_extension.ext.exe",
            "C:\\Program Files\\SplunkForwarder\\bin\\splunkd.exe",
            "C:\\Windows\\System32\\svchost.exe"
          ]
        }
      }
    }
  }
}
o

OpenPlgx

11/05/2021, 7:07 AM
Can you share your osquery.flags file?
l

lvferdi

11/05/2021, 3:09 PM
Copy code
--config_plugin=filesystem
--config_path=C:\Program Files\osquery\osquery.conf
--enable_monitor=false
--events_expiry=3600
--events_max=2500
--logger_plugin=filesystem
--logger_path=C:\Program Files\osquery\log
--logger_event_type=true
--database_path=C:\Program Files\osquery\osquery.db
--pidfile=C:\Program Files\osquery\osquery.pid
--disable_watchdog=true
--watchdog_level=-1
--disable_events=false
--enable_windows_events_publisher=true
--enable_windows_events_subscriber=true
--enable_powershell_events_subscriber=true
--enable_powershell_events_publisher=true
--enable_ntfs_event_publisher=true
--enable_ntfs_event_subscriber=true
--windows_event_channels=Microsoft-Windows-Powershell/Operational,Microsoft-Windows-AppLocker/EXE and DLL,Microsoft-Windows-AppLocker/MSI and Script,Microsoft-Windows-AppLocker/Packaged app-Deployment,Microsoft-Windows-AppLocker/Packaged app-Execution
--allow_unsafe
--extensions_autoload=C:\Program Files\osquery\extensions.load
--extensions_interval=10
--extensions_timeout=90
--database_dump=false
--schedule_default_interval=30
--pack_refresh_interval=30
--extensions_require=plgx_win_extension
h

himanshu

11/05/2021, 6:05 PM
i have just installed osquery 4.9.0 and used these files alongwith latest released extension. the osquery.conf includes the event filters published in https://github.com/polylogyx/osq-ext-bin/blob/master/osquery.conf Stopped osqueryd service and ran the following on command prompt: 
C:\Program Files\osquery>osqueryd\osqueryd.exe -S --flagfile osquery.flags
Filters got applied in registry and events are coming in table queries. can you try these and let us know it worked just to move a step further that osquery+extension is working in your environment.
osquery.confosquery.flagsextensions.load
registry should show the following if filters are successfully applied
and 
exclude
and 
include
keys should have registry value having entries of respective filters specified in conf file
l

lvferdi

11/05/2021, 7:14 PM
ok the config is running and I see the same reg entries as you have above but my keys are empty
I have data and here is what I see when running osqueryd in the foreground
but it still seems to ignore the filter. I added the parent_path for the plgx extension but I am still seeing it logged as a parent path
I am going to let it run for 30 minutes. thank you for the help so far. very appreciated
h

himanshu

11/07/2021, 7:47 AM
filters will be effective only if they are added in registry successfully. a sample of filters in my registry is as shown in the image. can you attach and share the osquery.conf file itself that you are using? if the osquery.conf has 
"plgx_event_filters":
field, you should not see 
No event_filter found
message in foreground.
l

lvferdi

11/07/2021, 4:25 PM
my filters are in a pack in a separate directory. my osquery.conf contains my pack section with a path to the events pack which contain the queries and filters for the extension. Do the filters HAVE to be in the osquery.conf file or can they reside in a pack.
I will share the conf file shortly
h

himanshu

11/08/2021, 4:17 AM
filters have to be in osquery.conf only
o

OpenPlgx

11/08/2021, 5:36 AM
@lvferdi, as @himanshu mentioned can you try with filters in osquery.conf file?
l

lvferdi

11/08/2021, 12:20 PM
I will try them this morning (sorry for the delay, the weekend and life got in the way)
here is my new osquery.conf file
Untitled.txt
running it now and will advise
I now see this filter in the 
win_proc_events
registry entry but socket events are not being filtered. I will check for process events
I had an extra set of parens in my events.conf file. Although it was valid json it made it so the extension was unable to see the filters past the first one (processes). I now see all of the registry entries from the above screenshot with filters in each registry entry. Waiting to see if the socket filter is function correctly now.
o

OpenPlgx

11/08/2021, 5:15 PM
Great. FWIW, socket filters are not stored in the registry, so if you don't see them, thats fine. Do let know if it all worked out fine
l

lvferdi

11/08/2021, 5:34 PM
so far so good. just playing with other filters now. The only ones giving me issues are ssl_events, but I'm not super worried about those. So far so good with the new config and filtering. Amazing product and something osquery has needed for some time.
❤️ 1
thanks for the help and extension
h

himanshu

11/08/2021, 5:36 PM
great to know
11 Views
Powered by