Hi everyone Can someone advise me what can be wrong with thi

Question

Hi everyone Can someone advise me what can be wrong with this queue is not detecting nc netcat connections to external IP addresses My goal is to get IP and SHA256 of a ` proc $ PID exe ` belongs to corresponding process ```SELECT DISTINCT processes path SELECT sha256 FROM hash WHERE path = concat proc socket pid exe AS sha256 socket remote address socket remote port FROM bpf socket events socket LEFT JOIN bpf process events processes ON socket pid = processes pid WHERE socket remote port NOT IN 0 443 993 4172 4195 AND socket remote address NOT LIKE 127 0 ```

seph · Answer

Is the `nc` process short lived

slevchenko · Answer

For tests I m setting up listener on remote machine `nc l 8983`

slevchenko · Answer

And then I m trying to connect from local one `nc v 8983 <http XXX XXX XXX XXX|XXX XXX XXX XXX>`

seph · Answer

Sounds like it s not short lived then

seph · Answer

I m not really sure but I m asking more details

slevchenko · Answer

Final goal is to detect malware on early bootstrap phases when potential malware tries to contact C2 point or download site

slevchenko · Answer

that s why I m trying to simulate this behavior with nc get sha256 from process procfs handle and ipaddress

slevchenko · Answer

If there s any better way to do this I would gladly use it

slevchenko · Answer

< seph> Hi Is there any reason why query can work in `osqueryi` but not from config Query always rerturns non empty result when executed interactively but never appears in results file if executed by daemon

seph · Answer

Not that I can think of

seph · Answer

The thing I was asking about short lived earlier if the query executes when nc isn t running obviously you won t see it I don t know if you d want to look at the process events table instead

slevchenko · Answer

I m trying to write a logger extension which reacts on presence of certain columns in query string If logger finds sha256 or IP it sends them to certain rest API

seph · Answer

Sounds interesting but unrelated to the thing you re asking about

slevchenko · Answer

As of now I m seeking a way to debug my extension to understand what extension actually gets < seph> Do you know if I can somehow check what comes from osquery to my extension

seph · Answer

Depending you could also switch off the query name

seph · Answer

mmmm I m not sure what simple debug routes there are I don t know that you can use a logger from osqueryi When I do this I add a bunch of printf or logging to my extension and debug via that

slevchenko · Answer

Sorry if I ve consfused you It s directly related So I ve wrote an extension which is being registered as a logger once it up I m trying to establish test connections to external host Amazon AWS machine and then checking if my extension detects it The result is negative I can t see nc connection not in results file nor in osqueryi

slevchenko · Answer

But my query detects bunch of ther things like browser connections and messenger connections via `osqueryi` while extension registered such test connection only once or twice

seph · Answer

You started this discussion how you re not seeing results in your logs Are you sending those logs to a file That seems weird I d debug the query and the behavior using plain osquery Separately you re talking about a logging extension that does some clever routing This is pretty cool But feels separate to debug Does your query work Does it produce the logs you expect locally And does your extension work as expected These should not be coupled

slevchenko · Answer

That s why I m searching for ways to hook up into data osquery lt gt extension interaction and check what s actually happens

seph · Answer

1 You can have osquery log to the filesystem and your extension 2 Use printf or other logging from the extension I find it easiest to log to a file

slevchenko · Answer

Yes I did `logger plugin= threat logger filesystem ` and it works

slevchenko · Answer

My issue is my query detects bunch of my laptop connections but not my test connection

slevchenko · Answer

while extension detects nothing at all

slevchenko · Answer

completelly nothing

seph · Answer

Which should be totally unrelated to your extension right

seph · Answer

And thus question about process length and whether you should be using the evented tables

slevchenko · Answer

Yes So first of all I m trying to understand what s wrong with query or my test approach Maybe nc is wrong tool for this test

seph · Answer

Though I guess `bpf socket events` is evented

seph · Answer

nc should be fine I m not sure Query doesn t look wrong

slevchenko · Answer

Since malware tends to hide itself ASAP I hoped to detect short living connections at bootstrap stages

slevchenko · Answer

Was I wrong assuming that s possible

slevchenko · Answer

That s what puzzles me too query registers DNS requests NTP requests which also have to be pretty short living

seph · Answer

I think you re correct you re using the bpf events So I m puzzled why it s not working

slevchenko · Answer

More than that I ve conducted like a dozen of tests and twice my test connection was registered which blew my mind

slevchenko · Answer

I even have this query result in `osqueryd results log`

seph · Answer

A couple of places to investigate If you make it simpler just the `bpf socket events` is it reliable I wonder if that subselect is being weird

slevchenko · Answer

```1130 2022 10 19 11 48 25 string name pack osquery process ioc process unknown outbound connection hostIdentifier HP Spectre x360 Convertible 13 aw0xxx calendarTime Wed Oct 19 08 48 25 2022 UTC unixTime 1666169305 epoch 0 counter 0 numerics false columns path remote address <http XXX XXX XXX XXX|XXX XXX XXX XXX> remote port 53 sha256 1a2b9a41d835898601a4778921c2504ab2e74f1cd76a4c48fcfda43 REDATED action added 2022 10 19 11 48 27 INFO VirusTotal client API sucessfully created with daily request limit 500 2022 10 19 11 48 27 ERROR Unable to canonicalize url 2022 10 19 11 49 00 deregistering extension write unix gt var osquery osquery em write broken pipe```

slevchenko · Answer

This is actual case when it worked as expected

seph · Answer

Those last couple lines seem notable ```2022 10 19 11 48 27 ERROR Unable to canonicalize url 2022 10 19 11 49 00 deregistering extension write unix gt var osquery osquery em write broken pipe```

slevchenko · Answer

< seph> Yes they are but I don t know how to interpret them

seph · Answer

Sounds like maybe your extension crashed

slevchenko · Answer

In particular I don t know how to investigate which url it tried to canonicalize or why socket was closed

slevchenko · Answer

maybe so Are there any examples of how multi result json strings look like

seph · Answer

I can t dig too much into this I would recommend adding debugging to your extension Via whatever means you like

slevchenko · Answer

Are there any examples of multi row json strings I mean cases when query returns more then one result

slevchenko · Answer

Anyway thanks for your time

seph · Answer

I d dig through the go code for the object definitions Or add a printf to the extension

slevchenko · Answer

thank you very much