Ahmed
11/09/2020, 1:20 PMMike Myers
11/09/2020, 8:58 PMAhmed
11/10/2020, 9:10 AMapt-get update -y
apt-get install git curl cmake python3 git python3 bison flex make wget -y
pip3 install timeout_decorator thrift==0.11.0 osquery pexpect==3.3
apt install --no-install-recommends rpm binutils -y
#wget <https://github.com/osquery/osquery-toolchain/releases/download/1.1.0/osquery-toolchain-1.1.0-x86_64.tar.xz>
tar xvf osquery-toolchain-1.1.0-x86_64.tar.xz -C /usr/local
git clone <https://github.com/osquery/osquery>
git clone --recurse-submodules <https://github.com/trailofbits/osquery-extensions.git>
#wget <https://github.com/Kitware/CMake/releases/download/v3.14.6/cmake-3.14.6-Linux-x86_64.tar.gz>
tar xvf cmake-3.14.6-Linux-x86_64.tar.gz -C /usr/local --strip 1
export PATH=/usr/local/bin:$PATH
cd osquery
ln -s ../../osquery-extensions ./external/extension_trailofbits
mkdir build; cd build
cmake -DOSQUERY_TOOLCHAIN_SYSROOT=/usr/local/osquery-toolchain ..
cmake --build . -j30 # where 30 is the number of parallel build jobs
Mike Myers
11/10/2020, 4:39 PMStefano Bonicatti
11/10/2020, 4:42 PMMike Myers
11/10/2020, 4:52 PM-j30
could be increasing your memory overhead too much and one of the parallelized jobs is failingAhmed
11/10/2020, 4:57 PM-j10
and i have enough ram, but i will take your advise and test with much lower number. thanks a lot both of youStefano Bonicatti
11/10/2020, 4:58 PMMike Myers
11/10/2020, 5:19 PMAhmed
11/11/2020, 8:21 AM-j4
but still getting killed.Stefano Bonicatti
11/11/2020, 10:02 AMfree -m
inside the Docker image that you have all 16GB?
Here https://docs.docker.com/docker-for-mac/#:~:text=Memory%3A%20By%20default%2C%20Docker%20Desktop,swap%20file%20size%20as%20needed. it seems like the default is to use only 2GB from the Host.Ahmed
11/11/2020, 10:03 AM-j4
and it did finish compiling. but when i tried to run the
osqueryi --extension osquery/build/external/trailofbits_osquery_extensions.ext
i dont see the tables mentioned, so trying to figure out why 😄iptables_ext
ip6tables_ext
and in side the build i see these files. i’m only interested to get the extension for dns.
thanks a lot for taking the time to look at this.dns_events
doesnot work when i run the extension using this way
osqueryi --allow_unsafe --extension osquery/build/external/extension_trailofbits/trailofbits_osquery_extensions.ext
and i get this message when quitting osquery
Thrift: Wed Nov 11 07:15:12 2020 TSocket::open() connect() <Host: Port: 0>No such file or directory
but it works with this way and generate info/error message and doenot get killed when i quit osquery i have to explicitly run kill
command against it.
osqueryi --allow_unsafe --extension osquery/build/external/extension_trailofbits/extensions/network_monitor/network_monitor.ext
and it keep generating this
I1111 07:16:53.047338 27998 extensions.cpp:298] Extension watcher ending: osquery core has gone away
I1111 07:16:53.047338 27998 extensions.cpp:298] Extension watcher ending: osquery core has gone away
osquery_src/network_monitor.ext --socket /home/user/.osquery/shell.em --timeout 3 --interval 3
Mike Myers
11/13/2020, 2:17 AMnetwork_monitor
produces no results for a query to dns_events
, but it produces results when run as a standard user? @alessandrogario do you know if this is expected behavior? It seems to me like if anything, it would be the opposite.i only see these tables i guessDid you set the environment variable to only build certain extensions? If you built them all, you should see many additional tables when loading the extension. https://github.com/trailofbits/osquery-extensions#specifying-the-extensions-to-be-built
dns_events
is not present. I can't see the reason for this, so I'll open another issue.addOsqueryExtension
rather than addOsqueryExtensionEx
although that might've been intentional since we had categorized this particular extension as 'experimental'Ahmed
11/13/2020, 3:04 PMExtension watcher ending: osquery core has gone away
and when its closed i never exists with osquery and keeps outputing to termial. and i have tested this with osqueryd and systemd i was getting same results.
Extension watcher failed to get extensions: Extension socket not available: /home/user/.osquery/shell.em
thanks a lot for those issues.Mike Myers
11/13/2020, 5:50 PMnetwork_monitor.ext
should be fixed now, thanks againAhmed
11/17/2020, 5:03 PMosqueryi --disable_extensions=false --extension network_monitor.ext
Using a virtual database. Need help, type '.help'
osquery> I1118 09:21:42.418730 954 pcapreaderservice.cpp:238] Listening on the following IPv4 addresses: 10.235.173.65/255.255.252.0
I1118 09:21:42.418823 954 pcapreaderservice.cpp:250] Listening on the following IPv6 addresses: 0:0:fe80::250:56ff/::ffff:ffff:ffff:ffff:0:0
I1118 09:21:45.404930 947 extensions.cpp:298] Extension watcher ending: osquery core has gone away
terminating
Thrift: Wed Nov 18 09:21:48 2020 TSocket::open() connect() <Host: Port: 0>Connection refused
and when i ran it with normal user, it worked very well and when terminated osqueryi it terminated well and printed this
terminating
Thrift: Wed Nov 18 09:24:41 2020 TSocket::open() connect() <Host: Port: 0>Connection refused
i got the same root issue with osqeuryd (i guess because its running with root)
Nov 18 09:34:23 pmlkqt2 osqueryd[2574]: I1118 09:34:23.135928 2592 extensions.cpp:298] Extension watcher ending: osquery core has gone away
Nov 18 09:34:24 pmlkqt2 osqueryd[2574]: terminating
Nov 18 09:34:26 pmlkqt2 osqueryd[2574]: I1118 09:34:26.137401 2585 extensions.cpp:341] Extension UUID 38367 has gone away
is there anything i should note here. thanks a lot.Mike Myers
11/18/2020, 2:46 PMalessandrogario
11/19/2020, 5:32 PMsudo osqueryd --verbose --disable_extensions=false --extensions_socket=/var/osquery/extensions_socket
2. start the extension sudo ./network_monitor.ext --verbose --extensions_socket=/var/osquery/extensions_socket
Ahmed
11/23/2020, 1:39 PMMike Myers
11/23/2020, 5:40 PM