Hey Folks trying to build network monitor extension from tra

Question

Hey Folks trying to build network monitor extension from trailofbits followed the build guide for osquery and their github page keep getting fatel errors i m building in a fresh debian latest docker image any advise i m only interested in building the extensions and not osquery itself thanks a lot

Mike Myers · Answer

The extensions do require all of osquery to be built along with them so they can t be compiled separately

Ahmed · Answer

ok that what i followed actually the compilation steps in osquery docs and their github which gave me those errors and keeps failing any advise help how to get a clean compilation thanks a lot

Ahmed · Answer

< Mike Myers> getting same failure on different debian versions and ubuntu versions and this is the command i m following ```apt get update y apt get install git curl cmake python3 git python3 bison flex make wget y pip3 install timeout decorator thrift==0 11 0 osquery pexpect==3 3 apt install no install recommends rpm binutils y wget tar xvf osquery toolchain 1 1 0 x86 64 tar xz C usr local git clone git clone recurse submodules wget tar xvf cmake 3 14 6 Linux x86 64 tar gz C usr local strip 1 export PATH= usr local bin $PATH cd osquery ln s osquery extensions external extension trailofbits mkdir build cd build cmake DOSQUERY TOOLCHAIN SYSROOT= usr local osquery toolchain cmake build j30 where 30 is the number of parallel build jobs```

Mike Myers · Answer

Yea those are weird errors but when LLVM fails during compilation sometimes it s because your build system doesn t have enough RAM

Mike Myers · Answer

< Stefano Bonicatti> what s the recommended RAM for build systems Like 6GB

Stefano Bonicatti · Answer

Depends on how many jobs you run ~4GB are required to build one of the C++ files from the AWS SDK and that s the only one really that has that huge requirement 6GB might be a safe bet for maybe up to 4 jobs

Stefano Bonicatti · Answer

Sometimes I run with 8GBs on my Linux VM Ubuntu and that s fine for 10 jobs but not always also depends on what else it was doing For a 30 jobs build I would give it at least 16GBs

Stefano Bonicatti · Answer

But again those are somewhat estimates the growth is non linear and it highly depends on what each job is doing and how many can run really parallel so a bit of testing is required ^^

Mike Myers · Answer

yea using ` j30` could be increasing your memory overhead too much and one of the parallelized jobs is failing

Ahmed · Answer

it was failing before with ` j10` and i have enough ram but i will take your advise and test with much lower number thanks a lot both of you

Stefano Bonicatti · Answer

< Ahmed> If the error is that clang gets killed it s most likely due to memory issues How much memory do you have

Mike Myers · Answer

Yea the OpenSSL library in particular is prone to failing in low memory situations in my experience

Ahmed · Answer

i have 16G on my mac which i m running the docker on and now i ran the same build with ` j4` but still getting killed

Stefano Bonicatti · Answer

< Ahmed> could you please check via `free m` inside the Docker image that you have all 16GB Here it seems like the default is to use only 2GB from the Host

Ahmed · Answer

i just ran it on another system with ` j4` and it did finish compiling but when i tried to run the ```osqueryi extension osquery build external trailofbits osquery extensions ext``` i dont see the tables mentioned so trying to figure out why smile

Ahmed · Answer

i only see these tables i guess ```iptables ext ip6tables ext``` and in side the build i see these files i m only interested to get the extension for dns thanks a lot for taking the time to look at this

Ahmed · Answer

the `dns events` doesnot work when i run the extension using this way ```osqueryi allow unsafe extension osquery build external extension trailofbits trailofbits osquery extensions ext``` and i get this message when quitting osquery ```Thrift Wed Nov 11 07 15 12 2020 TSocket open connect lt Host Port 0 gt No such file or directory``` but it works with this way and generate info error message and doenot get killed when i quit osquery i have to explicitly run `kill` command against it ```osqueryi allow unsafe extension osquery build external extension trailofbits extensions network monitor network monitor ext``` and it keep generating this ```I1111 07 16 53 047338 27998 extensions cpp 298 Extension watcher ending osquery core has gone away```

Ahmed · Answer

< Stefano Bonicatti> any advise why the extension doesnot terminate after quitting osquery and keeps running and outputing to the screen ```I1111 07 16 53 047338 27998 extensions cpp 298 Extension watcher ending osquery core has gone away```

Ahmed · Answer

when ran with normal user it generated the expected output but i still see its process running after quitting osquery ```osquery src network monitor ext socket home user osquery shell em timeout 3 interval 3```

Mike Myers · Answer

It sounds like there might be a bug if the extension does not exit when it should I ve opened an issue to track this <https github com trailofbits osquery extensions issues 62>

Mike Myers · Answer

You are saying that when run in an Administrator shell `network monitor` produces no results for a query to `dns events` but it produces results when run as a standard user < alessandrogario> do you know if this is expected behavior It seems to me like if anything it would be the opposite

Mike Myers · Answer

gt i only see these tables i guess Did you set the environment variable to only build certain extensions If you built them all you should see many additional tables when loading the extension <https github com trailofbits osquery extensions specifying the extensions to be built>

Mike Myers · Answer

Nevermind I see the same thing When loading the bundled extension `dns events` is not present I can t see the reason for this so I ll open another issue

Mike Myers · Answer

It s because <https github com trailofbits osquery extensions blob master network monitor CMakeLists txt L36> uses `addOsqueryExtension` rather than `addOsqueryExtensionEx` although that might ve been intentional since we had categorized this particular extension as experimental

Mike Myers · Answer

Ahmed · Answer

sorry for late reply and thanks a lot for your response i tested again with both normal user and sudo the extension works but it keep generating this error ```Extension watcher ending osquery core has gone away``` and when its closed i never exists with osquery and keeps outputing to termial and i have tested this with osqueryd and systemd i was getting same results ```Extension watcher failed to get extensions Extension socket not available home user osquery shell em``` thanks a lot for those issues

Mike Myers · Answer

you re welcome and thanks for bringing these issues to our attention

Mike Myers · Answer

Hi Ahmed the shutdown for `network monitor ext` should be fixed now thanks again

Ahmed · Answer

really appreciate all the efforts you did and the time taken thanks a lot

Ahmed · Answer

sorry minor question when i ran it with root account i got this ```osqueryi disable extensions=false extension network monitor ext Using a virtual database Need help type help osquery gt I1118 09 21 42 418730 954 pcapreaderservice cpp 238 Listening on the following IPv4 addresses 10 235 173 65 255 255 252 0 I1118 09 21 42 418823 954 pcapreaderservice cpp 250 Listening on the following IPv6 addresses 0 0 fe80 250 56ff ffff ffff ffff ffff 0 0 I1118 09 21 45 404930 947 extensions cpp 298 Extension watcher ending osquery core has gone away terminating Thrift Wed Nov 18 09 21 48 2020 TSocket open connect lt Host Port 0 gt Connection refused``` and when i ran it with normal user it worked very well and when terminated osqueryi it terminated well and printed this ```terminating Thrift Wed Nov 18 09 24 41 2020 TSocket open connect lt Host Port 0 gt Connection refused``` i got the same root issue with osqeuryd i guess because its running with root ```Nov 18 09 34 23 pmlkqt2 osqueryd 2574 I1118 09 34 23 135928 2592 extensions cpp 298 Extension watcher ending osquery core has gone away Nov 18 09 34 24 pmlkqt2 osqueryd 2574 terminating Nov 18 09 34 26 pmlkqt2 osqueryd 2574 I1118 09 34 26 137401 2585 extensions cpp 341 Extension UUID 38367 has gone away``` is there anything i should note here thanks a lot

Mike Myers · Answer

This seems like a Thrift IPC problem and if it only happens when running as root then I wonder if it s because the network monitor extension drops its privileges and loses rights to the IPC Does that sound like a valid theory < alessandrogario>

alessandrogario · Answer

One thing you could try to do 1 start osquery as root `sudo osqueryd verbose disable extensions=false extensions socket= var osquery extensions socket` 2 start the extension `sudo network monitor ext verbose extensions socket= var osquery extensions socket`

alessandrogario · Answer

As Mike said it could be caused by the ext dropping privileges

Ahmed · Answer

tried to use that but the daemon kept on failing thanks a lot your efforts if there re anything else i could do to get that logging would be appreciated

Mike Myers · Answer

when this happens it would maybe be useful to know the access controls on the Thrift file socket I bet it exists but has permissions that deny it from being opened Just my theory