osquery #extensions

hubert dulay

06/10/2022, 12:10 PM

Hi. Running osqueryi on macos using a python extension to send to apache pulsar. Getting this error below. Can anyone help? Thanks

W0609 160653.907390 13324288 watcher.cpp:624] Extension respawning too quickly: ./simple.ext

I0609 160653.908399 13324288 watcher.cpp:658] Created and monitoring extension child (95012): ./simple.ext

I tried both

osqueryi version 4.9.0

and

5.2.3

hubert dulay

06/10/2022, 12:43 PM

Also tried ubuntu with the same error. Thanks

Praveen Kumar

09/09/2022, 12:40 PM

I am facing the issue: Extension socket not available: /var/osquery/osquery.em when try to execute below cpp program!.. int main(int argc, char* argv[]) { osquery::Initializer runner(argc, argv, ToolType::EXTENSION); auto status = startExtension("example", "0.0.1"); if (!status.ok()) { LOG(ERROR) << status.getMessage(); runner.requestShutdown(status.getCode()); }} requesting a help here

seph

09/09/2022, 1:57 PM

Well, is osquery running?

seph

09/09/2022, 1:57 PM

Is it configured to have a socket there?

seph

09/09/2022, 1:57 PM

Do you have permission to read it?

Praveen Kumar

09/09/2022, 7:04 PM

s, it is running in the xcode

Praveen Kumar

09/09/2022, 7:05 PM

how to configure to have socket name there ?

Praveen Kumar

09/09/2022, 7:05 PM

s, I have a permission to read it

seph

09/09/2022, 7:44 PM

I don’t know much about xcode.

seph

09/09/2022, 7:45 PM

What socket is osquery running with? You’d need to pass that to the extension

Daniel Bretón Suárez

09/15/2022, 3:59 PM

I want to thank the people from Trail Of Bits for publishing https://github.com/trailofbits/osquery-extensions. It has been very useful to understand how to build an extension!!!

Daniel Bretón Suárez

10/06/2022, 7:14 AM

I'm trying to build a coverage report for an extension using the

gcovr

tool (on which I have very little knowledge).

.gcno

and

.gcda

files are generated but test coverage is always 0% in all the sources. At the moment I'm blocked, all the guides I've read say it should work. Maybe someone can point me in the right direction? I think it is related to the way extensions are built because sources are in a different folder than binaries. However,

gcovr

has a

--root

option on which source files can be specified so it might not be the problem. This are the steps I'm performing on Ubuntu

Copy code

cd osquery
ln -s /home/danielbreton/workspace/osquery-extension-hello/ external/extension_hello
cd build
cmake -DOSQUERY_TOOLCHAIN_SYSROOT=/usr/local/osquery-toolchain -DOSQUERY_BUILD_TESTS=ON -DENABLE_COVERAGE=true ..
cmake --build . -j$(nproc) --target hello_my_friend_extension_test
./external/extension_hello/hello_my_friend_extension_test
gcovr --xml-pretty --exclude-unreachable-branches --print-summary -o coverage.xml --root /home/danielbreton/workspace/osquery-extension-hello/

And this is the complete

CMakeLists.txt

for the extension

Copy code

project("hello_my_friend_extension")

addOsqueryExtension(
  "${PROJECT_NAME}"
  hello.cpp
  main.cpp
)

set(common_test_files
  hello.cpp
)

add_executable(
  "${PROJECT_NAME}_test"
  EXCLUDE_FROM_ALL
  ${common_test_files}
  test.cpp
)

if(ENABLE_COVERAGE)
  message("Enabling coverage")
  # set compiler flags
  set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -O0 -fprofile-arcs -ftest-coverage")
  set(CMAKE_CXX_FLAGS " ${CMAKE_CXX_FLAGS} -O0 -fprofile-arcs -ftest-coverage")
  # find required tools
  find_program(GCOVR gcovr REQUIRED)
endif()

target_link_libraries("${PROJECT_NAME}_test" PRIVATE
  thirdparty_googletest
  osquery_sdk_pluginsdk
  osquery_extensions_implthrift
)

if(ENABLE_COVERAGE)
  add_link_options("--coverage")
endif()

Project tree

Copy code

danielbreton@2022-EMEA-0022 ~/workspace/osquery-extension-hello (feature/UA-1101)$ tree
.
├── CMakeLists.txt
├── hello.cpp
├── hello.h
├── main.cpp
├── README.md
└── test.cpp

Daniel Bretón Suárez

10/06/2022, 7:17 AM

⬆️ sorry for a message so long

Mike Myers

10/06/2022, 7:20 PM

this is a possible CMake question for @alessandrogario (see above)

Daniel Bretón Suárez

11/08/2022, 1:27 PM

Hi! I'm trying to build a

config_parser

in a C++ extension. I'm using

REGISTER_EXTERNAL

macro and I see the debug message at

registry_factory.cpp

I1108 12:56:08.491199 231789 registry_factory.cpp:107] Extension 499 registered config_parser plugin devo_params

However, the update function is never called. If I add some debug messages to

osquery/config/config.cpp

and print the list of all config_parser modules it handles, I can't see it. However, if I add the same function at the extension code, it exists! Any ideas why this could be happening?

Copy code

void printAll()
{
  auto plugins = osquery::RegistryFactory::get().plugins("config_parser");
  for (auto & p : plugins) {
    printf("registered parser %s\n", p.first.c_str());
  }
}

Rupert

11/10/2022, 3:16 PM

I'm new to osquery & looking for some troubleshooting advice I built & installed this extension https://github.com/aquasecurity/kube-query & it works fine when used from the repl:

Copy code

KUBECONFIG="${HOME}/.kube/config" osqueryi --extension "${HOME}/.local/bin/kube-query"
osquery> -- this works ok
osquery> select * from kubernetes_pods limit 1;

when i pass in the query from the cli none of the

kubernetes_

tables show up & I can't query on them;

Copy code

# no kubernetes tables in this list
KUBECONFIG="${HOME}/.kube/config" osqueryi --extension "${HOME}/.local/bin/kube-query" '.tables'

# gives Error: no such table: kubernetes_pods
KUBECONFIG="${HOME}/.kube/config" osqueryi --extension "${HOME}/.local/bin/kube-query" 'select * from kubernetes_pods limit 1'

any advice appreciated!

Daniel Bretón Suárez

12/01/2022, 11:48 AM

I'm trying to send a post message to a server from an extension using

http_client

, somehow like this:

Copy code

osquery::http::Client client;
client.setOptions(getOptions()); /* Local function to fill options */
<http://client.post|client.post>(request, params); /* crashes on Windows */

Works fine on Ubuntu, but it crashes on Windows. https://github.com/osquery/osquery/blob/a5bc1a33f2ad5eecb1a814d472c00a30c410d394/osquery/remote/http_client.cpp#L119 There's also a comment that says there is a leak, Is there any way to avoid this?

Aayush Jain

12/13/2022, 11:05 AM

👋 Hello, team! I am new to osquery. Tried osquery python extension to execute osqueryi commands and stuck on this error.

Could not connect to any of ['/tmp/pyosqsockndnbir6z']

My code was :

Copy code

import osquery
 instance = osquery.SpawnInstance()
 instance.open()
 result = instance.client.query("SELECT interface FROM interface_details;")

Is the correct way or I am doing something wrong ?

Daniel Bretón Suárez

12/21/2022, 2:25 PM

Hi! Is the SQL plugin available to be called from extensions? I get responses from

static QueryData selectAllFrom(const std::string& table);

but not from

explicit SQL(const std::string& query, bool use_cache = false);

Stefano Bonicatti

12/21/2022, 2:27 PM

I don’t recall the difference on the top of my head but one of the two side steps the registry I think and tries to access the database plugin locally

Stefano Bonicatti

12/21/2022, 2:29 PM

Copy code

QueryData SQL::selectAllFrom(const std::string& table) {
  PluginResponse response;
  Registry::call("table", table, {{"action", "generate"}}, response);
  return response;
}

Daniel Bretón Suárez

12/21/2022, 2:31 PM

SQL calls a different registry

Copy code

auto status = Registry::call(
      "sql", "sql", {{"action", "columns"}, {"query", q}}, response);

Daniel Bretón Suárez

12/21/2022, 2:33 PM

The thing is, using the table API I'm not able to perform unions (or at least I don't know how)

Daniel Bretón Suárez

12/22/2022, 4:25 PM

Hey @Stefano Bonicatti, I've found out I must attach the table previously to perform the SQL command. Something like this:

Copy code

auto st = osquery::Registry::call("sql", "sql",
                                    {{"action", "attach"}, 
                                    {"table", "system_info"}}, response);

And then I can use the SQL engine with that table as usual

Copy code

osquery::SQL sql("SELECT * FROM system_info;");

So, it seems like not all tables are loaded by default into extension registry

Stefano Bonicatti

12/22/2022, 4:29 PM

Ah so I was misremembering then. And yeah, afaik this is because attaching tables happens when a table is registered, and table registering happens when the code of the table is linked against the osquery binary + static init time, all of which, for the majority of tables, only happens in the osquery binary.

Gilad Reich

02/02/2023, 5:42 PM

Hello! Curious question regarding this: https://github.com/osquery/osquery/blob/5.7.0/osquery/extensions/extensions.cpp#L49-L55 What was the rational behind enforcing Osquery extensions to end with

.ext

extension?

Adrian Junge

02/07/2023, 4:32 PM

Hello everyone, currently I'm trying to get osqueryd running with the custom table from https://github.com/osquery/osquery-python. I start osqueryd with

sudo osqueryd --ephemeral --disable_logging --disable_database --extensions_socket /home/adrian/.osquery/osqueryd.sock --config_path=./osquery/osquery.conf --disable_extensions false

and I start the table with

python3 test.py --socket /home/adrian/.osquery/osqueryd.sock

. But sadly an error "Could not connect to any of ['/home/adrian/.osquery/osqueryd.sock']" is raised every time in my logs. What am I doing wrong? Or do I have to bind the table in a different way?

Abhijit

02/21/2023, 5:44 AM

Seeing this error in a logger plugin: "The minimum events expiration timeout for process_file_events has been adjusted: 60" What exactly does it mean? Is it related to events_expiry flag?

Adrian Junge

02/22/2023, 12:59 PM

Hello everyone, I'm currently trying to add a custom osquery table with osquery-python to get the information of how many installed packages are outdated. Are there some better solutions than just simply using "os.system" or "subprocess.run" for similar tables?