https://github.com/osquery/osquery logo
Powered by
Title
y

Yves Dolce

01/27/2020, 9:44 PM
Can a Windows process run OSQUERY queries using (I guess) the ThriftAPI? If so, is it documented? Thanks. (BTW, the Thrift API https://github.com/osquery/osquery/blob/master/osquery.thrift link on https://osquery.readthedocs.io/en/stable/development/osquery-sdk/#thrift-api takes you to a github 404)
a

alessandrogario

01/28/2020, 12:30 AM
you can probably use selectAllFrom (look in the osquery source code). You can also call the registry directly, see here: https://github.com/zeek/zeek-agent/blob/master/components/zeekosqueryinterface/src/osquerytableplugin.cpp#L43
t

theopolis

01/28/2020, 1:20 AM
Do you mind creating a GitHub issue for the broken link?
y

Yves Dolce

01/28/2020, 6:51 PM
Thanks @alessandrogario. I have a hard time finding such an example that would target Windows. Do you know any?
a

alessandrogario

01/28/2020, 7:13 PM
There are no Windows specifics, just link the source folder under external and it should work
you can follow the build guide in the readme to setup the environment
y

Yves Dolce

01/28/2020, 7:25 PM
I was hoping for something that would not have involved the osquery tree (symlink'ing is still bringing the code in the tree). One that could be built with a `PS1`/`bat` or a 
vcxproj
, or even a 
CMakeFile
, but completely outside osquery. One that would reference its own `boost`/`rapidjson`/`googletest`/`sqllite`/`gflags`/... Am I making sense?
a

alessandrogario

01/28/2020, 7:26 PM
it is not supported, so it's uncharted territory
y

Yves Dolce

01/28/2020, 7:26 PM
Understood but are you aware of anyone that has done so w/ osquery 4.x?
a

alessandrogario

01/28/2020, 7:27 PM
no, as far as I know
refactoring the SDK to make it standalone is in wishlist but not currently planned
y

Yves Dolce

01/28/2020, 7:28 PM
Thanks for your patience. Buena serata!
t

theopolis

01/28/2020, 10:07 PM
I might not have all the context for what you are trying to do @Yves Dolce, but there are may ways a process on Windows can execute queries using the Thrift API within modifying osquery source code. What language is your Windows process/program written in? C/C++?
y

Yves Dolce

01/28/2020, 10:07 PM
Yes.
To be honest with you, I would have expected an SDK that included headers and libraries and possibly a couple of tools required to build ones executable. Not the current documented way.
a

alessandrogario

01/28/2020, 10:12 PM
There are many issues that the current implementation solves that may not be apparent from the outside
portability is the most important one, as it's extremely hard to build once and deploy everywhere when using Linux
and that not only has to do with what you build but also what you link when generating a binary
i.e.: you can't just link against any boost library and expect it to work on other distributions that are different than the one used to build
y

Yves Dolce

01/28/2020, 10:13 PM
Agreed. But you can build the BOOST libs for windows and thus do it, can't you?
a

alessandrogario

01/28/2020, 10:14 PM
the current way solves this and allow you to just focus on the extension code rather than forcing to think about how to build a toolchain that can do that, and then rebuilt all your dependencies with it
you can't really, as you are linking against osquery libraries
that are already importing boost
you have to use the same boost used to build osquery (as long as you are linking against osquery libs)
and that is true not just for boost but for any other library
CRT too, you don't want to mix different CRT libraries in the same process
y

Yves Dolce

01/28/2020, 10:15 PM
THat was just an example of how an open source project (i.e. BOOST) solve those issues.
a

alessandrogario

01/28/2020, 10:16 PM
Boost doesn't solve the issue, and actually make it worse
they are using their own build system called B2 that does not have good support for external toolchains
y

Yves Dolce

01/28/2020, 10:16 PM
Those are the positive sides to the current solution. But the negative side is that I can't build an extension if I don't have the whole osquery enchilada built at the same time on my build machine.
a

alessandrogario

01/28/2020, 10:16 PM
and we basically had to rewrite it in CMake
y

Yves Dolce

01/28/2020, 10:17 PM
What I'm saying is that somehow, you can get os and version specific boost libs and use them when building your project.
a

alessandrogario

01/28/2020, 10:18 PM
but that will not work and potentially cause issue
when linking the osquery SDK, the binary should link the same exact boost used to build osquery
and that can't be downloaded on the boost website
it doesn't just have to be the same exact version, it has to be same lib files
because compilation flags, settings, definition, compiler used, CRT used will greatly vary the lib files
even if the boost version is the same
the code written in your extension (compiled with a certain version of Boost/CRT/etc) can pass objects to osquery code (that may have been compiled with a different Boost)
It is possible to write a brand new SDK that is entirely based on just the thrift protocol specification (like the Go extension SDK). This would have 0 dependencies on osquery and could work with any library of your choice.
y

Yves Dolce

01/28/2020, 10:24 PM
Yes. But those issue are not specific to osquery. Windows has it and provide C/COM SDKs, all the OSes have that issue.
a

alessandrogario

01/28/2020, 10:25 PM
Going down this path sounds cool but unlike Go it will be impossible to redistribute binaries without a proper setup
most projects will attempt to implement reproducible builds, and using system libraries (i.e. Boost from the Ubuntu repository) or downloaded ones (from the Boost website) is never going to work
y

Yves Dolce

01/28/2020, 10:26 PM
I don't even want to try that 🙂 I'd rather listen to your wise advices and do the supported thing!
a

alessandrogario

01/28/2020, 10:27 PM
We currently do not have that issue, since we have reproducible (save for a small chunk related to glibc, sadly) builds
(and you can check this with ldd -d osqueryd)
or on WIndows, you can use something like the CFF Explorer
to look at the import table
(or Dependency Walker)
But I feel your pain, I'd like to be able to build with a standalone SDK
a standalone SDK could also be shipped with the osquery package
we want that (separate SDK that can be built with the custom toolchain so we can redistribute binaries), but unless someone sponsors that work it's hard to prioritize it
since it's not really causing major issues now compared to other high value targets (like better container introspection)
y

Yves Dolce

01/28/2020, 10:30 PM
And I understand the osquery project point of view.
This was a very useful conversation. Thank you so much @alessandrogario.
a

alessandrogario

01/28/2020, 10:32 PM
Sample use case: we have one of the Arch Linux maintainers here (Anatol) that really needs to use system libraries to package osquery for the distribution repositories
if you are familar with Thrift maybe we can get something started
if it's a strong requirement for you
y

Yves Dolce

01/28/2020, 10:33 PM
I'd love to but I'd first need to have a closer look at Thrift API.
I just did and experimented with 
osquery\extensions\thrift\osquery.thrift
. Again, I'm trying to perform osquery queries and get the corresponding result. That can be done using only Thrift?
t

theopolis

01/29/2020, 2:48 AM
It gets you 50% of the way, take a look at the go implementation for an example https://github.com/kolide/osquery-go/blob/master/README.md one could create a new repo called osquery-cpp and write code similar to what you see in that project
y

Yves Dolce

01/29/2020, 6:31 PM
Thanks but is this not also "_not supported and uncharted_" as @alessandrogario mentioned earlier here when I was entertaining the idea of building an extension/plugin "outside the osquery tree"?
#windowsJoin Slack
Powered by