I’m not sure what the right next steps are for this release.

I'm down for suggesting testing between website stable and Linux repo stable.

shasums on the website match the downloads

mmmmm. Google How many projects do I want anyhow. Probably need to think on that. osquery has a different balance than kolide. (note that in google cloud, project is the big isolation level between things)

@AoS that’s possibly because you haven’t specified any options to the columns you need. If you need some columns to be passed as constraints, depending on what those columns are and are used in your table logic, you have to mark them with a combination of required and/or additional and/or index.

(low pri) we might want to reconsider the in-progress work from https://github.com/osquery/osquery/pull/5257 https://github.com/osquery/osquery/pull/5268 https://github.com/osquery/osquery/pull/5285 if anyone has time to review + reopen as a new PR

I think the issue is the same as this: https://github.com/osquery/osquery/issues/6152

I am considering what larger effort I can help on (in small chunks) over the next month or so. I was considering working on having BUCK support the source-layer approach CMake uses. My guess is that we can make this work by first running CMake in a

buck-out/cmake

folder to checkout repos and patch files, then run BUCK with static configuration to expect third-party code in

./buck-out/cmake

. Then the significant changes to BUCK logic only include adapting the compile logic in CMake, which should be easy. This will enable us to move to newer third-party library versions and remove all dependence on the facebook pre-built layer. The downside of me working on it is that someone else might be able to move faster. Suggestions for what I should work on / how I could approach what I referenced above?

You are doing a lot of work on this regard, it would be nice to make it official by talking to him

is there any mechanism for sending config to a plugin?

https://github.com/osquery/osquery/issues/3027 implies it’s possible, and that alex has an early PR for it. https://github.com/osquery/osquery/pull/5285

How serious do we think this CVE is?

We could suggest providing a cert in the osquery configuration as a workaround. This would prevent the MITM attack that is possible due to the bug.

though the point is that you should be formatting before committing ^^'

I’d suggest we do several things: 1. Consider linting the whole files. This will produce larger diffs, but it will rip the bandaid off 2. Start fixing the linting everywhere, which will make that smoother 3. Provide a tool to either format the whole file, or run the diff and format it.

I appreciate the responses and movement though @seph, I think the folks internally are just cautious about ensuring we're responding to CVEs being cut appropriately.

@thor re #6212, you say it affects the tls plugin... i think some will be concerned it affects the aws plugins, but i'm like 99% sure that it does not because we have to specify a CA bundle or it doesn't work. just wanted to point that out!

Alright, #6170 shipped. @alessandrogario if you get a chance can you rebase #6197 and see if it passes tests? I'll see if I can rebase #6044 somehow without ted.

mmm if he means the change to ExitThread -> std::exit -> etc, it's not good as it is unfortunately... the python test we have which launches several osquery processes while using queries etc fails and several processes remain open.

It got kind of difficult to find solid information about the profile script and the stats it returns so I dug and documented. Dropping it here in case anyone else wants it The osquery

profile.py

script uses utils.py in

tools/tests/

which uses python’s

psutil

library to collect process stats for osqueryi as its running given queries.  The script turns 5 stats: Utilization (U): Utilization is calculated by taking the average of non-0 results of the

cpu_percent(interval=1)

function in

psutils.Process()

. This value can be greater than 100% for processes with threads running across different CPUs. The script sets an interval of 1 meaning that the function compares process time to system CPU times elapsed before and after the 1 second interval. This is a blocking call.  CPU time (C): CPU time uses the

psutils.Process()

's

cpu_times()

function. It returns a named tuple containing user, system, children_user, system_user, and iowait • user: time spent in user mode. • system: time spent in kernel mode. • children_user: user time of all child processes (always 0 on Windows and macOS). • system_user: user time of all child processes (always 0 on Windows and macOS). • iowait: (Linux) time spent waiting for blocking I/O to complete. This value is excluded from user and system times count (because the CPU is not doing any work). The profile script adds user and system together for the CPU Time output. Duration (D): Duration is calculated by taking the subtracting

start_time - 2

from the current time. The start time is set before the script starts the osqueryi process to run the query. (I’m not very sure about why the 2 is there. I don’t know how useful of a metric this is when compared to the cpu time, but someone might have better insights.) fds (F): Uses the

num_fds()

function and returns the file descriptors used by the osqueryi process during query execution Memory (M): Uses the

memory_info_ex()

command which is deprecated. Psutils documentation suggests using

memory_info()

instead. The function returns a named tuple and the script uses the

rss

value in the tuple. RSS stands for resident set size and is the non-swapped physical memory used by the process. This should match the RES column in

top

.

That is a great explanation and totally agreed with Seph that we'd love you to PR it. Thank you.

Another thing I'm noticing is that the results from the script differ when ran on one of our ec2 instances versus a docker container in gitlab. Testing with stress-ng to see if I can simulate a little system stress for the docker container to try to get more consistent results. More on that later.

@thor, @Stefano Bonicatti, @farfella I think I located the error Stefano referenced with the shutdown refactor. It was possible to notify the condition variable used by the main thread before the main thread started waiting. An example includes starting a service thread that quickly fails (for some reason or another) that then calls requestShutdown. I changed the code such that the CV uses a predicate and a quick/early request for shutdown makes that predicate hold.

#6190 needs a review and merge-- had to make a change to update with changes to master

Anything I can help with?

I know that osqueryd and osqueryi do not interact with one another (like watchdog can't kill osqueryi if it's running a performance heavy query). However, do the two share the RocksDB embedded database? I ask because we have

select * from osquery_schedule;

as a scheduled query and can see the results being returned in elasticsearch but if we try to do that query in

osqueryi

the fields are zeroed out.

Does

osqueryi

have knowledge of the scheduled queries but no knowledge of the metrics around them?

How do folks feel about https://github.com/osquery/osquery/pull/6236

OSX packages. Unsigned.

Ah yeah sure, I mean, wanted to help find a solution