hello guys, I want to know that Live query and Schedule query do they maintiain different queue and does LIVE query have more priority than SCHEDULE query ? and if yes then if any live query come does SCHEDULE query get preempted or it is like that SCHEDULE query will execute but for next query, osquery will Prioritize LIVE query over SCHEDULE query

Hi everyone, this is just an heads up. I believe

osquery::InternalRunnable

(as other places probably) is affected by clock adjustments (for sure on Windows, not sure on other platforms). Found this bug that basically say

std::this_thread::sleep_until

and

std::this_thread::sleep_for

implementation are based on a

system_clock

rather than a

steady_clock

. On top of it possibly other functions are affected such as:

std::condition_variable::wait_for
std::condition_variable::wait_until
std::condition_variable_any::wait_for
std::condition_variable_any::wait_until
std::future::wait_for
std::future::wait_until
std::recursive_timed_mutex::try_lock_for
std::recursive_timed_mutex::try_lock_until
std::shared_future::wait_for
std::shared_future::wait_until
std::shared_lock::try_lock_for
std::shared_lock::try_lock_until
std::shared_timed_mutex::try_lock_for
std::shared_timed_mutex::try_lock_until
std::shared_timed_mutex::try_lock_shared_for
std::shared_timed_mutex::try_lock_shared_until
std::timed_mutex::try_lock_for
std::timed_mutex::try_lock_until
std::unique_lock::try_lock_for
std::unique_lock::try_lock_until

shocked

Our builds are running out of disk on the AWS runners. https://github.com/osquery/osquery/actions/workflows/self_hosted_runners.yml @Stefano Bonicatti Did you fix this last time?

@seph There's a tag,

5.9.0.a

which is causing failures for the Windows builds on the latest commit (since the tag is there). Is that still necessary for tests or?

Hello, sorry for disturbing. I don't see a downloadable arm package for windows https://www.osquery.io/downloads/official/5.9.1 When will it be added, please?

Echoing it here: https://osquery.slack.com/archives/C08V7KTJB/p1696370043439059 And adding that we should maybe wait for https://github.com/osquery/osquery/pull/8052, but would really like if we could cut a pre-release this week, basically dropping the rest of the unfinished PRs (they all still need some work one way or another).

Micah and I have been working through a bug he found in

master

. https://github.com/osquery/osquery/pull/8037 introduced an issue. I was hoping we’d have a fix up today, but I think not. Worst case, I’ll back that PR out

@sephThe per row table call in a JOIN or the single constraint passed to the table per IN constraint, has always been the case since I remember. Although I know that we have functions to get multiple constraints per generate call, I never have seen them actually get more than 1.

I’m going to cut 5.10.0 as soon as CI on main is happy. (and I notice)

oops, I want to merge in

wifi_survey

as well. CI wait….

5.10.0 is cut. Waiting on CI there.

I can’t quite remember where I left the codesigning refactor. But I’ll try to drive signing with the new code. It’s at https://github.com/osquery/osquery-codesign/pull/43/files if people are up for looking

Hrm. Builds on main are failing. https://github.com/osquery/osquery/actions/runs/6437556025

5.10.1 exists, and there are packages at https://github.com/osquery/osquery/releases/tag/5.10.1 and the various test servers. It’s signed with the new workflow, I don’t think it’ll change anything, its the same code, just moved around

What’s left to get this marked as a final release?

It looks like the windows zip files didn’t get built. I’ll take a look

@seph for the JOIN, the sqlite logic to execute it is implemented as two nested for loops, and each table lands on the external or internal for loop based on the order in which the tables appear in the JOIN. The external table is already called only once, but then each row is used to call the internal table. I don't think this can be changed (for instance, having the external table collect all the constraint values, and then sending them all together).

Hey folks, @sharvil and I identified a potential issue with 5.10.1. The TL;DR is that when an osquery API manager triggers a config refresh, it can cause the file monitoring targets list to get kinda messed up and start monitoring all filepaths which causes a DB/logging blowup. What are the options for: 1. Waiting on the release until this is fixed or 2. Quickly pushing out a new release with a fix (but not waiting an entire release cycle to do so)?

hrm, something seems to be up with our

build_windows (Release, 64, windows-2019)

workflow

I’ve made a 5.10.2 to handle this ES FIM issue. There are binaries up at https://github.com/osquery/osquery/releases/tag/5.10.2

We can discuss whether to call 5.10.1 stable, or how to approach this in office hours, or here, I guess

I don’t seem to recall the answer to this one, and I couldn’t find it in the search — is there a difference between packages that are attached to a release on the github page and the one that gets pushed to osquery.io?

https://github.com/osquery/osquery-codesign/pull/43 seems to have worked. I think we should merge it

Oops. 5.10 broke `magic`:

pemberton:updates seph$ ./osqueryd/5.9.1/osqueryd -S 'SELECT * FROM magic WHERE path="/bin/ls"'
+---------+-----------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+---------------------------+--------------------+
| path    | magic_db_files                                      | data                                                                                                                            | mime_type                 | mime_encoding      |
+---------+-----------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+---------------------------+--------------------+
| /bin/ls | /usr/share/file/magic.mgc:/usr/share/misc/magic.mgc | Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e] | application/x-mach-binary | (null)(null)binary |
+---------+-----------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+---------------------------+--------------------+

pemberton:updates seph$ ./osqueryd/5.10.2/osqueryd -S 'SELECT * FROM magic WHERE path="/bin/ls"'
/usr/share/file/magic.mgc, 1: Warning: offset `?' invalid
Version=' invalidagic.mgc, 2: Warning: offset `
/usr/share/file/magic.mgc, 3: Warning: offset `' invalid
/usr/share/file/magic.mgc, 6: Warning: offset `Firmware v' invalid
/usr/share/file/magic.mgc, 12: Warning: offset `' invalid
Corel Corporation' invalid 13: Warning: offset `

I’m inclined to blame the libmagic upgrade in https://github.com/osquery/osquery/pull/8142

Hey team, I'm looking into building out nftables support to function similarly to the existing iptables table. As noted in the issue here, nftables does not seem to expose the same proc interface that the iptables functionality relies on. The nft docs here explain the available userspace interfaces. After exploring the options a bit it seems like we could bring in libnftables to export rules as JSON, and parse rows from there with a similar output structure to the iptables table. libnftables appears to be a simple, stable API from the history we have available, but I wanted to get some early feedback- how do others feel about bringing in a library like this? Is this a realistic approach?

@seph For the presumed WMI hang you were talking about in office hours, the first question is which version of osquery? The only issue in the past that should've been solved is https://github.com/osquery/osquery/issues/7424; maybe something related?

Hi, I have reviewed latest Osquery schema and I do not find anything to get Windows User Rights Assignment https://learn.microsoft.com/en-us/windows/security/threat-protection/security-policy-settings/user-rights-assignment . Did I miss something ?

Hi all! Please tell me, has anyone encountered the inability to use the

process_open_sockets

and

listening_ports

tables on Linux load balancers with open sockets highload? We have a number of servers acting as external load balancers that can have over 200,000 active TCP/UDP sockets at any time. And on these servers we cannot effectively use the tables described above, because such queries often exceed the watchdog memory limit, although we raised it to 400 megabytes. As result they got denylisted. As I think, at the C++ code level, osquery first receives the full set of all sockets, and then applies the specified filters to this set. Perhaps there are some opportunities for optimization here.

Would you please allow selects from ntfs_acl_permissions to use like rather than just = ?

@Lucas Rodriguez: I left a comment with a couple of notes, but I totally misremembered what was available as filtering via audit rules. So it's not possible to filter by path in the rules, when auditing those syscalls, my bad.