Channels
android_tests
apple-silicon
arm-architecture
auditing-warroom
aws
carving
code-review
community-feeds
core
darkbytes
doorman
ebpf
eclecticiq-polylogyx-extension
extensions
file-carving
fim
fleet
fleet-dev
fleetosquery
foundation
fuzzing
general
golang
goquery
infrastructure
jobs
kolide
linen-dev
linux
macos
officehours
osctrl
plugins
process-auditing
querycon
queryhub
random
selfgroup
sql
tls
uptycs
vendor-feeds
website
windows
zeek
zentral
zercurity
Title
b
Benjamin Herrenschmidt
08/13/2020, 1:49 AMHi Folks ! Is there a specific channel to discuss the aarch64 port ? Is anybody tackling the rebase yet ? (Looking at helping...)
a
alessandrogario
08/13/2020, 12:05 PMhello @Benjamin Herrenschmidt! I don't think there's a channel specific to the arch
I am currently stuck with BPF, so any help is super appreciated
b
Benjamin Herrenschmidt
08/13/2020, 9:57 PMAllright, going through the internal (Amazon) approval process to work on this. I'm trying to find the text of the CLA and has failed so far ... do you have a copy ? It looks like I would need to send a PR to get it but I need to get it vetted before I'm allowed to send a PR 🙂
Approval sorted, I've started looking at the rebase, it looks reasonably simple Makefile wrangling, shifting files around etc... so far. Please let me know if theres anything I need to be aware of, I know nothing about osquery at this point 🙂 (I'm a kernel guy but this needed doing so I volunteered)
a
alessandrogario
08/19/2020, 11:01 PMSo you will find out that our build system is weird for 2 major reasons
1. We use weird names; until recently we had to support the Buck build system used by Facebook, and we aimed to have 1:1 structure. Buck supports namespaces, and so the original targets re-used the same names over and over again. Buck is no longer there... we'll fix this
2. We re-import all the dependencies manually, by rewriting their build system in CMake
The goals of rewriting the build system of each lib are:
1. Not having to host pre-built dependencies anywhere
2. Automatically propagate all the compilation flags and settings top to bottom without worrying about outdated things
3. Not having to deal with incompatibilities between pre-built libraries and osquery or between pre-built libs and other pre-built libs
4. Preventing libraries from leaking dependencies against the host system (example: we don't want the curl makefiles to link against the system openssl. we want to be able to use the openssl version we have already imported)
5. Build everything statically
6. Fine tune all the config files generated by auto-tools, so that we can filter out things that are not supposed to be there (example: one script could detect we have a library on the system that is not supposed to be found as per official builds)
7. Make sure we target the correct glibc version for the reference system (Ubuntu 16.04) so that we can the binary can run on any distro >= recent than that
8. Help with reproducible builds
So to mark the port as beta: the branch is rebased on master, deleting the buck build system, then we merge it
b
Benjamin Herrenschmidt
08/19/2020, 11:05 PMOk, I'll try to identify the ""buck" builds. I was also thinking of splitting the patch to separate the moving files & CMake wrangling from code changes
for bisectability/review
also I need to check if the list of libs has changed, I might have to add/remove some etc...
a
alessandrogario
08/19/2020, 11:06 PMTo mark the port as stable: all the libraries that are used by AArch64 are updated. This means taking an AArch64 Ubuntu 16.04 VM, running the autogen scripts to generate the config.h files, and then go through them all and make sure that there are no unneeded deps that sneaked in due to things installed in the VM. The config.h files are then committed. The library is already build by CMake so that's already done by the awesome contributor!
b
Benjamin Herrenschmidt
08/19/2020, 11:06 PManyway, I''m going to spend quality time on this in the next few days (had somethying else tto finish while I was waiting for legal)
a
alessandrogario
08/19/2020, 11:06 PMTo mark the port as officially supported: we have to find a way to add a CI
b
Benjamin Herrenschmidt
08/19/2020, 11:06 PMok
a
alessandrogario
08/19/2020, 11:07 PMBut we can start merging things as soon as we rebase, and then I can start generating releases for the website
b
Benjamin Herrenschmidt
08/19/2020, 11:07 PMI might try to generate an Amazon Linux 2 RPM as well in Extras
once I sort out some python3 module deps, but all that can wait til I've done the basic stuff
a
alessandrogario
08/19/2020, 11:08 PMadding support for Amazon Linux 2 should be easy, the only thing to fix is the glibc package name that is added as a dependency
b
Benjamin Herrenschmidt
08/19/2020, 11:08 PMok
I''ll dig into all this and get back to you if I have blocking questions, thanks
a
alessandrogario
08/19/2020, 11:09 PMthat's another convenient thing we gain in having so much control (and maintenance burden!) over the deps
b
Benjamin Herrenschmidt
08/19/2020, 11:09 PMyup, that's going to help 🙂
a
alessandrogario
08/19/2020, 11:09 PMthanks for looking into this! can't express how happy I am 😄
b
Benjamin Herrenschmidt
08/19/2020, 11:11 PM🙂