https://github.com/osquery/osquery logo
Title
l

Luqi Pan

03/17/2021, 11:16 PM
Hey folks, my name is Luqi and I work at Apple with several other folks on osquery efforts. We're very interested in seeing official support of osquery on ARM become a reality and my understanding is that you folks are very much interested in it as well. We recently learned that folks at AWS are willing to provide engineering resources and compute credits to help with this goal, specifically CI for osquery on ARM. (Thank you AWS for the generous offer!) Would you all be interested in a meeting to discuss how do we make this a reality? I'm in no way familiar with the development process in osquery land but maybe we can target official support of osquery on ARM for 4.8.0 release?
Tagging folks who are potentially interested in this topic...
Apple team: @timb @Ivanlei @ikoniaris
AWS team: @Paul Roberts @Ali Saidi
Community: @Liz Fong-Jones @alessandrogario @zwass @seph @theopolis
l

Liz Fong-Jones

03/17/2021, 11:20 PM
affirmative, I can be a point of contact for community, although you're in good hands with @Ali Saidi
🎉 1
z

zwass

03/17/2021, 11:25 PM
I'd be happy to attend a meeting regarding this, however others you've mentioned are more qualified. Perhaps consider me a backup?
👍 1
a

alessandrogario

03/17/2021, 11:25 PM
I think @Marcello Golfieri was also tracking this; and yeah it would be cool to schedule a meeting with the tagged people 🙂
p

Paul Roberts

03/17/2021, 11:26 PM
Yes @Marcello Golfieri is also working on this from the AWS side, happy to help
❤️ 1
l

Luqi Pan

03/17/2021, 11:27 PM
Thanks for the tag Alessandro. Hey @Marcello Golfieri, didn't realize you're here as well 😛
s

seph

03/17/2021, 11:27 PM
Hi folks! I'm super psyched for folks help. I'm in the midst of child bedtime, but I generally have east coast availability.
❤️ 2
I think I can describe the various steps to do, and if folks can help that be great. This is volunteer time for me, and AWS has a lot of knobs.
Currently osquery ci is based on GitHub runners. While this isn't set in stone, I'm not sure changing that makes sense.
So I'm trying to get the GitHub runner self hosted in aws. It's simple enough on the command line, but baking it into an ami has lots of fiddly bits.
Comically, this sort of glue is what I used to do professionally. But time is always scarce.
a

Ali Saidi

03/17/2021, 11:34 PM
The benefit i see from another CI is just that they’re already doing the lifting for setting up the infrastructure so we don’t have to focus on that but it requires two CI setups.
a

alessandrogario

03/17/2021, 11:38 PM
I wouldn't mind to have a second CI to maintain if that's the best tool for the job (I haven't looked into how the GitHub Actions runner perform on ARM64, so refer to @seph for this)
The only requirements on my side is to be able to somehow acquire the artifacts for the new codesigning workflow we are preparing
t

theopolis

03/17/2021, 11:44 PM
@seph and @alessandrogario is something like this tenable: https://github.com/machulav/ec2-github-runner it seems to have very few moving parts
:this: 1
z

zwass

03/17/2021, 11:46 PM
I'd love to see us keep to 1 CI just to limit the spread of the codesigning keys. I trust y'all to make the best decision though.
a

alessandrogario

03/17/2021, 11:48 PM
The codesigning workflow we have prepared will remain on GitHub Actions, so it wouldn't be affected by a second CI (we just need to be able to download the artifact and we are good)
a

Ali Saidi

03/17/2021, 11:49 PM
@theopolis the trick is dealing with fact that given it’s a public CI service testing every PR someone can send a PR that tries to read a secret on the machine or something similar that have to be managed. The services that offer managed runners deal with these issues for us.
👆 1
a

alessandrogario

03/17/2021, 11:51 PM
I don't think we have any secret on the public CI, we have moved everything in a separate private repository that only the TSC can access to launch the final packaging and codesigning process
s

seph

03/17/2021, 11:51 PM
I can look at
ec2-github-runner
though the current thing I’m playing with is also pretty similar.
a

Ali Saidi

03/17/2021, 11:53 PM
The thing you’re currently playing with looks pretty good @seph. If we really want a single CI seems like a good way forward and we should just pile on make it work.
t

theopolis

03/17/2021, 11:53 PM
@Ali Saidi that makes sense, I can try to set up that runner workflow in a test repo and intentionally land a shell into the runner to see if secrets are exposed.
We don’t have any project specific secrets as Alessandro mentions, but it’s unclear if CI secrets would be available to the runner
a

Ali Saidi

03/17/2021, 11:55 PM
you need the secrets to register with github when the service starts initially, but then i assume they can be deleted (except for how to stash the output).
l

Luqi Pan

03/17/2021, 11:55 PM
Would it be helpful if folks from AWS built a PoC that kicks off osquery tests on EC2 Graviton with
ec2-github-runner
?
s

seph

03/17/2021, 11:56 PM
@Ali Saidi That matches my understanding.
@Luqi Pan Honestly, I don’t know. I think it depends where on the PoC / production scale things are.
a

Ali Saidi

03/17/2021, 11:56 PM
@seph has been using something around https://github.com/envoyproxy/ci-infra which is another starting point
s

seph

03/17/2021, 11:56 PM
Okay, I just pushed my current work in progress to https://github.com/osquery/infrastructure/pull/7 It is very much a draft.
That ami does build. But I have not written scripts that would, on boot: 1. fetch credentials 2. detach from an ASG 3. configure a runner 4. start a runner 5. shut down the machine when a run completes I don’t think the github runner has obvious hooks, but I’m assuming we can figure out something
With some side projects around going over security groups and VPCs and whatnot to make sure those are least privledged
l

Luqi Pan

03/18/2021, 12:05 AM
Got it. Maybe folks from AWS can build on top of https://github.com/osquery/infrastructure/pull/7 and fill in the gaps?
s

seph

03/18/2021, 12:06 AM
Yes, I’d love that. Though I would want to be pretty clear in what’s happening and coordination lest we step on each other.
t

theopolis

03/18/2021, 12:06 AM
(But my vote is always for the simplest and safest solution to start out, then we can iterate and consolidate if needed)
s

seph

03/18/2021, 12:07 AM
It’s different axis of simple. I think either we put complexity into an AMI to manage a runner. Or we put complexity into maintaining 2 CI systems.
And if we look at the next likely platform, Apple m1, I suspect we’re going to have a similar set of questions
l

Luqi Pan

03/18/2021, 12:09 AM
For sure, I think we can coordinate here to avoid stepping on each other's toes @Ali Saidi @Paul Roberts @Marcello Golfieri what do y'all think ^?
👍 1
s

seph

03/18/2021, 12:15 AM
I’m not opposed to using AWS codebuild either — mostly I’m trying to do what I think has less general overhead? Are there macOS and windows options for codebuild?
p

Paul Roberts

03/18/2021, 12:28 AM
We're open to the community's feedback. If AWS can make things easier to help automate the ARM builds then we're willing to help.
m

Marcello Golfieri

03/18/2021, 12:29 AM
Absolutely. Amen to that
a

Ali Saidi

03/18/2021, 1:56 AM
should we create a channel for discussing this further?
s

seph

03/18/2021, 2:02 AM
I’d say just use #infrastructure or this one