Hi folks! I know there’s been a lot of interest in the arm64 CI work, and I wanted to give y’all an update on what @theopolis and I have been working on. (In thread)
We’ve been pursuing two different strategies.
The first mechanism is to have github actions spin up a dedicated runner, run the CI, and then turn off that runner. This approach is fairly simple, and should be pretty quick to implement. It works works by giving github appropriate AWS credentials. Because github only shares credentials with the primary org (no forks), this is reasonably secure. But, it cannot work on non-osquery forks. Still, having a test run post merge is hugely valuable.This is represented by https://github.com/osquery/osquery/pull/7014While not merged, this is currently running, and feels pretty close.
The second approach is to keep standing github runners around read to process jobs. Because these runners are running public code, we need to take several steps to protect them. They have somewhat complicated permissions scheme where they start with enough to configure them, then they drop permission.This work is represented by https://github.com/osquery/infrastructure/pull/7 and is less far along than the first approach.
Happy to answer any questions!
03/23/2021, 2:24 AM
so the gist here is that approach one is more secure because only maintainers/contributors are launching the builds and the second approach is dangerous because anyone could execute code and we'd want to make sure the actual worker is properly removed/secured/re-instantiated?
What's everyone's preference?
03/23/2021, 3:08 AM
I think we’d like to have runners that can handle the PRs. But we’d also like to have something sooner.
I think we’ll see the first approach (runners spun up on branches in the osquery org) and while we work on suitable isolation for the other
03/23/2021, 5:25 PM
Being able to do ARM-based CI on PR merges would at least generate that level of reassurance needed to be able to mark ARM packages as 'release', right
The second approach, is a persistent hosted ARM-based CI runner that anyone can run jobs on? Yea I don't know if that's advisable. I acknowledge the benefit —anyone can run the CI jobs even from their forks— but, maybe for this we should just wait for GitHub Actions to provide a runner.
03/23/2021, 8:39 PM
Couple things, regarding #1 - it sounds like that approach is more secure because not everyone will be able to perform builds due to the lack of access to GH secrets.
Regarding #2, there are definite security concerns, however, @David Gibb from the AWS side is happy to put together a secure environment where we could test this.
Are we good to move forward with @theopolis solution to enable regular testing while @David Gibb tries to flush out @seph approach based on what envoy didt and then we can evaluate the security posture of that second solution?
03/24/2021, 3:48 PM
That is certainly @theopolis and my intent.
03/24/2021, 3:59 PM
awesome. I plan on kicking the tires today - I'm sure I'll have questions, ideas, etc. in a day or two
03/24/2021, 4:00 PM
It’s, um, very drafty still. I think it communicates my intent, and direction, but I’m still working through enough of the terraform to get a working VPC, to actually test the permissions dropping.(I feel like I’m making excusing, but there are a lot of moving parts)
I think I could probably create some kind of readonly role on the AWS accounts if that would be interesting
03/24/2021, 4:07 PM
my entire life is drafty 🙂
03/25/2021, 5:50 PM
@David Gibb i assume you’re just create cdk code or similar in your test account so it could be easily stood up other places?
03/25/2021, 5:51 PM
It looks like Seph and team are using Terraform, so I'm running with that. My test account isn't as complex as their organization, so I'm standing up a simplified version
03/25/2021, 5:54 PM
@seph i thought you didn’t want to go the TF route it was just that envoy started there?
03/25/2021, 6:00 PM
I ended up using TF because it was ultimately easier to work through my STS setup with it, than to get the VPC parameters right without it.