Title
#general
s

seph

09/14/2021, 3:12 PM
Hi folks. I’m making some changes to how we serve packages. This probably won’t break anyone, but if it does, please speak up!
3:12 PM
There will be ongoing changes, as I try to find an inexpensive way to server content. Turns out, we’d doing about 75TB/month. Which is pretty hefty.
3:13 PM
But. most of that is entirely inside AWS.
3:13 PM
So. I’m starting to try to build some direct-to-bucket-link things, since the data transfer from buckets in-region is free
3:14 PM
Alternately, if you’re running a site with a large number machines that download osquery every couple seconds, please let’s find another way?
Gavin

Gavin

09/14/2021, 8:05 PM
So this may not be possible for the foundation's specific use case but I see some popular projects such as emby using Github Releases & Github Pages to host APT repos. Would moving off AWS and onto a purely GH hosted solution something you would consider ?
s

seph

09/14/2021, 8:06 PM
I didn’t see a good approach to that. But maybe I didn’t look hard enough
8:07 PM
I did initially consider it. But… apt doesn’t use full URLs, just relative ones. And there’s a lot of weird/gnarly overhead in making that go.
8:07 PM
I’d be pretty happy to try to revisit it though
Gavin

Gavin

09/14/2021, 8:08 PM
This looks somewhat like a good starting point for Linux packaging. https://github.com/marketplace/actions/github-pages-apt-repo
s

seph

09/14/2021, 8:14 PM
I suspect that requires (if automated) checking the packages into GH pages. On the other hand, is that bad? I mean, surely a 300gig repo is fine. Right…
Gavin

Gavin

09/14/2021, 8:27 PM
Yeah let me think about that
s

seph

09/14/2021, 8:42 PM
It’s a good thing to keep thinking about. I’ll make sure it’s in my notes