Hi folks. I’m making some changes to how we serve packages. This probably won’t break anyone, but if it does, please speak up!
There will be ongoing changes, as I try to find an inexpensive way to server content. Turns out, we’d doing about 75TB/month. Which is pretty hefty.
😮 1
🚀 1
But. most of that is entirely inside AWS.
So. I’m starting to try to build some direct-to-bucket-link things, since the data transfer from buckets in-region is free
Alternately, if you’re running a site with a large number machines that download osquery every couple seconds, please let’s find another way?
g
Gavin
09/14/2021, 8:05 PM
So this may not be possible for the foundation's specific use case but I see some popular projects such as emby using Github Releases & Github Pages to host APT repos.
Would moving off AWS and onto a purely GH hosted solution something you would consider ?
s
seph
09/14/2021, 8:06 PM
I didn’t see a good approach to that. But maybe I didn’t look hard enough
I did initially consider it. But… apt doesn’t use full URLs, just relative ones. And there’s a lot of weird/gnarly overhead in making that go.
I suspect that requires (if automated) checking the packages into GH pages. On the other hand, is that bad? I mean, surely a 300gig repo is fine. Right…
g
Gavin
09/14/2021, 8:27 PM
Yeah let me think about that
s
seph
09/14/2021, 8:42 PM
It’s a good thing to keep thinking about. I’ll make sure it’s in my notes