Title
#general
a

alessandrogario

11/14/2018, 10:43 PM
@fmanco will it be possible to easily setup a system that builds a binary that has the same hash as the binary released upstream? (i'm talking about 100% reproducible builds)
fmanco

fmanco

11/14/2018, 10:56 PM
In some sense yes. Buck is actually much better at guaranteeing reproducible builds than CMake. If you take a look at https://buckbuild.com/ this is the second thing highlighted. However we're not going to provide a toolchain anymore, so as far as you use the same toolchain this should be possible, but that's something that we're not providing, at least yet.
g

groob

11/14/2018, 10:59 PM
so, the real answer is no, not possible
a

alessandrogario

11/14/2018, 11:01 PM
Is the toolchain you are going to use internally public? Is it possible to replicate it?
fmanco

fmanco

11/15/2018, 2:42 AM
@groob indeed no. But just to clarify are you talking about getting any distro building osquery and getting the exact same binary, or just building twice out of the same VM image giving the same binary?
2:44 AM
@alessandrogario the toolchain we're going to use depends on the base system. We're still working out the details, but the idea is to install whatever comes with your distro and build out of that.
a

alessandrogario

11/15/2018, 2:44 AM
I see; but doesn't that mean that osquery will have to build once for each supported system and distribution?
2:46 AM
Having a standard toolchain is great right now because everything is almost 100% static and the same binaries (built once) can work everywhere
2:47 AM
And this makes it possible to output the same binary from two different systems if both are compiling from the same commit
fmanco

fmanco

11/15/2018, 2:52 AM
Possibly yes we would need to build for each os if we don't statically link everything. I'll check whether we'll be able to provide a toolchain as well. In fact the external build is something we're still finalising. What is clear is that toolchain will be out of the code dependencies (so you can build with whatever toolchain you like), but we can try to make it part of provisioning.
g

groob

11/15/2018, 2:53 AM
static binaries are awesome. please keep them ❤️
fmanco

fmanco

11/15/2018, 2:54 AM
Indeed this would theoretically make it possible to have completely reproducible builds but AFAIK the current system doesn't do that either. If you build the same commit out of Ubuntu and Centos you'll get different binaries. Buck puts us in a better place to make this happen.
a

alessandrogario

11/15/2018, 2:54 AM
A small note though; it is already possible to use an arbitrary toolchain (and call CMake directly without the current Makefile wrapper)
fmanco

fmanco

11/15/2018, 2:58 AM
@groob yes we will. But if you look at the current binary it is not 100% statically linked, we still have some dynamic libs.
g

groob

11/15/2018, 2:59 AM
that’s mostly fine
a

alessandrogario

11/15/2018, 3:03 AM
I think that having at least a small readme that shows how build the same toolchain used upstream is really important
fmanco

fmanco

11/15/2018, 10:13 AM
What exactly do you mean by upstream? Notice that we'll test everything and build packages out of a public toolchain (either provided by us or packaged by the distro).