https://github.com/osquery/osquery logo
Title
s

seph

10/03/2019, 12:46 PM
So you think it is git version dependent? If it’s consistently just augeas, does making it not-shallow help? Does using a release that’s within the last 50 commits help?
a

alessandrogario

10/03/2019, 12:47 PM
Yeah, it is; if you were to try it on the same git version that we use on the CI, it will work
The augeas repository is not huge, so I would just clone it fully; there are some others however that are causing the issue and they are huge
I think we have this issue with around ~4 submodules
s

seph

10/03/2019, 12:48 PM
Interesting. Do you know how far back the commits for the other broken ones are?
Alternately, we could just slap something in that complains if the git version is below XXX
a

alessandrogario

10/03/2019, 12:49 PM
it is very erratic, I don't think we can just filter with a simple
version > xxx
I think the shallow modules were squashed inside the main commit that contains the third-party refactor
Yeah, it's all in 6481b34e2382ffb5b4c469de3157c63e0ed23ea3
s

seph

10/03/2019, 12:54 PM
I was trying to read it last night — it looks like the submodules stuff is all cmake we wrote?
Are we linking to specific shas or tags? What happens if you add something like
cd ...moduledir && git fetch -t
a

alessandrogario

10/03/2019, 12:57 PM
I would: 1. Remove the shallow everywhere; I'm not really a fan of that 2. Add ~/.git/modules to the cache 3. CMake will automatically get the delta from the remote repo OR 1. Remove the shallow everywhere 2. Add a pre-build script Script would do this: 1. Fully clone all the remotes inside the cached CI folder 2. Go though all the submodules in osquery and update the remote to point inside the cached submodules 3. git submodule sync --recursive 4. Resume normal operations
Yeah, all the submodules are pointing to an upstream release tag; exceptions are bad upstream repositories such as libudev ('cause it's now systemd and they kind of nuked the version history)
(Some of them got upgraded to a new tagged version)
The script would be easy to build with
git submodule foreach
s

seph

10/03/2019, 1:00 PM
I dunno… I really like using shallow for this sort of thing.
I sorta think you’re running into submodules being kinda a mess. ISTR they’re not super well supported
Either way though, if there’s a simple enough reproduce case, I’d try to play with fetch, and keep shallow.
Or think a lot harder and ditch the submodules
a

alessandrogario

10/03/2019, 1:03 PM
I can reproduce it 100% of the times on my machine
it is really that bad on recent git versions (I have 2.23.0 here)
s

seph

10/03/2019, 1:04 PM
If you’re up for it, try adding that fetch in? I don’t think it unshallows
a

alessandrogario

10/03/2019, 1:04 PM
It doesn't, it needs
git fetch --unshallow
(there's a submodule command for that, i don't remember it)
sometimes you just restart the cmake configuration, git tries again and it works
but some modules are stubborn and will never work (like google logging)
s

seph

10/03/2019, 1:27 PM
Weird. Is there enough of a reproduce case that I can test this with bare git commands in a test repo? I can’t easily interate inside the osquery dependency system.
a

alessandrogario

10/03/2019, 1:28 PM
since this happens at configure time, it should be usable anywhere without installing anything but git and cmake
this should reproduce the issue
git clone <https://github.com/osquery/osquery> && mkdir build && cd build && cmake ../osquery
s

seph

10/03/2019, 1:30 PM
That’s not something I can iterate from — I can’t quickly tinkering with the git commands in the depths of cmake
Maybe I’ll try though. What git version is expected to fail
version 2.21.0 on macOS works
works fine on macOS git version 2.23.0
a

alessandrogario

10/03/2019, 2:17 PM
Interesting, then it's more random than I thought
we can maybe ask more info to the other contributors that encountered this issue
s

seph

10/03/2019, 2:18 PM
macOS vs linux may matter
s

sharvil

10/03/2019, 4:35 PM
works fine on git 2.17.2 on macOS as well
t

theopolis

10/04/2019, 12:09 PM
Let’s try MITMing our connection to GH for failing versions, and/or stracing and narrowing this down. I’m interested in the root cause.
s

seph

10/04/2019, 3:03 PM
Is there an OS this is likely to fail on?
t

theopolis

10/04/2019, 8:04 PM
for more context, I had tried switching all the URIs to
git://
and adding
.git
and was still able to reproduce. I was testing inside of `oss-fuzz`'s Dockerfile, which is:
FROM <http://gcr.io/oss-fuzz-base/base-builder|gcr.io/oss-fuzz-base/base-builder>
and I tried the
git
available from a basic
apt-get install
as well as the 2.23 version available here: https://askubuntu.com/questions/568591/how-do-i-install-the-latest-version-of-git-with-apt/568596
s

seph

10/04/2019, 9:42 PM
Trying to make a test case… The docker container
<http://gcr.io/oss-fuzz-base/base-builder|gcr.io/oss-fuzz-base/base-builder>
has a cmake that’s way too old. Ditto ubuntu16.04. Though now I’m trying with the cmake in stefano’s docker files
t

theopolis

10/04/2019, 9:42 PM
you can just download an install a new cmake
s

seph

10/04/2019, 9:43 PM
Yes, that’s what’s in stefano’s dockerfile
Anyhow, I don’t have a good reproduce case yet. Entirely because cmake does a bunch of toolchain sanity checks, and I don’t have a linux build environment for it.
a

alessandrogario

10/04/2019, 9:47 PM
you can in theory just launch the git command manually
git submodule update --init --depth=1 libraries/cmake/source/augeas/src
should do the trick
s

seph

10/05/2019, 2:24 AM
Woo! That’s a reproduce case.
In
ubuntu:16.04
I can get that easily
Using the above instruction, it fails on ubuntu16, git version 2.7.4 But it works with ubuntu18, git version 2.17.1
<http://gcr.io/oss-fuzz-base/base-builder|gcr.io/oss-fuzz-base/base-builder>
uses git 2.7
https://github.com/boostorg/boost/issues/245 seems to match both what we’re seeing here, and my understanding of this error.
t

theopolis

10/05/2019, 8:15 PM
sometimes you just restart the cmake configuration, git tries again and it works
this is so odd, I am seeing this behavior with 2.23.0 too
I think I figured it out this commit breaks the behavior we want (merged between 2.21.0 and 2.22.0-rc0) https://github.com/git/git/commit/34066f06614d6dbc1955c92ba0c3a17dbfeb0077#diff-7891df559f581cdcc9168363e0f932a1
👍 1
s

seph

10/06/2019, 12:54 AM
Awesome finding that! I don’t know if git is developed on github, or just mirrored there — I don’t know if that comment will get seen.