@seph && @theopolis https://osquery.slack.com/archives/C08V7KTJB/p1569351776173900?thread_ts=1569342404.172100&cid=C08V7KTJB

So you think it is git version dependent? If it’s consistently just augeas, does making it not-shallow help? Does using a release that’s within the last 50 commits help?

Yeah, it is; if you were to try it on the same git version that we use on the CI, it will work

Interesting. Do you know how far back the commits for the other broken ones are?

it is very erratic, I don't think we can just filter with a simple

version > xxx

I was trying to read it last night — it looks like the submodules stuff is all cmake we wrote?

I would: 1. Remove the shallow everywhere; I'm not really a fan of that 2. Add ~/.git/modules to the cache 3. CMake will automatically get the delta from the remote repo OR 1. Remove the shallow everywhere 2. Add a pre-build script Script would do this: 1. Fully clone all the remotes inside the cached CI folder 2. Go though all the submodules in osquery and update the remote to point inside the cached submodules 3. git submodule sync --recursive 4. Resume normal operations

I dunno… I really like using shallow for this sort of thing.

I can reproduce it 100% of the times on my machine

If you’re up for it, try adding that fetch in? I don’t think it unshallows

It doesn't, it needs

git fetch --unshallow

(there's a submodule command for that, i don't remember it)

Weird. Is there enough of a reproduce case that I can test this with bare git commands in a test repo? I can’t easily interate inside the osquery dependency system.

since this happens at configure time, it should be usable anywhere without installing anything but git and cmake

That’s not something I can iterate from — I can’t quickly tinkering with the git commands in the depths of cmake

Interesting, then it's more random than I thought

macOS vs linux may matter

works fine on git 2.17.2 on macOS as well

Let’s try MITMing our connection to GH for failing versions, and/or stracing and narrowing this down. I’m interested in the root cause.

Is there an OS this is likely to fail on?

for more context, I had tried switching all the URIs to

git://

and adding

.git

and was still able to reproduce. I was testing inside of `oss-fuzz`'s Dockerfile, which is:

FROM <http://gcr.io/oss-fuzz-base/base-builder|gcr.io/oss-fuzz-base/base-builder>

and I tried the

git

available from a basic

apt-get install

as well as the 2.23 version available here: https://askubuntu.com/questions/568591/how-do-i-install-the-latest-version-of-git-with-apt/568596

Trying to make a test case… The docker container

<http://gcr.io/oss-fuzz-base/base-builder|gcr.io/oss-fuzz-base/base-builder>

has a cmake that’s way too old. Ditto ubuntu16.04. Though now I’m trying with the cmake in stefano’s docker files

you can just download an install a new cmake

Yes, that’s what’s in stefano’s dockerfile

you can in theory just launch the git command manually

Woo! That’s a reproduce case.

sometimes you just restart the cmake configuration, git tries again and it works

this is so odd, I am seeing this behavior with 2.23.0 too

Awesome finding that! I don’t know if git is developed on github, or just mirrored there — I don’t know if that comment will get seen.