https://github.com/osquery/osquery logo
Powered by
Title
d

Daisukixci

07/07/2021, 12:05 PM
👋 The macos pkg doesn’t have the same hash on github and on the osquery website, is it expected
a

allister

07/07/2021, 12:06 PM
did you check the signature?
d

Daisukixci

07/07/2021, 12:06 PM
Only the sha512
a

allister

07/07/2021, 12:06 PM
URLDownloader
{'Input': {'filename': 'osquery.pkg',
           'url': '<https://osquery-packages.s3.amazonaws.com/darwin/osquery.pkg'}}>
URLDownloader: No value supplied for prefetch_filename, setting default value of: False
URLDownloader: No value supplied for CHECK_FILESIZE_ONLY, setting default value of: False
URLDownloader: Storing new Last-Modified header: Tue, 22 Jun 2021 19:04:54 GMT
URLDownloader: Storing new ETag header: "9180d84001b51fd92f85e8d5ba418db8"
URLDownloader: Downloaded /Users/allister/Library/AutoPkg/Cache/com.github.keeleysam.recipes.osquery.download/downloads/osquery.pkg
{'Output': {'download_changed': True,
            'etag': '"9180d84001b51fd92f85e8d5ba418db8"',
            'last_modified': 'Tue, 22 Jun 2021 19:04:54 GMT',
            'pathname': '/Users/allister/Library/AutoPkg/Cache/com.github.keeleysam.recipes.osquery.download/downloads/osquery.pkg',
            'url_downloader_summary_result': {'data': {'download_path': '/Users/allister/Library/AutoPkg/Cache/com.github.keeleysam.recipes.osquery.download/downloads/osquery.pkg'},
                                              'summary_text': 'The following '
                                                              'new items were '
                                                              'downloaded:'}}}
EndOfCheckPhase
{'Input': {}}
{'Output': {}}
CodeSignatureVerifier
{'Input': {'expected_authority_names': ['Developer ID Installer: Theodore Reed '
                                        '(B89LNTUADM)',
                                        'Developer ID Certification Authority',
                                        'Apple Root CA'],
           'input_path': '/Users/allister/Library/AutoPkg/Cache/com.github.keeleysam.recipes.osquery.download/downloads/osquery.pkg'}}
…512? not 256?
d

Daisukixci

07/07/2021, 12:07 PM
512
a

allister

07/07/2021, 12:07 PM
I don't remember what's posted
d

Daisukixci

07/07/2021, 12:07 PM
it’s the 256 is posted
a

allister

07/07/2021, 12:07 PM
3f9ab772596f4da69687a2d7db9a382535b5eabf2346abd452b24666b8f25102
d

Daisukixci

07/07/2021, 12:07 PM
a

allister

07/07/2021, 12:08 PM
% santactl fileinfo /Users/allister/Library/AutoPkg/Cache/com.github.keeleysam.recipes.osquery.download/downloads/osquery.pkg
Path                   : /Users/allister/Library/AutoPkg/Cache/com.github.keeleysam.recipes.osquery.download/downloads/osquery.pkg
SHA-256                : 3f9ab772596f4da69687a2d7db9a382535b5eabf2346abd452b24666b8f25102
d

Daisukixci

07/07/2021, 12:08 PM
a

allister

07/07/2021, 12:08 PM
matches osquery.io/downloads/official/4.9.0
where is 512 posted?
oh github, one sec
you are correct
osqueryd itself differs
install size differs according to pkginfo
lsbom only shows the size of osqueryd as differing
d

Daisukixci

07/07/2021, 12:18 PM
Was it the same with previous version ? 4.8 is not on github anymore
a

allister

07/07/2021, 12:19 PM
not sure, autopkg grabs the one on s3
not sure what's worthwhile to diff about the 40.2MB compiled binary
not long before the rest of those on the right coast come online
they've always been interested in repeatable builds so there should be a reason they can suss out
s

Stefano Bonicatti

07/07/2021, 12:34 PM
Hum interesting, I have downloaded from github, website and s3. s3 and the website have the same pkg sha, but extracting all of them result in the same content.
The difference in the pkg is likely due to the difference in timestamps between signatures. @allister you said you see the osqueryd binary being different, what's the command you used to check?
a

allister

07/07/2021, 12:44 PM
lsbom
I expanded the pkgs with e.g. 
pkgutil --expand autopkgpath /tmp/s3
then used archive utility to expand the payload and diff'd them which pointed to osqueryd (and the pkginfo, since it looks at exact size in bytes of the payload, and bom as per the content you see converted by lsbom) being different
s

Stefano Bonicatti

07/07/2021, 12:56 PM
ok nvm PEBCAK, I see the difference now in the osqueryd binary too
a

allister

07/07/2021, 1:11 PM
the signature timestamp differing wouldn't alter the filesize, though?
s

Stefano Bonicatti

07/07/2021, 1:11 PM
They have different signatures, the github one is using the new LF keys, while the website one still uses Teddy keys.
I mean, this is on the binary
a

allister

07/07/2021, 1:11 PM
pkg sig's are both teddy
s

Stefano Bonicatti

07/07/2021, 1:11 PM
yep
codesign --verbose=4 -dv ../website/osquery/Payload/usr/local/bin/osqueryd
Executable=/private/tmp/osquery/website/osquery/Payload/usr/local/bin/osqueryd
Identifier=osqueryd
Format=Mach-O thin (x86_64)
CodeDirectory v=20500 size=311636 flags=0x10000(runtime) hashes=9733+2 location=embedded
VersionPlatform=1
VersionMin=658432
VersionSDK=721152
Hash type=sha256 size=32
CandidateCDHash sha256=4010e5a75de2c29387cdfd76ed20ddaa9c0f438e
CandidateCDHashFull sha256=4010e5a75de2c29387cdfd76ed20ddaa9c0f438e4bb851c486de2b46e0476cd0
Hash choices=sha256
CMSDigest=4010e5a75de2c29387cdfd76ed20ddaa9c0f438e4bb851c486de2b46e0476cd0
CMSDigestType=2
Page size=4096
    -2=695ae83a2af26948741ec57ed0f661fa69ffece7e4fc6e7c81b3550cde5492a7
CDHash=4010e5a75de2c29387cdfd76ed20ddaa9c0f438e
Signature size=8996
Authority=Developer ID Application: Theodore Reed (B89LNTUADM)
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Timestamp=22 Jun 2021 at 21:04:40
Info.plist=not bound
TeamIdentifier=B89LNTUADM
Runtime Version=11.1.0
Sealed Resources=none
Internal requirements count=1 size=168
codesign --verbose=4 -dv osquery/Payload/usr/local/bin/osqueryd
Executable=/private/tmp/osquery/github/osquery/Payload/usr/local/bin/osqueryd
Identifier=osqueryd
Format=Mach-O thin (x86_64)
CodeDirectory v=20500 size=311732 flags=0x10000(runtime) hashes=9733+5 location=embedded
VersionPlatform=1
VersionMin=658432
VersionSDK=721152
Hash type=sha256 size=32
CandidateCDHash sha256=70e03c536569ef97594a762c8f428a69752acf40
CandidateCDHashFull sha256=70e03c536569ef97594a762c8f428a69752acf40220b6ae74cd22c4cb96fcbec
Hash choices=sha256
CMSDigest=70e03c536569ef97594a762c8f428a69752acf40220b6ae74cd22c4cb96fcbec
CMSDigestType=2
Page size=4096
    -5=cf468074bceac0fbab1058e86626b0357adf297c18c7a4fc22929f6644b4f6e0
    -4=0000000000000000000000000000000000000000000000000000000000000000
    -3=0000000000000000000000000000000000000000000000000000000000000000
    -2=eedc00297f1acf7b04a82fa071552093e345ee06d62595e66c766aeeef05665a
CDHash=70e03c536569ef97594a762c8f428a69752acf40
Signature size=9022
Authority=Developer ID Application: OSQUERY A Series of LF Projects, LLC (3522FA9PXF)
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Timestamp=15 Jun 2021 at 03:38:09
Info.plist=not bound
TeamIdentifier=3522FA9PXF
Runtime Version=11.1.0
Sealed Resources=none
Internal requirements count=1 size=168
second one is github
a

allister

07/07/2021, 1:14 PM
it's a difference of an additional ~480 bytes, I'd think the signing would be like-for-like blobs differing
s

Stefano Bonicatti

07/07/2021, 1:14 PM
Also a very quick bin diff shows that there's a big blob at the end of the binary that's different.
I see some additional zero padding at the end of the biggest one
s

seph

07/07/2021, 1:45 PM
Hi! I’ll take a look. It would not surprise me if we used different signing keys. or packaging steps on these to. 4.9.0 was somewhat transitional.
d

Daisukixci

07/07/2021, 1:46 PM
Ok thanks
s

seph

07/07/2021, 1:56 PM
Thank you bringing this up! This was not intentional, but represents a packaging building error specific to 4.9.0. The website one is correct, and I am updating github with it. This happened because we’re in the midst of moving the codesigning to GitHub driven automation. The automated packages are using the new osquery codesigning certificates. But, we consider that a breaking change. So the 4.x process was generally had Teddy go through manual re-sign steps. The github one was not correct re-signed (only the outer package was resigned, not the inner binary) The 5.0 release moves the signing to being automated and based on the osquery certs directly.
✅ 1
a

allister

07/07/2021, 2:16 PM
!m @Daisukixci (you're doing good work, 大好きち!)
(thanks for the explanation!)
d

Daisukixci

07/07/2021, 3:24 PM
Thanks for the explanation
2 Views
#macosJoin Slack
Powered by