https://github.com/osquery/osquery logo
Powered by
#kolide
Title
# kolide
s

Sherwin

12/12/2019, 3:51 PM
Hi All, I have kolide fleet installed to a centos 7 box in azure - what's the best way to install a persistent launcher to my endpoints?
Some more questions related: Some of the documentation mentions to install osquery and then install launcher on top of that - is that correct or am I misinterpreting it? I did a POC running the launcher from the command line and it worked well but it isn't persistent that way - I could maybe make a custom way to launch that process in the background but I imagine this is something already solved - Thanks for your help!
Also - endpoints are windows 10 (later mac os)
s

seph

12/12/2019, 4:01 PM
That is not what the documentation should be saying. You should install either launcher or osquery. (Launcher carries it’s own osquery with it)
The best way is very site dependent. A common way is to use the 
package-builder
tool to build a MSI containing launcher
s

Sherwin

12/12/2019, 4:48 PM
hi @seph - there are prebuilt binaries in "releases" I shouldn't use those? and where's the guide for building an msi? I couldn't really find one which is why I came here - sorry if it's obvious and I just missed it
s

seph

12/12/2019, 5:41 PM
The binary releases are just binaries. You can use them, there’s no MSI there.
Mostly though, you should do what makes sense for your fleet
s

Sherwin

12/12/2019, 5:41 PM
ok but where's the documentation for building the msi -- is there something I'm missing?
the documentation there barely mentions windows - it mentions how to build it for mac
s

seph

12/12/2019, 5:42 PM
There is almost nothing in that document that is platform specific.
s

Sherwin

12/12/2019, 5:43 PM
So to recap, to use 
package-builder
, you must: • Be on macOS • Be able to 
docker run
 something • Build 
package-builder
 from source
😕
s

seph

12/12/2019, 5:43 PM
No.
s

Sherwin

12/12/2019, 5:43 PM
very confusing
so I can build on windows right?
s

seph

12/12/2019, 5:43 PM
The macOS package is built and signed using the pkgbuild command, which is only found on macOS. There are plans to allow for the building of mac packages on Linux but until then, it is a requirement to use a macOS machine when running the package-builder tool.
is misleading. That’s saying you can only build a mac package on macOS. Just as you only build a windows package on windows.
pkgbuild
is not the same as 
package-builder
s

Sherwin

12/12/2019, 5:44 PM
so to use pkgbuild I need to install GO? or does it use a C compiler?
s

seph

12/12/2019, 5:44 PM
pkgbuild
is a macOS command to build mac packages
s

Sherwin

12/12/2019, 5:45 PM
er sorry package-builder
s

seph

12/12/2019, 5:45 PM
package-builder must be built using go. yes. You don’t need docker to build windows packages.
s

Sherwin

12/12/2019, 5:46 PM
I'd be happy to help flesh out that documentation... but I'll probably have a lot more questions along the way
would you be willing to help guide me?
s

seph

12/12/2019, 5:46 PM
Feel free to open a PR with updates! I’m happy to read and consider them
s

Sherwin

12/12/2019, 5:46 PM
PR?
s

seph

12/12/2019, 5:47 PM
“Pull Request” github tool for proposing changes to things
s

Sherwin

12/12/2019, 5:51 PM
ah I see
mind if I PM you while I go through the process? ill compile a guide once I finish creating an msi and deploying to an endpoint
s

seph

12/12/2019, 5:52 PM
I’m not very available to answer questions. I would generally recommend messaging here. I will try to answer, but others may also
s

Sherwin

12/12/2019, 5:52 PM
ok
oh - you're the owner of OSQuery?
s

seph

12/12/2019, 5:55 PM
I am a kolide developer, and a member of the osquery technical steering committee. (So, yes, but I’m one of several)
s

Sherwin

12/12/2019, 5:55 PM
Cool
besides go I wont need any other prerequisites yeah?
I can
I can't seem to run make package-builder... probably because the guide is written for mac or linux 😕
s

sundsta

12/12/2019, 7:13 PM
In order to build the MSI you must build on Windows and have WiX installed. My build command looks like this:
Copy code
./package-builder.exe make \
      --hostname="$(FLEET_HOSTNAME)" \
      --enroll_secret="$(FLEET_ENROLL_SECRET)" \
      --update_channel stable \
      --osquery_version ./osqueryd.exe \
      --launcher_version ./launcher.exe \
      --extension_version ./osquery-extension.exe \
      --notary_url="$(NOTARY_URL)" \
      --mirror_url="$(MIRROR_URL)" \
      --notary_prefix="$(NOTARY_PREFIX)" \
      --targets=windows-service-msi \
      --package_version=$(PACKAGE_VERSION) \
      --output_dir .
s

seph

12/12/2019, 7:15 PM
While I am primarily a unix person, I did not write that to exclude windows. But, it does have a bunch of assumptions. (like the wix install path)
g

groob

12/12/2019, 7:18 PM
It might be more useful if you ask questions like “I’m running 
make package builder
and getting 
Some Error I copy Pasted
, what do I do?” Instead you’re just asking questions that don’t have good answers.
s

Sherwin

12/12/2019, 7:34 PM
make package-builder 'make' is not recognized as an internal or external command, operable program or batch file. not a terribly useful error - I'll check out what sundsta did
so @sundsta can you elaborate - where do I get wix?
s

seph

12/12/2019, 7:36 PM
make
is a common build tool. If you don’t have make installed, you might be able to run the underlying files. But it’s simpler if you have make. I wonder if I should ship package-builder with the launcher releases. “maybe”
wix is… a common open source tool for making packages
s

Sherwin

12/12/2019, 7:37 PM
honestly if there was an MSI already there that I could just feed arguments to that would be ideal
if I was working on this my build server would build major distributable packages that can take command line arguments when called
Id rather learn how to use OSQuery/fleet than have to learn how to build something with WiX which I used to think was a terrible website building tool but apparently something with the same name makes msi files
s

seph

12/12/2019, 7:39 PM
Most people who want an MSI, want it to include the enrollment secret. So they must be built for a site
s

Sherwin

12/12/2019, 7:40 PM
I can easily open a prebuild msi and modify the internal argument though
s

sundsta

12/12/2019, 7:40 PM
You can also just run osqueryd from a shell with the correct arguments to connect it to Fleet without installing it
s

Sherwin

12/12/2019, 7:41 PM
brb
so I'm looking at WiX @sundsta I gotta download visual studio to compile with it?
or would it work with your script out of the box?
@seph couldn't I just grab and MSI prebuilt from you and just edit the secret with Orca?
orca lets you modify msi files internal arguments
s

seph

12/12/2019, 8:23 PM
I do not distribute MSIs without secrets. I understand the desire, but generally I’ve opted to enable this by distributing the tools we use to build MSIs
s

Sherwin

12/12/2019, 8:23 PM
could make it not work unless the secret is changed with something like orca
@seph I guess I'm wondering why - most deployment tools will use an msi and pass in arguments
for example this is how you install tenable agents msiexec /i NessusAgent-<version number>-x64.msi NESSUS_GROUPS="Agent Group Name" NESSUS_SERVER="192.168.0.1:8834" NESSUS_KEY=00abcd00000efgh11111i0k222lmopq3333st4455u66v777777w88xy9999zabc00 /qn
copied from docs.tenable.com
s

seph

12/12/2019, 9:39 PM
To hypothesize…. Building and signing MSIs is work. Most people don’t have that automated. So it’s easier to build a single MSI, and push the work to end users and clients. We took the other route. We build customer specific MSIs. This enables our customers and end users to have a much smoother setup path.
While other patterns are wrong, this is the one we find works for us.
s

Sherwin

12/12/2019, 9:44 PM
other patterns are wrong?
you can even install and link without a key in your settings
s

seph

12/12/2019, 9:45 PM
Ha. Missed a negative.
s

Sherwin

12/12/2019, 9:45 PM
it just seems arbitrary
s

seph

12/12/2019, 9:45 PM
Other patterns are not wrong, But this is what works for us
s

Sherwin

12/12/2019, 9:45 PM
mmm
change it. LOL jk
so is WiX the method I should be using to build?
is that how your team builds msi's?
I'm happy to learn something new but it just seems kinda silly to build an MSI for each company...
either way - todays been a long one
ill tty tomorrow
s

seph

12/12/2019, 9:50 PM
We build an MSI for each customer. I have some amount of automation around that.
How other open source uses do things I cannot say. You could probably build an MSI that does that.
s

sundsta

12/12/2019, 10:32 PM
launcher and osquery both read their configs from a file. If you have automation for deploying the MSI you can probably also automate updating the config on disk after
s

Sherwin

12/13/2019, 6:53 PM
ok so @sundsta - you got like a set of steps I can follow? or atleast where you learned how to use wix?
a bit lost lol
there seems to be a wix folder in packagekit
I started a new thread here: https://osquery.slack.com/archives/C1XCLA5DZ/p1576264327004300 so that if you ended up here you can go here to see replies on how to build using wix or other
@seph I have no idea where package-builder.exe resides - I've seen package-builder.go in the CMD page
er cmd folder
s

seph

12/16/2019, 7:32 PM
s

Sherwin

12/16/2019, 7:39 PM
PS C:\Users\Security\Downloads\launcher-master> make package-builder ./build/package-builder make --help make : The term 'make' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. At line:1 char:1 + make package-builder + ~~ + CategoryInfo : ObjectNotFound: (make:String) [], CommandNotFoundException + FullyQualifiedErrorId : CommandNotFoundException ./build/package-builder : The term './build/package-builder' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. At line:2 char:1 + ./build/package-builder make --help + ~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : ObjectNotFound: (./build/package-builder:String) [], CommandNotFoundException + FullyQualifiedErrorId : CommandNotFoundExceptionPS C:\Users\Security\Downloads\launcher-master> make package-builder ./build/package-builder make --help make : The term 'make' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. At line:1 char:1 + make package-builder + ~~ + CategoryInfo : ObjectNotFound: (make:String) [], CommandNotFoundException + FullyQualifiedErrorId : CommandNotFoundException ./build/package-builder : The term './build/package-builder' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. At line:2 char:1 + ./build/package-builder make --help + ~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : ObjectNotFound: (./build/package-builder:String) [], CommandNotFoundException + FullyQualifiedErrorId : CommandNotFoundException
g

groob

12/16/2019, 7:46 PM
You need make
s

Sherwin

12/16/2019, 7:47 PM
where do I get that
do I install visual studio?
g

groob

12/16/2019, 7:50 PM
chocolatey maybe? I’d google “how to install make on windows”
to be honest, if the make message is not self evident, the rest of the build is likely going to be very hard
s

Sherwin

12/16/2019, 7:51 PM
so I've used make - when I compile C code
and I've used make on linux
Powered by