Title
#kolide
l

Lawrence D'Anna

04/02/2020, 10:13 PM
:\Program Files\osquery>launcher.exe --hostname="<http://osquery-dev-fleet.com:8080|osquery-dev-fleet.com:8080>" --root_directory="C:\ProgramData\osquery" --enroll_secret=foobarbaz --insecure
{"caller":"main.go:26","msg":"Launcher starting up","revision":"6ff84fba146ed3d2070faa30bd4947b2e16d7072","severity":"info","ts":"2020-04-02T22:07:25.2657548Z","version":"0.11.9"}
{"caller":"main.go:57","msg":"Nothing new","severity":"info","ts":"2020-04-02T22:07:25.266744Z"}
{"caller":"client_grpc.go:111","cert_pinning":false,"msg":"dialing grpc server","server":"<http://osquery-dev-fleet.com:8080|osquery-dev-fleet.com:8080>","severity":"info","tls_secure":false,"transport_secure":true,"ts":"2020-04-02T22:07:25.2707474Z"}
{"build":"6ff84fba146ed3d2070faa30bd4947b2e16d7072","caller":"launcher.go:158","msg":"started kolide launcher","severity":"info","ts":"2020-04-02T22:07:25.2848206Z","version":"0.11.9"}
{"caller":"query_target_updater.go:21","msg":"query target updater started","severity":"info","ts":"2020-04-02T22:07:25.2848206Z"}
{"arg0":"osqueryd.exe","args":"osqueryd.exe --pidfile=C:\\ProgramData\\osquery\\osquery.pid --database_path=C:\\ProgramData\\osquery\\osquery.db --extensions_socket=\\\\.\\pipe\\kolide.em --extensions_autoload=C:\\ProgramData\\osquery\\osquery.autoload --extensions_timeout=10 --config_plugin=kolide_grpc --logger_plugin=kolide_grpc --distributed_plugin=kolide_grpc --disable_distributed=false --distributed_interval=5 --pack_delimiter=: --host_identifier=uuid --force=true --disable_watchdog --utc --config_refresh=300 --config_accelerated_refresh=30 --allow_unsafe","caller":"runtime.go:546","msg":"launching osqueryd","severity":"info","ts":"2020-04-02T22:07:25.2867445Z"}
{"caller":"init.cpp:509","component":"osquery","level":"stderr","msg":"E0402 15:07:25.349478 12040 init.cpp:509] Cannot activate kolide_grpc config plugin: Unknown registry plugin: kolide_grpc","severity":"info","ts":"2020-04-02T22:07:25.364451Z"}
{"caller":"init.cpp:596","component":"osquery","level":"stderr","msg":"W0402 15:07:25.364450 12040 init.cpp:596] Error reading config: Missing config plugin","severity":"info","ts":"2020-04-02T22:07:25.364451Z"}
{"caller":"init.cpp:509","component":"osquery","level":"stderr","msg":"E0402 15:07:25.364450 12040 init.cpp:509] Cannot activate kolide_grpc logger plugin: Unknown registry plugin: kolide_grpc\r\nE0402 15:07:25.364450 12040 init.cpp:509] Cannot activate kolide_grpc distributed plugin: Unknown registry plugin: kolide_grpc\r\nI0402 15:07:25.364450 12040 events.cpp:863] Event publisher not enabled: ntfs_event_publisher: NTFS event publisher disabled via configuration","severity":"info","ts":"2020-04-02T22:07:25.3684534Z"}
{"caller":"","component":"osquery","level":"stderr","msg":"T","severity":"info","ts":"2020-04-02T22:07:25.3714472Z"}
{"caller":"","component":"osquery","level":"stderr","msg":"hrift: Thu Apr  2 15:07:25 2020 TPipeServer ConnectNamedPipe GLE=errno = 995","severity":"info","ts":"2020-04-02T22:07:25.3714472Z"}
{"caller":"runtime.go:585","err":"exit status 78","mode":"-rw-rw-rw-","msg":"Error running osquery command","path":"osqueryd.exe","severity":"info","sha256":"4dbf2babae608e4eea7d6cc97dbf2affa7ba3f83626b58c7f0937790737a99b7","sizeBytes":11177984,"ts":"2020-04-02T22:07:25.8488886Z"}
{"caller":"launcher.go:125","err":"launching osquery instance: starting instance: could not create extension manager server at \\\\.\\pipe\\kolide.em: dialing pipe '\\\\.\\pipe\\kolide.em': open \\\\.\\pipe\\kolide.em: The system cannot find the file specified.","msg":"interrupted","severity":"info","ts":"2020-04-02T22:07:35.2926968Z"}
{"caller":"query_target_updater.go:26","msg":"query target updater interrupted","severity":"info","ts":"2020-04-02T22:07:35.2926968Z"}
{"caller":"launcher.go:121","msg":"beginnning shutdown via signal","severity":"info","ts":"2020-04-02T22:07:35.2926968Z"}
{"caller":"extension.go:135","err":"launching osquery instance: starting instance: could not create extension manager server at \\\\.\\pipe\\kolide.em: dialing pipe '\\\\.\\pipe\\kolide.em': open \\\\.\\pipe\\kolide.em: The system cannot find the file specified.","msg":"extension interrupted","severity":"info","ts":"2020-04-02T22:07:35.2966917Z"}
{"caller":"extension.go:140","err":"while shutting down instance: running osqueryd command: exit status 78","msg":"error shutting down runtime","severity":"info","ts":"2020-04-02T22:07:35.2986922Z"}
{"caller":"logutil.go:13","run service: launching osquery instance: starting instance: could not create extension manager server at \\\\.\\pipe\\kolide.em: dialing pipe '\\\\.\\pipe\\kolide.em': open \\\\.\\pipe\\kolide.em: The system cannot find the file specified.":"run launcher","severity":"info","ts":"2020-04-02T22:07:35.300691Z"}
zwass

zwass

04/02/2020, 10:55 PM
Possibly a permissions issue?
l

Lawrence D'Anna

04/02/2020, 11:29 PM
possibly, but the really strange thign is that sometimes it happens and sometimes it does not
11:29 PM
same machine, same terminal window
11:30 PM
just, sometimes it works but usually it does that
s

seph

04/03/2020, 1:06 AM
A couple of notes…* --root_directory=“C:\ProgramData\osquery` seems wrong. Strongly recommend a dedicated directory for launcher’s root, not something that might be shared with osquery like that. *
--insecure
is, well, insecure. Not recommended for production* I’ve seen errors like
Unknown registry plugin: kolide_grpc
if you’re running multiple launchers. You should be able to run more than one, as long as they have different root directories
1:06 AM
But there’s been something kinda weird on windows, where that pops up. It feels like something about a conflicting path, but I haven’t dug into it
1:07 AM
(er, by conflicting path. I mean multiple launchers fighting. And if you’re on windows, it’s possuble one of them is some kind of stale process holding a pipe open)
l

Lawrence D'Anna

04/03/2020, 1:08 AM
hrm. there’s just three proceseess right, launcher, osqeryd, and osquery-extensions?
s

seph

04/03/2020, 1:09 AM
Correct.
1:09 AM
(though you might see multiple launchers running bewcause that’s how the update system works)
l

Lawrence D'Anna

04/03/2020, 1:11 AM
that’s what’s weird, i don’t see any of those three processes hanging around. It’s not the root_directory either, same thing happens if i set it to “c:\foobar” or anything else
s

seph

04/03/2020, 1:11 AM
What does rebooting get you?
l

Lawrence D'Anna

04/03/2020, 1:14 AM
hrm, not sure. it’s a shared machine so I haven’t tried that. I’ll give it a try.
1:15 AM
thought maybe the named pipe was still open somehow but that doesn’t seem to be the case either
s

seph

04/03/2020, 1:15 AM
Yeah, that’s the bit that seems most suspicious. But I thought that pipe was in the launcher root directory
zwass

zwass

04/03/2020, 1:16 AM
Pipes are in sort of their own namespace in Windows
1:16 AM
\.\pipe*
l

Lawrence D'Anna

04/03/2020, 1:16 AM
yea
s

seph

04/03/2020, 1:17 AM
@zwass … So that means that pipe is (a) globally unique to the machine, and not rooted in the launcher db?
zwass

zwass

04/03/2020, 1:18 AM
Yes
s

seph

04/03/2020, 1:18 AM
And probably something about how if something leaves it open, shit breaks
1:18 AM
Okay, I think there’s an obvious bug about making the name launcher_db dependant. Thanks for pointing that out to me.
1:18 AM
But I’m less sure about the issue here — I bet something leaves it open.
zwass

zwass

04/03/2020, 1:19 AM
Possibly, but in theory shouldn't you be able to see the pipe is open if that's the case?
s

seph

04/03/2020, 1:19 AM
I don’t know how. Maybe @Lawrence D'Anna does
l

Lawrence D'Anna

04/03/2020, 1:20 AM
[System.IO.Directory]::GetFiles("\\.\\pipe\\")
1:20 AM
in powershell
1:20 AM
but
kolide.em
isn’t in there
s

seph

04/03/2020, 1:25 AM
I made https://github.com/kolide/launcher/issues/598 and https://github.com/kolide/launcher/issues/597 for the two things I just mentioned. Not sure I have any clever ideas on the pipe issue
1:26 AM
I wonder if you can start osqueryd with the same command line launcher is using, and see what it gets you.
1:27 AM
Digging a bit, this is probably bubbling up from the go sdk
l

Lawrence D'Anna

04/03/2020, 1:28 AM
How can I get the command line it is using
s

seph

04/03/2020, 1:29 AM
It’s in the log output you pasted 🙂
1:29 AM
{
  "arg0": "osqueryd.exe",
  "args": "osqueryd.exe --pidfile=C:\\ProgramData\\osquery\\osquery.pid --database_path=C:\\ProgramData\\osquery\\osquery.db --extensions_socket=\\\\.\\pipe\\kolide.em --extensions_autoload=C:\\ProgramData\\osquery\\osquery.autoload --extensions_timeout=10 --config_plugin=kolide_grpc --logger_plugin=kolide_grpc --distributed_plugin=kolide_grpc --disable_distributed=false --distributed_interval=5 --pack_delimiter=: --host_identifier=uuid --force=true --disable_watchdog --utc --config_refresh=300 --config_accelerated_refresh=30 --allow_unsafe",
  "caller": "runtime.go:546",
  "msg": "launching osqueryd",
  "severity": "info",
  "ts": "2020-04-02T22:07:25.2867445Z"
}
1:29 AM
I don’t remember, maybe @zwass does. Does launcher create the pipe, or does osquery?
l

Lawrence D'Anna

04/03/2020, 1:36 AM
Ya it says named pipe path is invalid if i just launch osqueryd
1:42 AM
oh wait that's wrong I just had the backslashes messed up
1:42 AM
it just gives the same error as launcher does
1:58 AM
what make do you use to build it on windows?
s

seph

04/03/2020, 1:59 AM
I wonder if theres a permissions issue. Like if administrator runs launcher, creates the pipe, then a uer tries to do it later?
1:59 AM
windows is cross compiled from a mac, so I use a standard gnu make.
l

Lawrence D'Anna

04/03/2020, 2:00 AM
hah oh wow
2:00 AM
i don't know much about go
2:01 AM
um, i don't suppose there are instructions on how to build it somewhere?
s

seph

04/03/2020, 2:02 AM
On windows, or cross compiled from mac or linux?
l

Lawrence D'Anna

04/03/2020, 2:02 AM
any of the above
2:03 AM
🙂
l

Lawrence D'Anna

04/03/2020, 2:03 AM
thanks
s

seph

04/03/2020, 2:04 AM
It’s pretty simple though. On a machine with go and make, run
make -j xp
(or look at that file for some more specific target). You should end up with windows binaries in
./build/windows
2:04 AM
You should be able to run it in docker. using the official go container
2:39 AM
I stared at this a bit more, and concluded pipes should probably have random names. https://github.com/kolide/launcher/pull/599 if you’re curious