I've been struggling with a launcher/osquery problem for a few days now. We have a few Server 2019 systems that this is happening on, and from what I can tell it appears that launcher can't communicate with the osquery process, but I'm definitely no expert at reading the debug information. I'll post more in replies.

That’s a weird

--root_directory

— it’s not the root of the launcher install, it should be the data directory where launcher and osquery write their db to

Thank you Seph those are good observations and I'll give those things a try and get back to you here.

Honestly it's been awhile since I've done that. What's the error?

Here's the output that I capture in a file. This looks pretty similar on both systems but I'm going to compare a little closer. The output I'm sending you here is from the system where launcher is failing.

I0527 19:57:25.045593  5312 init.cpp:342] osquery initialized [version=4.7.0]
I0527 19:57:25.076864  5312 system.cpp:342] Found stale process for osqueryd (4076)
I0527 19:57:25.076864  5312 system.cpp:374] Writing osqueryd pid (8712) to C:\Program Files\Kolide\Launcher-launcher\data\osquery.pid
I0527 19:57:25.076864  5312 extensions.cpp:438] Found autoloadable extension: C:\Program Files\Kolide\Launcher-launcher\bin\osquery-extension.exe
I0527 19:57:25.076864  5312 dispatcher.cpp:78] Adding new service: WatcherRunner (0000014492AFD010) to thread: 4548 (0000014492AF7AC0) in process 8712
I0527 19:57:25.076864  5312 rocksdb.cpp:132] Opening RocksDB handle: C:\Program Files\Kolide\Launcher-launcher\data\osquery.db
I0527 19:57:25.092486  4548 watcher.cpp:659] Created and monitoring extension child (4160): C:\Program Files\Kolide\Launcher-launcher\bin\osquery-extension.exe
[C:\Program Files\Kolide\Launcher-launcher\bin\osquery-extension.exe --verbose --socket \\.\pipe\kolide-osquery-01F6QZD64HQ6J03NNP01ZREWSK --timeout 10 --interval 3]I0527 19:57:25.352573  5312 dispatcher.cpp:78] Adding new service: ExtensionWatcher (0000014492B159E0) to thread: 7896 (0000014492B56630) in process 8712
I0527 19:57:25.352573  5312 dispatcher.cpp:78] Adding new service: ExtensionRunnerCore (0000014492AB18F0) to thread: 2504 (0000014492B56530) in process 8712
I0527 19:57:25.352573  2504 interface.cpp:270] Extension manager service starting: \\.\pipe\kolide-osquery-01F6QZD64HQ6J03NNP01ZREWSK
E0527 19:57:40.675662  5312 shutdown.cpp:69] Cannot activate kolide_grpc config plugin: Unknown registry plugin: kolide_grpc
I0527 19:57:40.675662  5312 auto_constructed_tables.cpp:97] Removing stale ATC entries
W0527 19:57:40.675662  5312 init.cpp:589] Error reading config: Missing config plugin 
E0527 19:57:56.008266  5312 shutdown.cpp:69] Cannot activate kolide_grpc logger plugin: Unknown registry plugin: kolide_grpc
E0527 19:58:11.259007  5312 shutdown.cpp:69] Cannot activate kolide_grpc distributed plugin: Unknown registry plugin: kolide_grpc
I0527 19:58:11.259007  5312 eventfactory.cpp:156] Event publisher not enabled: ntfs_event_publisher: NTFS event publisher disabled via configuration
I0527 19:58:11.259007  5312 events.cpp:36] Skipping subscriber: powershell_events: Required publisher is disabled by configuration
I0527 19:58:11.259007  5312 events.cpp:36] Skipping subscriber: windows_events: Required publisher is disabled by configuration
I0527 19:58:11.259007  5260 eventfactory.cpp:391] Starting event publisher run loop: WindowsEventLogPublisher
I0527 19:58:11.259007  5260 eventfactory.cpp:411] Event publisher WindowsEventLogPublisher run loop terminated for reason: Publisher disabled by configuration
I0527 19:58:11.259007  5312 dispatcher.cpp:149] Thread: 5312 requesting a stop
I0527 19:58:11.259007  5312 dispatcher.cpp:156] Service: 0000014492AFD010 has been interrupted
I0527 19:58:11.259007  5312 dispatcher.cpp:156] Service: 0000014492B159E0 has been interrupted
I0527 19:58:11.259007  5312 dispatcher.cpp:156] Service: 0000014492AB18F0 has been interrupted
I0527 19:58:11.259007  5312 dispatcher.cpp:122] Thread: 5312 requesting a join
I0527 19:58:11.259007  5312 dispatcher.cpp:140] Service thread: 0000014492B56530 has joined
I0527 19:58:11.259007  5312 dispatcher.cpp:140] Service thread: 0000014492B56630 has joined
I0527 19:58:11.259007  5312 dispatcher.cpp:140] Service thread: 0000014492AF7AC0 has joined
I0527 19:58:11.259007  5312 dispatcher.cpp:144] Services and threads have been cleared

Hrm. It may be harder to make it just work. Some of that is coming in via launcher

That's sort of what I thought. Maybe I can get some debug going on osquery and run it through launcher.

You’re already getting that — the osquery args include

--verbose

.

Ok, gotcha. Well, there's got to be some difference between these systems. It seems like these might be newer OS's that I've deployed, so I'll continue to hunt for differences.

I’d generally bet something in permissions. But I’m still surprised there’s no logging.

Good ideas, thanks.

I keep wondering what’s up with

Extension socket not available: \\\\.\\pipe\\kolide-osquery-01F6QPMW4JVZVQZAWQS94WTCEJ.30828"

That path is, AFAIK, reasonable

\\.\pipe\<whatever>

is where windows keeps named pipes. And the rest is a random ulid to ensure uniqueness. So, why would that fail? It’s weird

I think you may be on to something with the pipe. If I look at a working system, there are two pipes:

\\.\\pipe\\kolide-osquery-01F6RS33G9Y895NC1HWN79ZDAD
\\.\\pipe\\kolide-osquery-01F6RS33G9Y895NC1HWN79ZDAD.13599

Note that one of them has the 5 digit extension. If I watch the pipe list on a non-working system right after I start the launcher service, I only see 1 pipe:

\\.\\pipe\\kolide-osquery-01F6RTP7R619269W0ZS6ZWDVAN

Is there any significance to that? It seems to be the pipe name with the 5 digit extension that the error message indicates launcher can't connect to, and on my non-working systems that pipe name is never created.