hey team, i'm testing out orbit's ability to build .pkg files on a mac (running catalina 10.15.x specifically in case that matters). I'm able to generate the .pkg file, but after installing it on the mac it isn't registering the endpoint w/ the fleet server. I've run debug w/ the

go run ./cmd/package

command when building the package to trace the issue. OSQueryi is already installed correctly in this case, however I haven't configured OSQueryd yet on the machine, in an effort to simulate how the installation would look for a new endpoint being enrolled. Here's the error I'm getting; any thoughts on what I'm doing wrong?

DBG stat file error="stat /tmp/orbit-package645165662/root/var/lib/orbit/bin/osqueryd/macos/stable/osqueryd: no such file or directory"

This is an error during packaging?

yes correct. Here's what I was running when I received that error referenced

go run ./cmd/package --type=pkg --fleet-url=<myserver> --insecure --enroll-secret=<mysecret> --debug

That should be more of an informational message (debug level)

hey @zwass, yes. thanks for reviewing it for me.

2021-05-26T08:41:40-06:00 INF Failed to connect to Fleet server. Osquery connection may fail. error="dial for validate: verify certificate: x509: certificate signed by unknown authority"

2021-05-26T08:41:40-06:00 INF using insecure TLS proxy addr=localhost:62974 target=<my_fleet_server_domainname>:443

2021-05-26T08:41:40-06:00 INF run osqueryd cmd="/var/lib/orbit/bin/osqueryd/macos/stable/osqueryd --pidfile=/var/lib/orbit/osquery.pid --database_path=/var/lib/orbit/osquery.db --extensions_socket=/var/lib/orbit/osquery.em --enroll_secret_env=ENROLL_SECRET --tls_hostname=localhost:62974 --enroll_tls_endpoint=/api/v1/osquery/enroll --config_plugin=tls --config_tls_endpoint=/api/v1/osquery/config --config_refresh=60 --disable_distributed=false --distributed_plugin=tls --distributed_tls_max_attempts=10 --distributed_tls_read_endpoint=/api/v1/osquery/distributed/read --distributed_tls_write_endpoint=/api/v1/osquery/distributed/write --logger_plugin=tls --logger_tls_endpoint=/api/v1/osquery/log --disable_carver=false --carver_start_endpoint=/api/v1/osquery/carve/begin --carver_continue_endpoint=/api/v1/osquery/carve/block --carver_block_size=2000000 --tls_server_certs /tmp/fleet.crt --force"

W0526 08:41:41.190591 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...

W0526 08:41:42.422358 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...

W0526 08:41:46.701145 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...

W0526 08:41:47.917557 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...

W0526 08:41:53.411298 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...

W0526 08:41:58.960552 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...

2021-05-26T10:41:53-06:00 INF update failed error="update metadata: update metadata: tuf: failed to download timestamp.json: Get \"<https://tuf.fleetctl.com/timestamp.json>\": dial tcp: lookup <http://tuf.fleetctl.com|tuf.fleetctl.com> on 192.168.1.1:53: write udp 192.168.1.32:61238->192.168.1.1:53: write: network is down"

2021/05/26 10:41:55 http: proxy error: dial tcp: lookup <my_fleet_server_domainname> on 192.168.1.1:53: dial udp 192.168.1.1:53: connect: network is unreachable

2021/05/26 10:41:56 http: proxy error: dial tcp: lookup <my_fleet_server_domainname> on 192.168.1.1:53: dial udp 192.168.1.1:53: connect: network is unreachable

2021/05/26 10:41:56 http: proxy error: dial tcp: lookup <my_fleet_server_domainname> on 192.168.1.1:53: dial udp 192.168.1.1:53: connect: network is unreachable

W0526 10:41:56.474023 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...

W0526 10:41:58.437203 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...

W0526 10:42:02.752142 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...

W0526 10:42:03.836588 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...

E0526 10:42:08.139662 361979328 shutdown.cpp:69] Cannot activate tls logger plugin: No node key, TLS logging disabled.

W0526 10:42:08.217161 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...

W0526 10:42:09.440047 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...

W0526 10:42:13.730883 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...

W0526 10:42:14.941220 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...

I0526 10:42:19.282280 361979328 eventfactory.cpp:156] Event publisher not enabled: openbsm: Publisher disabled via configuration

I0526 10:42:19.282408 361979328 eventfactory.cpp:156] Event publisher not enabled: scnetwork: Publisher not used

I0526 10:42:19.282495 361979328 eventfactory.cpp:156] Event publisher not enabled: event_tapping: Publisher disabled via configuration

E0526 10:42:22.985704 17145856 shutdown.cpp:69] Worker returned exit status

2021-05-26T10:42:23-06:00 ERR unexpected exit error="osqueryd exited with error: exit status 78"

2021/05/26 10:42:23 WARNING: proto: file "pb.proto" is already registered

A future release will panic on registration conflicts. See:

<https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict>

I see some network errors in there. Are there logs in the Fleet server when you do that? Can you

curl

the Fleet server

/api/v1/osquery/enroll

endpoint successfully from that machine?

would that be osquery logs specifically on the fleetserver or another fleet server log specifically? when I try & curl that endpoint you listed, I don't get any response from the fleetserver

In my first question I was referring to the

stderr

logs of the Fleet server. Can you please paste the

curl

command and output you got (redacting any URLs if necessary)?

Hey @zwass, thanks for the follow up on this. I was able to figure it out after checking out those logs. Many thanks!