Title
#fleet
d

David Edwards

05/25/2021, 9:25 PM
hey team, i'm testing out orbit's ability to build .pkg files on a mac (running catalina 10.15.x specifically in case that matters). I'm able to generate the .pkg file, but after installing it on the mac it isn't registering the endpoint w/ the fleet server. I've run debug w/ the
go run ./cmd/package
command when building the package to trace the issue. OSQueryi is already installed correctly in this case, however I haven't configured OSQueryd yet on the machine, in an effort to simulate how the installation would look for a new endpoint being enrolled. Here's the error I'm getting; any thoughts on what I'm doing wrong?
DBG stat file error="stat /tmp/orbit-package645165662/root/var/lib/orbit/bin/osqueryd/macos/stable/osqueryd: no such file or directory"
zwass

zwass

05/25/2021, 9:50 PM
This is an error during packaging?
d

David Edwards

05/25/2021, 9:52 PM
yes correct. Here's what I was running when I received that error referenced
go run ./cmd/package --type=pkg --fleet-url=<myserver> --insecure --enroll-secret=<mysecret> --debug
zwass

zwass

05/25/2021, 9:53 PM
That should be more of an informational message (debug level)
9:53 PM
Do you have any logs in
/var/log/orbit/orbit.stderr.log
?
d

David Edwards

05/26/2021, 6:42 PM
hey @zwass, yes. thanks for reviewing it for me.
2021-05-26T08:41:40-06:00 INF Failed to connect to Fleet server. Osquery connection may fail. error="dial for validate: verify certificate: x509: certificate signed by unknown authority"
2021-05-26T08:41:40-06:00 INF using insecure TLS proxy addr=localhost:62974 target=<my_fleet_server_domainname>:443
2021-05-26T08:41:40-06:00 INF run osqueryd cmd="/var/lib/orbit/bin/osqueryd/macos/stable/osqueryd --pidfile=/var/lib/orbit/osquery.pid --database_path=/var/lib/orbit/osquery.db --extensions_socket=/var/lib/orbit/osquery.em --enroll_secret_env=ENROLL_SECRET --tls_hostname=localhost:62974 --enroll_tls_endpoint=/api/v1/osquery/enroll --config_plugin=tls --config_tls_endpoint=/api/v1/osquery/config --config_refresh=60 --disable_distributed=false --distributed_plugin=tls --distributed_tls_max_attempts=10 --distributed_tls_read_endpoint=/api/v1/osquery/distributed/read --distributed_tls_write_endpoint=/api/v1/osquery/distributed/write --logger_plugin=tls --logger_tls_endpoint=/api/v1/osquery/log --disable_carver=false --carver_start_endpoint=/api/v1/osquery/carve/begin --carver_continue_endpoint=/api/v1/osquery/carve/block --carver_block_size=2000000 --tls_server_certs /tmp/fleet.crt --force"
W0526 08:41:41.190591 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...
W0526 08:41:42.422358 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...
W0526 08:41:46.701145 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...
W0526 08:41:47.917557 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...
W0526 08:41:53.411298 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...
W0526 08:41:58.960552 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...
2021-05-26T10:41:53-06:00 INF update failed error="update metadata: update metadata: tuf: failed to download timestamp.json: Get \"<https://tuf.fleetctl.com/timestamp.json>\": dial tcp: lookup <http://tuf.fleetctl.com|tuf.fleetctl.com> on 192.168.1.1:53: write udp 192.168.1.32:61238->192.168.1.1:53: write: network is down"
2021/05/26 10:41:55 http: proxy error: dial tcp: lookup <my_fleet_server_domainname> on 192.168.1.1:53: dial udp 192.168.1.1:53: connect: network is unreachable
2021/05/26 10:41:56 http: proxy error: dial tcp: lookup <my_fleet_server_domainname> on 192.168.1.1:53: dial udp 192.168.1.1:53: connect: network is unreachable
2021/05/26 10:41:56 http: proxy error: dial tcp: lookup <my_fleet_server_domainname> on 192.168.1.1:53: dial udp 192.168.1.1:53: connect: network is unreachable
W0526 10:41:56.474023 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...
W0526 10:41:58.437203 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...
W0526 10:42:02.752142 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...
W0526 10:42:03.836588 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...
E0526 10:42:08.139662 361979328 shutdown.cpp:69] Cannot activate tls logger plugin: No node key, TLS logging disabled.
W0526 10:42:08.217161 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...
W0526 10:42:09.440047 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...
W0526 10:42:13.730883 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...
W0526 10:42:14.941220 361979328 tls_enroll.cpp:77] Failed enrollment request to <https://localhost:62974/api/v1/osquery/enroll> (No node key returned from TLS enroll plugin) retrying...
I0526 10:42:19.282280 361979328 eventfactory.cpp:156] Event publisher not enabled: openbsm: Publisher disabled via configuration
I0526 10:42:19.282408 361979328 eventfactory.cpp:156] Event publisher not enabled: scnetwork: Publisher not used
I0526 10:42:19.282495 361979328 eventfactory.cpp:156] Event publisher not enabled: event_tapping: Publisher disabled via configuration
E0526 10:42:22.985704 17145856 shutdown.cpp:69] Worker returned exit status
2021-05-26T10:42:23-06:00 ERR unexpected exit error="osqueryd exited with error: exit status 78"
2021/05/26 10:42:23 WARNING: proto: file "pb.proto" is already registered
A future release will panic on registration conflicts. See:
<https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict>
zwass

zwass

05/26/2021, 10:17 PM
I see some network errors in there. Are there logs in the Fleet server when you do that? Can you
curl
the Fleet server
/api/v1/osquery/enroll
endpoint successfully from that machine?
d

David Edwards

05/27/2021, 9:25 PM
would that be osquery logs specifically on the fleetserver or another fleet server log specifically? when I try & curl that endpoint you listed, I don't get any response from the fleetserver
zwass

zwass

05/27/2021, 9:38 PM
In my first question I was referring to the
stderr
logs of the Fleet server. Can you please paste the
curl
command and output you got (redacting any URLs if necessary)?
d

David Edwards

06/01/2021, 2:09 PM
Hey @zwass, thanks for the follow up on this. I was able to figure it out after checking out those logs. Many thanks!
🍻 1