https://github.com/osquery/osquery logo
Join Slack
Powered by
trying to enroll a container with osquery on it in...
# fleet
p
trying to enroll a container with osquery on it into Fleet but this error occurs when running 
osqueryd
and the container never gets enrolled successfully. Any ideas why? 🤔 (btw same error occurs upon running 
osqueryi
as well)
Copy code
# osqueryd --verbose --tls_dump
I0426 15:59:05.960875   265 init.cpp:357] osquery initialized [version=5.2.3]
I0426 15:59:05.960927   265 init.cpp:364] Using default flagfile: /etc/osquery/osquery.flags.default
I0426 15:59:05.979229   265 system.cpp:354] Found stale process for osqueryd (157)
I0426 15:59:05.979316   265 system.cpp:386] Writing osqueryd pid (265) to /var/run/osqueryd.pidfile
I0426 15:59:05.979460   265 extensions.cpp:453] Could not autoload extensions: Cannot open file for reading: /etc/osquery/extensions.load
I0426 15:59:05.979648   265 dispatcher.cpp:78] Adding new service: WatcherRunner (0x55ba6f2c72d8) to thread: 140231381698112 (0x55ba6f2b4dc0) in process 265
I0426 15:59:05.980381   266 watcher.cpp:656] osqueryd watcher (265) executing worker (267)
I0426 15:59:05.986223   267 init.cpp:354] osquery worker initialized [watcher=265]
I0426 15:59:05.986275   267 init.cpp:364] Using default flagfile: /etc/osquery/osquery.flags.default
I0426 15:59:05.986357   267 dispatcher.cpp:78] Adding new service: WatcherWatcherRunner (0x5567575220d8) to thread: 140552803788352 (0x55675751a9d0) in process 267
I0426 15:59:05.986425   267 rocksdb.cpp:132] Opening RocksDB handle: /var/osquery/osquery.db
I0426 15:59:06.008972   267 dispatcher.cpp:78] Adding new service: ExtensionWatcher (0x5567575fef38) to thread: 140552259495488 (0x556757608310) in process 267
I0426 15:59:06.009070   267 dispatcher.cpp:78] Adding new service: ExtensionRunnerCore (0x5567575fed18) to thread: 140552267888192 (0x556757529ee0) in process 267
I0426 15:59:06.009122   366 interface.cpp:299] Extension manager service starting: /var/osquery/osquery.em
E0426 15:59:06.009130   267 shutdown.cpp:79] Cannot activate tls && --enroll_secret_path=/etc/osquery/osquery.secret && --enroll_tls_endpoint=/api/v1/osquery/enroll&& --config_tls_endpoint=/api/v1/osquery/config && --tls_hostname=<http://fleetdm.segment.com|fleetdm.segment.com> && --config_refresh=300 && --config_tls_accelerated_refresh=300 && --config_tls_max_attempts=9999 && --distributed_tls_read_endpoint=/api/v1/osquery/distributed/read && --distributed_tls_write_endpoint=/api/v1/osquery/distributed/write && --carver_start_endpoint=/api/v1/osquery/carve/begin && --carver_continue_endpoint=/api/v1/osquery/carve/block config plugin: Unknown registry plugin: tls && --enroll_secret_path=/etc/osquery/osquery.secret && --enroll_tls_endpoint=/api/v1/osquery/enroll&& --config_tls_endpoint=/api/v1/osquery/config && --tls_hostname=<http://fleetdm.segment.com|fleetdm.segment.com> && --config_refresh=300 && --config_tls_accelerated_refresh=300 && --config_tls_max_attempts=9999 && --distributed_tls_read_endpoint=/api/v1/osquery/distributed/read && --distributed_tls_write_endpoint=/api/v1/osquery/distributed/write && --carver_start_endpoint=/api/v1/osquery/carve/begin && --carver_continue_endpoint=/api/v1/osquery/carve/block
I0426 15:59:06.009331   267 dispatcher.cpp:149] Thread: 140552803806784 requesting a stop
I0426 15:59:06.009389   267 dispatcher.cpp:156] Service: 0x5567575220d8 has been interrupted
I0426 15:59:06.009459   267 dispatcher.cpp:156] Service: 0x5567575fef38 has been interrupted
I0426 15:59:06.009536   267 dispatcher.cpp:156] Service: 0x5567575fed18 has been interrupted
I0426 15:59:06.009622   267 dispatcher.cpp:122] Thread: 140552803806784 requesting a join
I0426 15:59:06.010021   267 dispatcher.cpp:140] Service thread: 0x556757529ee0 has joined
I0426 15:59:06.010056   267 dispatcher.cpp:140] Service thread: 0x556757608310 has joined
I0426 15:59:06.010100   267 dispatcher.cpp:140] Service thread: 0x55675751a9d0 has joined
I0426 15:59:06.010123   267 dispatcher.cpp:144] Services and threads have been cleared
E0426 15:59:08.981992   266 shutdown.cpp:79] Worker returned exit status
I0426 15:59:08.983098   265 dispatcher.cpp:149] Thread: 140231381716544 requesting a stop
I0426 15:59:08.983296   265 dispatcher.cpp:122] Thread: 140231381716544 requesting a join
I0426 15:59:08.983441   265 dispatcher.cpp:140] Service thread: 0x55ba6f2b4dc0 has joined
I0426 15:59:08.983824   265 dispatcher.cpp:144] Services and threads have been cleared
It's an Ubuntu container and this is what the Dockerfile looks like
Copy code
FROM ubuntu:22.04

RUN apt update && apt-get install --yes \
    curl \
    wget \
    software-properties-common

# Add Osquery Binary
#Adding GPG settings
RUN gpg_key_url="<https://packagecloud.io/segment/infra/gpgkey>" \
    && gpg_keyring_path="/usr/share/keyrings/segment_infra-archive-keyring.gpg" \
    && echo -n "Importing packagecloud gpg key... " \
    && curl -fsSL "${gpg_key_url}" | gpg --dearmor > ${gpg_keyring_path} \
    && echo "done."

# Install Osquery
RUN apt-key adv --keyserver <http://keyserver.ubuntu.com|keyserver.ubuntu.com> --recv-keys 1484120AC4E9F8A1A577AEEE97A80C63C9D8B80B \
    && add-apt-repository "deb [arch=amd64] <https://osquery-packages.s3.amazonaws.com/deb> deb main" \
    && apt-get install osquery

RUN rm -f /etc/osquery/osquery.secret \
    && rm -f /etc/osquery/osquery.flags \
    && rm -f /etc/osquery/osquery.flags.default

RUN echo "28ZojsUf7zpOTQWAOrqSuadZCxWBEX14" > /etc/osquery/osquery.secret

# Fill osquery.flags.default with stuff
RUN echo "--config_plugin=tls \
&& --enroll_secret_path=/etc/osquery/osquery.secret \
&& --enroll_tls_endpoint=/api/v1/osquery/enroll\
&& --config_tls_endpoint=/api/v1/osquery/config \
&& --tls_hostname=<our_hostname> \
&& --config_refresh=300 \
&& --config_tls_accelerated_refresh=300 \
&& --config_tls_max_attempts=9999 \
&& --distributed_tls_read_endpoint=/api/v1/osquery/distributed/read \
&& --distributed_tls_write_endpoint=/api/v1/osquery/distributed/write \
&& --carver_start_endpoint=/api/v1/osquery/carve/begin \
&& --carver_continue_endpoint=/api/v1/osquery/carve/block" > /etc/osquery/osquery.flags.default
RUN ln -s /etc/osquery/osquery.flags.default /etc/osquery/osquery.flags

CMD ["bash"]
z
Ah, I think the issue is all the 
&&
in the flagfile.
p
geez good catch! Thanks Zach!
🍻 1
spoke too soon. Same error even after removing the `&&`s
will continue to debug with our SRE folks and report here once I find a solution
in case I haven't mentioned, this config works perfectly fine on our EC2 base images. It only fails on Docker for some reason.
z
Can you just copy the working flagfile into the container in your build?
p
yeah .. Punted on it for now. I was gonna start with a clean default Ubuntu base image and run the commands manually but we mostly have the data we need so decided no point spending much time trying to get it to work just yet ... I'll keep doing it on the side in evenings if/when I have time
z
The same flagfile will almost definitely work assuming the container has network connectivity. Happy to help when you get back to it.
ty 1
💯 1
14 Views
Open in Slack
PreviousNext
Powered by