Title
#general
n

nyanshak

11/12/2019, 10:07 PM
osqueryd worker respawning too quickly: 3 times
^ I keep seeing that in a short time span, followed by osqueryd exiting, but I'm having trouble understanding why. I've got
--verbose
flag, but I still don't see anything to really help in diagnosing further.
theopolis

theopolis

11/12/2019, 10:55 PM
Can you try with
--disable_watchdog
n

nyanshak

11/12/2019, 10:57 PM
Yup, now it just dies after getting the config
theopolis

theopolis

11/12/2019, 10:58 PM
womp, and
lldb /path/to/osqueryd
(
run INSERT_FLAGS_HERE
) can you see where it's crashing?
10:58 PM
10:59 PM
(has symbols)
10:59 PM
I don't know why I assumed you're running on macOS 😕
n

nyanshak

11/12/2019, 10:59 PM
I am indeed not running on macOS
11:00 PM
linux 5.3.10-arch1-1
theopolis

theopolis

11/12/2019, 11:02 PM
Ok then let me know how I can help, assumptions:- You have
gdb
and know how to use it- You are using our pre-built pkgs, either RPM/DEB and could install the debug versions Loading the config should not cause a crashing (hehe) but rather give a useful error message about malformed JSON/etc.
11:04 PM
I'm going to be off-the-grid for a few hours. If this is indeed a crash we should promote this investigation to a github issue and triage as a bug report.
n

nyanshak

11/12/2019, 11:58 PM
- You have
gdb
and know how to use it
Yeah... not really. I'm a bit lost on this.
sudo osqueryd --flagfile=/etc/osquery/osquery.flags --verbose --config_dump --disable_watchdog
I1112 17:51:49.555171 70714 init.cpp:421] osquery initialized [version=4.0.2]
I1112 17:51:49.567677 70714 system.cpp:330] Found stale process for osqueryd (70546)
I1112 17:51:49.567737 70714 system.cpp:362] Writing osqueryd pid (70714) to /var/run/osqueryd.pidfile
I1112 17:51:49.567826 70714 extensions.cpp:345] Could not autoload extensions: Failed reading: /etc/osquery/extensions.load
I1112 17:51:49.567909 70714 rocksdb.cpp:131] Opening RocksDB handle: /var/osquery/osquery.db
I1112 17:51:49.624684 70714 database.cpp:570] Checking database version for migration
I1112 17:51:49.624918 70714 init.cpp:657] Cannot start extension manager: Extensions disabled
I1112 17:51:49.624989 70714 auto_constructed_tables.cpp:93] Removing stale ATC entries
I1112 17:51:49.625424 70714 tls.cpp:253] TLS/HTTPS POST request to URI: https://<tls-server>/api/v1/osquery/config
{"node_key":"<node_key>"}

{"tls_plugin": {"decorators":{...},"options":{"audit_allow_config":true,"aws_access_key_id":"...","aws_kinesis_stream":"...","aws_region":"...","aws_secret_access_key":"...","aws_sts_arn_role":"arn:aws:iam::<account>:role/<role>","aws_sts_region":"us-west-1","aws_sts_session_name":"osquery","aws_sts_timeout":3600,"decorations_top_level":true,"disable_audit":false,"disable_distributed":false,"disable_events":false,"distributed_interval":30,"distributed_plugin":"tls","distributed_tls_max_attempts":3,"distributed_tls_read_endpoint":"/api/v1/osquery/distributed/read","distributed_tls_write_endpoint":"/api/v1/osquery/distributed/write","host_identifier":"hostname","logger_min_status":2,"logger_plugin":"aws_kinesis","pack_delimiter":"_","worker_threads":2}}}
[ NO MORE OUTPUT... IT JUST DIES]
echo $?
0
12:14 AM
flag file:
--force=true
--host_identifier=uuid
--tls_hostname=<tls-server>
--config_plugin=tls
--enroll_tls_endpoint=/api/v1/osquery/enroll
--config_tls_endpoint=/api/v1/osquery/config
--enroll_secret_path=/etc/osquery/kolide.enrollment_secret
I've tried dumping the configs (
--config_dump
) putting it in
/etc/osquery/osquery.conf
(minus the distributed bits) but can't seem to reproduce except when using the distributed bits so far.
s

seph

11/13/2019, 1:22 PM
What osquery version are you running?
j

Jean M

01/22/2020, 4:10 PM
This problem (in case others also have it) can be due to the usage of --config_dump ! thanks @nyanshak for the information