osqueryd is erroring out throwing following error ...
# general
osqueryd is erroring out throwing following error for few of the queries I am trying to run: ErrorLogTag":"read unix @-\u003e/var/osquery/sock/osquery.em: i/o timeout","level":"error" Is there anything that I need to change in config file? What can I do to resolve this error? worker_threads has been set to 4 currently. "worker_threads": "4"
I would expect that socket to be used to communicate to extensions. Are your queries using tables that come from an extension?
This is happening for all tables. Like tables as basic as syslog_events
journalctl logs for osqueryd shows the followiing log entries: 5914 eventsubscriberplugin.cpp:577] Removed 212 event batches (with 18446744073709551422 delete errors) for subscriber: syslog.syslog_events (limit: 100000, last query: never)
journalctl logs error has been resolved by deleting the db and restarting the service
But strangely, ErrorLogTag":"read unix @-\u003e/var/osquery/sock/osquery.em: i/o timeout","level":"error" is still showing up for some queries randomly. Like in one run, for some queries there is no error and for some queries this error creeps up. I'm unable to understand why!
I see this majorly happening on syslog_events, last and process_open_sockets tables. And not on file_events, hardware_events and usb_devices tables. These are all the tables I am using so far.
But a few seconds ago, even syslog_events, last and process_open_sockets tables queries ran and gave the desired outputs. But not anymore.
Now syslog_events, last and process_open_sockets tables are only throwing ErrorLogTag":"read unix @-\u003e/var/osquery/sock/osquery.em: i/o timeout","level":"error"
Is this osqueryi? or osqueryd? Are you using any extensions? What options are you running with?
This is osqueryd. Not getting this error when I ran with osqueryi. This is what osqueryd is using /usr/bin/osqueryd --flagfile /etc/osquery/osquery.flags --config_path /etc/osquery/osquery.conf I don't see any use of extensions though. This is how the flags file look. cat osquery.flags --extensions_socket=/var/osquery/sock/osquery.em --watchdog_memory_limit=1000
@seph these are the options we use:
Copy code
"host_identifier": "PLACEHOLDER_HOSTNAME",
"schedule_splay_percent": 10,
"config_plugin": "filesystem",
"logger_plugin": "filesystem",
"logger_path": "/var/log/osquery",
"disable_logging": "false",
"pidfile": "/var/osquery/osquery.pidfile",
"events_expiry": "1",
"database_path": "/var/osquery/osquery.db",
"worker_threads": "4",
"disable_events": "false",
"disable_audit": "false",
"audit_allow_config": "true",
"enable_syslog": "true",
"syslog_pipe_path": "/var/osquery/syslog_pipe",
"force": "true",
"audit_allow_sockets": "true",
"enable_file_events": "true",
"enable_ntfs_publisher": "true",
"schedule_default_interval": "3600"
our queries run on a schedule and these errors are happeing when osquery is running on deamon mode. Strange part is, we are able to open the socket successfully
but the query is giving these errors
I don't think I've seen this before. In another thread here, AP has talked about docker and other things. Is there anything unusual happening here?
we trigger the queries from inside a docker container, but I believe that should not be an issue
as the errors are random. How many parallel threads can osquery handle? Is that decided by "worker_threads": "4", ? Are there guidelines on setting this number?