I've been struggling with this for a few days and I've got to the point where I'm hoping someone else can guide me... I've installed kolide fleet, elasticsearch, kibana, and i've installed logstash all on the same server. Everything is fine (kolide fleet, elasticsearch, kibana are all running) and my server is humming along at under 10% utilized, but as soon as I try starting logstash my server becomes close to 100% utilized and very sluggish, and while logstash has created /var/log/logstash, it never writes anything in there so I have no logs to check to see what is going on. I suspect there is a problem with my logstash configuration and maybe that is causing it, as I've not really been able to get a handle on how the configuration should be done - most people it seems use filebeat to send the osquery logs to logstash to send them to elasticsearch, but I read that filebeat is not necessary if you are using logstash on the same machine as the osquery logs are being stored on. So this is my logstash configuration file (/etc/logstash/conf.d/47-osquery-output.conf): input { file { path => "/var/log/osquery/osqueryd.results.log" type => "osquery_json" codec => "json" } } filter { if [type] == "osquery_json" { date { match => [ "unixTime", "UNIX" ] } } } output { stdout {} elasticsearch { hosts=> "127.0.0.1:9200" } } Can anyone suggest why logstash is overloading my server? and why it is not creating any logs? can anyone confirm I've got a correct configuration in this conf file? /var/log/osquery/osqueryd.results.log is the file being output from kolide fleet. I notice that it is owned by root and in the root group but it is read for owner, group and other so it shouldn't be a prolem. and elasticsearch is listening on 127.0.0.1:9200. I don't understand what the filter in the middle of the config is for though - is it really necessary? what does it do? Thanks in advance!

Have you done any initial debugging? Is a process consuming a lot of CPU? A lot of RAM? What does 100% utilized mean?

I don't know where else to look. All I know is that until I start logstash the server is running perfectly and then it goes to near 100%. I'm taking these numbers from the linux top command

Knowing you ran

top

and it shows logstash is consuming a lot of CPU is valuable. That wasn’t clear from your initial comment.

Thanks for helping clarify Seph. That's the weird thing. It doesn't show logstash is consuming a lot of resources, I see some strange "shim" processes and other random things but not logstash. But the high utilization only occurs when I start logstash. I was also thinking more about this overnight and realised that I did do some other troubleshooting last week that I didn't think was relevant at the time but now with the syslog results mentioning trouble talking to elasticsearch it seems more relevant: ubuntu@ip-172-30-0-162:~$ curl http://127.0.0.1:9200 { "name" : "3sHDhp8", "cluster_name" : "elasticsearch", "cluster_uuid" : "_RRhXzMkQRi2cl9ree9S5Q", "version" : { "number" : "6.6.1", "build_flavor" : "default", "build_type" : "deb", "build_hash" : "1fd8f69", "build_date" : "2019-02-13T17🔟04.160291Z", "build_snapshot" : false, "lucene_version" : "7.6.0", "minimum_wire_compatibility_version" : "5.6.0", "minimum_index_compatibility_version" : "5.0.0" }, "tagline" : "You Know, for Search" } This tells me elastic search is actually listening

what does top show?

I'm not in a position to bring the server to a grinding halt again until Saturday so I'm afraid I can't tell you at the moment. However, over in the logstash community it seems to be that there is a configuration error in the logstash configuration file that tells it how to pick up the files from osquery/kolide and this is causing logstash to go into an endless restarting cycle. Apparently the JVM that it runs in starts running very heavily in such a circumstance. So if anyone here has experience in configuring logstash to pickup osquery logs on the same machine directly, rather than unnecessarily using filebeat to pick them up from the same machine and shuffle them to logstash which logstash can apparently do itself when it is on the same machine, I'd appreciate any advice on how to correct the configuration.