Thanks for helping clarify Seph. That's the weird thing. It doesn't show logstash is consuming a lot of resources, I see some strange "shim" processes and other random things but not logstash. But the high utilization only occurs when I start logstash.
I was also thinking more about this overnight and realised that I did do some other troubleshooting last week that I didn't think was relevant at the time but now with the syslog results mentioning trouble talking to elasticsearch it seems more relevant:
ubuntu@ip-172-30-0-162:~$ curl
http://127.0.0.1:9200
{
"name" : "3sHDhp8",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "_RRhXzMkQRi2cl9ree9S5Q",
"version" : {
"number" : "6.6.1",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "1fd8f69",
"build_date" : "2019-02-13T17
1004.160291Z",
"build_snapshot" : false,
"lucene_version" : "7.6.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
This tells me elastic search is actually listening