Hi - I have an odd message in my launcher logs (us...
# kolide
j
Hi - I have an odd message in my launcher logs (using the github kolide). Anyone have an idea of what might be causing this? Even just a point in the right direction would be fantastic.
Copy code
{
  "caller": "query_target.go:75",
  "err": "fetching target memberships: rpc error: code = Unknown desc = OK: HTTP status code 200; transport: received the unexpected content-type \"text/html; charset=utf-8\"",
  "msg": "updating kolide_target_membership data",
  "severity": "error",
  "ts": "2020-01-28T18:28:29.977507Z"
}
s
kolide_target_membership
is a feature related an earlier version of our SaaS. It’s a debug level warning. I think it should be supressed in current launcher versions.
Well… It’s a launcher feature that was only ever implemented in one of the kolide SaaS offerings. I don’t think we ever ported it to fleet.
j
interesting. i’ve recently rebuilt launcher from the .11.6 tag
I can try a fresh checkout and build again I suppose…
s
Hrm. Clearly not supressed then.
Regardless, it’s ignorable
j
yeah, I just have IT admins complaining (especially on windows) that fills up the event logs
s
Interesting. https://github.com/kolide/launcher/pull/416/files was supposed to move that to debug
j
looks like it did not, then
s
I think it did, but that the error is coming back somewhat differently, so the
if
isn’t firing
If you fix it, PR welcome 😉
j
you know, I can follow go, but haven't taken the time to really learn. it would be garbage 🙂 I will poke around and perhaps file an issue for someone else to do a more proper PR
s
I’ll see if I can find time. It shouldn’t be hard to replicate
Is this effecting platforms other than windows?
j
yeah, it’s on my mac as well. What I posed is from my mac. It’s just easier to ignore on the mac.
haven’t checked linux, but I can.
s
Nah. mac is enough. Lemme see if I can re-produce quickly
j
thx
s
A quick test case here, does not reproduce this error.
j
crazy. ok, let me recompile launcher and build a binary and see what happens. Very odd.
thanks for looking so fast!
s
And, as I look at your error more… You’re getting html back on an GRPC stream:
Copy code
desc = OK: HTTP status code 200; transport: received the unexpected content-type \"text/html
Does anything work? Are you running this with some http based load balancer?
j
oh, everything works
s
Still… It seems like you’re getting an grpc connection to an http endpoint.
j
it’s behind nginx acting as a proxy to restrict access to the admin interface.
let me expose kolide “naked” for short bit and see if that goes away.
s
I’m not really sure how nginx handles this. but I strongly suspect it’s doing something weird. I know there’s been conversation about how to do load balancing and endpoint restrictions, but there are some gotchas/
Yeah. testing without nginx seems like a good step. And digging though slack for conversations about ELB vs ALB might also help
j
yeah, I will probably move to something else (maybe traefik?) or just like you say, native AWS ALB
s
I don’t run fleet, nor have I tried to run grpc stuff in AWS. But I know I’ve seen it discussed here in #kolide
j
thanks! big big help
s
np!
j
thanks btw - just got a chance to really look, and found where I was proxying that grpc endpoint as http/1 and not grpc. all fixed.
s
Nice! Glad it worked out.
TBH I’m surprise anything worked at all
j
my memory is really foggy on this, but I had found a few endpoints, the ones relevant here are
/kolide.agent.Api
and
kolide.launcher.QueryTarget
it’s the latter that caused the errors not being proxied via grpc