Is there any known incompatibility with connecting...
# kolideg
grant seltzer
05/14/2019, 4:47 PMIs there any known incompatibility with connecting osqueryd to fleet through a proxy server?
z
zwass
05/14/2019, 4:51 PM
If you are using Launcher it tries to connect with grpc using HTTP2 which is not supported (or requires special configuration) with many proxies. With regular osqueryd I am not aware of any issues when the proxy is configured properly.
zwass
05/14/2019, 4:51 PM
What error are you seeing?
g
grant seltzer
05/14/2019, 4:53 PMStill the same one we were looking at a few days ago, the proxy is giving me a 503 error with osqueryd but works fine with curl, or directly to the host
grant seltzer
05/14/2019, 4:53 PMcurl through the proxy as well
grant seltzer
05/14/2019, 4:54 PMIs osqueryd definitely using https as oppose to grpc?
z
zwass
05/14/2019, 4:55 PM
Yes osqueryd has no grpc capabilities
zwass
05/14/2019, 4:56 PM
Does the request seem to hit the Fleet server when you run it through the proxy with osqueryd (is there a log entry)? Or is it erroring at the proxy?
zwass
05/14/2019, 4:56 PM
Also what proxy is this?
g
grant seltzer
05/14/2019, 4:57 PMhaproxy, and no it doesn't (erroring at the proxy),
grant seltzer
05/14/2019, 4:57 PMwith curl it reaches fleet
z
zwass
05/14/2019, 4:58 PM
I haven't heard of such issues before... Maybe osqueryd provides (or does not provide) headers in a way that HAProxy doesn't like?
g
grant seltzer
05/14/2019, 4:58 PMI see there's a
proxy_hostnamegrant seltzer
05/14/2019, 4:59 PMI'll try taking a close look at pcaps
z
zwass
05/14/2019, 4:59 PM
This generally sounds like a proxy issue so I would dig into what might make HAProxy return a 503.
g
grant seltzer
05/14/2019, 5:35 PMHm yea our theory right now is that osqueryd is not setting the host field of the http header
grant seltzer
05/14/2019, 5:35 PMwhich HAProxy uses to route traffic
grant seltzer
05/15/2019, 6:33 PMSo... crazy solution
grant seltzer
05/15/2019, 6:33 PMWe realized that osqueryd was not populating the "Host" field of the HTTP header
grant seltzer
05/15/2019, 6:33 PMHAProxy uses that to route to the fleet service
grant seltzer
05/15/2019, 6:34 PMso we just wrote a one line patch to osquery that fixed the issue
grant seltzer
05/15/2019, 6:34 PMWe're going to open a PR soon. Does this come to mind as something that makes sense?
2 Views