<@U7QP20JQH> regarding the (TLS gzip flag naming)[...
# core
z
@seph regarding the (TLS gzip flag naming)[https://github.com/osquery/osquery/pull/8731#issuecomment-3661415819} I was finding it very appealing to make the single flag
tls_gzip
that would control both requests and responses but then I realized that could end up being problematic if we want to turn it on by default and not all servers support gzip. I think the viable paths are: 1) Single flag, never enabled by default (or maybe enabled by default in a major version release? with documentation that servers MUST support it?) 2) Separate flags for requests/responses, with requests enabled by default in a future release (server can always choose to ignore and return an uncompressed response) What are your thoughts?
l
(server can always choose to ignore and return an uncompressed response)
Though the requests may fail if gzip decoding is not supported by the server, maybe at this point almost all flavors of servers do support it? In any case, users can turn gzip encoding on requests when this happens.
Separate flags makes sense to me (my 2 cents).
Guess: if a server supports request decoding it most likely supports response encoding?
z
Ah sorry I was not clear at all with my wording in #2 there. We would turn on accepting gzip by default but continue to leave sending gzip off by default. The reason being the request can set an accept encoding for gzip and the server can just choose to return uncompressed data even if osquery supports gzip.
s
<thinks>
So my general goals are to do things that feel right. Defaults should be the common use case, and named well.
So… if I think about it. Osquery ought just be able to handle server gzipped responses always. We can send the appropriate accept header and assume the right things. Right? Maybe worth an option in the override a normal behavior to debug something weird.
But sending data is tricky. Since osquery starts with a post, we don't have a way to know what's supported. (Without tracking it, which feels messy)
z
Yeah this all makes sense to me. I could see removing the flag entirely to accept gzip once we have some more confidence in the feature.
s
So I think separate options, with separate defaults? Do we think most osquery servers support gzip? That would bias the defaults.
z
Yeah okay so I think we do
tls_accept_gzip
(default false right now, default true once we build confidence in the stability)
tls_send_gzip
(default false for the foreseeable future)
1
l
Flags looks good. What would be a reason for users to set
tls_accept_gzip=false
?
z
Fleet for example supports neither at the moment. Could be fronted by a proxy that does though. But we'll add it.
Immediately so that the feature can be toggled for testing. Later, in case they decide they don't want to use the CPU on decompression? Or if a critical bug comes up?
l
Later, in case they decide they don't want to use the CPU on decompression? Or if a critical bug comes up?
Good point.
z
I could see removing the option in a release or two if we decide it's plenty stable
👍 1
Okay so I think this PR goes ahead with the
tls_accept_gzip
flag and future work handles the rest.
s
The thing I don't like, is a flag structure like
tls_accept_gzip=false
. But I don't have a great suggestion and maybe this is really a complaint about GFLAG anyhow
So I think we go with it
z
I have updated the PR per this discussion
l
PR looks good!, left some comments around troubleshooting (possibly adding a few `VLOG(1)`s). Future work:
distributed/write
requests usually have very large bodies (query responses), so compression would be even more handy here. But I'm guessing it's a bit more challenging because AFAICS the only way to know for sure if a server supports gzip decoding is by running a probe request before the actual request.
I'm happy to approve, or wait for your requested VLOG
z
K I've updated with the logging