When coding the new tls_logger backoff feature, I realized that the following configs are not being updated dynamically via

config_tls_endpoint

. They are only set once at startup and cannot be changed afterwards.

--logger_tls_endpoint
--logger_tls_period
--logger_tls_max_lines

Is this a bug? Or is the fact that these settings are set once and cannot be updated until osquery restart documented somewhere? I'd like to see the ability to remotely restart osquery if a setting cannot be applied -- either as a separate config

--osquery_restart=true

or automatically if an updated setting via config_tls_endpoint cannot be immediately applied.

I don't think that can be considered a bug per se. It's just a choice of the moment when writing the code. The loggers are constructed with those parameters and so they do not react when changed. Also, osquery has the concept of CLI_FLAG, which are flags that can be given only through the process arguments or the flagfile, and some are marked as that because they can only be changed once at startup, others because one wants to enforce that they can't be changed remotely easily. Right now is all quite inconsistent, because there definitely should be more CLI_FLAG, even only to mark which flags will not react to change post startup.

Maybe there should be a concept of DYNAMIC_FLAG, which can be overwritten remotely at any time?

There is already, FLAG

As I user, if I find a flag in osquery documentation and try to set it remotely, how do I know if it will actually be applied or not?

This is not something that's communicated in the readthedocs doc about each flag, but can be seen in the

osquery --help

which always has been the ultimate truth. Again FLAGs will react to remote config file changes if the code has been written to do so, and that flag is not erroneously marked as FLAG CLI_FLAGs will not react to remote config file changes

Doesn't seem like a good user experience. If I find a flag in the documentation that I want to set dynamically, I need to go look in the code to see if the code has been written to pick up the flag change.

I agree, but fixing it is done via marking each flag correctly.

The behavior is a bit different here. These flags are set remotely initially, but then afterwards they cannot be updated. So they should not be labeled as CLI_FLAGS. That's why I was proposing a new label for them.

Well, but now they do not get picked up and modified, before they are read. That been said if the "modify once but remotely" behavior is what you want, then yes we would need a new variation of a flag. Although at this point I wonder if it makes sense to create another category or it's confusing. As I mentioned earlier, period and max_lines could be kept as FLAG and moved in the code where they are read each time, so when the config file changes them, then they are picked up. Is there are reason to why you want to change them only once?

I'm just thinking of this new label for the future (for other flags). These flags should be modified dynamically. I made this fix as part of my tls_logger backoff PR: https://github.com/osquery/osquery/pull/8230

And talking about the documentation being out of sync, that's mostly because it's a manual procedure. The macros could be grepped and a script be written to list them all, presenting them with the info they have. Couple of issues with that is that until now for the flags that are documented on readthedocs, they have a longer explanation too which is manually written in the docs, and cannot be therefore extracted from the flags themselves. I think if we want to ensure that everything stays in sync, code should be the truth and FLAGS macros should support a short and long description, so that then that can be parsed and used for the documentation too (I'm mainly referring to generate a page like this: https://osquery.readthedocs.io/en/latest/installation/cli-flags/, instead of manually updating it everytime after adding a new flag. I also just realized the misnomer of the page itself, which clashes with the concept of CLI_FLAGs, since there you have both types). We also have other pages that have longer, in context explanation of some flags, but those obviously will stay manually written.

I’m coming in a little late here, and I’m not sure what the state of this discussion is… As I understand it, today things are setup around two flags styles. CLI_FLAG and FLAG. The former must be specified on the command line, or flags file. The latter may also show up in the config, This can primarily be seen in the

--help

output. This difference is partly around security/privacy, and partly around what makes sense implementation-wise As Stefano said — there’s a lot of manual updating around this, so there may well be bug in how the docs are written. And even in which flags show up where. There’s almost certainly place to PR fixes… It sounds like there is a separate question around some flags which can be set once via a remote config but then cannot be changed. I cannot decide if this is a bug, or if it’s a category of flag we should embrace. I’m curious where people are leaning.

For the new category, some of those flags are like that due to construction and, if you want, limitations of the current code. Like some of the flags connected to event publishers, because we currently cannot disable or enable multiple times a publisher at runtime, without restarting.

I’m kinda nosing around whether we should make a new flag type, or just add a note to the flag descriptions. If we consider it a bug, a note feels more appropriate. If we think of it as intentional, than a new flag catagory

I think the bug right now is the incorrect classification on being a FLAG. Until now the intention of CLI_FLAG was both to say "something you have to give at startup, via flagfile or process args" and "something you can't change later". Therefore where people remembered and wrote code where the logic did not re-read the value, they would mark it as CLI_FLAG.

I thought we added a warning if a CLI_FLAG shows up in the config?

Yeah, sorry, I'm talking about two kinds of enforcing here. 1. Although the guideline of marking FLAG flags that we want to change from a remote config, and that can affect the code, because they are re-read, and the other guideline for CLI_FLAG I mentioned above existed since a while, enforcing the correct labeling of the flags did not happen often, result: some flags are mislabeled. 2. To further "prove" the inconsistency, I was bringing the CLI_FLAG issue where previously we would not enforce that CLI_FLAGs could not be changed via remote config (and we would not even warn about it). We later changed that, and now is enforced, and it also prints warnings.

Yeah, I agree.

I'm less sure about the security separation between CLI_FLAG and FLAGs. Meaning, I think it's good to have some flags that cannot be changed easily via a remote config, without too much noise; this was also part of the reason I wanted to actually enforce that CLI_FLAGs could not be changed remotely (and a customer asked too ^^'). But this also maybe hinged a bit on the backend not also having other functionality that can simply push files to machines and restart osquery when they want. Basically their question was "what if backend X gets owned?". In any case I guess having to make "more noise" to apply some settings is better.

I'm on board with just having CLI_FLAG and FLAG, and that there are bugs where some FLAG should actually be CLI_FLAG. I fixed the above-mentioned logger_tls configs to have FLAG functionality in my PR.

set once but remotely

Feels like it would only make sense with a lot of additional monitoring. And it’s feels like it’s getting a bit far outside our usual use case