Hi I m trying to build a `config parser` in a C++ extension

Question

Hi I m trying to build a `config parser` in a C++ extension I m using `REGISTER EXTERNAL` macro and I see the debug message at `registry factory cpp` `I1108 12 56 08 491199 231789 registry factory cpp 107 Extension 499 registered config parser plugin devo params` However the update function is never called If I add some debug messages to `osquery config config cpp` and print the list of all config parser modules it handles I can t see it However if I add the same function at the extension code it exists Any ideas why this could be happening ```void printAll auto plugins = osquery RegistryFactory get plugins config parser for auto amp p plugins printf registered parser s n p first c str ```

Stefano Bonicatti · Answer

How are you starting the extension and osquery What flags

Daniel Bretón Suárez · Answer

I m running osquery daemon in verbose mode ```clear sudo osqueryd flagfile etc osquery osquery flags verbose 2 gt amp 1 | tee devo ea agent verbose out``` With this flagfile ``` enroll secret path= etc osquery certs secret tls server certs= etc osquery certs devo ea manager crt tls hostname=devo ea manager 8080 enroll tls endpoint= api v1 osquery enroll config plugin=tls config tls endpoint= api v1 osquery config config refresh=10 disable distributed=false distributed plugin=tls distributed interval=3 distributed tls max attempts=3 distributed tls read endpoint= api v1 osquery distributed read distributed tls write endpoint= api v1 osquery distributed write logger plugin=tls logger tls endpoint= api v1 osquery log logger tls period=10 logger tls max lines=8192 extensions autoload= etc osquery extensions load watchdog memory limit=512 enable extensions watchdog=true extensions require=system stats fetchfiles devo wevent logger disable tables=curl``` An this extension load devo wevent logger is the one I m working on ``` opt osquery share osquery extensions fetchfiles ext opt osquery share osquery extensions devo wevent logger ext opt osquery share osquery extensions system stats ext``` I m also running a fleet server in a VM The ultimate goal is to distribute the configuration Just to be clear I can access the parameters defined as `custom ` flags under `options` as in the first image But I want to implement a schema as the second image

Stefano Bonicatti · Answer

Nvm I misread the image So it is indeed a separate section already got confused by the first image

Stefano Bonicatti · Answer

I see though that in your previous example the plugin is named devo params while the section you re trying to parse is called devo things

Stefano Bonicatti · Answer

Afaik they have to match for the reason of why printAll doesn t print your config parser not sure where exactly have you placed that printAll call

Daniel Bretón Suárez · Answer

The invocation of printAll that does not print the parser is placed at `Config refresh ` under `osquery config config cpp` The one that prints the parser is under the extension

Daniel Bretón Suárez · Answer

I m changing it for the names to match but I m not able to get the update function from my config parser called when I change the configuration on the manager I think that if it is not listed under the registered parsers on the core process it won t be called However it is listed in the extension process I was thinking that could be the root problem but now I m not sure Am I missing something

Stefano Bonicatti · Answer

Ok sorry my extension game is rusty As far as I can see we don t support external config parsers while a config plugin we do The plugin gets registered in a separate registry the external one but the parsers use only the local one

Stefano Bonicatti · Answer

More specifically at a certain point in core you get the `RegisteryFactory addBroadcast` function called Then on line 100 we register the plugin provided by the extension using the specific registry `addExternal` function that ends up calling the RegistryInterface relative function which at line 270 registers the route to that plugin in `external ` before that it calls the `addExternalPlugin` which you can follow in our plugin h which is basically empty and always returns success the only thing that implements it is a TablePlugin which uses SQL to make the table appear in the local sqlite database Now a Config plugin works because the code uses the `call` function of the Registry which automatically uses all the registry types you can see its implementation here As you can see that also uses the routes previously registered in the external registry

Stefano Bonicatti · Answer

Sorry forgot to point where the Config plugin does the call And instead you can see that the applyParsers function uses the function you were trying to use to get the registered plugins But that `plugins config parser ` function only gets local plugins after having select the `config parsers` registry it ends up in the `plugins ` function of the `RegistryInterface` here As you can see it s returning the `items ` member but in RegistryInterface you also have `external ` which is where the external plugins are

Stefano Bonicatti · Answer

TLDR You need to use a ConfigPlugin which has to be provided via ` config plugin` which will receive the whole config and will have to select and modify only what s interested in

Stefano Bonicatti · Answer

So the TLDR might need to be expanded the situation is a bit messy you can request from the extension that each osquery local plugin should generate again the config and results to be sent to the extension which if nothing changed it would be the same as what osquery sees You would do that through the registry using a call similar to what the core does in reverse here <https github com osquery osquery blob 0b4ec101e86a35d5855a45bb03324abbfe5688a6 osquery config config cpp L484> That though has its problems because for some plugins that might means an additional remote call for the TLS plugin you re also getting the individual plugins configs each potentially at different points in time which might not match with what osquery has applied I think that mostly works if the config you need to read is the config that exists in the config file on the host and so you request it to the filesystem plugin Though by its nature that config cannot be changed through Fleet or you can but that actually is a layer on top that comes from the TLS plugin that layer won t be seen from the extension The ConfigPlugin way should be technically there to add to the config view osquery has not just to read moreover to get the full view you need to put your own config plugin name as the last in the list in ` config plugin` In theory each plugin before should compose a final response which contains everything the core has to use so the last plugin in the list should get all

Daniel Bretón Suárez · Answer

Wow awesome explanation I need some time to process all this info but is really really useful Thank you a lot for your patience

Daniel Bretón Suárez · Answer

FYI I was finally able to get the whole distributed config using `osquery Registry call config action genConfig config ` and then parsing it into a map the same way it is done at <https github com osquery osquery blob f8bd96e1ad499397596e3fb551ac8f0065e43c32 plugins config parsers options cpp L76> ```std map lt std string std string gt Config parseDistConfig const osquery PluginResponse amp config std string plugin std string cat std map lt std string std string gt ret for auto conf config for auto c1 conf osquery JSON json json fromString c1 second const auto amp options = json doc plugin cat for const auto amp option options GetObject std string name = option name GetString std string value if option value IsString value = option value GetString else if option value IsBool value = option value GetBool true false else if option value IsInt value = std to string option value GetInt else if option value IsNumber value = std to string option value GetUint64 else if option value IsObject || option value IsArray auto doc = osquery JSON newFromValue option value doc toString value else DEVOLOG WARNING lt lt Cannot parse unknown value type for option lt lt name ret insert name value return ret ```