Configuration System

This page can be used as a reference while discussing the configuration system. This page should be updated as the discussion progresses.

Introduction

One of the Google Summer of Code project ideas for 2012 was a "configuration system". The configuration system was not among the selected projects, but the system was seen as a priority, so the plan now is to design and implement the important bits (the client API and the server-side infrastructure) ourselves within May 2012 or so (the reason for the tight schedule is that we'd like the GSoC students to benefit from this new system). Porting the existing configuration to the new system is not a matter of urgency. All the existing stuff has to be kept in mind while doing the design, though, so that the porting work is possible to do when that time comes.

The configuration system should make it easy to implement runtime-modifiable persistent configuration options in the core and modules. Currently that involves writing protocol extensions (for multiple protocols, if you aim for completeness) and managing the data storage yourself. The goal is to reduce that work, and also to make life easier for applications.

There are some notes from previous discussion at http://piratepad.net/BLGxYG2lf3.

Use cases

• Daemon configuration (daemon.conf, additional modules to load)
• Client configuration (client.conf)
• Global module configuration (echo canceller parameters)
• Per-instance module configuration (ALSA ignore_dB)

Scheme

• Configuration will be defined as a (key, value) pair, each being a string
• All known configuration will need to be registered before first use
  • Mechanism for modules to publish configuration out of band?
• Keys can be namespaced
  • daemon.flat-volumes, daemon.defaults.rate, daemon.defaults.alternate-rate
  • client.autospawn, client.defaults.sink, client-defaults.source
  • modules.echo-cancel.engine, modules.beamformer.geometry
  • modules.alsa.card..ignore_dB
  • '.' characters in object names can be replaced by '_'
• Identifying instance for per-instance configuration must be handled by module itself
  • Can just be responsible for module.alsa.* for example

Internal API

• Register a config key
  • pa_config_register_key(core, key, default_value)
• Query a config key
  • pa_config_query_key(core, key, ptr)
• Register a handler for when a key is set
  • pa_config_register_single_handler(core, key, func)
• Register a handler for a namespace
  • pa_config_register_prefix_handler(core, prefix, func)
• More than one handler can be registered
  • Modules might be interested in certain core config, for example
  • Do we need to worry about order? Early/normal/late?
• Modules that have not yet been loaded might still need to validate their configuration
  • For this, we could add a new entry point to modules -- validate()
  • The core will load the corresponding module when a module..* configuration option is set
  • The pa__validate_config() function in that module is meant to accept a (key, value) tuple and return true/false on validate
  • The core can unload the module after a while if no configuration is set on it for a while
  • An alternative to the core understanding the "module." prefix is that we build a registry at PA start time of modules and the prefixes they will manage so that we know up-front which module(s) to load to validate a given configuration option.

External API

• Configuration key names become part of the client API/ABI.
• Clients can listen for configuration changes
  • pa_context_set_config_callback()
  • Should we allow a prefix filter?
• Clients can set configuration (the name is assumed to be well known)
  • pa_context_set_config(context, key, value, success_cb)
  • Server will return a success/failure status which will be provided to the client via the success callback
• Clients can query configuration
  • pa_context_get_config(context, key, value_ptr)

Storage

• Configuration storage will be in plain text
  • This would allow us to set up initial config without messy binary files
  • Loading configuration amendments from .d directories will be supported too.
  • Arun prefers a single config file, Tanu prefers one file per prefix (daemon.conf, client.conf, per-module)
• An alternative is a plain-text file
• All access happens in the main thread context
• No support for listening to changes made outside of the client API

Previous Proposal

Goals

• The client API should be easy to use.
• The set of supported configuration options should be easy to extend.
  • No protocol extensions.
  • Make it as easy as possible to add a new option.
• Support also other things than statically named global options.
  • For example, options for ports require dynamic keys to identify the port.
  • We want (or do we?) to support also persistent data that is less configuration-like, like the stream-restore database or the equalizer data.
• Support change notifications both at client side and at server side.
• Support reading and modifying client.conf. (?)

Design

The configuration system stores "options". Options have a value and a (multi-part) identifier ("option id"). The option id has three parts: "object type", "object id" and "option name". For example, the maximum volume of a port could be stored in an option with the following id parts: "core.Port", "foocard:barport", "max-volume". The object that the object type and id describe is one of the objects in Pulseaudio: it could be a card, sink, source, port or stream-restore entry etc. The object id is whatever property identifies the object in question: in case of ports it would be the card name + port name (the card name is needed, because port names are not globally unique), and in case of stream-restore entries it would be the entry key. The object type has a namespace part, for example "core" in "core.Port" or "stream-restore" in "stream-restore.Entry". This can be used to divide the option storage into multiple files, one per namespace (one namespace for core and one namespace for each module would be the suggested way to group the options).

The client API doesn't have a concept of namespaces. The applications only need to know that if they want to access port options, they have use "core.Port" as the object_type string.

It may be sometimes useful to refer to an option by using just one string instead of three. That can be done by just concatenating the three identifiers. The suggested separator is slash. Example: "core.Port/foocard:barport/max-volume".

The configuration system knows which object types support which options. This is achieved by each "object type owner" (i.e. the code that implements the objects of that type) registering the object type to the configuration system. The registration includes the information about which options the object type supports, i.e. which option names are valid. The configuration system doesn't have any hardcoded knowledge about which object types exist.

The set of options that an object type supports can not be changed at runtime (except by unloading the object type owner module and then loading another module that registers the same object type with different options, but that should never happen). There may be situations where a module would like to extend a core object type. For example, module-alsa-card might want to have a "tsched" option for core.Card. The way to handle this is to make module-alsa-card register its own object type for cards, e.g. "alsa.Card", and specify in the documentation that the "alsa.Card" type is a subtype of "core.Card". Being a subtype means it's guaranteed that any "alsa.Card" object also has a corresponding "core.Card" object. The association between the two objects is made by using the same object id for both.

From the configuration system's point of view, all option values are strings. Adding type information would add a lot of complexity and not have very big benefits (is it so?). The options will of course have an implicit type to be useful. The actual users of the options (applications and object type owners in the server) have to validate the values themselves. This means that we should provide parsing functions for clients, at least for all complex types, but preferably also for simple stuff like integers. At server side, values are validated when they are read from the disk and when they are set by clients or by some code in the server other than the object type owner.

Since validation is done by the object type owner code, and since modules can implement object types, some of the options can be only validated when a specific module is loaded. The way to handle the case, where an application sets an option for a module that is not loaded, is to refuse to set the option. If the module required isn't loaded, from the configuration system's point of view the option doesn't exist.

Client API

This is how the client API will look like:

typedef void (*pa_server_configuration_value_change_cb_t)(
                pa_context *c,
                const char *object_type,
                const char *object_id,
                const char *option_name,
                const char *value,
                int eol,
                void *userdata);

typedef void (*pa_server_configuration_object_type_event_cb_t)(
                pa_context *c,
                const char *object_type,
                int added, /* 1 for added, 0 for removed */
                int eol,
                void *userdata);

pa_operation *pa_server_configuration_set(
                pa_context *c,
                const char *object_type,
                const char *object_id,
                const char *option_name,
                const char *value,
                pa_context_success_cb_t cb,
                void *userdata);

/* object_type, object_id and option_name can be NULL for wildcard selection.
 * If object_type is NULL, then all other identifiers have to be NULL also. In
 * case of wildcards, the callback may get called multiple times. The last call
 * has the object_type, object_id, option_name and value parameters of
 * pa_server_configuration_cb_t set to NULL and eol set to 1. */
pa_operation *pa_server_configuration_get(
                pa_context *c,
                const char *object_type,
                const char *object_id,
                const char *option_name,
                pa_server_configuration_cb_t cb,
                void *userdata);

/* The wildcard rules described with pa_server_configuration_get apply with
 * the value change items too. */
typedef struct pa_server_configuration_value_change_item {
        char *object_type;
        char *object_id;
        char *option_name;
} pa_server_configuration_value_change_item;

/* The items parameter is an array with n_items elements. If there are multiple
 * value change items given, then the value change callback is called when any
 * of the items match the event. If you want to end the subscription, or you
 * want to get updates only about added and removed object types, you can set
 * items to NULL and n_items to 0.
 *
 * The object_types parameter is an array with n_object_types elements. If an
 * object type is added or removed in the server, and it's one of the types
 * you have listed in the object_types array here, the server will send
 * a notification. For getting notifications for all object types, set
 * object_types to a pointer to a NULL pointer and set n_object_types to 1. If
 * you don't want any object type event notifications, set object_types to NULL
 * and n_object_types to 0.
 *
 * If called multiple times, the value change items and object types of the
 * last call will replace all the previous items and object types. */
pa_operation *pa_server_configuration_subscribe(
                pa_context *c,
                const pa_server_configuration_value_change_item *items,
                unsigned n_items,
                const char *const *object_types,
                unsigned n_object_types,
                pa_context_success_cb_t cb,
                void *userdata);

/* When there are option value changes that match the subscription, the
 * callback that is set here will be called multiple times. The last call has
 * the object_type, object_id, option_name and value parameters of
 * pa_server_configuration_value_change_cb_t set to NULL and eol set to 1. */
void pa_server_configuration_set_value_change_callback(
                pa_context *c,
                pa_server_configuration_value_change_cb_t cb,
                void *userdata);

/* When there are object type add/remove events that match the subscription,
 * the callback that is set here will be called multiple times. The last call
 * has the object_type parameter of
 * pa_server_configuration_object_type_event_cb_t set to NULL and eol set to
 * 1. */
void pa_server_configuration_set_object_type_event_callback(
                pa_context *c,
                pa_server_configuration_object_type_event_cb_t cb,
                void *userdata);

There has been some discussion about supporting also reading and modifying client.conf. client.conf is not centrally managed, it stores simple key-value pairs with static keys, it doesn't support change notifications and working with it probably doesn't require an asynchronous API. Therefore, the API is quite different, and maybe it doesn't need to be a part of this configuration system effort. The main point is that we might want such API now or in the future, so we should think how the configuration system API can co-exist with the client.conf API. That means basically that we probably don't want to use "pa_configuration_" as the server configuration prefix, because it's too generic.

Parsing functions

TODO

Storage format

TODO

• I like ini files as described here: http://article.gmane.org/gmane.comp.audio.pulseaudio.general/12845 --Tanu

Open issues

The namespace concept might need some refinement. The idea of splitting the storage into one file per module sounds nice, but it might not make sense to dedicate each object type to one module. Modules may want to add options to their own object types and to core object types, and possibly also to other modules' types. Let's say there's an object type "Card" in namespace "core", i.e. "core.Card". The alsa modules might want to add an option to core.Cards for enabling and disabling timer-based scheduling. Maybe the options should have namespacing too, i.e. the new option would be "core.Card/foocard/alsa.tsched"? Would every option name have a namespace part? That would be very ugly. How would this example be handled in storage? Is splitting the storage into many files a bad idea anyway?

• This is solved by not allowing modules to add options to existing object types. If everybody is happy, this issue can be deleted from here. --Tanu

Is there need for adding an API for clients for querying whether a specific option is supported? Such API could be added also later, if we are not certain yet.

One limitation with this proposal is that the client API supports only modifying existing objects. For example, adding a new entry to stream-restore is not possible in a nice way. You could set an option of a non-existent object, but that would leave the other fields of the entry uninitialized. Maybe that's tolerable, but if we want to support object creation, it should probably be done in a better way. And object deletion is not possible at all with the current API.

• I'm a bit against supporting object creation/deletion through the configuration API. Protocol extensions are still needed for that then. Maybe exposing the stream-restore database to clients through the configuration API isn't so good idea after all. The storage part of the system could still be useful also for stream-restore. --Tanu

Should we have a "reset to default" API? What about an "ask what the default is" API?

• My opinion: yes for "reset to default", no for "ask what the default is" until a good use case is found. --Tanu

Should we have an API for querying the currently supported object types?

• My opinion: yes. --Tanu