libpref¶

libpref is a generic key/value store that is used to implement prefs, a term that encompasses a variety of things.

Feature enable/disable flags (e.g. dom.IntersectionObserver.enabled, xpinstall.signatures.required).
User preferences (e.g. things set from about:preferences)
Internal application parameters (e.g. javascript.options.mem.nursery.max_kb).
Testing and debugging flags (e.g. network.dns.native-is-localhost).
Things that might need locking in an enterprise installation.
Application data (e.g. browser.onboarding.tour.onboarding-tour-addons.completed, services.sync.clients.lastSync).
A cheap and dirty form of IPC(!) (some devtools prefs).

Some of these (particularly the last two) are not an ideal use of libpref.

The C++ API is in the Preferences class. The XPIDL API is in the nsIPrefService and nsIPrefBranch interfaces.

High-level design¶

Keys¶

Keys (a.k.a. pref names) are 8-bit strings, and ASCII in practice. The convention is to use a dotted segmented form, e.g. foo.bar.baz, but the segments have no built-in meaning.

Naming is inconsistent, e.g. segments have various forms: foo_bar, foo-bar, fooBar, etc. Pref names for feature flags are likewise inconsistent: foo.enabled, foo.enable, foo.disable, fooEnabled, enable-foo, foo.enabled.bar, etc.

The grouping of prefs into families, via pref name segments, is ad hoc. Some of these families are closely related, e.g. there are many font prefs that are present for every language script.

Some prefs only make sense when considered in combination with other prefs.

Many pref names are known at compile time, but some are computed at runtime.

Basic values¶

The basic types of pref values are bools, 32-bit ints, and 8-bit C strings.

Strings are used to encode many types of data: identifiers, alphanumeric IDs, UUIDs, SHA1 hashes, CSS color hex values, large integers that don’t fit into 32-bit ints (e.g. timestamps), directory names, URLs, comma-separated lists, space-separated lists, JSON blobs, etc. There is a 1 MiB length limit on string values; longer strings are rejected outright.

Problem: The C string encoding is unclear; some API functions deal with unrestricted 8-bit strings (i.e. Latin1), but some require UTF-8.

There is some API support for faking floats, by converting them from/to strings when getting/setting.

Problem: confusion between ints and floats can lead to bugs.

Each pref consists of a default value and/or a user value. Default values can be initialized from file at startup, and can be added and modified at runtime via the API. User values can be initialized from file at startup, and can be added, modified and removed at runtime via the API and about:config.

If both values are present the user value takes precedence for most operations, though there are operations that specifically work on the default value.

If a user value is set to the same value as the default value, the user value is removed, unless the pref is marked as sticky at startup.

Problem: it would be better to have a clear notion of “reset to default”, at least for prefs that have a default value.

Prefs can be locked. This prevents them from being given a user value, or hides the existing user value if there is one.

Complex values¶

There is API support for some complex values.

nsIFile objects are handled by storing the filename as a string, similar to how floats are faked by storing them as strings.

nsIPrefLocalizedString objects are ones for which the default value specifies a properties file that contains an entry whose name matches the prefname. When gotten, the value from that entry is put into the user value. When set, the given value just overwrites the user value, like a string pref.

Problem: this is weird and unlike all the other pref types.

nsIRelativeFilePref objects are only used in comm-central.

Pref Branches¶

XPIDL-based access to prefs is via nsIPrefBranch/nsPrefBranch, which lets you specify a branch of the pref tree (e.g. font.) and pref names work relative to that point.

This API can be used from C++, but for C++ code there is also direct access through the Preferences class, which uses absolute pref names.

Threads¶

For the most part, all the basic API functions only work on the main thread. However, there are two exceptions to this.

The narrow exception is that the Servo traversal thread is allowed to get pref values. This only occurs when the main thread is paused, which makes it safe. (Note: bug 1474789 indicates that this may not be true.)

The broad exception is that static prefs can have a cached copy of a pref value that can be accessed from other threads. See below.

Notifications¶

There is a notification API for being told when a pref’s value changes. C++ code can register a callback function and JS code can register an observer (via nsIObserver, which requires XPCOM). In both cases, the registered entity will be notified when the value of the named pref value changes, or when the value of any pref matching a given prefix changes. E.g. all font pref changes can be observed by adding a font. prefix-matching observer.

See also the section on static prefs below.

Static prefs¶

There is a special kind of pref called a static pref. Static prefs are defined in StaticPrefList.yaml.

If a static pref is defined in both StaticPrefList.yaml and a pref data file, the latter definition will take precedence. A pref shouldn’t appear in both StaticPrefList.yaml and all.js, but it may make sense for a pref to appear in both StaticPrefList.yaml and an app-specific pref data file such as firefox.js.

Each static pref has a mirror kind.

always: A C++ mirror variable is associated with the pref. The variable is always kept in sync with the pref value. This kind is common.
once: A C++ mirror variable is associated with the pref. The variable is synced once with the pref’s value at startup, and then does not change. This kind is less common, and mostly used for graphics prefs.
never: No C++ mirror variable is associated with the pref. This is much like a normal pref.

An always or once static pref can only be used for prefs with bool/int/float values, not strings or complex values.

Each mirror variable is read-only, accessible via a getter function.

Mirror variables have two benefits. First, they allow C++ and Rust code to get the pref value directly from the variable instead of requiring a slow hash table lookup, which is important for prefs that are consulted frequently. Second, they allow C++ and Rust code to get the pref value off the main thread. The mirror variable must have an atomic type if it is read off the main thread, and assertions ensure this.

Note that mirror variables could be implemented via vanilla callbacks without API support, except for one detail: libpref gives their callbacks higher priority than normal callbacks, ensuring that any static pref will be up-to-date if read by a normal callback.

Problem: It is not clear what should happen to a static pref’s mirror variable if the pref is deleted? Currently there is a missing NotifyCallbacks() call so the mirror variable keeps its value from before the deletion. The cleanest solution is probably to disallow static prefs from being deleted.

Loading and Saving¶

Default pref values are initialized from various pref data files. Notable ones include:

modules/libpref/init/all.js, used by all products;
browser/app/profile/firefox.js, used by Firefox desktop;
mobile/android/app/mobile.js, used by Firefox mobile;
mail/app/profile/all-thunderbird.js, used by Thunderbird (in comm-central);
suite/browser/browser-prefs.js, used by SeaMonkey (in comm-central).

In release builds these are all put into omni.ja.

User pref values are initialized from prefs.js and (if present) user.js, in the user’s profile. This only happens once, in the parent process. Note that prefs.js is managed by Firefox, and regularly overwritten. user.js is created and managed by the user, and Firefox only reads it.

These files are not JavaScript; the .js suffix is present for historical reasons. They are read by a custom parser within libpref.

User pref file syntax is slightly more restrictive than default pref file syntax. In user pref files user_pref definitions are allowed but pref and sticky_pref definitions are not, and attributes (such as locked) are not allowed.

Problem: geckodriver has a separate prefs parser in the mozprofile crate.

Problem: there is no versioning of these files, for either the syntax or the data. This makes changing the file format difficult.

There are API functions to save modified prefs, either synchronously or asynchronously (via an off-main-thread runnable), either to the default file (prefs.js) or to a named file. When saving to the default file, no action will take place if no prefs have been modified.

Also, whenever a pref is modified, we wait 500ms and then automatically do an off-main-thread save to prefs.js. This provides an approximation of durability, but it is still possible for something to go wrong (e.g. a parent process crash) and end up with recently changed prefs not being saved. (If such a thing happens, it compromises atomicity, i.e. a sequence of multiple related pref changes might only get partially written.)

Only prefs whose values have changed from the default are saved to prefs.js.

Problem: Each time prefs are saved, the entire file is overwritten – 10s or even 100s of KiBs – even if only a single value has changed. This happens at least every 5 minutes, due to sync. Furthermore, various prefs are changed during and shortly after startup, which can result in 10s of MiBs of disk activity.

about:support¶

about:support contains an “Important Modified Preferences” table. It contains all prefs that (a) have had their value changed from the default, and (b) whose prefix match a whitelist in Troubleshoot.jsm. The whitelist matching is to avoid exposing pref values that might be privacy-sensitive.

Problem: The whitelist of prefixes is specified separately from the prefs themselves. Having an attribute on a pref definition would be better.

Sync¶

On desktop, a pref is synced onto a device via Sync if there is an accompanying services.sync.prefs.sync.-prefixed pref. I.e. the pref foo.bar is synced if the pref services.sync.prefs.sync.foo.bar exists and is true.

Previously, one could push prefs onto a device even if a local services.sync.prefs.sync.-prefixed pref was not present; however this behavior changed in bug 1538015 to require the local prefixed pref to be present. The old (insecure) behavior can be re-enabled by setting a single pref services.sync.prefs.dangerously_allow_arbitrary to true on the target browser - subsequently any pref can be pushed there by creating a remote services.sync.prefs.sync.-prefixed pref.

In practice, only a small subset of prefs (about 70) have a services.sync.prefs.sync.-prefixed pref by default.

Problem: This is gross. An attribute on the pref definition would be better, but it might be hard to change that at this point.

The number of synced prefs is small because prefs are synced across versions; any pref whose meaning might change shouldn’t be synced. Also, we don’t sync prefs that may differ across different devices (such as a desktop machine vs. a notebook).

Prefs are not synced on mobile.

Rust¶

Static prefs mirror variables can be accessed from Rust code via the static_prefs::pref! macro. Other prefs currently cannot be accessed. Parts of libpref’s C++ API could be made accessible to Rust code fairly straightforwardly via C bindings, either hand-made or generated.

Cost of a pref¶

The cost of a single pref is low, but the cost of several thousand prefs is reasonably high, and includes the following.

Parsing and initializing at startup.
IPC costs at startup and on pref value changes.
Disk writing costs of pref value changes, especially during startup.
Memory usage for storing the prefs, callbacks and observers, and C++ mirror variables.
Complexity: most pref combinations are untested. Some can be set to a bogus value by a curious user, which can have serious effects (read the comments). Prefs can also have bugs. Real-life examples include mistyped prefnames, all.js entries with incorrect types (e.g. confusing int vs. float), both of which mean changing the pref value via about:config or the API would have no effect (see bug 1414150 for examples of both).
Sync cost, for synced prefs.

Guidelines¶

We have far too many prefs. This is at least partly because we have had, for a long time, a culture of “when in doubt, add a pref”. Also, we don’t have any system — either technical or cultural — for removing unnecessary prefs. See [bug 90440] (https://bugzilla.mozilla.org/show_bug.cgi?id=90440) for a pref that was unused for 17 years.

In short, prefs are Firefox’s equivalent of the Windows Registry: a dumping ground for anything and everything. We should have guidelines for when to add a pref.

Here are some good reasons to add a pref.

A user may genuinely want to change it. E.g. it controls a feature that is adjustable in about:preferences.
To enable/disable new features. Once a feature is mature, consider removing the pref. A pref expiry mechanism would help with this.
For certain testing/debugging flags. Ideally, these would not be visible in about:config.

Here are some less good reasons to add a pref.

I’m not confident about this numeric parameter (cache size, timeout, etc.) Get confident! In practice, few if any users will change it. Adding a pref doesn’t absolve you of the responsibility of finding a good default. Then make it a code constant.
I need to experiment with different parameters during development. This is reasonable, but consider removing the pref before landing or once the feature has matured. An expiry mechanism would help with this.
I sometimes fiddle with this value for debugging or testing. Is it worth exposing it to the whole world to save yourself a recompile every once in a while? Consider making it a code constant.
Different values are needed on different platforms. This can be done in other ways, e.g. #ifdef in C++ code.

These guidelines do not consider application data prefs (i.e. ones that typically don’t have a default value). They are quite different from the other kinds. They arguably shouldn’t prefs at all, and should be stored via some other mechanism.

Low-level details¶

The key idea is that the prefs database consists of two pieces. The first is an initial snapshot of pref values that is created when the first child process is created. This snapshot is stored in immutable, shared memory, and shared by all processes.

Pref value changes that occur after this point are stored in a second hash table. Each process has its own copy of this hash table. When pref values change in the parent process, it performs IPC to inform child processes about the changes, so they can update their copy.

The motivation for this design is memory usage. It’s not tenable for every child process to have a full copy of the prefs database.

Not all child processes need access to prefs. Those that do include web content processes, the GPU process, and the RDD process.

Parent process startup¶

The parent process initially has only a hash table.

Early in startup, the parent process loads all of the static prefs and default prefs (mainly from omni.ja) into that hash table. The parent process also registers C++ mirror variables for static prefs, initializes them, and registers callbacks so they will be updated appropriately for all subsequent updates.

Slightly later in startup, the parent process loads all user prefs files, mainly from the profile directory.

When the first getter for a once static pref is called, all the once static prefs have their mirror variables set and special frozen prefs are put into the hash table. These frozen prefs are copies of the once prefs that are given $$$ prefixes and suffixes on their names. They are also marked specially so they are ignored for all cases except when starting a new child process. They exist so that all child processes can be given the same once values as the parent process.

Child process startup (parent side)¶

When the first child process is created, the parent process serializes its hash table into a shared, immutable snapshot. This snapshot is stored in a shared memory region managed by a SharedPrefMap instance. The parent process then clears the hash table. The hash table is subsequently used only to store changed pref values.

When any child process is created, the parent process serializes all pref values present in the hash table (i.e. those that have changed since the snapshot was made) and stores them in a second, short-lived shared memory region. This represents the set of changes the child process needs to apply on top of the snapshot, and allows it to build a hash table which should exactly match the parent’s.

The parent process passes two file descriptors to the child process, one for each region of memory. The snapshot is the same for all child processes.

Child process startup (child side)¶

Early in child process startup, the prefs service maps in and deserializes both shared memory regions sent from the parent process, but defers further initialization until requested by XPCOM initialization. Once that happens, mirror variables are initialized for static prefs, but no default values are set in the hash table, and no prefs files are loaded.

Once the mirror variables have been initialized, we dispatch pref change callbacks for any prefs in the shared snapshot which have user values or are locked. This causes the mirror variables to be updated.

After that, the changed pref values received from the parent process (via changedPrefsFd) are added to the prefs database. Their values override the values in the snapshot, and pref change callbacks are dispatched for them as appropriate. once mirror variable are initialized from the special frozen pref values.

Pref lookups¶

Each prefs database has both a hash table and a shared memory snapshot. A given pref may have an entry in either or both of these. If a pref exists in both, the hash table entry takes precedence.

For pref lookups, the hash table is checked first, followed by the shared snapshot. The entry in the hash table may have the type None, in which case the pref is treated as if it did not exist. The entry in the static snapshot never has the type None.

For pref enumeration, both maps are enumerated, starting with the hash table. While iterating over the hash table, any entry with the type None is skipped. While iterating over the shared snapshot, any entry which also exists in the hash table is skipped. The combined result of the two iterations represents the full contents of the prefs database.

Pref changes¶

Pref changes can only be initiated in the parent process. All API methods that modify prefs fail noisily (with NS_ERROR) if run outside the parent process.

Pref changes that happen before the initial snapshot have been made are simple, and take place in the hash table. There is no shared snapshot to update, and no child processes to synchronize with.

Once a snapshot has been created, any changes need to happen in the hash table.

If an entry for a changed pref already exists in the hash table, that entry can be updated directly. Likewise for prefs that do not exist in either the hash table or the shared snapshot: a new hash table entry can be created.

More care is needed when a changed pref exists in the snapshot but not in the hash table. In that case, we create a hash table entry with the same values as the snapshot entry, and then update it… but only if the changes will have an effect. If a caller attempts to set a pref to its existing value, we do not want to waste memory creating an unnecessary hash table entry.

Content processes must be told about any visible pref value changes. (A change to a default value that is hidden by a user value is unimportant.) When this happens, ContentParent detects the change (via an observer). It checks the pref name against a small blacklist of prefixes that child processes should not care about (this is an optimization to reduce IPC rather than a capabilities/security consideration), and for string prefs it also checks the value(s) don’t exceed 4 KiB. If the checks pass, it sends an IPC message (PreferenceUpdate) to the child process, and the child process updates the pref (default and user value) accordingly.

Problem: The blacklist of prefixes is specified separately from the prefs themselves. Having an attribute on a pref definition would be better.

Problem: The 4 KiB limit can lead to inconsistencies between the parent process and child processes. E.g. see bug 1303051.

Pref deletions¶

Pref deletion is more complicated. If a pref to be deleted exists only in the hash table of the parent process, its entry can simply be removed. If it exists in the shared snapshot, however, its hash table entry needs to be kept (or created), and its type changed to None. The presence of this entry masks the snapshot entry, causing it to be ignored by pref enumerators.