From cfb9ecc3c75fd76756486eb44a4df4057afa3310 Mon Sep 17 00:00:00 2001 From: Rob Wu Date: Thu, 27 Mar 2025 10:01:27 +0100 Subject: [PATCH 1/2] Add proposal: Synchronous data at startup --- proposals/synchronous_data_at_startup.md | 240 +++++++++++++++++++++++ 1 file changed, 240 insertions(+) create mode 100644 proposals/synchronous_data_at_startup.md diff --git a/proposals/synchronous_data_at_startup.md b/proposals/synchronous_data_at_startup.md new file mode 100644 index 00000000..a513b9f8 --- /dev/null +++ b/proposals/synchronous_data_at_startup.md @@ -0,0 +1,240 @@ +# Proposal: Synchronous data at startup + +**Summary** + +A mechanism for extensions to specify values that are synchronously available when a content script or background script executes. + +**Document Metadata** + +**Author:** Rob--W + +**Sponsoring Browser:** Mozilla + +**Contributors:** @oliverdunk + +**Created:** 2025-03-26 + +**Related Issues:** + +* [Issue 536 comment](https://github.com/w3c/webextensions/issues/536#issuecomment-2200692043): `scripting.globalParams` in content scripts +* [Issue 703](https://github.com/w3c/webextensions/issues/703): State in background scripts, synchronously available across restarts +* Seemingly related but explicitly out of scope: [Issue 284](https://github.com/w3c/webextensions/issues/284): Main world User Script shared params +* Resolves use case: [Issue 501](https://github.com/w3c/webextensions/issues/501): Proposal: Toggleable event listeners +* Resolves use case: [issue 747](https://github.com/w3c/webextensions/issues/747): API to invalidate BFCache'd webpage with outdated content added by the extension + +## Motivation + +### Objective + +Provide a way for extensions to set and update state that is synchronously +available at the start of script execution. The state can be set from +privileged extension contexts, such as background scripts and options pages. +Once the state has propagated (asynchronously), the state can *synchronously* +be read from content scripts or other privileged contexts. + +This state may be session-scoped or persist across browser restarts and +extension updates. + +Content scripts have the following additional limitations: +- They can only read this shared state, and not propagate modifications. +- They may not access state specific to privileged contexts. + +#### Use Cases + +Some use cases depend on state to be synchronously available at the start of +script execution. This applies both to content scripts and background scripts. + +In the context of content scripts, initial state may be needed to run logic +at `document_start`, before the web page has had a chance to run. Content +scripts may also want to revalidate their state without latency after returning +from the bfcache ([issue 747](https://github.com/w3c/webextensions/issues/747)). + +This concept has also been discussed before as `globalParams` in the context of +dynamic content scripts (see https://crbug.com/1054624). + +In the context of background scripts, initial state may be needed for +initialization, for example registering listeners based on user configuration +([issue 501](https://github.com/w3c/webextensions/issues/501)). + + +### Known Consumers + +Classes of privacy extentensions, such as NoScript +([issue 536](https://github.com/w3c/webextensions/issues/536)). + +Classes of extensions that modify web pages based on user input, +where latency in state retrieval degrades the user experience. +The Violentmonkey author expressed a desire to use such an API in +[issue 747](https://github.com/w3c/webextensions/issues/747). + + +## Specification + +### Schema + +This proposal builds upon the existing `browser.storage` API. The `StorageArea` +type that commonly describes the various areas within `browser.storage` is +extended with new methods to allow synchronous access to the data. + +The `set` method includes a new optional `options` parameter to configure the +persistency of the specified parameters. + +``` +// When execution context is allowed to read synchronously. + any ConfigStorageArea.getSync(keyOrKeysOrObjectWithDefaults: string | string[] | null | object) + string[] ConfigStorageArea.getKeysSync(); + +// These are the same interfaces as storage.StorageArea + Promise ConfigStorageArea.get(keyOrKeysOrObjectWithDefaults: string | string[] | null | object) + Promise ConfigStorageArea.getBytesInUse() + Promise ConfigStorageArea.getKeys() +Promise ConfigStorageArea.set(items: object, options?: ConfigStorageAreaSetOptions) +Promise ConfigStorageArea.remove(keyOrKeys: string | string[]) +Promise ConfigStorageArea.clear() + + ExtensionEvent ConfigStorageArea.onChanged; + number ConfigStorageArea.QUOTA_BYTES; + +// Read-only to content scripts; no synchronous reading from extension contexts. +ConfigStorageArea browser.storage.contentScriptConfig; +// Extension contexts only; can read synchronously and write. +ConfigStorageArea browser.storage.extensionConfig; + +dictionary ConfigStorageAreaSetOptions { + persistent: boolean // default false +} +``` + +### Behavior + +#### ConfigStorageArea + +The `ConfigStorageArea` type is a composition of the existing `StorageArea` +interface and two new methods, `getSync` and `getKeysSync`. Upon invoking the +methods that update the storage area, the data is propagated to all processes +where a context may exists with a need for synchronous data access. + +Content scripts can *synchronously* access a copy of the available data with: + +- `browser.storage.contentScriptConfig.getSync(...)` +- `browser.storage.contentScriptConfig.getKeysSync(...)` + +These methods are unavailable to privileged extension contexts, to encourage +extensions to use the `extensionConfig` storage area that is hidden from +content scripts and only available to privileged extension contexts: + +- `browser.storage.extensionConfig.getSync(...)` +- `browser.storage.extensionConfig.getKeysSync(...)` + +The asynchronous methods (inherit from the `StorageArea` interface) are only +available to privileged extension contexts, and allow them to update the data. +Notably, the asynchronous `get`, `getKeys` and `getBytesInUse` methods are not +available to content scripts, despite their read-only capabilities. + +#### getSync + +The `getSync()` method has similar semantics as the `get()` method, except it +*synchronously* returns keys and values that are known locally in the process. + +#### getKeysSync + +The `getKeysSync()` method has similar semantics as the `getKeys()` method, +except it *synchronously* returns keys that are known locally in the process. + +#### set + +The `set()` method takes a (new) optional options object as a second parameter. + +The `persistent` option defaults to false, which means that the data is not +persisted across browser restarts. When `persistent` is set to `true`, the +specified key-value pairs are persisted across browser and extension restarts. + +Each key has its own associated `persistent` flag, which affect the behavior of +the latest `set()` call, and any following `remove()` or `clear()` calls. + +Persisted data should be stored locally, comparable to `storage.local`. + +#### onChanged + +When a storage change is observed, the `browser.storage.onChanged`, +`browser.storage.contentScriptConfig.onChanged` or +`browser.storage.extensionConfig.onChanged` events are dispatched as needed. + +The `onChanged` event receives an object with all modified keys. + +TODO: Should the `oldValue` and `newValue` fields be omitted? This would enable +the browser to lazily deserialize values on demand, as an optimization. +The recipient can use `getSync` to look up the actual value if desired. +See also https://github.com/w3c/webextensions/issues/475 + +#### QUOTA_BYTES + +The maximum amount (in bytes) of data that can be stored, as measured by the +key's string length plus the JSON stringification of every value, or the byte +consumption if the agent supports structured cloning. + +The value is a balance between utility for extensions and performance impact. + +The quota is `102400`. + +### New Permissions + +This API builds upon the `storage` API, which already requires the `storage` +permission. The `storage` permission does not trigger a warning message. + +### Manifest File Changes + +This proposal does not specify manifest changes. + +## Security and Privacy + +### Exposed Sensitive Data + +This API does not expose sensitive data. + +### Abuse Mitigations + +This API specifies data that should be made available across all processes, +which may affect the memory usage and startup cost of all processes. +To limit abuse, a reasonably tight storage quota is chosen. + +### Additional Security Considerations + +The existing `storage.local` API is sometimes used by extension to store +sensitive information. Due to its default availability in content scripts, +this exposes extensions to data leakage in compromised content processes. + +The proposed API design separates content script storage from the storage of +the higher-privileged parts of the extension, to encourage safe defaults. + +## Alternatives + +### Existing Workarounds + +In content scripts, extensions use synchronous XMLHttpRequest to block the main +thread of the web page to fetch dynamic configuration that has been specified +by the extension. + +In document-based background contexts, extensions can use the `localStorage` +API from the web platform as a synchronous storage mechanism. A limitation of +`localStorage` is that the data may not persist when the background context +runs in private browsing (incognito) mode, similar to +[issue 534](https://github.com/w3c/webextensions/issues/534). + +Service worker-based background contexts do not have alternatives for +synchronous storage. + +### Open Web API + +`contentScriptConfig` does not make sense on the web, as the concept of content +scripts is non-existent on the regular web platform. + +Service workers use asynchronous APIs by design. Synchronously blocking storage +is incompatible with that design. + +## Implementation Notes + + +## Future Work + +Highlight any planned or prospective future work. From 4440fe70aceae601d10fe3f2c8ab3578ad469ec9 Mon Sep 17 00:00:00 2001 From: Rob Wu Date: Thu, 9 Apr 2026 02:49:46 +0100 Subject: [PATCH 2/2] Update based on feedback from WECG F2F Changes to the API: - Drop getSync / getKeysSync; make `get` and `getKeys` sync instead. - To avoid the odd situation of being to write without reading, the `contentScriptConfig` area is now also synchronously readable by extension contexts. Updates to the proposal: - Explicitly list all exposed APIs for clarity. - Emphasize independence of storage areas for clarity. - Add expectations for flushing and method resolution behavior. - Drop "TODO" remark about potentially excluding oldValue/newValue from onChanged event as an optimization, because the consensus is to maintain the same semantics as other storage `onChanged` events. - Expand "Future work" --- proposals/synchronous_data_at_startup.md | 157 +++++++++++++++++------ 1 file changed, 115 insertions(+), 42 deletions(-) diff --git a/proposals/synchronous_data_at_startup.md b/proposals/synchronous_data_at_startup.md index a513b9f8..eefb5147 100644 --- a/proposals/synchronous_data_at_startup.md +++ b/proposals/synchronous_data_at_startup.md @@ -18,7 +18,7 @@ A mechanism for extensions to specify values that are synchronously available wh * [Issue 536 comment](https://github.com/w3c/webextensions/issues/536#issuecomment-2200692043): `scripting.globalParams` in content scripts * [Issue 703](https://github.com/w3c/webextensions/issues/703): State in background scripts, synchronously available across restarts -* Seemingly related but explicitly out of scope: [Issue 284](https://github.com/w3c/webextensions/issues/284): Main world User Script shared params +* Seemingly related but explicitly out of scope: [Issue 284](https://github.com/w3c/webextensions/issues/284): Main world Content Script shared params * Resolves use case: [Issue 501](https://github.com/w3c/webextensions/issues/501): Proposal: Toggleable event listeners * Resolves use case: [issue 747](https://github.com/w3c/webextensions/issues/747): API to invalidate BFCache'd webpage with outdated content added by the extension @@ -59,7 +59,7 @@ initialization, for example registering listeners based on user configuration ### Known Consumers -Classes of privacy extentensions, such as NoScript +Classes of privacy extensions, such as NoScript ([issue 536](https://github.com/w3c/webextensions/issues/536)). Classes of extensions that modify web pages based on user input, @@ -72,32 +72,31 @@ The Violentmonkey author expressed a desire to use such an API in ### Schema -This proposal builds upon the existing `browser.storage` API. The `StorageArea` -type that commonly describes the various areas within `browser.storage` is -extended with new methods to allow synchronous access to the data. +This proposal extends the `browser.storage` namespace with two new storage +areas. Their interface is inspired by the `StorageArea` type, except the +methods to look up stored keys/values are synchronous (`get` and `getKeys`). The `set` method includes a new optional `options` parameter to configure the -persistency of the specified parameters. +persistence of the specified parameters. ``` -// When execution context is allowed to read synchronously. - any ConfigStorageArea.getSync(keyOrKeysOrObjectWithDefaults: string | string[] | null | object) - string[] ConfigStorageArea.getKeysSync(); +// Methods available to extension contexts and content scripts: + any ConfigStorageArea.get(keyOrKeysOrObjectWithDefaults: string | string[] | null | object) + string[] ConfigStorageArea.getKeys(); + ExtensionEvent ConfigStorageArea.onChanged; -// These are the same interfaces as storage.StorageArea - Promise ConfigStorageArea.get(keyOrKeysOrObjectWithDefaults: string | string[] | null | object) +// Methods available to extension contexts only, not content scripts: Promise ConfigStorageArea.getBytesInUse() - Promise ConfigStorageArea.getKeys() Promise ConfigStorageArea.set(items: object, options?: ConfigStorageAreaSetOptions) Promise ConfigStorageArea.remove(keyOrKeys: string | string[]) Promise ConfigStorageArea.clear() - ExtensionEvent ConfigStorageArea.onChanged; number ConfigStorageArea.QUOTA_BYTES; -// Read-only to content scripts; no synchronous reading from extension contexts. +// In content scripts, the set/remove/clear methods are unavailable: ConfigStorageArea browser.storage.contentScriptConfig; -// Extension contexts only; can read synchronously and write. + +// Extension contexts only: ConfigStorageArea browser.storage.extensionConfig; dictionary ConfigStorageAreaSetOptions { @@ -105,41 +104,80 @@ dictionary ConfigStorageAreaSetOptions { } ``` +#### Explicit list of APIs + +This section describes the full list of APIs specified by this proposal. + +APIs available to content scripts: + +- `browser.storage.contentScriptConfig.get` +- `browser.storage.contentScriptConfig.getKeys` +- `browser.storage.contentScriptConfig.onChanged` +- `browser.storage.contentScriptConfig.QUOTA_BYTES` + +APIs available to privileged extension contexts: + +- `browser.storage.contentScriptConfig.get` +- `browser.storage.contentScriptConfig.getKeys` +- `browser.storage.contentScriptConfig.getBytesInUse` +- `browser.storage.contentScriptConfig.set` +- `browser.storage.contentScriptConfig.remove` +- `browser.storage.contentScriptConfig.clear` +- `browser.storage.contentScriptConfig.onChanged` +- `browser.storage.contentScriptConfig.QUOTA_BYTES` +- `browser.storage.extensionConfig.get` +- `browser.storage.extensionConfig.getKeys` +- `browser.storage.extensionConfig.getBytesInUse` +- `browser.storage.extensionConfig.set` +- `browser.storage.extensionConfig.remove` +- `browser.storage.extensionConfig.clear` +- `browser.storage.extensionConfig.onChanged` +- `browser.storage.extensionConfig.QUOTA_BYTES` + ### Behavior #### ConfigStorageArea -The `ConfigStorageArea` type is a composition of the existing `StorageArea` -interface and two new methods, `getSync` and `getKeysSync`. Upon invoking the -methods that update the storage area, the data is propagated to all processes -where a context may exists with a need for synchronous data access. +This proposal defines two fully independent `ConfigStorageArea` instances, +`contentScriptConfig` and `extensionConfig`. Privileged extension contexts have +full read and write access to both areas, whereas content scripts can only +read from `contentScriptConfig`. -Content scripts can *synchronously* access a copy of the available data with: +The `ConfigStorageArea` type is based on the `StorageArea` interface, except +with the `get` and `getKeys` methods returning data synchronously instead of +asynchronously. -- `browser.storage.contentScriptConfig.getSync(...)` -- `browser.storage.contentScriptConfig.getKeysSync(...)` +Updates to the storage area are propagated to all processes where a context may +exists with a need for synchronous data access. This proposal specifies the +`set`, `remove` and `clear` methods to update data, that eventually flushes. -These methods are unavailable to privileged extension contexts, to encourage -extensions to use the `extensionConfig` storage area that is hidden from -content scripts and only available to privileged extension contexts: +Once flushed, saved data should immediately be readable. In particular: -- `browser.storage.extensionConfig.getSync(...)` -- `browser.storage.extensionConfig.getKeysSync(...)` +- Background scripts / extension service workers should be able to read from + `extensionConfig` at any stage of their life cycle, including startup. +- Content scripts should be able to read from `contentScriptConfig`, even at + the earliest execution (`document_start`). -The asynchronous methods (inherit from the `StorageArea` interface) are only -available to privileged extension contexts, and allow them to update the data. -Notably, the asynchronous `get`, `getKeys` and `getBytesInUse` methods are not -available to content scripts, despite their read-only capabilities. +By default, data only lasts for the duration of the browser session. The `set` +method has a `persistent` option that extends the lifetime past browser and +extension restarts. The data may be cleared when an extension is uninstalled. -#### getSync +#### get -The `getSync()` method has similar semantics as the `get()` method, except it +The `get()` method has similar semantics as `StorageArea.get`, except it *synchronously* returns keys and values that are known locally in the process. -#### getKeysSync +#### getKeys + +The `getKeys()` method has similar semantics as `StorageArea.getKeys`, except +it *synchronously* returns keys that are known locally in the process. + +#### getBytesInUse + +The `getBytesInUse()` method returns the amount of bytes in use by the data. + +See `QUOTA_BYTES` for remarks about how quota is measured. -The `getKeysSync()` method has similar semantics as the `getKeys()` method, -except it *synchronously* returns keys that are known locally in the process. #### set @@ -151,9 +189,30 @@ specified key-value pairs are persisted across browser and extension restarts. Each key has its own associated `persistent` flag, which affect the behavior of the latest `set()` call, and any following `remove()` or `clear()` calls. +The `set` method can update multiple keys at once, and the `persistent` option +applies to all specified keys. To have different `persistent` flags for keys, +call the `set()` method again, with a different `persistent` option value. Persisted data should be stored locally, comparable to `storage.local`. +Updates are asynchronous. The returned `Promise` may await writes to disk when +the persistent flag is set, but does not guarantee the delivery of data to +other processes. This ensures that non-responsive processes cannot delay the +resolution of the `Promise`. The caller resides in the extension process and +is guaranteed to receive the updated value when `get` or `getSync` are called. + +#### remove + +Removes data associated with the specified keys. + +See the `set` method for remarks about this method's asynchronous behavior. + +#### clear + +Clears all data in this storage area. + +See the `set` method for remarks about this method's asynchronous behavior. + #### onChanged When a storage change is observed, the `browser.storage.onChanged`, @@ -162,11 +221,6 @@ When a storage change is observed, the `browser.storage.onChanged`, The `onChanged` event receives an object with all modified keys. -TODO: Should the `oldValue` and `newValue` fields be omitted? This would enable -the browser to lazily deserialize values on demand, as an optimization. -The recipient can use `getSync` to look up the actual value if desired. -See also https://github.com/w3c/webextensions/issues/475 - #### QUOTA_BYTES The maximum amount (in bytes) of data that can be stored, as measured by the @@ -237,4 +291,23 @@ is incompatible with that design. ## Future Work -Highlight any planned or prospective future work. +### Domain or Origin specific data + +This proposal offers one global storage area for content scripts shared between +all websites. A potential enhancement could be to offer a way to scope storage +to specific domains or origins. While the proposal currently allows extensions +to implement the domain-specific values themselves by including the domain or +origin in the key, the data would still be exposed across all processes, which +makes the mechanism less suited for data that ought to only be shared with +processes from a specific origin. + +### onChanged event filter + +The `onChanged` event fires for any storage change, which is inefficient if a +content script is only interested in changes to a specific key. + +There is a proposal to add filter options to `StorageArea.onChanged` at +[Issue 475](https://github.com/w3c/webextensions/issues/475). + +An alternative for extensions is to use `tabs.sendMessage` as a mechanism to +notify specific content scripts of data changes, but that would add overhead.