Release 2026.2.0

CMakeLists.txt

@@ -7,7 +7,7 @@ project(mapget LANGUAGES CXX C)
 # Allow version to be set from command line for CI/CD
 # For local development, use the default version
 if(NOT DEFINED MAPGET_VERSION)
-  set(MAPGET_VERSION 2026.1.3)
+  set(MAPGET_VERSION 2026.2.0)
 endif()
 
 set(CMAKE_CXX_STANDARD 20)

cmake/deps.cmake

@@ -34,7 +34,7 @@ if (NOT "${_mapget_simfil_source_dir}" STREQUAL "")
             "SIMFIL_SHARED OFF")
 else()
     CPMAddPackage(
-        URI "gh:Klebert-Engineering/simfil#v0.7.0"
+        URI "gh:Klebert-Engineering/simfil@0.7.2"
         OPTIONS
             "SIMFIL_WITH_MODEL_JSON ON"
             "SIMFIL_SHARED OFF")
@@ -139,7 +139,7 @@ endif()
     endif()
 
     CPMAddPackage(
-        URI "gh:jbeder/yaml-cpp#aa8d4e@0.8.0" # Use > 0.8.0 once available.
+        URI "gh:jbeder/yaml-cpp#yaml-cpp-0.9.0@0.9.0"
         GIT_SHALLOW OFF
         OPTIONS
             "YAML_CPP_BUILD_TESTS OFF"
@@ -161,7 +161,7 @@ if (MAPGET_WITH_SERVICE OR MAPGET_WITH_HTTPLIB OR MAPGET_ENABLE_TESTING)
 endif()
 
 if (MAPGET_WITH_WHEEL AND NOT TARGET python-cmake-wheel)
-    CPMAddPackage("gh:Klebert-Engineering/python-cmake-wheel#v1.2.7@1.2.7")
+    CPMAddPackage("gh:Klebert-Engineering/python-cmake-wheel#v1.2.8@1.2.8")
 endif()
 
 if (MAPGET_ENABLE_TESTING)

docs/mapget-api.md

@@ -80,6 +80,14 @@ To cancel an in-flight HTTP stream, close the HTTP connection.
   - `Status` frames contain UTF-8 JSON describing per-request `RequestStatus` transitions and a human-readable message. The final status frame has `"allDone": true`.
   - `LoadStateChange` exists in the protocol but is currently not emitted by the HTTP service.
 
+Each entry in a status frame's `requests` array contains `index`, `mapId`, `layerId`, numeric `status`, and `statusText`. For `NoDataSource` statuses, servers may also include `noDataSourceReason`:
+
+- `emptySources`
+- `allSourcesDisabled`
+- `datasourceInitializationFailed`
+- `missingMapOrLayer`
+- `noConfig`
+
 To cancel, either send a new request message on the same connection (which replaces the current one) or close the WebSocket connection.
 
 ## `/tiles/next` – pull binary tile frames
@@ -111,11 +119,7 @@ Each line in the JSONL response is a GeoJSON-like FeatureCollection with additio
   "mapgetTileId": 281479271743500,
   "mapId": "EuropeHD",
   "mapgetLayerId": "Roads",
-  "idPrefix": {
-    "areaId": 123,
-    "tileId": 456
-  },
-  "timestamp": "2025-01-14T10:30:00.000000Z",
+  "timestamp": 1736850600000000,
   "ttl": 3600000,
   "error": {
     "code": 404,
@@ -131,8 +135,7 @@ Each line in the JSONL response is a GeoJSON-like FeatureCollection with additio
 | `mapgetTileId` | integer | The mapget tile ID (64-bit decimal) |
 | `mapId` | string | Map identifier |
 | `mapgetLayerId` | string | Layer identifier within the map |
-| `idPrefix` | object | Common ID parts shared by all features in this tile (optional) |
-| `timestamp` | string | ISO 8601 timestamp when the tile was created |
+| `timestamp` | integer | Tile creation time in microseconds since the Unix epoch |
 | `ttl` | integer | Time-to-live in milliseconds (optional) |
 | `error` | object | Error information if tile creation failed (optional) |
 | `error.code` | integer | Numeric error code, e.g., HTTP status or database error (optional) |
@@ -238,10 +241,17 @@ The page shows the number of active datasources and worker threads, cache statis
 The response contains:
 
 - `timestampMs`
-- `service`: service statistics, datasource info, cache occupancy, and optional tile-size-distribution data
+- `service`: service statistics, datasource info, cache occupancy, datasource-config counts, and optional tile-size-distribution data
 - `cache`: cache hit/miss counters and cache sizes
 - `tilesWebsocket`: control-channel metrics such as active sessions, pending queued frames for `/tiles/next`, blocked pull requests, and total forwarded bytes / frames
 
+`service.datasource-config` reports datasource YAML load diagnostics:
+
+- `configured`: number of entries under `sources`.
+- `enabled`: number of entries not disabled by `enabled: false`.
+- `disabled`: number of entries skipped because `enabled: false`.
+- `construction-failed`: number of enabled entries whose datasource construction failed.
+
 ## `/locate` – resolve external feature IDs
 
 `POST /locate` resolves external feature references to the tile IDs and feature IDs that contain them. This is commonly used together with feature search results or external databases that store map references.
@@ -262,7 +272,7 @@ Datasources are free to implement more advanced resolution schemes (for example
 
 ## `/config` – inspect and update configuration
 
-The `/config` endpoint family exposes the YAML configuration used by `mapget` for datasource wiring and HTTP settings. It is optional and can be enabled or disabled from the command line.
+The `/config` endpoint family exposes the YAML configuration used by `mapget` for datasource wiring and HTTP settings. Command-line flags control whether datasource config is exposed and whether updates are accepted.
 
 <!-- --8<-- [start:config-endpoints] -->
 
@@ -271,21 +281,37 @@ The `/config` endpoint family exposes the YAML configuration used by `mapget` fo
 - **Method:** `GET`
 - **Request body:** none
 - **Response:** `application/json` object with the keys:
-  - `model`: JSON representation of the current YAML config, limited to the `sources` and `http-settings` top‑level keys.
-  - `schema`: JSON Schema used to validate incoming configurations.
+  - `schema`: JSON Schema used to validate datasource-model configurations.
+  - `model`: JSON representation of the current YAML config, limited to top-level keys in the active datasource schema. The built-in schema includes `sources`; deployments can add keys such as `http-settings` through `--config-schema`.
   - `readOnly`: boolean flag indicating whether `POST /config` is enabled.
+  - `datasourceConfigUnavailable`: boolean flag indicating that datasource config could not or must not be exposed.
+  - `datasourceConfigUnavailableReason`: `null` on success, otherwise a stable reason string.
+  - Additional public sections registered by the embedding application, returned as top-level siblings of `model`.
+
+When the endpoint handler is reached, `GET /config` returns HTTP `200`. `readOnly` reflects whether `POST /config` is enabled. If `--no-get-config` is set, `datasourceConfigUnavailable` is `true`, `datasourceConfigUnavailableReason` is `getConfigDisabled`, and `model` is empty. In that state, writable servers still return `schema` so clients can present an empty replacement editor; read-only servers return an empty schema.
+
+Unavailable reason values are:
+
+- `getConfigDisabled`
+- `configPathUnset`
+- `configFileMissing`
+- `configFileOpenFailed`
+- `configParseFailed`
+- `configValidationFailed`
+
+On a successful datasource-config response, `datasourceConfigUnavailable` is `false` and `datasourceConfigUnavailableReason` is `null`. The returned model masks sensitive fields: any `password` or `api-key` values are replaced with stable masked tokens.
 
-This call is disabled if the server is started with `--no-get-config`. When enabled, it provides a safe way for tools and UIs to read the active configuration without exposing secrets in plain text: any `password` or `api-key` fields are replaced with stable masked tokens.
+Registered public sections are read-only. They are included as top-level siblings of `model` when the YAML config can be read and parsed, even if the datasource model itself is hidden through `--no-get-config`. If the YAML config cannot be read or parsed, registered public sections are still present but empty.
 
 ### `POST /config`
 
 - **Method:** `POST`
 - **Request body:** `application/json` matching the schema returned by `GET /config`.
-  - Must contain both `sources` and `http-settings` keys at the top level.
+  - Must contain the datasource-model keys required by the schema.
 - **Response:**
   - `text/plain` success message when the configuration was validated, written to disk and successfully applied.
   - `text/plain` error description and a 4xx/5xx status code if validation or application failed.
 
-This call is only accepted if the server is started with `--allow-post-config`. When a valid configuration is posted, mapget rewrites the underlying YAML file, preserving real secret values where masked tokens were supplied, and then reloads the datasource configuration. Clients should be prepared for temporary 5xx errors if reloading fails.
+This call is only accepted if the server is started with `--allow-post-config`. When a valid configuration is posted, mapget rewrites the datasource-model fields in the underlying YAML file, preserving real secret values where masked tokens were supplied, and then reloads the datasource configuration. Unknown top-level YAML sections, including registered public sections, are preserved but not edited through this endpoint. Clients should be prepared for temporary 5xx errors if reloading fails.
 
 <!-- --8<-- [end:config-endpoints] -->

docs/mapget-config.md

@@ -22,7 +22,9 @@ Two top‑level keys are relevant for mapget itself:
 
 For integration with configuration UIs there is an additional top‑level key:
 
-- `http-settings` (optional) stores HTTP‑related settings used by frontends or tooling. Mapget itself does not interpret its contents, but exposes it via `/config` and ensures that sensitive fields such as `password` and `api-key` are masked in responses.
+- `http-settings` (optional) stores HTTP‑related settings used by frontends or tooling. Mapget itself does not interpret its contents. It is exposed through `/config.model` only when the active datasource schema includes `http-settings`, typically via a deployment-specific `--config-schema` patch.
+
+Embedded applications can also register additional public top-level sections for `GET /config`. These sections are returned as siblings of `model`, not inside `model`, and are outside mapget's datasource schema. For example, MapViewer registers an `erdblick` section for frontend defaults. `POST /config` remains scoped to datasource-model keys and preserves unknown public sections in the YAML file.
 
 Changes to the `sources` section take effect while the server is running. Changes to options under `mapget` only apply after the server is restarted.
 
@@ -35,10 +37,19 @@ By default, the HTTP service registers the following datasource types:
 - `DataSourceHost` – connect to a remote `DataSourceServer` over HTTP.
 - `DataSourceProcess` – spawn a datasource process locally and connect to it.
 - `GridDataSource` – generate synthetic tiles on the fly for testing and benchmarking.
-- `GeoJsonFolder` – serve features from a directory containing `.geojson` files named by tile ID.
+- `GeoJsonFolder` – serve features from local GeoJSON files.
+- `GeoJsonEndpoint` – serve features from GeoJSON files fetched over HTTP.
 
 Additional datasource types can be registered from C++ code using `DataSourceConfigService`, but those are outside the scope of this guide.
 
+Every datasource entry accepts these generic fields in addition to type-specific fields:
+
+- `enabled` (optional, default `true`): when set to `false`, the entry is skipped before its type-specific constructor is called.
+- `ttl` (optional): cache time-to-live override in seconds. `0` means infinite.
+- `auth-header` (optional): header-to-regular-expression map as described below.
+
+Disabled entries are counted separately in service diagnostics and are not treated as construction failures.
+
 ### Restricting access with `auth-header`
 
 <!-- --8<-- [start:restrict-access] -->
@@ -205,6 +216,8 @@ Optional fields:
 
 - `mapId`: optional map ID override. If omitted, mapget derives a display name from the input directory.
 - `withAttrLayers` (default: `true`): boolean flag. If `true`, nested objects in the GeoJSON `properties` are converted to mapget attribute layers; if `false`, only scalar top‑level properties are emitted and nested objects are silently dropped.
+- `dataSourceInfo`: inline datasource info object, local YAML/JSON file path, or HTTP(S) URL.
+- `tilePathTemplate`: relative path template used when `dataSourceInfo` is configured. Supported placeholders are `{tileId}` and `{layerId}`.
 
 Example:
 
@@ -216,9 +229,26 @@ sources:
     withAttrLayers: true
 ```
 
-#### Manifest Mode (Recommended)
+Example with explicit layer metadata and template-based file lookup:
+
+```yaml
+sources:
+  - type: GeoJsonFolder
+    folder: /data/tiles
+    dataSourceInfo: /data/tiles/info.yaml
+    tilePathTemplate: "{layerId}/{tileId}.geojson"
+```
+
+If `dataSourceInfo` is omitted, mapget falls back to legacy discovery:
+
+- first it looks for a legacy `manifest.json`
+- if no manifest exists, it scans the folder for files named `<tileId>.geojson`
 
-If a `manifest.json` file exists in the input directory, it is used to map filenames to tile IDs and layers. This allows arbitrary filenames and multi‑layer support.
+In that fallback mode, metadata is synthesized as a single `GeoJsonAny` layer and conversion is best-effort.
+
+#### Manifest Mode (Legacy)
+
+If no explicit `dataSourceInfo` is configured and a `manifest.json` file exists in the input directory, it is used to map filenames to tile IDs and layers. This allows arbitrary filenames and multi‑layer support.
 
 **Manifest Structure:**
 
@@ -276,11 +306,39 @@ If no `manifest.json` exists, the datasource falls back to scanning for files na
 
 <!-- --8<-- [end:geojson] -->
 
+### `GeoJsonEndpoint`
+
+`GeoJsonEndpoint` serves GeoJSON tiles from an HTTP(S) endpoint. It uses the same GeoJSON conversion logic as `GeoJsonFolder`, but fetches each tile body over the network.
+
+Required fields:
+
+- `type`: must be `GeoJsonEndpoint`.
+- `baseUrl`: base HTTP(S) URL used to fetch GeoJSON tiles.
+
+Optional fields:
+
+- `mapId`: optional map ID override. If omitted, mapget derives a display name from the base URL.
+- `withAttrLayers` (default: `true`): converts nested GeoJSON property objects to mapget attribute layers.
+- `dataSourceInfo`: inline datasource info object, local YAML/JSON file path, or HTTP(S) URL.
+- `tileUrlTemplate`: URL or relative path template used to fetch tiles. Supported placeholders are `{tileId}`, `{layerId}`, and `{baseUrl}`.
+
+Example:
+
+```yaml
+sources:
+  - type: GeoJsonEndpoint
+    baseUrl: https://example.test/tiles
+    dataSourceInfo: https://example.test/info.yaml
+    tileUrlTemplate: "{layerId}/{tileId}.geojson"
+```
+
+If `dataSourceInfo` is omitted, mapget emits a strong warning and falls back to a synthesized single-layer `GeoJsonAny` datasource with empty coverage. In that fallback mode, conversion is still attempted, but service discovery remains intentionally limited.
+
 ## HTTP settings for tools and UIs
 
 The optional `http-settings` top‑level key is reserved for HTTP‑related configuration used by tools and user interfaces. It is typically a list of objects that may contain fields such as `scope`, `api-key` or `password`.
 
-Mapget itself treats this section as opaque data: it is read and written via the `/config` endpoint but not interpreted when serving tiles. When returning the configuration, mapget replaces the values of any `api-key` or `password` fields with masked tokens. When a modified configuration is posted back, these tokens are resolved to the original secret values before the YAML file is updated.
+Mapget itself treats this section as opaque data and does not interpret it when serving tiles. It is included in `/config.model` only when the active datasource schema contains an `http-settings` property, for example through a deployment-specific `--config-schema` patch. When returning the configuration, mapget replaces the values of any `api-key` or `password` fields with masked tokens. When a modified configuration is posted back, these tokens are resolved to the original secret values before the YAML file is updated.
 
 ## Environment variables
 
@@ -313,14 +371,26 @@ mapget:
     cache-max-tiles: 20000      # --cache-max-tiles (0 disables the limit)
     clear-cache: false          # --clear-cache
     allow-post-config: true     # --allow-post-config (enables POST /config)
-    no-get-config: false        # --no-get-config (set to true to disable GET /config)
+    no-get-config: false        # --no-get-config (hides datasource model in GET /config)
+    webapp: /srv/my-ui          # --webapp, one application document root
+    static-mount:               # --static-mount, additional static aliases
+      - /assets:/srv/assets
     memory-trim-binary-interval: 100  # --memory-trim-binary-interval
     memory-trim-json-interval: 0      # --memory-trim-json-interval
 
 http-settings: ...
 sources: ...
 ```
 
-Adjust or omit fields as needed; unspecified options fall back to the same defaults as the CLI flags (for example, in-memory cache, port 0, GET `/config` enabled, POST `/config` disabled).
+Adjust or omit fields as needed; unspecified options fall back to the same defaults as the CLI flags (for example, in-memory cache, port 0, GET `/config` enabled, POST `/config` disabled). Static mount entries use `[<url-scope>:]<filesystem-path>` syntax and are served as plain files; mapget does not attach application-specific meaning to those files.
+
+Datasource editor visibility is controlled by `allow-post-config` and `no-get-config`:
+
+| `allow-post-config` | `no-get-config` | `/config` datasource model | Editor behaviour |
+| --- | --- | --- | --- |
+| `false` | `false` | Current model is returned. | Read-only. |
+| `true` | `false` | Current model is returned. | Editable. |
+| `false` | `true` | Model and schema are empty. | Disabled. |
+| `true` | `true` | Model is empty, schema is returned. | Empty editor; applying overwrites datasource config. |
 
 <!-- --8<-- [end:yamlconf] -->