Config Improvements

cmake/deps.cmake

@@ -34,7 +34,7 @@ if (NOT "${_mapget_simfil_source_dir}" STREQUAL "")
             "SIMFIL_SHARED OFF")
 else()
     CPMAddPackage(
-        URI "gh:Klebert-Engineering/simfil@0.7.2#modelnode-data-accessor"
+        URI "gh:Klebert-Engineering/simfil@0.7.2"
         OPTIONS
             "SIMFIL_WITH_MODEL_JSON ON"
             "SIMFIL_SHARED OFF")

docs/mapget-api.md

@@ -80,6 +80,14 @@ To cancel an in-flight HTTP stream, close the HTTP connection.
   - `Status` frames contain UTF-8 JSON describing per-request `RequestStatus` transitions and a human-readable message. The final status frame has `"allDone": true`.
   - `LoadStateChange` exists in the protocol but is currently not emitted by the HTTP service.
 
+Each entry in a status frame's `requests` array contains `index`, `mapId`, `layerId`, numeric `status`, and `statusText`. For `NoDataSource` statuses, servers may also include `noDataSourceReason`:
+
+- `emptySources`
+- `allSourcesDisabled`
+- `datasourceInitializationFailed`
+- `missingMapOrLayer`
+- `noConfig`
+
 To cancel, either send a new request message on the same connection (which replaces the current one) or close the WebSocket connection.
 
 ## `/tiles/next` – pull binary tile frames
@@ -233,10 +241,17 @@ The page shows the number of active datasources and worker threads, cache statis
 The response contains:
 
 - `timestampMs`
-- `service`: service statistics, datasource info, cache occupancy, and optional tile-size-distribution data
+- `service`: service statistics, datasource info, cache occupancy, datasource-config counts, and optional tile-size-distribution data
 - `cache`: cache hit/miss counters and cache sizes
 - `tilesWebsocket`: control-channel metrics such as active sessions, pending queued frames for `/tiles/next`, blocked pull requests, and total forwarded bytes / frames
 
+`service.datasource-config` reports datasource YAML load diagnostics:
+
+- `configured`: number of entries under `sources`.
+- `enabled`: number of entries not disabled by `enabled: false`.
+- `disabled`: number of entries skipped because `enabled: false`.
+- `construction-failed`: number of enabled entries whose datasource construction failed.
+
 ## `/locate` – resolve external feature IDs
 
 `POST /locate` resolves external feature references to the tile IDs and feature IDs that contain them. This is commonly used together with feature search results or external databases that store map references.
@@ -257,7 +272,7 @@ Datasources are free to implement more advanced resolution schemes (for example
 
 ## `/config` – inspect and update configuration
 
-The `/config` endpoint family exposes the YAML configuration used by `mapget` for datasource wiring and HTTP settings. It is optional and can be enabled or disabled from the command line.
+The `/config` endpoint family exposes the YAML configuration used by `mapget` for datasource wiring and HTTP settings. Command-line flags control whether datasource config is exposed and whether updates are accepted.
 
 <!-- --8<-- [start:config-endpoints] -->
 
@@ -266,21 +281,37 @@ The `/config` endpoint family exposes the YAML configuration used by `mapget` fo
 - **Method:** `GET`
 - **Request body:** none
 - **Response:** `application/json` object with the keys:
-  - `model`: JSON representation of the current YAML config, limited to the `sources` and `http-settings` top‑level keys.
-  - `schema`: JSON Schema used to validate incoming configurations.
+  - `schema`: JSON Schema used to validate datasource-model configurations.
+  - `model`: JSON representation of the current YAML config, limited to datasource-model top-level keys such as `sources` and `http-settings`.
   - `readOnly`: boolean flag indicating whether `POST /config` is enabled.
+  - `datasourceConfigUnavailable`: boolean flag indicating that datasource config could not or must not be exposed.
+  - `datasourceConfigUnavailableReason`: `null` on success, otherwise a stable reason string.
+  - Additional public sections registered by the embedding application, returned as top-level siblings of `model`.
+
+When the endpoint handler is reached, `GET /config` returns HTTP `200`. If datasource configuration is unavailable, `schema` and `model` are empty objects, `readOnly` is `true`, and `datasourceConfigUnavailable` is `true`.
+
+Unavailable reason values are:
+
+- `getConfigDisabled`
+- `configPathUnset`
+- `configFileMissing`
+- `configFileOpenFailed`
+- `configParseFailed`
+- `configValidationFailed`
+
+On a successful datasource-config response, `datasourceConfigUnavailable` is `false` and `datasourceConfigUnavailableReason` is `null`. The returned model masks sensitive fields: any `password` or `api-key` values are replaced with stable masked tokens.
 
-This call is disabled if the server is started with `--no-get-config`. When enabled, it provides a safe way for tools and UIs to read the active configuration without exposing secrets in plain text: any `password` or `api-key` fields are replaced with stable masked tokens.
+Registered public sections are read-only. They are included as top-level siblings of `model` on successful responses. On datasource-config unavailable responses, registered public sections are still present but empty.
 
 ### `POST /config`
 
 - **Method:** `POST`
 - **Request body:** `application/json` matching the schema returned by `GET /config`.
-  - Must contain both `sources` and `http-settings` keys at the top level.
+  - Must contain the datasource-model keys required by the schema.
 - **Response:**
   - `text/plain` success message when the configuration was validated, written to disk and successfully applied.
   - `text/plain` error description and a 4xx/5xx status code if validation or application failed.
 
-This call is only accepted if the server is started with `--allow-post-config`. When a valid configuration is posted, mapget rewrites the underlying YAML file, preserving real secret values where masked tokens were supplied, and then reloads the datasource configuration. Clients should be prepared for temporary 5xx errors if reloading fails.
+This call is only accepted if the server is started with `--allow-post-config`. When a valid configuration is posted, mapget rewrites the datasource-model fields in the underlying YAML file, preserving real secret values where masked tokens were supplied, and then reloads the datasource configuration. Unknown top-level YAML sections, including registered public sections, are preserved but not edited through this endpoint. Clients should be prepared for temporary 5xx errors if reloading fails.
 
 <!-- --8<-- [end:config-endpoints] -->

docs/mapget-config.md

@@ -24,6 +24,8 @@ For integration with configuration UIs there is an additional top‑level key:
 
 - `http-settings` (optional) stores HTTP‑related settings used by frontends or tooling. Mapget itself does not interpret its contents, but exposes it via `/config` and ensures that sensitive fields such as `password` and `api-key` are masked in responses.
 
+Embedded applications can also register additional public top-level sections for `GET /config`. These sections are returned as siblings of `model`, not inside `model`, and are outside mapget's datasource schema. For example, MapViewer registers an `erdblick` section for frontend defaults. `POST /config` remains scoped to datasource-model keys and preserves unknown public sections in the YAML file.
+
 Changes to the `sources` section take effect while the server is running. Changes to options under `mapget` only apply after the server is restarted.
 
 ## The `sources` section
@@ -40,6 +42,14 @@ By default, the HTTP service registers the following datasource types:
 
 Additional datasource types can be registered from C++ code using `DataSourceConfigService`, but those are outside the scope of this guide.
 
+Every datasource entry accepts these generic fields in addition to type-specific fields:
+
+- `enabled` (optional, default `true`): when set to `false`, the entry is skipped before its type-specific constructor is called.
+- `ttl` (optional): cache time-to-live override in seconds. `0` means infinite.
+- `auth-header` (optional): header-to-regular-expression map as described below.
+
+Disabled entries are counted separately in service diagnostics and are not treated as construction failures.
+
 ### Restricting access with `auth-header`
 
 <!-- --8<-- [start:restrict-access] -->
@@ -361,14 +371,17 @@ mapget:
     cache-max-tiles: 20000      # --cache-max-tiles (0 disables the limit)
     clear-cache: false          # --clear-cache
     allow-post-config: true     # --allow-post-config (enables POST /config)
-    no-get-config: false        # --no-get-config (set to true to disable GET /config)
+    no-get-config: false        # --no-get-config (set to true to hide datasource config in GET /config)
+    webapp: /srv/my-ui          # --webapp, one application document root
+    static-mount:               # --static-mount, additional static aliases
+      - /assets:/srv/assets
     memory-trim-binary-interval: 100  # --memory-trim-binary-interval
     memory-trim-json-interval: 0      # --memory-trim-json-interval
 
 http-settings: ...
 sources: ...
 ```
 
-Adjust or omit fields as needed; unspecified options fall back to the same defaults as the CLI flags (for example, in-memory cache, port 0, GET `/config` enabled, POST `/config` disabled).
+Adjust or omit fields as needed; unspecified options fall back to the same defaults as the CLI flags (for example, in-memory cache, port 0, GET `/config` enabled, POST `/config` disabled). Static mount entries use `[<url-scope>:]<filesystem-path>` syntax and are served as plain files; mapget does not attach application-specific meaning to those files.
 
 <!-- --8<-- [end:yamlconf] -->

docs/mapget-dev-guide.md

@@ -221,6 +221,8 @@ For interactive clients, tile streaming uses WebSocket `GET /tiles` as a control
 - map HTTP/WebSocket endpoints to service calls (`/sources`, `/tiles`, `/tiles/next`, `/status`, `/status-data`, `/locate`, `/config`),
 - parse JSON requests and build `LayerTilesRequest` objects,
 - serialize tile responses as JSONL or binary streams,
+- mount static filesystem roots configured through `--webapp` and `--static-mount`,
+- expose `mapget::ensureStaticMount()` and `mapget::removeStaticMount()` for embedding applications that discover additional static roots after startup,
 - provide `/config` as a JSON view on the YAML config file.
 
 ### Tile streaming
@@ -249,10 +251,12 @@ This means `priorityTileIds` affects both backend scheduling and already-produce
 
 The HTTP service also implements `/config`:
 
-- `GET /config` reads the YAML configuration file, extracts the `sources` and `http-settings` keys, masks any `password` or `api-key` values and returns the result alongside the JSON Schema configured via `--config-schema`.
-- `POST /config` validates an incoming JSON document against that schema, merges it back into the YAML file (unmasking secrets) and relies on `DataSourceConfigService` to apply the updated datasource configuration.
+- `GET /config` reads the YAML configuration file, extracts datasource-model keys such as `sources` and `http-settings`, masks any `password` or `api-key` values and returns the result alongside the JSON Schema configured via `--config-schema`.
+- `GET /config` also merges public read-only sections registered through `DataSourceConfigService::registerPublicConfigSection(...)` as top-level siblings of `model`. Mapget does not interpret those sections itself.
+- If datasource configuration cannot be exposed, `GET /config` still returns HTTP `200` from the handler and reports the problem through `datasourceConfigUnavailable` plus `datasourceConfigUnavailableReason`.
+- `POST /config` validates an incoming datasource-model JSON document against the schema, merges it back into the YAML file (unmasking secrets), preserves unknown top-level sections, and relies on `DataSourceConfigService` to apply the updated datasource configuration.
 
-These endpoints are guarded by command‑line flags: `--no-get-config` disables the GET endpoint, and `--allow-post-config` enables the POST endpoint. They are primarily intended for configuration editors and admin tools.
+These endpoints are guarded by command-line flags: `--no-get-config` makes `GET /config` return the unavailable response shape, and `--allow-post-config` enables the POST endpoint. They are primarily intended for configuration editors and admin tools.
 
 ## Binary streaming and simfil integration
 

docs/mapget-setup.md

@@ -53,6 +53,21 @@ mapget --config examples/config/sample-service.yaml serve
 
 When `--config` is used, the HTTP service subscribes to the referenced YAML file. Any changes to its `sources` section are picked up while the server is running. Details of that file format are described in `mapget-config.md`.
 
+## Serving static files
+
+`mapget serve` can expose static filesystem directories in addition to datasource endpoints:
+
+```bash
+mapget serve \
+  --port 8080 \
+  --webapp /srv/my-ui \
+  --static-mount /assets:/srv/assets
+```
+
+`--webapp` is the single application document root, usually mounted at `/`. `--static-mount` is for additional static aliases and can be specified multiple times. Both options use `[<url-scope>:]<filesystem-path>` syntax; if the URL scope is omitted, the directory is mounted at `/`.
+
+Static mounts are generic HTTP serving functionality. Mapget does not interpret or validate the files beyond requiring the filesystem path to exist and be a directory. Higher-level applications such as MapViewer may use static mounts for their own assets, but application-specific configuration semantics live in those applications, not in mapget. Embedded applications that discover static roots after startup can call `mapget::ensureStaticMount()` instead of synthesizing command-line options.
+
 ## Using the fetch client
 
 The `fetch` subcommand is a simple HTTP client for `/tiles`. It requests one map layer for a set of tiles and prints each tile as a JSON feature collection.
@@ -78,4 +93,3 @@ For local experimentation it is often useful to increase log verbosity. You can
 - `MAPGET_LOG_FILE_MAXSIZE` limits the size of a log file before it is rotated.
 
 These settings apply to both the Python entry point and the native executable and are especially handy when debugging datasources or cache behaviour.
-

docs/mapget-user-guide.md

@@ -9,7 +9,7 @@ Most readers will only need the setup and API chapters. The model and configurat
 The guide is split into several focused documents:
 
 - [**Setup Guide**](mapget-setup.md) explains how to install mapget via `pip`, how to build the native executable from source, and how to start a server or use the built‑in `fetch` client for quick experiments.
-- [**Configuration Guide**](mapget-config.md) documents the YAML configuration file used with `--config`, the supported datasource types (`DataSourceHost`, `DataSourceProcess`, `GridDataSource`, `GeoJsonFolder`, `GeoJsonEndpoint`) and the optional `http-settings` section used by tools and UIs.
+- [**Configuration Guide**](mapget-config.md) documents the YAML configuration file used with `--config`, the supported datasource types (`DataSourceHost`, `DataSourceProcess`, `GridDataSource`, `GeoJsonFolder`, `GeoJsonEndpoint`), generic datasource fields such as `enabled`, and optional UI-facing sections such as `http-settings`.
 - [**HTTP / WebSocket API Guide**](mapget-api.md) describes the endpoints exposed by `mapget serve`, including `/sources`, `/tiles`, `/tiles/next`, `/status`, `/status-data`, `/locate` and `/config`, along with their request and response formats and example calls.
 - [**Caching Guide**](mapget-cache.md) covers the available cache modes (`memory`, `persistent`, `none`), explains how to configure cache size and location, and shows how to inspect cache statistics via the status endpoint.
 - [**Simfil Language Extensions**](mapget-simfil-extensions.md) introduces the feature model, tiling scheme, geometry and validity concepts, and the binary tile stream format. This chapter is especially relevant if you are writing datasources or low‑level clients.

libs/http-datasource/include/mapget/detail/http-server.h

@@ -1,6 +1,7 @@
 #pragma once
 
 #include <cstdint>
+#include <filesystem>
 #include <memory>
 #include <string>
 
@@ -90,4 +91,30 @@ class HttpServer
     std::unique_ptr<Impl> impl_;
 };
 
+/**
+ * Ensure that a static filesystem mount is available to the running HTTP server.
+ * This is intended for embedding applications that discover additional static
+ * assets after the server has already started. The mount is process-global
+ * because Drogon exposes a process-global application instance.
+ *
+ * @param urlPrefix URL path prefix, e.g. `/assets`.
+ * @param filesystemRoot Directory to serve under the prefix.
+ * @return true when the mount exists or was added successfully.
+ */
+bool ensureStaticMount(
+    std::string const& urlPrefix,
+    std::filesystem::path const& filesystemRoot);
+
+/**
+ * Ensure a static mount using the same `[<url-scope>:]<filesystem-path>` syntax
+ * as HttpServer::mountFileSystem().
+ */
+bool ensureStaticMount(std::string const& pathFromTo);
+
+/**
+ * Remove a runtime static mount previously added through ensureStaticMount().
+ * Startup mounts added through HttpServer::mountFileSystem() are not affected.
+ */
+bool removeStaticMount(std::string const& urlPrefix);
+
 }  // namespace mapget