fix: Updated instructions in drop-in readme, changed default KLL parameter

asap-dropin/.env

@@ -1,28 +1,30 @@
 # ── User's existing Prometheus ────────────────────────────────────────────────
-# URL of your running Prometheus instance, reachable from inside Docker.
-#   - Docker Desktop (Mac/Windows): http://host.docker.internal:9090
-#   - Linux (host networking):      http://172.17.0.1:9090   (default Docker bridge gateway)
-#   - Prometheus in another compose: use a shared Docker network and the service name
-PROMETHEUS_URL=http://prometheus:9090
+# URL of your running Prometheus, reachable from inside the ASAPQuery container.
+#   - Mode A / B (Prometheus on bare metal or Docker host network):
+#       http://localhost:9090  (ASAPQuery runs on host network — see README Step 2a)
+#   - Mode C (Prometheus in a Docker bridge/compose network):
+#       http://<prometheus-service-name>:9090  (shared Docker network — see README Step 2b)
+PROMETHEUS_URL=http://localhost:9090
 
 # Scrape interval configured in your Prometheus (in seconds).
 PROMETHEUS_SCRAPE_INTERVAL=15
 
 # ── Exposed ports ────────────────────────────────────────────────────────────
-# Port on the host where the remote-write receiver listens.
-# Add this to your prometheus.yml:
+# Port where ASAPQuery's remote-write receiver listens (Prometheus sends data here).
+# Add to your prometheus.yml:
 #   remote_write:
-#     - url: http://localhost:${REMOTE_WRITE_PORT}/api/v1/write
-REMOTE_WRITE_PORT=9091
+#     - url: http://localhost:${ASAPQUERY_DATA_PORT}/api/v1/write   # Modes A/B
+#     - url: http://queryengine:${ASAPQUERY_DATA_PORT}/api/v1/write  # Mode C
+ASAPQUERY_DATA_PORT=9091
 
-# Port on the host where the ASAPQuery query engine listens.
-# Point your Grafana Prometheus datasource here:
-#   http://localhost:${QUERY_ENGINE_PORT}
-QUERY_ENGINE_PORT=8088
+# Port where the ASAPQuery query engine listens (point your Grafana datasource here).
+#   - Grafana on bare metal: http://localhost:${ASAPQUERY_QUERY_PORT}
+#   - Grafana in Docker (same network as ASAPQuery): http://queryengine:${ASAPQUERY_QUERY_PORT}
+ASAPQUERY_QUERY_PORT=8088
 
 # ── Observation window ────────────────────────────────────────────────────────
 # How long ASAPQuery observes Grafana queries before generating an acceleration
 # plan (in seconds). All queries are forwarded to Prometheus during this window.
 # Rule of thumb: set this to at least 3× your Grafana dashboard refresh interval.
 # Example: 30s refresh → 90s minimum; 1m refresh → 180s minimum.
-TRACKER_OBSERVATION_WINDOW_SECS=180
+TRACKER_OBSERVATION_WINDOW_SECS=60

asap-dropin/README.md

@@ -2,22 +2,25 @@
 
 A self-contained single-container Docker Compose that adds ASAPQuery to an existing Prometheus and Grafana deployment.
 
-On startup, all queries are forwarded transparently to your upstream Prometheus. After one observation window (default 180s), the engine automatically plans and activates sketch-based acceleration based on the real queries it observed from Grafana.
+On startup, all queries are forwarded transparently to your upstream Prometheus. After one observation window (default 60s), the engine automatically plans and activates sketch-based acceleration based on the real queries it observed from Grafana.
 
 ## Prerequisites
 
 - Docker and Docker Compose
 - A running Prometheus instance
 - A running Grafana instance (with a Prometheus datasource)
 
-## Architecture
+## Deployment modes
 
-```
-Prometheus ──remote_write──▶ ASAPQuery (:9091)
-    ▲                              │
-    │ unsupported queries          ▼ builds sketches
-    └──────────── ASAPQuery (:8088) ◀── Grafana
-```
+How you configure `PROMETHEUS_URL` (the URL ASAPQuery uses to reach Prometheus) and the `remote_write` URL (the URL Prometheus uses to reach ASAPQuery) depends on how Prometheus is running:
+
+| Mode | Description |
+|---|---|
+| **A — Prometheus on bare metal** | Prometheus runs as a process directly on the host |
+| **B — Prometheus in Docker, host network** | Prometheus runs in a container with `--network host` |
+| **C — Prometheus in Docker, bridge/compose network** | Prometheus runs in a container on a named Docker network |
+
+Modes A and B behave identically from a networking perspective. Mode C requires an extra step to join ASAPQuery to Prometheus's Docker network.
 
 ## Setup
 
@@ -27,23 +30,34 @@ Edit `.env` to match your deployment:
 
 | Variable | Default | Description |
 |---|---|---|
-| `PROMETHEUS_URL` | `http://host.docker.internal:9090` | URL of your Prometheus, reachable from inside the ASAPQuery container |
+| `PROMETHEUS_URL` | `http://localhost:9090` | URL of your Prometheus, reachable from inside the ASAPQuery container |
 | `PROMETHEUS_SCRAPE_INTERVAL` | `15` | Your Prometheus scrape interval in seconds |
-| `REMOTE_WRITE_PORT` | `9091` | ASAPQuery data ingest port — must be free on the host |
-| `QUERY_ENGINE_PORT` | `8088` | ASAPQuery query endpoint port — must be free on the host |
-| `TRACKER_OBSERVATION_WINDOW_SECS` | `180` | How long to observe queries before planning (see note below) |
+| `ASAPQUERY_DATA_PORT` | `9091` | ASAPQuery data ingest port — must be free on the host |
+| `ASAPQUERY_QUERY_PORT` | `8088` | ASAPQuery query endpoint port — must be free on the host |
+| `TRACKER_OBSERVATION_WINDOW_SECS` | `60` | How long to observe queries before planning (see note below) |
 
-**Finding the right `PROMETHEUS_URL`:**
-- **Prometheus on the same host as Docker:** `http://172.17.0.1:9090` (default Docker bridge gateway on Linux)
-- **Prometheus in another Docker Compose:** use a shared external Docker network and the Prometheus service name
+**Finding the right `PROMETHEUS_URL`** (from inside the ASAPQuery container):
+
+- **Mode A / B:** `http://localhost:9090` — ASAPQuery runs on the host network (see Step 2a), so `localhost` resolves to the host ← default
+- **Mode C:** use a shared Docker network (see Step 2b) and the Prometheus service name, e.g. `http://prometheus:9090`
 
 **Setting `TRACKER_OBSERVATION_WINDOW_SECS`:**
 Set this to at least 3× your Grafana dashboard refresh interval so ASAPQuery sees enough query repetitions to build a useful plan.
+- Grafana refresh 10s → set to 30 or higher
 - Grafana refresh 30s → set to 90 or higher
-- Grafana refresh 1m → set to 180 or higher (default)
-- Grafana refresh 5m → set to 900 or higher
+- Grafana refresh 1m → set to 180 or higher
+
+### Step 2a — Start ASAPQuery (Mode A and B)
+
+Run ASAPQuery on the host network so it can reach Prometheus at `localhost`. Create a `docker-compose.override.yml`:
 
-### Step 2 — Start ASAPQuery
+```yaml
+services:
+  queryengine:
+    network_mode: host
+```
+
+Then start:
 
 ```bash
 docker compose up -d
@@ -55,13 +69,70 @@ Verify it started:
 docker compose logs queryengine
 ```
 
-You should see a line confirming Prometheus is reachable, then the engine waiting for the observation window.
+You should see these two lines confirming Prometheus is reachable and the tracker is running:
+
+```
+INFO  Prometheus reachable at http://localhost:9090
+INFO  Query tracker enabled (observation window: 60s)
+```
+
+### Step 2b — Start ASAPQuery and join Prometheus's network (Mode C)
+
+First, identify the name of the Docker network Prometheus is attached to:
+
+```bash
+docker inspect <prometheus-container-name> --format '{{json .NetworkSettings.Networks}}' | jq 'keys'
+```
+
+Add that network as an external network in a `docker-compose.override.yml`:
+
+```yaml
+networks:
+  prometheus-net:
+    external: true
+    name: <the-network-name-from-above>
+
+services:
+  queryengine:
+    networks:
+      - prometheus-net
+```
+
+Set `PROMETHEUS_URL` in `.env` to the Prometheus service name on that network, e.g.:
+
+```
+PROMETHEUS_URL=http://prometheus:9090
+```
+
+Then start:
+
+```bash
+docker compose up -d
+```
+
+Verify it started:
+
+```bash
+docker compose logs queryengine
+```
+
+You should see:
+
+```
+INFO  Prometheus reachable at http://prometheus:9090
+INFO  Query tracker enabled (observation window: 60s)
+```
 
 ### Step 3 — Configure Prometheus remote_write
 
 Prometheus needs to send all ingested samples to ASAPQuery so it can build sketches.
 
-**Add this block to your `prometheus.yml`:**
+The `remote_write` URL is written from **Prometheus's perspective**, so it depends on where Prometheus is running.
+Change `9091` if you set a different `ASAPQUERY_DATA_PORT` in `.env`.
+
+#### Mode A and B — Prometheus on bare metal or host network
+
+ASAPQuery is on the host network (Step 2a), so Prometheus reaches it at `localhost`:
 
 ```yaml
 remote_write:
@@ -71,10 +142,17 @@ remote_write:
       sample_age_limit: 5m
 ```
 
-> **Finding the right `remote_write` URL:** The URL is from Prometheus's perspective.
-> - **Prometheus on the same host as Docker:** `http://localhost:9091/api/v1/write` (default above)
-> - **Prometheus in Docker on the same host:** `http://host.docker.internal:9091/api/v1/write` (Mac/Windows) or `http://172.17.0.1:9091/api/v1/write` (Linux)
-> - Change `9091` if you set a different `REMOTE_WRITE_PORT` in `.env`
+#### Mode C — Prometheus in Docker, bridge/compose network
+
+After joining the shared network (Step 2b), Prometheus can reach ASAPQuery by container hostname:
+
+```yaml
+remote_write:
+  - url: http://queryengine:9091/api/v1/write
+    queue_config:
+      batch_send_deadline: 1s
+      sample_age_limit: 5m
+```
 
 **Reload Prometheus to apply the change:**
 
@@ -97,12 +175,12 @@ Create a new datasource in Grafana pointing at ASAPQuery, then switch your dashb
 1. Open Grafana in your browser
 2. Go to **Connections → Data Sources**
 3. Click **Add new data source** and select **Prometheus**
-4. Set the **Name** to something like `ASAPQuery`
-5. Set the **URL** to:
-   ```
-   http://localhost:8088
-   ```
-   (Change the port if you set a different `QUERY_ENGINE_PORT` in `.env`)
+4. Set the **Name** to `ASAPQuery`
+5. Set the **URL** to the ASAPQuery query endpoint, as seen from Grafana:
+   - **Grafana on bare metal (Mode A/B, or any mode):** `http://localhost:8088`
+   - **Grafana in Docker on the same network as ASAPQuery (Mode C):** `http://queryengine:8088`
+
+   Change the port if you set a different `ASAPQUERY_QUERY_PORT` in `.env`.
 6. Click **Save & Test** — you should see "Data source is working"
 7. Open your dashboards and switch their datasource to `ASAPQuery`
 
@@ -118,9 +196,10 @@ After the observation window elapses, check the ASAPQuery logs:
 docker compose logs queryengine | grep query_tracker
 ```
 
-You should see lines like:
+You should see a line like:
+
 ```
-query_tracker: planner succeeded — streaming aggregations: N, inference queries: M
+INFO query_tracker: planner succeeded — streaming aggregations: N, inference queries: M, punted: P
 ```
 
 From this point on, check the routing in the logs:
@@ -129,7 +208,7 @@ From this point on, check the routing in the logs:
 docker compose logs queryengine | grep "destination="
 ```
 
-Lines with `destination=asap` are served by ASAPQuery; lines with `destination=prometheus` are forwarded to your upstream Prometheus.
+Lines with `destination=asap` are served by ASAPQuery; lines with `destination=prometheus` are forwarded to your upstream Prometheus; lines with `destination=none_unsupported` are queries ASAPQuery does not support (also forwarded).
 
 ## Development
 

asap-dropin/docker-compose.yml

@@ -5,11 +5,11 @@ name: asapquery-dropin
 # Usage:
 #   1. Adjust PROMETHEUS_URL in .env to point at your Prometheus.
 #   2. docker compose up -d
-#   3. Add remote_write to your prometheus.yml  ->  http://localhost:${REMOTE_WRITE_PORT}/receive
-#   4. Point your Grafana datasource URL        ->  http://localhost:${QUERY_ENGINE_PORT}
+#   3. Add remote_write to your prometheus.yml  ->  http://localhost:${ASAPQUERY_DATA_PORT}/api/v1/write
+#   4. Point your Grafana datasource URL        ->  http://localhost:${ASAPQUERY_QUERY_PORT}
 #
 # The query engine starts with an empty plan and forwards all queries to Prometheus.
-# After the observation window (default 180s), it automatically generates a plan
+# After the observation window (default 60s), it automatically generates a plan
 # based on real query patterns and begins precomputing sketches.
 
 networks:
@@ -24,15 +24,15 @@ services:
     networks:
       - asap-network
     ports:
-      - "${QUERY_ENGINE_PORT:-8088}:8088"
-      - "${REMOTE_WRITE_PORT:-9091}:9091"
+      - "${ASAPQUERY_QUERY_PORT:-8088}:8088"
+      - "${ASAPQUERY_DATA_PORT:-9091}:9091"
     environment:
       - RUST_LOG=INFO
       - RUST_BACKTRACE=1
     volumes:
       - ./output/queryengine:/app/outputs
     command:
-      - "--prometheus-server=${PROMETHEUS_URL:-http://host.docker.internal:9090}"
+      - "--prometheus-server=${PROMETHEUS_URL:-http://localhost:9090}"
       - "--prometheus-scrape-interval=${PROMETHEUS_SCRAPE_INTERVAL:-15}"
       - "--streaming-engine=precompute"
       - "--prometheus-remote-write-port=9091"

asap-planner-rs/src/planner/logics.rs

@@ -10,7 +10,7 @@ use std::collections::HashMap;
 const DEFAULT_CMS_DEPTH: u64 = 3;
 const DEFAULT_CMS_WIDTH: u64 = 1024;
 const DEFAULT_CMS_HEAP_MULT: u64 = 4;
-const DEFAULT_KLL_K: u64 = 20;
+const DEFAULT_KLL_K: u64 = 500;
 const DEFAULT_HYDRA_ROW: u64 = 3;
 const DEFAULT_HYDRA_COL: u64 = 1024;
 const DEFAULT_HYDRA_K: u64 = 20;