diff --git a/README.md b/README.md index fd8f465..e0470f5 100644 --- a/README.md +++ b/README.md @@ -8,11 +8,11 @@ - **Configurable query accuracy** - **Ease-of-use with your existing tech stack** -![ASAPQuery v0.1.0 intercepts queries between Grafana and Prometheus, and accelerates them. Currently, it ingests data using Prometheus' remote_write interface](assets/img/asapquery_intro_figure.jpg) +![ASAPQuery intercepts queries between Grafana and Prometheus, and accelerates them. It ingests data using Prometheus' remote_write interface](assets/img/asapquery_intro_figure.jpg) -ASAPQuery v0.1.0 sits between Prometheus and Grafana. It intercepts queries from Grafana and answers them using streaming sketches, instead of scanning large volumes of raw data in Prometheus. -Currently, it ingests data using Prometheus' remote_write interface. -Future versions of ASAPQuery will accelerate queries against other observability systems (e.g. VictoriaMetrics) and time-series databases (e.g. Clickhouse, Elastic). +ASAPQuery sits between Prometheus and Grafana. It intercepts queries from Grafana and answers them using streaming sketches, instead of scanning large volumes of raw data in Prometheus. +It ingests data using Prometheus' remote_write interface. +ASAPQuery also targets other observability systems (e.g. VictoriaMetrics) and time-series databases (e.g. Clickhouse, Elastic). ## Quick Start @@ -27,6 +27,16 @@ Open http://localhost:3000 and see ASAPQuery vs Prometheus side-by-side! Full quickstart instructions at [**Quickstart Guide**](asap-quickstart/README.md) +**Already have Prometheus and Grafana running?** Use the drop-in instead: + +```bash +cd asap-dropin +# Edit .env to point at your Prometheus, then: +docker compose up -d +``` + +Full drop-in instructions at [**Drop-in Guide**](asap-dropin/README.md) + ## Why ASAPQuery? ### The Problem @@ -47,29 +57,25 @@ ASAPQuery uses **streaming sketches** to: ## Architecture -ASAPQuery has four main components: the **asap-planner-rs** generates sketch configurations from your query workload, **asap-summary-ingest** deploys streaming pipelines in **Arroyo** that continuously build sketches from live Prometheus metrics, and **asap-query-engine** intercepts PromQL queries and serves them from those pre-computed sketches. +ASAPQuery has two main components: **asap-planner-rs** analyzes your PromQL query workload and generates sketch configurations, and **asap-query-engine** intercepts PromQL queries and serves them from pre-computed sketches. The query engine includes a built-in **precompute engine** that continuously builds sketches directly from live metrics via Prometheus `remote_write` — no external streaming infrastructure required. ### Components -- **[asap-planner-rs](asap-planner-rs/)** - Analyzes a PromQL query workload and auto-generates sketch configurations for asap-summary-ingest and asap-query-engine -- **[asap-summary-ingest](asap-summary-ingest/)** - Deploys Arroyo streaming pipelines that continuously compute and publish sketches from live metrics -- **[arroyo](https://github.com/ProjectASAP/arroyo)** - Fork of the [Arroyo](https://github.com/ArroyoSystems/arroyo) stream processing engine that runs the sketch-building SQL pipelines -- **[asap-query-engine](asap-query-engine/)** - Intercepts incoming PromQL queries and serves them from pre-computed sketches, falling back to Prometheus for unsupported queries +- **[asap-planner-rs](asap-planner-rs/)** - Analyzes a PromQL query workload and auto-generates sketch configurations for asap-query-engine +- **[asap-query-engine](asap-query-engine/)** - Intercepts incoming PromQL queries and serves them from pre-computed sketches, falling back to Prometheus for unsupported queries; includes a built-in precompute engine that continuously builds sketches from live metrics via Prometheus `remote_write` ### Repository Structure ``` ├── asap-quickstart/ # Self-contained demo (start here!) +├── asap-dropin/ # Drop-in for existing Prometheus-Grafana stacks ├── asap-planner-rs/ # Auto-configuration service -├── asap-summary-ingest/ # Arroyo pipeline deployer -└── asap-query-engine/ # Query serving engine -# Note: Arroyo fork lives at https://github.com/ProjectASAP/arroyo +└── asap-query-engine/ # Query serving and sketch precomputation engine ``` ## Coming soon -1. Drop-in ASAPQuery artifact that works with your existing pre-configured Prometheus-Grafana stack -2. Drop-in ASAPQuery artifact that accelerates Clickhouse queries +1. Drop-in ASAPQuery artifact that accelerates Clickhouse queries ## Current state diff --git a/asap-quickstart/README.md b/asap-quickstart/README.md index e014241..645f706 100644 --- a/asap-quickstart/README.md +++ b/asap-quickstart/README.md @@ -2,7 +2,7 @@ ## What is ASAPQuery? -ASAPQuery is a drop-in accelerator that reduces query latency by 100x. The current version of ASAPQuery (v0.1.0) sits between Prometheus and Grafana and accelerates repeating PromQL queries. +ASAPQuery is a drop-in accelerator that reduces query latency by 100x. It sits between Prometheus and Grafana and accelerates repeating PromQL queries. ASAPQuery is compatible with the Prometheus query API and integrates seamlessly with existing Grafana dashboards. ## What This Demo Shows @@ -13,9 +13,7 @@ This quickstart simulates a typical monitoring deployment with components you mi - Grafana - send queries to Prometheus and visualizing query results Then it adds ASAPQuery's components on top: -- **Query Engine** - Prometheus-compatible API with sketch-based acceleration -- **[Arroyo](https://github.com/ProjectASAP/arroyo) + asap-summary-ingest** - Streaming engine with pipelines configured for building sketches -- **Kafka** - Message broker for streaming data from Arroyo to the Query Engine +- **Query Engine** - Prometheus-compatible API with sketch-based acceleration and built-in precompute engine - **asap-planner-rs** - Automatically configures sketches from PromQL queries Once you run the quickstart, you will see a pre-configured Grafana dashboard that compares Prometheues and ASAPQuery side-by-side. You will see **visually indistinguishable** results from Prometheus and ASAPQuery, with ASAPQuery being 100x faster @@ -23,7 +21,7 @@ Once you run the quickstart, you will see a pre-configured Grafana dashboard tha ## Prerequisites - **Docker & Docker Compose** v2.0+ -- **Ports available**: 3000 (Grafana), 5115 (Arroyo), 8088 (ASAPQuery), 9090 (Prometheus) +- **Ports available**: 3000 (Grafana), 8088 (ASAPQuery query), 9091 (ASAPQuery ingest), 9090 (Prometheus) ## Quick Start @@ -117,4 +115,4 @@ docker compose restart ### Running ASAPQuery with your own Grafana and Prometheus setup -**Coming Soon**: A drop-in ASAPQuery artifact that works with your existing already-configured Grafana and Prometheus deployments. +See [asap-dropin/](../asap-dropin/README.md) for a single-container setup that integrates with an existing Prometheus and Grafana deployment. diff --git a/asap-quickstart/config/prometheus.yml b/asap-quickstart/config/prometheus.yml index 6ec5ffc..c336930 100644 --- a/asap-quickstart/config/prometheus.yml +++ b/asap-quickstart/config/prometheus.yml @@ -4,9 +4,9 @@ global: scrape_interval: 1s evaluation_interval: 1s -# Remote write configuration to send metrics to Arroyo for sketch building +# Remote write configuration to send metrics to ASAPQuery for sketch building remote_write: - - url: http://arroyo:9091/receive + - url: http://queryengine:9091/receive queue_config: batch_send_deadline: 1s # Drop samples older than 5 minutes before enqueuing — prevents WAL replay diff --git a/asap-quickstart/docker-compose-precompute.dev.yml b/asap-quickstart/docker-compose-precompute.dev.yml deleted file mode 100644 index 9786ea9..0000000 --- a/asap-quickstart/docker-compose-precompute.dev.yml +++ /dev/null @@ -1,12 +0,0 @@ -name: asapquery-quickstart-precompute - -# Development override for precompute variant: builds query engine from local source. -# -# Usage: -# docker compose -f docker-compose-precompute.yml -f docker-compose-precompute.dev.yml up -d --build - -services: - queryengine: - build: - context: .. - dockerfile: asap-query-engine/Dockerfile diff --git a/asap-quickstart/docker-compose-precompute.local.yml b/asap-quickstart/docker-compose-precompute.local.yml deleted file mode 100644 index c2a2320..0000000 --- a/asap-quickstart/docker-compose-precompute.local.yml +++ /dev/null @@ -1,12 +0,0 @@ -name: asapquery-quickstart-precompute - -# Override: builds query engine from pre-built local binary (fast). -# Usage: -# cargo build --release -p query_engine_rust # build locally first -# sudo docker compose -f docker-compose-precompute.yml -f docker-compose-precompute.local.yml up -d --build - -services: - queryengine: - build: - context: .. - dockerfile: asap-quickstart/Dockerfile.queryengine-local diff --git a/asap-quickstart/docker-compose-precompute.yml b/asap-quickstart/docker-compose-precompute.yml deleted file mode 100644 index e0353ad..0000000 --- a/asap-quickstart/docker-compose-precompute.yml +++ /dev/null @@ -1,322 +0,0 @@ -name: asapquery-quickstart-precompute - -# Docker Compose file for pattern-based fake exporters demo -# Uses the hand-crafted precompute engine instead of Arroyo for streaming aggregation -# This eliminates the need for Kafka, Arroyo, and asap-summary-ingest - -networks: - asap-network: - driver: bridge - ipam: - driver: default - config: - - subnet: 172.25.0.0/16 - -volumes: - prometheus-data: - grafana-data: - asap-planner-output: - -services: - ############################################################################# - # INFRASTRUCTURE SERVICES - ############################################################################# - - prometheus: - image: prom/prometheus:v3.9.1 - container_name: asap-prometheus - hostname: prometheus - networks: - - asap-network - ports: - - "9090:9090" - volumes: - - ./config/prometheus-precompute.yml:/etc/prometheus/prometheus.yml:ro - - prometheus-data:/prometheus - command: - - "--config.file=/etc/prometheus/prometheus.yml" - - "--storage.tsdb.path=/prometheus" - - "--web.console.libraries=/usr/share/prometheus/console_libraries" - - "--web.console.templates=/usr/share/prometheus/consoles" - - "--web.enable-lifecycle" - healthcheck: - test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider http://localhost:9090/-/healthy || exit 1"] - interval: 10s - timeout: 5s - retries: 5 - depends_on: - queryengine: - condition: service_healthy - restart: no - - grafana: - image: grafana/grafana-enterprise:12.3.3 - container_name: asap-grafana - hostname: grafana - networks: - - asap-network - ports: - - "3000:3000" - environment: - - GF_SECURITY_ADMIN_PASSWORD=admin - - GF_SECURITY_ADMIN_USER=admin - - GF_USERS_ALLOW_SIGN_UP=false - - GF_SERVER_ROOT_URL=http://localhost:3000 - - GF_SECURITY_ALLOW_EMBEDDING=true - volumes: - - grafana-data:/var/lib/grafana - - ./config/grafana/provisioning:/etc/grafana/provisioning:ro - healthcheck: - test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider http://localhost:3000/api/health || exit 1"] - interval: 10s - timeout: 5s - retries: 5 - restart: no - - ############################################################################# - # INIT CONTAINERS - ############################################################################# - - asap-planner-rs: - image: ghcr.io/projectasap/asap-planner-rs:v0.2.0 - container_name: asap-planner-rs - hostname: asap-planner-rs - networks: - - asap-network - command: - - "--input_config=/config/controller-config.yaml" - - "--output_dir=/asap-planner-output" - - "--prometheus_scrape_interval=1" - - "--streaming_engine=arroyo" - - "--range-duration=300" - - "--step=10" - volumes: - - ./config/controller-config.yaml:/config/controller-config.yaml:ro - - asap-planner-output:/asap-planner-output - restart: "no" - - ############################################################################# - # CORE SERVICES - ############################################################################# - - queryengine: - image: ghcr.io/projectasap/asap-query-engine:v0.2.0-precompute - container_name: asap-queryengine - hostname: queryengine - networks: - - asap-network - ports: - - "8088:8088" - expose: - - "9091" - environment: - - RUST_LOG=INFO - - RUST_BACKTRACE=1 - volumes: - - asap-planner-output:/asap-planner-output:ro - - ./output/queryengine:/app/outputs - command: - - "--config=/asap-planner-output/inference_config.yaml" - - "--streaming-config=/asap-planner-output/streaming_config.yaml" - - "--prometheus-server=http://prometheus:9090" - - "--prometheus-scrape-interval=1" - - "--streaming-engine=precompute" - - "--prometheus-remote-write-port=9091" - - "--delete-existing-db" - - "--log-level=INFO" - - "--output-dir=/app/outputs" - - "--query-language=PROMQL" - - "--lock-strategy=per-key" - - "--forward-unsupported-queries" - healthcheck: - test: ["CMD-SHELL", "bash -c 'echo > /dev/tcp/localhost/8088' 2>/dev/null || exit 1"] - interval: 10s - timeout: 5s - retries: 10 - start_period: 15s - depends_on: - asap-planner-rs: - condition: service_completed_successfully - restart: no - - ############################################################################# - # PATTERN-BASED FAKE EXPORTERS - # Each exporter generates one pattern type with the 'pattern' label - # All series within an exporter follow the same pattern shape with variation - ############################################################################# - - # Constant values - baseline for comparison - fake-exporter-constant: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 - container_name: asap-fake-exporter-constant - hostname: fake-exporter-constant - networks: - - asap-network - expose: - - "50000" - command: - - "--port=50000" - - "--valuescale=1000" - - "--dataset=constant" - - "--num-labels=3" - - "--num-values-per-label=30,30,30" - - "--metric-type=gauge" - - "--metric-name=sensor_reading" - - "--label-names=region,service,host" - - "--label-value-prefixes=region,svc,host" - - "--add-pattern-label" - restart: no - - # Linear increasing - tests trend preservation - fake-exporter-linear-up: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 - container_name: asap-fake-exporter-linear-up - hostname: fake-exporter-linear-up - networks: - - asap-network - expose: - - "50001" - command: - - "--port=50001" - - "--valuescale=1000" - - "--dataset=linear-up" - - "--num-labels=3" - - "--num-values-per-label=30,30,30" - - "--metric-type=gauge" - - "--metric-name=sensor_reading" - - "--label-names=region,service,host" - - "--label-value-prefixes=region,svc,host" - - "--add-pattern-label" - restart: no - - # Linear decreasing - tests trend preservation - fake-exporter-linear-down: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 - container_name: asap-fake-exporter-linear-down - hostname: fake-exporter-linear-down - networks: - - asap-network - expose: - - "50002" - command: - - "--port=50002" - - "--valuescale=1000" - - "--dataset=linear-down" - - "--num-labels=3" - - "--num-values-per-label=30,30,30" - - "--metric-type=gauge" - - "--metric-name=sensor_reading" - - "--label-names=region,service,host" - - "--label-value-prefixes=region,svc,host" - - "--add-pattern-label" - restart: no - - # Sine wave - tests periodicity preservation - fake-exporter-sine: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 - container_name: asap-fake-exporter-sine - hostname: fake-exporter-sine - networks: - - asap-network - expose: - - "50003" - command: - - "--port=50003" - - "--valuescale=1000" - - "--dataset=sine" - - "--num-labels=3" - - "--num-values-per-label=30,30,30" - - "--metric-type=gauge" - - "--metric-name=sensor_reading" - - "--label-names=region,service,host" - - "--label-value-prefixes=region,svc,host" - - "--add-pattern-label" - restart: no - - # Sine with noise - tests signal extraction / smoothing - fake-exporter-sine-noise: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 - container_name: asap-fake-exporter-sine-noise - hostname: fake-exporter-sine-noise - networks: - - asap-network - expose: - - "50004" - command: - - "--port=50004" - - "--valuescale=1000" - - "--dataset=sine-noise" - - "--num-labels=3" - - "--num-values-per-label=30,30,30" - - "--metric-type=gauge" - - "--metric-name=sensor_reading" - - "--label-names=region,service,host" - - "--label-value-prefixes=region,svc,host" - - "--add-pattern-label" - restart: no - - # Step function - tests edge preservation - fake-exporter-step: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 - container_name: asap-fake-exporter-step - hostname: fake-exporter-step - networks: - - asap-network - expose: - - "50005" - command: - - "--port=50005" - - "--valuescale=1000" - - "--dataset=step" - - "--num-labels=3" - - "--num-values-per-label=30,30,30" - - "--metric-type=gauge" - - "--metric-name=sensor_reading" - - "--label-names=region,service,host" - - "--label-value-prefixes=region,svc,host" - - "--add-pattern-label" - restart: no - - # Spiky pattern - tests sudden spikes/drops - fake-exporter-spiky: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 - container_name: asap-fake-exporter-spiky - hostname: fake-exporter-spiky - networks: - - asap-network - expose: - - "50006" - command: - - "--port=50006" - - "--valuescale=1000" - - "--dataset=spiky" - - "--num-labels=3" - - "--num-values-per-label=30,30,30" - - "--metric-type=gauge" - - "--metric-name=sensor_reading" - - "--label-names=region,service,host" - - "--label-value-prefixes=region,svc,host" - - "--add-pattern-label" - restart: no - - # Exponential growth - tests non-linear patterns - fake-exporter-exp-up: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 - container_name: asap-fake-exporter-exp-up - hostname: fake-exporter-exp-up - networks: - - asap-network - expose: - - "50007" - command: - - "--port=50007" - - "--valuescale=1000" - - "--dataset=exp-up" - - "--num-labels=3" - - "--num-values-per-label=30,30,30" - - "--metric-type=gauge" - - "--metric-name=sensor_reading" - - "--label-names=region,service,host" - - "--label-value-prefixes=region,svc,host" - - "--add-pattern-label" - restart: no diff --git a/asap-quickstart/docker-compose.dev.yml b/asap-quickstart/docker-compose.dev.yml index 96b5fc0..10531c8 100644 --- a/asap-quickstart/docker-compose.dev.yml +++ b/asap-quickstart/docker-compose.dev.yml @@ -14,10 +14,6 @@ services: context: .. dockerfile: asap-planner-rs/Dockerfile - asap-summary-ingest: - build: - context: ../asap-summary-ingest - queryengine: build: context: .. diff --git a/asap-quickstart/docker-compose.yml b/asap-quickstart/docker-compose.yml index 9622d72..fd83f94 100644 --- a/asap-quickstart/docker-compose.yml +++ b/asap-quickstart/docker-compose.yml @@ -13,7 +13,6 @@ networks: - subnet: 172.25.0.0/16 volumes: - kafka-data: prometheus-data: grafana-data: asap-planner-output: @@ -23,97 +22,6 @@ services: # INFRASTRUCTURE SERVICES ############################################################################# - kafka: - image: apache/kafka:3.7.0 - container_name: asap-kafka - hostname: kafka - networks: - - asap-network - environment: - KAFKA_NODE_ID: 1 - KAFKA_PROCESS_ROLES: broker,controller - KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092,CONTROLLER://0.0.0.0:9093 - KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092 - KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093 - KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT - KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 - KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1 - KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1 - KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0 - KAFKA_MESSAGE_MAX_BYTES: 20971520 - KAFKA_REPLICA_FETCH_MAX_BYTES: 20971520 - KAFKA_LOG_RETENTION_HOURS: 1 - KAFKA_LOG_DIRS: /tmp/kraft-combined-logs - CLUSTER_ID: MkU3OEVBNTcwNTJENDM2Qk - volumes: - - kafka-data:/tmp/kraft-combined-logs - user: "0:0" - entrypoint: /bin/bash - command: - - -c - - | - chown -R appuser:appuser /tmp/kraft-combined-logs - chmod -R 755 /tmp/kraft-combined-logs - exec su appuser -c "/etc/kafka/docker/run" - healthcheck: - test: ["CMD-SHELL", "/opt/kafka/bin/kafka-broker-api-versions.sh --bootstrap-server localhost:9092 || exit 1"] - interval: 10s - timeout: 5s - retries: 5 - start_period: 30s - restart: no - - kafka-init: - image: apache/kafka:3.7.0 - container_name: asap-kafka-init - networks: - - asap-network - depends_on: - kafka: - condition: service_healthy - entrypoint: /bin/bash - command: - - -c - - | - echo "Creating Kafka topics..." - /opt/kafka/bin/kafka-topics.sh --bootstrap-server kafka:9092 \ - --create --if-not-exists --topic flink_input \ - --partitions 1 --replication-factor 1 \ - --config max.message.bytes=20971520 - - /opt/kafka/bin/kafka-topics.sh --bootstrap-server kafka:9092 \ - --create --if-not-exists --topic flink_output \ - --partitions 1 --replication-factor 1 \ - --config max.message.bytes=20971520 - - echo "Kafka topics created successfully" - restart: "no" - - arroyo: - image: ghcr.io/projectasap/asap-arroyo:v0.1.0 - container_name: asap-arroyo - hostname: arroyo - networks: - - asap-network - ports: - - "5115:5115" - volumes: - - ./config/arroyo-config.yaml:/config.yaml:ro - command: ["--config", "/config.yaml", "cluster"] - environment: - - ARROYO__API__RUN_HTTP_PORT=5115 - - KAFKA_BOOTSTRAP_SERVERS=kafka:9092 - depends_on: - kafka: - condition: service_healthy - healthcheck: - test: ["CMD-SHELL", "curl -f http://localhost:5115/api/v1/pipelines || exit 1"] - interval: 10s - timeout: 5s - retries: 5 - restart: no - prometheus: image: prom/prometheus:v3.9.1 container_name: asap-prometheus @@ -137,9 +45,6 @@ services: timeout: 5s retries: 10 start_period: 3m - depends_on: - asap-summary-ingest: - condition: service_completed_successfully restart: no grafana: @@ -171,7 +76,7 @@ services: ############################################################################# asap-planner-rs: - image: ghcr.io/projectasap/asap-planner-rs:v0.2.0 + image: ghcr.io/projectasap/asap-planner-rs:v0.3.0 container_name: asap-planner-rs hostname: asap-planner-rs networks: @@ -180,7 +85,7 @@ services: - "--input_config=/config/controller-config.yaml" - "--output_dir=/asap-planner-output" - "--prometheus_scrape_interval=1" - - "--streaming_engine=arroyo" + - "--streaming_engine=precompute" - "--range-duration=300" - "--step=10" - "-v" @@ -189,47 +94,19 @@ services: - asap-planner-output:/asap-planner-output restart: "no" - asap-summary-ingest: - image: ghcr.io/projectasap/asap-summary-ingest:v0.2.0 - container_name: asap-summary-ingest - hostname: asap-summary-ingest - networks: - - asap-network - command: - - "--config_file_path=/asap-planner-output/streaming_config.yaml" - - "--source_type=prometheus_remote_write" - - "--prometheus_base_port=9091" - - "--prometheus_path=/receive" - - "--prometheus_bind_ip=0.0.0.0" - - "--parallelism=1" - - "--output_kafka_topic=flink_output" - - "--output_format=json" - - "--pipeline_name=asap-demo" - - "--output_dir=/asap-summary-ingest-output" - - "--arroyo_url=http://arroyo:5115/api/v1" - - "--bootstrap_servers=kafka:9092" - volumes: - - asap-planner-output:/asap-planner-output:ro - - ./output/asap-summary-ingest:/asap-summary-ingest-output - depends_on: - asap-planner-rs: - condition: service_completed_successfully - arroyo: - condition: service_healthy - restart: "no" - ############################################################################# # CORE SERVICES ############################################################################# queryengine: - image: ghcr.io/projectasap/asap-query-engine:v0.2.0 + image: ghcr.io/projectasap/asap-query-engine:v0.3.0 container_name: asap-queryengine hostname: queryengine networks: - asap-network ports: - "8088:8088" + - "9091:9091" environment: - RUST_LOG=INFO - RUST_BACKTRACE=1 @@ -237,27 +114,20 @@ services: - asap-planner-output:/asap-planner-output:ro - ./output/queryengine:/app/outputs command: - - "--kafka-topic=flink_output" - - "--kafka-broker=kafka:9092" - - "--input-format=json" - "--config=/asap-planner-output/inference_config.yaml" - "--streaming-config=/asap-planner-output/streaming_config.yaml" - "--prometheus-server=http://prometheus:9090" - "--prometheus-scrape-interval=1" - - "--streaming-engine=arroyo" + - "--streaming-engine=precompute" + - "--prometheus-remote-write-port=9091" - "--delete-existing-db" - "--log-level=INFO" - "--output-dir=/app/outputs" - "--query-language=PROMQL" - "--lock-strategy=per-key" - - "--decompress-json" - "--forward-unsupported-queries" depends_on: - asap-summary-ingest: - condition: service_completed_successfully - kafka: - condition: service_healthy - kafka-init: + asap-planner-rs: condition: service_completed_successfully prometheus: condition: service_healthy @@ -271,7 +141,7 @@ services: # Constant values - baseline for comparison fake-exporter-constant: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 + image: ghcr.io/projectasap/asap-fake-exporter:v0.3.0 container_name: asap-fake-exporter-constant hostname: fake-exporter-constant networks: @@ -293,7 +163,7 @@ services: # Linear increasing - tests trend preservation fake-exporter-linear-up: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 + image: ghcr.io/projectasap/asap-fake-exporter:v0.3.0 container_name: asap-fake-exporter-linear-up hostname: fake-exporter-linear-up networks: @@ -315,7 +185,7 @@ services: # Linear decreasing - tests trend preservation fake-exporter-linear-down: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 + image: ghcr.io/projectasap/asap-fake-exporter:v0.3.0 container_name: asap-fake-exporter-linear-down hostname: fake-exporter-linear-down networks: @@ -337,7 +207,7 @@ services: # Sine wave - tests periodicity preservation fake-exporter-sine: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 + image: ghcr.io/projectasap/asap-fake-exporter:v0.3.0 container_name: asap-fake-exporter-sine hostname: fake-exporter-sine networks: @@ -359,7 +229,7 @@ services: # Sine with noise - tests signal extraction / smoothing fake-exporter-sine-noise: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 + image: ghcr.io/projectasap/asap-fake-exporter:v0.3.0 container_name: asap-fake-exporter-sine-noise hostname: fake-exporter-sine-noise networks: @@ -381,7 +251,7 @@ services: # Step function - tests edge preservation fake-exporter-step: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 + image: ghcr.io/projectasap/asap-fake-exporter:v0.3.0 container_name: asap-fake-exporter-step hostname: fake-exporter-step networks: @@ -403,7 +273,7 @@ services: # Exponential growth - tests non-linear patterns fake-exporter-exp-up: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 + image: ghcr.io/projectasap/asap-fake-exporter:v0.3.0 container_name: asap-fake-exporter-exp-up hostname: fake-exporter-exp-up networks: