qa-event-performance-lab

Portfolio-grade QA automation lab for testing a small distributed order system.

This repository is designed for QA Automation Engineers who want practical, interview-ready experience with REST API automation, schema validation, Redis cache/rate-limit behavior, RabbitMQ retry/DLQ flows, Kafka-compatible event processing with Redpanda, and performance testing with k6, JMeter, and Locust.

What This Repo Demonstrates

• Java 21 Spring Boot backend with layered controller/service/repository design.
• PostgreSQL persistence with Flyway migrations and deterministic product seed data.
• REST API validation, consistent error responses, and idempotent order creation.
• Redis order-status cache with TTL, cache-miss fallback, and fixed-window rate limiting.
• RabbitMQ invoice generation using exchange, queue, routing key, retry, and DLQ.
• Redpanda/Kafka ORDER_CREATED event flow with an idempotent notification consumer.
• Opt-in black-box RestAssured API tests with Awaitility for async behavior.
• JSON schema validation for event payloads.
• k6, JMeter, and Locust performance assets that exercise the real order flow.
• Offline-safe Maven build and a simple GitHub Actions CI workflow.

Architecture Overview

Client / API tests / performance tools
  |
  v
order-api Spring Boot service
  |
  +-- REST controllers
  +-- validation and global error handling
  +-- product/order/idempotency services
  +-- Redis cache and rate-limit services
  +-- RabbitMQ invoice publisher and in-process listener
  +-- Kafka ORDER_CREATED publisher and in-process notification listener
  |
  +--> PostgreSQL
  +--> Redis
  +--> RabbitMQ invoice.generate.q -> invoice.generate.dlq
  +--> Redpanda topic orders.events

The invoice and notification workers run in-process to keep the lab manageable. The messaging boundaries are still real: RabbitMQ and Redpanda run as external Docker services, and the consumers are written so they could later be moved into separate services.

Tech Stack

Area	Tools
Backend	Java 21, Spring Boot, Maven
API testing	JUnit 5, RestAssured, AssertJ, Awaitility
Persistence	PostgreSQL, Flyway, Spring Data JPA
Cache/rate limit	Redis, Spring Data Redis
Queue	RabbitMQ with management UI, Spring AMQP
Streaming	Redpanda as Kafka-compatible broker, Spring Kafka
Performance	k6, JMeter, Locust
CI	GitHub Actions, Maven verify

Prerequisites

• Java 21
• Maven 3.9+
• Docker Desktop or compatible Docker Compose
• Optional for performance tests: k6, JMeter, Python with Locust

Run The Root Build

The root build is intentionally offline-safe. It compiles the app and tests, but does not require Docker, PostgreSQL, Redis, RabbitMQ, Redpanda, or a running API.

mvn clean verify

Run Locally

Start the full local stack:

docker compose up --build

Or use the helper script:

.\scripts\start-local.ps1

Verify health:

Invoke-RestMethod http://localhost:8080/api/health
Invoke-RestMethod http://localhost:8080/actuator/health

If port 8080 is busy:

$env:ORDER_API_PORT="18080"
docker compose up --build

Run With Testtools

The reset and internal inspection endpoints are disabled by default. Enable them only for local black-box testing:

$env:SPRING_PROFILES_ACTIVE="docker,testtools"
docker compose up --build

Or:

.\scripts\start-testtools.ps1

Run API Tests

Full black-box API tests are opt-in because they require a live Dockerized stack with testtools enabled.

mvn -pl tests/api-tests test -DrunApiTests=true "-Dapi.base-url=http://localhost:8080"

Or:

.\scripts\run-api-tests.ps1

You can also use BASE_URL:

$env:BASE_URL="http://localhost:18080"
.\scripts\run-api-tests.ps1

Run Performance Tests

Performance tests create real orders, invoices, notifications, Redis keys, RabbitMQ activity, and Kafka events. For clean local results, recreate Docker volumes:

docker compose down -v
$env:SPRING_PROFILES_ACTIVE="docker,performance"
docker compose up --build

Or:

.\scripts\start-performance.ps1

k6:

$env:BASE_URL="http://localhost:8080"
k6 run performance/k6/create-order-smoke.js
k6 run performance/k6/create-order-load.js
k6 run performance/k6/create-order-spike.js
k6 run performance/k6/create-order-soak.js

PowerShell wrappers:

.\scripts\run-k6-smoke.ps1
.\scripts\run-k6-load.ps1

JMeter:

jmeter -n -t performance/jmeter/create-order-load-test.jmx -l performance-results.jtl

Locust:

locust -f performance/locust/locustfile.py --host http://localhost:8080
locust -f performance/locust/locustfile.py --host http://localhost:8080 --headless -u 20 -r 5 -t 2m

Performance tools are not part of the Maven lifecycle.

Ports

Service	Default URL/Port	Notes
order-api	http://localhost:8080	Override with ORDER_API_PORT
PostgreSQL	localhost:5432	Override with POSTGRES_PORT
Redis	localhost:6379	Override with REDIS_PORT
RabbitMQ AMQP	localhost:5672	Override with RABBITMQ_PORT
RabbitMQ UI	http://localhost:15672	qa_lab_user / qa_lab_password
Redpanda Kafka API	localhost:9092	Override with REDPANDA_KAFKA_PORT
Redpanda Admin	localhost:9644	Override with REDPANDA_ADMIN_PORT
Redpanda Console	http://localhost:8082	Override with REDPANDA_CONSOLE_PORT

API Endpoints

Method	Endpoint	Purpose
GET	/api/health	Lab health response
GET	/actuator/health	Spring Actuator health
GET	/api/products/{productId}	Product lookup
POST	/api/orders	Create order with validation and idempotency
GET	/api/orders/{orderId}	Fetch order
GET	/api/orders/{orderId}/status	Fetch order status from Redis or PostgreSQL fallback
GET	/api/invoices/order/{orderId}	Fetch eventually created invoice
GET	/api/notifications/order/{orderId}	Fetch eventually created notification

Testtools-only endpoints:

Method	Endpoint	Purpose
POST	/api/test/reset	Reset lab data while keeping products
GET	/api/internal/cache/orders/{orderId}	Inspect Redis order-status cache
DELETE	/api/internal/cache/orders/{orderId}	Delete one Redis order-status cache key
GET	/api/internal/rate-limit/{customerId}	Inspect Redis rate-limit key
GET	/api/internal/rabbitmq/invoice-dlq/count	Inspect invoice DLQ count
POST	/api/internal/rabbitmq/publish-invalid-invoice-job	Publish invalid invoice job
POST	/api/internal/rabbitmq/publish-duplicate-invoice-job	Publish duplicate invoice job
GET	/api/internal/invoices/order/{orderId}/count	Count invoices for one order
GET	/api/internal/notifications/order/{orderId}/count	Count notifications for one order
GET	/api/internal/kafka/processed-events/{eventId}	Inspect processed Kafka event ledger
POST	/api/internal/kafka/publish-duplicate-order-created-event	Publish duplicate order-created events
POST	/api/internal/kafka/publish-invalid-order-created-event	Publish missing-order event

Example Order Request

{
  "customerId": "CUST-1001",
  "productId": "PRD-1001",
  "quantity": 2,
  "paymentMethod": "CARD",
  "idempotencyKey": "idem-12345"
}

Expected new-order behavior:

• Returns 201.
• Stores order in PostgreSQL.
• Writes order:status:{orderId} to Redis with TTL.
• Publishes an invoice job to RabbitMQ after commit.
• Publishes ORDER_CREATED to Redpanda after commit.
• Creates invoice and notification asynchronously.

Idempotency replay with the same normalized payload returns the same order with 200 and does not republish messages/events. A different payload with the same key returns 409.

Testing Strategy Summary

• mvn clean verify is safe for local development and CI because external black-box tests are skipped by default.
• RestAssured tests run only with -DrunApiTests=true.
• Awaitility waits for async invoice and notification outcomes without hard sleeps.
• Testtools endpoints provide deterministic reset and diagnostics only when qa-lab.testtools.enabled=true.
• k6, JMeter, and Locust are manual performance tools and are not executed by Maven or default CI.

GitHub Actions

The first CI workflow is intentionally conservative:

checkout -> setup Java 21 with Maven cache -> mvn clean verify

It does not start Docker Compose or run performance tests by default. Dockerized API smoke tests can be added later as a separate job.

Known Limitations

• No transactional outbox. If an order commits and broker publishing fails after commit, invoice jobs or order events can be lost.
• Invoice and notification workers are in-process for lab simplicity.
• Kafka topics are append-only in this lab; reset clears database state but does not purge orders.events.
• No production authentication, authorization, TLS, secrets management, or tenant isolation.
• Performance results are local-machine dependent and should be treated as baselines, not production capacity.
• No advanced observability stack yet.

Future Improvements

• Testcontainers-based integration tests.
• Allure reporting.
• Dockerized CI integration-test job.
• Optional k6 smoke job in CI.
• Prometheus and Grafana dashboards.
• Split invoice-worker and notification-worker into separate services.
• Transactional outbox pattern and relay.
• Schema Registry with Avro or Protobuf.
• Broader contract testing for REST responses and message payloads.

Interview-Ready Explanation

This project demonstrates how I test a distributed order workflow end to end. I start with deterministic REST API behavior, then verify side effects through PostgreSQL, Redis, RabbitMQ, and Redpanda using black-box tests and controlled diagnostics. Async assertions use bounded polling instead of sleeps, idempotency is tested at both API and consumer levels, and performance scripts exercise the real system path rather than isolated mock endpoints.