diff --git a/content/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes.md b/content/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes.md new file mode 100644 index 00000000..5ae9bbe3 --- /dev/null +++ b/content/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes.md @@ -0,0 +1,291 @@ +--- +title: "LLM Costs and Observability with agentgateway on Kubernetes (Part 2)" +seoTitle: "agentgateway Observability: Token Cost, Metrics, and Grafana on Kubernetes" +seoDescription: "Part 2: scrape agentgateway with Prometheus, build a Grafana dashboard of token cost and per-tool usage, see blocked tool calls, and alert on spend." +datePublished: 2026-06-30T10:00:00.000Z +slug: llm-costs-and-observability-with-agentgateway-on-kubernetes +author: shubham-katara +cover: /img/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes/cover.png +tags: ["kubernetes", "ai-agents", "observability", "prometheus", "grafana"] +draft: true +--- + +This is Part 2 of a two-part series. In [Part 1](https://blog.kubesimplify.com/controlling-mcp-tools-with-agentgateway-on-kubernetes) you put a Google ADK agent behind **agentgateway** on Kubernetes: the agent holds zero secrets, its model and tool calls flow through one proxy, and a policy blocks any tool you have not allowed. That is the governance half. This part is the question governance cannot answer on its own: what is all of this actually costing you, and can you see when an agent misbehaves? + +By the end you will have Prometheus and Grafana on the same `kind` cluster, a dashboard that shows token throughput and an estimated dollar figure as your agents run, and blocked tool-call attempts visible on the same screen. + +All the manifests and the dashboard JSON are in the companion repo: https://github.com/shkatara/agentgateway-security-observability + +What you'll build in Part 2: + +- Prometheus and Grafana running in the cluster via `kube-prometheus-stack`. +- A `PodMonitor` that scrapes the gateway proxy's metrics. +- A Grafana dashboard with token cost, per-tool call counts, and HTTP status, imported from a single file. +- A view of blocked tool-call attempts, and an explanation of why the `405`s you will see are not blocked tools. +- A `PrometheusRule` that alerts when estimated daily spend crosses a budget. + +> **Note:** agentgateway is a fast-moving project. These commands target the `v1.2.x` charts and the `agentgateway.dev/v1alpha1` API. Pin versions and expect metric names and fields to evolve. + +## Prerequisites + +You need the setup from Part 1 up and running: + +- The `kind` cluster with the agentgateway control plane and proxy. +- The LLM route (`/v1`) and the MCP route (`/mcp-github`), with the ADK agent working. +- Ideally the `AgentgatewayPolicy` from Part 1 applied (allow only `get_me`), so blocked attempts show up on the dashboard. +- `helm` v3.8+ and the port-forward on `localhost:8080` still available. + +## Why agents need observability, specifically + +Two of the three questions from Part 1 are observability questions. Who spent the money, and on which model? When an agent goes off the rails, where is the trace? A normal service dashboard does not answer these, because it does not understand tokens or tool calls. agentgateway does. It counts tokens and MCP calls for every request and exposes them as Prometheus metrics, so the answers become PromQL queries instead of a shrug. + +## The quick look: the metrics endpoint + +Before installing anything, confirm the gateway is already emitting what we need. Port-forward the proxy's metrics port and read it: + +```sh +kubectl port-forward deployment/agentgateway-proxy -n agentgateway-system 15020 & +curl -s localhost:15020/metrics | grep gen_ai_client_token_usage +``` + +You will see the token-usage histogram that powers all cost tracking: + +```console +agentgateway_gen_ai_client_token_usage_sum{gen_ai_operation_name="chat",gen_ai_request_model="gpt-4o-mini",gen_ai_system="openai",gen_ai_token_type="input"} 342 +agentgateway_gen_ai_client_token_usage_count{gen_ai_operation_name="chat",gen_ai_request_model="gpt-4o-mini",gen_ai_system="openai",gen_ai_token_type="input"} 5 +``` + +The labels (`gen_ai_request_model`, `gen_ai_system`, `gen_ai_token_type`) are what let you slice spend by model, provider, and direction. The metrics endpoint is fine for a peek. The payoff is a dashboard, so let us wire one up. + +## The observability architecture + +Everything stays inside the same cluster. The proxy exposes metrics on port `15020`. We add Prometheus to scrape it and Grafana to draw it. + +![AgentGateway Observability Architecture](/img/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes/agw-monitoring.png) + +## Step 1: Install Prometheus and Grafana + +The `kube-prometheus-stack` chart bundles Prometheus, Grafana, the Prometheus Operator, and a Grafana sidecar that auto-loads dashboards from labeled ConfigMaps. The three `NilUsesHelmValues=false` flags matter. Without them, the operator only picks up `PodMonitor` and `PrometheusRule` resources that carry its own release label, and the ones we create next would be silently ignored. + +```sh +helm upgrade --install kube-prometheus-stack kube-prometheus-stack \ + --repo https://prometheus-community.github.io/helm-charts \ + --namespace telemetry --create-namespace \ + --set prometheus.prometheusSpec.podMonitorSelectorNilUsesHelmValues=false \ + --set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false \ + --set prometheus.prometheusSpec.ruleSelectorNilUsesHelmValues=false \ + --wait +``` + +It pulls a few images, so give it a minute. Confirm the stack is up: + +```sh +kubectl get pods -n telemetry +``` + +## Step 2: Make Prometheus scrape the gateway + +First, find the label that uniquely identifies the proxy pod. This one step saves the most debugging time. + +```sh +kubectl get pods -n agentgateway-system --show-labels | grep proxy +``` + +agentgateway's own scrape config keys on `kgateway=kube-gateway` for data plane proxies, which is what the `PodMonitor` below uses. If your output shows a different label (for example `app.kubernetes.io/name=agentgateway-proxy`), use that in the `selector`. + +```sh +kubectl apply -f- < /dev/null + echo "request $i sent" + sleep 2 +done +``` + +You can also just chat with the ADK agent a few more times. Either way, every call increments the token metric. + +## Step 4: Confirm the metric in Prometheus + +Before touching Grafana, confirm the data landed. With the port-forward on `9090`, open `http://localhost:9090/graph` and run: + +```promql +agentgateway_gen_ai_client_token_usage_sum +``` + +You should see series with labels like `gen_ai_request_model="gpt-4o-mini"` and `gen_ai_token_type="input"`. If they are here, Grafana is just drawing what Prometheus already has. + +## Step 5: Import the cost dashboard + +The dashboard ships as [`agentgateway-cost-dashboard.json`](https://github.com/shkatara/agentgateway-security-observability/blob/main/agentgateway-cost-dashboard.json) in the repo. The Grafana sidecar auto-imports any ConfigMap in the `telemetry` namespace labeled `grafana_dashboard=1`, so from the repo root: + +```sh +kubectl -n telemetry create configmap agentgateway-cost-dashboard \ + --from-file=agentgateway-cost-dashboard.json +kubectl -n telemetry label configmap agentgateway-cost-dashboard grafana_dashboard=1 +``` + +Port-forward Grafana and log in: + +```sh +kubectl port-forward -n telemetry deployment/kube-prometheus-stack-grafana 3000 & +# username: admin +kubectl -n telemetry get secret kube-prometheus-stack-grafana -o jsonpath='{.data.admin-password}' | base64 -d +``` + +Open the **agentgateway: Agent LLM Cost and Usage** dashboard. With the traffic loop running, the panels fill in within a scrape interval or two. + +![agentgateway cost and tool-usage dashboard in Grafana](/img/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes/cost-dashboard.png) + +Top row: MCP tool calls over time, and totals per tool. Middle: MCP transport HTTP by method and status, where the `GET 405/406` lines are normal transport negotiation, not blocked tools (more on that below). Bottom: blocked tool-call attempts, everything outside the allow-list. + +## The panels and the PromQL behind them + +If you would rather build the dashboard by hand, or you just want to understand what each panel asks Prometheus, here is every query. + +| Panel | What it shows | PromQL | +| --- | --- | --- | +| Token throughput (input vs output) | Tokens per second, split by direction | `sum by (gen_ai_token_type) (rate(agentgateway_gen_ai_client_token_usage_sum[5m]))` | +| Token throughput by model | Which models are burning tokens | `sum by (gen_ai_request_model) (rate(agentgateway_gen_ai_client_token_usage_sum[5m]))` | +| Estimated spend (last 1h) | A live dollar figure | `(sum(increase(agentgateway_gen_ai_client_token_usage_sum{gen_ai_token_type="input"}[1h])) / 1000000) * 0.15 + (sum(increase(agentgateway_gen_ai_client_token_usage_sum{gen_ai_token_type="output"}[1h])) / 1000000) * 0.60` | +| LLM requests per second by status | HTTP status of LLM-route traffic only | `sum by (status) (rate(agentgateway_requests_total{route="agentgateway-system/openai"}[5m]))` | +| MCP transport requests (by method and status) | Raw HTTP on the MCP route | `sum by (method, status) (rate(agentgateway_requests_total{route="agentgateway-system/mcp-github"}[5m]))` | +| MCP tool calls over time (by tool) | Which tools are being called | `sum by (resource) (rate(agentgateway_mcp_requests_total{method="tools/call"}[5m]))` | +| Total MCP tool calls (by tool) | How many times each tool was called | `sum by (resource) (agentgateway_mcp_requests_total{method="tools/call"})` | +| Blocked tool-call attempts (total) | Calls to tools outside the allow-list | `sum(agentgateway_mcp_requests_total{method="tools/call", resource!="get_me"})` | +| Blocked tool-call attempts over time (by tool) | When and which blocked tools were attempted | `sum by (resource) (rate(agentgateway_mcp_requests_total{method="tools/call", resource!="get_me"}[5m]))` | + +A note on the MCP metric. Tool activity is `agentgateway_mcp_requests_total`, with a `method` label (`tools/list`, `tools/call`, `initialize`) and, for tool calls, a `resource` label holding the tool name. So `...{method="tools/call", resource="get_me"}` is the per-tool call count. The gateway counts the attempt even when a tool is blocked, so a tool you have hidden by policy still shows a count here. The metric records that a call was attempted, not whether it was allowed, so the proxy logs are where you confirm the verdict. If names differ on your build, run `curl -s localhost:15020/metrics | grep mcp`. + +## Turning tokens into dollars + +There is no magic in the spend panel. It multiplies token counts by your provider's price: + +``` +cost = (input_tokens / 1,000,000 × input_price) + (output_tokens / 1,000,000 × output_price) +``` + +The dashboard keeps the two prices in hidden variables (`price_in`, `price_out`), defaulting to example values for a small model. Prices change and vary by model, so treat them as placeholders and set your real rates in the dashboard's Variables settings. The `increase(...[1h])` window means the stat reads as spend over the last hour. + +## See the blocked tool call on the dashboard + +This is where the two halves of the series meet. When you applied the `AgentgatewayPolicy` that allows only `get_me`, every attempt to call another tool is rejected. With the dashboard open, run a handful of blocked calls: + +```sh +for i in $(seq 1 5); do + npx @modelcontextprotocol/inspector@0.21.2 \ + --cli http://localhost:8080/mcp-github \ + --transport http \ + --method tools/call \ + --tool-name search_repositories \ + --tool-arg query="kubernetes" + sleep 1 +done +``` + +Each one comes back as `Unknown tool`, because the policy hides `search_repositories`. Even so, the gateway records the attempt, so `search_repositories` shows up on the per-tool panels with its own count, and on the "blocked tool-call attempts" panels because it is not in the allow-list. A blocked tool with a rising count is a useful signal in itself: something is trying to reach a tool it is not allowed to. + +## Why you see 405s, and why they are not blocked tools + +Look at the MCP transport panel and you will see a lot of `GET 405` and `GET 406`. It is tempting to read those as blocked tools. They are not. + +A blocked tool call returns HTTP `200` with a JSON-RPC error in the body (`Unknown tool`). + +MCP errors are application-level, not HTTP status codes, so blocking never shows up as a `4xx`. The `405` and `406` responses are MCP Streamable HTTP transport: the client issues a `GET` to open an SSE stream, or a `DELETE` to end a session, and that method is not allowed on the endpoint. It is normal protocol negotiation. + +That is exactly why the dashboard scopes the LLM status panel to the OpenAI route and gives the MCP route its own transport panel. Otherwise the MCP transport noise leaks into your LLM error rate and you chase a problem that is not there. + +## Alert on spend + +You do not want to watch a dashboard all day. A `PrometheusRule` fires when estimated daily spend crosses a threshold. The `release: kube-prometheus-stack` label is what makes the operator pick the rule up. + +```sh +kubectl apply -f- < 50 + for: 5m + labels: + severity: warning + annotations: + summary: "Estimated LLM spend over the last 24h exceeded 50 USD" +EOF +``` + +Adjust the prices and the `> 50` threshold to your budget. The alert appears in Prometheus under **Alerts**, and routes through AlertManager if you have wired up a receiver. + +## Common Issues and How to Solve Them + +| Symptom | Likely cause | Fix | +| --- | --- | --- | +| Proxy target missing or `DOWN` in Prometheus | The `PodMonitor` selector does not match the proxy pod, or the operator ignores the monitor | Run `kubectl get pods -n agentgateway-system --show-labels` and fix the `selector`; confirm the `podMonitorSelectorNilUsesHelmValues=false` flag | +| Target `UP` but no `agentgateway_gen_ai_*` series | No LLM traffic yet, or the wrong port | Run the traffic loop and confirm metrics on `:15020/metrics` | +| Grafana panels say "datasource not found" | The dashboard datasource variable did not auto-select | Pick Prometheus from the datasource dropdown at the top of the dashboard | +| Dashboard never appears in Grafana | The ConfigMap is unlabeled or in the wrong namespace | It must be in the `telemetry` namespace and labeled `grafana_dashboard=1` | +| Estimated spend shows 0 | `increase()` needs at least two samples in the window | Send traffic and wait a couple of scrape intervals | +| MCP panels are empty | The tool-call metric is named differently on your build | Run `curl -s localhost:15020/metrics | grep -E 'mcp|tool'` and adjust the query | +| A lot of `GET 405/406` on the MCP route | Normal MCP Streamable HTTP transport negotiation, not blocked tools | Nothing to fix; blocked tools are HTTP `200` with `Unknown tool` in the body | + +## Conclusion + +That completes the series. In Part 1 you made agents safe to run: no secrets in the runtime, and tools locked down by policy. In Part 2 you made them legible: token cost per model on a live graph, per-tool usage, blocked attempts you can watch, and an alert when spend runs away. + +Here's what you accomplished in Part 2: + +1. Installed Prometheus and Grafana on the same cluster as the gateway. +2. Scraped the proxy with a `PodMonitor` and verified the target. +3. Imported a cost dashboard showing token throughput, per-model usage, per-tool calls, and an estimated dollar figure. +4. Made blocked tool-call attempts visible, and learned why the `405`s are transport noise, not policy denials. +5. Added an alert that fires when estimated daily spend crosses a budget. + +Put together, the two parts answer the questions you could not answer before: who spent what, who can call which tool, and where the trace is. That single control point is the difference between a pile of agents and a platform. + +All the manifests, the agent, and the dashboard are in the repo: https://github.com/shkatara/agentgateway-security-observability. The agentgateway project lives at [agentgateway.dev](https://agentgateway.dev/). diff --git a/lib/_blog-feed-data.js b/lib/_blog-feed-data.js index aab16acd..455fd969 100644 --- a/lib/_blog-feed-data.js +++ b/lib/_blog-feed-data.js @@ -1,5 +1,17 @@ // AUTO-GENERATED by scripts/generate-feeds.mjs. Do not edit by hand. export const FEED_POSTS = [ + { + "slug": "llm-costs-and-observability-with-agentgateway-on-kubernetes", + "title": "LLM Costs and Observability with agentgateway on Kubernetes (Part 2)", + "description": "Part 2: scrape agentgateway with Prometheus, build a Grafana dashboard of token cost and per-tool usage, see blocked tool calls, and alert on spend.", + "datePublished": "2026-06-30T10:00:00.000Z", + "cover": "/img/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes/cover.png", + "tags": [ + "kubernetes", + "ai-agents", + "observability" + ] + }, { "slug": "controlling-mcp-tools-with-agentgateway-on-kubernetes", "title": "Controlling MCP Tools with agentgateway on Kubernetes (Part 1)", @@ -339,17 +351,5 @@ export const FEED_POSTS = [ "new", "newrelease" ] - }, - { - "slug": "ditch-the-overheating-laptop-supercharge-your-docker-workflow-with-docker-offload", - "title": "Ditch the Overheating Laptop: Supercharge Your Docker Workflow with Docker Offload", - "description": "Running multiple Docker containers can slow down your laptop and drain your battery. In this blog, we explore Docker Offload — a game-changing feature", - "datePublished": "2025-08-26T06:40:46.134Z", - "cover": "/img/blog/ditch-the-overheating-laptop-supercharge-your-docker-workflow-with-docker-offload/365cc324-2eb9-4656-9515-fc69b74abb3e.png", - "tags": [ - "ai", - "cloud", - "docker" - ] } ]; diff --git a/lib/series.js b/lib/series.js index 44422afc..0a705845 100644 --- a/lib/series.js +++ b/lib/series.js @@ -45,6 +45,20 @@ export const SERIES = [ 'day-4-quantization-demystified-bf16-fp8-nvfp4-mxfp4-int4-gguf-and-why-it-all-matters', ], }, + { + slug: 'agentgateway-on-kubernetes', + name: 'agentgateway on Kubernetes', + tagline: 'Put your AI agents behind a gateway: lock down their tools, then see what they cost.', + description: + "A two-part, hands-on series on running AI agents safely on Kubernetes with agentgateway. Part 1 puts a Google ADK agent behind the gateway so it holds no secrets and can only call the tools you allow. Part 2 adds Prometheus and Grafana for token-cost and per-tool observability.", + author: 'shubham-katara', + color: '#C15F3C', + cover: null, + posts: [ + 'controlling-mcp-tools-with-agentgateway-on-kubernetes', + 'llm-costs-and-observability-with-agentgateway-on-kubernetes', + ], + }, ]; export function getAllSeries() { diff --git a/public/_redirects b/public/_redirects index 95637037..bed01dc5 100644 --- a/public/_redirects +++ b/public/_redirects @@ -142,6 +142,7 @@ /blog/lets-talk-about-ansible /lets-talk-about-ansible 301! /blog/linux-boot-process-simplified /linux-boot-process-simplified 301! /blog/linux-system-directories-explained /linux-system-directories-explained 301! +/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes /llm-costs-and-observability-with-agentgateway-on-kubernetes 301! /blog/managing-contexts-in-kubernetes-with-plugins /managing-contexts-in-kubernetes-with-plugins 301! /blog/managing-your-operating-system-with-package-managers /managing-your-operating-system-with-package-managers 301! /blog/mastering-kubernetes-costs-from-monitoring-to-automation /mastering-kubernetes-costs-from-monitoring-to-automation 301! @@ -349,6 +350,7 @@ /lets-talk-about-ansible /blog/lets-talk-about-ansible 200! /linux-boot-process-simplified /blog/linux-boot-process-simplified 200! /linux-system-directories-explained /blog/linux-system-directories-explained 200! +/llm-costs-and-observability-with-agentgateway-on-kubernetes /blog/llm-costs-and-observability-with-agentgateway-on-kubernetes 200! /managing-contexts-in-kubernetes-with-plugins /blog/managing-contexts-in-kubernetes-with-plugins 200! /managing-your-operating-system-with-package-managers /blog/managing-your-operating-system-with-package-managers 200! /mastering-kubernetes-costs-from-monitoring-to-automation /blog/mastering-kubernetes-costs-from-monitoring-to-automation 200! diff --git a/public/_worker.js b/public/_worker.js index aece69ef..38503d2c 100644 --- a/public/_worker.js +++ b/public/_worker.js @@ -3,7 +3,7 @@ // Replaces _redirects (Pages picks _worker.js over _redirects when both exist). const KUBESIMPLIFY_ROUTES = new Set(['/about', '/workshops', '/partnerships', '/resources']); -const BLOG_SLUGS = new Set(["10-things-you-might-not-know-about-k9s","12-practical-grep-command-examples-in-linux","a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-1","a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2","a-complete-walk-through-of-devops","a-kubeconfig-for-gke-that-doesnt-need-gcloud","a-simple-way-to-structure-your-terraform-code","a-simplified-guide-to-yaml","about-my-pdf-editor-project","an-overview-of-gitops-and-argocd","announcing-buildsafe","api-response-in-go","arkade","automate-repetitive-tasks-shell-scripting","automated-github-releases-with-github-actions-and-conventional-commits","avoid-overspending-with-kubecost","aws-elastic-cloud-compute","bake-your-container-images-with-bake","become-a-hashicorp-certified-terraform-associate-preparation-guide","best-devops-tools-2025","breaking-down-docker","building-a-zero-cve-strategy","building-apigateway-with-lambda-using-pulumi","certified-kubernetes-security-specialist-cks-2022-exam-guide","cicd-pipeline-github-actions-with-aws-ecs","ckad-exam-april-2022","claude-code-leak-what-the-source-actually-teaches","clawspark-your-private-openclaw-ai-assistant-that-never-phones-home","cloud-computing","cloud-native-buildpacks-concepts","confidential-containers-running-on-kubernetes","container-and-kubernetes-security","controlling-mcp-tools-with-agentgateway-on-kubernetes","coolify","creating-multi-node-kubernetes-cluster-locally","day-1-the-local-llm-revolution-why-your-desk-just-became-the-new-datacenter","day-1-what-actually-happens-when-you-type-docker-run","day-2-anatomy-of-an-llm-inference-request-from-prompt-to-answer-step-by-step","day-2-your-images-are-a-supply-chain-and-it-s-probably-broken","day-3-stop-writing-dockerfiles-from-scratch","day-3-the-dgx-spark-unpacked-gb10-unified-memory-sm-121-and-the-one-reason-this-hardware-exists","day-4-breaking-isolation-on-purpose-volumes-networks-and-the-real-world","day-4-quantization-demystified-bf16-fp8-nvfp4-mxfp4-int4-gguf-and-why-it-all-matters","day-5-docker-compose-how-docker-actually-gets-used","day-6-run-an-llm-on-your-laptop-with-docker","day-7-ship-it-and-what-comes-next","deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws","deploy-a-simple-server-using-aws-terraform","deploying-java-application-using-docker-and-kubernetes-devops-project","ditch-the-overheating-laptop-supercharge-your-docker-workflow-with-docker-offload","diy-how-to-build-a-kubernetes-policy-engine","docker-captain-journey","docker-mcp-catalog","docker-networking-demystified","embed-http-servers-in-wasm-with-rust-and-csharp","enhancing-runtime-security-with-falco-my-hands-on-experience","ephemeral-pull-request-environment-using-vcluster","essential-linux-commands-for-devops","event-driven-architecture-simplified-monolith-to-microservices","everything-you-need-to-know-about-docker-compose","everything-you-need-to-know-about-the-linux-ls-command","exploiting-metasploitable2-using-msfconsole-kali-linux-lab","firewall-a-networks-gatekeeper","four-pillars-of-observability-in-kubernetes","get-good-at-git","getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster","getting-started-with-ko-a-fast-container-image-builder-for-your-go-applications","getting-started-with-kyverno","git-and-github-a-beginners-guide","github-actions-101-what-are-github-actions-and-how-to-use-them-a-beginners-guide","gitops-demystified","ha-kubernetes","how-a-kubernetes-service-actually-works-and-all-5-types-you-need","how-get-started-with-hashicorp-vault","how-kubernetes-endpointslices-actually-work-and-why-endpoints-had-to-die","how-to-backup-kubernetes-with-kasten-community-edition","how-to-change-directory-in-shell-scripts","how-to-install-a-kubernetes-cluster-with-kubeadm-containerd-and-cilium-a-hands-on-guide","how-to-setup-your-ftp-server-in-linux","implementing-kubernetes-network-policies-a-comprehensive-guide","important-concepts-of-operating-systems","ing-switch-119-annotations-gateway-api-traefik-impact-ratings","ing-switch-migrate-from-ingress-nginx-to-traefik-or-gateway-api-in-minutes-not-days","installing-prometheus-with-selinux","introducing-unikraft-lightweight-virtualization-using-unikernels","introduction-of-jenkins-pipeline","introduction-to-cicd-and-cicd-pipeline","introduction-to-cri","introduction-to-developer-platforms-with-gimlet","introduction-to-helm","introduction-to-jenkins","introduction-to-kubernetes","introduction-to-terraform","iptables-demo","istio-service-mesh","k8sgpt-tutorial-when-kubernetes-meets-ai","keptn-getting-started","ksctl-making-kubernetes-easy-across-clouds","kube-proxy-deep-dive","kube-scheduler-deep-dive","kubecon-cloudnativecon-north-america-2024-recap-themes-innovations-and-community-spirit","kubecon-cloudnativecon-rejekts-and-wasm-io-wrap-up-a-leap-into-the-future-with-webassembly-ai-and-sustainable-cloud-practices","kubectl-run-nginx-inside","kubeflow-machine-learning-on-kubernetes-part-1","kubeflow-notebooks-ml-experimentation-made-easier-part-2","kubeflow-pipelines-orchestrating-machine-learning-workflows-part-3","kubernetes-125-dockerd","kubernetes-126","kubernetes-access-control-with-authentication-authorization-admission-control","kubernetes-adoption-key-challenges-in-migrating-to-kubernetes","kubernetes-backup-using-cloudcasa","kubernetes-containerd-setup","kubernetes-crio","kubernetes-management-with-rust-a-dive-into-generic-client-go-controller-abstractions-and-crd-macros-with-kubers","kubernetes-on-apple-macbooks-m-series","kubernetes-scheduling-the-complete-guide","kubernetes-v133-key-features-updates-and-what-you-need-to-know","kubernetes-v135-whats-new-whats-changing-and-what-you-should-know","kubesimplify-a-journey-to-remember","kubesimplify-at-wasmio-and-kubecon-eu-2024","kyverno-and-cosign","kyverno-cli","lets-learn-terraform","lets-simplify-golang-part-1","lets-simplify-golang-part-2","lets-simplify-golang-part-3","lets-talk-about-ansible","linux-boot-process-simplified","linux-system-directories-explained","managing-contexts-in-kubernetes-with-plugins","managing-your-operating-system-with-package-managers","mastering-kubernetes-costs-from-monitoring-to-automation","microservices","mlxcel-rust-native-inference-engine-tested-on-m1-max","moving-code-between-git-repositories-with-copybara","multi-stage-docker-build","multi-tenancy-in-2025-and-beyond","my-first-international-conference-open-source-summit-2022","my-journey-to-kubestronaut-on-kubernetes-10th-birthday","my-kubecon-euvirtual-experience","my-schedule-for-kubecon-cloudnativecon-eu-2022","navigating-through-cncf-landscape","nemotron3-on-dgx-spark","networking-fundamentals-for-devops","nexus-repository-manager-what-is-it-and-how-to-configure-it-on-a-digital-ocean-droplet","nvcf-is-now-open-source-inside-nvidia-s-gpu-function-platform","operating-systems-101-essential-knowledge-for-devopssre-engineers","optimizing-kubernetes-costs-balancing-spot-and-on-demand-instances-with-topology-spread-constraints","optimizing-scalability-a-deep-dive-into-load-testing-with-locust-on-eks","package-managers-demystified","perform-crud-operations-on-kubernetes-using-golang","platform-engineering-demystified-navigating-the-basics","pods-in-kubernetes","practical-guide-to-kubernetes-api","progressive-rollouts-with-argo-cd-rollouts","prometheus-explained","pure-cilium-a-guide-for-local-load-balancing-and-bgp","quick-bites-of-fluxcd-health-assessment","rancher-desktop-evolution","ready-for-wasm-day-2023","simplified-introduction-to-bacalhau","speeding-up-using-microk8s","ssh-into-your-dgx-spark-from-anywhere-in-the-world-using-tailscale","starting-your-devops-journey-as-a-windows-user","statefulsets","supply-chain-security-using-slsa-part-1-fundamentals","supply-chain-security-using-slsa-part-2-the-framework","terraform-best-practices","testing-docker-ais-gordon-how-smart-is-it","the-complete-guide-to-the-dd-command-in-linux","the-secret-gems-behind-building-container-images-enter-buildkit-and-docker-buildx","the-ultimate-guide-to-audit-logging-in-kubernetes-from-setup-to-analysis","the-webassembly-course","tutorial-build-a-cloud-cost-monitoring-system-with-terraform-ansible-and-komiser","understanding-docker-desktop-all-in-one-platform-for-containers","understanding-etcd-in-kubernetes-a-beginners-guide","understanding-how-containers-work-behind-the-scenes","understanding-the-architecture-of-kubernetes-a-beginners-guide","understanding-the-ins-and-outs-of-git-using-github","wandler-local-openai-compatible-inference-transformersjs-webgpu","what-is-reproducibility-and-why-does-it-matter","what-is-shell-scripting","why-are-network-policies-in-kubernetes-so-hard-to-understand","why-devops-case-study","wtf-is-linux-shell-command-substitution","yours-kindly-drone"]); +const BLOG_SLUGS = new Set(["10-things-you-might-not-know-about-k9s","12-practical-grep-command-examples-in-linux","a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-1","a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2","a-complete-walk-through-of-devops","a-kubeconfig-for-gke-that-doesnt-need-gcloud","a-simple-way-to-structure-your-terraform-code","a-simplified-guide-to-yaml","about-my-pdf-editor-project","an-overview-of-gitops-and-argocd","announcing-buildsafe","api-response-in-go","arkade","automate-repetitive-tasks-shell-scripting","automated-github-releases-with-github-actions-and-conventional-commits","avoid-overspending-with-kubecost","aws-elastic-cloud-compute","bake-your-container-images-with-bake","become-a-hashicorp-certified-terraform-associate-preparation-guide","best-devops-tools-2025","breaking-down-docker","building-a-zero-cve-strategy","building-apigateway-with-lambda-using-pulumi","certified-kubernetes-security-specialist-cks-2022-exam-guide","cicd-pipeline-github-actions-with-aws-ecs","ckad-exam-april-2022","claude-code-leak-what-the-source-actually-teaches","clawspark-your-private-openclaw-ai-assistant-that-never-phones-home","cloud-computing","cloud-native-buildpacks-concepts","confidential-containers-running-on-kubernetes","container-and-kubernetes-security","controlling-mcp-tools-with-agentgateway-on-kubernetes","coolify","creating-multi-node-kubernetes-cluster-locally","day-1-the-local-llm-revolution-why-your-desk-just-became-the-new-datacenter","day-1-what-actually-happens-when-you-type-docker-run","day-2-anatomy-of-an-llm-inference-request-from-prompt-to-answer-step-by-step","day-2-your-images-are-a-supply-chain-and-it-s-probably-broken","day-3-stop-writing-dockerfiles-from-scratch","day-3-the-dgx-spark-unpacked-gb10-unified-memory-sm-121-and-the-one-reason-this-hardware-exists","day-4-breaking-isolation-on-purpose-volumes-networks-and-the-real-world","day-4-quantization-demystified-bf16-fp8-nvfp4-mxfp4-int4-gguf-and-why-it-all-matters","day-5-docker-compose-how-docker-actually-gets-used","day-6-run-an-llm-on-your-laptop-with-docker","day-7-ship-it-and-what-comes-next","deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws","deploy-a-simple-server-using-aws-terraform","deploying-java-application-using-docker-and-kubernetes-devops-project","ditch-the-overheating-laptop-supercharge-your-docker-workflow-with-docker-offload","diy-how-to-build-a-kubernetes-policy-engine","docker-captain-journey","docker-mcp-catalog","docker-networking-demystified","embed-http-servers-in-wasm-with-rust-and-csharp","enhancing-runtime-security-with-falco-my-hands-on-experience","ephemeral-pull-request-environment-using-vcluster","essential-linux-commands-for-devops","event-driven-architecture-simplified-monolith-to-microservices","everything-you-need-to-know-about-docker-compose","everything-you-need-to-know-about-the-linux-ls-command","exploiting-metasploitable2-using-msfconsole-kali-linux-lab","firewall-a-networks-gatekeeper","four-pillars-of-observability-in-kubernetes","get-good-at-git","getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster","getting-started-with-ko-a-fast-container-image-builder-for-your-go-applications","getting-started-with-kyverno","git-and-github-a-beginners-guide","github-actions-101-what-are-github-actions-and-how-to-use-them-a-beginners-guide","gitops-demystified","ha-kubernetes","how-a-kubernetes-service-actually-works-and-all-5-types-you-need","how-get-started-with-hashicorp-vault","how-kubernetes-endpointslices-actually-work-and-why-endpoints-had-to-die","how-to-backup-kubernetes-with-kasten-community-edition","how-to-change-directory-in-shell-scripts","how-to-install-a-kubernetes-cluster-with-kubeadm-containerd-and-cilium-a-hands-on-guide","how-to-setup-your-ftp-server-in-linux","implementing-kubernetes-network-policies-a-comprehensive-guide","important-concepts-of-operating-systems","ing-switch-119-annotations-gateway-api-traefik-impact-ratings","ing-switch-migrate-from-ingress-nginx-to-traefik-or-gateway-api-in-minutes-not-days","installing-prometheus-with-selinux","introducing-unikraft-lightweight-virtualization-using-unikernels","introduction-of-jenkins-pipeline","introduction-to-cicd-and-cicd-pipeline","introduction-to-cri","introduction-to-developer-platforms-with-gimlet","introduction-to-helm","introduction-to-jenkins","introduction-to-kubernetes","introduction-to-terraform","iptables-demo","istio-service-mesh","k8sgpt-tutorial-when-kubernetes-meets-ai","keptn-getting-started","ksctl-making-kubernetes-easy-across-clouds","kube-proxy-deep-dive","kube-scheduler-deep-dive","kubecon-cloudnativecon-north-america-2024-recap-themes-innovations-and-community-spirit","kubecon-cloudnativecon-rejekts-and-wasm-io-wrap-up-a-leap-into-the-future-with-webassembly-ai-and-sustainable-cloud-practices","kubectl-run-nginx-inside","kubeflow-machine-learning-on-kubernetes-part-1","kubeflow-notebooks-ml-experimentation-made-easier-part-2","kubeflow-pipelines-orchestrating-machine-learning-workflows-part-3","kubernetes-125-dockerd","kubernetes-126","kubernetes-access-control-with-authentication-authorization-admission-control","kubernetes-adoption-key-challenges-in-migrating-to-kubernetes","kubernetes-backup-using-cloudcasa","kubernetes-containerd-setup","kubernetes-crio","kubernetes-management-with-rust-a-dive-into-generic-client-go-controller-abstractions-and-crd-macros-with-kubers","kubernetes-on-apple-macbooks-m-series","kubernetes-scheduling-the-complete-guide","kubernetes-v133-key-features-updates-and-what-you-need-to-know","kubernetes-v135-whats-new-whats-changing-and-what-you-should-know","kubesimplify-a-journey-to-remember","kubesimplify-at-wasmio-and-kubecon-eu-2024","kyverno-and-cosign","kyverno-cli","lets-learn-terraform","lets-simplify-golang-part-1","lets-simplify-golang-part-2","lets-simplify-golang-part-3","lets-talk-about-ansible","linux-boot-process-simplified","linux-system-directories-explained","llm-costs-and-observability-with-agentgateway-on-kubernetes","managing-contexts-in-kubernetes-with-plugins","managing-your-operating-system-with-package-managers","mastering-kubernetes-costs-from-monitoring-to-automation","microservices","mlxcel-rust-native-inference-engine-tested-on-m1-max","moving-code-between-git-repositories-with-copybara","multi-stage-docker-build","multi-tenancy-in-2025-and-beyond","my-first-international-conference-open-source-summit-2022","my-journey-to-kubestronaut-on-kubernetes-10th-birthday","my-kubecon-euvirtual-experience","my-schedule-for-kubecon-cloudnativecon-eu-2022","navigating-through-cncf-landscape","nemotron3-on-dgx-spark","networking-fundamentals-for-devops","nexus-repository-manager-what-is-it-and-how-to-configure-it-on-a-digital-ocean-droplet","nvcf-is-now-open-source-inside-nvidia-s-gpu-function-platform","operating-systems-101-essential-knowledge-for-devopssre-engineers","optimizing-kubernetes-costs-balancing-spot-and-on-demand-instances-with-topology-spread-constraints","optimizing-scalability-a-deep-dive-into-load-testing-with-locust-on-eks","package-managers-demystified","perform-crud-operations-on-kubernetes-using-golang","platform-engineering-demystified-navigating-the-basics","pods-in-kubernetes","practical-guide-to-kubernetes-api","progressive-rollouts-with-argo-cd-rollouts","prometheus-explained","pure-cilium-a-guide-for-local-load-balancing-and-bgp","quick-bites-of-fluxcd-health-assessment","rancher-desktop-evolution","ready-for-wasm-day-2023","simplified-introduction-to-bacalhau","speeding-up-using-microk8s","ssh-into-your-dgx-spark-from-anywhere-in-the-world-using-tailscale","starting-your-devops-journey-as-a-windows-user","statefulsets","supply-chain-security-using-slsa-part-1-fundamentals","supply-chain-security-using-slsa-part-2-the-framework","terraform-best-practices","testing-docker-ais-gordon-how-smart-is-it","the-complete-guide-to-the-dd-command-in-linux","the-secret-gems-behind-building-container-images-enter-buildkit-and-docker-buildx","the-ultimate-guide-to-audit-logging-in-kubernetes-from-setup-to-analysis","the-webassembly-course","tutorial-build-a-cloud-cost-monitoring-system-with-terraform-ansible-and-komiser","understanding-docker-desktop-all-in-one-platform-for-containers","understanding-etcd-in-kubernetes-a-beginners-guide","understanding-how-containers-work-behind-the-scenes","understanding-the-architecture-of-kubernetes-a-beginners-guide","understanding-the-ins-and-outs-of-git-using-github","wandler-local-openai-compatible-inference-transformersjs-webgpu","what-is-reproducibility-and-why-does-it-matter","what-is-shell-scripting","why-are-network-policies-in-kubernetes-so-hard-to-understand","why-devops-case-study","wtf-is-linux-shell-command-substitution","yours-kindly-drone"]); export default { async fetch(request, env) { diff --git a/public/atom.xml b/public/atom.xml index 01f61d1c..c8c588b4 100644 --- a/public/atom.xml +++ b/public/atom.xml @@ -5,11 +5,24 @@ https://blog.kubesimplify.com/ - 2026-06-29T20:14:38.552Z + 2026-07-06T12:14:30.534Z Kubesimplify hello@kubesimplify.com + + LLM Costs and Observability with agentgateway on Kubernetes (Part 2) + + https://blog.kubesimplify.com/llm-costs-and-observability-with-agentgateway-on-kubernetes + 2026-06-30T10:00:00.000Z + 2026-06-30T10:00:00.000Z + Part 2: scrape agentgateway with Prometheus, build a Grafana dashboard of token cost and per-tool usage, see blocked tool calls, and alert on spend. + + + + + + Controlling MCP Tools with agentgateway on Kubernetes (Part 1) diff --git a/public/img/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes/agw-monitoring.png b/public/img/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes/agw-monitoring.png new file mode 100644 index 00000000..5447d363 Binary files /dev/null and b/public/img/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes/agw-monitoring.png differ diff --git a/public/img/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes/cost-dashboard.png b/public/img/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes/cost-dashboard.png new file mode 100644 index 00000000..46daa459 Binary files /dev/null and b/public/img/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes/cost-dashboard.png differ diff --git a/public/img/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes/cover.png b/public/img/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes/cover.png new file mode 100644 index 00000000..e629bcd1 Binary files /dev/null and b/public/img/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes/cover.png differ diff --git a/public/llms.txt b/public/llms.txt index bb8de4bc..b2488ea9 100644 --- a/public/llms.txt +++ b/public/llms.txt @@ -4,7 +4,7 @@ ## About -Kubesimplify is a community-driven publication on cloud-native technologies, with 186 in-depth technical articles by 62 practitioner authors. We cover Kubernetes (kubelet internals, scheduling, networking, operators), container runtimes (containerd, CRI-O, Docker), GitOps (Argo CD, Flux), service meshes, observability, AI/ML infrastructure on Kubernetes, GPU workloads, platform engineering, and the broader CNCF ecosystem. +Kubesimplify is a community-driven publication on cloud-native technologies, with 187 in-depth technical articles by 62 practitioner authors. We cover Kubernetes (kubelet internals, scheduling, networking, operators), container runtimes (containerd, CRI-O, Docker), GitOps (Argo CD, Flux), service meshes, observability, AI/ML infrastructure on Kubernetes, GPU workloads, platform engineering, and the broader CNCF ecosystem. Authoritative, practitioner-written, citation-friendly. Articles include code examples, diagrams, and references. @@ -34,8 +34,9 @@ Authoritative, practitioner-written, citation-friendly. Articles include code ex - Cloud Native Security: https://blog.kubesimplify.com/hub/security (network policies, Falco, Kyverno, SLSA supply-chain) - Linux Fundamentals: https://blog.kubesimplify.com/hub/linux (shell, sysadmin, networking primitives) -## Recent posts (most recent 30 of 186) +## Recent posts (most recent 30 of 187) +- [LLM Costs and Observability with agentgateway on Kubernetes (Part 2)](https://blog.kubesimplify.com/llm-costs-and-observability-with-agentgateway-on-kubernetes) (2026-06-30). Part 2: scrape agentgateway with Prometheus, build a Grafana dashboard of token cost and per-tool usage, see blocked tool calls, and alert on spend. - [Controlling MCP Tools with agentgateway on Kubernetes (Part 1)](https://blog.kubesimplify.com/controlling-mcp-tools-with-agentgateway-on-kubernetes) (2026-06-29). Run AI agents behind agentgateway on Kubernetes: route their LLM and MCP tool calls through one proxy, keep secrets out of the agent, and block tools by policy. - [Day 4: Quantization Demystified. BF16, FP8, NVFP4, MXFP4, INT4, GGUF, and Why It All Matters](https://blog.kubesimplify.com/day-4-quantization-demystified-bf16-fp8-nvfp4-mxfp4-int4-gguf-and-why-it-all-matters) (2026-06-10). A practical, beginner-friendly guide to BF16, FP8, NVFP4, MXFP4, INT4, and GGUF Q4_K_M on NVIDIA DGX Spark. Bytes per parameter, quality vs size, and which format to pick when. - [Day 3: The DGX Spark Unpacked. GB10, Unified Memory, sm_121, and the One Reason This Hardware Exists](https://blog.kubesimplify.com/day-3-the-dgx-spark-unpacked-gb10-unified-memory-sm-121-and-the-one-reason-this-hardware-exists) (2026-06-05). A practical teardown of NVIDIA DGX Spark's GB10 Grace Blackwell Superchip, unified memory, sm_121, NVFP4 tensor cores, memory reporting, and decode limits. @@ -65,11 +66,10 @@ Authoritative, practitioner-written, citation-friendly. Articles include code ex - [ing-switch: Migrate from Ingress NGINX to Traefik or Gateway API in Minutes, Not Days](https://blog.kubesimplify.com/ing-switch-migrate-from-ingress-nginx-to-traefik-or-gateway-api-in-minutes-not-days) (2026-02-25) - [Exploiting Metasploitable2 Using msfconsole (Kali Linux Lab)](https://blog.kubesimplify.com/exploiting-metasploitable2-using-msfconsole-kali-linux-lab) (2026-01-17) - [Kubernetes v1.35 – What’s New, What’s Changing, and What You Should Know](https://blog.kubesimplify.com/kubernetes-v135-whats-new-whats-changing-and-what-you-should-know) (2025-12-19). Learn about the latest Kubenretes release Kubernetes 1.35 -- [Ditch the Overheating Laptop: Supercharge Your Docker Workflow with Docker Offload](https://blog.kubesimplify.com/ditch-the-overheating-laptop-supercharge-your-docker-workflow-with-docker-offload) (2025-08-26). Running multiple Docker containers can slow down your laptop and drain your battery. In this blog, we explore Docker Offload — a game-changing feature ## Topics covered (auto-derived from tags) -- kubernetes (93 articles): https://blog.kubesimplify.com/tag/kubernetes +- kubernetes (94 articles): https://blog.kubesimplify.com/tag/kubernetes - devops (71 articles): https://blog.kubesimplify.com/tag/devops - docker (31 articles): https://blog.kubesimplify.com/tag/docker - k8s (27 articles): https://blog.kubesimplify.com/tag/k8s diff --git a/public/rss.xml b/public/rss.xml index 114df719..2a25cecf 100644 --- a/public/rss.xml +++ b/public/rss.xml @@ -6,8 +6,16 @@ Deep dives on Kubernetes, AI infrastructure, GitOps, and the cloud-native stack, written by practitioners. en-us - Mon, 29 Jun 2026 10:00:00 GMT + Tue, 30 Jun 2026 10:00:00 GMT Kubesimplify static blog + + LLM Costs and Observability with agentgateway on Kubernetes (Part 2) + https://blog.kubesimplify.com/llm-costs-and-observability-with-agentgateway-on-kubernetes + https://blog.kubesimplify.com/llm-costs-and-observability-with-agentgateway-on-kubernetes + Tue, 30 Jun 2026 10:00:00 GMT + Part 2: scrape agentgateway with Prometheus, build a Grafana dashboard of token cost and per-tool usage, see blocked tool calls, and alert on spend. + kubernetesai-agentsobservabilityprometheusgrafana + Controlling MCP Tools with agentgateway on Kubernetes (Part 1) https://blog.kubesimplify.com/controlling-mcp-tools-with-agentgateway-on-kubernetes diff --git a/vercel.json b/vercel.json index 513aff4c..73ee3717 100644 --- a/vercel.json +++ b/vercel.json @@ -696,6 +696,11 @@ "destination": "https://blog.kubesimplify.com/linux-system-directories-explained", "permanent": true }, + { + "source": "/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes", + "destination": "https://blog.kubesimplify.com/llm-costs-and-observability-with-agentgateway-on-kubernetes", + "permanent": true + }, { "source": "/blog/managing-contexts-in-kubernetes-with-plugins", "destination": "https://blog.kubesimplify.com/managing-contexts-in-kubernetes-with-plugins", @@ -2400,6 +2405,17 @@ } ] }, + { + "source": "/llm-costs-and-observability-with-agentgateway-on-kubernetes", + "destination": "https://blog.kubesimplify.com/llm-costs-and-observability-with-agentgateway-on-kubernetes", + "permanent": true, + "has": [ + { + "type": "host", + "value": "kubesimplify.com" + } + ] + }, { "source": "/managing-contexts-in-kubernetes-with-plugins", "destination": "https://blog.kubesimplify.com/managing-contexts-in-kubernetes-with-plugins",