LLM Router is a service that can be deployed on‑premises or in the cloud. It adds a layer between any application and the LLM provider. In real time it controls traffic, distributes a load among providers of a specific LLM, and enables analysis of outgoing requests from a security perspective (masking, anonymization, prohibited content).
security automation cloud rest-api prometheus model-management load-balancing pii on-prem llm genai local-llm llm-router llm-gateway local-llm-integration llm-gateway-system llm-balancing llm-router-models llm-router-plugins
-
Updated
Jan 3, 2026 - Python