NVIDIA NeMo Agent Toolkit

NVIDIA NeMo Agent Toolkit adds intelligence to AI agents across any framework—enhancing speed, accuracy, and decision-making through enterprise-grade instrumentation, observability, and continuous learning.

🔥 New Features

• LangGraph Agent Automatic Wrapper: Easily onboard existing LangGraph agents to NeMo Agent Toolkit. Use the automatic wrapper to access NeMo Agent Toolkit advanced features with very little modification of LangGraph agents.

• Automatic Reinforcement Learning (RL): Improve your agent quality by fine-tuning open LLMs to better understand your agent's workflows, tools, and prompts. Perform GRPO with OpenPipe ART or DPO with NeMo Customizer using NeMo Agent Toolkit built-in evaluation system as a verifier.

• Initial NVIDIA Dynamo Integration: Accelerate end-to-end deployment of agentic workflows with initial Dynamo support. Utilize the new agent-aware router to improve worker latency by predicting future agent behavior.

• A2A Support: Build teams of distributed agents using the A2A protocol.

• Safety and Security Engine: Strengthen safety and security workflows by simulating scenario-based attacks, profiling risk, running guardrail-ready evaluations, and applying defenses with red teaming. Validate defenses, profile risk, monitor behavior, and harden agents across any framework.

• Amazon Bedrock AgentCore and Strands Agents Support: Build agents using Strands Agents framework and deploy them securely on Amazon Bedrock AgentCore runtime.

• Microsoft AutoGen Support: Build agents using the Microsoft AutoGen framework.

• Per-User Functions: Use per-user functions for deferred instantiation, enabling per-user stateful functions, per-user resources, and other features.

✨ Key Features

• 🛠️ Building Agents: Accelerate your agent development with tools that make it easier to get your agent into production.
  • 🧩 Framework Agnostic: Work side-by-side with agentic frameworks to add the instrumentation necessary for observing, profiling, and optimizing your agents. Use the toolkit with popular frameworks such as LangChain, LlamaIndex, CrewAI, Microsoft Semantic Kernel, and Google ADK, as well as custom enterprise agentic frameworks and simple Python agents.
  • 🔁 Reusability: Build components once and use them multiple times to maximize the value from development effort.
  • ⚡ Customization: Start with a pre-built agent, tool, or workflow, and customize it to your needs.
  • 💬 Built-In User Interface: Use the NeMo Agent Toolkit UI chat interface to interact with your agents, visualize output, and debug workflows.
• 📈 Agent Insights: Utilize NeMo Agent Toolkit instrumentation to better understand how your agents function at runtime.
  • 📊 Profiling: Profile entire workflows from the agent level all the way down to individual tokens to identify bottlenecks, analyze token efficiency, and guide developers in optimizing their agents.
  • 🔎 Observability: Track performance, trace execution flows, and gain insights into your agent behaviors in production.
• 🚀 Agent Optimization: Improve your agent's quality, accuracy, and performance with a suite of tools for all phases of the agent lifecycle.
  • 🧪 Evaluation System: Validate and maintain accuracy of agentic workflows with a suite of tools for offline evaluation.
  • 🎯 Hyper-Parameter and Prompt Optimizer: Automatically identify the best configuration and prompts to ensure you are getting the most out of your agent.
  • 🧠 Fine-tuning with Reinforcement Learning: Fine-tune LLMs specifically for your agent and train intrinsic information about your workflow directly into the model.
  • ⚡ NVIDIA Dynamo Integration: Use Dynamo and NeMo Agent Toolkit together to improve agent performance at scale.
• 🔌 Protocol Support: Integrate with common protocols used to build agents.
  • 🔗 Model Context Protocol (MCP): Integrate MCP tools into your agents or serve your tools and agents as an MCP server for others to consume.
  • 🤝 Agent-to-Agent (A2A) Protocol: Build teams of distributed agents with full support for authentication.

With NeMo Agent Toolkit, you can move quickly, experiment freely, and ensure reliability across all your agent-driven projects.

🚀 Installation

Before you begin using NeMo Agent Toolkit, ensure that you have Python 3.11, 3.12, or 3.13 installed on your system.

Note

For users who want to run the examples, it's required to clone the repository and install from source to get the necessary files required to run the examples. Please refer to the Examples documentation for more information.

To install the latest stable version of NeMo Agent Toolkit from PyPI, run the following command:

pip install nvidia-nat

NeMo Agent Toolkit has many optional dependencies that can be installed with the core package. Optional dependencies are grouped by framework. For example, to install the LangChain/LangGraph plugin, run the following:

pip install "nvidia-nat[langchain]"

Detailed installation instructions, including the full list of optional dependencies and their conflicts, can be found in the Installation Guide.

🌟 Hello World Example

Before getting started, it's possible to run this simple workflow and many other examples in Google Colab with no setup. Click here to open the introduction notebook: .

1. Ensure you have set the NVIDIA_API_KEY environment variable to allow the example to use NVIDIA NIMs. An API key can be obtained by visiting build.nvidia.com and creating an account.

   export NVIDIA_API_KEY=<your_api_key>

2. Create the NeMo Agent Toolkit workflow configuration file. This file will define the agents, tools, and workflows that will be used in the example. Save the following as workflow.yml:

   functions:
      # Add a tool to search wikipedia
      wikipedia_search:
         _type: wiki_search
         max_results: 2
   
   llms:
      # Tell NeMo Agent Toolkit which LLM to use for the agent
      nim_llm:
         _type: nim
         model_name: nvidia/nemotron-3-nano-30b-a3b
         temperature: 0.0
         chat_template_kwargs:
            enable_thinking: false
   
   workflow:
      # Use an agent that 'reasons' and 'acts'
      _type: react_agent
      # Give it access to our wikipedia search tool
      tool_names: [wikipedia_search]
      # Tell it which LLM to use
      llm_name: nim_llm
      # Make it verbose
      verbose: true
      # Retry up to 3 times
      parse_agent_response_max_retries: 3

3. Run the Hello World example using the nat CLI and the workflow.yml file.

   nat run --config_file workflow.yml --input "List five subspecies of Aardvarks"

   This will run the workflow and output the results to the console.

   Workflow Result:
   ['Here are five subspecies of Aardvarks:\n\n1. Orycteropus afer afer (Southern aardvark)\n2. O. a. adametzi  Grote, 1921 (Western aardvark)\n3. O. a. aethiopicus  Sundevall, 1843\n4. O. a. angolensis  Zukowsky & Haltenorth, 1957\n5. O. a. erikssoni  Lönnberg, 1906']

📚 Additional Resources

• 📖 Documentation: Explore the full documentation for NeMo Agent Toolkit.
• 🧭 Get Started Guide: Set up your environment and start building with NeMo Agent Toolkit.
• 🤝 Contributing: Learn how to contribute to NeMo Agent Toolkit and set up your development environment.
• 🧪 Examples: Explore examples of NeMo Agent Toolkit workflows located in the examples directory of the source repository.
• 🛠️ Create and Customize NeMo Agent Toolkit Workflows: Learn how to create and customize NeMo Agent Toolkit workflows.
• 🎯 Evaluate with NeMo Agent Toolkit: Learn how to evaluate your NeMo Agent Toolkit workflows.
• 🆘 Troubleshooting: Get help with common issues.

🛣️ Roadmap

• Automatic Reinforcement Learning (RL) to fine-tune LLMs for a specific agent.
• Integration with NVIDIA Dynamo to reduce LLM latency at scale.
• Improve agent throughput with KV-Cache optimization.
• Integration with NeMo Guardrails to improve agent safety and security.
• Improved memory interface to support self-improving agents.

💬 Feedback

We would love to hear from you! Please file an issue on GitHub if you have any feedback or feature requests.

🤝 Acknowledgements

We would like to thank the following groups for their contribution to the toolkit:

• Synopsys
  • Google ADK framework support.
  • Microsoft AutoGen framework support.
• W&B Weave Team
  • Contributions to the evaluation and telemetry system.

In addition, we would like to thank the following open source projects that made NeMo Agent Toolkit possible:

• Agent2Agent (A2A) Protocol
• CrewAI
• Dynamo
• FastAPI
• Google Agent Development Kit (ADK)
• LangChain
• Llama-Index
• Mem0ai
• Microsoft AutoGen
• MinIO
• Model Context Protocol (MCP)
• OpenTelemetry
• Phoenix
• Ragas
• Redis
• Semantic Kernel
• Strands
• uv
• Weave