Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 139 additions & 3 deletions docs/ai-quickstart.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ async function getWeather(ctx: restate.Context, city: string) {
// AGENT
const run = async (ctx: restate.Context, { prompt }: { prompt: string }) => {
const model = wrapLanguageModel({
model: openai("gpt-4o"),
model: openai("gpt-5.4"),
// Persist LLM responses
middleware: durableCalls(ctx, { maxRetryAttempts: 3 }),
});
Expand Down Expand Up @@ -589,7 +589,7 @@ class WeatherPrompt(BaseModel):

# AGENT
weather_agent = Agent(
"openai:gpt-4o-mini",
"openai:gpt-5.4",
system_prompt="You are a helpful agent that provides weather updates.",
)

Expand Down Expand Up @@ -649,6 +649,142 @@ Once you restart the service, the agent resumes at the weather tool call and suc
**Next step:**
Learn more about [Durable Agents](/ai/patterns/durable-agents) and how Restate makes your AI agents resilient to failures.

</Step>
</Steps>
</GlobalTab>
<GlobalTab title="LangChain" icon={"/img/languages/python.svg"}>

<Info>
**Prerequisites**:
- Python >= v3.12
- [uv](https://docs.astral.sh/uv/getting-started/installation/)
- OpenAI API key (get one at [OpenAI](https://platform.openai.com/))
</Info>

<Steps>
<Step title="Install Restate Server & CLI">
<InstallRestate/>
</Step>

<Step title="Get the AI Agent template">
Get the weather agent template for [LangChain](https://www.langchain.com/) and Restate:
```shell
restate example python-langchain-template && cd python-langchain-template
```
<GitHubLink url={"https://github.com/restatedev/ai-examples/tree/langchain/langchain-python/template"}/>
</Step>

<Step title="Run the AI Agent service">
Export your OpenAI key and run the agent:
```shell
export OPENAI_API_KEY=your_openai_api_key_here
uv run .
```

The weather agent is now listening on port 9080.
</Step>

<Step title="Register the agent service">
Tell Restate where the service is running (`http://localhost:9080`), so Restate can discover and register the services and handlers behind this endpoint.
You can do this via the UI (`http://localhost:9070`):

<Frame>
<img src={"/img/ai/registration.png"} alt="Restate UI Playground"/>
</Frame>

If you run Restate with Docker, register `http://host.docker.internal:9080` instead of `http://localhost:9080`.

<Accordion title="Restate Cloud">
When using [Restate Cloud](https://restate.dev/cloud), your service must be accessible over the public internet so Restate can invoke it.
If you want to develop with a local service, you can expose it using our [tunnel](/deploy/server/cloud/#registering-restate-services-with-your-environment) feature.
</Accordion>
</Step>

<Step title="Send weather requests to the AI Agent">
Invoke the agent via the Restate UI playground: go to `http://localhost:9070`, click on your service and then on playground.

Or invoke via `curl`:
```shell
curl localhost:8080/agent/run --json '{"message": "What is the weather in San Francisco?"}'
```
Output: `The weather in San Francisco is sunny and 23°C.`

</Step>

<Step title="Congratulations, you just ran a Durable AI Agent!">

The agent you just invoked uses Durable Execution to make agents resilient to failures. Restate persisted all LLM calls and tool execution steps, so if anything fails, the agent can resume exactly where it left off.

We did this by attaching Restate's `RestateMiddleware` to the LangChain agent so every LLM call is journaled, and by using [Restate Context actions](/foundations/actions) (e.g. `restate_context().run_typed`) inside tools to make side effects durable:

```python expandable {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/langchain-python/template/agent.py"}
import restate
from langchain.agents import create_agent
from langchain_core.messages import AnyMessage
from langchain_core.tools import tool
from langchain.chat_models import init_chat_model
from pydantic import BaseModel
from restate.ext.langchain import RestateMiddleware, restate_context


class WeatherPrompt(BaseModel):
message: str = "What is the weather in San Francisco?"


# TOOL
@tool
async def get_weather(city: str) -> dict:
"""Get the current weather for a given city."""

async def call_weather_api() -> dict:
return {"temperature": 23, "description": "Sunny and warm."}

# Durable step: results are journaled, so on retry we replay the value
# rather than re-hitting the API.
return await restate_context().run_typed(f"Get weather {city}", call_weather_api)


# AGENT
weather_agent = create_agent(
model=init_chat_model("openai:gpt-5.4"),
tools=[get_weather],
system_prompt="You are a helpful agent that provides weather updates.",
middleware=[RestateMiddleware()],
)


# AGENT SERVICE
agent_service = restate.Service("agent")


@agent_service.handler()
async def run(_ctx: restate.Context, req: WeatherPrompt) -> str:
result = await weather_agent.ainvoke(
{"messages": [{"role": "user", "content": req.message}]}
)
return result["messages"][-1].content
```
<GitHubLink url={"https://github.com/restatedev/ai-examples/blob/main/langchain-python/template/agent.py"}/>

The Invocations tab of the Restate UI shows us how Restate captured each LLM call and tool step in a journal.

<Accordion title="See how a failing tool call is retried">
Let the weather tool raise an exception by adding the following line to the `get_weather` function:
```python
raise Exception("[👻 SIMULATED] Fetching weather failed: Weather API down...")
```

You can see in the Restate UI how each LLM call and tool step gets durably executed.
We can see how the weather tool is currently stuck, because the weather API is down.

Fix the problem, by removing the exception again.

Once you restart the service, the agent resumes at the weather tool call and successfully completes the request.
</Accordion>

**Next step:**
Learn more about [Durable Agents](/ai/patterns/durable-agents) and how Restate makes your AI agents resilient to failures.

</Step>
</Steps>
</GlobalTab>
Expand Down Expand Up @@ -952,7 +1088,7 @@ async def run(ctx: restate.Context, message: WeatherPrompt) -> str | None:
# Call the LLM
async def call_llm() -> Message:
resp = await acompletion(
model="gpt-4o-mini", messages=messages, tools=TOOLS
model="gpt-5.4", messages=messages, tools=TOOLS
)
return resp.choices[0].message

Expand Down
1 change: 1 addition & 0 deletions docs/ai/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ Whether you're building chatbots, autonomous agents, or AI-powered workflows, Re
<Card title="OpenAI Agents SDK" icon="/img/ai/sdk-integrations/openai.webp" href="/ai/sdk-integrations/openai-agents-sdk" horizontal/>
<Card title="Google ADK" href="/ai/sdk-integrations/google-adk" icon="/img/ai/sdk-integrations/google-adk.png" horizontal/>
<Card title="Pydantic AI" href="/ai/sdk-integrations/pydantic-ai" icon="/img/ai/sdk-integrations/pydantic-ai.png" horizontal/>
<Card title="LangChain" href="/ai/sdk-integrations/langchain" icon="/img/ai/sdk-integrations/langchain.svg" horizontal/>
<Card title="Integrating with other SDKs" href="/ai/sdk-integrations/integration-guide" icon={"puzzle-piece"} horizontal/>
</CardGroup>

Expand Down
76 changes: 73 additions & 3 deletions docs/ai/patterns/durable-agents.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ Select your SDK:
<GlobalTab title="OpenAI Agents" icon={"/img/languages/python.svg"}/>
<GlobalTab title="Google ADK" icon={"/img/languages/python.svg"}/>
<GlobalTab title="Pydantic AI" icon={"/img/languages/python.svg"}/>
<GlobalTab title="LangChain" icon={"/img/languages/python.svg"}/>
<GlobalTab title="Restate TS" icon={"/img/languages/typescript.svg"}/>
<GlobalTab title="Restate Py" icon={"/img/languages/python.svg"}/>
</GlobalTabs>
Expand Down Expand Up @@ -65,7 +66,7 @@ async function getWeather(ctx: restate.Context, city: string) {
// AGENT
const run = async (ctx: restate.Context, { prompt }: { prompt: string }) => {
const model = wrapLanguageModel({
model: openai("gpt-4o"),
model: openai("gpt-5.4"),
// Persist LLM responses
middleware: durableCalls(ctx, { maxRetryAttempts: 3 }),
});
Expand Down Expand Up @@ -277,7 +278,7 @@ class WeatherPrompt(BaseModel):

# AGENT
weather_agent = Agent(
"openai:gpt-4o-mini",
"openai:gpt-5.4",
system_prompt="You are a helpful agent that provides weather updates.",
)

Expand Down Expand Up @@ -313,6 +314,70 @@ The main difference is the use of the **Restate Context** at key points:
2. **Persisting LLM responses**: Wrap your agent with `RestateAgent` so every LLM response is saved in the Restate Server and replayed during recovery. The `RestateAgent` is provided via the Pydantic AI extensions in the Restate SDK.
3. **Resilient tool execution**: Use `restate_context().run_typed()` inside tools to make steps durable. The result is persisted and retried until it succeeds.

</GlobalTab>
<GlobalTab title="LangChain">

To implement a durable agent, you use the Restate SDK in combination with [LangChain](https://www.langchain.com/).

Here's a weather agent that looks up the weather for a city:

```python agent.py {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/langchain-python/template/agent.py"}
import restate
from langchain.agents import create_agent
from langchain_core.messages import AnyMessage
from langchain_core.tools import tool
from langchain.chat_models import init_chat_model
from pydantic import BaseModel
from restate.ext.langchain import RestateMiddleware, restate_context


class WeatherPrompt(BaseModel):
message: str = "What is the weather in San Francisco?"


# TOOL
@tool
async def get_weather(city: str) -> dict:
"""Get the current weather for a given city."""

async def call_weather_api() -> dict:
return {"temperature": 23, "description": "Sunny and warm."}

# Durable step: results are journaled, so on retry we replay the value
# rather than re-hitting the API.
return await restate_context().run_typed(f"Get weather {city}", call_weather_api)


# AGENT
weather_agent = create_agent(
model=init_chat_model("openai:gpt-5.4"),
tools=[get_weather],
system_prompt="You are a helpful agent that provides weather updates.",
middleware=[RestateMiddleware()],
)


# AGENT SERVICE
agent_service = restate.Service("agent")


@agent_service.handler()
async def run(_ctx: restate.Context, req: WeatherPrompt) -> str:
result = await weather_agent.ainvoke(
{"messages": [{"role": "user", "content": req.message}]}
)
return result["messages"][-1].content
```

<GitHubLink url={"https://github.com/restatedev/ai-examples/blob/main/langchain-python/template/agent.py"}/>

You define your agent and tools as you normally would with LangChain.

The main difference is the use of the **Restate Context** at key points:
1. **Restate service handler**: The agent runs inside a Restate service handler, giving it a durable execution context. Restate exposes the handler as an HTTP endpoint you can call via `curl`, the Restate UI, or any HTTP client.
2. **Persisting LLM responses**: Attach `RestateMiddleware()` to your agent so every LLM call is journaled and replayed during recovery. The middleware is provided via the LangChain extensions in the Restate SDK.
3. **Resilient tool execution**: Wrap durable side effects in tools with `restate_context().run_typed()`. The middleware does not auto-journal tool calls — you choose which steps are durable.

</GlobalTab>
<GlobalTab title="Restate TS">

Expand Down Expand Up @@ -463,7 +528,7 @@ async def run(ctx: restate.Context, message: WeatherPrompt) -> str | None:
# Call the LLM
async def call_llm() -> Message:
resp = await acompletion(
model="gpt-4o-mini", messages=messages, tools=TOOLS
model="gpt-5.4", messages=messages, tools=TOOLS
)
return resp.choices[0].message

Expand Down Expand Up @@ -521,6 +586,11 @@ The Restate UI (`http://localhost:9070`) shows the step-by-step execution trace
<img src="/img/tour/agents/pydantic/weather-agent.png" alt="Agent execution trace in Restate UI" />
</Frame>
</GlobalTab>
<GlobalTab title="LangChain">
<Frame>
<img src="/img/tour/agents/weather-agent.png" alt="Agent execution trace in Restate UI" />
</Frame>
</GlobalTab>
<GlobalTab title="Restate TS">
<Frame>
<img src="/img/tour/agents/weather-agent.png" alt="Agent execution trace in Restate UI" />
Expand Down
Loading
Loading