From f91515da36941cd4b7ae6b188ac192a2e7017c3e Mon Sep 17 00:00:00 2001 From: lioZ129 <1907814837@qq.com> Date: Tue, 7 Apr 2026 17:04:24 +0800 Subject: [PATCH] docs: add HPC-AI chat generator docs --- .../docs/pipeline-components/generators.mdx | 1 + .../generators/hpcaichatgenerator.mdx | 141 ++++++++++++++++++ docs-website/sidebars.js | 1 + 3 files changed, 143 insertions(+) create mode 100644 docs-website/docs/pipeline-components/generators/hpcaichatgenerator.mdx diff --git a/docs-website/docs/pipeline-components/generators.mdx b/docs-website/docs/pipeline-components/generators.mdx index f49778444f..6e70f73103 100644 --- a/docs-website/docs/pipeline-components/generators.mdx +++ b/docs-website/docs/pipeline-components/generators.mdx @@ -28,6 +28,7 @@ Generators are responsible for generating text after you give them a prompt. The | [GoogleAIGeminiChatGenerator](generators/googleaigeminichatgenerator.mdx) | Enables chat completion using Google Gemini models. **_This integration will be deprecated soon. We recommend using [GoogleGenAIChatGenerator](generators/googlegenaichatgenerator.mdx) integration instead._** | ✅ | | [GoogleAIGeminiGenerator](generators/googleaigeminigenerator.mdx) | Enables text generation using Google Gemini models. **_This integration will be deprecated soon. We recommend using [GoogleGenAIChatGenerator](generators/googlegenaichatgenerator.mdx) integration instead._** | ✅ | | [GoogleGenAIChatGenerator](generators/googlegenaichatgenerator.mdx) | Enables chat completion using Google Gemini models through Google Gen AI SDK. | ✅ | +| [HPCAIChatGenerator](generators/hpcaichatgenerator.mdx) | Enables chat completion using HPC-AI's OpenAI-compatible API. | ✅ | | [HuggingFaceAPIChatGenerator](generators/huggingfaceapichatgenerator.mdx) | Enables chat completion using various Hugging Face APIs. | ✅ | | [HuggingFaceAPIGenerator](generators/huggingfaceapigenerator.mdx) | Enables text generation using various Hugging Face APIs. | ✅ | | [HuggingFaceLocalChatGenerator](generators/huggingfacelocalchatgenerator.mdx) | Provides an interface for chat completion using a Hugging Face model that runs locally. | ✅ | diff --git a/docs-website/docs/pipeline-components/generators/hpcaichatgenerator.mdx b/docs-website/docs/pipeline-components/generators/hpcaichatgenerator.mdx new file mode 100644 index 0000000000..c73e6b6e78 --- /dev/null +++ b/docs-website/docs/pipeline-components/generators/hpcaichatgenerator.mdx @@ -0,0 +1,141 @@ +--- +title: "HPCAIChatGenerator" +id: hpcaichatgenerator +slug: "/hpcaichatgenerator" +description: "HPCAIChatGenerator enables chat completion using HPC-AI's OpenAI-compatible API." +--- + +# HPCAIChatGenerator + +`HPCAIChatGenerator` enables chat completion using HPC-AI's OpenAI-compatible API. + +
+ +| | | +| --- | --- | +| **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) | +| **Mandatory init variables** | `api_key`: The HPC-AI API key. Can be set with `HPC_AI_API_KEY` env var. | +| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | +| **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects

`meta`: A list of dictionaries with the metadata associated with each reply, such as token count and finish reason | +| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/hpc_ai | + +
+ +## Overview + +`HPCAIChatGenerator` reuses Haystack's OpenAI chat generation logic and points it at HPC-AI's compatible base URL: `https://api.hpc-ai.com/inference/v1`. + +This integration officially supports these models: + +- `minimax/minimax-m2.5` (default) +- `moonshotai/kimi-k2.5` + +You can provide credentials with: + +- The `HPC_AI_API_KEY` environment variable (recommended) +- The `api_key` init parameter and Haystack [Secret](../../concepts/secret-management.mdx) API + +If you need to override the endpoint, set `HPC_AI_BASE_URL` or pass `api_base_url` explicitly. + +### Structured Output + +`HPCAIChatGenerator` supports structured output generation for compatible models through `generation_kwargs["response_format"]`. You can pass either a Pydantic model or a JSON schema, just like with Haystack's OpenAI-compatible generators. + +### Tool Support + +`HPCAIChatGenerator` supports function calling through the `tools` parameter. You can pass a list of Tool objects, a Toolset, or a mix of both. + +### Streaming + +`HPCAIChatGenerator` supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) tokens through the `streaming_callback` init parameter. + +## Usage + +Install the `hpc-ai-haystack` package: + +```shell +pip install hpc-ai-haystack +``` + +### On its own + +```python +from haystack.dataclasses import ChatMessage +from haystack_integrations.components.generators.hpc_ai import HPCAIChatGenerator + +client = HPCAIChatGenerator() +response = client.run([ChatMessage.from_user("What are agentic pipelines? Be brief.")]) + +print(response["replies"][0].text) +``` + +With structured output: + +```python +from pydantic import BaseModel +from haystack.dataclasses import ChatMessage +from haystack_integrations.components.generators.hpc_ai import HPCAIChatGenerator + + +class CityInfo(BaseModel): + city_name: str + country: str + + +client = HPCAIChatGenerator( + model="moonshotai/kimi-k2.5", + generation_kwargs={"response_format": CityInfo}, +) + +response = client.run([ + ChatMessage.from_user("Return the capital city and country for Germany.") +]) + +print(response["replies"][0].text) +``` + +With streaming: + +```python +from haystack.components.generators.utils import print_streaming_chunk +from haystack.dataclasses import ChatMessage +from haystack_integrations.components.generators.hpc_ai import HPCAIChatGenerator + +client = HPCAIChatGenerator( + model="moonshotai/kimi-k2.5", + streaming_callback=print_streaming_chunk, +) + +client.run([ChatMessage.from_user("Summarize RAG in two lines.")]) +``` + +### In a Pipeline + +```python +from haystack import Pipeline +from haystack.components.builders import ChatPromptBuilder +from haystack.dataclasses import ChatMessage +from haystack_integrations.components.generators.hpc_ai import HPCAIChatGenerator + +prompt_builder = ChatPromptBuilder() +llm = HPCAIChatGenerator() + +pipe = Pipeline() +pipe.add_component("prompt_builder", prompt_builder) +pipe.add_component("llm", llm) +pipe.connect("prompt_builder.prompt", "llm.messages") + +messages = [ + ChatMessage.from_system("Always respond concisely."), + ChatMessage.from_user("Tell me about {{topic}}"), +] + +pipe.run( + data={ + "prompt_builder": { + "template_variables": {"topic": "Haystack"}, + "template": messages, + } + } +) +``` diff --git a/docs-website/sidebars.js b/docs-website/sidebars.js index df530938bc..4f4885a253 100644 --- a/docs-website/sidebars.js +++ b/docs-website/sidebars.js @@ -400,6 +400,7 @@ export default { 'pipeline-components/generators/googleaigeminichatgenerator', 'pipeline-components/generators/googleaigeminigenerator', 'pipeline-components/generators/googlegenaichatgenerator', + 'pipeline-components/generators/hpcaichatgenerator', 'pipeline-components/generators/huggingfaceapichatgenerator', 'pipeline-components/generators/huggingfaceapigenerator', 'pipeline-components/generators/huggingfacelocalchatgenerator',