community: integration of kluster.ai with tests, examples and documentation

[troels-im]

How to test

To test, checkout out the repo and from the root directory, run the following commands:

uv sync
cd libs/community
uv sync --group test
cd ../../
uv run --group test jupyter notebook

Then navigate to docs/docs/integrations and check out the examples in chat and llms.

LangChain PR template

Thank you for contributing to LangChain!

• [v] PR title: "community: integration of kluster.ai with tests, examples and documentation"

  • Where "package" is whichever of langchain, community, core, etc. is being modified. Use "docs: ..." for purely docs changes, "infra: ..." for CI changes.
  • Example: "community: add foobar LLM"

• PR message: Delete this entire checklist and replace with

  • Description: a description of the change
  • Issue: the issue # it fixes, if applicable
  • Dependencies: any dependencies required for this change
  • Twitter handle: if your PR gets announced, and you'd like a mention, we'll gladly shout you out!

1. **Introduction of KlusterAi Notebooks**:
   - Two new Jupyter Notebooks have been created:
     - `docs/docs/integrations/chat/klusterai.ipynb`: A detailed guide on how to use LangChain with KlusterAi's chat models.
     - `docs/docs/integrations/llms/klusterai.ipynb`: Instructions on leveraging KlusterAi's language models with LangChain.

2. **Documentation Updates**:
   - Added a new Markdown document (`docs/docs/integrations/providers/klusterai.mdx`) describing the KlusterAi service, including its setup, installation, and available models.

3. **Implementation of ChatKlusterAi Model**:
   - A new class `ChatKlusterAi` is wrapped around the KlusterAi API for chat functionalities. It allows for various messaging formats, system messages, and tool calls.
   - Implements both synchronous and asynchronous invocation of chat models.

4. **Implementation of KlusterAi Model**:
   - A new class `KlusterAi` has been added, which facilitates access to language models offered by KlusterAi service.
   - Includes error handling and supports streaming responses.

5. **Integration Testing**:
   - Comprehensive integration tests have been added for both `ChatKlusterAi` and `KlusterAi` classes to ensure functionality under various conditions.

6. **Unit Testing**:
   - New unit tests validate the behavior of the KlusterAi models, ensuring model parameters are correctly set and functionality operates as expected.

issue: n/a
dependencies: no new dependencies
twitter handle: klusterai

• [v] Add tests and docs: If you're adding a new integration, please include

  1. a test for the integration, preferably unit tests that do not rely on network access (TODO),
  2. an example notebook showing its use. It lives in docs/docs/integrations directory.

• Lint and test: Run (v) make format, make lint and make test from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/

Additional guidelines:

• Make sure optional dependencies are imported within a function.
• Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests.
• Most PRs should not touch more than one package.
• Changes should be backwards compatible.
• If you are adding something to community, do not re-import it in langchain.

If no one reviews your PR within a few days, please @-mention one of baskaryan, eyurtsev, ccurme, vbarda, hwchase17.

[rymc]

is KlusterAi (lowercase 'i') the standard? Looks a little odd to me that it's not "AI"..

[troels-im]

is KlusterAi (lowercase 'i') the standard? Looks a little odd to me that it's not "AI"..
Yeah, you are right. it's updated to KlusterAI

[themacexpert]

I noticed that the ipynb has a slightly different introduction to Kluster than the LLMS file...

KlusterAI is a serverless inference service that provides access to a variety of LLMs. This notebook goes over how to use LangChain with KlusterAI for language models."

vs

kluster.ai is a inference service that provides access to a variety of high-performance LLMs including Meta's Llama 3.1 and Llama 3.3 models. This notebook goes over how to use LangChain with kluster.ai for chat models.

I also wanted to bring up a fresh intro that Anjin put together (feel free to truncate) etc but I like the first sentence of this intro a bit more as it feels more recent / descriptive

kluster.ai is a developer-focused AI platform offering a versatile and growing suite of powerful models, including Llama 3.3, 3.1, DeepSeek-R1, DeepSeekV3, Gemma 3, and Qwen2.5, with new models being added regularly to stay at the forefront of AI innovation. kluster.ai was the first platform to host DeepSeek-R1, highlighting its commitment to delivering cutting-edge tools for developers. With seamless fine-tuning capabilities, developers can customize models to fit their specific project requirements, ensuring optimal performance while maintaining cost efficiency. By integrating kluster.ai, open-source creators gain access to state-of-the-art AI tools and developer-friendly solutions, empowering them to enhance their projects with precision and flexibility.

[troels-im]

Thanks! I will update this - I haven't paid much attention to the copy, more to the functionality.

[albertov19]

Why model_id and not model? Found this from OpenAI and this from anthropic

Also HuggingFace