Add an API Endpoint Node

[RoyHEyono]

Chainforge should add an API Endpoint Node to ChainForge. This will allow users to prompt engineer and systematically battletest prompts against their own LLM-integrated systems via API calls instead of a LLM provider like OpenAI, and Claude.

---

Potential Capabilities:

• Send prompts directly to external APIs.
• Evaluate API responses within ChainForge workflows.
• Enable automated testing flows callable via the ChainForge CLI

[ianarawjo]

Hi there, this should be modified to not be an independent node but a Custom Provider.

[RoyHEyono]

Although it could achieve the same thing, from a design perspective I believe it introduces several hidden affordances:

• Custom Provider is a misleading term
• Have to go to settings to activate this feature
• Chaining several API responses is a lot more intuitive with the node design pattern

If we were to leverage the custom provider, would that mean that we would need to add the endpoint on the prompt node? Would the prompt template, now act as a POST/GET template?

[ianarawjo]

The endpoint as a separate node makes it no longer possible to compare with other providers.

You can make it less “hidden” in three ways: adding a special menu item to the Add button like + Add Endpoint, renaming Model to Provider in the UI, and adding a docs page.

Really, this should be dealt with as adding a Custom Provider that persists across sessions.

[ianarawjo]

If you dislike Custom Provider as a term, you could rename it. However this term is now used by other APIs/tools too (not just us). Par exemple: https://www.promptfoo.dev/docs/providers/