Promptulate V2 Proposal

[Undertone0809]

Proposal: Promptulate v2 路线图与功能规划

Promptulate v2 旨在通过模块化、易用性和强大的工程最佳实践实现框架的重大升级。本路线图详细列出了关键改进和新功能，以确保 Promptulate 始终是 LLM 应用开发的领先解决方案，让开发者可以用最低的成本上手大模型应用开发。

---

核心改进

1. 移除冗余模块

   • 简化框架，去除不必要的模型相关组件。
   • 聚焦于 LLM、Agent、Tool 和 RAG 集成的核心模块。

2. 更强的数据类型推断

   • 使用 pydantic v2，增强类型验证和推断。
   • 提升类型安全性和开发体验，使集成更直观。
   • 不再兼容 pydantic v1

3. 优化 Tool 模块

   • 重构 Tool 模块，提供更好的可用性和灵活性。
   • 支持 音频模型 和 视觉模型 的集成。

4. Model Factory

   • 引入统一的 Model Factory，标准化模型管理和初始化流程。
   • 支持各种 LLM 的无缝扩展和集成。

5. 优化函数调用与 Tool 调用兼容性

   • 重构函数和工具调用逻辑，提高原生兼容性。
   • 确保与外部库和 API 的流畅交互。

6. 重构 LLM 逻辑

   • 重新设计 LLM 核心工作流，提高模块化和性能。
   • 优化响应解析和错误处理机制。
   • 更方便的使用 function call 的逻辑

7. Hooks 系统重构

   • 重建 hooks 架构，提供更灵活的定制化能力。
   • 参考 langchain callback handler 的设计理念来设计
     • Token Counting Handler，ref: https://docs.llamaindex.ai/en/stable/module_guides/examples/callbacks/TokenCountingHandler.ipynb

8. 优化重试机制

   • 增强重试逻辑，引入指数退避和自定义异常处理。
   • 降低高负载或网络不稳定场景下的失败率。

9. mcp 协议兼容

   • doc: https://modelcontextprotocol.io/introduction

---

生态系统改进

1. 集成 Zeeland-Core

   • 基于轻量级的 zeeland-core 库，实现高性能和一致性。

2. 与 LangChain 的无缝衔接

   • 确保与 LangChain 的兼容性，支持链式工作流构建。
   • 便于开发者迁移或结合 LangChain 使用。

3. Agent 模块重构

   • 重建 Agent 模块，符合 v2 精简架构。
   • 增强模块化，支持多 Agent 协作，引入过去 Cogit 设计的 Mulit-Agent 自助编排与任务调度框架。
   • 这一块还需要进一步讨论，当前的想法是基于 LangChain Agent Protocol 的生态进行 Agent Infra 的构建。

4. 本地模型密钥管理

   • 目的：方便本地开发调试与管理，不鼓励在生产中使用。
   • 引入安全的 Model Manager，支持本地存储和检索模型密钥。
   • 调研并借鉴 LlamaIndex 的 Model Manager 实践。
   • 将配置存入 ~/.zeeland/pne 下

5. 多种 Prompting 范式支持

   • 增加多种 Prompting 样式的支持，配套 回调函数 实现运行时自定义。

6. RAG 支持

   • 通过 pip install pne[rag] 实现模块化 RAG 集成，按需使用。
   • 简化配置，让 RAG 快速上手。

---

开发者体验

1. 基于 StackBlitz 的 Streamlit 项目体验

   • 让开发者可以快速使用 Streamlit 和 StackBlitz 构建 PNE 驱动的原型项目。

2. 精炼示例项目

   • 优化示例项目，聚焦常见的真实使用场景。
   • 提供从最小化到高级配置的逐步指导。

3. 极致的文档支持

   • 注重 know-why，不仅仅是 know-how，帮助开发者深入理解最佳实践。
   • 强调工具、Agent 和工作流设计的实用原则。
   • 通过文档赋能开发者解决实际问题，提供可操作的深度指导。

4. 代码生成器（待定）

   • 可选地集成一个 代码生成器，帮助用户快速生成项目模板和 Prompting 范式代码。
   • 支持快速原型开发，减少配置和学习成本。

---

技术优化

1. 依赖精简

   • 减少不必要的依赖，降低框架体积和安装开销。
   • 专注于轻量化和高性能的库。

2. 增强 Trace 和日志支持

   • 添加内置追踪机制，便于调试和性能分析。
   • 支持工具、Agent 和链式工作流的执行追踪。
   • OpenTelemetry 协议集成

3. 分离 PNE Client 库

   • 将 PNE Client 独立为一个库：pip install pne[client]。
   • 允许用户按需选择模块。

4. 重构项目结构

• 使用 poetry 进行多包管理，在 libs 目录下
• 管理 pne, promptulate, pne-chat, pne[client], pne[langchain] 等包

---

开放性问题

1. Agent 的必要性

   • Agent 是否应作为 PNE 核心部分，还是独立为可选模块？
   • 讨论 Agent 的依赖对生态系统的一致性影响。

2. 代码生成器的可行性

   • 评估加入代码生成器的价值，是否符合 PNE 简单灵活的设计理念。
   • 确定其是否能提升开发者体验。

3. Real time API 的设计

   • Realtime API： https://platform.openai.com/docs/guides/realtime

---

交付目标

• Promptulate v2：包含所有核心改进和生态系统优化，Agent 处于 P1，pne.chat 为 P0 核心功能。
• 模块化插件式架构，支持按需加载功能（如 RAG 和高级客户端）。
• 全面的文档、示例和入门资料。
• 卓越的开发者体验，强调清晰性、灵活性和最佳实践。

欢迎大家在此 Issue 中反馈意见和建议

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Proposal: Promptulate v2 roadmap and feature planning

Promptulate v2 aims to be a major upgrade to the framework through modularity, ease of use, and strong engineering best practices. This roadmap details key improvements and new features to ensure that Promptulate remains the leading solution for LLM application development, allowing developers to get started with large model application development at the lowest cost.

---

Core Improvements

1. Remove redundant modules

   • Simplify the framework and remove unnecessary model-related components.
   • Focus on core modules for LLM, Agent, Tool and RAG integration.

2. Stronger data type inference

   • Enhanced type validation and inference using pydantic v2.
   • Improve type safety and development experience, making integration more intuitive.
   • No longer compatible with pydantic v1

3. Optimize Tool module

   • Refactor the Tool module to provide better usability and flexibility.
   • Supports the integration of Audio Model and Visual Model.

4. Model Factory

   • Introducing a unified Model Factory to standardize model management and initialization processes.
   • Supports seamless extension and integration of various LLMs.

5. Optimize compatibility of function calls and Tool calls

   • Reconstruct function and tool calling logic to improve native compatibility.
   • Ensure smooth interaction with external libraries and APIs.

6. Refactor LLM logic

   • Redesigned LLM core workflow to improve modularity and performance.
   • Optimize response parsing and error handling mechanisms.

7. Hooks system reconstruction

   • Reconstruct the hooks architecture to provide more flexible customization capabilities.
   • Supports pre- and post-execution hooks for easy debugging and expansion.

8. Optimize retry mechanism

   • Enhanced retry logic, introduced exponential backoff and custom exception handling.
   • Reduce the failure rate under high load or unstable network scenarios.

---

Ecosystem Improvements

1. Integrated Zeeland-Core

   • Based on the lightweight zeeland-core library to achieve high performance and consistency.

2. Seamless connection with LangChain

   • Ensure compatibility with LangChain and support chain workflow construction.
   • Convenient for developers to migrate or use it in combination with LangChain.

3. Agent module reconstruction

   • Rebuilt Agent module to conform to v2 streamlined architecture.
   • Enhance modularity, support multi-Agent collaboration, and introduce the Mulit-Agent self-service orchestration and task scheduling framework designed by Cogit in the past.
   • This area needs further discussion. The current idea is to build Agent Infra based on the ecology of LangChain Agent Protocol.

4. Local model key management

   • Purpose: To facilitate local development, debugging and management, use in production is not encouraged.
   • Introducing a secure Model Manager that supports local storage and retrieval of model keys.
   • Research and learn from LlamaIndex’s Model Manager practices.
   • Save the configuration under ~/.zeeland/pne

5. Multiple Prompting paradigms supported

   • Add support for multiple Prompting styles, and support callback function to achieve runtime customization.

6. RAG Support

   • Modular RAG integration via pip install pne[rag], use on demand.
   • Simplified configuration to get started with RAG quickly.

---

Developer Experience

1. Streamlit project experience based on StackBlitz

   • Allow developers to quickly use Streamlit and StackBlitz to build PNE-driven prototype projects.

2. Refined example project

   • Optimize sample projects to focus on common real-life usage scenarios.
   • Provides step-by-step guidance from minimal to advanced configuration.

3. Extreme document support

   • Focus on know-why, not just know-how, to help developers deeply understand best practices.
   • Emphasis on practical principles of tool, agent, and workflow design.
   • Empower developers to solve practical problems through documentation and provide actionable and in-depth guidance.

4. Code Generator (TBD)

   • Optionally integrates a code generator to help users quickly generate project templates and Prompting paradigm codes.
   • Support rapid prototype development and reduce configuration and learning costs.

---

Technical Optimization

1. Dependency Streamlining

   • Reduce unnecessary dependencies, reduce framework size and installation overhead.
   • A library focused on lightweight and high performance.

2. Enhanced Trace and log support

   • Added built-in tracing mechanism to facilitate debugging and performance analysis.
   • Supports execution tracking of tools, agents, and chained workflows.

3. Separate PNE Client library

   • Separate PNE Client into a library: pip install pne[client].
   • Allow users to select modules on demand.

---

Open question

1. Necessity of Agent

   • Should Agent be a core part of PNE or standalone as an optional module?
   • Discuss the impact of Agent dependencies on the consistency of the ecosystem.

2. Feasibility of code generator

   • Evaluate the value of adding a code generator and whether it is consistent with PNE's simple and flexible design philosophy.
   • Determine if it improves the developer experience.

---

Delivery target

• Promptulate v2: Contains all core improvements and ecosystem optimizations, Agent is in P1, pne.chat is P0 core functionality.
• Modular plug-in architecture that supports on-demand loading of features (such as RAG and advanced clients).
• Comprehensive documentation, examples and getting started.
• A great developer experience that emphasizes clarity, flexibility, and best practices.

Feedback and suggestions are welcome in this Issue