Problem
When users configure custom prompts in their GitHub repository, there's no validation to ensure the prompt output matches the format expected by the response parser.
This leads to "Summary could not be extracted from response" errors when the custom prompt returns a different JSON structure.
Example
Custom prompt returned:
{
"🎯 Overview": "...",
"💡 Key Technical Points": [...],
"🔧 Action Items": [...]
}
Parser expected:
{
"summary": "...",
"key_points": [...],
"action_items": [...]
}
Proposed Solution
Add a validation step when loading custom prompts from GitHub:
- Schema Validator: Create a JSON schema for the expected output format
- Prompt Testing: When a custom prompt is loaded, include instructions about the required output format
- Validation Endpoint: Add a
/config validate-prompt command that tests the prompt with sample data
- Documentation: Update GitHub prompt template with clear schema requirements
Alternative
Make the parser more flexible to handle multiple JSON formats (backward compatible approach).
Priority
Medium - affects users who want to customize prompts but creates poor UX when format doesn't match
Problem
When users configure custom prompts in their GitHub repository, there's no validation to ensure the prompt output matches the format expected by the response parser.
This leads to "Summary could not be extracted from response" errors when the custom prompt returns a different JSON structure.
Example
Custom prompt returned:
{ "🎯 Overview": "...", "💡 Key Technical Points": [...], "🔧 Action Items": [...] }Parser expected:
{ "summary": "...", "key_points": [...], "action_items": [...] }Proposed Solution
Add a validation step when loading custom prompts from GitHub:
/config validate-promptcommand that tests the prompt with sample dataAlternative
Make the parser more flexible to handle multiple JSON formats (backward compatible approach).
Priority
Medium - affects users who want to customize prompts but creates poor UX when format doesn't match