Staging

[paullizer]

Feature Release

(v0.238.025)

New Features

• Conversation Export

  • Export one or multiple conversations from the Chat page in JSON or Markdown format.
  • Single Export: Use the ellipsis menu on any conversation to quickly export it.
  • Multi-Export: Enter selection mode, check the conversations you want, and click the export button.
  • A guided 4-step wizard walks you through selection review, format choice, packaging options (single file or ZIP archive), and download.
  • Sensitive internal metadata is automatically stripped from exported data for security.

• Retention Policy UI for Groups and Public Workspaces

  • Can now configure conversation and document retention periods directly from the workspace and group management page.
  • Choose from preset retention periods ranging from 7 days to 10 years, use the organization default, or disable automatic deletion entirely.

• Owner-Only Group Agent and Action Management

  • New admin setting to restrict group agent and group action management (create, edit, delete) to only the group Owner role.
  • Admin Toggle: "Require Owner to Manage Group Agents and Actions" located in Admin Settings > My Groups section, under the existing group creation membership setting.
  • Default Off: When disabled, both Owner and Admin roles can manage group agents and actions (preserving existing behavior).
  • When Enabled: Only the group Owner can create, edit, and delete group agents and group actions. Group Admins and other roles are restricted to read-only access.
  • Backend Enforcement: Server-side validation returns 403 for non-Owner users attempting create, update, or delete operations on group agents and actions.
  • Frontend Enforcement: "New Agent" and "New Action" buttons are hidden, edit/delete controls are removed, and a permission warning is displayed for non-Owner users.
  • Files Modified: functions_settings.py, admin_settings.html, route_frontend_admin_settings.py, route_backend_agents.py, route_backend_plugins.py, group_workspaces.html, group_agents.js, group_plugins.js.
  • (Ref: require_owner_for_group_agent_management setting, assert_group_role permission check)

• Enforce Workspace Scope Lock

  • New admin setting to control whether users can unlock workspace scope in chat conversations.
  • Enabled by Default: When enabled, workspace scope automatically locks after the first AI search and users cannot unlock it, preventing accidental cross-contamination between data sources.
  • Informational Modal: Users can still click the lock icon to view which workspaces are locked, but the "Unlock Scope" button is hidden and replaced with an informational message.
  • Backend Enforcement: Server-side validation rejects unlock API requests when the setting is enabled, providing defense-in-depth security.
  • Admin Toggle: Located in Admin Settings > Workspace tab in the new "Workspace Scope Lock" section.
  • Files Modified: config.py, functions_settings.py, route_frontend_admin_settings.py, admin_settings.html, chats.html, chat-documents.js, route_backend_conversations.py.
  • (Ref: ENFORCE_WORKSPACE_SCOPE_LOCK.md)

• Blob Metadata Tag Propagation

  • Document tags now propagate to Azure Blob Storage metadata when enhanced citations is enabled.
  • Automatic Sync: When tags are added, removed, or updated on a document, the corresponding blob's metadata is updated with a document_tags field containing a comma-separated list of tags.
  • Conditional: Only active when enable_enhanced_citations is enabled in admin settings; no blob metadata changes occur otherwise.
  • Cross-Workspace: Works for personal, group, and public workspace documents.
  • Non-Blocking: Blob metadata update failures are logged but do not prevent the primary tag propagation to AI Search chunks.
  • Files Modified: functions_documents.py.
  • (Ref: BLOB_METADATA_TAG_PROPAGATION.md)

• Document Tag System

  • Comprehensive tag management system for organizing documents across personal, group, and public workspaces.
  • Tag Definitions: Tags with custom colors from a 10-color default palette (blue, green, amber, red, purple, pink, cyan, lime, orange, indigo) or user-specified hex codes. Colors assigned deterministically via character-sum hash.
  • Full CRUD API: 15 endpoints (5 per workspace type) for listing, creating, bulk tagging, renaming/recoloring, and deleting tags. Consistent API pattern across /api/documents/tags, /api/group_documents/<id>/tags, and /api/public_workspace_documents/<id>/tags.
  • Bulk Tag Operations: Apply, remove, or replace tags on multiple documents in a single operation with per-document success/error reporting.
  • AI Search Integration: Tags propagate to all document chunks via propagate_tags_to_chunks(), enabling OData tag filtering during hybrid search with AND logic (document_tags/any(t: t eq 'tag')).
  • Tag Validation: Max 50 characters, alphanumeric plus hyphens/underscores only, normalized to lowercase, duplicates silently deduplicated.
  • Tag Storage: Personal tags in user settings, group tags on group Cosmos document, public workspace tags on workspace Cosmos document.
  • Files Modified: functions_documents.py, functions_search.py, route_backend_documents.py, route_backend_group_documents.py, route_backend_public_documents.py.
  • Files Added: static/json/ai_search-index-user.json, static/json/ai_search-index-group.json, static/json/ai_search-index-public.json.
  • (Ref: Document Tag System, AI Search OData filtering, cross-workspace tags, DOCUMENT_TAG_SYSTEM.md)

• Workspace Folder View (Grid View)

  • Toggle between traditional list view and folder-based grid view for workspace documents via radio buttons.
  • Tag Folders: Color-coded folder cards displaying tag name, document count, folder icon, and context menu (rename, recolor, delete).
  • Special Folders: "Untagged" folder for documents with no tags and "Unclassified" folder for documents without classification (when classification is enabled).
  • Folder Drill-Down: Click a folder to view its contents with breadcrumb navigation, in-folder search, configurable page sizes (10, 20, 50), and sort by filename or title.
  • Grid Sort Controls: Sort folder overview by name or file count with ascending/descending toggle.
  • View Persistence: Selected view preference saved to localStorage and restored on page load.
  • Tag Management Modal: Step-through workflow for creating, editing, renaming, recoloring, and deleting tags with color picker.
  • Cross-Workspace Support: Equivalent grid view and tag management available in group workspaces (inline JS) and public workspaces.
  • Files Added: workspace-tags.js (1257 lines), workspace-tag-management.js (732 lines).
  • Files Modified: workspace.html, group_workspaces.html, public_workspaces.html, public_workspace.js.
  • (Ref: Folder view, tag management modal, grid rendering, WORKSPACE_FOLDER_VIEW.md)

• Multi-Workspace Scope Management

  • Select from Personal, multiple Group, and multiple Public workspaces simultaneously in the chat interface.
  • Hierarchical Scope Dropdown: Organized sections with checkbox multi-selection and "Select All / Clear All" toggle with indeterminate state support.
  • Scope Locking: Per-conversation lock that freezes workspace selection after the first AI Search. Three-state machine: null (auto-lockable) → true (locked) → false (user-unlocked) → true (re-lockable).
  • Lock Indicator: Visual lock icon with tooltip showing locked workspace names. Locked workspaces appear grayed out in the dropdown.
  • Lock/Unlock Modal: Dialog for manually toggling scope lock per conversation.
  • Lock Persistence: Lock state stored in conversation metadata via PATCH /api/conversations/<id>/scope_lock.
  • Workspace Search Container: Multi-column flex layout (Scope → Tags → Documents) with connected card UI and viewport boundary detection.
  • Files Modified: chat-documents.js, chat-messages.js, chats.html, route_backend_chats.py, route_backend_conversations.py.
  • (Ref: Multi-workspace selection, scope locking, search container layout, MULTI_WORKSPACE_SCOPE_MANAGEMENT.md)

• Chat Document and Tag Filtering

  • Checkbox-based multi-document selection replacing the legacy single-document dropdown in the chat interface.
  • Custom Document Dropdown: Checkboxes for each document with real-time search, "All Documents" option, and selected count display ("3 Documents").
  • Scope Indicators: Each document labeled with its source workspace: [Personal], [Group: Name], or [Public: Name].
  • Multi-Tag Filtering: Checkbox dropdown for selecting tags to filter the document list. Classification categories shown with color coding when enabled.
  • Dynamic Tag Loading: Tags load and merge across all selected scope workspaces with aggregated counts.
  • DOM-Based Filtering: Non-matching documents removed from the DOM (not hidden via CSS), following project conventions. Removed items stored for restoration when filters change.
  • Backend Integration: Selected document IDs and tags sent in chat request body. Backend constructs OData AND filter: document_tags/any(t: t eq 'tag1') and document_tags/any(t: t eq 'tag2').
  • Files Modified: chat-documents.js, chat-messages.js, functions_search.py, route_backend_chats.py, chats.html.
  • (Ref: Multi-document selection, tag filtering, OData search integration, CHAT_DOCUMENT_AND_TAG_FILTERING.md)

Bug Fixes

• Public Workspace setActive 403 Fix
  • Fixed issue where non-owner/admin/document-manager users received a 403 "Not a member" error when trying to activate a public workspace for chat.
  • Root cause was an overly restrictive membership check on the /api/public_workspaces/setActive endpoint that only allowed owners, admins, and document managers — even though public workspaces are intended to be accessible to all authenticated users for chatting.
  • Removed the membership verification from the setActive endpoint; the route still requires authentication (@login_required, @user_required) and the public workspaces feature flag (@enabled_required).
  • Other admin-level endpoints (listing members, viewing stats, ownership transfer) retain their membership checks.
  • (Ref: route_backend_public_workspaces.py, api_set_active_public_workspace)
• Chats Page User Settings Hardening
  • Fixed a user-specific chats page failure where only one affected user could not load /chats due to malformed per-user settings data.
  • Root Cause: The chats route assumed user_settings["settings"] was always a dictionary. If that field existed but had an invalid type (for example string, null, or list), the page could fail before rendering.
  • Solution: Hardened get_user_settings() to normalize missing/malformed settings to {} and persist the repaired document. Hardened the chats route to use safe dictionary fallbacks when reading nested settings values.
  • Telemetry: Added repair logging ([UserSettings] Malformed settings repaired) to improve diagnostics for future user-specific data-shape issues.
  • Files Modified: functions_settings.py, route_frontend_chats.py, config.py.
  • Files Added: test_chats_user_settings_hardening_fix.py, CHATS_USER_SETTINGS_HARDENING_FIX.md.
  • (Ref: user settings normalization, /chats route resilience, functional_tests/test_chats_user_settings_hardening_fix.py, docs/explanation/fixes/CHATS_USER_SETTINGS_HARDENING_FIX.md)
• Tag Filter Input Sanitization (Injection Prevention)
  • Added sanitize_tags_for_filter() function to validate tag filter inputs against the same ^[a-z0-9_-]+$ character whitelist enforced when saving tags.
  • Previously, tag filter values from query parameters only passed through normalize_tag() (strip + lowercase) without character validation, allowing arbitrary characters to reach OData filter construction in build_tags_filter().
  • Hardened build_tags_filter() in functions_search.py to validate tags before interpolating into OData expressions, eliminating the OData injection vector.
  • Updated tag filter parsing in personal, group, and public document routes to use sanitize_tags_for_filter() for defense-in-depth.
  • Invalid tag filter values are silently dropped (they cannot match any stored tag).
  • Files Modified: functions_documents.py, functions_search.py, route_backend_documents.py, route_backend_group_documents.py, route_backend_public_documents.py.
  • (Ref: TAG_FILTER_INJECTION_FIX.md, sanitize_tags_for_filter)
• Citation Parsing Bug Fix
  • Fixed citation parsing edge cases where page range references (e.g., "Pages: 1-5") failed to generate correct clickable links when not all pages had explicit reference IDs in the bracketed citation section of the AI response.
  • Root Cause: The parseCitations() function only generated links for pages with existing [doc_prefix_N] bracket references, leaving pages without explicit references as non-functional text.
  • Solution: Added auto-fill logic using getDocPrefix() to extract the document ID prefix from known reference patterns and construct missing page references (e.g., if [doc_abc_1] exists, infer doc_abc_2 through doc_abc_5).
  • Files Modified: chat-citations.js.
  • (Ref: Citation parsing, page range handling, CITATION_IMPROVEMENTS.md)

User Interface Enhancements

• Extended Document Dropdown Width

  • Widened the document selection dropdown in the chat interface for improved readability of long filenames. Dropdown width now dynamically adapts to the parent container.
  • Files Modified: chat-documents.js.
  • (Ref: Document dropdown, UI readability)

• Enhanced Citation Links

  • Robust inline citation links with support for both inline source references and hybrid citation buttons.
  • Metadata citation support for viewing extracted document metadata including OCR text, vision analysis, and detected objects via the enhanced citation modal.
  • Improved error handling in citation JSON parsing with graceful fallback for malformed citation strings.
  • Files Modified: chat-citations.js, chat-enhanced-citations.js.
  • (Ref: Citation rendering, metadata citations, enhanced citation modal, CITATION_IMPROVEMENTS.md)