Skip to content

v0.238.024 - Development into Staging#733

Merged
paullizer merged 134 commits intoStagingfrom
Development
Mar 2, 2026
Merged

v0.238.024 - Development into Staging#733
paullizer merged 134 commits intoStagingfrom
Development

Conversation

@paullizer
Copy link
Contributor

@paullizer paullizer commented Feb 23, 2026

Feature Release

(v0.238.025)

New Features

  • Owner-Only Group Agent and Action Management

    • New admin setting to restrict group agent and group action management (create, edit, delete) to only the group Owner role.
    • Admin Toggle: "Require Owner to Manage Group Agents and Actions" located in Admin Settings > My Groups section, under the existing group creation membership setting.
    • Default Off: When disabled, both Owner and Admin roles can manage group agents and actions (preserving existing behavior).
    • When Enabled: Only the group Owner can create, edit, and delete group agents and group actions. Group Admins and other roles are restricted to read-only access.
    • Backend Enforcement: Server-side validation returns 403 for non-Owner users attempting create, update, or delete operations on group agents and actions.
    • Frontend Enforcement: "New Agent" and "New Action" buttons are hidden, edit/delete controls are removed, and a permission warning is displayed for non-Owner users.
    • Files Modified: functions_settings.py, admin_settings.html, route_frontend_admin_settings.py, route_backend_agents.py, route_backend_plugins.py, group_workspaces.html, group_agents.js, group_plugins.js.
    • (Ref: require_owner_for_group_agent_management setting, assert_group_role permission check)
  • Enforce Workspace Scope Lock

    • New admin setting to control whether users can unlock workspace scope in chat conversations.
    • Enabled by Default: When enabled, workspace scope automatically locks after the first AI search and users cannot unlock it, preventing accidental cross-contamination between data sources.
    • Informational Modal: Users can still click the lock icon to view which workspaces are locked, but the "Unlock Scope" button is hidden and replaced with an informational message.
    • Backend Enforcement: Server-side validation rejects unlock API requests when the setting is enabled, providing defense-in-depth security.
    • Admin Toggle: Located in Admin Settings > Workspace tab in the new "Workspace Scope Lock" section.
    • Files Modified: config.py, functions_settings.py, route_frontend_admin_settings.py, admin_settings.html, chats.html, chat-documents.js, route_backend_conversations.py.
    • (Ref: ENFORCE_WORKSPACE_SCOPE_LOCK.md)
  • Blob Metadata Tag Propagation

    • Document tags now propagate to Azure Blob Storage metadata when enhanced citations is enabled.
    • Automatic Sync: When tags are added, removed, or updated on a document, the corresponding blob's metadata is updated with a document_tags field containing a comma-separated list of tags.
    • Conditional: Only active when enable_enhanced_citations is enabled in admin settings; no blob metadata changes occur otherwise.
    • Cross-Workspace: Works for personal, group, and public workspace documents.
    • Non-Blocking: Blob metadata update failures are logged but do not prevent the primary tag propagation to AI Search chunks.
    • Files Modified: functions_documents.py.
    • (Ref: BLOB_METADATA_TAG_PROPAGATION.md)
  • Document Tag System

    • Comprehensive tag management system for organizing documents across personal, group, and public workspaces.
    • Tag Definitions: Tags with custom colors from a 10-color default palette (blue, green, amber, red, purple, pink, cyan, lime, orange, indigo) or user-specified hex codes. Colors assigned deterministically via character-sum hash.
    • Full CRUD API: 15 endpoints (5 per workspace type) for listing, creating, bulk tagging, renaming/recoloring, and deleting tags. Consistent API pattern across /api/documents/tags, /api/group_documents/<id>/tags, and /api/public_workspace_documents/<id>/tags.
    • Bulk Tag Operations: Apply, remove, or replace tags on multiple documents in a single operation with per-document success/error reporting.
    • AI Search Integration: Tags propagate to all document chunks via propagate_tags_to_chunks(), enabling OData tag filtering during hybrid search with AND logic (document_tags/any(t: t eq 'tag')).
    • Tag Validation: Max 50 characters, alphanumeric plus hyphens/underscores only, normalized to lowercase, duplicates silently deduplicated.
    • Tag Storage: Personal tags in user settings, group tags on group Cosmos document, public workspace tags on workspace Cosmos document.
    • Files Modified: functions_documents.py, functions_search.py, route_backend_documents.py, route_backend_group_documents.py, route_backend_public_documents.py.
    • Files Added: static/json/ai_search-index-user.json, static/json/ai_search-index-group.json, static/json/ai_search-index-public.json.
    • (Ref: Document Tag System, AI Search OData filtering, cross-workspace tags, DOCUMENT_TAG_SYSTEM.md)
  • Workspace Folder View (Grid View)

    • Toggle between traditional list view and folder-based grid view for workspace documents via radio buttons.
    • Tag Folders: Color-coded folder cards displaying tag name, document count, folder icon, and context menu (rename, recolor, delete).
    • Special Folders: "Untagged" folder for documents with no tags and "Unclassified" folder for documents without classification (when classification is enabled).
    • Folder Drill-Down: Click a folder to view its contents with breadcrumb navigation, in-folder search, configurable page sizes (10, 20, 50), and sort by filename or title.
    • Grid Sort Controls: Sort folder overview by name or file count with ascending/descending toggle.
    • View Persistence: Selected view preference saved to localStorage and restored on page load.
    • Tag Management Modal: Step-through workflow for creating, editing, renaming, recoloring, and deleting tags with color picker.
    • Cross-Workspace Support: Equivalent grid view and tag management available in group workspaces (inline JS) and public workspaces.
    • Files Added: workspace-tags.js (1257 lines), workspace-tag-management.js (732 lines).
    • Files Modified: workspace.html, group_workspaces.html, public_workspaces.html, public_workspace.js.
    • (Ref: Folder view, tag management modal, grid rendering, WORKSPACE_FOLDER_VIEW.md)
  • Multi-Workspace Scope Management

    • Select from Personal, multiple Group, and multiple Public workspaces simultaneously in the chat interface.
    • Hierarchical Scope Dropdown: Organized sections with checkbox multi-selection and "Select All / Clear All" toggle with indeterminate state support.
    • Scope Locking: Per-conversation lock that freezes workspace selection after the first AI Search. Three-state machine: null (auto-lockable) → true (locked) → false (user-unlocked) → true (re-lockable).
    • Lock Indicator: Visual lock icon with tooltip showing locked workspace names. Locked workspaces appear grayed out in the dropdown.
    • Lock/Unlock Modal: Dialog for manually toggling scope lock per conversation.
    • Lock Persistence: Lock state stored in conversation metadata via PATCH /api/conversations/<id>/scope_lock.
    • Workspace Search Container: Multi-column flex layout (Scope → Tags → Documents) with connected card UI and viewport boundary detection.
    • Files Modified: chat-documents.js, chat-messages.js, chats.html, route_backend_chats.py, route_backend_conversations.py.
    • (Ref: Multi-workspace selection, scope locking, search container layout, MULTI_WORKSPACE_SCOPE_MANAGEMENT.md)
  • Chat Document and Tag Filtering

    • Checkbox-based multi-document selection replacing the legacy single-document dropdown in the chat interface.
    • Custom Document Dropdown: Checkboxes for each document with real-time search, "All Documents" option, and selected count display ("3 Documents").
    • Scope Indicators: Each document labeled with its source workspace: [Personal], [Group: Name], or [Public: Name].
    • Multi-Tag Filtering: Checkbox dropdown for selecting tags to filter the document list. Classification categories shown with color coding when enabled.
    • Dynamic Tag Loading: Tags load and merge across all selected scope workspaces with aggregated counts.
    • DOM-Based Filtering: Non-matching documents removed from the DOM (not hidden via CSS), following project conventions. Removed items stored for restoration when filters change.
    • Backend Integration: Selected document IDs and tags sent in chat request body. Backend constructs OData AND filter: document_tags/any(t: t eq 'tag1') and document_tags/any(t: t eq 'tag2').
    • Files Modified: chat-documents.js, chat-messages.js, functions_search.py, route_backend_chats.py, chats.html.
    • (Ref: Multi-document selection, tag filtering, OData search integration, CHAT_DOCUMENT_AND_TAG_FILTERING.md)

Bug Fixes

  • Public Workspace setActive 403 Fix
    • Fixed issue where non-owner/admin/document-manager users received a 403 "Not a member" error when trying to activate a public workspace for chat.
    • Root cause was an overly restrictive membership check on the /api/public_workspaces/setActive endpoint that only allowed owners, admins, and document managers — even though public workspaces are intended to be accessible to all authenticated users for chatting.
    • Removed the membership verification from the setActive endpoint; the route still requires authentication (@login_required, @user_required) and the public workspaces feature flag (@enabled_required).
    • Other admin-level endpoints (listing members, viewing stats, ownership transfer) retain their membership checks.
    • (Ref: route_backend_public_workspaces.py, api_set_active_public_workspace)
  • Chats Page User Settings Hardening
    • Fixed a user-specific chats page failure where only one affected user could not load /chats due to malformed per-user settings data.
    • Root Cause: The chats route assumed user_settings["settings"] was always a dictionary. If that field existed but had an invalid type (for example string, null, or list), the page could fail before rendering.
    • Solution: Hardened get_user_settings() to normalize missing/malformed settings to {} and persist the repaired document. Hardened the chats route to use safe dictionary fallbacks when reading nested settings values.
    • Telemetry: Added repair logging ([UserSettings] Malformed settings repaired) to improve diagnostics for future user-specific data-shape issues.
    • Files Modified: functions_settings.py, route_frontend_chats.py, config.py.
    • Files Added: test_chats_user_settings_hardening_fix.py, CHATS_USER_SETTINGS_HARDENING_FIX.md.
    • (Ref: user settings normalization, /chats route resilience, functional_tests/test_chats_user_settings_hardening_fix.py, docs/explanation/fixes/CHATS_USER_SETTINGS_HARDENING_FIX.md)
  • Tag Filter Input Sanitization (Injection Prevention)
    • Added sanitize_tags_for_filter() function to validate tag filter inputs against the same ^[a-z0-9_-]+$ character whitelist enforced when saving tags.
    • Previously, tag filter values from query parameters only passed through normalize_tag() (strip + lowercase) without character validation, allowing arbitrary characters to reach OData filter construction in build_tags_filter().
    • Hardened build_tags_filter() in functions_search.py to validate tags before interpolating into OData expressions, eliminating the OData injection vector.
    • Updated tag filter parsing in personal, group, and public document routes to use sanitize_tags_for_filter() for defense-in-depth.
    • Invalid tag filter values are silently dropped (they cannot match any stored tag).
    • Files Modified: functions_documents.py, functions_search.py, route_backend_documents.py, route_backend_group_documents.py, route_backend_public_documents.py.
    • (Ref: TAG_FILTER_INJECTION_FIX.md, sanitize_tags_for_filter)
  • Citation Parsing Bug Fix
    • Fixed citation parsing edge cases where page range references (e.g., "Pages: 1-5") failed to generate correct clickable links when not all pages had explicit reference IDs in the bracketed citation section of the AI response.
    • Root Cause: The parseCitations() function only generated links for pages with existing [doc_prefix_N] bracket references, leaving pages without explicit references as non-functional text.
    • Solution: Added auto-fill logic using getDocPrefix() to extract the document ID prefix from known reference patterns and construct missing page references (e.g., if [doc_abc_1] exists, infer doc_abc_2 through doc_abc_5).
    • Files Modified: chat-citations.js.
    • (Ref: Citation parsing, page range handling, CITATION_IMPROVEMENTS.md)

User Interface Enhancements

  • Extended Document Dropdown Width

    • Widened the document selection dropdown in the chat interface for improved readability of long filenames. Dropdown width now dynamically adapts to the parent container.
    • Files Modified: chat-documents.js.
    • (Ref: Document dropdown, UI readability)
  • Enhanced Citation Links

    • Robust inline citation links with support for both inline source references and hybrid citation buttons.
    • Metadata citation support for viewing extracted document metadata including OCR text, vision analysis, and detected objects via the enhanced citation modal.
    • Improved error handling in citation JSON parsing with graceful fallback for malformed citation strings.
    • Files Modified: chat-citations.js, chat-enhanced-citations.js.
    • (Ref: Citation rendering, metadata citations, enhanced citation modal, CITATION_IMPROVEMENTS.md)

Chen, Vivien and others added 30 commits January 2, 2026 10:42
- Added custom_subdomain_name to OpenAI resource for managed identity authentication
- Created Speech Service resource with custom subdomain configuration
- Added RBAC role assignments for Speech Service (Managed Identity and App Service MI)
- Includes Cognitive Services Speech User and Speech Contributor roles
- Documentation: Azure Speech managed identity setup guide
- Add comprehensive ServiceNow integration guide with OAuth 2.0 setup
- Include OpenAPI specifications for Incident Management and Knowledge Base APIs
- Add agent instructions for ServiceNow support agent
- Fix GROUP_ACTION_OAUTH_SCHEMA_MERGING: Ensure additionalFields preserved during schema merge
- Fix GROUP_AGENT_LOADING: Improve group agent loading reliability
- Fix OPENAPI_BASIC_AUTH: Support basic authentication in OpenAPI actions
- Fix AZURE_AI_SEARCH_TEST_CONNECTION: Improve AI Search connection testing
- Update version to 0.236.012
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…_incident_api.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…factory.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
- Changed 'indexes = list(...)' to '_ = list(...)'
- Follows Python convention for discarded return values
- AI Search connection test only needs to verify the API call succeeds
- OPENAPI_BASIC_AUTH_FIX.md: 0.235.026 → 0.236.012
- GROUP_ACTION_OAUTH_SCHEMA_MERGING_FIX.md: 0.235.028 → 0.236.012
- GROUP_AGENT_LOADING_FIX.md: 0.235.027 → 0.236.012
- AZURE_AI_SEARCH_TEST_CONNECTION_FIX.md: 0.235.004 → 0.236.012"
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…dge_latest_spec.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…ION_FIX.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…dge_latest_spec_basicauth.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…_incident_api_basicauth.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Reverted route_backend_settings.py to origin/development version and removed
AZURE_AI_SEARCH_TEST_CONNECTION_FIX.md documentation. These changes will be
submitted in a dedicated PR to keep the ServiceNow integration PR focused.
…factory.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@paullizer paullizer changed the title Development v0.238.024 - Development into Staging Feb 24, 2026
paullizer and others added 5 commits February 24, 2026 13:52
…te-group-agent

optional only allow owner to create/edit/delete group agents and actions
* Added retention policy UI for groups and public workspaces

* Added retention policy UI for groups and public workspaces

---------

Co-authored-by: Eldon Gormsen <Eldon.Gormsen@microsoft.com>
Co-authored-by: Paul Lizer <paullizer@microsoft.com>
…tings` to `{}` and persist the repaired document.
paullizer and others added 11 commits March 2, 2026 16:47
fixed potential sql injection with new tags feature
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* update wrapper

* fix for json schema validation for plugins

* add video indexer fix for custom env

* fix fragility bug

* add handling for multi-decorated function

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>
fixed active workspace bug for users switching public workspaces
@paullizer paullizer merged commit 7a3d467 into Staging Mar 2, 2026
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants