LibreChat/api/server/routes/roles.js

183 lines
5.6 KiB
JavaScript
Raw Normal View History

🗨️ feat: Prompts (#3131) * 🗨️ feat: Prompts (#7) * WIP: MERGE prompts/frontend (#1) * added schema for prompt and promptgroup, added model methods for prompts, added routes for prompts * * updated promptGroup Schema * updated model methods for prompts (get, add, delete) * slight fixes in prompt routes * * Created Files Management components * Created Vector Stores components * Added file management route in the routes folder * Completed UI for Files list, Compeleted UI for vector stores list, Completed UI for upload file modal, Completed UI for preview file, Completed UI for preview vector store * Fixed style and UI fixes for file dashboard, file list and vector stores list * added responsiveness classes for vector store page * fixed responsiveness of file page, dashboard page, and main page * fixed styling and responsiveness issues on dashboard page, file list page and vector store page * added queries and mutations for prompts and promptGroups, added relevant endpoints in data-provider, added relevant components prompts, added and updated relevant APIs * added types on mutation queries data service, updated prompt attributes * feature: Prompts and prompt groups management, added relevant APIs, added types for data service/queries/mutations, added relevant mutation and queries * chore: typing clarifications * added drop down on prompts mgmt dashboard * Fixes: fixed version switching issue on tags update or labels update, added cross button on create prompt group, fixed list updation on prompt group renaiming, added CSV upload button * Feature: Added oneliner and category attributes in prompt group, added schema for categories, added schema methods and route for categories * chore: typing and lint issues * chore: more type and linter fixes * chore: linting * chore: prompt controller and backend typing example; MOVE TO CONTROLLER DIRECTORY * chore: more type fixes * style: prompt name changes * chore: more type changes, and stateful prompt name change without flickering * fix: Return result of savePrompt in patchPrompt API endpoint * fix: navigation prompt queries; refactor: name 'prompt-groups' to just 'groups' * refactor: fetch prompt groups rewrite * refactor(prompts): query/mutation statefulness * refactor: remove `isActive` field * refactor: remove labels, consolidate logic * style: width, layout shift * refactor: improve hover toggle behavior and styling * refactor: add useParams hook to PromptListItem for dynamic rendering and add timeout ref for blur timeout * chore: hide upload button * refactor: import Button component from correct location in PromptSidePanel * style: prompt editor styling * style: fix more layout shifts * style: container scroll * refactor: Rename CreatePrompt component to CreatePromptForm * refactor: use react-hook-form * refactor: Add Prompts components and routes to Dashboard * style: skeletons for loading * fix: optimize makePromptProduction * refactor: consolidate variables * feat: create prompt form validation * refactor: Consolidate variables and update mutation hooks * style: minor touchups * chore: Update lucide-react npm dependency to version 0.394.0 and npm audit fix * refactor: add a new icon for the Prompts heading. * style: Update PromptsView heading to use h1 instead of h2 and other minor margin issues * chore: wording * refactor: Update PromptsView heading to use h1 instead of h2, consolidate variables, and add new icons * refactor: Prompts Button for Mobile * feature: added category field in prompt group, added relevant API and static data on BE to support FE UI for category in prompt group * chore: template for prompt cards --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * WIP: Prompts/frontend Continued (#2) * chore: loading style, remove unused component * feat: Add CategorySelector component for prompt group category selection * feat: add categories to create prompt * feat: prompt versions styling * feat: optimistic updates for prompt production state * refactor: optimize form state and show if prompt field is dirty with cross icon, also other styling changes * chore: remove unused code and localizations * fix: light mode styling * WIP: SidePanel Prompts * refactor: move to groups directory * refactor: rename GroupsSidePanel to GroupSidePanel and update imports * style: ListCard * refactor: isProduction changes * refactor: infinite query with productionPrompt * refactor: optimize snippets and prompts, and styling * refactor: Update getSnippet function to accept a length parameter * chore: localizations * feat: prompts navigation to chat and vice versa * fix: create prompt * feat: remember last selected category for creating prompts * fix(promptGroups): fix pagination and add usePromptGroupsNav hook * Prompts/frontend 3 (#3) * fix: stateful issues with prompt groups * style: improved layout * refactor: improve variable naming in Eng.ts * refactor: theme selector styling improvements * added prompt cards on chat new page, with dark mode, added API to fetch random prompts, added types for useQuery Slightly improved usePromptGroupNav logic to fetch updated result for pageSize, updated prompt cards view with darkmode and responsiveness fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page * WIP: Prompts/frontend (#4) * fix: optimize and fix paginated query * fix: remove unique constraint on names * refactor: button links and styling * style: menu border light mode * feat: Add Auto-Send Switch component for prompts groups * refactor(ChatView): use form context for submission text * chore: clear convo state on navigation to dashboard routes * chore: save prompt edit name on tab, remove console log * feat: basic prompt submission * refactor: move Auto-Send Switch * style(ListCard): border styling * feat: Add function to detect variables in text * feat: Add OriginalDialog component to UI library * chore(ui): Update SelectDropDown options list class to use text-xs size * refactor: submitMessage hook now includes submitPrompt, make compatible to document query selector * WIP: Variable Dialog * feat: variable submission working for both auto-send and non-autosend * feat: dashboard breadcrumbs and prompts/chat navigation * refactor: dashboard breadcrumb and dashboard link to chat navigation * refactor: Update VariableDialog and VariableForm styles * Prompts: Admin features (#5) * fix: link issue * fix: usePromptGroupsNav add missing dep. * style: dashbreadcrumb and sidepanel text color * temp fix: remove refetch on pageNumber change * fix: handle multiple variable replacement * WIP: create project schema and add project groups to fetch * feat: Add functionality to add prompt group IDs to a project * feat: Add caching for startup config in config route * chore: remove prompt landing * style: Update Skeleton component with additional background styling * chore: styling and types * WIP: SharePrompt first draft * feat(SharePrompt): form validation * feat: shared global indicators * refactor: prompt details * refactor: change NoPromptGroup directory * feat: preview prompt * feat: remove/add global prompts, add rbac-related enums * refactor: manage prompts location * WIP: first draft admin settings for prompts * feat: SystemRoles enum * refactor: update PromptDetails component styling * style: ellipsis custom class for showing more preview text * WIP: initial role schema and initialization * style: improved margins for single unordered lists * fix: use custom chat form context to prevent re-renders from FormProvider * feat: Role mutations for Prompt Permissions * feat: fetch user role * feat: update AdminSettings form default values from user role values * refactor: rename PromptPermissions to Permissions for general definitions * feat: initial role checks * feat: Add optional `bodyProps` parameter to generateCheckAccess middleware * refactor: UI access checks * Prompts: delete (#6) * Fixed delete prompt version API, fixed types and logic for prompt version deletion, updated prompt delete mutation logic * chore: Update return type of deletePrompt function in Prompt.js --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * chore: Update package-lock.json version to 0.7.4-rc1 and fast-xml-parser to 4.4.0 * feat: toast for saving admin settings, add timer no-access navigation * feat: always make prod * feat: Add localization to category labels in CategorySelector component * feat: Update category label localization in CategorySelector component * fix: Enable making prompt production in Prompt API --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * feat: Add helper fn for dark mode detection in ThemeProvider * style: surface-primary definition * fix(useHasAccess): utilize user.role and not just USER role * fix: empty category and role fetch * refactort: increase max height to options list and use label if no localization is found * fix: update CategorySelector to handle empty category value and improve localization * refactor: move prompts to own store/reactquery modules, add in filter WIP * refactor: Rename AutoSendSwitch to AutoSendPrompt * style: theming commit * style: fix slight coloring issue for convos in dark mode * style: better composition for prompts side panel * style: remove gray-750 and make it gray-850 * chore: adjust theming * feat: filter all prompt groups and properly remove prompts from projects * refactor: optimize delete prompt groups further * chore: localization * feat: Add uniqueProperty filtering to normalizeData function * WIP: filter prompts * chore: Update FilterPrompts component to include User icon in FilterItem * feat(FilterPrompts): set categories * feat: more system filters and show selected category icon * style: always make prod, flips switch to avoid mis-clicks * style: ui/ux loading/no prompts * chore: style FilterPrompts ChatView * fix: handle missing role edge case * style: special variables * feat: special variables * refactor: improve replaceSpecialVars function in prompts.ts * feat: simple/advanced editor modes * chore: bump versions * feat: localizations and hide production button on simple mode * fix: error connecting layout shift * fix: prompts CRUD for admins * fix: secure single group fetch * style: sidepanel styling * style(PromptName): bring edit button closer to name * style: mobile prompts header * style: mobile prompts header continued * style: align send prompts switch right * feat: description * Update special variables description in Eng.ts * feat: update/create/preview oneliner * fix: allow empty oneliner update * style: loading improvement and always make selected prompt Production if simple mode * fix: production index set and remove unused props * fix(ci): mock initializeRoles * fix: address #3128 * fix: address #3128 * feat: add deletion confirmation dialog * fix: mobile UI issues * style: prompt library UI update * style: focus, logcal tab order * style: Refactor SelectDropDown component to improve code readability and maintainability * chore: bump data-provider * chore: fix labels * refactor: confirm delete prompt version --------- Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
2024-06-20 20:24:32 -04:00
const express = require('express');
📜 feat: Implement System Grants for Capability-Based Authorization (#11896) * feat: Implement System Grants for Role-Based Capabilities - Added a new `systemGrant` model and associated methods to manage role-based capabilities within the application. - Introduced middleware functions `hasCapability` and `requireCapability` to check user permissions based on their roles. - Updated the database seeding process to include system grants for the ADMIN role, ensuring all necessary capabilities are assigned on startup. - Enhanced type definitions and schemas to support the new system grant functionality, improving overall type safety and clarity in the codebase. * test: Add unit tests for capabilities middleware and system grant methods - Introduced comprehensive unit tests for the capabilities middleware, including `hasCapability` and `requireCapability`, ensuring proper permission checks based on user roles. - Added tests for the `SystemGrant` methods, verifying the seeding of system grants, capability granting, and revocation processes. - Enhanced test coverage for edge cases, including idempotency of grant operations and handling of unexpected errors in middleware. - Utilized mocks for database interactions to isolate tests and improve reliability. * refactor: Transition to Capability-Based Access Control - Replaced role-based access checks with capability-based checks across various middleware and routes, enhancing permission management. - Introduced `hasCapability` and `requireCapability` functions to streamline capability verification for user actions. - Updated relevant routes and middleware to utilize the new capability system, ensuring consistent permission enforcement. - Enhanced type definitions and added tests for the new capability functions, improving overall code reliability and maintainability. * test: Enhance capability-based access tests for ADMIN role - Updated tests to reflect the new capability-based access control, specifically for the ADMIN role. - Modified test descriptions to clarify that users with the MANAGE_AGENTS capability can bypass permission checks. - Seeded capabilities for the ADMIN role in multiple test files to ensure consistent permission checks across different routes and middleware. - Improved overall test coverage for capability verification, ensuring robust permission management. * test: Update capability tests for MCP server access - Renamed test to reflect the correct capability for bypassing permission checks, changing from MANAGE_AGENTS to MANAGE_MCP_SERVERS. - Updated seeding of capabilities for the ADMIN role to align with the new capability structure. - Ensured consistency in capability definitions across tests and middleware for improved permission management. * feat: Add hasConfigCapability for enhanced config access control - Introduced `hasConfigCapability` function to check user permissions for managing or reading specific config sections. - Updated middleware to export the new capability function, ensuring consistent access control across the application. - Enhanced unit tests to cover various scenarios for the new capability, improving overall test coverage and reliability. * fix: Update tenantId filter in createSystemGrantMethods - Added a condition to set tenantId filter to { $exists: false } when tenantId is null, ensuring proper handling of cases where tenantId is not provided. - This change improves the robustness of the system grant methods by explicitly managing the absence of tenantId in the filter logic. * fix: account deletion capability check - Updated the `canDeleteAccount` middleware to ensure that the `hasManageUsers` capability check only occurs if a user is present, preventing potential errors when the user object is undefined. - This change improves the robustness of the account deletion logic by ensuring proper handling of user permissions. * refactor: Optimize seeding of system grants for ADMIN role - Replaced sequential capability granting with parallel execution using Promise.all in the seedSystemGrants function. - This change improves performance and efficiency during the initialization of system grants, ensuring all capabilities are granted concurrently. * refactor: Simplify systemGrantSchema index definition - Removed the sparse option from the unique index on principalType, principalId, capability, and tenantId in the systemGrantSchema. - This change streamlines the index definition, potentially improving query performance and clarity in the schema design. * refactor: Reorganize role capability check in roles route - Moved the capability check for reading roles to occur after parsing the roleName, improving code clarity and structure. - This change ensures that the authorization logic is consistently applied before fetching role details, enhancing overall permission management. * refactor: Remove unused ISystemGrant interface from systemCapabilities.ts - Deleted the ISystemGrant interface as it was no longer needed, streamlining the code and improving clarity. - This change helps reduce clutter in the file and focuses on relevant capabilities for the system. * refactor: Migrate SystemCapabilities to data-schemas - Replaced imports of SystemCapabilities from 'librechat-data-provider' with imports from '@librechat/data-schemas' across multiple files. - This change centralizes the management of system capabilities, improving code organization and maintainability. * refactor: Update account deletion middleware and capability checks - Modified the `canDeleteAccount` middleware to ensure that the account deletion permission is only granted to users with the `MANAGE_USERS` capability, improving security and clarity in permission management. - Enhanced error logging for unauthorized account deletion attempts, providing better insights into permission issues. - Updated the `capabilities.ts` file to ensure consistent handling of user authentication checks, improving robustness in capability verification. - Refined type definitions in `systemGrant.ts` and `systemGrantMethods.ts` to utilize the `PrincipalType` enum, enhancing type safety and code clarity. * refactor: Extract principal ID normalization into a separate function - Introduced `normalizePrincipalId` function to streamline the normalization of principal IDs based on their type, enhancing code clarity and reusability. - Updated references in `createSystemGrantMethods` to utilize the new normalization function, improving maintainability and reducing code duplication. * test: Add unit tests for principalId normalization in systemGrant - Introduced tests for the `grantCapability`, `revokeCapability`, and `getCapabilitiesForPrincipal` methods to verify correct handling of principalId normalization between string and ObjectId formats. - Enhanced the `capabilities.ts` middleware to utilize the `PrincipalType` enum for improved type safety. - Added a new utility function `normalizePrincipalId` to streamline principal ID normalization logic, ensuring consistent behavior across the application. * feat: Introduce capability implications and enhance system grant methods - Added `CapabilityImplications` to define relationships between broader and implied capabilities, allowing for more intuitive permission checks. - Updated `createSystemGrantMethods` to expand capability queries to include implied capabilities, improving authorization logic. - Enhanced `systemGrantSchema` to include an `expiresAt` field for future TTL enforcement of grants, and added validation to ensure `tenantId` is not set to null. - Documented authorization requirements for prompt group and prompt deletion methods to clarify access control expectations. * test: Add unit tests for canDeleteAccount middleware - Introduced unit tests for the `canDeleteAccount` middleware to verify account deletion permissions based on user roles and capabilities. - Covered scenarios for both allowed and blocked account deletions, including checks for ADMIN users with the `MANAGE_USERS` capability and handling of undefined user cases. - Enhanced test structure to ensure clarity and maintainability of permission checks in the middleware. * fix: Add principalType enum validation to SystemGrant schema Without enum validation, any string value was accepted for principalType and silently stored. Invalid documents would never match capability queries, creating phantom grants impossible to diagnose without raw DB inspection. All other ACL models in the codebase validate this field. * fix: Replace seedSystemGrants Promise.all with bulkWrite for concurrency safety When two server instances start simultaneously (K8s rolling deploy, PM2 cluster), both call seedSystemGrants. With Promise.all + findOneAndUpdate upsert, both instances may attempt to insert the same documents, causing E11000 duplicate key errors that crash server startup. bulkWrite with ordered:false handles concurrent upserts gracefully and reduces 17 individual round trips to a single network call. The returned documents (previously discarded) are no longer fetched. * perf: Add AsyncLocalStorage per-request cache for capability checks Every hasCapability call previously required 2 DB round trips (getUserPrincipals + SystemGrant.exists) — replacing what were O(1) string comparisons. Routes like patchPromptGroup triggered this twice, and hasConfigCapability's fallback path resolved principals twice. This adds a per-request AsyncLocalStorage cache that: - Caches resolved principals (same for all checks within one request) - Caches capability check results (same user+cap = same answer) - Automatically scoped to request lifetime (no stale grants) - Falls through to DB when no store exists (background jobs, tests) - Requires no signature changes to hasCapability The capabilityContextMiddleware is registered at the app level before all routes, initializing a fresh store per request. * fix: Add error handling for inline hasCapability calls canDeleteAccount, fetchAssistants, and validateAuthor all call hasCapability without try-catch. These were previously O(1) string comparisons that could never throw. Now they hit the database and can fail on connection timeout or transient errors. Wrap each call in try-catch, defaulting to deny (false) on error. This ensures a DB hiccup returns a clean 403 instead of an unhandled 500 with a stack trace. * test: Add canDeleteAccount DB-error resilience test Tests that hasCapability rejection (e.g., DB timeout) results in a clean 403 rather than an unhandled exception. Validates the error handling added in the previous commit. * refactor: Use barrel import for hasCapability in validateAuthor Import from ~/server/middleware barrel instead of directly from ~/server/middleware/roles/capabilities for consistency with other non-middleware consumers. Files within the middleware barrel itself must continue using direct imports to avoid circular requires. * refactor: Remove misleading pre('save') hook from SystemGrant schema The pre('save') hook normalized principalId for USER/GROUP principals, but the primary write path (grantCapability) uses findOneAndUpdate — which does not trigger save hooks. The normalization was already handled explicitly in grantCapability itself. The hook created a false impression of schema-level enforcement that only covered save()/create() paths. Replace with a comment documenting that all writes must go through grantCapability. * feat: Add READ_ASSISTANTS capability to complete manage/read pair Every other managed resource had a paired READ_X / MANAGE_X capability except assistants. This adds READ_ASSISTANTS and registers the MANAGE_ASSISTANTS → READ_ASSISTANTS implication in CapabilityImplications, enabling future read-only assistant visibility grants. * chore: Reorder systemGrant methods for clarity Moved hasCapabilityForPrincipals to a more logical position in the returned object of createSystemGrantMethods, improving code readability. This change also maintains the inclusion of seedSystemGrants in the export, ensuring all necessary methods are available. * fix: Wrap seedSystemGrants in try-catch to avoid blocking startup Seeding capabilities is idempotent and will succeed on the next restart. A transient DB error during seeding should not prevent the server from starting — log the error and continue. * refactor: Improve capability check efficiency and add audit logging Move hasCapability calls after cheap early-exits in validateAuthor and fetchAssistants so the DB check only runs when its result matters. Add logger.debug on every capability bypass grant across all 7 call sites for auditability, and log errors in catch blocks instead of silently swallowing them. * test: Add integration tests for AsyncLocalStorage capability caching Exercises the full vertical — ALS context, generateCapabilityCheck, real getUserPrincipals, real hasCapabilityForPrincipals, real MongoDB via MongoMemoryServer. Covers per-request caching, cross-context isolation, concurrent request isolation, negative caching, capability implications, tenant scoping, group-based grants, and requireCapability middleware. * test: Add systemGrant data-layer and ALS edge-case integration tests systemGrant.spec.ts (51 tests): Full integration tests for all systemGrant methods against real MongoDB — grant/revoke lifecycle, principalId normalization (string→ObjectId for USER/GROUP, string for ROLE), capability implications (both directions), tenant scoping, schema validation (null tenantId, invalid enum, required fields, unique compound index). capabilities.integration.spec.ts (27 tests): Adds ALS edge cases — missing context degrades gracefully with no caching (background jobs, child processes), nested middleware creates independent inner context, optional-chaining safety when store is undefined, mid-request grant changes are invisible due to result caching, requireCapability works without ALS, and interleaved concurrent contexts maintain isolation. * fix: Add worker thread guards to capability ALS usage Detect when hasCapability or capabilityContextMiddleware is called from a worker thread (where ALS context does not propagate from the parent). hasCapability logs a warn-once per factory instance; the middleware logs an error since mounting Express middleware in a worker is likely a misconfiguration. Both continue to function correctly — the guard is observability, not a hard block. * fix: Include tenantId in ALS principal cache key for tenant isolation The principal cache key was user.id:user.role, which would reuse cached principals across tenants for the same user within a request. When getUserPrincipals gains tenant-scoped group resolution, principals from tenant-a would incorrectly serve tenant-b checks. Changed to user.id:user.role:user.tenantId to prevent cross-tenant cache hits. Adds integration test proving separate principal lookups per tenantId. * test: Remove redundant mocked capabilities.spec.js The JS wrapper test (7 tests, all mocked) is a strict subset of capabilities.integration.spec.ts (28 tests, real MongoDB). Every scenario it covered — hasCapability true/false, tenantId passthrough, requireCapability 403/500, error handling — is tested with higher fidelity in the integration suite. * test: Replace mocked canDeleteAccount tests with real MongoDB integration Remove hasCapability mock — tests now exercise the full capability chain against real MongoDB (getUserPrincipals, hasCapabilityForPrincipals, SystemGrant collection). Only mocks remaining are logger and cache. Adds new coverage: admin role without grant is blocked, user-level grant bypasses deletion restriction, null user handling. * test: Add comprehensive tests for ACL entry management and user group methods Introduces new tests for `deleteAclEntries`, `bulkWriteAclEntries`, and `findPublicResourceIds` in `aclEntry.spec.ts`, ensuring proper functionality for deleting and bulk managing ACL entries. Additionally, enhances `userGroup.spec.ts` with tests for finding groups by ID and name pattern, including external ID matching and source filtering. These changes improve coverage and validate the integrity of ACL and user group operations against real MongoDB interactions. * refactor: Update capability checks and logging for better clarity and error handling Replaced `MANAGE_USERS` with `ACCESS_ADMIN` in the `canDeleteAccount` middleware and related tests to align with updated permission structure. Enhanced logging in various middleware functions to use `logger.warn` for capability check failures, providing clearer error messages. Additionally, refactored capability checks in the `patchPromptGroup` and `validateAuthor` functions to improve readability and maintainability. This commit also includes adjustments to the `systemGrant` methods to implement retry logic for transient failures during capability seeding, ensuring robustness in the face of database errors. * refactor: Enhance logging and retry logic in seedSystemGrants method Updated the logging format in the seedSystemGrants method to include error messages for better clarity. Improved the retry mechanism by explicitly mocking multiple failures in tests, ensuring robust error handling during transient database issues. Additionally, refined imports in the systemGrant schema for better type management. * refactor: Consolidate imports in canDeleteAccount middleware Merged logger and SystemCapabilities imports from the data-schemas module into a single line for improved readability and maintainability of the code. This change streamlines the import statements in the canDeleteAccount middleware. * test: Enhance systemGrant tests for error handling and capability validation Added tests to the systemGrant methods to handle various error scenarios, including E11000 race conditions, invalid ObjectId strings for USER and GROUP principals, and invalid capability strings. These enhancements improve the robustness of the capability granting and revoking logic, ensuring proper error propagation and validation of inputs. * fix: Wrap hasCapability calls in deny-by-default try-catch at remaining sites canAccessResource, files.js, and roles.js all had hasCapability inside outer try-catch blocks that returned 500 on DB failure instead of falling through to the regular ACL check. This contradicts the deny-by-default pattern used everywhere else. Also removes raw error.message from the roles.js 500 response to prevent internal host/connection info leaking to clients. * fix: Normalize user ID in canDeleteAccount before passing to hasCapability requireCapability normalizes req.user.id via _id?.toString() fallback, but canDeleteAccount passed raw req.user directly. If req.user.id is absent (some auth layers only populate _id), getUserPrincipals received undefined, silently returning empty principals and blocking the bypass. * fix: Harden systemGrant schema and type safety - Reject empty string tenantId in schema validator (was only blocking null; empty string silently orphaned documents) - Fix reverseImplications to use BaseSystemCapability[] instead of string[], preserving the narrow discriminated type - Document READ_ASSISTANTS as reserved/unenforced * test: Use fake timers for seedSystemGrants retry tests and add tenantId validation - Switch retry tests to jest.useFakeTimers() to eliminate 3+ seconds of real setTimeout delays per test run - Add regression test for empty-string tenantId rejection * docs: Add TODO(#12091) comments for tenant-scoped capability gaps In multi-tenant mode, platform-level grants (no tenantId) won't match tenant-scoped queries, breaking admin access. getUserPrincipals also returns cross-tenant group memberships. Both need fixes in #12091.
2026-03-07 13:56:32 -05:00
const { logger, SystemCapabilities } = require('@librechat/data-schemas');
🗨️ feat: Prompts (#3131) * 🗨️ feat: Prompts (#7) * WIP: MERGE prompts/frontend (#1) * added schema for prompt and promptgroup, added model methods for prompts, added routes for prompts * * updated promptGroup Schema * updated model methods for prompts (get, add, delete) * slight fixes in prompt routes * * Created Files Management components * Created Vector Stores components * Added file management route in the routes folder * Completed UI for Files list, Compeleted UI for vector stores list, Completed UI for upload file modal, Completed UI for preview file, Completed UI for preview vector store * Fixed style and UI fixes for file dashboard, file list and vector stores list * added responsiveness classes for vector store page * fixed responsiveness of file page, dashboard page, and main page * fixed styling and responsiveness issues on dashboard page, file list page and vector store page * added queries and mutations for prompts and promptGroups, added relevant endpoints in data-provider, added relevant components prompts, added and updated relevant APIs * added types on mutation queries data service, updated prompt attributes * feature: Prompts and prompt groups management, added relevant APIs, added types for data service/queries/mutations, added relevant mutation and queries * chore: typing clarifications * added drop down on prompts mgmt dashboard * Fixes: fixed version switching issue on tags update or labels update, added cross button on create prompt group, fixed list updation on prompt group renaiming, added CSV upload button * Feature: Added oneliner and category attributes in prompt group, added schema for categories, added schema methods and route for categories * chore: typing and lint issues * chore: more type and linter fixes * chore: linting * chore: prompt controller and backend typing example; MOVE TO CONTROLLER DIRECTORY * chore: more type fixes * style: prompt name changes * chore: more type changes, and stateful prompt name change without flickering * fix: Return result of savePrompt in patchPrompt API endpoint * fix: navigation prompt queries; refactor: name 'prompt-groups' to just 'groups' * refactor: fetch prompt groups rewrite * refactor(prompts): query/mutation statefulness * refactor: remove `isActive` field * refactor: remove labels, consolidate logic * style: width, layout shift * refactor: improve hover toggle behavior and styling * refactor: add useParams hook to PromptListItem for dynamic rendering and add timeout ref for blur timeout * chore: hide upload button * refactor: import Button component from correct location in PromptSidePanel * style: prompt editor styling * style: fix more layout shifts * style: container scroll * refactor: Rename CreatePrompt component to CreatePromptForm * refactor: use react-hook-form * refactor: Add Prompts components and routes to Dashboard * style: skeletons for loading * fix: optimize makePromptProduction * refactor: consolidate variables * feat: create prompt form validation * refactor: Consolidate variables and update mutation hooks * style: minor touchups * chore: Update lucide-react npm dependency to version 0.394.0 and npm audit fix * refactor: add a new icon for the Prompts heading. * style: Update PromptsView heading to use h1 instead of h2 and other minor margin issues * chore: wording * refactor: Update PromptsView heading to use h1 instead of h2, consolidate variables, and add new icons * refactor: Prompts Button for Mobile * feature: added category field in prompt group, added relevant API and static data on BE to support FE UI for category in prompt group * chore: template for prompt cards --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * WIP: Prompts/frontend Continued (#2) * chore: loading style, remove unused component * feat: Add CategorySelector component for prompt group category selection * feat: add categories to create prompt * feat: prompt versions styling * feat: optimistic updates for prompt production state * refactor: optimize form state and show if prompt field is dirty with cross icon, also other styling changes * chore: remove unused code and localizations * fix: light mode styling * WIP: SidePanel Prompts * refactor: move to groups directory * refactor: rename GroupsSidePanel to GroupSidePanel and update imports * style: ListCard * refactor: isProduction changes * refactor: infinite query with productionPrompt * refactor: optimize snippets and prompts, and styling * refactor: Update getSnippet function to accept a length parameter * chore: localizations * feat: prompts navigation to chat and vice versa * fix: create prompt * feat: remember last selected category for creating prompts * fix(promptGroups): fix pagination and add usePromptGroupsNav hook * Prompts/frontend 3 (#3) * fix: stateful issues with prompt groups * style: improved layout * refactor: improve variable naming in Eng.ts * refactor: theme selector styling improvements * added prompt cards on chat new page, with dark mode, added API to fetch random prompts, added types for useQuery Slightly improved usePromptGroupNav logic to fetch updated result for pageSize, updated prompt cards view with darkmode and responsiveness fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page * WIP: Prompts/frontend (#4) * fix: optimize and fix paginated query * fix: remove unique constraint on names * refactor: button links and styling * style: menu border light mode * feat: Add Auto-Send Switch component for prompts groups * refactor(ChatView): use form context for submission text * chore: clear convo state on navigation to dashboard routes * chore: save prompt edit name on tab, remove console log * feat: basic prompt submission * refactor: move Auto-Send Switch * style(ListCard): border styling * feat: Add function to detect variables in text * feat: Add OriginalDialog component to UI library * chore(ui): Update SelectDropDown options list class to use text-xs size * refactor: submitMessage hook now includes submitPrompt, make compatible to document query selector * WIP: Variable Dialog * feat: variable submission working for both auto-send and non-autosend * feat: dashboard breadcrumbs and prompts/chat navigation * refactor: dashboard breadcrumb and dashboard link to chat navigation * refactor: Update VariableDialog and VariableForm styles * Prompts: Admin features (#5) * fix: link issue * fix: usePromptGroupsNav add missing dep. * style: dashbreadcrumb and sidepanel text color * temp fix: remove refetch on pageNumber change * fix: handle multiple variable replacement * WIP: create project schema and add project groups to fetch * feat: Add functionality to add prompt group IDs to a project * feat: Add caching for startup config in config route * chore: remove prompt landing * style: Update Skeleton component with additional background styling * chore: styling and types * WIP: SharePrompt first draft * feat(SharePrompt): form validation * feat: shared global indicators * refactor: prompt details * refactor: change NoPromptGroup directory * feat: preview prompt * feat: remove/add global prompts, add rbac-related enums * refactor: manage prompts location * WIP: first draft admin settings for prompts * feat: SystemRoles enum * refactor: update PromptDetails component styling * style: ellipsis custom class for showing more preview text * WIP: initial role schema and initialization * style: improved margins for single unordered lists * fix: use custom chat form context to prevent re-renders from FormProvider * feat: Role mutations for Prompt Permissions * feat: fetch user role * feat: update AdminSettings form default values from user role values * refactor: rename PromptPermissions to Permissions for general definitions * feat: initial role checks * feat: Add optional `bodyProps` parameter to generateCheckAccess middleware * refactor: UI access checks * Prompts: delete (#6) * Fixed delete prompt version API, fixed types and logic for prompt version deletion, updated prompt delete mutation logic * chore: Update return type of deletePrompt function in Prompt.js --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * chore: Update package-lock.json version to 0.7.4-rc1 and fast-xml-parser to 4.4.0 * feat: toast for saving admin settings, add timer no-access navigation * feat: always make prod * feat: Add localization to category labels in CategorySelector component * feat: Update category label localization in CategorySelector component * fix: Enable making prompt production in Prompt API --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * feat: Add helper fn for dark mode detection in ThemeProvider * style: surface-primary definition * fix(useHasAccess): utilize user.role and not just USER role * fix: empty category and role fetch * refactort: increase max height to options list and use label if no localization is found * fix: update CategorySelector to handle empty category value and improve localization * refactor: move prompts to own store/reactquery modules, add in filter WIP * refactor: Rename AutoSendSwitch to AutoSendPrompt * style: theming commit * style: fix slight coloring issue for convos in dark mode * style: better composition for prompts side panel * style: remove gray-750 and make it gray-850 * chore: adjust theming * feat: filter all prompt groups and properly remove prompts from projects * refactor: optimize delete prompt groups further * chore: localization * feat: Add uniqueProperty filtering to normalizeData function * WIP: filter prompts * chore: Update FilterPrompts component to include User icon in FilterItem * feat(FilterPrompts): set categories * feat: more system filters and show selected category icon * style: always make prod, flips switch to avoid mis-clicks * style: ui/ux loading/no prompts * chore: style FilterPrompts ChatView * fix: handle missing role edge case * style: special variables * feat: special variables * refactor: improve replaceSpecialVars function in prompts.ts * feat: simple/advanced editor modes * chore: bump versions * feat: localizations and hide production button on simple mode * fix: error connecting layout shift * fix: prompts CRUD for admins * fix: secure single group fetch * style: sidepanel styling * style(PromptName): bring edit button closer to name * style: mobile prompts header * style: mobile prompts header continued * style: align send prompts switch right * feat: description * Update special variables description in Eng.ts * feat: update/create/preview oneliner * fix: allow empty oneliner update * style: loading improvement and always make selected prompt Production if simple mode * fix: production index set and remove unused props * fix(ci): mock initializeRoles * fix: address #3128 * fix: address #3128 * feat: add deletion confirmation dialog * fix: mobile UI issues * style: prompt library UI update * style: focus, logcal tab order * style: Refactor SelectDropDown component to improve code readability and maintainability * chore: bump data-provider * chore: fix labels * refactor: confirm delete prompt version --------- Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
2024-06-20 20:24:32 -04:00
const {
SystemRoles,
roleDefaults,
PermissionTypes,
agentPermissionsSchema,
🗨️ feat: Prompts (#3131) * 🗨️ feat: Prompts (#7) * WIP: MERGE prompts/frontend (#1) * added schema for prompt and promptgroup, added model methods for prompts, added routes for prompts * * updated promptGroup Schema * updated model methods for prompts (get, add, delete) * slight fixes in prompt routes * * Created Files Management components * Created Vector Stores components * Added file management route in the routes folder * Completed UI for Files list, Compeleted UI for vector stores list, Completed UI for upload file modal, Completed UI for preview file, Completed UI for preview vector store * Fixed style and UI fixes for file dashboard, file list and vector stores list * added responsiveness classes for vector store page * fixed responsiveness of file page, dashboard page, and main page * fixed styling and responsiveness issues on dashboard page, file list page and vector store page * added queries and mutations for prompts and promptGroups, added relevant endpoints in data-provider, added relevant components prompts, added and updated relevant APIs * added types on mutation queries data service, updated prompt attributes * feature: Prompts and prompt groups management, added relevant APIs, added types for data service/queries/mutations, added relevant mutation and queries * chore: typing clarifications * added drop down on prompts mgmt dashboard * Fixes: fixed version switching issue on tags update or labels update, added cross button on create prompt group, fixed list updation on prompt group renaiming, added CSV upload button * Feature: Added oneliner and category attributes in prompt group, added schema for categories, added schema methods and route for categories * chore: typing and lint issues * chore: more type and linter fixes * chore: linting * chore: prompt controller and backend typing example; MOVE TO CONTROLLER DIRECTORY * chore: more type fixes * style: prompt name changes * chore: more type changes, and stateful prompt name change without flickering * fix: Return result of savePrompt in patchPrompt API endpoint * fix: navigation prompt queries; refactor: name 'prompt-groups' to just 'groups' * refactor: fetch prompt groups rewrite * refactor(prompts): query/mutation statefulness * refactor: remove `isActive` field * refactor: remove labels, consolidate logic * style: width, layout shift * refactor: improve hover toggle behavior and styling * refactor: add useParams hook to PromptListItem for dynamic rendering and add timeout ref for blur timeout * chore: hide upload button * refactor: import Button component from correct location in PromptSidePanel * style: prompt editor styling * style: fix more layout shifts * style: container scroll * refactor: Rename CreatePrompt component to CreatePromptForm * refactor: use react-hook-form * refactor: Add Prompts components and routes to Dashboard * style: skeletons for loading * fix: optimize makePromptProduction * refactor: consolidate variables * feat: create prompt form validation * refactor: Consolidate variables and update mutation hooks * style: minor touchups * chore: Update lucide-react npm dependency to version 0.394.0 and npm audit fix * refactor: add a new icon for the Prompts heading. * style: Update PromptsView heading to use h1 instead of h2 and other minor margin issues * chore: wording * refactor: Update PromptsView heading to use h1 instead of h2, consolidate variables, and add new icons * refactor: Prompts Button for Mobile * feature: added category field in prompt group, added relevant API and static data on BE to support FE UI for category in prompt group * chore: template for prompt cards --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * WIP: Prompts/frontend Continued (#2) * chore: loading style, remove unused component * feat: Add CategorySelector component for prompt group category selection * feat: add categories to create prompt * feat: prompt versions styling * feat: optimistic updates for prompt production state * refactor: optimize form state and show if prompt field is dirty with cross icon, also other styling changes * chore: remove unused code and localizations * fix: light mode styling * WIP: SidePanel Prompts * refactor: move to groups directory * refactor: rename GroupsSidePanel to GroupSidePanel and update imports * style: ListCard * refactor: isProduction changes * refactor: infinite query with productionPrompt * refactor: optimize snippets and prompts, and styling * refactor: Update getSnippet function to accept a length parameter * chore: localizations * feat: prompts navigation to chat and vice versa * fix: create prompt * feat: remember last selected category for creating prompts * fix(promptGroups): fix pagination and add usePromptGroupsNav hook * Prompts/frontend 3 (#3) * fix: stateful issues with prompt groups * style: improved layout * refactor: improve variable naming in Eng.ts * refactor: theme selector styling improvements * added prompt cards on chat new page, with dark mode, added API to fetch random prompts, added types for useQuery Slightly improved usePromptGroupNav logic to fetch updated result for pageSize, updated prompt cards view with darkmode and responsiveness fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page * WIP: Prompts/frontend (#4) * fix: optimize and fix paginated query * fix: remove unique constraint on names * refactor: button links and styling * style: menu border light mode * feat: Add Auto-Send Switch component for prompts groups * refactor(ChatView): use form context for submission text * chore: clear convo state on navigation to dashboard routes * chore: save prompt edit name on tab, remove console log * feat: basic prompt submission * refactor: move Auto-Send Switch * style(ListCard): border styling * feat: Add function to detect variables in text * feat: Add OriginalDialog component to UI library * chore(ui): Update SelectDropDown options list class to use text-xs size * refactor: submitMessage hook now includes submitPrompt, make compatible to document query selector * WIP: Variable Dialog * feat: variable submission working for both auto-send and non-autosend * feat: dashboard breadcrumbs and prompts/chat navigation * refactor: dashboard breadcrumb and dashboard link to chat navigation * refactor: Update VariableDialog and VariableForm styles * Prompts: Admin features (#5) * fix: link issue * fix: usePromptGroupsNav add missing dep. * style: dashbreadcrumb and sidepanel text color * temp fix: remove refetch on pageNumber change * fix: handle multiple variable replacement * WIP: create project schema and add project groups to fetch * feat: Add functionality to add prompt group IDs to a project * feat: Add caching for startup config in config route * chore: remove prompt landing * style: Update Skeleton component with additional background styling * chore: styling and types * WIP: SharePrompt first draft * feat(SharePrompt): form validation * feat: shared global indicators * refactor: prompt details * refactor: change NoPromptGroup directory * feat: preview prompt * feat: remove/add global prompts, add rbac-related enums * refactor: manage prompts location * WIP: first draft admin settings for prompts * feat: SystemRoles enum * refactor: update PromptDetails component styling * style: ellipsis custom class for showing more preview text * WIP: initial role schema and initialization * style: improved margins for single unordered lists * fix: use custom chat form context to prevent re-renders from FormProvider * feat: Role mutations for Prompt Permissions * feat: fetch user role * feat: update AdminSettings form default values from user role values * refactor: rename PromptPermissions to Permissions for general definitions * feat: initial role checks * feat: Add optional `bodyProps` parameter to generateCheckAccess middleware * refactor: UI access checks * Prompts: delete (#6) * Fixed delete prompt version API, fixed types and logic for prompt version deletion, updated prompt delete mutation logic * chore: Update return type of deletePrompt function in Prompt.js --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * chore: Update package-lock.json version to 0.7.4-rc1 and fast-xml-parser to 4.4.0 * feat: toast for saving admin settings, add timer no-access navigation * feat: always make prod * feat: Add localization to category labels in CategorySelector component * feat: Update category label localization in CategorySelector component * fix: Enable making prompt production in Prompt API --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * feat: Add helper fn for dark mode detection in ThemeProvider * style: surface-primary definition * fix(useHasAccess): utilize user.role and not just USER role * fix: empty category and role fetch * refactort: increase max height to options list and use label if no localization is found * fix: update CategorySelector to handle empty category value and improve localization * refactor: move prompts to own store/reactquery modules, add in filter WIP * refactor: Rename AutoSendSwitch to AutoSendPrompt * style: theming commit * style: fix slight coloring issue for convos in dark mode * style: better composition for prompts side panel * style: remove gray-750 and make it gray-850 * chore: adjust theming * feat: filter all prompt groups and properly remove prompts from projects * refactor: optimize delete prompt groups further * chore: localization * feat: Add uniqueProperty filtering to normalizeData function * WIP: filter prompts * chore: Update FilterPrompts component to include User icon in FilterItem * feat(FilterPrompts): set categories * feat: more system filters and show selected category icon * style: always make prod, flips switch to avoid mis-clicks * style: ui/ux loading/no prompts * chore: style FilterPrompts ChatView * fix: handle missing role edge case * style: special variables * feat: special variables * refactor: improve replaceSpecialVars function in prompts.ts * feat: simple/advanced editor modes * chore: bump versions * feat: localizations and hide production button on simple mode * fix: error connecting layout shift * fix: prompts CRUD for admins * fix: secure single group fetch * style: sidepanel styling * style(PromptName): bring edit button closer to name * style: mobile prompts header * style: mobile prompts header continued * style: align send prompts switch right * feat: description * Update special variables description in Eng.ts * feat: update/create/preview oneliner * fix: allow empty oneliner update * style: loading improvement and always make selected prompt Production if simple mode * fix: production index set and remove unused props * fix(ci): mock initializeRoles * fix: address #3128 * fix: address #3128 * feat: add deletion confirmation dialog * fix: mobile UI issues * style: prompt library UI update * style: focus, logcal tab order * style: Refactor SelectDropDown component to improve code readability and maintainability * chore: bump data-provider * chore: fix labels * refactor: confirm delete prompt version --------- Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
2024-06-20 20:24:32 -04:00
promptPermissionsSchema,
🧠 feat: User Memories for Conversational Context (#7760) * 🧠 feat: User Memories for Conversational Context chore: mcp typing, use `t` WIP: first pass, Memories UI - Added MemoryViewer component for displaying, editing, and deleting user memories. - Integrated data provider hooks for fetching, updating, and deleting memories. - Implemented pagination and loading states for better user experience. - Created unit tests for MemoryViewer to ensure functionality and interaction with data provider. - Updated translation files to include new UI strings related to memories. chore: move mcp-related files to own directory chore: rename librechat-mcp to librechat-api WIP: first pass, memory processing and data schemas chore: linting in fileSearch.js query description chore: rename librechat-api to @librechat/api across the project WIP: first pass, functional memory agent feat: add MemoryEditDialog and MemoryViewer components for managing user memories - Introduced MemoryEditDialog for editing memory entries with validation and toast notifications. - Updated MemoryViewer to support editing and deleting memories, including pagination and loading states. - Enhanced data provider to handle memory updates with optional original key for better management. - Added new localization strings for memory-related UI elements. feat: add memory permissions management - Implemented memory permissions in the backend, allowing roles to have specific permissions for using, creating, updating, and reading memories. - Added new API endpoints for updating memory permissions associated with roles. - Created a new AdminSettings component for managing memory permissions in the frontend. - Integrated memory permissions into the existing roles and permissions schemas. - Updated the interface to include memory settings and permissions. - Enhanced the MemoryViewer component to conditionally render admin settings based on user roles. - Added localization support for memory permissions in the translation files. feat: move AdminSettings component to a new position in MemoryViewer for better visibility refactor: clean up commented code in MemoryViewer component feat: enhance MemoryViewer with search functionality and improve MemoryEditDialog integration - Added a search input to filter memories in the MemoryViewer component. - Refactored MemoryEditDialog to accept children for better customization. - Updated MemoryViewer to utilize the new EditMemoryButton and DeleteMemoryButton components for editing and deleting memories. - Improved localization support by adding new strings for memory filtering and deletion confirmation. refactor: optimize memory filtering in MemoryViewer using match-sorter - Replaced manual filtering logic with match-sorter for improved search functionality. - Enhanced performance and readability of the filteredMemories computation. feat: enhance MemoryEditDialog with triggerRef and improve updateMemory mutation handling feat: implement access control for MemoryEditDialog and MemoryViewer components refactor: remove commented out code and create runMemory method refactor: rename role based files feat: implement access control for memory usage in AgentClient refactor: simplify checkVisionRequest method in AgentClient by removing commented-out code refactor: make `agents` dir in api package refactor: migrate Azure utilities to TypeScript and consolidate imports refactor: move sanitizeFilename function to a new file and update imports, add related tests refactor: update LLM configuration types and consolidate Azure options in the API package chore: linting chore: import order refactor: replace getLLMConfig with getOpenAIConfig and remove unused LLM configuration file chore: update winston-daily-rotate-file to version 5.0.0 and add object-hash dependency in package-lock.json refactor: move primeResources and optionalChainWithEmptyCheck functions to resources.ts and update imports refactor: move createRun function to a new run.ts file and update related imports fix: ensure safeAttachments is correctly typed as an array of TFile chore: add node-fetch dependency and refactor fetch-related functions into packages/api/utils, removing the old generators file refactor: enhance TEndpointOption type by using Pick to streamline endpoint fields and add new properties for model parameters and client options feat: implement initializeOpenAIOptions function and update OpenAI types for enhanced configuration handling fix: update types due to new TEndpointOption typing fix: ensure safe access to group parameters in initializeOpenAIOptions function fix: remove redundant API key validation comment in initializeOpenAIOptions function refactor: rename initializeOpenAIOptions to initializeOpenAI for consistency and update related documentation refactor: decouple req.body fields and tool loading from initializeAgentOptions chore: linting refactor: adjust column widths in MemoryViewer for improved layout refactor: simplify agent initialization by creating loadAgent function and removing unused code feat: add memory configuration loading and validation functions WIP: first pass, memory processing with config feat: implement memory callback and artifact handling feat: implement memory artifacts display and processing updates feat: add memory configuration options and schema validation for validKeys fix: update MemoryEditDialog and MemoryViewer to handle memory state and display improvements refactor: remove padding from BookmarkTable and MemoryViewer headers for consistent styling WIP: initial tokenLimit config and move Tokenizer to @librechat/api refactor: update mongoMeili plugin methods to use callback for better error handling feat: enhance memory management with token tracking and usage metrics - Added token counting for memory entries to enforce limits and provide usage statistics. - Updated memory retrieval and update routes to include total token usage and limit. - Enhanced MemoryEditDialog and MemoryViewer components to display memory usage and token information. - Refactored memory processing functions to handle token limits and provide feedback on memory capacity. feat: implement memory artifact handling in attachment handler - Enhanced useAttachmentHandler to process memory artifacts when receiving updates. - Introduced handleMemoryArtifact utility to manage memory updates and deletions. - Updated query client to reflect changes in memory state based on incoming data. refactor: restructure web search key extraction logic - Moved the logic for extracting API keys from the webSearchAuth configuration into a dedicated function, getWebSearchKeys. - Updated webSearchKeys to utilize the new function for improved clarity and maintainability. - Prevents build time errors feat: add personalization settings and memory preferences management - Introduced a new Personalization tab in settings to manage user memory preferences. - Implemented API endpoints and client-side logic for updating memory preferences. - Enhanced user interface components to reflect personalization options and memory usage. - Updated permissions to allow users to opt out of memory features. - Added localization support for new settings and messages related to personalization. style: personalization switch class feat: add PersonalizationIcon and align Side Panel UI feat: implement memory creation functionality - Added a new API endpoint for creating memory entries, including validation for key and value. - Introduced MemoryCreateDialog component for user interface to facilitate memory creation. - Integrated token limit checks to prevent exceeding user memory capacity. - Updated MemoryViewer to include a button for opening the memory creation dialog. - Enhanced localization support for new messages related to memory creation. feat: enhance message processing with configurable window size - Updated AgentClient to use a configurable message window size for processing messages. - Introduced messageWindowSize option in memory configuration schema with a default value of 5. - Improved logic for selecting messages to process based on the configured window size. chore: update librechat-data-provider version to 0.7.87 in package.json and package-lock.json chore: remove OpenAPIPlugin and its associated tests chore: remove MIGRATION_README.md as migration tasks are completed ci: fix backend tests chore: remove unused translation keys from localization file chore: remove problematic test file and unused var in AgentClient chore: remove unused import and import directly for JSDoc * feat: add api package build stage in Dockerfile for improved modularity * docs: reorder build steps in contributing guide for clarity
2025-06-07 18:52:22 -04:00
memoryPermissionsSchema,
🛸 feat: Remote Agent Access with External API Support (#11503) * 🪪 feat: Microsoft Graph Access Token Placeholder for MCP Servers (#10867) * feat: MCP Graph Token env var * Addressing copilot remarks * Addressed Copilot review remarks * Fixed graphtokenservice mock in MCP test suite * fix: remove unnecessary type check and cast in resolveGraphTokensInRecord * ci: add Graph Token integration tests in MCPManager * refactor: update user type definitions to use Partial<IUser> in multiple functions * test: enhance MCP tests for graph token processing and user placeholder resolution - Added comprehensive tests to validate the interaction between preProcessGraphTokens and processMCPEnv. - Ensured correct resolution of graph tokens and user placeholders in various configurations. - Mocked OIDC utilities to facilitate testing of token extraction and validation. - Verified that original options remain unchanged after processing. * chore: import order * chore: imports --------- Co-authored-by: Danny Avila <danny@librechat.ai> * WIP: OpenAI-compatible API for LibreChat agents - Added OpenAIChatCompletionController for handling chat completions. - Introduced ListModelsController and GetModelController for listing and retrieving agent details. - Created routes for OpenAI API endpoints, including /v1/chat/completions and /v1/models. - Developed event handlers for streaming responses in OpenAI format. - Implemented request validation and error handling for API interactions. - Integrated content aggregation and response formatting to align with OpenAI specifications. This commit establishes a foundational API for interacting with LibreChat agents in a manner compatible with OpenAI's chat completion interface. * refactor: OpenAI-spec content aggregation for improved performance and clarity * fix: OpenAI chat completion controller with safe user handling for correct tool loading * refactor: Remove conversation ID from OpenAI response context and related handlers * refactor: OpenAI chat completion handling with streaming support - Introduced a lightweight tracker for streaming responses, allowing for efficient tracking of emitted content and usage metadata. - Updated the OpenAIChatCompletionController to utilize the new tracker, improving the handling of streaming and non-streaming responses. - Refactored event handlers to accommodate the new streaming logic, ensuring proper management of tool calls and content aggregation. - Adjusted response handling to streamline error reporting during streaming sessions. * WIP: Open Responses API with core service, types, and handlers - Added Open Responses API module with comprehensive types and enums. - Implemented core service for processing requests, including validation and input conversion. - Developed event handlers for streaming responses and non-streaming aggregation. - Established response building logic and error handling mechanisms. - Created detailed types for input and output content, ensuring compliance with Open Responses specification. * feat: Implement response storage and retrieval in Open Responses API - Added functionality to save user input messages and assistant responses to the database when the `store` flag is set to true. - Introduced a new endpoint to retrieve stored responses by ID, allowing users to access previous interactions. - Enhanced the response creation process to include database operations for conversation and message storage. - Implemented tests to validate the storage and retrieval of responses, ensuring correct behavior for both existing and non-existent response IDs. * refactor: Open Responses API with additional token tracking and validation - Added support for tracking cached tokens in response usage, improving token management. - Updated response structure to include new properties for top log probabilities and detailed usage metrics. - Enhanced tests to validate the presence and types of new properties in API responses, ensuring compliance with updated specifications. - Refactored response handling to accommodate new fields and improve overall clarity and performance. * refactor: Update reasoning event handlers and types for consistency - Renamed reasoning text events to simplify naming conventions, changing `emitReasoningTextDelta` to `emitReasoningDelta` and `emitReasoningTextDone` to `emitReasoningDone`. - Updated event types in the API to reflect the new naming, ensuring consistency across the codebase. - Added `logprobs` property to output events for enhanced tracking of log probabilities. * feat: Add validation for streaming events in Open Responses API tests * feat: Implement response.created event in Open Responses API - Added emitResponseCreated function to emit the response.created event as the first event in the streaming sequence, adhering to the Open Responses specification. - Updated createResponse function to emit response.created followed by response.in_progress. - Enhanced tests to validate the order of emitted events, ensuring response.created is triggered before response.in_progress. * feat: Responses API with attachment event handling - Introduced `createResponsesToolEndCallback` to handle attachment events in the Responses API, emitting `librechat:attachment` events as per the Open Responses extension specification. - Updated the `createResponse` function to utilize the new callback for processing tool outputs and emitting attachments during streaming. - Added helper functions for writing attachment events and defined types for attachment data, ensuring compatibility with the Open Responses protocol. - Enhanced tests to validate the integration of attachment events within the Responses API workflow. * WIP: remote agent auth * fix: Improve loading state handling in AgentApiKeys component - Updated the rendering logic to conditionally display loading spinner and API keys based on the loading state. - Removed unnecessary imports and streamlined the component for better readability. * refactor: Update API key access handling in routes - Replaced `checkAccess` with `generateCheckAccess` for improved access control. - Consolidated access checks into a single `checkApiKeyAccess` function, enhancing code readability and maintainability. - Streamlined route definitions for creating, listing, retrieving, and deleting API keys. * fix: Add permission handling for REMOTE_AGENT resource type * feat: Enhance permission handling for REMOTE_AGENT resources - Updated the deleteAgent and deleteUserAgents functions to handle permissions for both AGENT and REMOTE_AGENT resource types. - Introduced new functions to enrich REMOTE_AGENT principals and backfill permissions for AGENT owners. - Modified createAgentHandler and duplicateAgentHandler to grant permissions for REMOTE_AGENT alongside AGENT. - Added utility functions for retrieving effective permissions for REMOTE_AGENT resources, ensuring consistent access control across the application. * refactor: Rename and update roles for remote agent access - Changed role name from API User to Editor in translation files for clarity. - Updated default editor role ID from REMOTE_AGENT_USER to REMOTE_AGENT_EDITOR in resource configurations. - Adjusted role localization to reflect the new Editor role. - Modified access permissions to align with the updated role definitions across the application. * feat: Introduce remote agent permissions and update access handling - Added support for REMOTE_AGENTS in permission schemas, including use, create, share, and share_public permissions. - Updated the interface configuration to include remote agent settings. - Modified middleware and API key access checks to align with the new remote agent permission structure. - Enhanced role defaults to incorporate remote agent permissions, ensuring consistent access control across the application. * refactor: Update AgentApiKeys component and permissions handling - Refactored the AgentApiKeys component to improve structure and readability, including the introduction of ApiKeysContent for better separation of concerns. - Updated CreateKeyDialog to accept an onKeyCreated callback, enhancing its functionality. - Adjusted permission checks in Data component to use REMOTE_AGENTS and USE permissions, aligning with recent permission schema changes. - Enhanced loading state handling and dialog management for a smoother user experience. * refactor: Update remote agent access checks in API routes - Replaced existing access checks with `generateCheckAccess` for remote agents in the API keys and agents routes. - Introduced specific permission checks for creating, listing, retrieving, and deleting API keys, enhancing access control. - Improved code structure by consolidating permission handling for remote agents across multiple routes. * fix: Correct query parameters in ApiKeysContent component - Updated the useGetAgentApiKeysQuery call to include an object for the enabled parameter, ensuring proper functionality when the component is open. - This change improves the handling of API key retrieval based on the component's open state. * feat: Implement remote agents permissions and update API routes - Added new API route for updating remote agents permissions, enhancing role management capabilities. - Introduced remote agents permissions handling in the AgentApiKeys component, including a dedicated settings dialog. - Updated localization files to include new remote agents permission labels for better user experience. - Refactored data provider to support remote agents permissions updates, ensuring consistent access control across the application. * feat: Add remote agents permissions to role schema and interface - Introduced new permissions for REMOTE_AGENTS in the role schema, including USE, CREATE, SHARE, and SHARE_PUBLIC. - Updated the IRole interface to reflect the new remote agents permissions structure, enhancing role management capabilities. * feat: Add remote agents settings button to API keys dialog * feat: Update AgentFooter to include remote agent sharing permissions - Refactored access checks to incorporate permissions for sharing remote agents. - Enhanced conditional rendering logic to allow sharing by users with remote agent permissions. - Improved loading state handling for remote agent permissions, ensuring a smoother user experience. * refactor: Update API key creation access check and localization strings - Replaced the access check for creating API keys to use the existing remote agents access check. - Updated localization strings to correct the descriptions for remote agent permissions, ensuring clarity in user interface. * fix: resource permission mapping to include remote agents - Changed the resourceToPermissionMap to use a Partial<Record> for better flexibility. - Added mapping for REMOTE_AGENT permissions, enhancing the sharing capabilities for remote agents. * feat: Implement remote access checks for agent models - Enhanced ListModelsController and GetModelController to include checks for user permissions on remote agents. - Integrated findAccessibleResources to filter agents based on VIEW permission for REMOTE_AGENT. - Updated response handling to ensure users can only access agents they have permissions for, improving security and access control. * fix: Update user parameter type in processUserPlaceholders function - Changed the user parameter type in the processUserPlaceholders function from Partial<Partial<IUser>> to Partial<IUser> for improved type clarity and consistency. * refactor: Simplify integration test structure by removing conditional describe - Replaced conditional describeWithApiKey with a standard describe for all integration tests in responses.spec.js. - This change enhances test clarity and ensures all tests are executed consistently, regardless of the SKIP_INTEGRATION_TESTS flag. * test: Update AgentFooter tests to reflect new grant access dialog ID - Changed test IDs for the grant access dialog in AgentFooter tests to include the resource type, ensuring accurate identification in the test cases. - This update improves test clarity and aligns with recent changes in the component's implementation. * test: Enhance integration tests for Open Responses API - Updated integration tests in responses.spec.js to utilize an authRequest helper for consistent authorization handling across all test cases. - Introduced a test user and API key creation to improve test setup and ensure proper permission checks for remote agents. - Added checks for existing access roles and created necessary roles if they do not exist, enhancing test reliability and coverage. * feat: Extend accessRole schema to include remoteAgent resource type - Updated the accessRole schema to add 'remoteAgent' to the resourceType enum, enhancing the flexibility of role assignments and permissions management. * test: refactored test setup to create a minimal Express app for responses routes, enhancing test structure and maintainability. * test: Enhance abort.spec.js by mocking additional modules for improved test isolation - Updated the test setup in abort.spec.js to include actual implementations of '@librechat/data-schemas' and '@librechat/api' while maintaining mock functionality. - This change improves test reliability and ensures that the tests are more representative of the actual module behavior. * refactor: Update conversation ID generation to use UUID - Replaced the nanoid with uuidv4 for generating conversation IDs in the createResponse function, enhancing uniqueness and consistency in ID generation. * test: Add remote agent access roles to AccessRole model tests - Included additional access roles for remote agents (REMOTE_AGENT_EDITOR, REMOTE_AGENT_OWNER, REMOTE_AGENT_VIEWER) in the AccessRole model tests to ensure comprehensive coverage of role assignments and permissions management. * chore: Add deletion of user agent API keys in user deletion process - Updated the user deletion process in UserController and delete-user.js to include the removal of user agent API keys, ensuring comprehensive cleanup of user data upon account deletion. * test: Add remote agents permissions to permissions.spec.ts - Enhanced the permissions tests by including comprehensive permission settings for remote agents across various scenarios, ensuring accurate validation of access controls for remote agent roles. * chore: Update remote agents translations for clarity and consistency - Removed outdated remote agents translation entries and added revised entries to improve clarity on API key creation and sharing permissions for remote agents. This enhances user understanding of the available functionalities. * feat: Add indexing and TTL for agent API keys - Introduced an index on the `key` field for improved query performance. - Added a TTL index on the `expiresAt` field to enable automatic cleanup of expired API keys, ensuring efficient management of stored keys. * chore: Update API route documentation for clarity - Revised comments in the agents route file to clarify the handling of API key authentication. - Removed outdated endpoint listings to streamline the documentation and focus on current functionality. --------- Co-authored-by: Max Sanna <max@maxsanna.com>
2026-01-26 10:50:30 -05:00
mcpServersPermissionsSchema,
marketplacePermissionsSchema,
peoplePickerPermissionsSchema,
🛸 feat: Remote Agent Access with External API Support (#11503) * 🪪 feat: Microsoft Graph Access Token Placeholder for MCP Servers (#10867) * feat: MCP Graph Token env var * Addressing copilot remarks * Addressed Copilot review remarks * Fixed graphtokenservice mock in MCP test suite * fix: remove unnecessary type check and cast in resolveGraphTokensInRecord * ci: add Graph Token integration tests in MCPManager * refactor: update user type definitions to use Partial<IUser> in multiple functions * test: enhance MCP tests for graph token processing and user placeholder resolution - Added comprehensive tests to validate the interaction between preProcessGraphTokens and processMCPEnv. - Ensured correct resolution of graph tokens and user placeholders in various configurations. - Mocked OIDC utilities to facilitate testing of token extraction and validation. - Verified that original options remain unchanged after processing. * chore: import order * chore: imports --------- Co-authored-by: Danny Avila <danny@librechat.ai> * WIP: OpenAI-compatible API for LibreChat agents - Added OpenAIChatCompletionController for handling chat completions. - Introduced ListModelsController and GetModelController for listing and retrieving agent details. - Created routes for OpenAI API endpoints, including /v1/chat/completions and /v1/models. - Developed event handlers for streaming responses in OpenAI format. - Implemented request validation and error handling for API interactions. - Integrated content aggregation and response formatting to align with OpenAI specifications. This commit establishes a foundational API for interacting with LibreChat agents in a manner compatible with OpenAI's chat completion interface. * refactor: OpenAI-spec content aggregation for improved performance and clarity * fix: OpenAI chat completion controller with safe user handling for correct tool loading * refactor: Remove conversation ID from OpenAI response context and related handlers * refactor: OpenAI chat completion handling with streaming support - Introduced a lightweight tracker for streaming responses, allowing for efficient tracking of emitted content and usage metadata. - Updated the OpenAIChatCompletionController to utilize the new tracker, improving the handling of streaming and non-streaming responses. - Refactored event handlers to accommodate the new streaming logic, ensuring proper management of tool calls and content aggregation. - Adjusted response handling to streamline error reporting during streaming sessions. * WIP: Open Responses API with core service, types, and handlers - Added Open Responses API module with comprehensive types and enums. - Implemented core service for processing requests, including validation and input conversion. - Developed event handlers for streaming responses and non-streaming aggregation. - Established response building logic and error handling mechanisms. - Created detailed types for input and output content, ensuring compliance with Open Responses specification. * feat: Implement response storage and retrieval in Open Responses API - Added functionality to save user input messages and assistant responses to the database when the `store` flag is set to true. - Introduced a new endpoint to retrieve stored responses by ID, allowing users to access previous interactions. - Enhanced the response creation process to include database operations for conversation and message storage. - Implemented tests to validate the storage and retrieval of responses, ensuring correct behavior for both existing and non-existent response IDs. * refactor: Open Responses API with additional token tracking and validation - Added support for tracking cached tokens in response usage, improving token management. - Updated response structure to include new properties for top log probabilities and detailed usage metrics. - Enhanced tests to validate the presence and types of new properties in API responses, ensuring compliance with updated specifications. - Refactored response handling to accommodate new fields and improve overall clarity and performance. * refactor: Update reasoning event handlers and types for consistency - Renamed reasoning text events to simplify naming conventions, changing `emitReasoningTextDelta` to `emitReasoningDelta` and `emitReasoningTextDone` to `emitReasoningDone`. - Updated event types in the API to reflect the new naming, ensuring consistency across the codebase. - Added `logprobs` property to output events for enhanced tracking of log probabilities. * feat: Add validation for streaming events in Open Responses API tests * feat: Implement response.created event in Open Responses API - Added emitResponseCreated function to emit the response.created event as the first event in the streaming sequence, adhering to the Open Responses specification. - Updated createResponse function to emit response.created followed by response.in_progress. - Enhanced tests to validate the order of emitted events, ensuring response.created is triggered before response.in_progress. * feat: Responses API with attachment event handling - Introduced `createResponsesToolEndCallback` to handle attachment events in the Responses API, emitting `librechat:attachment` events as per the Open Responses extension specification. - Updated the `createResponse` function to utilize the new callback for processing tool outputs and emitting attachments during streaming. - Added helper functions for writing attachment events and defined types for attachment data, ensuring compatibility with the Open Responses protocol. - Enhanced tests to validate the integration of attachment events within the Responses API workflow. * WIP: remote agent auth * fix: Improve loading state handling in AgentApiKeys component - Updated the rendering logic to conditionally display loading spinner and API keys based on the loading state. - Removed unnecessary imports and streamlined the component for better readability. * refactor: Update API key access handling in routes - Replaced `checkAccess` with `generateCheckAccess` for improved access control. - Consolidated access checks into a single `checkApiKeyAccess` function, enhancing code readability and maintainability. - Streamlined route definitions for creating, listing, retrieving, and deleting API keys. * fix: Add permission handling for REMOTE_AGENT resource type * feat: Enhance permission handling for REMOTE_AGENT resources - Updated the deleteAgent and deleteUserAgents functions to handle permissions for both AGENT and REMOTE_AGENT resource types. - Introduced new functions to enrich REMOTE_AGENT principals and backfill permissions for AGENT owners. - Modified createAgentHandler and duplicateAgentHandler to grant permissions for REMOTE_AGENT alongside AGENT. - Added utility functions for retrieving effective permissions for REMOTE_AGENT resources, ensuring consistent access control across the application. * refactor: Rename and update roles for remote agent access - Changed role name from API User to Editor in translation files for clarity. - Updated default editor role ID from REMOTE_AGENT_USER to REMOTE_AGENT_EDITOR in resource configurations. - Adjusted role localization to reflect the new Editor role. - Modified access permissions to align with the updated role definitions across the application. * feat: Introduce remote agent permissions and update access handling - Added support for REMOTE_AGENTS in permission schemas, including use, create, share, and share_public permissions. - Updated the interface configuration to include remote agent settings. - Modified middleware and API key access checks to align with the new remote agent permission structure. - Enhanced role defaults to incorporate remote agent permissions, ensuring consistent access control across the application. * refactor: Update AgentApiKeys component and permissions handling - Refactored the AgentApiKeys component to improve structure and readability, including the introduction of ApiKeysContent for better separation of concerns. - Updated CreateKeyDialog to accept an onKeyCreated callback, enhancing its functionality. - Adjusted permission checks in Data component to use REMOTE_AGENTS and USE permissions, aligning with recent permission schema changes. - Enhanced loading state handling and dialog management for a smoother user experience. * refactor: Update remote agent access checks in API routes - Replaced existing access checks with `generateCheckAccess` for remote agents in the API keys and agents routes. - Introduced specific permission checks for creating, listing, retrieving, and deleting API keys, enhancing access control. - Improved code structure by consolidating permission handling for remote agents across multiple routes. * fix: Correct query parameters in ApiKeysContent component - Updated the useGetAgentApiKeysQuery call to include an object for the enabled parameter, ensuring proper functionality when the component is open. - This change improves the handling of API key retrieval based on the component's open state. * feat: Implement remote agents permissions and update API routes - Added new API route for updating remote agents permissions, enhancing role management capabilities. - Introduced remote agents permissions handling in the AgentApiKeys component, including a dedicated settings dialog. - Updated localization files to include new remote agents permission labels for better user experience. - Refactored data provider to support remote agents permissions updates, ensuring consistent access control across the application. * feat: Add remote agents permissions to role schema and interface - Introduced new permissions for REMOTE_AGENTS in the role schema, including USE, CREATE, SHARE, and SHARE_PUBLIC. - Updated the IRole interface to reflect the new remote agents permissions structure, enhancing role management capabilities. * feat: Add remote agents settings button to API keys dialog * feat: Update AgentFooter to include remote agent sharing permissions - Refactored access checks to incorporate permissions for sharing remote agents. - Enhanced conditional rendering logic to allow sharing by users with remote agent permissions. - Improved loading state handling for remote agent permissions, ensuring a smoother user experience. * refactor: Update API key creation access check and localization strings - Replaced the access check for creating API keys to use the existing remote agents access check. - Updated localization strings to correct the descriptions for remote agent permissions, ensuring clarity in user interface. * fix: resource permission mapping to include remote agents - Changed the resourceToPermissionMap to use a Partial<Record> for better flexibility. - Added mapping for REMOTE_AGENT permissions, enhancing the sharing capabilities for remote agents. * feat: Implement remote access checks for agent models - Enhanced ListModelsController and GetModelController to include checks for user permissions on remote agents. - Integrated findAccessibleResources to filter agents based on VIEW permission for REMOTE_AGENT. - Updated response handling to ensure users can only access agents they have permissions for, improving security and access control. * fix: Update user parameter type in processUserPlaceholders function - Changed the user parameter type in the processUserPlaceholders function from Partial<Partial<IUser>> to Partial<IUser> for improved type clarity and consistency. * refactor: Simplify integration test structure by removing conditional describe - Replaced conditional describeWithApiKey with a standard describe for all integration tests in responses.spec.js. - This change enhances test clarity and ensures all tests are executed consistently, regardless of the SKIP_INTEGRATION_TESTS flag. * test: Update AgentFooter tests to reflect new grant access dialog ID - Changed test IDs for the grant access dialog in AgentFooter tests to include the resource type, ensuring accurate identification in the test cases. - This update improves test clarity and aligns with recent changes in the component's implementation. * test: Enhance integration tests for Open Responses API - Updated integration tests in responses.spec.js to utilize an authRequest helper for consistent authorization handling across all test cases. - Introduced a test user and API key creation to improve test setup and ensure proper permission checks for remote agents. - Added checks for existing access roles and created necessary roles if they do not exist, enhancing test reliability and coverage. * feat: Extend accessRole schema to include remoteAgent resource type - Updated the accessRole schema to add 'remoteAgent' to the resourceType enum, enhancing the flexibility of role assignments and permissions management. * test: refactored test setup to create a minimal Express app for responses routes, enhancing test structure and maintainability. * test: Enhance abort.spec.js by mocking additional modules for improved test isolation - Updated the test setup in abort.spec.js to include actual implementations of '@librechat/data-schemas' and '@librechat/api' while maintaining mock functionality. - This change improves test reliability and ensures that the tests are more representative of the actual module behavior. * refactor: Update conversation ID generation to use UUID - Replaced the nanoid with uuidv4 for generating conversation IDs in the createResponse function, enhancing uniqueness and consistency in ID generation. * test: Add remote agent access roles to AccessRole model tests - Included additional access roles for remote agents (REMOTE_AGENT_EDITOR, REMOTE_AGENT_OWNER, REMOTE_AGENT_VIEWER) in the AccessRole model tests to ensure comprehensive coverage of role assignments and permissions management. * chore: Add deletion of user agent API keys in user deletion process - Updated the user deletion process in UserController and delete-user.js to include the removal of user agent API keys, ensuring comprehensive cleanup of user data upon account deletion. * test: Add remote agents permissions to permissions.spec.ts - Enhanced the permissions tests by including comprehensive permission settings for remote agents across various scenarios, ensuring accurate validation of access controls for remote agent roles. * chore: Update remote agents translations for clarity and consistency - Removed outdated remote agents translation entries and added revised entries to improve clarity on API key creation and sharing permissions for remote agents. This enhances user understanding of the available functionalities. * feat: Add indexing and TTL for agent API keys - Introduced an index on the `key` field for improved query performance. - Added a TTL index on the `expiresAt` field to enable automatic cleanup of expired API keys, ensuring efficient management of stored keys. * chore: Update API route documentation for clarity - Revised comments in the agents route file to clarify the handling of API key authentication. - Removed outdated endpoint listings to streamline the documentation and focus on current functionality. --------- Co-authored-by: Max Sanna <max@maxsanna.com>
2026-01-26 10:50:30 -05:00
remoteAgentsPermissionsSchema,
🗨️ feat: Prompts (#3131) * 🗨️ feat: Prompts (#7) * WIP: MERGE prompts/frontend (#1) * added schema for prompt and promptgroup, added model methods for prompts, added routes for prompts * * updated promptGroup Schema * updated model methods for prompts (get, add, delete) * slight fixes in prompt routes * * Created Files Management components * Created Vector Stores components * Added file management route in the routes folder * Completed UI for Files list, Compeleted UI for vector stores list, Completed UI for upload file modal, Completed UI for preview file, Completed UI for preview vector store * Fixed style and UI fixes for file dashboard, file list and vector stores list * added responsiveness classes for vector store page * fixed responsiveness of file page, dashboard page, and main page * fixed styling and responsiveness issues on dashboard page, file list page and vector store page * added queries and mutations for prompts and promptGroups, added relevant endpoints in data-provider, added relevant components prompts, added and updated relevant APIs * added types on mutation queries data service, updated prompt attributes * feature: Prompts and prompt groups management, added relevant APIs, added types for data service/queries/mutations, added relevant mutation and queries * chore: typing clarifications * added drop down on prompts mgmt dashboard * Fixes: fixed version switching issue on tags update or labels update, added cross button on create prompt group, fixed list updation on prompt group renaiming, added CSV upload button * Feature: Added oneliner and category attributes in prompt group, added schema for categories, added schema methods and route for categories * chore: typing and lint issues * chore: more type and linter fixes * chore: linting * chore: prompt controller and backend typing example; MOVE TO CONTROLLER DIRECTORY * chore: more type fixes * style: prompt name changes * chore: more type changes, and stateful prompt name change without flickering * fix: Return result of savePrompt in patchPrompt API endpoint * fix: navigation prompt queries; refactor: name 'prompt-groups' to just 'groups' * refactor: fetch prompt groups rewrite * refactor(prompts): query/mutation statefulness * refactor: remove `isActive` field * refactor: remove labels, consolidate logic * style: width, layout shift * refactor: improve hover toggle behavior and styling * refactor: add useParams hook to PromptListItem for dynamic rendering and add timeout ref for blur timeout * chore: hide upload button * refactor: import Button component from correct location in PromptSidePanel * style: prompt editor styling * style: fix more layout shifts * style: container scroll * refactor: Rename CreatePrompt component to CreatePromptForm * refactor: use react-hook-form * refactor: Add Prompts components and routes to Dashboard * style: skeletons for loading * fix: optimize makePromptProduction * refactor: consolidate variables * feat: create prompt form validation * refactor: Consolidate variables and update mutation hooks * style: minor touchups * chore: Update lucide-react npm dependency to version 0.394.0 and npm audit fix * refactor: add a new icon for the Prompts heading. * style: Update PromptsView heading to use h1 instead of h2 and other minor margin issues * chore: wording * refactor: Update PromptsView heading to use h1 instead of h2, consolidate variables, and add new icons * refactor: Prompts Button for Mobile * feature: added category field in prompt group, added relevant API and static data on BE to support FE UI for category in prompt group * chore: template for prompt cards --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * WIP: Prompts/frontend Continued (#2) * chore: loading style, remove unused component * feat: Add CategorySelector component for prompt group category selection * feat: add categories to create prompt * feat: prompt versions styling * feat: optimistic updates for prompt production state * refactor: optimize form state and show if prompt field is dirty with cross icon, also other styling changes * chore: remove unused code and localizations * fix: light mode styling * WIP: SidePanel Prompts * refactor: move to groups directory * refactor: rename GroupsSidePanel to GroupSidePanel and update imports * style: ListCard * refactor: isProduction changes * refactor: infinite query with productionPrompt * refactor: optimize snippets and prompts, and styling * refactor: Update getSnippet function to accept a length parameter * chore: localizations * feat: prompts navigation to chat and vice versa * fix: create prompt * feat: remember last selected category for creating prompts * fix(promptGroups): fix pagination and add usePromptGroupsNav hook * Prompts/frontend 3 (#3) * fix: stateful issues with prompt groups * style: improved layout * refactor: improve variable naming in Eng.ts * refactor: theme selector styling improvements * added prompt cards on chat new page, with dark mode, added API to fetch random prompts, added types for useQuery Slightly improved usePromptGroupNav logic to fetch updated result for pageSize, updated prompt cards view with darkmode and responsiveness fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page * WIP: Prompts/frontend (#4) * fix: optimize and fix paginated query * fix: remove unique constraint on names * refactor: button links and styling * style: menu border light mode * feat: Add Auto-Send Switch component for prompts groups * refactor(ChatView): use form context for submission text * chore: clear convo state on navigation to dashboard routes * chore: save prompt edit name on tab, remove console log * feat: basic prompt submission * refactor: move Auto-Send Switch * style(ListCard): border styling * feat: Add function to detect variables in text * feat: Add OriginalDialog component to UI library * chore(ui): Update SelectDropDown options list class to use text-xs size * refactor: submitMessage hook now includes submitPrompt, make compatible to document query selector * WIP: Variable Dialog * feat: variable submission working for both auto-send and non-autosend * feat: dashboard breadcrumbs and prompts/chat navigation * refactor: dashboard breadcrumb and dashboard link to chat navigation * refactor: Update VariableDialog and VariableForm styles * Prompts: Admin features (#5) * fix: link issue * fix: usePromptGroupsNav add missing dep. * style: dashbreadcrumb and sidepanel text color * temp fix: remove refetch on pageNumber change * fix: handle multiple variable replacement * WIP: create project schema and add project groups to fetch * feat: Add functionality to add prompt group IDs to a project * feat: Add caching for startup config in config route * chore: remove prompt landing * style: Update Skeleton component with additional background styling * chore: styling and types * WIP: SharePrompt first draft * feat(SharePrompt): form validation * feat: shared global indicators * refactor: prompt details * refactor: change NoPromptGroup directory * feat: preview prompt * feat: remove/add global prompts, add rbac-related enums * refactor: manage prompts location * WIP: first draft admin settings for prompts * feat: SystemRoles enum * refactor: update PromptDetails component styling * style: ellipsis custom class for showing more preview text * WIP: initial role schema and initialization * style: improved margins for single unordered lists * fix: use custom chat form context to prevent re-renders from FormProvider * feat: Role mutations for Prompt Permissions * feat: fetch user role * feat: update AdminSettings form default values from user role values * refactor: rename PromptPermissions to Permissions for general definitions * feat: initial role checks * feat: Add optional `bodyProps` parameter to generateCheckAccess middleware * refactor: UI access checks * Prompts: delete (#6) * Fixed delete prompt version API, fixed types and logic for prompt version deletion, updated prompt delete mutation logic * chore: Update return type of deletePrompt function in Prompt.js --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * chore: Update package-lock.json version to 0.7.4-rc1 and fast-xml-parser to 4.4.0 * feat: toast for saving admin settings, add timer no-access navigation * feat: always make prod * feat: Add localization to category labels in CategorySelector component * feat: Update category label localization in CategorySelector component * fix: Enable making prompt production in Prompt API --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * feat: Add helper fn for dark mode detection in ThemeProvider * style: surface-primary definition * fix(useHasAccess): utilize user.role and not just USER role * fix: empty category and role fetch * refactort: increase max height to options list and use label if no localization is found * fix: update CategorySelector to handle empty category value and improve localization * refactor: move prompts to own store/reactquery modules, add in filter WIP * refactor: Rename AutoSendSwitch to AutoSendPrompt * style: theming commit * style: fix slight coloring issue for convos in dark mode * style: better composition for prompts side panel * style: remove gray-750 and make it gray-850 * chore: adjust theming * feat: filter all prompt groups and properly remove prompts from projects * refactor: optimize delete prompt groups further * chore: localization * feat: Add uniqueProperty filtering to normalizeData function * WIP: filter prompts * chore: Update FilterPrompts component to include User icon in FilterItem * feat(FilterPrompts): set categories * feat: more system filters and show selected category icon * style: always make prod, flips switch to avoid mis-clicks * style: ui/ux loading/no prompts * chore: style FilterPrompts ChatView * fix: handle missing role edge case * style: special variables * feat: special variables * refactor: improve replaceSpecialVars function in prompts.ts * feat: simple/advanced editor modes * chore: bump versions * feat: localizations and hide production button on simple mode * fix: error connecting layout shift * fix: prompts CRUD for admins * fix: secure single group fetch * style: sidepanel styling * style(PromptName): bring edit button closer to name * style: mobile prompts header * style: mobile prompts header continued * style: align send prompts switch right * feat: description * Update special variables description in Eng.ts * feat: update/create/preview oneliner * fix: allow empty oneliner update * style: loading improvement and always make selected prompt Production if simple mode * fix: production index set and remove unused props * fix(ci): mock initializeRoles * fix: address #3128 * fix: address #3128 * feat: add deletion confirmation dialog * fix: mobile UI issues * style: prompt library UI update * style: focus, logcal tab order * style: Refactor SelectDropDown component to improve code readability and maintainability * chore: bump data-provider * chore: fix labels * refactor: confirm delete prompt version --------- Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
2024-06-20 20:24:32 -04:00
} = require('librechat-data-provider');
⚗️ feat: Agent Context Compaction/Summarization (#12287) * chore: imports/types Add summarization config and package-level summarize handler contracts Register summarize handlers across server controller paths Port cursor dual-read/dual-write summary support and UI status handling Selectively merge cursor branch files for BaseClient summary content block detection (last-summary-wins), dual-write persistence, summary block unit tests, and on_summarize_status SSE event handling with started/completed/failed branches. Co-authored-by: Cursor <cursoragent@cursor.com> refactor: type safety feat: add localization for summarization status messages refactor: optimize summary block detection in BaseClient Updated the logic for identifying existing summary content blocks to use a reverse loop for improved efficiency. Added a new test case to ensure the last summary content block is updated correctly when multiple summary blocks exist. chore: add runName to chainOptions in AgentClient refactor: streamline summarization configuration and handler integration Removed the deprecated summarizeNotConfigured function and replaced it with a more flexible createSummarizeFn. Updated the summarization handler setup across various controllers to utilize the new function, enhancing error handling and configuration resolution. Improved overall code clarity and maintainability by consolidating summarization logic. feat(summarization): add staged chunk-and-merge fallback feat(usage): track summarization usage separately from messages feat(summarization): resolve prompt from config in runtime fix(endpoints): use @librechat/api provider config loader refactor(agents): import getProviderConfig from @librechat/api chore: code order feat(app-config): auto-enable summarization when configured feat: summarization config refactor(summarization): streamline persist summary handling and enhance configuration validation Removed the deprecated createDeferredPersistSummary function and integrated a new createPersistSummary function for MongoDB persistence. Updated summarization handlers across various controllers to utilize the new persistence method. Enhanced validation for summarization configuration to ensure provider, model, and prompt are properly set, improving error handling and overall robustness. refactor(summarization): update event handling and remove legacy summarize handlers Replaced the deprecated summarization handlers with new event-driven handlers for summarization start and completion across multiple controllers. This change enhances the clarity of the summarization process and improves the integration of summarization events in the application. Additionally, removed unused summarization functions and streamlined the configuration loading process. refactor(summarization): standardize event names in handlers Updated event names in the summarization handlers to use constants from GraphEvents for consistency and clarity. This change improves maintainability and reduces the risk of errors related to string literals in event handling. feat(summarization): enhance usage tracking for summarization events Added logic to track summarization usage in multiple controllers by checking the current node type. If the node indicates a summarization task, the usage type is set accordingly. This change improves the granularity of usage data collected during summarization processes. feat(summarization): integrate SummarizationConfig into AppSummarizationConfig type Enhanced the AppSummarizationConfig type by extending it with the SummarizationConfig type from librechat-data-provider. This change improves type safety and consistency in the summarization configuration structure. test: add end-to-end tests for summarization functionality Introduced a comprehensive suite of end-to-end tests for the summarization feature, covering the full LibreChat pipeline from message creation to summarization. This includes a new setup file for environment configuration and a Jest configuration specifically for E2E tests. The tests utilize real API keys and ensure proper integration with the summarization process, enhancing overall test coverage and reliability. refactor(summarization): include initial summary in formatAgentMessages output Updated the formatAgentMessages function to return an initial summary alongside messages and index token count map. This change is reflected in multiple controllers and the corresponding tests, enhancing the summarization process by providing additional context for each agent's response. refactor: move hydrateMissingIndexTokenCounts to tokenMap utility Extracted the hydrateMissingIndexTokenCounts function from the AgentClient and related tests into a new tokenMap utility file. This change improves code organization and reusability, allowing for better management of token counting logic across the application. refactor(summarization): standardize step event handling and improve summary rendering Refactored the step event handling in the useStepHandler and related components to utilize constants for event names, enhancing consistency and maintainability. Additionally, improved the rendering logic in the Summary component to conditionally display the summary text based on its availability, providing a better user experience during the summarization process. feat(summarization): introduce baseContextTokens and reserveTokensRatio for improved context management Added baseContextTokens to the InitializedAgent type to calculate the context budget based on agentMaxContextNum and maxOutputTokensNum. Implemented reserveTokensRatio in the createRun function to allow configurable context token management. Updated related tests to validate these changes and ensure proper functionality. feat(summarization): add minReserveTokens, context pruning, and overflow recovery configurations Introduced new configuration options for summarization, including minReserveTokens, context pruning settings, and overflow recovery parameters. Updated the createRun function to accommodate these new options and added a comprehensive test suite to validate their functionality and integration within the summarization process. feat(summarization): add updatePrompt and reserveTokensRatio to summarization configuration Introduced an updatePrompt field for updating existing summaries with new messages, enhancing the flexibility of the summarization process. Additionally, added reserveTokensRatio to the configuration schema, allowing for improved management of token allocation during summarization. Updated related tests to validate these new features. feat(logging): add on_agent_log event handler for structured logging Implemented an on_agent_log event handler in both the agents' callbacks and responses to facilitate structured logging of agent activities. This enhancement allows for better tracking and debugging of agent interactions by logging messages with associated metadata. Updated the summarization process to ensure proper handling of log events. fix: remove duplicate IBalanceUpdate interface declaration perf(usage): single-pass partition of collectedUsage Replace two Array.filter() passes with a single for-of loop that partitions message vs. summarization usages in one iteration. fix(BaseClient): shallow-copy message content before mutating and preserve string content Avoid mutating the original message.content array in-place when appending a summary block. Also convert string content to a text content part instead of silently discarding it. fix(ui): fix Part.tsx indentation and useStepHandler summarize-complete handling - Fix SUMMARY else-if branch indentation in Part.tsx to match chain level - Guard ON_SUMMARIZE_COMPLETE with didFinalize flag to avoid unnecessary re-renders when no summarizing parts exist - Protect against undefined completeData.summary instead of unsafe spread fix(agents): use strict enabled check for summarization handlers Change summarizationConfig?.enabled !== false to === true so handlers are not registered when summarizationConfig is undefined. chore: fix initializeClient JSDoc and move DEFAULT_RESERVE_RATIO to module scope refactor(Summary): align collapse/expand behavior with Reasoning component - Single render path instead of separate streaming vs completed branches - Use useMessageContext for isSubmitting/isLatestMessage awareness so the "Summarizing..." label only shows during active streaming - Default to collapsed (matching Reasoning), user toggles to expand - Add proper aria attributes (aria-hidden, role, aria-controls, contentId) - Hide copy button while actively streaming feat(summarization): default to self-summarize using agent's own provider/model When no summarization config is provided (neither in librechat.yaml nor on the agent), automatically enable summarization using the agent's own provider and model. The agents package already provides default prompts, so no prompt configuration is needed. Also removes the dead resolveSummarizationLLMConfig in summarize.ts (and its spec) — run.ts buildAgentContext is the single source of truth for summarization config resolution. Removes the duplicate RuntimeSummarizationConfig local type in favor of the canonical SummarizationConfig from data-provider. chore: schema and type cleanup for summarization - Add trigger field to summarizationAgentOverrideSchema so per-agent trigger overrides in librechat.yaml are not silently stripped by Zod - Remove unused SummarizationStatus type from runs.ts - Make AppSummarizationConfig.enabled non-optional to reflect the invariant that loadSummarizationConfig always sets it refactor(responses): extract duplicated on_agent_log handler refactor(run): use agents package types for summarization config Import SummarizationConfig, ContextPruningConfig, and OverflowRecoveryConfig from @librechat/agents and use them to type-check the translation layer in buildAgentContext. This ensures the config object passed to the agent graph matches what it expects. - Use `satisfies AgentSummarizationConfig` on the config object - Cast contextPruningConfig and overflowRecoveryConfig to agents types - Properly narrow trigger fields from DeepPartial to required shape feat(config): add maxToolResultChars to base endpoint schema Add maxToolResultChars to baseEndpointSchema so it can be configured on any endpoint in librechat.yaml. Resolved during agent initialization using getProviderConfig's endpoint resolution: custom endpoint config takes precedence, then the provider-specific endpoint config, then the shared `all` config. Passed through to the agents package ToolNode, which uses it to cap tool result length before it enters the context window. When not configured, the agents package computes a sensible default from maxContextTokens. fix(summarization): forward agent model_parameters in self-summarize default When no explicit summarization config exists, the self-summarize default now forwards the agent's model_parameters as the summarization parameters. This ensures provider-specific settings (e.g. Bedrock region, credentials, endpoint host) are available when the agents package constructs the summarization LLM. fix(agents): register summarization handlers by default Change the enabled gate from === true to !== false so handlers register when no explicit summarization config exists. This aligns with the self-summarize default where summarization is always on unless explicitly disabled via enabled: false. refactor(summarization): let agents package inherit clientOptions for self-summarize Remove model_parameters forwarding from the self-summarize default. The agents package now reuses the agent's own clientOptions when the summarization provider matches the agent's provider, inheriting all provider-specific settings (region, credentials, proxy, etc.) automatically. refactor(summarization): use MessageContentComplex[] for summary content Unify summary content to always use MessageContentComplex[] arrays, matching the pattern used by on_message_delta. No more string | array unions — content is always an array of typed blocks ({ type: 'text', text: '...' } for text, { type: 'reasoning_content', ... } for reasoning). Agents package: - SummaryContentBlock.content: MessageContentComplex[] (was string) - tokenCount now optional (not sent on deltas) - Removed reasoning field — reasoning is now a content block type - streamAndCollect normalizes all chunks to content block arrays - Delta events pass content blocks directly LibreChat: - SummaryContentPart.content: Agents.MessageContentComplex[] - Updated Part.tsx, Summary.tsx, useStepHandler.ts, BaseClient.js - Summary.tsx derives display text from content blocks via useMemo - Aggregator uses simple array spread refactor(summarization): enhance summary handling and text extraction - Updated BaseClient.js to improve summary text extraction, accommodating both legacy and new content formats. - Modified summarization logic to ensure consistent handling of summary content across different message formats. - Adjusted test cases in summarization.e2e.spec.js to utilize the new summary text extraction method. - Refined SSE useStepHandler to initialize summary content as an array. - Updated configuration schema by removing unused minReserveTokens field. - Cleaned up SummaryContentPart type by removing rangeHash property. These changes streamline the summarization process and ensure compatibility with various content structures. refactor(summarization): streamline usage tracking and logging - Removed direct checks for summarization nodes in ModelEndHandler and replaced them with a dedicated markSummarizationUsage function for better readability and maintainability. - Updated OpenAIChatCompletionController and responses handlers to utilize the new markSummarizationUsage function for setting usage types. - Enhanced logging functionality by ensuring the logger correctly handles different log levels. - Introduced a new useCopyToClipboard hook in the Summary component to encapsulate clipboard copy logic, improving code reusability and clarity. These changes improve the overall structure and efficiency of the summarization handling and logging processes. refactor(summarization): update summary content block documentation - Removed outdated comment regarding the last summary content block in BaseClient.js. - Added a new comment to clarify the purpose of the findSummaryContentBlock method, ensuring consistency in documentation. These changes enhance code clarity and maintainability by providing accurate descriptions of the summarization logic. refactor(summarization): update summary content structure in tests - Modified the summarization content structure in e2e tests to use an array format for text, aligning with recent changes in summary handling. - Updated test descriptions to clarify the behavior of context token calculations, ensuring consistency and clarity in the tests. These changes enhance the accuracy and maintainability of the summarization tests by reflecting the updated content structure. refactor(summarization): remove legacy E2E test setup and configuration - Deleted the e2e-setup.js and jest.e2e.config.js files, which contained legacy configurations for E2E tests using real API keys. - Introduced a new summarization.e2e.ts file that implements comprehensive E2E backend integration tests for the summarization process, utilizing real AI providers and tracking summaries throughout the run. These changes streamline the testing framework by consolidating E2E tests into a single, more robust file while removing outdated configurations. refactor(summarization): enhance E2E tests and error handling - Added a cleanup step to force exit after all tests to manage Redis connections. - Updated the summarization model to 'claude-haiku-4-5-20251001' for consistency across tests. - Improved error handling in the processStream function to capture and return processing errors. - Enhanced logging for cross-run tests and tight context scenarios to provide better insights into test execution. These changes improve the reliability and clarity of the E2E tests for the summarization process. refactor(summarization): enhance test coverage for maxContextTokens behavior - Updated run-summarization.test.ts to include a new test case ensuring that maxContextTokens does not exceed user-defined limits, even when calculated ratios suggest otherwise. - Modified summarization.e2e.ts to replace legacy UsageMetadata type with a more appropriate type for collectedUsage, improving type safety and clarity in the test setup. These changes improve the robustness of the summarization tests by validating context token constraints and refining type definitions. feat(summarization): add comprehensive E2E tests for summarization process - Introduced a new summarization.e2e.test.ts file that implements extensive end-to-end integration tests for the summarization pipeline, covering the full flow from LibreChat to agents. - The tests utilize real AI providers and include functionality to track summaries during and between runs. - Added necessary cleanup steps to manage Redis connections post-tests and ensure proper exit. These changes enhance the testing framework by providing robust coverage for the summarization process, ensuring reliability and performance under real-world conditions. fix(service): import logger from winston configuration - Removed the import statement for logger from '@librechat/data-schemas' and replaced it with an import from '~/config/winston'. - This change ensures that the logger is correctly sourced from the updated configuration, improving consistency in logging practices across the application. refactor(summary): simplify Summary component and enhance token display - Removed the unused `meta` prop from the `SummaryButton` component to streamline its interface. - Updated the token display logic to use a localized string for better internationalization support. - Adjusted the rendering of the `meta` information to improve its visibility within the `Summary` component. These changes enhance the clarity and usability of the Summary component while ensuring better localization practices. feat(summarization): add maxInputTokens configuration for summarization - Introduced a new `maxInputTokens` property in the summarization configuration schema to control the amount of conversation context sent to the summarizer, with a default value of 10000. - Updated the `createRun` function to utilize the new `maxInputTokens` setting, allowing for more flexible summarization based on agent context. These changes enhance the summarization capabilities by providing better control over input token limits, improving the overall summarization process. refactor(summarization): simplify maxInputTokens logic in createRun function - Updated the logic for the `maxInputTokens` property in the `createRun` function to directly use the agent's base context tokens when the resolved summarization configuration does not specify a value. - This change streamlines the configuration process and enhances clarity in how input token limits are determined for summarization. These modifications improve the maintainability of the summarization configuration by reducing complexity in the token calculation logic. feat(summary): enhance Summary component to display meta information - Updated the SummaryContent component to accept an optional `meta` prop, allowing for additional contextual information to be displayed above the main content. - Adjusted the rendering logic in the Summary component to utilize the new `meta` prop, improving the visibility of supplementary details. These changes enhance the user experience by providing more context within the Summary component, making it clearer and more informative. refactor(summarization): standardize reserveRatio configuration in summarization logic - Replaced instances of `reserveTokensRatio` with `reserveRatio` in the `createRun` function and related tests to unify the terminology across the codebase. - Updated the summarization configuration schema to reflect this change, ensuring consistency in how the reserve ratio is defined and utilized. - Removed the per-agent override logic for summarization configuration, simplifying the overall structure and enhancing clarity. These modifications improve the maintainability and readability of the summarization logic by standardizing the configuration parameters. * fix: circular dependency of `~/models` * chore: update logging scope in agent log handlers Changed log scope from `[agentus:${data.scope}]` to `[agents:${data.scope}]` in both the callbacks and responses controllers to ensure consistent logging format across the application. * feat: calibration ratio * refactor(tests): update summarizationConfig tests to reflect changes in enabled property Modified tests to check for the new `summarizationEnabled` property instead of the deprecated `enabled` field in the summarization configuration. This change ensures that the tests accurately validate the current configuration structure and behavior of the agents. * feat(tests): add markSummarizationUsage mock for improved test coverage Introduced a mock for the markSummarizationUsage function in the responses unit tests to enhance the testing of summarization usage tracking. This addition supports better validation of summarization-related functionalities and ensures comprehensive test coverage for the agents' response handling. * refactor(tests): simplify event handler setup in createResponse tests Removed redundant mock implementations for event handlers in the createResponse unit tests, streamlining the setup process. This change enhances test clarity and maintainability while ensuring that the tests continue to validate the correct behavior of usage tracking during on_chat_model_end events. * refactor(agents): move calibration ratio capture to finally block Reorganized the logic for capturing the calibration ratio in the AgentClient class to ensure it is executed in the finally block. This change guarantees that the ratio is captured even if the run is aborted, enhancing the reliability of the response message persistence. Removed redundant code and improved clarity in the handling of context metadata. * refactor(agents): streamline bulk write logic in recordCollectedUsage function Removed redundant bulk write operations and consolidated document handling in the recordCollectedUsage function. The logic now combines all documents into a single bulk write operation, improving efficiency and reducing error handling complexity. Updated logging to provide consistent error messages for bulk write failures. * refactor(agents): enhance summarization configuration resolution in createRun function Streamlined the summarization configuration logic by introducing a base configuration and allowing for overrides from agent-specific settings. This change improves clarity and maintainability, ensuring that the summarization configuration is consistently applied while retaining flexibility for customization. Updated the handling of summarization parameters to ensure proper integration with the agent's model and provider settings. * refactor(agents): remove unused tokenCountMap and streamline calibration ratio handling Eliminated the unused tokenCountMap variable from the AgentClient class to enhance code clarity. Additionally, streamlined the logic for capturing the calibration ratio by using optional chaining and a fallback value, ensuring that context metadata is consistently defined. This change improves maintainability and reduces potential confusion in the codebase. * refactor(agents): extract agent log handler for improved clarity and reusability Refactored the agent log handling logic by extracting it into a dedicated function, `agentLogHandler`, enhancing code clarity and reusability across different modules. Updated the event handlers in both the OpenAI and responses controllers to utilize the new handler, ensuring consistent logging behavior throughout the application. * test: add summarization event tests for useStepHandler Implemented a series of tests for the summarization events in the useStepHandler hook. The tests cover scenarios for ON_SUMMARIZE_START, ON_SUMMARIZE_DELTA, and ON_SUMMARIZE_COMPLETE events, ensuring proper handling of summarization logic, including message accumulation and finalization. This addition enhances test coverage and validates the correct behavior of the summarization process within the application. * refactor(config): update summarizationTriggerSchema to use enum for type validation Changed the type of the `type` field in the summarizationTriggerSchema from a string to an enum with a single value 'token_count'. This modification enhances type safety and ensures that only valid types are accepted in the configuration, improving overall clarity and maintainability of the schema. * test(usage): add bulk write tests for message and summarization usage Implemented tests for the bulk write functionality in the recordCollectedUsage function, covering scenarios for combined message and summarization usage, summarization-only usage, and message-only usage. These tests ensure correct document handling and token rollup calculations, enhancing test coverage and validating the behavior of the usage tracking logic. * refactor(Chat): enhance clipboard copy functionality and type definitions in Summary component Updated the Summary component to improve the clipboard copy functionality by handling clipboard permission errors. Refactored type definitions for SummaryProps to use a more specific type, enhancing type safety. Adjusted the SummaryButton and FloatingSummaryBar components to accept isCopied and onCopy props, promoting better separation of concerns and reusability. * chore(translations): remove unused "Expand Summary" key from English translations Deleted the "Expand Summary" key from the English translation file to streamline the localization resources and improve clarity in the user interface. This change helps maintain an organized and efficient translation structure. * refactor: adjust token counting for Claude model to account for API discrepancies Implemented a correction factor for token counting when using the Claude model, addressing discrepancies between Anthropic's API and local tokenizer results. This change ensures accurate token counts by applying a scaling factor, improving the reliability of token-related functionalities. * refactor(agents): implement token count adjustment for Claude model messages Added a method to adjust token counts for messages processed by the Claude model, applying a correction factor to align with API expectations. This enhancement improves the accuracy of token counting, ensuring reliable functionality when interacting with the Claude model. * refactor(agents): token counting for media content in messages Introduced a new method to estimate token costs for image and document blocks in messages, improving the accuracy of token counting. This enhancement ensures that media content is properly accounted for, particularly for the Claude model, by integrating additional token estimation logic for various content types. Updated the token counting function to utilize this new method, enhancing overall reliability and functionality. * chore: fix missing import * fix(agents): clamp baseContextTokens and document reserve ratio change Prevent negative baseContextTokens when maxOutputTokens exceeds the context window (misconfigured models). Document the 10%→5% default reserve ratio reduction introduced alongside summarization. * fix(agents): include media tokens in hydrated token counts Add estimateMediaTokensForMessage to createTokenCounter so the hydration path (used by hydrateMissingIndexTokenCounts) matches the precomputed path in AgentClient.getTokenCountForMessage. Without this, messages containing images or documents were systematically undercounted during hydration, risking context window overflow. Add 34 unit tests covering all block-type branches of estimateMediaTokensForMessage. * fix(agents): include summarization output tokens in usage return value The returned output_tokens from recordCollectedUsage now reflects all billed LLM calls (message + summarization). Previously, summarization completions were billed but excluded from the returned metadata, causing a discrepancy between what users were charged and what the response message reported. * fix(tests): replace process.exit with proper Redis cleanup in e2e test The summarization E2E test used process.exit(0) to work around a Redis connection opened at import time, which killed the Jest runner and bypassed teardown. Use ioredisClient.quit() and keyvRedisClient.disconnect() for graceful cleanup instead. * fix(tests): update getConvo imports in OpenAI and response tests Refactor test files to import getConvo from the main models module instead of the Conversation submodule. This change ensures consistency across tests and simplifies the import structure, enhancing maintainability. * fix(clients): improve summary text validation in BaseClient Refactor the summary extraction logic to ensure that only non-empty summary texts are considered valid. This change enhances the robustness of the message processing by utilizing a dedicated method for summary text retrieval, improving overall reliability. * fix(config): replace z.any() with explicit union in summarization schema Model parameters (temperature, top_p, etc.) are constrained to primitive types rather than the policy-violating z.any(). * refactor(agents): deduplicate CLAUDE_TOKEN_CORRECTION constant Export from the TS source in packages/api and import in the JS client, eliminating the static class property that could drift out of sync. * refactor(agents): eliminate duplicate selfProvider in buildAgentContext selfProvider and provider were derived from the same expression with different type casts. Consolidated to a single provider variable. * refactor(agents): extract shared SSE handlers and restrict log levels - buildSummarizationHandlers() factory replaces triplicated handler blocks across responses.js and openai.js - agentLogHandlerObj exported from callbacks.js for consistent reuse - agentLogHandler restricted to an allowlist of safe log levels (debug, info, warn, error) instead of accepting arbitrary strings * fix(SSE): batch summarize deltas, add exhaustiveness check, conditional error announcement - ON_SUMMARIZE_DELTA coalesces rapid-fire renders via requestAnimationFrame instead of calling setMessages per chunk - Exhaustive never-check on TStepEvent catches unhandled variants at compile time when new StepEvents are added - ON_SUMMARIZE_COMPLETE error announcement only fires when a summary part was actually present and removed * feat(agents): persist instruction overhead in contextMeta and seed across runs Extend contextMeta with instructionOverhead and toolCount so the provider-observed instruction overhead is persisted on the response message and seeded into the pruner on subsequent runs. This enables the pruner to use a calibrated budget from the first call instead of waiting for a provider observation, preventing the ratio collapse caused by local tokenizer overestimating tool schema tokens. The seeded overhead is only used when encoding and tool count match between runs, ensuring stale values from different configurations are discarded. * test(agents): enhance OpenAI test mocks for summarization handlers Updated the OpenAI test suite to include additional mock implementations for summarization handlers, including buildSummarizationHandlers, markSummarizationUsage, and agentLogHandlerObj. This improves test coverage and ensures consistent behavior during testing. * fix(agents): address review findings for summarization v2 Cancel rAF on unmount to prevent stale Recoil writes from dead component context. Clear orphaned summarizing:true parts when ON_SUMMARIZE_COMPLETE arrives without a summary payload. Add null guard and safe spread to agentLogHandler. Handle Anthropic-format base64 image/* documents in estimateMediaTokensForMessage. Use role="region" for expandable summary content. Add .describe() to contextMeta Zod fields. Extract duplicate usage loop into helper. * refactor: simplify contextMeta to calibrationRatio + encoding only Remove instructionOverhead and toolCount from cross-run persistence — instruction tokens change too frequently between runs (prompt edits, tool changes) for a persisted seed to be reliable. The intra-run calibration in the pruner still self-corrects via provider observations. contextMeta now stores only the tokenizer-bias ratio and encoding, which are stable across instruction changes. * test(SSE): enhance useStepHandler tests for ON_SUMMARIZE_COMPLETE behavior Updated the test for ON_SUMMARIZE_COMPLETE to clarify that it finalizes the existing part with summarizing set to false when the summary is undefined. Added assertions to verify the correct behavior of message updates and the state of summary parts. * refactor(BaseClient): remove handleContextStrategy and truncateToolCallOutputs functions Eliminated the handleContextStrategy method from BaseClient to streamline message handling. Also removed the truncateToolCallOutputs function from the prompts module, simplifying the codebase and improving maintainability. * refactor: add AGENT_DEBUG_LOGGING option and refactor token count handling in BaseClient Introduced AGENT_DEBUG_LOGGING to .env.example for enhanced debugging capabilities. Refactored token count handling in BaseClient by removing the handleTokenCountMap method and simplifying token count updates. Updated AgentClient to log detailed token count recalculations and adjustments, improving traceability during message processing. * chore: update dependencies in package-lock.json and package.json files Bumped versions of several dependencies, including @librechat/agents to ^3.1.62 and various AWS SDK packages to their latest versions. This ensures compatibility and incorporates the latest features and fixes. * chore: imports order * refactor: extract summarization config resolution from buildAgentContext * refactor: rename and simplify summarization configuration shaping function * refactor: replace AgentClient token counting methods with single-pass pure utility Extract getTokenCount() and getTokenCountForMessage() from AgentClient into countFormattedMessageTokens(), a pure function in packages/api that handles text, tool_call, image, and document content types in one loop. - Decompose estimateMediaTokensForMessage into block-level helpers (estimateImageDataTokens, estimateImageBlockTokens, estimateDocumentBlockTokens) shared by both estimateMediaTokensForMessage and the new single-pass function - Remove redundant per-call getEncoding() resolution (closure captures once) - Remove deprecated gpt-3.5-turbo-0301 model branching - Drop this.getTokenCount guard from BaseClient.sendMessage * refactor: streamline token counting in createTokenCounter function Simplified the createTokenCounter function by removing the media token estimation and directly calculating the token count. This change enhances clarity and performance by consolidating the token counting logic into a single pass, while maintaining compatibility with Claude's token correction. * refactor: simplify summarization configuration types Removed the AppSummarizationConfig type and directly used SummarizationConfig in the AppConfig interface. This change streamlines the type definitions and enhances consistency across the codebase. * chore: import order * fix: summarization event handling in useStepHandler - Cancel pending summarizeDeltaRaf in clearStepMaps to prevent stale frames firing after map reset or component unmount - Move announcePolite('summarize_completed') inside the didFinalize guard so screen readers only announce when finalization actually occurs - Remove dead cleanup closure returned from stepHandler useCallback body that was never invoked by any caller * fix: estimate tokens for non-PDF/non-image base64 document blocks Previously estimateDocumentBlockTokens returned 0 for unrecognized MIME types (e.g. text/plain, application/json), silently underestimating context budget. Fall back to character-based heuristic or countTokens. * refactor: return cloned usage from markSummarizationUsage Avoid mutating LangChain's internal usage_metadata object by returning a shallow clone with the usage_type tag. Update all call sites in callbacks, openai, and responses controllers to use the returned value. * refactor: consolidate debug logging loops in buildMessages Merge the two sequential O(n) debug-logging passes over orderedMessages into a single pass inside the map callback where all data is available. * refactor: narrow SummaryContentPart.content type Replace broad Agents.MessageContentComplex[] with the specific Array<{ type: ContentTypes.TEXT; text: string }> that all producers and consumers already use, improving compile-time safety. * refactor: use single output array in recordCollectedUsage Have processUsageGroup append to a shared array instead of returning separate arrays that are spread into a third, reducing allocations. * refactor: use for...in in hydrateMissingIndexTokenCounts Replace Object.entries with for...in to avoid allocating an intermediate tuple array during token map hydration.
2026-03-21 12:03:10 -04:00
const { hasCapability, requireCapability } = require('~/server/middleware/roles/capabilities');
📦 refactor: Consolidate DB models, encapsulating Mongoose usage in `data-schemas` (#11830) * chore: move database model methods to /packages/data-schemas * chore: add TypeScript ESLint rule to warn on unused variables * refactor: model imports to streamline access - Consolidated model imports across various files to improve code organization and reduce redundancy. - Updated imports for models such as Assistant, Message, Conversation, and others to a unified import path. - Adjusted middleware and service files to reflect the new import structure, ensuring functionality remains intact. - Enhanced test files to align with the new import paths, maintaining test coverage and integrity. * chore: migrate database models to packages/data-schemas and refactor all direct Mongoose Model usage outside of data-schemas * test: update agent model mocks in unit tests - Added `getAgent` mock to `client.test.js` to enhance test coverage for agent-related functionality. - Removed redundant `getAgent` and `getAgents` mocks from `openai.spec.js` and `responses.unit.spec.js` to streamline test setup and reduce duplication. - Ensured consistency in agent mock implementations across test files. * fix: update types in data-schemas * refactor: enhance type definitions in transaction and spending methods - Updated type definitions in `checkBalance.ts` to use specific request and response types. - Refined `spendTokens.ts` to utilize a new `SpendTxData` interface for better clarity and type safety. - Improved transaction handling in `transaction.ts` by introducing `TransactionResult` and `TxData` interfaces, ensuring consistent data structures across methods. - Adjusted unit tests in `transaction.spec.ts` to accommodate new type definitions and enhance robustness. * refactor: streamline model imports and enhance code organization - Consolidated model imports across various controllers and services to a unified import path, improving code clarity and reducing redundancy. - Updated multiple files to reflect the new import structure, ensuring all functionalities remain intact. - Enhanced overall code organization by removing duplicate import statements and optimizing the usage of model methods. * feat: implement loadAddedAgent and refactor agent loading logic - Introduced `loadAddedAgent` function to handle loading agents from added conversations, supporting multi-convo parallel execution. - Created a new `load.ts` file to encapsulate agent loading functionalities, including `loadEphemeralAgent` and `loadAgent`. - Updated the `index.ts` file to export the new `load` module instead of the deprecated `loadAgent`. - Enhanced type definitions and improved error handling in the agent loading process. - Adjusted unit tests to reflect changes in the agent loading structure and ensure comprehensive coverage. * refactor: enhance balance handling with new update interface - Introduced `IBalanceUpdate` interface to streamline balance update operations across the codebase. - Updated `upsertBalanceFields` method signatures in `balance.ts`, `transaction.ts`, and related tests to utilize the new interface for improved type safety. - Adjusted type imports in `balance.spec.ts` to include `IBalanceUpdate`, ensuring consistency in balance management functionalities. - Enhanced overall code clarity and maintainability by refining type definitions related to balance operations. * feat: add unit tests for loadAgent functionality and enhance agent loading logic - Introduced comprehensive unit tests for the `loadAgent` function, covering various scenarios including null and empty agent IDs, loading of ephemeral agents, and permission checks. - Enhanced the `initializeClient` function by moving `getConvoFiles` to the correct position in the database method exports, ensuring proper functionality. - Improved test coverage for agent loading, including handling of non-existent agents and user permissions. * chore: reorder memory method exports for consistency - Moved `deleteAllUserMemories` to the correct position in the exported memory methods, ensuring a consistent and logical order of method exports in `memory.ts`.
2026-02-17 18:23:44 -05:00
const { updateRoleByName, getRoleByName } = require('~/models');
⚗️ feat: Agent Context Compaction/Summarization (#12287) * chore: imports/types Add summarization config and package-level summarize handler contracts Register summarize handlers across server controller paths Port cursor dual-read/dual-write summary support and UI status handling Selectively merge cursor branch files for BaseClient summary content block detection (last-summary-wins), dual-write persistence, summary block unit tests, and on_summarize_status SSE event handling with started/completed/failed branches. Co-authored-by: Cursor <cursoragent@cursor.com> refactor: type safety feat: add localization for summarization status messages refactor: optimize summary block detection in BaseClient Updated the logic for identifying existing summary content blocks to use a reverse loop for improved efficiency. Added a new test case to ensure the last summary content block is updated correctly when multiple summary blocks exist. chore: add runName to chainOptions in AgentClient refactor: streamline summarization configuration and handler integration Removed the deprecated summarizeNotConfigured function and replaced it with a more flexible createSummarizeFn. Updated the summarization handler setup across various controllers to utilize the new function, enhancing error handling and configuration resolution. Improved overall code clarity and maintainability by consolidating summarization logic. feat(summarization): add staged chunk-and-merge fallback feat(usage): track summarization usage separately from messages feat(summarization): resolve prompt from config in runtime fix(endpoints): use @librechat/api provider config loader refactor(agents): import getProviderConfig from @librechat/api chore: code order feat(app-config): auto-enable summarization when configured feat: summarization config refactor(summarization): streamline persist summary handling and enhance configuration validation Removed the deprecated createDeferredPersistSummary function and integrated a new createPersistSummary function for MongoDB persistence. Updated summarization handlers across various controllers to utilize the new persistence method. Enhanced validation for summarization configuration to ensure provider, model, and prompt are properly set, improving error handling and overall robustness. refactor(summarization): update event handling and remove legacy summarize handlers Replaced the deprecated summarization handlers with new event-driven handlers for summarization start and completion across multiple controllers. This change enhances the clarity of the summarization process and improves the integration of summarization events in the application. Additionally, removed unused summarization functions and streamlined the configuration loading process. refactor(summarization): standardize event names in handlers Updated event names in the summarization handlers to use constants from GraphEvents for consistency and clarity. This change improves maintainability and reduces the risk of errors related to string literals in event handling. feat(summarization): enhance usage tracking for summarization events Added logic to track summarization usage in multiple controllers by checking the current node type. If the node indicates a summarization task, the usage type is set accordingly. This change improves the granularity of usage data collected during summarization processes. feat(summarization): integrate SummarizationConfig into AppSummarizationConfig type Enhanced the AppSummarizationConfig type by extending it with the SummarizationConfig type from librechat-data-provider. This change improves type safety and consistency in the summarization configuration structure. test: add end-to-end tests for summarization functionality Introduced a comprehensive suite of end-to-end tests for the summarization feature, covering the full LibreChat pipeline from message creation to summarization. This includes a new setup file for environment configuration and a Jest configuration specifically for E2E tests. The tests utilize real API keys and ensure proper integration with the summarization process, enhancing overall test coverage and reliability. refactor(summarization): include initial summary in formatAgentMessages output Updated the formatAgentMessages function to return an initial summary alongside messages and index token count map. This change is reflected in multiple controllers and the corresponding tests, enhancing the summarization process by providing additional context for each agent's response. refactor: move hydrateMissingIndexTokenCounts to tokenMap utility Extracted the hydrateMissingIndexTokenCounts function from the AgentClient and related tests into a new tokenMap utility file. This change improves code organization and reusability, allowing for better management of token counting logic across the application. refactor(summarization): standardize step event handling and improve summary rendering Refactored the step event handling in the useStepHandler and related components to utilize constants for event names, enhancing consistency and maintainability. Additionally, improved the rendering logic in the Summary component to conditionally display the summary text based on its availability, providing a better user experience during the summarization process. feat(summarization): introduce baseContextTokens and reserveTokensRatio for improved context management Added baseContextTokens to the InitializedAgent type to calculate the context budget based on agentMaxContextNum and maxOutputTokensNum. Implemented reserveTokensRatio in the createRun function to allow configurable context token management. Updated related tests to validate these changes and ensure proper functionality. feat(summarization): add minReserveTokens, context pruning, and overflow recovery configurations Introduced new configuration options for summarization, including minReserveTokens, context pruning settings, and overflow recovery parameters. Updated the createRun function to accommodate these new options and added a comprehensive test suite to validate their functionality and integration within the summarization process. feat(summarization): add updatePrompt and reserveTokensRatio to summarization configuration Introduced an updatePrompt field for updating existing summaries with new messages, enhancing the flexibility of the summarization process. Additionally, added reserveTokensRatio to the configuration schema, allowing for improved management of token allocation during summarization. Updated related tests to validate these new features. feat(logging): add on_agent_log event handler for structured logging Implemented an on_agent_log event handler in both the agents' callbacks and responses to facilitate structured logging of agent activities. This enhancement allows for better tracking and debugging of agent interactions by logging messages with associated metadata. Updated the summarization process to ensure proper handling of log events. fix: remove duplicate IBalanceUpdate interface declaration perf(usage): single-pass partition of collectedUsage Replace two Array.filter() passes with a single for-of loop that partitions message vs. summarization usages in one iteration. fix(BaseClient): shallow-copy message content before mutating and preserve string content Avoid mutating the original message.content array in-place when appending a summary block. Also convert string content to a text content part instead of silently discarding it. fix(ui): fix Part.tsx indentation and useStepHandler summarize-complete handling - Fix SUMMARY else-if branch indentation in Part.tsx to match chain level - Guard ON_SUMMARIZE_COMPLETE with didFinalize flag to avoid unnecessary re-renders when no summarizing parts exist - Protect against undefined completeData.summary instead of unsafe spread fix(agents): use strict enabled check for summarization handlers Change summarizationConfig?.enabled !== false to === true so handlers are not registered when summarizationConfig is undefined. chore: fix initializeClient JSDoc and move DEFAULT_RESERVE_RATIO to module scope refactor(Summary): align collapse/expand behavior with Reasoning component - Single render path instead of separate streaming vs completed branches - Use useMessageContext for isSubmitting/isLatestMessage awareness so the "Summarizing..." label only shows during active streaming - Default to collapsed (matching Reasoning), user toggles to expand - Add proper aria attributes (aria-hidden, role, aria-controls, contentId) - Hide copy button while actively streaming feat(summarization): default to self-summarize using agent's own provider/model When no summarization config is provided (neither in librechat.yaml nor on the agent), automatically enable summarization using the agent's own provider and model. The agents package already provides default prompts, so no prompt configuration is needed. Also removes the dead resolveSummarizationLLMConfig in summarize.ts (and its spec) — run.ts buildAgentContext is the single source of truth for summarization config resolution. Removes the duplicate RuntimeSummarizationConfig local type in favor of the canonical SummarizationConfig from data-provider. chore: schema and type cleanup for summarization - Add trigger field to summarizationAgentOverrideSchema so per-agent trigger overrides in librechat.yaml are not silently stripped by Zod - Remove unused SummarizationStatus type from runs.ts - Make AppSummarizationConfig.enabled non-optional to reflect the invariant that loadSummarizationConfig always sets it refactor(responses): extract duplicated on_agent_log handler refactor(run): use agents package types for summarization config Import SummarizationConfig, ContextPruningConfig, and OverflowRecoveryConfig from @librechat/agents and use them to type-check the translation layer in buildAgentContext. This ensures the config object passed to the agent graph matches what it expects. - Use `satisfies AgentSummarizationConfig` on the config object - Cast contextPruningConfig and overflowRecoveryConfig to agents types - Properly narrow trigger fields from DeepPartial to required shape feat(config): add maxToolResultChars to base endpoint schema Add maxToolResultChars to baseEndpointSchema so it can be configured on any endpoint in librechat.yaml. Resolved during agent initialization using getProviderConfig's endpoint resolution: custom endpoint config takes precedence, then the provider-specific endpoint config, then the shared `all` config. Passed through to the agents package ToolNode, which uses it to cap tool result length before it enters the context window. When not configured, the agents package computes a sensible default from maxContextTokens. fix(summarization): forward agent model_parameters in self-summarize default When no explicit summarization config exists, the self-summarize default now forwards the agent's model_parameters as the summarization parameters. This ensures provider-specific settings (e.g. Bedrock region, credentials, endpoint host) are available when the agents package constructs the summarization LLM. fix(agents): register summarization handlers by default Change the enabled gate from === true to !== false so handlers register when no explicit summarization config exists. This aligns with the self-summarize default where summarization is always on unless explicitly disabled via enabled: false. refactor(summarization): let agents package inherit clientOptions for self-summarize Remove model_parameters forwarding from the self-summarize default. The agents package now reuses the agent's own clientOptions when the summarization provider matches the agent's provider, inheriting all provider-specific settings (region, credentials, proxy, etc.) automatically. refactor(summarization): use MessageContentComplex[] for summary content Unify summary content to always use MessageContentComplex[] arrays, matching the pattern used by on_message_delta. No more string | array unions — content is always an array of typed blocks ({ type: 'text', text: '...' } for text, { type: 'reasoning_content', ... } for reasoning). Agents package: - SummaryContentBlock.content: MessageContentComplex[] (was string) - tokenCount now optional (not sent on deltas) - Removed reasoning field — reasoning is now a content block type - streamAndCollect normalizes all chunks to content block arrays - Delta events pass content blocks directly LibreChat: - SummaryContentPart.content: Agents.MessageContentComplex[] - Updated Part.tsx, Summary.tsx, useStepHandler.ts, BaseClient.js - Summary.tsx derives display text from content blocks via useMemo - Aggregator uses simple array spread refactor(summarization): enhance summary handling and text extraction - Updated BaseClient.js to improve summary text extraction, accommodating both legacy and new content formats. - Modified summarization logic to ensure consistent handling of summary content across different message formats. - Adjusted test cases in summarization.e2e.spec.js to utilize the new summary text extraction method. - Refined SSE useStepHandler to initialize summary content as an array. - Updated configuration schema by removing unused minReserveTokens field. - Cleaned up SummaryContentPart type by removing rangeHash property. These changes streamline the summarization process and ensure compatibility with various content structures. refactor(summarization): streamline usage tracking and logging - Removed direct checks for summarization nodes in ModelEndHandler and replaced them with a dedicated markSummarizationUsage function for better readability and maintainability. - Updated OpenAIChatCompletionController and responses handlers to utilize the new markSummarizationUsage function for setting usage types. - Enhanced logging functionality by ensuring the logger correctly handles different log levels. - Introduced a new useCopyToClipboard hook in the Summary component to encapsulate clipboard copy logic, improving code reusability and clarity. These changes improve the overall structure and efficiency of the summarization handling and logging processes. refactor(summarization): update summary content block documentation - Removed outdated comment regarding the last summary content block in BaseClient.js. - Added a new comment to clarify the purpose of the findSummaryContentBlock method, ensuring consistency in documentation. These changes enhance code clarity and maintainability by providing accurate descriptions of the summarization logic. refactor(summarization): update summary content structure in tests - Modified the summarization content structure in e2e tests to use an array format for text, aligning with recent changes in summary handling. - Updated test descriptions to clarify the behavior of context token calculations, ensuring consistency and clarity in the tests. These changes enhance the accuracy and maintainability of the summarization tests by reflecting the updated content structure. refactor(summarization): remove legacy E2E test setup and configuration - Deleted the e2e-setup.js and jest.e2e.config.js files, which contained legacy configurations for E2E tests using real API keys. - Introduced a new summarization.e2e.ts file that implements comprehensive E2E backend integration tests for the summarization process, utilizing real AI providers and tracking summaries throughout the run. These changes streamline the testing framework by consolidating E2E tests into a single, more robust file while removing outdated configurations. refactor(summarization): enhance E2E tests and error handling - Added a cleanup step to force exit after all tests to manage Redis connections. - Updated the summarization model to 'claude-haiku-4-5-20251001' for consistency across tests. - Improved error handling in the processStream function to capture and return processing errors. - Enhanced logging for cross-run tests and tight context scenarios to provide better insights into test execution. These changes improve the reliability and clarity of the E2E tests for the summarization process. refactor(summarization): enhance test coverage for maxContextTokens behavior - Updated run-summarization.test.ts to include a new test case ensuring that maxContextTokens does not exceed user-defined limits, even when calculated ratios suggest otherwise. - Modified summarization.e2e.ts to replace legacy UsageMetadata type with a more appropriate type for collectedUsage, improving type safety and clarity in the test setup. These changes improve the robustness of the summarization tests by validating context token constraints and refining type definitions. feat(summarization): add comprehensive E2E tests for summarization process - Introduced a new summarization.e2e.test.ts file that implements extensive end-to-end integration tests for the summarization pipeline, covering the full flow from LibreChat to agents. - The tests utilize real AI providers and include functionality to track summaries during and between runs. - Added necessary cleanup steps to manage Redis connections post-tests and ensure proper exit. These changes enhance the testing framework by providing robust coverage for the summarization process, ensuring reliability and performance under real-world conditions. fix(service): import logger from winston configuration - Removed the import statement for logger from '@librechat/data-schemas' and replaced it with an import from '~/config/winston'. - This change ensures that the logger is correctly sourced from the updated configuration, improving consistency in logging practices across the application. refactor(summary): simplify Summary component and enhance token display - Removed the unused `meta` prop from the `SummaryButton` component to streamline its interface. - Updated the token display logic to use a localized string for better internationalization support. - Adjusted the rendering of the `meta` information to improve its visibility within the `Summary` component. These changes enhance the clarity and usability of the Summary component while ensuring better localization practices. feat(summarization): add maxInputTokens configuration for summarization - Introduced a new `maxInputTokens` property in the summarization configuration schema to control the amount of conversation context sent to the summarizer, with a default value of 10000. - Updated the `createRun` function to utilize the new `maxInputTokens` setting, allowing for more flexible summarization based on agent context. These changes enhance the summarization capabilities by providing better control over input token limits, improving the overall summarization process. refactor(summarization): simplify maxInputTokens logic in createRun function - Updated the logic for the `maxInputTokens` property in the `createRun` function to directly use the agent's base context tokens when the resolved summarization configuration does not specify a value. - This change streamlines the configuration process and enhances clarity in how input token limits are determined for summarization. These modifications improve the maintainability of the summarization configuration by reducing complexity in the token calculation logic. feat(summary): enhance Summary component to display meta information - Updated the SummaryContent component to accept an optional `meta` prop, allowing for additional contextual information to be displayed above the main content. - Adjusted the rendering logic in the Summary component to utilize the new `meta` prop, improving the visibility of supplementary details. These changes enhance the user experience by providing more context within the Summary component, making it clearer and more informative. refactor(summarization): standardize reserveRatio configuration in summarization logic - Replaced instances of `reserveTokensRatio` with `reserveRatio` in the `createRun` function and related tests to unify the terminology across the codebase. - Updated the summarization configuration schema to reflect this change, ensuring consistency in how the reserve ratio is defined and utilized. - Removed the per-agent override logic for summarization configuration, simplifying the overall structure and enhancing clarity. These modifications improve the maintainability and readability of the summarization logic by standardizing the configuration parameters. * fix: circular dependency of `~/models` * chore: update logging scope in agent log handlers Changed log scope from `[agentus:${data.scope}]` to `[agents:${data.scope}]` in both the callbacks and responses controllers to ensure consistent logging format across the application. * feat: calibration ratio * refactor(tests): update summarizationConfig tests to reflect changes in enabled property Modified tests to check for the new `summarizationEnabled` property instead of the deprecated `enabled` field in the summarization configuration. This change ensures that the tests accurately validate the current configuration structure and behavior of the agents. * feat(tests): add markSummarizationUsage mock for improved test coverage Introduced a mock for the markSummarizationUsage function in the responses unit tests to enhance the testing of summarization usage tracking. This addition supports better validation of summarization-related functionalities and ensures comprehensive test coverage for the agents' response handling. * refactor(tests): simplify event handler setup in createResponse tests Removed redundant mock implementations for event handlers in the createResponse unit tests, streamlining the setup process. This change enhances test clarity and maintainability while ensuring that the tests continue to validate the correct behavior of usage tracking during on_chat_model_end events. * refactor(agents): move calibration ratio capture to finally block Reorganized the logic for capturing the calibration ratio in the AgentClient class to ensure it is executed in the finally block. This change guarantees that the ratio is captured even if the run is aborted, enhancing the reliability of the response message persistence. Removed redundant code and improved clarity in the handling of context metadata. * refactor(agents): streamline bulk write logic in recordCollectedUsage function Removed redundant bulk write operations and consolidated document handling in the recordCollectedUsage function. The logic now combines all documents into a single bulk write operation, improving efficiency and reducing error handling complexity. Updated logging to provide consistent error messages for bulk write failures. * refactor(agents): enhance summarization configuration resolution in createRun function Streamlined the summarization configuration logic by introducing a base configuration and allowing for overrides from agent-specific settings. This change improves clarity and maintainability, ensuring that the summarization configuration is consistently applied while retaining flexibility for customization. Updated the handling of summarization parameters to ensure proper integration with the agent's model and provider settings. * refactor(agents): remove unused tokenCountMap and streamline calibration ratio handling Eliminated the unused tokenCountMap variable from the AgentClient class to enhance code clarity. Additionally, streamlined the logic for capturing the calibration ratio by using optional chaining and a fallback value, ensuring that context metadata is consistently defined. This change improves maintainability and reduces potential confusion in the codebase. * refactor(agents): extract agent log handler for improved clarity and reusability Refactored the agent log handling logic by extracting it into a dedicated function, `agentLogHandler`, enhancing code clarity and reusability across different modules. Updated the event handlers in both the OpenAI and responses controllers to utilize the new handler, ensuring consistent logging behavior throughout the application. * test: add summarization event tests for useStepHandler Implemented a series of tests for the summarization events in the useStepHandler hook. The tests cover scenarios for ON_SUMMARIZE_START, ON_SUMMARIZE_DELTA, and ON_SUMMARIZE_COMPLETE events, ensuring proper handling of summarization logic, including message accumulation and finalization. This addition enhances test coverage and validates the correct behavior of the summarization process within the application. * refactor(config): update summarizationTriggerSchema to use enum for type validation Changed the type of the `type` field in the summarizationTriggerSchema from a string to an enum with a single value 'token_count'. This modification enhances type safety and ensures that only valid types are accepted in the configuration, improving overall clarity and maintainability of the schema. * test(usage): add bulk write tests for message and summarization usage Implemented tests for the bulk write functionality in the recordCollectedUsage function, covering scenarios for combined message and summarization usage, summarization-only usage, and message-only usage. These tests ensure correct document handling and token rollup calculations, enhancing test coverage and validating the behavior of the usage tracking logic. * refactor(Chat): enhance clipboard copy functionality and type definitions in Summary component Updated the Summary component to improve the clipboard copy functionality by handling clipboard permission errors. Refactored type definitions for SummaryProps to use a more specific type, enhancing type safety. Adjusted the SummaryButton and FloatingSummaryBar components to accept isCopied and onCopy props, promoting better separation of concerns and reusability. * chore(translations): remove unused "Expand Summary" key from English translations Deleted the "Expand Summary" key from the English translation file to streamline the localization resources and improve clarity in the user interface. This change helps maintain an organized and efficient translation structure. * refactor: adjust token counting for Claude model to account for API discrepancies Implemented a correction factor for token counting when using the Claude model, addressing discrepancies between Anthropic's API and local tokenizer results. This change ensures accurate token counts by applying a scaling factor, improving the reliability of token-related functionalities. * refactor(agents): implement token count adjustment for Claude model messages Added a method to adjust token counts for messages processed by the Claude model, applying a correction factor to align with API expectations. This enhancement improves the accuracy of token counting, ensuring reliable functionality when interacting with the Claude model. * refactor(agents): token counting for media content in messages Introduced a new method to estimate token costs for image and document blocks in messages, improving the accuracy of token counting. This enhancement ensures that media content is properly accounted for, particularly for the Claude model, by integrating additional token estimation logic for various content types. Updated the token counting function to utilize this new method, enhancing overall reliability and functionality. * chore: fix missing import * fix(agents): clamp baseContextTokens and document reserve ratio change Prevent negative baseContextTokens when maxOutputTokens exceeds the context window (misconfigured models). Document the 10%→5% default reserve ratio reduction introduced alongside summarization. * fix(agents): include media tokens in hydrated token counts Add estimateMediaTokensForMessage to createTokenCounter so the hydration path (used by hydrateMissingIndexTokenCounts) matches the precomputed path in AgentClient.getTokenCountForMessage. Without this, messages containing images or documents were systematically undercounted during hydration, risking context window overflow. Add 34 unit tests covering all block-type branches of estimateMediaTokensForMessage. * fix(agents): include summarization output tokens in usage return value The returned output_tokens from recordCollectedUsage now reflects all billed LLM calls (message + summarization). Previously, summarization completions were billed but excluded from the returned metadata, causing a discrepancy between what users were charged and what the response message reported. * fix(tests): replace process.exit with proper Redis cleanup in e2e test The summarization E2E test used process.exit(0) to work around a Redis connection opened at import time, which killed the Jest runner and bypassed teardown. Use ioredisClient.quit() and keyvRedisClient.disconnect() for graceful cleanup instead. * fix(tests): update getConvo imports in OpenAI and response tests Refactor test files to import getConvo from the main models module instead of the Conversation submodule. This change ensures consistency across tests and simplifies the import structure, enhancing maintainability. * fix(clients): improve summary text validation in BaseClient Refactor the summary extraction logic to ensure that only non-empty summary texts are considered valid. This change enhances the robustness of the message processing by utilizing a dedicated method for summary text retrieval, improving overall reliability. * fix(config): replace z.any() with explicit union in summarization schema Model parameters (temperature, top_p, etc.) are constrained to primitive types rather than the policy-violating z.any(). * refactor(agents): deduplicate CLAUDE_TOKEN_CORRECTION constant Export from the TS source in packages/api and import in the JS client, eliminating the static class property that could drift out of sync. * refactor(agents): eliminate duplicate selfProvider in buildAgentContext selfProvider and provider were derived from the same expression with different type casts. Consolidated to a single provider variable. * refactor(agents): extract shared SSE handlers and restrict log levels - buildSummarizationHandlers() factory replaces triplicated handler blocks across responses.js and openai.js - agentLogHandlerObj exported from callbacks.js for consistent reuse - agentLogHandler restricted to an allowlist of safe log levels (debug, info, warn, error) instead of accepting arbitrary strings * fix(SSE): batch summarize deltas, add exhaustiveness check, conditional error announcement - ON_SUMMARIZE_DELTA coalesces rapid-fire renders via requestAnimationFrame instead of calling setMessages per chunk - Exhaustive never-check on TStepEvent catches unhandled variants at compile time when new StepEvents are added - ON_SUMMARIZE_COMPLETE error announcement only fires when a summary part was actually present and removed * feat(agents): persist instruction overhead in contextMeta and seed across runs Extend contextMeta with instructionOverhead and toolCount so the provider-observed instruction overhead is persisted on the response message and seeded into the pruner on subsequent runs. This enables the pruner to use a calibrated budget from the first call instead of waiting for a provider observation, preventing the ratio collapse caused by local tokenizer overestimating tool schema tokens. The seeded overhead is only used when encoding and tool count match between runs, ensuring stale values from different configurations are discarded. * test(agents): enhance OpenAI test mocks for summarization handlers Updated the OpenAI test suite to include additional mock implementations for summarization handlers, including buildSummarizationHandlers, markSummarizationUsage, and agentLogHandlerObj. This improves test coverage and ensures consistent behavior during testing. * fix(agents): address review findings for summarization v2 Cancel rAF on unmount to prevent stale Recoil writes from dead component context. Clear orphaned summarizing:true parts when ON_SUMMARIZE_COMPLETE arrives without a summary payload. Add null guard and safe spread to agentLogHandler. Handle Anthropic-format base64 image/* documents in estimateMediaTokensForMessage. Use role="region" for expandable summary content. Add .describe() to contextMeta Zod fields. Extract duplicate usage loop into helper. * refactor: simplify contextMeta to calibrationRatio + encoding only Remove instructionOverhead and toolCount from cross-run persistence — instruction tokens change too frequently between runs (prompt edits, tool changes) for a persisted seed to be reliable. The intra-run calibration in the pruner still self-corrects via provider observations. contextMeta now stores only the tokenizer-bias ratio and encoding, which are stable across instruction changes. * test(SSE): enhance useStepHandler tests for ON_SUMMARIZE_COMPLETE behavior Updated the test for ON_SUMMARIZE_COMPLETE to clarify that it finalizes the existing part with summarizing set to false when the summary is undefined. Added assertions to verify the correct behavior of message updates and the state of summary parts. * refactor(BaseClient): remove handleContextStrategy and truncateToolCallOutputs functions Eliminated the handleContextStrategy method from BaseClient to streamline message handling. Also removed the truncateToolCallOutputs function from the prompts module, simplifying the codebase and improving maintainability. * refactor: add AGENT_DEBUG_LOGGING option and refactor token count handling in BaseClient Introduced AGENT_DEBUG_LOGGING to .env.example for enhanced debugging capabilities. Refactored token count handling in BaseClient by removing the handleTokenCountMap method and simplifying token count updates. Updated AgentClient to log detailed token count recalculations and adjustments, improving traceability during message processing. * chore: update dependencies in package-lock.json and package.json files Bumped versions of several dependencies, including @librechat/agents to ^3.1.62 and various AWS SDK packages to their latest versions. This ensures compatibility and incorporates the latest features and fixes. * chore: imports order * refactor: extract summarization config resolution from buildAgentContext * refactor: rename and simplify summarization configuration shaping function * refactor: replace AgentClient token counting methods with single-pass pure utility Extract getTokenCount() and getTokenCountForMessage() from AgentClient into countFormattedMessageTokens(), a pure function in packages/api that handles text, tool_call, image, and document content types in one loop. - Decompose estimateMediaTokensForMessage into block-level helpers (estimateImageDataTokens, estimateImageBlockTokens, estimateDocumentBlockTokens) shared by both estimateMediaTokensForMessage and the new single-pass function - Remove redundant per-call getEncoding() resolution (closure captures once) - Remove deprecated gpt-3.5-turbo-0301 model branching - Drop this.getTokenCount guard from BaseClient.sendMessage * refactor: streamline token counting in createTokenCounter function Simplified the createTokenCounter function by removing the media token estimation and directly calculating the token count. This change enhances clarity and performance by consolidating the token counting logic into a single pass, while maintaining compatibility with Claude's token correction. * refactor: simplify summarization configuration types Removed the AppSummarizationConfig type and directly used SummarizationConfig in the AppConfig interface. This change streamlines the type definitions and enhances consistency across the codebase. * chore: import order * fix: summarization event handling in useStepHandler - Cancel pending summarizeDeltaRaf in clearStepMaps to prevent stale frames firing after map reset or component unmount - Move announcePolite('summarize_completed') inside the didFinalize guard so screen readers only announce when finalization actually occurs - Remove dead cleanup closure returned from stepHandler useCallback body that was never invoked by any caller * fix: estimate tokens for non-PDF/non-image base64 document blocks Previously estimateDocumentBlockTokens returned 0 for unrecognized MIME types (e.g. text/plain, application/json), silently underestimating context budget. Fall back to character-based heuristic or countTokens. * refactor: return cloned usage from markSummarizationUsage Avoid mutating LangChain's internal usage_metadata object by returning a shallow clone with the usage_type tag. Update all call sites in callbacks, openai, and responses controllers to use the returned value. * refactor: consolidate debug logging loops in buildMessages Merge the two sequential O(n) debug-logging passes over orderedMessages into a single pass inside the map callback where all data is available. * refactor: narrow SummaryContentPart.content type Replace broad Agents.MessageContentComplex[] with the specific Array<{ type: ContentTypes.TEXT; text: string }> that all producers and consumers already use, improving compile-time safety. * refactor: use single output array in recordCollectedUsage Have processUsageGroup append to a shared array instead of returning separate arrays that are spread into a third, reducing allocations. * refactor: use for...in in hydrateMissingIndexTokenCounts Replace Object.entries with for...in to avoid allocating an intermediate tuple array during token map hydration.
2026-03-21 12:03:10 -04:00
const { requireJwtAuth } = require('~/server/middleware');
🗨️ feat: Prompts (#3131) * 🗨️ feat: Prompts (#7) * WIP: MERGE prompts/frontend (#1) * added schema for prompt and promptgroup, added model methods for prompts, added routes for prompts * * updated promptGroup Schema * updated model methods for prompts (get, add, delete) * slight fixes in prompt routes * * Created Files Management components * Created Vector Stores components * Added file management route in the routes folder * Completed UI for Files list, Compeleted UI for vector stores list, Completed UI for upload file modal, Completed UI for preview file, Completed UI for preview vector store * Fixed style and UI fixes for file dashboard, file list and vector stores list * added responsiveness classes for vector store page * fixed responsiveness of file page, dashboard page, and main page * fixed styling and responsiveness issues on dashboard page, file list page and vector store page * added queries and mutations for prompts and promptGroups, added relevant endpoints in data-provider, added relevant components prompts, added and updated relevant APIs * added types on mutation queries data service, updated prompt attributes * feature: Prompts and prompt groups management, added relevant APIs, added types for data service/queries/mutations, added relevant mutation and queries * chore: typing clarifications * added drop down on prompts mgmt dashboard * Fixes: fixed version switching issue on tags update or labels update, added cross button on create prompt group, fixed list updation on prompt group renaiming, added CSV upload button * Feature: Added oneliner and category attributes in prompt group, added schema for categories, added schema methods and route for categories * chore: typing and lint issues * chore: more type and linter fixes * chore: linting * chore: prompt controller and backend typing example; MOVE TO CONTROLLER DIRECTORY * chore: more type fixes * style: prompt name changes * chore: more type changes, and stateful prompt name change without flickering * fix: Return result of savePrompt in patchPrompt API endpoint * fix: navigation prompt queries; refactor: name 'prompt-groups' to just 'groups' * refactor: fetch prompt groups rewrite * refactor(prompts): query/mutation statefulness * refactor: remove `isActive` field * refactor: remove labels, consolidate logic * style: width, layout shift * refactor: improve hover toggle behavior and styling * refactor: add useParams hook to PromptListItem for dynamic rendering and add timeout ref for blur timeout * chore: hide upload button * refactor: import Button component from correct location in PromptSidePanel * style: prompt editor styling * style: fix more layout shifts * style: container scroll * refactor: Rename CreatePrompt component to CreatePromptForm * refactor: use react-hook-form * refactor: Add Prompts components and routes to Dashboard * style: skeletons for loading * fix: optimize makePromptProduction * refactor: consolidate variables * feat: create prompt form validation * refactor: Consolidate variables and update mutation hooks * style: minor touchups * chore: Update lucide-react npm dependency to version 0.394.0 and npm audit fix * refactor: add a new icon for the Prompts heading. * style: Update PromptsView heading to use h1 instead of h2 and other minor margin issues * chore: wording * refactor: Update PromptsView heading to use h1 instead of h2, consolidate variables, and add new icons * refactor: Prompts Button for Mobile * feature: added category field in prompt group, added relevant API and static data on BE to support FE UI for category in prompt group * chore: template for prompt cards --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * WIP: Prompts/frontend Continued (#2) * chore: loading style, remove unused component * feat: Add CategorySelector component for prompt group category selection * feat: add categories to create prompt * feat: prompt versions styling * feat: optimistic updates for prompt production state * refactor: optimize form state and show if prompt field is dirty with cross icon, also other styling changes * chore: remove unused code and localizations * fix: light mode styling * WIP: SidePanel Prompts * refactor: move to groups directory * refactor: rename GroupsSidePanel to GroupSidePanel and update imports * style: ListCard * refactor: isProduction changes * refactor: infinite query with productionPrompt * refactor: optimize snippets and prompts, and styling * refactor: Update getSnippet function to accept a length parameter * chore: localizations * feat: prompts navigation to chat and vice versa * fix: create prompt * feat: remember last selected category for creating prompts * fix(promptGroups): fix pagination and add usePromptGroupsNav hook * Prompts/frontend 3 (#3) * fix: stateful issues with prompt groups * style: improved layout * refactor: improve variable naming in Eng.ts * refactor: theme selector styling improvements * added prompt cards on chat new page, with dark mode, added API to fetch random prompts, added types for useQuery Slightly improved usePromptGroupNav logic to fetch updated result for pageSize, updated prompt cards view with darkmode and responsiveness fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page * WIP: Prompts/frontend (#4) * fix: optimize and fix paginated query * fix: remove unique constraint on names * refactor: button links and styling * style: menu border light mode * feat: Add Auto-Send Switch component for prompts groups * refactor(ChatView): use form context for submission text * chore: clear convo state on navigation to dashboard routes * chore: save prompt edit name on tab, remove console log * feat: basic prompt submission * refactor: move Auto-Send Switch * style(ListCard): border styling * feat: Add function to detect variables in text * feat: Add OriginalDialog component to UI library * chore(ui): Update SelectDropDown options list class to use text-xs size * refactor: submitMessage hook now includes submitPrompt, make compatible to document query selector * WIP: Variable Dialog * feat: variable submission working for both auto-send and non-autosend * feat: dashboard breadcrumbs and prompts/chat navigation * refactor: dashboard breadcrumb and dashboard link to chat navigation * refactor: Update VariableDialog and VariableForm styles * Prompts: Admin features (#5) * fix: link issue * fix: usePromptGroupsNav add missing dep. * style: dashbreadcrumb and sidepanel text color * temp fix: remove refetch on pageNumber change * fix: handle multiple variable replacement * WIP: create project schema and add project groups to fetch * feat: Add functionality to add prompt group IDs to a project * feat: Add caching for startup config in config route * chore: remove prompt landing * style: Update Skeleton component with additional background styling * chore: styling and types * WIP: SharePrompt first draft * feat(SharePrompt): form validation * feat: shared global indicators * refactor: prompt details * refactor: change NoPromptGroup directory * feat: preview prompt * feat: remove/add global prompts, add rbac-related enums * refactor: manage prompts location * WIP: first draft admin settings for prompts * feat: SystemRoles enum * refactor: update PromptDetails component styling * style: ellipsis custom class for showing more preview text * WIP: initial role schema and initialization * style: improved margins for single unordered lists * fix: use custom chat form context to prevent re-renders from FormProvider * feat: Role mutations for Prompt Permissions * feat: fetch user role * feat: update AdminSettings form default values from user role values * refactor: rename PromptPermissions to Permissions for general definitions * feat: initial role checks * feat: Add optional `bodyProps` parameter to generateCheckAccess middleware * refactor: UI access checks * Prompts: delete (#6) * Fixed delete prompt version API, fixed types and logic for prompt version deletion, updated prompt delete mutation logic * chore: Update return type of deletePrompt function in Prompt.js --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * chore: Update package-lock.json version to 0.7.4-rc1 and fast-xml-parser to 4.4.0 * feat: toast for saving admin settings, add timer no-access navigation * feat: always make prod * feat: Add localization to category labels in CategorySelector component * feat: Update category label localization in CategorySelector component * fix: Enable making prompt production in Prompt API --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * feat: Add helper fn for dark mode detection in ThemeProvider * style: surface-primary definition * fix(useHasAccess): utilize user.role and not just USER role * fix: empty category and role fetch * refactort: increase max height to options list and use label if no localization is found * fix: update CategorySelector to handle empty category value and improve localization * refactor: move prompts to own store/reactquery modules, add in filter WIP * refactor: Rename AutoSendSwitch to AutoSendPrompt * style: theming commit * style: fix slight coloring issue for convos in dark mode * style: better composition for prompts side panel * style: remove gray-750 and make it gray-850 * chore: adjust theming * feat: filter all prompt groups and properly remove prompts from projects * refactor: optimize delete prompt groups further * chore: localization * feat: Add uniqueProperty filtering to normalizeData function * WIP: filter prompts * chore: Update FilterPrompts component to include User icon in FilterItem * feat(FilterPrompts): set categories * feat: more system filters and show selected category icon * style: always make prod, flips switch to avoid mis-clicks * style: ui/ux loading/no prompts * chore: style FilterPrompts ChatView * fix: handle missing role edge case * style: special variables * feat: special variables * refactor: improve replaceSpecialVars function in prompts.ts * feat: simple/advanced editor modes * chore: bump versions * feat: localizations and hide production button on simple mode * fix: error connecting layout shift * fix: prompts CRUD for admins * fix: secure single group fetch * style: sidepanel styling * style(PromptName): bring edit button closer to name * style: mobile prompts header * style: mobile prompts header continued * style: align send prompts switch right * feat: description * Update special variables description in Eng.ts * feat: update/create/preview oneliner * fix: allow empty oneliner update * style: loading improvement and always make selected prompt Production if simple mode * fix: production index set and remove unused props * fix(ci): mock initializeRoles * fix: address #3128 * fix: address #3128 * feat: add deletion confirmation dialog * fix: mobile UI issues * style: prompt library UI update * style: focus, logcal tab order * style: Refactor SelectDropDown component to improve code readability and maintainability * chore: bump data-provider * chore: fix labels * refactor: confirm delete prompt version --------- Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
2024-06-20 20:24:32 -04:00
const router = express.Router();
router.use(requireJwtAuth);
📜 feat: Implement System Grants for Capability-Based Authorization (#11896) * feat: Implement System Grants for Role-Based Capabilities - Added a new `systemGrant` model and associated methods to manage role-based capabilities within the application. - Introduced middleware functions `hasCapability` and `requireCapability` to check user permissions based on their roles. - Updated the database seeding process to include system grants for the ADMIN role, ensuring all necessary capabilities are assigned on startup. - Enhanced type definitions and schemas to support the new system grant functionality, improving overall type safety and clarity in the codebase. * test: Add unit tests for capabilities middleware and system grant methods - Introduced comprehensive unit tests for the capabilities middleware, including `hasCapability` and `requireCapability`, ensuring proper permission checks based on user roles. - Added tests for the `SystemGrant` methods, verifying the seeding of system grants, capability granting, and revocation processes. - Enhanced test coverage for edge cases, including idempotency of grant operations and handling of unexpected errors in middleware. - Utilized mocks for database interactions to isolate tests and improve reliability. * refactor: Transition to Capability-Based Access Control - Replaced role-based access checks with capability-based checks across various middleware and routes, enhancing permission management. - Introduced `hasCapability` and `requireCapability` functions to streamline capability verification for user actions. - Updated relevant routes and middleware to utilize the new capability system, ensuring consistent permission enforcement. - Enhanced type definitions and added tests for the new capability functions, improving overall code reliability and maintainability. * test: Enhance capability-based access tests for ADMIN role - Updated tests to reflect the new capability-based access control, specifically for the ADMIN role. - Modified test descriptions to clarify that users with the MANAGE_AGENTS capability can bypass permission checks. - Seeded capabilities for the ADMIN role in multiple test files to ensure consistent permission checks across different routes and middleware. - Improved overall test coverage for capability verification, ensuring robust permission management. * test: Update capability tests for MCP server access - Renamed test to reflect the correct capability for bypassing permission checks, changing from MANAGE_AGENTS to MANAGE_MCP_SERVERS. - Updated seeding of capabilities for the ADMIN role to align with the new capability structure. - Ensured consistency in capability definitions across tests and middleware for improved permission management. * feat: Add hasConfigCapability for enhanced config access control - Introduced `hasConfigCapability` function to check user permissions for managing or reading specific config sections. - Updated middleware to export the new capability function, ensuring consistent access control across the application. - Enhanced unit tests to cover various scenarios for the new capability, improving overall test coverage and reliability. * fix: Update tenantId filter in createSystemGrantMethods - Added a condition to set tenantId filter to { $exists: false } when tenantId is null, ensuring proper handling of cases where tenantId is not provided. - This change improves the robustness of the system grant methods by explicitly managing the absence of tenantId in the filter logic. * fix: account deletion capability check - Updated the `canDeleteAccount` middleware to ensure that the `hasManageUsers` capability check only occurs if a user is present, preventing potential errors when the user object is undefined. - This change improves the robustness of the account deletion logic by ensuring proper handling of user permissions. * refactor: Optimize seeding of system grants for ADMIN role - Replaced sequential capability granting with parallel execution using Promise.all in the seedSystemGrants function. - This change improves performance and efficiency during the initialization of system grants, ensuring all capabilities are granted concurrently. * refactor: Simplify systemGrantSchema index definition - Removed the sparse option from the unique index on principalType, principalId, capability, and tenantId in the systemGrantSchema. - This change streamlines the index definition, potentially improving query performance and clarity in the schema design. * refactor: Reorganize role capability check in roles route - Moved the capability check for reading roles to occur after parsing the roleName, improving code clarity and structure. - This change ensures that the authorization logic is consistently applied before fetching role details, enhancing overall permission management. * refactor: Remove unused ISystemGrant interface from systemCapabilities.ts - Deleted the ISystemGrant interface as it was no longer needed, streamlining the code and improving clarity. - This change helps reduce clutter in the file and focuses on relevant capabilities for the system. * refactor: Migrate SystemCapabilities to data-schemas - Replaced imports of SystemCapabilities from 'librechat-data-provider' with imports from '@librechat/data-schemas' across multiple files. - This change centralizes the management of system capabilities, improving code organization and maintainability. * refactor: Update account deletion middleware and capability checks - Modified the `canDeleteAccount` middleware to ensure that the account deletion permission is only granted to users with the `MANAGE_USERS` capability, improving security and clarity in permission management. - Enhanced error logging for unauthorized account deletion attempts, providing better insights into permission issues. - Updated the `capabilities.ts` file to ensure consistent handling of user authentication checks, improving robustness in capability verification. - Refined type definitions in `systemGrant.ts` and `systemGrantMethods.ts` to utilize the `PrincipalType` enum, enhancing type safety and code clarity. * refactor: Extract principal ID normalization into a separate function - Introduced `normalizePrincipalId` function to streamline the normalization of principal IDs based on their type, enhancing code clarity and reusability. - Updated references in `createSystemGrantMethods` to utilize the new normalization function, improving maintainability and reducing code duplication. * test: Add unit tests for principalId normalization in systemGrant - Introduced tests for the `grantCapability`, `revokeCapability`, and `getCapabilitiesForPrincipal` methods to verify correct handling of principalId normalization between string and ObjectId formats. - Enhanced the `capabilities.ts` middleware to utilize the `PrincipalType` enum for improved type safety. - Added a new utility function `normalizePrincipalId` to streamline principal ID normalization logic, ensuring consistent behavior across the application. * feat: Introduce capability implications and enhance system grant methods - Added `CapabilityImplications` to define relationships between broader and implied capabilities, allowing for more intuitive permission checks. - Updated `createSystemGrantMethods` to expand capability queries to include implied capabilities, improving authorization logic. - Enhanced `systemGrantSchema` to include an `expiresAt` field for future TTL enforcement of grants, and added validation to ensure `tenantId` is not set to null. - Documented authorization requirements for prompt group and prompt deletion methods to clarify access control expectations. * test: Add unit tests for canDeleteAccount middleware - Introduced unit tests for the `canDeleteAccount` middleware to verify account deletion permissions based on user roles and capabilities. - Covered scenarios for both allowed and blocked account deletions, including checks for ADMIN users with the `MANAGE_USERS` capability and handling of undefined user cases. - Enhanced test structure to ensure clarity and maintainability of permission checks in the middleware. * fix: Add principalType enum validation to SystemGrant schema Without enum validation, any string value was accepted for principalType and silently stored. Invalid documents would never match capability queries, creating phantom grants impossible to diagnose without raw DB inspection. All other ACL models in the codebase validate this field. * fix: Replace seedSystemGrants Promise.all with bulkWrite for concurrency safety When two server instances start simultaneously (K8s rolling deploy, PM2 cluster), both call seedSystemGrants. With Promise.all + findOneAndUpdate upsert, both instances may attempt to insert the same documents, causing E11000 duplicate key errors that crash server startup. bulkWrite with ordered:false handles concurrent upserts gracefully and reduces 17 individual round trips to a single network call. The returned documents (previously discarded) are no longer fetched. * perf: Add AsyncLocalStorage per-request cache for capability checks Every hasCapability call previously required 2 DB round trips (getUserPrincipals + SystemGrant.exists) — replacing what were O(1) string comparisons. Routes like patchPromptGroup triggered this twice, and hasConfigCapability's fallback path resolved principals twice. This adds a per-request AsyncLocalStorage cache that: - Caches resolved principals (same for all checks within one request) - Caches capability check results (same user+cap = same answer) - Automatically scoped to request lifetime (no stale grants) - Falls through to DB when no store exists (background jobs, tests) - Requires no signature changes to hasCapability The capabilityContextMiddleware is registered at the app level before all routes, initializing a fresh store per request. * fix: Add error handling for inline hasCapability calls canDeleteAccount, fetchAssistants, and validateAuthor all call hasCapability without try-catch. These were previously O(1) string comparisons that could never throw. Now they hit the database and can fail on connection timeout or transient errors. Wrap each call in try-catch, defaulting to deny (false) on error. This ensures a DB hiccup returns a clean 403 instead of an unhandled 500 with a stack trace. * test: Add canDeleteAccount DB-error resilience test Tests that hasCapability rejection (e.g., DB timeout) results in a clean 403 rather than an unhandled exception. Validates the error handling added in the previous commit. * refactor: Use barrel import for hasCapability in validateAuthor Import from ~/server/middleware barrel instead of directly from ~/server/middleware/roles/capabilities for consistency with other non-middleware consumers. Files within the middleware barrel itself must continue using direct imports to avoid circular requires. * refactor: Remove misleading pre('save') hook from SystemGrant schema The pre('save') hook normalized principalId for USER/GROUP principals, but the primary write path (grantCapability) uses findOneAndUpdate — which does not trigger save hooks. The normalization was already handled explicitly in grantCapability itself. The hook created a false impression of schema-level enforcement that only covered save()/create() paths. Replace with a comment documenting that all writes must go through grantCapability. * feat: Add READ_ASSISTANTS capability to complete manage/read pair Every other managed resource had a paired READ_X / MANAGE_X capability except assistants. This adds READ_ASSISTANTS and registers the MANAGE_ASSISTANTS → READ_ASSISTANTS implication in CapabilityImplications, enabling future read-only assistant visibility grants. * chore: Reorder systemGrant methods for clarity Moved hasCapabilityForPrincipals to a more logical position in the returned object of createSystemGrantMethods, improving code readability. This change also maintains the inclusion of seedSystemGrants in the export, ensuring all necessary methods are available. * fix: Wrap seedSystemGrants in try-catch to avoid blocking startup Seeding capabilities is idempotent and will succeed on the next restart. A transient DB error during seeding should not prevent the server from starting — log the error and continue. * refactor: Improve capability check efficiency and add audit logging Move hasCapability calls after cheap early-exits in validateAuthor and fetchAssistants so the DB check only runs when its result matters. Add logger.debug on every capability bypass grant across all 7 call sites for auditability, and log errors in catch blocks instead of silently swallowing them. * test: Add integration tests for AsyncLocalStorage capability caching Exercises the full vertical — ALS context, generateCapabilityCheck, real getUserPrincipals, real hasCapabilityForPrincipals, real MongoDB via MongoMemoryServer. Covers per-request caching, cross-context isolation, concurrent request isolation, negative caching, capability implications, tenant scoping, group-based grants, and requireCapability middleware. * test: Add systemGrant data-layer and ALS edge-case integration tests systemGrant.spec.ts (51 tests): Full integration tests for all systemGrant methods against real MongoDB — grant/revoke lifecycle, principalId normalization (string→ObjectId for USER/GROUP, string for ROLE), capability implications (both directions), tenant scoping, schema validation (null tenantId, invalid enum, required fields, unique compound index). capabilities.integration.spec.ts (27 tests): Adds ALS edge cases — missing context degrades gracefully with no caching (background jobs, child processes), nested middleware creates independent inner context, optional-chaining safety when store is undefined, mid-request grant changes are invisible due to result caching, requireCapability works without ALS, and interleaved concurrent contexts maintain isolation. * fix: Add worker thread guards to capability ALS usage Detect when hasCapability or capabilityContextMiddleware is called from a worker thread (where ALS context does not propagate from the parent). hasCapability logs a warn-once per factory instance; the middleware logs an error since mounting Express middleware in a worker is likely a misconfiguration. Both continue to function correctly — the guard is observability, not a hard block. * fix: Include tenantId in ALS principal cache key for tenant isolation The principal cache key was user.id:user.role, which would reuse cached principals across tenants for the same user within a request. When getUserPrincipals gains tenant-scoped group resolution, principals from tenant-a would incorrectly serve tenant-b checks. Changed to user.id:user.role:user.tenantId to prevent cross-tenant cache hits. Adds integration test proving separate principal lookups per tenantId. * test: Remove redundant mocked capabilities.spec.js The JS wrapper test (7 tests, all mocked) is a strict subset of capabilities.integration.spec.ts (28 tests, real MongoDB). Every scenario it covered — hasCapability true/false, tenantId passthrough, requireCapability 403/500, error handling — is tested with higher fidelity in the integration suite. * test: Replace mocked canDeleteAccount tests with real MongoDB integration Remove hasCapability mock — tests now exercise the full capability chain against real MongoDB (getUserPrincipals, hasCapabilityForPrincipals, SystemGrant collection). Only mocks remaining are logger and cache. Adds new coverage: admin role without grant is blocked, user-level grant bypasses deletion restriction, null user handling. * test: Add comprehensive tests for ACL entry management and user group methods Introduces new tests for `deleteAclEntries`, `bulkWriteAclEntries`, and `findPublicResourceIds` in `aclEntry.spec.ts`, ensuring proper functionality for deleting and bulk managing ACL entries. Additionally, enhances `userGroup.spec.ts` with tests for finding groups by ID and name pattern, including external ID matching and source filtering. These changes improve coverage and validate the integrity of ACL and user group operations against real MongoDB interactions. * refactor: Update capability checks and logging for better clarity and error handling Replaced `MANAGE_USERS` with `ACCESS_ADMIN` in the `canDeleteAccount` middleware and related tests to align with updated permission structure. Enhanced logging in various middleware functions to use `logger.warn` for capability check failures, providing clearer error messages. Additionally, refactored capability checks in the `patchPromptGroup` and `validateAuthor` functions to improve readability and maintainability. This commit also includes adjustments to the `systemGrant` methods to implement retry logic for transient failures during capability seeding, ensuring robustness in the face of database errors. * refactor: Enhance logging and retry logic in seedSystemGrants method Updated the logging format in the seedSystemGrants method to include error messages for better clarity. Improved the retry mechanism by explicitly mocking multiple failures in tests, ensuring robust error handling during transient database issues. Additionally, refined imports in the systemGrant schema for better type management. * refactor: Consolidate imports in canDeleteAccount middleware Merged logger and SystemCapabilities imports from the data-schemas module into a single line for improved readability and maintainability of the code. This change streamlines the import statements in the canDeleteAccount middleware. * test: Enhance systemGrant tests for error handling and capability validation Added tests to the systemGrant methods to handle various error scenarios, including E11000 race conditions, invalid ObjectId strings for USER and GROUP principals, and invalid capability strings. These enhancements improve the robustness of the capability granting and revoking logic, ensuring proper error propagation and validation of inputs. * fix: Wrap hasCapability calls in deny-by-default try-catch at remaining sites canAccessResource, files.js, and roles.js all had hasCapability inside outer try-catch blocks that returned 500 on DB failure instead of falling through to the regular ACL check. This contradicts the deny-by-default pattern used everywhere else. Also removes raw error.message from the roles.js 500 response to prevent internal host/connection info leaking to clients. * fix: Normalize user ID in canDeleteAccount before passing to hasCapability requireCapability normalizes req.user.id via _id?.toString() fallback, but canDeleteAccount passed raw req.user directly. If req.user.id is absent (some auth layers only populate _id), getUserPrincipals received undefined, silently returning empty principals and blocking the bypass. * fix: Harden systemGrant schema and type safety - Reject empty string tenantId in schema validator (was only blocking null; empty string silently orphaned documents) - Fix reverseImplications to use BaseSystemCapability[] instead of string[], preserving the narrow discriminated type - Document READ_ASSISTANTS as reserved/unenforced * test: Use fake timers for seedSystemGrants retry tests and add tenantId validation - Switch retry tests to jest.useFakeTimers() to eliminate 3+ seconds of real setTimeout delays per test run - Add regression test for empty-string tenantId rejection * docs: Add TODO(#12091) comments for tenant-scoped capability gaps In multi-tenant mode, platform-level grants (no tenantId) won't match tenant-scoped queries, breaking admin access. getUserPrincipals also returns cross-tenant group memberships. Both need fixes in #12091.
2026-03-07 13:56:32 -05:00
const manageRoles = requireCapability(SystemCapabilities.MANAGE_ROLES);
🗨️ feat: Prompts (#3131) * 🗨️ feat: Prompts (#7) * WIP: MERGE prompts/frontend (#1) * added schema for prompt and promptgroup, added model methods for prompts, added routes for prompts * * updated promptGroup Schema * updated model methods for prompts (get, add, delete) * slight fixes in prompt routes * * Created Files Management components * Created Vector Stores components * Added file management route in the routes folder * Completed UI for Files list, Compeleted UI for vector stores list, Completed UI for upload file modal, Completed UI for preview file, Completed UI for preview vector store * Fixed style and UI fixes for file dashboard, file list and vector stores list * added responsiveness classes for vector store page * fixed responsiveness of file page, dashboard page, and main page * fixed styling and responsiveness issues on dashboard page, file list page and vector store page * added queries and mutations for prompts and promptGroups, added relevant endpoints in data-provider, added relevant components prompts, added and updated relevant APIs * added types on mutation queries data service, updated prompt attributes * feature: Prompts and prompt groups management, added relevant APIs, added types for data service/queries/mutations, added relevant mutation and queries * chore: typing clarifications * added drop down on prompts mgmt dashboard * Fixes: fixed version switching issue on tags update or labels update, added cross button on create prompt group, fixed list updation on prompt group renaiming, added CSV upload button * Feature: Added oneliner and category attributes in prompt group, added schema for categories, added schema methods and route for categories * chore: typing and lint issues * chore: more type and linter fixes * chore: linting * chore: prompt controller and backend typing example; MOVE TO CONTROLLER DIRECTORY * chore: more type fixes * style: prompt name changes * chore: more type changes, and stateful prompt name change without flickering * fix: Return result of savePrompt in patchPrompt API endpoint * fix: navigation prompt queries; refactor: name 'prompt-groups' to just 'groups' * refactor: fetch prompt groups rewrite * refactor(prompts): query/mutation statefulness * refactor: remove `isActive` field * refactor: remove labels, consolidate logic * style: width, layout shift * refactor: improve hover toggle behavior and styling * refactor: add useParams hook to PromptListItem for dynamic rendering and add timeout ref for blur timeout * chore: hide upload button * refactor: import Button component from correct location in PromptSidePanel * style: prompt editor styling * style: fix more layout shifts * style: container scroll * refactor: Rename CreatePrompt component to CreatePromptForm * refactor: use react-hook-form * refactor: Add Prompts components and routes to Dashboard * style: skeletons for loading * fix: optimize makePromptProduction * refactor: consolidate variables * feat: create prompt form validation * refactor: Consolidate variables and update mutation hooks * style: minor touchups * chore: Update lucide-react npm dependency to version 0.394.0 and npm audit fix * refactor: add a new icon for the Prompts heading. * style: Update PromptsView heading to use h1 instead of h2 and other minor margin issues * chore: wording * refactor: Update PromptsView heading to use h1 instead of h2, consolidate variables, and add new icons * refactor: Prompts Button for Mobile * feature: added category field in prompt group, added relevant API and static data on BE to support FE UI for category in prompt group * chore: template for prompt cards --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * WIP: Prompts/frontend Continued (#2) * chore: loading style, remove unused component * feat: Add CategorySelector component for prompt group category selection * feat: add categories to create prompt * feat: prompt versions styling * feat: optimistic updates for prompt production state * refactor: optimize form state and show if prompt field is dirty with cross icon, also other styling changes * chore: remove unused code and localizations * fix: light mode styling * WIP: SidePanel Prompts * refactor: move to groups directory * refactor: rename GroupsSidePanel to GroupSidePanel and update imports * style: ListCard * refactor: isProduction changes * refactor: infinite query with productionPrompt * refactor: optimize snippets and prompts, and styling * refactor: Update getSnippet function to accept a length parameter * chore: localizations * feat: prompts navigation to chat and vice versa * fix: create prompt * feat: remember last selected category for creating prompts * fix(promptGroups): fix pagination and add usePromptGroupsNav hook * Prompts/frontend 3 (#3) * fix: stateful issues with prompt groups * style: improved layout * refactor: improve variable naming in Eng.ts * refactor: theme selector styling improvements * added prompt cards on chat new page, with dark mode, added API to fetch random prompts, added types for useQuery Slightly improved usePromptGroupNav logic to fetch updated result for pageSize, updated prompt cards view with darkmode and responsiveness fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page * WIP: Prompts/frontend (#4) * fix: optimize and fix paginated query * fix: remove unique constraint on names * refactor: button links and styling * style: menu border light mode * feat: Add Auto-Send Switch component for prompts groups * refactor(ChatView): use form context for submission text * chore: clear convo state on navigation to dashboard routes * chore: save prompt edit name on tab, remove console log * feat: basic prompt submission * refactor: move Auto-Send Switch * style(ListCard): border styling * feat: Add function to detect variables in text * feat: Add OriginalDialog component to UI library * chore(ui): Update SelectDropDown options list class to use text-xs size * refactor: submitMessage hook now includes submitPrompt, make compatible to document query selector * WIP: Variable Dialog * feat: variable submission working for both auto-send and non-autosend * feat: dashboard breadcrumbs and prompts/chat navigation * refactor: dashboard breadcrumb and dashboard link to chat navigation * refactor: Update VariableDialog and VariableForm styles * Prompts: Admin features (#5) * fix: link issue * fix: usePromptGroupsNav add missing dep. * style: dashbreadcrumb and sidepanel text color * temp fix: remove refetch on pageNumber change * fix: handle multiple variable replacement * WIP: create project schema and add project groups to fetch * feat: Add functionality to add prompt group IDs to a project * feat: Add caching for startup config in config route * chore: remove prompt landing * style: Update Skeleton component with additional background styling * chore: styling and types * WIP: SharePrompt first draft * feat(SharePrompt): form validation * feat: shared global indicators * refactor: prompt details * refactor: change NoPromptGroup directory * feat: preview prompt * feat: remove/add global prompts, add rbac-related enums * refactor: manage prompts location * WIP: first draft admin settings for prompts * feat: SystemRoles enum * refactor: update PromptDetails component styling * style: ellipsis custom class for showing more preview text * WIP: initial role schema and initialization * style: improved margins for single unordered lists * fix: use custom chat form context to prevent re-renders from FormProvider * feat: Role mutations for Prompt Permissions * feat: fetch user role * feat: update AdminSettings form default values from user role values * refactor: rename PromptPermissions to Permissions for general definitions * feat: initial role checks * feat: Add optional `bodyProps` parameter to generateCheckAccess middleware * refactor: UI access checks * Prompts: delete (#6) * Fixed delete prompt version API, fixed types and logic for prompt version deletion, updated prompt delete mutation logic * chore: Update return type of deletePrompt function in Prompt.js --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * chore: Update package-lock.json version to 0.7.4-rc1 and fast-xml-parser to 4.4.0 * feat: toast for saving admin settings, add timer no-access navigation * feat: always make prod * feat: Add localization to category labels in CategorySelector component * feat: Update category label localization in CategorySelector component * fix: Enable making prompt production in Prompt API --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * feat: Add helper fn for dark mode detection in ThemeProvider * style: surface-primary definition * fix(useHasAccess): utilize user.role and not just USER role * fix: empty category and role fetch * refactort: increase max height to options list and use label if no localization is found * fix: update CategorySelector to handle empty category value and improve localization * refactor: move prompts to own store/reactquery modules, add in filter WIP * refactor: Rename AutoSendSwitch to AutoSendPrompt * style: theming commit * style: fix slight coloring issue for convos in dark mode * style: better composition for prompts side panel * style: remove gray-750 and make it gray-850 * chore: adjust theming * feat: filter all prompt groups and properly remove prompts from projects * refactor: optimize delete prompt groups further * chore: localization * feat: Add uniqueProperty filtering to normalizeData function * WIP: filter prompts * chore: Update FilterPrompts component to include User icon in FilterItem * feat(FilterPrompts): set categories * feat: more system filters and show selected category icon * style: always make prod, flips switch to avoid mis-clicks * style: ui/ux loading/no prompts * chore: style FilterPrompts ChatView * fix: handle missing role edge case * style: special variables * feat: special variables * refactor: improve replaceSpecialVars function in prompts.ts * feat: simple/advanced editor modes * chore: bump versions * feat: localizations and hide production button on simple mode * fix: error connecting layout shift * fix: prompts CRUD for admins * fix: secure single group fetch * style: sidepanel styling * style(PromptName): bring edit button closer to name * style: mobile prompts header * style: mobile prompts header continued * style: align send prompts switch right * feat: description * Update special variables description in Eng.ts * feat: update/create/preview oneliner * fix: allow empty oneliner update * style: loading improvement and always make selected prompt Production if simple mode * fix: production index set and remove unused props * fix(ci): mock initializeRoles * fix: address #3128 * fix: address #3128 * feat: add deletion confirmation dialog * fix: mobile UI issues * style: prompt library UI update * style: focus, logcal tab order * style: Refactor SelectDropDown component to improve code readability and maintainability * chore: bump data-provider * chore: fix labels * refactor: confirm delete prompt version --------- Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
2024-06-20 20:24:32 -04:00
/**
* Permission configuration mapping
* Maps route paths to their corresponding schemas and permission types
*/
const permissionConfigs = {
prompts: {
schema: promptPermissionsSchema,
permissionType: PermissionTypes.PROMPTS,
errorMessage: 'Invalid prompt permissions.',
},
agents: {
schema: agentPermissionsSchema,
permissionType: PermissionTypes.AGENTS,
errorMessage: 'Invalid agent permissions.',
},
memories: {
schema: memoryPermissionsSchema,
permissionType: PermissionTypes.MEMORIES,
errorMessage: 'Invalid memory permissions.',
},
'people-picker': {
schema: peoplePickerPermissionsSchema,
permissionType: PermissionTypes.PEOPLE_PICKER,
errorMessage: 'Invalid people picker permissions.',
},
🏗️ feat: Dynamic MCP Server Infrastructure with Access Control (#10787) * Feature: Dynamic MCP Server with Full UI Management * 🚦 feat: Add MCP Connection Status icons to MCPBuilder panel (#10805) * feature: Add MCP server connection status icons to MCPBuilder panel * refactor: Simplify MCPConfigDialog rendering in MCPBuilderPanel --------- Co-authored-by: Atef Bellaaj <slalom.bellaaj@external.daimlertruck.com> Co-authored-by: Danny Avila <danny@librechat.ai> * fix: address code review feedback for MCP server management - Fix OAuth secret preservation to avoid mutating input parameter by creating a merged config copy in ServerConfigsDB.update() - Improve error handling in getResourcePermissionsMap to propagate critical errors instead of silently returning empty Map - Extract duplicated MCP server filter logic by exposing selectableServers from useMCPServerManager hook and using it in MCPSelect component * test: Update PermissionService tests to throw errors on invalid resource types - Changed the test for handling invalid resource types to ensure it throws an error instead of returning an empty permissions map. - Updated the expectation to check for the specific error message when an invalid resource type is provided. * feat: Implement retry logic for MCP server creation to handle race conditions - Enhanced the createMCPServer method to include retry logic with exponential backoff for handling duplicate key errors during concurrent server creation. - Updated tests to verify that all concurrent requests succeed and that unique server names are generated. - Added a helper function to identify MongoDB duplicate key errors, improving error handling during server creation. * refactor: StatusIcon to use CircleCheck for connected status - Replaced the PlugZap icon with CircleCheck in the ConnectedStatusIcon component to better represent the connected state. - Ensured consistent icon usage across the component for improved visual clarity. * test: Update AccessControlService tests to throw errors on invalid resource types - Modified the test for invalid resource types to ensure it throws an error with a specific message instead of returning an empty permissions map. - This change enhances error handling and improves test coverage for the AccessControlService. * fix: Update error message for missing server name in MCP server retrieval - Changed the error message returned when the server name is not provided from 'MCP ID is required' to 'Server name is required' for better clarity and accuracy in the API response. --------- Co-authored-by: Atef Bellaaj <slalom.bellaaj@external.daimlertruck.com> Co-authored-by: Danny Avila <danny@librechat.ai>
2025-12-04 21:37:23 +01:00
'mcp-servers': {
schema: mcpServersPermissionsSchema,
permissionType: PermissionTypes.MCP_SERVERS,
errorMessage: 'Invalid MCP servers permissions.',
},
marketplace: {
schema: marketplacePermissionsSchema,
permissionType: PermissionTypes.MARKETPLACE,
errorMessage: 'Invalid marketplace permissions.',
},
🛸 feat: Remote Agent Access with External API Support (#11503) * 🪪 feat: Microsoft Graph Access Token Placeholder for MCP Servers (#10867) * feat: MCP Graph Token env var * Addressing copilot remarks * Addressed Copilot review remarks * Fixed graphtokenservice mock in MCP test suite * fix: remove unnecessary type check and cast in resolveGraphTokensInRecord * ci: add Graph Token integration tests in MCPManager * refactor: update user type definitions to use Partial<IUser> in multiple functions * test: enhance MCP tests for graph token processing and user placeholder resolution - Added comprehensive tests to validate the interaction between preProcessGraphTokens and processMCPEnv. - Ensured correct resolution of graph tokens and user placeholders in various configurations. - Mocked OIDC utilities to facilitate testing of token extraction and validation. - Verified that original options remain unchanged after processing. * chore: import order * chore: imports --------- Co-authored-by: Danny Avila <danny@librechat.ai> * WIP: OpenAI-compatible API for LibreChat agents - Added OpenAIChatCompletionController for handling chat completions. - Introduced ListModelsController and GetModelController for listing and retrieving agent details. - Created routes for OpenAI API endpoints, including /v1/chat/completions and /v1/models. - Developed event handlers for streaming responses in OpenAI format. - Implemented request validation and error handling for API interactions. - Integrated content aggregation and response formatting to align with OpenAI specifications. This commit establishes a foundational API for interacting with LibreChat agents in a manner compatible with OpenAI's chat completion interface. * refactor: OpenAI-spec content aggregation for improved performance and clarity * fix: OpenAI chat completion controller with safe user handling for correct tool loading * refactor: Remove conversation ID from OpenAI response context and related handlers * refactor: OpenAI chat completion handling with streaming support - Introduced a lightweight tracker for streaming responses, allowing for efficient tracking of emitted content and usage metadata. - Updated the OpenAIChatCompletionController to utilize the new tracker, improving the handling of streaming and non-streaming responses. - Refactored event handlers to accommodate the new streaming logic, ensuring proper management of tool calls and content aggregation. - Adjusted response handling to streamline error reporting during streaming sessions. * WIP: Open Responses API with core service, types, and handlers - Added Open Responses API module with comprehensive types and enums. - Implemented core service for processing requests, including validation and input conversion. - Developed event handlers for streaming responses and non-streaming aggregation. - Established response building logic and error handling mechanisms. - Created detailed types for input and output content, ensuring compliance with Open Responses specification. * feat: Implement response storage and retrieval in Open Responses API - Added functionality to save user input messages and assistant responses to the database when the `store` flag is set to true. - Introduced a new endpoint to retrieve stored responses by ID, allowing users to access previous interactions. - Enhanced the response creation process to include database operations for conversation and message storage. - Implemented tests to validate the storage and retrieval of responses, ensuring correct behavior for both existing and non-existent response IDs. * refactor: Open Responses API with additional token tracking and validation - Added support for tracking cached tokens in response usage, improving token management. - Updated response structure to include new properties for top log probabilities and detailed usage metrics. - Enhanced tests to validate the presence and types of new properties in API responses, ensuring compliance with updated specifications. - Refactored response handling to accommodate new fields and improve overall clarity and performance. * refactor: Update reasoning event handlers and types for consistency - Renamed reasoning text events to simplify naming conventions, changing `emitReasoningTextDelta` to `emitReasoningDelta` and `emitReasoningTextDone` to `emitReasoningDone`. - Updated event types in the API to reflect the new naming, ensuring consistency across the codebase. - Added `logprobs` property to output events for enhanced tracking of log probabilities. * feat: Add validation for streaming events in Open Responses API tests * feat: Implement response.created event in Open Responses API - Added emitResponseCreated function to emit the response.created event as the first event in the streaming sequence, adhering to the Open Responses specification. - Updated createResponse function to emit response.created followed by response.in_progress. - Enhanced tests to validate the order of emitted events, ensuring response.created is triggered before response.in_progress. * feat: Responses API with attachment event handling - Introduced `createResponsesToolEndCallback` to handle attachment events in the Responses API, emitting `librechat:attachment` events as per the Open Responses extension specification. - Updated the `createResponse` function to utilize the new callback for processing tool outputs and emitting attachments during streaming. - Added helper functions for writing attachment events and defined types for attachment data, ensuring compatibility with the Open Responses protocol. - Enhanced tests to validate the integration of attachment events within the Responses API workflow. * WIP: remote agent auth * fix: Improve loading state handling in AgentApiKeys component - Updated the rendering logic to conditionally display loading spinner and API keys based on the loading state. - Removed unnecessary imports and streamlined the component for better readability. * refactor: Update API key access handling in routes - Replaced `checkAccess` with `generateCheckAccess` for improved access control. - Consolidated access checks into a single `checkApiKeyAccess` function, enhancing code readability and maintainability. - Streamlined route definitions for creating, listing, retrieving, and deleting API keys. * fix: Add permission handling for REMOTE_AGENT resource type * feat: Enhance permission handling for REMOTE_AGENT resources - Updated the deleteAgent and deleteUserAgents functions to handle permissions for both AGENT and REMOTE_AGENT resource types. - Introduced new functions to enrich REMOTE_AGENT principals and backfill permissions for AGENT owners. - Modified createAgentHandler and duplicateAgentHandler to grant permissions for REMOTE_AGENT alongside AGENT. - Added utility functions for retrieving effective permissions for REMOTE_AGENT resources, ensuring consistent access control across the application. * refactor: Rename and update roles for remote agent access - Changed role name from API User to Editor in translation files for clarity. - Updated default editor role ID from REMOTE_AGENT_USER to REMOTE_AGENT_EDITOR in resource configurations. - Adjusted role localization to reflect the new Editor role. - Modified access permissions to align with the updated role definitions across the application. * feat: Introduce remote agent permissions and update access handling - Added support for REMOTE_AGENTS in permission schemas, including use, create, share, and share_public permissions. - Updated the interface configuration to include remote agent settings. - Modified middleware and API key access checks to align with the new remote agent permission structure. - Enhanced role defaults to incorporate remote agent permissions, ensuring consistent access control across the application. * refactor: Update AgentApiKeys component and permissions handling - Refactored the AgentApiKeys component to improve structure and readability, including the introduction of ApiKeysContent for better separation of concerns. - Updated CreateKeyDialog to accept an onKeyCreated callback, enhancing its functionality. - Adjusted permission checks in Data component to use REMOTE_AGENTS and USE permissions, aligning with recent permission schema changes. - Enhanced loading state handling and dialog management for a smoother user experience. * refactor: Update remote agent access checks in API routes - Replaced existing access checks with `generateCheckAccess` for remote agents in the API keys and agents routes. - Introduced specific permission checks for creating, listing, retrieving, and deleting API keys, enhancing access control. - Improved code structure by consolidating permission handling for remote agents across multiple routes. * fix: Correct query parameters in ApiKeysContent component - Updated the useGetAgentApiKeysQuery call to include an object for the enabled parameter, ensuring proper functionality when the component is open. - This change improves the handling of API key retrieval based on the component's open state. * feat: Implement remote agents permissions and update API routes - Added new API route for updating remote agents permissions, enhancing role management capabilities. - Introduced remote agents permissions handling in the AgentApiKeys component, including a dedicated settings dialog. - Updated localization files to include new remote agents permission labels for better user experience. - Refactored data provider to support remote agents permissions updates, ensuring consistent access control across the application. * feat: Add remote agents permissions to role schema and interface - Introduced new permissions for REMOTE_AGENTS in the role schema, including USE, CREATE, SHARE, and SHARE_PUBLIC. - Updated the IRole interface to reflect the new remote agents permissions structure, enhancing role management capabilities. * feat: Add remote agents settings button to API keys dialog * feat: Update AgentFooter to include remote agent sharing permissions - Refactored access checks to incorporate permissions for sharing remote agents. - Enhanced conditional rendering logic to allow sharing by users with remote agent permissions. - Improved loading state handling for remote agent permissions, ensuring a smoother user experience. * refactor: Update API key creation access check and localization strings - Replaced the access check for creating API keys to use the existing remote agents access check. - Updated localization strings to correct the descriptions for remote agent permissions, ensuring clarity in user interface. * fix: resource permission mapping to include remote agents - Changed the resourceToPermissionMap to use a Partial<Record> for better flexibility. - Added mapping for REMOTE_AGENT permissions, enhancing the sharing capabilities for remote agents. * feat: Implement remote access checks for agent models - Enhanced ListModelsController and GetModelController to include checks for user permissions on remote agents. - Integrated findAccessibleResources to filter agents based on VIEW permission for REMOTE_AGENT. - Updated response handling to ensure users can only access agents they have permissions for, improving security and access control. * fix: Update user parameter type in processUserPlaceholders function - Changed the user parameter type in the processUserPlaceholders function from Partial<Partial<IUser>> to Partial<IUser> for improved type clarity and consistency. * refactor: Simplify integration test structure by removing conditional describe - Replaced conditional describeWithApiKey with a standard describe for all integration tests in responses.spec.js. - This change enhances test clarity and ensures all tests are executed consistently, regardless of the SKIP_INTEGRATION_TESTS flag. * test: Update AgentFooter tests to reflect new grant access dialog ID - Changed test IDs for the grant access dialog in AgentFooter tests to include the resource type, ensuring accurate identification in the test cases. - This update improves test clarity and aligns with recent changes in the component's implementation. * test: Enhance integration tests for Open Responses API - Updated integration tests in responses.spec.js to utilize an authRequest helper for consistent authorization handling across all test cases. - Introduced a test user and API key creation to improve test setup and ensure proper permission checks for remote agents. - Added checks for existing access roles and created necessary roles if they do not exist, enhancing test reliability and coverage. * feat: Extend accessRole schema to include remoteAgent resource type - Updated the accessRole schema to add 'remoteAgent' to the resourceType enum, enhancing the flexibility of role assignments and permissions management. * test: refactored test setup to create a minimal Express app for responses routes, enhancing test structure and maintainability. * test: Enhance abort.spec.js by mocking additional modules for improved test isolation - Updated the test setup in abort.spec.js to include actual implementations of '@librechat/data-schemas' and '@librechat/api' while maintaining mock functionality. - This change improves test reliability and ensures that the tests are more representative of the actual module behavior. * refactor: Update conversation ID generation to use UUID - Replaced the nanoid with uuidv4 for generating conversation IDs in the createResponse function, enhancing uniqueness and consistency in ID generation. * test: Add remote agent access roles to AccessRole model tests - Included additional access roles for remote agents (REMOTE_AGENT_EDITOR, REMOTE_AGENT_OWNER, REMOTE_AGENT_VIEWER) in the AccessRole model tests to ensure comprehensive coverage of role assignments and permissions management. * chore: Add deletion of user agent API keys in user deletion process - Updated the user deletion process in UserController and delete-user.js to include the removal of user agent API keys, ensuring comprehensive cleanup of user data upon account deletion. * test: Add remote agents permissions to permissions.spec.ts - Enhanced the permissions tests by including comprehensive permission settings for remote agents across various scenarios, ensuring accurate validation of access controls for remote agent roles. * chore: Update remote agents translations for clarity and consistency - Removed outdated remote agents translation entries and added revised entries to improve clarity on API key creation and sharing permissions for remote agents. This enhances user understanding of the available functionalities. * feat: Add indexing and TTL for agent API keys - Introduced an index on the `key` field for improved query performance. - Added a TTL index on the `expiresAt` field to enable automatic cleanup of expired API keys, ensuring efficient management of stored keys. * chore: Update API route documentation for clarity - Revised comments in the agents route file to clarify the handling of API key authentication. - Removed outdated endpoint listings to streamline the documentation and focus on current functionality. --------- Co-authored-by: Max Sanna <max@maxsanna.com>
2026-01-26 10:50:30 -05:00
'remote-agents': {
schema: remoteAgentsPermissionsSchema,
permissionType: PermissionTypes.REMOTE_AGENTS,
errorMessage: 'Invalid remote agents permissions.',
},
};
/**
* Generic handler for updating permissions
* @param {string} permissionKey - The key from permissionConfigs
* @returns {Function} Express route handler
*/
const createPermissionUpdateHandler = (permissionKey) => {
const config = permissionConfigs[permissionKey];
return async (req, res) => {
const { roleName: _r } = req.params;
// TODO: TEMP, use a better parsing for roleName
const roleName = _r.toUpperCase();
const updates = req.body;
try {
const parsedUpdates = config.schema.partial().parse(updates);
const role = await getRoleByName(roleName);
if (!role) {
return res.status(404).send({ message: 'Role not found' });
}
const currentPermissions =
role.permissions?.[config.permissionType] || role[config.permissionType] || {};
const mergedUpdates = {
permissions: {
...role.permissions,
[config.permissionType]: {
...currentPermissions,
...parsedUpdates,
},
},
};
const updatedRole = await updateRoleByName(roleName, mergedUpdates);
res.status(200).send(updatedRole);
} catch (error) {
return res.status(400).send({ message: config.errorMessage, error: error.errors });
}
};
};
🗨️ feat: Prompts (#3131) * 🗨️ feat: Prompts (#7) * WIP: MERGE prompts/frontend (#1) * added schema for prompt and promptgroup, added model methods for prompts, added routes for prompts * * updated promptGroup Schema * updated model methods for prompts (get, add, delete) * slight fixes in prompt routes * * Created Files Management components * Created Vector Stores components * Added file management route in the routes folder * Completed UI for Files list, Compeleted UI for vector stores list, Completed UI for upload file modal, Completed UI for preview file, Completed UI for preview vector store * Fixed style and UI fixes for file dashboard, file list and vector stores list * added responsiveness classes for vector store page * fixed responsiveness of file page, dashboard page, and main page * fixed styling and responsiveness issues on dashboard page, file list page and vector store page * added queries and mutations for prompts and promptGroups, added relevant endpoints in data-provider, added relevant components prompts, added and updated relevant APIs * added types on mutation queries data service, updated prompt attributes * feature: Prompts and prompt groups management, added relevant APIs, added types for data service/queries/mutations, added relevant mutation and queries * chore: typing clarifications * added drop down on prompts mgmt dashboard * Fixes: fixed version switching issue on tags update or labels update, added cross button on create prompt group, fixed list updation on prompt group renaiming, added CSV upload button * Feature: Added oneliner and category attributes in prompt group, added schema for categories, added schema methods and route for categories * chore: typing and lint issues * chore: more type and linter fixes * chore: linting * chore: prompt controller and backend typing example; MOVE TO CONTROLLER DIRECTORY * chore: more type fixes * style: prompt name changes * chore: more type changes, and stateful prompt name change without flickering * fix: Return result of savePrompt in patchPrompt API endpoint * fix: navigation prompt queries; refactor: name 'prompt-groups' to just 'groups' * refactor: fetch prompt groups rewrite * refactor(prompts): query/mutation statefulness * refactor: remove `isActive` field * refactor: remove labels, consolidate logic * style: width, layout shift * refactor: improve hover toggle behavior and styling * refactor: add useParams hook to PromptListItem for dynamic rendering and add timeout ref for blur timeout * chore: hide upload button * refactor: import Button component from correct location in PromptSidePanel * style: prompt editor styling * style: fix more layout shifts * style: container scroll * refactor: Rename CreatePrompt component to CreatePromptForm * refactor: use react-hook-form * refactor: Add Prompts components and routes to Dashboard * style: skeletons for loading * fix: optimize makePromptProduction * refactor: consolidate variables * feat: create prompt form validation * refactor: Consolidate variables and update mutation hooks * style: minor touchups * chore: Update lucide-react npm dependency to version 0.394.0 and npm audit fix * refactor: add a new icon for the Prompts heading. * style: Update PromptsView heading to use h1 instead of h2 and other minor margin issues * chore: wording * refactor: Update PromptsView heading to use h1 instead of h2, consolidate variables, and add new icons * refactor: Prompts Button for Mobile * feature: added category field in prompt group, added relevant API and static data on BE to support FE UI for category in prompt group * chore: template for prompt cards --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * WIP: Prompts/frontend Continued (#2) * chore: loading style, remove unused component * feat: Add CategorySelector component for prompt group category selection * feat: add categories to create prompt * feat: prompt versions styling * feat: optimistic updates for prompt production state * refactor: optimize form state and show if prompt field is dirty with cross icon, also other styling changes * chore: remove unused code and localizations * fix: light mode styling * WIP: SidePanel Prompts * refactor: move to groups directory * refactor: rename GroupsSidePanel to GroupSidePanel and update imports * style: ListCard * refactor: isProduction changes * refactor: infinite query with productionPrompt * refactor: optimize snippets and prompts, and styling * refactor: Update getSnippet function to accept a length parameter * chore: localizations * feat: prompts navigation to chat and vice versa * fix: create prompt * feat: remember last selected category for creating prompts * fix(promptGroups): fix pagination and add usePromptGroupsNav hook * Prompts/frontend 3 (#3) * fix: stateful issues with prompt groups * style: improved layout * refactor: improve variable naming in Eng.ts * refactor: theme selector styling improvements * added prompt cards on chat new page, with dark mode, added API to fetch random prompts, added types for useQuery Slightly improved usePromptGroupNav logic to fetch updated result for pageSize, updated prompt cards view with darkmode and responsiveness fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page * WIP: Prompts/frontend (#4) * fix: optimize and fix paginated query * fix: remove unique constraint on names * refactor: button links and styling * style: menu border light mode * feat: Add Auto-Send Switch component for prompts groups * refactor(ChatView): use form context for submission text * chore: clear convo state on navigation to dashboard routes * chore: save prompt edit name on tab, remove console log * feat: basic prompt submission * refactor: move Auto-Send Switch * style(ListCard): border styling * feat: Add function to detect variables in text * feat: Add OriginalDialog component to UI library * chore(ui): Update SelectDropDown options list class to use text-xs size * refactor: submitMessage hook now includes submitPrompt, make compatible to document query selector * WIP: Variable Dialog * feat: variable submission working for both auto-send and non-autosend * feat: dashboard breadcrumbs and prompts/chat navigation * refactor: dashboard breadcrumb and dashboard link to chat navigation * refactor: Update VariableDialog and VariableForm styles * Prompts: Admin features (#5) * fix: link issue * fix: usePromptGroupsNav add missing dep. * style: dashbreadcrumb and sidepanel text color * temp fix: remove refetch on pageNumber change * fix: handle multiple variable replacement * WIP: create project schema and add project groups to fetch * feat: Add functionality to add prompt group IDs to a project * feat: Add caching for startup config in config route * chore: remove prompt landing * style: Update Skeleton component with additional background styling * chore: styling and types * WIP: SharePrompt first draft * feat(SharePrompt): form validation * feat: shared global indicators * refactor: prompt details * refactor: change NoPromptGroup directory * feat: preview prompt * feat: remove/add global prompts, add rbac-related enums * refactor: manage prompts location * WIP: first draft admin settings for prompts * feat: SystemRoles enum * refactor: update PromptDetails component styling * style: ellipsis custom class for showing more preview text * WIP: initial role schema and initialization * style: improved margins for single unordered lists * fix: use custom chat form context to prevent re-renders from FormProvider * feat: Role mutations for Prompt Permissions * feat: fetch user role * feat: update AdminSettings form default values from user role values * refactor: rename PromptPermissions to Permissions for general definitions * feat: initial role checks * feat: Add optional `bodyProps` parameter to generateCheckAccess middleware * refactor: UI access checks * Prompts: delete (#6) * Fixed delete prompt version API, fixed types and logic for prompt version deletion, updated prompt delete mutation logic * chore: Update return type of deletePrompt function in Prompt.js --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * chore: Update package-lock.json version to 0.7.4-rc1 and fast-xml-parser to 4.4.0 * feat: toast for saving admin settings, add timer no-access navigation * feat: always make prod * feat: Add localization to category labels in CategorySelector component * feat: Update category label localization in CategorySelector component * fix: Enable making prompt production in Prompt API --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * feat: Add helper fn for dark mode detection in ThemeProvider * style: surface-primary definition * fix(useHasAccess): utilize user.role and not just USER role * fix: empty category and role fetch * refactort: increase max height to options list and use label if no localization is found * fix: update CategorySelector to handle empty category value and improve localization * refactor: move prompts to own store/reactquery modules, add in filter WIP * refactor: Rename AutoSendSwitch to AutoSendPrompt * style: theming commit * style: fix slight coloring issue for convos in dark mode * style: better composition for prompts side panel * style: remove gray-750 and make it gray-850 * chore: adjust theming * feat: filter all prompt groups and properly remove prompts from projects * refactor: optimize delete prompt groups further * chore: localization * feat: Add uniqueProperty filtering to normalizeData function * WIP: filter prompts * chore: Update FilterPrompts component to include User icon in FilterItem * feat(FilterPrompts): set categories * feat: more system filters and show selected category icon * style: always make prod, flips switch to avoid mis-clicks * style: ui/ux loading/no prompts * chore: style FilterPrompts ChatView * fix: handle missing role edge case * style: special variables * feat: special variables * refactor: improve replaceSpecialVars function in prompts.ts * feat: simple/advanced editor modes * chore: bump versions * feat: localizations and hide production button on simple mode * fix: error connecting layout shift * fix: prompts CRUD for admins * fix: secure single group fetch * style: sidepanel styling * style(PromptName): bring edit button closer to name * style: mobile prompts header * style: mobile prompts header continued * style: align send prompts switch right * feat: description * Update special variables description in Eng.ts * feat: update/create/preview oneliner * fix: allow empty oneliner update * style: loading improvement and always make selected prompt Production if simple mode * fix: production index set and remove unused props * fix(ci): mock initializeRoles * fix: address #3128 * fix: address #3128 * feat: add deletion confirmation dialog * fix: mobile UI issues * style: prompt library UI update * style: focus, logcal tab order * style: Refactor SelectDropDown component to improve code readability and maintainability * chore: bump data-provider * chore: fix labels * refactor: confirm delete prompt version --------- Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
2024-06-20 20:24:32 -04:00
/**
* GET /api/roles/:roleName
* Get a specific role by name
*/
router.get('/:roleName', async (req, res) => {
const { roleName: _r } = req.params;
// TODO: TEMP, use a better parsing for roleName
const roleName = _r.toUpperCase();
try {
📜 feat: Implement System Grants for Capability-Based Authorization (#11896) * feat: Implement System Grants for Role-Based Capabilities - Added a new `systemGrant` model and associated methods to manage role-based capabilities within the application. - Introduced middleware functions `hasCapability` and `requireCapability` to check user permissions based on their roles. - Updated the database seeding process to include system grants for the ADMIN role, ensuring all necessary capabilities are assigned on startup. - Enhanced type definitions and schemas to support the new system grant functionality, improving overall type safety and clarity in the codebase. * test: Add unit tests for capabilities middleware and system grant methods - Introduced comprehensive unit tests for the capabilities middleware, including `hasCapability` and `requireCapability`, ensuring proper permission checks based on user roles. - Added tests for the `SystemGrant` methods, verifying the seeding of system grants, capability granting, and revocation processes. - Enhanced test coverage for edge cases, including idempotency of grant operations and handling of unexpected errors in middleware. - Utilized mocks for database interactions to isolate tests and improve reliability. * refactor: Transition to Capability-Based Access Control - Replaced role-based access checks with capability-based checks across various middleware and routes, enhancing permission management. - Introduced `hasCapability` and `requireCapability` functions to streamline capability verification for user actions. - Updated relevant routes and middleware to utilize the new capability system, ensuring consistent permission enforcement. - Enhanced type definitions and added tests for the new capability functions, improving overall code reliability and maintainability. * test: Enhance capability-based access tests for ADMIN role - Updated tests to reflect the new capability-based access control, specifically for the ADMIN role. - Modified test descriptions to clarify that users with the MANAGE_AGENTS capability can bypass permission checks. - Seeded capabilities for the ADMIN role in multiple test files to ensure consistent permission checks across different routes and middleware. - Improved overall test coverage for capability verification, ensuring robust permission management. * test: Update capability tests for MCP server access - Renamed test to reflect the correct capability for bypassing permission checks, changing from MANAGE_AGENTS to MANAGE_MCP_SERVERS. - Updated seeding of capabilities for the ADMIN role to align with the new capability structure. - Ensured consistency in capability definitions across tests and middleware for improved permission management. * feat: Add hasConfigCapability for enhanced config access control - Introduced `hasConfigCapability` function to check user permissions for managing or reading specific config sections. - Updated middleware to export the new capability function, ensuring consistent access control across the application. - Enhanced unit tests to cover various scenarios for the new capability, improving overall test coverage and reliability. * fix: Update tenantId filter in createSystemGrantMethods - Added a condition to set tenantId filter to { $exists: false } when tenantId is null, ensuring proper handling of cases where tenantId is not provided. - This change improves the robustness of the system grant methods by explicitly managing the absence of tenantId in the filter logic. * fix: account deletion capability check - Updated the `canDeleteAccount` middleware to ensure that the `hasManageUsers` capability check only occurs if a user is present, preventing potential errors when the user object is undefined. - This change improves the robustness of the account deletion logic by ensuring proper handling of user permissions. * refactor: Optimize seeding of system grants for ADMIN role - Replaced sequential capability granting with parallel execution using Promise.all in the seedSystemGrants function. - This change improves performance and efficiency during the initialization of system grants, ensuring all capabilities are granted concurrently. * refactor: Simplify systemGrantSchema index definition - Removed the sparse option from the unique index on principalType, principalId, capability, and tenantId in the systemGrantSchema. - This change streamlines the index definition, potentially improving query performance and clarity in the schema design. * refactor: Reorganize role capability check in roles route - Moved the capability check for reading roles to occur after parsing the roleName, improving code clarity and structure. - This change ensures that the authorization logic is consistently applied before fetching role details, enhancing overall permission management. * refactor: Remove unused ISystemGrant interface from systemCapabilities.ts - Deleted the ISystemGrant interface as it was no longer needed, streamlining the code and improving clarity. - This change helps reduce clutter in the file and focuses on relevant capabilities for the system. * refactor: Migrate SystemCapabilities to data-schemas - Replaced imports of SystemCapabilities from 'librechat-data-provider' with imports from '@librechat/data-schemas' across multiple files. - This change centralizes the management of system capabilities, improving code organization and maintainability. * refactor: Update account deletion middleware and capability checks - Modified the `canDeleteAccount` middleware to ensure that the account deletion permission is only granted to users with the `MANAGE_USERS` capability, improving security and clarity in permission management. - Enhanced error logging for unauthorized account deletion attempts, providing better insights into permission issues. - Updated the `capabilities.ts` file to ensure consistent handling of user authentication checks, improving robustness in capability verification. - Refined type definitions in `systemGrant.ts` and `systemGrantMethods.ts` to utilize the `PrincipalType` enum, enhancing type safety and code clarity. * refactor: Extract principal ID normalization into a separate function - Introduced `normalizePrincipalId` function to streamline the normalization of principal IDs based on their type, enhancing code clarity and reusability. - Updated references in `createSystemGrantMethods` to utilize the new normalization function, improving maintainability and reducing code duplication. * test: Add unit tests for principalId normalization in systemGrant - Introduced tests for the `grantCapability`, `revokeCapability`, and `getCapabilitiesForPrincipal` methods to verify correct handling of principalId normalization between string and ObjectId formats. - Enhanced the `capabilities.ts` middleware to utilize the `PrincipalType` enum for improved type safety. - Added a new utility function `normalizePrincipalId` to streamline principal ID normalization logic, ensuring consistent behavior across the application. * feat: Introduce capability implications and enhance system grant methods - Added `CapabilityImplications` to define relationships between broader and implied capabilities, allowing for more intuitive permission checks. - Updated `createSystemGrantMethods` to expand capability queries to include implied capabilities, improving authorization logic. - Enhanced `systemGrantSchema` to include an `expiresAt` field for future TTL enforcement of grants, and added validation to ensure `tenantId` is not set to null. - Documented authorization requirements for prompt group and prompt deletion methods to clarify access control expectations. * test: Add unit tests for canDeleteAccount middleware - Introduced unit tests for the `canDeleteAccount` middleware to verify account deletion permissions based on user roles and capabilities. - Covered scenarios for both allowed and blocked account deletions, including checks for ADMIN users with the `MANAGE_USERS` capability and handling of undefined user cases. - Enhanced test structure to ensure clarity and maintainability of permission checks in the middleware. * fix: Add principalType enum validation to SystemGrant schema Without enum validation, any string value was accepted for principalType and silently stored. Invalid documents would never match capability queries, creating phantom grants impossible to diagnose without raw DB inspection. All other ACL models in the codebase validate this field. * fix: Replace seedSystemGrants Promise.all with bulkWrite for concurrency safety When two server instances start simultaneously (K8s rolling deploy, PM2 cluster), both call seedSystemGrants. With Promise.all + findOneAndUpdate upsert, both instances may attempt to insert the same documents, causing E11000 duplicate key errors that crash server startup. bulkWrite with ordered:false handles concurrent upserts gracefully and reduces 17 individual round trips to a single network call. The returned documents (previously discarded) are no longer fetched. * perf: Add AsyncLocalStorage per-request cache for capability checks Every hasCapability call previously required 2 DB round trips (getUserPrincipals + SystemGrant.exists) — replacing what were O(1) string comparisons. Routes like patchPromptGroup triggered this twice, and hasConfigCapability's fallback path resolved principals twice. This adds a per-request AsyncLocalStorage cache that: - Caches resolved principals (same for all checks within one request) - Caches capability check results (same user+cap = same answer) - Automatically scoped to request lifetime (no stale grants) - Falls through to DB when no store exists (background jobs, tests) - Requires no signature changes to hasCapability The capabilityContextMiddleware is registered at the app level before all routes, initializing a fresh store per request. * fix: Add error handling for inline hasCapability calls canDeleteAccount, fetchAssistants, and validateAuthor all call hasCapability without try-catch. These were previously O(1) string comparisons that could never throw. Now they hit the database and can fail on connection timeout or transient errors. Wrap each call in try-catch, defaulting to deny (false) on error. This ensures a DB hiccup returns a clean 403 instead of an unhandled 500 with a stack trace. * test: Add canDeleteAccount DB-error resilience test Tests that hasCapability rejection (e.g., DB timeout) results in a clean 403 rather than an unhandled exception. Validates the error handling added in the previous commit. * refactor: Use barrel import for hasCapability in validateAuthor Import from ~/server/middleware barrel instead of directly from ~/server/middleware/roles/capabilities for consistency with other non-middleware consumers. Files within the middleware barrel itself must continue using direct imports to avoid circular requires. * refactor: Remove misleading pre('save') hook from SystemGrant schema The pre('save') hook normalized principalId for USER/GROUP principals, but the primary write path (grantCapability) uses findOneAndUpdate — which does not trigger save hooks. The normalization was already handled explicitly in grantCapability itself. The hook created a false impression of schema-level enforcement that only covered save()/create() paths. Replace with a comment documenting that all writes must go through grantCapability. * feat: Add READ_ASSISTANTS capability to complete manage/read pair Every other managed resource had a paired READ_X / MANAGE_X capability except assistants. This adds READ_ASSISTANTS and registers the MANAGE_ASSISTANTS → READ_ASSISTANTS implication in CapabilityImplications, enabling future read-only assistant visibility grants. * chore: Reorder systemGrant methods for clarity Moved hasCapabilityForPrincipals to a more logical position in the returned object of createSystemGrantMethods, improving code readability. This change also maintains the inclusion of seedSystemGrants in the export, ensuring all necessary methods are available. * fix: Wrap seedSystemGrants in try-catch to avoid blocking startup Seeding capabilities is idempotent and will succeed on the next restart. A transient DB error during seeding should not prevent the server from starting — log the error and continue. * refactor: Improve capability check efficiency and add audit logging Move hasCapability calls after cheap early-exits in validateAuthor and fetchAssistants so the DB check only runs when its result matters. Add logger.debug on every capability bypass grant across all 7 call sites for auditability, and log errors in catch blocks instead of silently swallowing them. * test: Add integration tests for AsyncLocalStorage capability caching Exercises the full vertical — ALS context, generateCapabilityCheck, real getUserPrincipals, real hasCapabilityForPrincipals, real MongoDB via MongoMemoryServer. Covers per-request caching, cross-context isolation, concurrent request isolation, negative caching, capability implications, tenant scoping, group-based grants, and requireCapability middleware. * test: Add systemGrant data-layer and ALS edge-case integration tests systemGrant.spec.ts (51 tests): Full integration tests for all systemGrant methods against real MongoDB — grant/revoke lifecycle, principalId normalization (string→ObjectId for USER/GROUP, string for ROLE), capability implications (both directions), tenant scoping, schema validation (null tenantId, invalid enum, required fields, unique compound index). capabilities.integration.spec.ts (27 tests): Adds ALS edge cases — missing context degrades gracefully with no caching (background jobs, child processes), nested middleware creates independent inner context, optional-chaining safety when store is undefined, mid-request grant changes are invisible due to result caching, requireCapability works without ALS, and interleaved concurrent contexts maintain isolation. * fix: Add worker thread guards to capability ALS usage Detect when hasCapability or capabilityContextMiddleware is called from a worker thread (where ALS context does not propagate from the parent). hasCapability logs a warn-once per factory instance; the middleware logs an error since mounting Express middleware in a worker is likely a misconfiguration. Both continue to function correctly — the guard is observability, not a hard block. * fix: Include tenantId in ALS principal cache key for tenant isolation The principal cache key was user.id:user.role, which would reuse cached principals across tenants for the same user within a request. When getUserPrincipals gains tenant-scoped group resolution, principals from tenant-a would incorrectly serve tenant-b checks. Changed to user.id:user.role:user.tenantId to prevent cross-tenant cache hits. Adds integration test proving separate principal lookups per tenantId. * test: Remove redundant mocked capabilities.spec.js The JS wrapper test (7 tests, all mocked) is a strict subset of capabilities.integration.spec.ts (28 tests, real MongoDB). Every scenario it covered — hasCapability true/false, tenantId passthrough, requireCapability 403/500, error handling — is tested with higher fidelity in the integration suite. * test: Replace mocked canDeleteAccount tests with real MongoDB integration Remove hasCapability mock — tests now exercise the full capability chain against real MongoDB (getUserPrincipals, hasCapabilityForPrincipals, SystemGrant collection). Only mocks remaining are logger and cache. Adds new coverage: admin role without grant is blocked, user-level grant bypasses deletion restriction, null user handling. * test: Add comprehensive tests for ACL entry management and user group methods Introduces new tests for `deleteAclEntries`, `bulkWriteAclEntries`, and `findPublicResourceIds` in `aclEntry.spec.ts`, ensuring proper functionality for deleting and bulk managing ACL entries. Additionally, enhances `userGroup.spec.ts` with tests for finding groups by ID and name pattern, including external ID matching and source filtering. These changes improve coverage and validate the integrity of ACL and user group operations against real MongoDB interactions. * refactor: Update capability checks and logging for better clarity and error handling Replaced `MANAGE_USERS` with `ACCESS_ADMIN` in the `canDeleteAccount` middleware and related tests to align with updated permission structure. Enhanced logging in various middleware functions to use `logger.warn` for capability check failures, providing clearer error messages. Additionally, refactored capability checks in the `patchPromptGroup` and `validateAuthor` functions to improve readability and maintainability. This commit also includes adjustments to the `systemGrant` methods to implement retry logic for transient failures during capability seeding, ensuring robustness in the face of database errors. * refactor: Enhance logging and retry logic in seedSystemGrants method Updated the logging format in the seedSystemGrants method to include error messages for better clarity. Improved the retry mechanism by explicitly mocking multiple failures in tests, ensuring robust error handling during transient database issues. Additionally, refined imports in the systemGrant schema for better type management. * refactor: Consolidate imports in canDeleteAccount middleware Merged logger and SystemCapabilities imports from the data-schemas module into a single line for improved readability and maintainability of the code. This change streamlines the import statements in the canDeleteAccount middleware. * test: Enhance systemGrant tests for error handling and capability validation Added tests to the systemGrant methods to handle various error scenarios, including E11000 race conditions, invalid ObjectId strings for USER and GROUP principals, and invalid capability strings. These enhancements improve the robustness of the capability granting and revoking logic, ensuring proper error propagation and validation of inputs. * fix: Wrap hasCapability calls in deny-by-default try-catch at remaining sites canAccessResource, files.js, and roles.js all had hasCapability inside outer try-catch blocks that returned 500 on DB failure instead of falling through to the regular ACL check. This contradicts the deny-by-default pattern used everywhere else. Also removes raw error.message from the roles.js 500 response to prevent internal host/connection info leaking to clients. * fix: Normalize user ID in canDeleteAccount before passing to hasCapability requireCapability normalizes req.user.id via _id?.toString() fallback, but canDeleteAccount passed raw req.user directly. If req.user.id is absent (some auth layers only populate _id), getUserPrincipals received undefined, silently returning empty principals and blocking the bypass. * fix: Harden systemGrant schema and type safety - Reject empty string tenantId in schema validator (was only blocking null; empty string silently orphaned documents) - Fix reverseImplications to use BaseSystemCapability[] instead of string[], preserving the narrow discriminated type - Document READ_ASSISTANTS as reserved/unenforced * test: Use fake timers for seedSystemGrants retry tests and add tenantId validation - Switch retry tests to jest.useFakeTimers() to eliminate 3+ seconds of real setTimeout delays per test run - Add regression test for empty-string tenantId rejection * docs: Add TODO(#12091) comments for tenant-scoped capability gaps In multi-tenant mode, platform-level grants (no tenantId) won't match tenant-scoped queries, breaking admin access. getUserPrincipals also returns cross-tenant group memberships. Both need fixes in #12091.
2026-03-07 13:56:32 -05:00
let hasReadRoles = false;
try {
hasReadRoles = await hasCapability(req.user, SystemCapabilities.READ_ROLES);
} catch (err) {
logger.warn(`[GET /roles/:roleName] capability check failed: ${err.message}`);
}
if (!hasReadRoles && (roleName === SystemRoles.ADMIN || !roleDefaults[roleName])) {
return res.status(403).send({ message: 'Unauthorized' });
}
🗨️ feat: Prompts (#3131) * 🗨️ feat: Prompts (#7) * WIP: MERGE prompts/frontend (#1) * added schema for prompt and promptgroup, added model methods for prompts, added routes for prompts * * updated promptGroup Schema * updated model methods for prompts (get, add, delete) * slight fixes in prompt routes * * Created Files Management components * Created Vector Stores components * Added file management route in the routes folder * Completed UI for Files list, Compeleted UI for vector stores list, Completed UI for upload file modal, Completed UI for preview file, Completed UI for preview vector store * Fixed style and UI fixes for file dashboard, file list and vector stores list * added responsiveness classes for vector store page * fixed responsiveness of file page, dashboard page, and main page * fixed styling and responsiveness issues on dashboard page, file list page and vector store page * added queries and mutations for prompts and promptGroups, added relevant endpoints in data-provider, added relevant components prompts, added and updated relevant APIs * added types on mutation queries data service, updated prompt attributes * feature: Prompts and prompt groups management, added relevant APIs, added types for data service/queries/mutations, added relevant mutation and queries * chore: typing clarifications * added drop down on prompts mgmt dashboard * Fixes: fixed version switching issue on tags update or labels update, added cross button on create prompt group, fixed list updation on prompt group renaiming, added CSV upload button * Feature: Added oneliner and category attributes in prompt group, added schema for categories, added schema methods and route for categories * chore: typing and lint issues * chore: more type and linter fixes * chore: linting * chore: prompt controller and backend typing example; MOVE TO CONTROLLER DIRECTORY * chore: more type fixes * style: prompt name changes * chore: more type changes, and stateful prompt name change without flickering * fix: Return result of savePrompt in patchPrompt API endpoint * fix: navigation prompt queries; refactor: name 'prompt-groups' to just 'groups' * refactor: fetch prompt groups rewrite * refactor(prompts): query/mutation statefulness * refactor: remove `isActive` field * refactor: remove labels, consolidate logic * style: width, layout shift * refactor: improve hover toggle behavior and styling * refactor: add useParams hook to PromptListItem for dynamic rendering and add timeout ref for blur timeout * chore: hide upload button * refactor: import Button component from correct location in PromptSidePanel * style: prompt editor styling * style: fix more layout shifts * style: container scroll * refactor: Rename CreatePrompt component to CreatePromptForm * refactor: use react-hook-form * refactor: Add Prompts components and routes to Dashboard * style: skeletons for loading * fix: optimize makePromptProduction * refactor: consolidate variables * feat: create prompt form validation * refactor: Consolidate variables and update mutation hooks * style: minor touchups * chore: Update lucide-react npm dependency to version 0.394.0 and npm audit fix * refactor: add a new icon for the Prompts heading. * style: Update PromptsView heading to use h1 instead of h2 and other minor margin issues * chore: wording * refactor: Update PromptsView heading to use h1 instead of h2, consolidate variables, and add new icons * refactor: Prompts Button for Mobile * feature: added category field in prompt group, added relevant API and static data on BE to support FE UI for category in prompt group * chore: template for prompt cards --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * WIP: Prompts/frontend Continued (#2) * chore: loading style, remove unused component * feat: Add CategorySelector component for prompt group category selection * feat: add categories to create prompt * feat: prompt versions styling * feat: optimistic updates for prompt production state * refactor: optimize form state and show if prompt field is dirty with cross icon, also other styling changes * chore: remove unused code and localizations * fix: light mode styling * WIP: SidePanel Prompts * refactor: move to groups directory * refactor: rename GroupsSidePanel to GroupSidePanel and update imports * style: ListCard * refactor: isProduction changes * refactor: infinite query with productionPrompt * refactor: optimize snippets and prompts, and styling * refactor: Update getSnippet function to accept a length parameter * chore: localizations * feat: prompts navigation to chat and vice versa * fix: create prompt * feat: remember last selected category for creating prompts * fix(promptGroups): fix pagination and add usePromptGroupsNav hook * Prompts/frontend 3 (#3) * fix: stateful issues with prompt groups * style: improved layout * refactor: improve variable naming in Eng.ts * refactor: theme selector styling improvements * added prompt cards on chat new page, with dark mode, added API to fetch random prompts, added types for useQuery Slightly improved usePromptGroupNav logic to fetch updated result for pageSize, updated prompt cards view with darkmode and responsiveness fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page * WIP: Prompts/frontend (#4) * fix: optimize and fix paginated query * fix: remove unique constraint on names * refactor: button links and styling * style: menu border light mode * feat: Add Auto-Send Switch component for prompts groups * refactor(ChatView): use form context for submission text * chore: clear convo state on navigation to dashboard routes * chore: save prompt edit name on tab, remove console log * feat: basic prompt submission * refactor: move Auto-Send Switch * style(ListCard): border styling * feat: Add function to detect variables in text * feat: Add OriginalDialog component to UI library * chore(ui): Update SelectDropDown options list class to use text-xs size * refactor: submitMessage hook now includes submitPrompt, make compatible to document query selector * WIP: Variable Dialog * feat: variable submission working for both auto-send and non-autosend * feat: dashboard breadcrumbs and prompts/chat navigation * refactor: dashboard breadcrumb and dashboard link to chat navigation * refactor: Update VariableDialog and VariableForm styles * Prompts: Admin features (#5) * fix: link issue * fix: usePromptGroupsNav add missing dep. * style: dashbreadcrumb and sidepanel text color * temp fix: remove refetch on pageNumber change * fix: handle multiple variable replacement * WIP: create project schema and add project groups to fetch * feat: Add functionality to add prompt group IDs to a project * feat: Add caching for startup config in config route * chore: remove prompt landing * style: Update Skeleton component with additional background styling * chore: styling and types * WIP: SharePrompt first draft * feat(SharePrompt): form validation * feat: shared global indicators * refactor: prompt details * refactor: change NoPromptGroup directory * feat: preview prompt * feat: remove/add global prompts, add rbac-related enums * refactor: manage prompts location * WIP: first draft admin settings for prompts * feat: SystemRoles enum * refactor: update PromptDetails component styling * style: ellipsis custom class for showing more preview text * WIP: initial role schema and initialization * style: improved margins for single unordered lists * fix: use custom chat form context to prevent re-renders from FormProvider * feat: Role mutations for Prompt Permissions * feat: fetch user role * feat: update AdminSettings form default values from user role values * refactor: rename PromptPermissions to Permissions for general definitions * feat: initial role checks * feat: Add optional `bodyProps` parameter to generateCheckAccess middleware * refactor: UI access checks * Prompts: delete (#6) * Fixed delete prompt version API, fixed types and logic for prompt version deletion, updated prompt delete mutation logic * chore: Update return type of deletePrompt function in Prompt.js --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * chore: Update package-lock.json version to 0.7.4-rc1 and fast-xml-parser to 4.4.0 * feat: toast for saving admin settings, add timer no-access navigation * feat: always make prod * feat: Add localization to category labels in CategorySelector component * feat: Update category label localization in CategorySelector component * fix: Enable making prompt production in Prompt API --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * feat: Add helper fn for dark mode detection in ThemeProvider * style: surface-primary definition * fix(useHasAccess): utilize user.role and not just USER role * fix: empty category and role fetch * refactort: increase max height to options list and use label if no localization is found * fix: update CategorySelector to handle empty category value and improve localization * refactor: move prompts to own store/reactquery modules, add in filter WIP * refactor: Rename AutoSendSwitch to AutoSendPrompt * style: theming commit * style: fix slight coloring issue for convos in dark mode * style: better composition for prompts side panel * style: remove gray-750 and make it gray-850 * chore: adjust theming * feat: filter all prompt groups and properly remove prompts from projects * refactor: optimize delete prompt groups further * chore: localization * feat: Add uniqueProperty filtering to normalizeData function * WIP: filter prompts * chore: Update FilterPrompts component to include User icon in FilterItem * feat(FilterPrompts): set categories * feat: more system filters and show selected category icon * style: always make prod, flips switch to avoid mis-clicks * style: ui/ux loading/no prompts * chore: style FilterPrompts ChatView * fix: handle missing role edge case * style: special variables * feat: special variables * refactor: improve replaceSpecialVars function in prompts.ts * feat: simple/advanced editor modes * chore: bump versions * feat: localizations and hide production button on simple mode * fix: error connecting layout shift * fix: prompts CRUD for admins * fix: secure single group fetch * style: sidepanel styling * style(PromptName): bring edit button closer to name * style: mobile prompts header * style: mobile prompts header continued * style: align send prompts switch right * feat: description * Update special variables description in Eng.ts * feat: update/create/preview oneliner * fix: allow empty oneliner update * style: loading improvement and always make selected prompt Production if simple mode * fix: production index set and remove unused props * fix(ci): mock initializeRoles * fix: address #3128 * fix: address #3128 * feat: add deletion confirmation dialog * fix: mobile UI issues * style: prompt library UI update * style: focus, logcal tab order * style: Refactor SelectDropDown component to improve code readability and maintainability * chore: bump data-provider * chore: fix labels * refactor: confirm delete prompt version --------- Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
2024-06-20 20:24:32 -04:00
const role = await getRoleByName(roleName, '-_id -__v');
if (!role) {
return res.status(404).send({ message: 'Role not found' });
}
res.status(200).send(role);
} catch (error) {
📜 feat: Implement System Grants for Capability-Based Authorization (#11896) * feat: Implement System Grants for Role-Based Capabilities - Added a new `systemGrant` model and associated methods to manage role-based capabilities within the application. - Introduced middleware functions `hasCapability` and `requireCapability` to check user permissions based on their roles. - Updated the database seeding process to include system grants for the ADMIN role, ensuring all necessary capabilities are assigned on startup. - Enhanced type definitions and schemas to support the new system grant functionality, improving overall type safety and clarity in the codebase. * test: Add unit tests for capabilities middleware and system grant methods - Introduced comprehensive unit tests for the capabilities middleware, including `hasCapability` and `requireCapability`, ensuring proper permission checks based on user roles. - Added tests for the `SystemGrant` methods, verifying the seeding of system grants, capability granting, and revocation processes. - Enhanced test coverage for edge cases, including idempotency of grant operations and handling of unexpected errors in middleware. - Utilized mocks for database interactions to isolate tests and improve reliability. * refactor: Transition to Capability-Based Access Control - Replaced role-based access checks with capability-based checks across various middleware and routes, enhancing permission management. - Introduced `hasCapability` and `requireCapability` functions to streamline capability verification for user actions. - Updated relevant routes and middleware to utilize the new capability system, ensuring consistent permission enforcement. - Enhanced type definitions and added tests for the new capability functions, improving overall code reliability and maintainability. * test: Enhance capability-based access tests for ADMIN role - Updated tests to reflect the new capability-based access control, specifically for the ADMIN role. - Modified test descriptions to clarify that users with the MANAGE_AGENTS capability can bypass permission checks. - Seeded capabilities for the ADMIN role in multiple test files to ensure consistent permission checks across different routes and middleware. - Improved overall test coverage for capability verification, ensuring robust permission management. * test: Update capability tests for MCP server access - Renamed test to reflect the correct capability for bypassing permission checks, changing from MANAGE_AGENTS to MANAGE_MCP_SERVERS. - Updated seeding of capabilities for the ADMIN role to align with the new capability structure. - Ensured consistency in capability definitions across tests and middleware for improved permission management. * feat: Add hasConfigCapability for enhanced config access control - Introduced `hasConfigCapability` function to check user permissions for managing or reading specific config sections. - Updated middleware to export the new capability function, ensuring consistent access control across the application. - Enhanced unit tests to cover various scenarios for the new capability, improving overall test coverage and reliability. * fix: Update tenantId filter in createSystemGrantMethods - Added a condition to set tenantId filter to { $exists: false } when tenantId is null, ensuring proper handling of cases where tenantId is not provided. - This change improves the robustness of the system grant methods by explicitly managing the absence of tenantId in the filter logic. * fix: account deletion capability check - Updated the `canDeleteAccount` middleware to ensure that the `hasManageUsers` capability check only occurs if a user is present, preventing potential errors when the user object is undefined. - This change improves the robustness of the account deletion logic by ensuring proper handling of user permissions. * refactor: Optimize seeding of system grants for ADMIN role - Replaced sequential capability granting with parallel execution using Promise.all in the seedSystemGrants function. - This change improves performance and efficiency during the initialization of system grants, ensuring all capabilities are granted concurrently. * refactor: Simplify systemGrantSchema index definition - Removed the sparse option from the unique index on principalType, principalId, capability, and tenantId in the systemGrantSchema. - This change streamlines the index definition, potentially improving query performance and clarity in the schema design. * refactor: Reorganize role capability check in roles route - Moved the capability check for reading roles to occur after parsing the roleName, improving code clarity and structure. - This change ensures that the authorization logic is consistently applied before fetching role details, enhancing overall permission management. * refactor: Remove unused ISystemGrant interface from systemCapabilities.ts - Deleted the ISystemGrant interface as it was no longer needed, streamlining the code and improving clarity. - This change helps reduce clutter in the file and focuses on relevant capabilities for the system. * refactor: Migrate SystemCapabilities to data-schemas - Replaced imports of SystemCapabilities from 'librechat-data-provider' with imports from '@librechat/data-schemas' across multiple files. - This change centralizes the management of system capabilities, improving code organization and maintainability. * refactor: Update account deletion middleware and capability checks - Modified the `canDeleteAccount` middleware to ensure that the account deletion permission is only granted to users with the `MANAGE_USERS` capability, improving security and clarity in permission management. - Enhanced error logging for unauthorized account deletion attempts, providing better insights into permission issues. - Updated the `capabilities.ts` file to ensure consistent handling of user authentication checks, improving robustness in capability verification. - Refined type definitions in `systemGrant.ts` and `systemGrantMethods.ts` to utilize the `PrincipalType` enum, enhancing type safety and code clarity. * refactor: Extract principal ID normalization into a separate function - Introduced `normalizePrincipalId` function to streamline the normalization of principal IDs based on their type, enhancing code clarity and reusability. - Updated references in `createSystemGrantMethods` to utilize the new normalization function, improving maintainability and reducing code duplication. * test: Add unit tests for principalId normalization in systemGrant - Introduced tests for the `grantCapability`, `revokeCapability`, and `getCapabilitiesForPrincipal` methods to verify correct handling of principalId normalization between string and ObjectId formats. - Enhanced the `capabilities.ts` middleware to utilize the `PrincipalType` enum for improved type safety. - Added a new utility function `normalizePrincipalId` to streamline principal ID normalization logic, ensuring consistent behavior across the application. * feat: Introduce capability implications and enhance system grant methods - Added `CapabilityImplications` to define relationships between broader and implied capabilities, allowing for more intuitive permission checks. - Updated `createSystemGrantMethods` to expand capability queries to include implied capabilities, improving authorization logic. - Enhanced `systemGrantSchema` to include an `expiresAt` field for future TTL enforcement of grants, and added validation to ensure `tenantId` is not set to null. - Documented authorization requirements for prompt group and prompt deletion methods to clarify access control expectations. * test: Add unit tests for canDeleteAccount middleware - Introduced unit tests for the `canDeleteAccount` middleware to verify account deletion permissions based on user roles and capabilities. - Covered scenarios for both allowed and blocked account deletions, including checks for ADMIN users with the `MANAGE_USERS` capability and handling of undefined user cases. - Enhanced test structure to ensure clarity and maintainability of permission checks in the middleware. * fix: Add principalType enum validation to SystemGrant schema Without enum validation, any string value was accepted for principalType and silently stored. Invalid documents would never match capability queries, creating phantom grants impossible to diagnose without raw DB inspection. All other ACL models in the codebase validate this field. * fix: Replace seedSystemGrants Promise.all with bulkWrite for concurrency safety When two server instances start simultaneously (K8s rolling deploy, PM2 cluster), both call seedSystemGrants. With Promise.all + findOneAndUpdate upsert, both instances may attempt to insert the same documents, causing E11000 duplicate key errors that crash server startup. bulkWrite with ordered:false handles concurrent upserts gracefully and reduces 17 individual round trips to a single network call. The returned documents (previously discarded) are no longer fetched. * perf: Add AsyncLocalStorage per-request cache for capability checks Every hasCapability call previously required 2 DB round trips (getUserPrincipals + SystemGrant.exists) — replacing what were O(1) string comparisons. Routes like patchPromptGroup triggered this twice, and hasConfigCapability's fallback path resolved principals twice. This adds a per-request AsyncLocalStorage cache that: - Caches resolved principals (same for all checks within one request) - Caches capability check results (same user+cap = same answer) - Automatically scoped to request lifetime (no stale grants) - Falls through to DB when no store exists (background jobs, tests) - Requires no signature changes to hasCapability The capabilityContextMiddleware is registered at the app level before all routes, initializing a fresh store per request. * fix: Add error handling for inline hasCapability calls canDeleteAccount, fetchAssistants, and validateAuthor all call hasCapability without try-catch. These were previously O(1) string comparisons that could never throw. Now they hit the database and can fail on connection timeout or transient errors. Wrap each call in try-catch, defaulting to deny (false) on error. This ensures a DB hiccup returns a clean 403 instead of an unhandled 500 with a stack trace. * test: Add canDeleteAccount DB-error resilience test Tests that hasCapability rejection (e.g., DB timeout) results in a clean 403 rather than an unhandled exception. Validates the error handling added in the previous commit. * refactor: Use barrel import for hasCapability in validateAuthor Import from ~/server/middleware barrel instead of directly from ~/server/middleware/roles/capabilities for consistency with other non-middleware consumers. Files within the middleware barrel itself must continue using direct imports to avoid circular requires. * refactor: Remove misleading pre('save') hook from SystemGrant schema The pre('save') hook normalized principalId for USER/GROUP principals, but the primary write path (grantCapability) uses findOneAndUpdate — which does not trigger save hooks. The normalization was already handled explicitly in grantCapability itself. The hook created a false impression of schema-level enforcement that only covered save()/create() paths. Replace with a comment documenting that all writes must go through grantCapability. * feat: Add READ_ASSISTANTS capability to complete manage/read pair Every other managed resource had a paired READ_X / MANAGE_X capability except assistants. This adds READ_ASSISTANTS and registers the MANAGE_ASSISTANTS → READ_ASSISTANTS implication in CapabilityImplications, enabling future read-only assistant visibility grants. * chore: Reorder systemGrant methods for clarity Moved hasCapabilityForPrincipals to a more logical position in the returned object of createSystemGrantMethods, improving code readability. This change also maintains the inclusion of seedSystemGrants in the export, ensuring all necessary methods are available. * fix: Wrap seedSystemGrants in try-catch to avoid blocking startup Seeding capabilities is idempotent and will succeed on the next restart. A transient DB error during seeding should not prevent the server from starting — log the error and continue. * refactor: Improve capability check efficiency and add audit logging Move hasCapability calls after cheap early-exits in validateAuthor and fetchAssistants so the DB check only runs when its result matters. Add logger.debug on every capability bypass grant across all 7 call sites for auditability, and log errors in catch blocks instead of silently swallowing them. * test: Add integration tests for AsyncLocalStorage capability caching Exercises the full vertical — ALS context, generateCapabilityCheck, real getUserPrincipals, real hasCapabilityForPrincipals, real MongoDB via MongoMemoryServer. Covers per-request caching, cross-context isolation, concurrent request isolation, negative caching, capability implications, tenant scoping, group-based grants, and requireCapability middleware. * test: Add systemGrant data-layer and ALS edge-case integration tests systemGrant.spec.ts (51 tests): Full integration tests for all systemGrant methods against real MongoDB — grant/revoke lifecycle, principalId normalization (string→ObjectId for USER/GROUP, string for ROLE), capability implications (both directions), tenant scoping, schema validation (null tenantId, invalid enum, required fields, unique compound index). capabilities.integration.spec.ts (27 tests): Adds ALS edge cases — missing context degrades gracefully with no caching (background jobs, child processes), nested middleware creates independent inner context, optional-chaining safety when store is undefined, mid-request grant changes are invisible due to result caching, requireCapability works without ALS, and interleaved concurrent contexts maintain isolation. * fix: Add worker thread guards to capability ALS usage Detect when hasCapability or capabilityContextMiddleware is called from a worker thread (where ALS context does not propagate from the parent). hasCapability logs a warn-once per factory instance; the middleware logs an error since mounting Express middleware in a worker is likely a misconfiguration. Both continue to function correctly — the guard is observability, not a hard block. * fix: Include tenantId in ALS principal cache key for tenant isolation The principal cache key was user.id:user.role, which would reuse cached principals across tenants for the same user within a request. When getUserPrincipals gains tenant-scoped group resolution, principals from tenant-a would incorrectly serve tenant-b checks. Changed to user.id:user.role:user.tenantId to prevent cross-tenant cache hits. Adds integration test proving separate principal lookups per tenantId. * test: Remove redundant mocked capabilities.spec.js The JS wrapper test (7 tests, all mocked) is a strict subset of capabilities.integration.spec.ts (28 tests, real MongoDB). Every scenario it covered — hasCapability true/false, tenantId passthrough, requireCapability 403/500, error handling — is tested with higher fidelity in the integration suite. * test: Replace mocked canDeleteAccount tests with real MongoDB integration Remove hasCapability mock — tests now exercise the full capability chain against real MongoDB (getUserPrincipals, hasCapabilityForPrincipals, SystemGrant collection). Only mocks remaining are logger and cache. Adds new coverage: admin role without grant is blocked, user-level grant bypasses deletion restriction, null user handling. * test: Add comprehensive tests for ACL entry management and user group methods Introduces new tests for `deleteAclEntries`, `bulkWriteAclEntries`, and `findPublicResourceIds` in `aclEntry.spec.ts`, ensuring proper functionality for deleting and bulk managing ACL entries. Additionally, enhances `userGroup.spec.ts` with tests for finding groups by ID and name pattern, including external ID matching and source filtering. These changes improve coverage and validate the integrity of ACL and user group operations against real MongoDB interactions. * refactor: Update capability checks and logging for better clarity and error handling Replaced `MANAGE_USERS` with `ACCESS_ADMIN` in the `canDeleteAccount` middleware and related tests to align with updated permission structure. Enhanced logging in various middleware functions to use `logger.warn` for capability check failures, providing clearer error messages. Additionally, refactored capability checks in the `patchPromptGroup` and `validateAuthor` functions to improve readability and maintainability. This commit also includes adjustments to the `systemGrant` methods to implement retry logic for transient failures during capability seeding, ensuring robustness in the face of database errors. * refactor: Enhance logging and retry logic in seedSystemGrants method Updated the logging format in the seedSystemGrants method to include error messages for better clarity. Improved the retry mechanism by explicitly mocking multiple failures in tests, ensuring robust error handling during transient database issues. Additionally, refined imports in the systemGrant schema for better type management. * refactor: Consolidate imports in canDeleteAccount middleware Merged logger and SystemCapabilities imports from the data-schemas module into a single line for improved readability and maintainability of the code. This change streamlines the import statements in the canDeleteAccount middleware. * test: Enhance systemGrant tests for error handling and capability validation Added tests to the systemGrant methods to handle various error scenarios, including E11000 race conditions, invalid ObjectId strings for USER and GROUP principals, and invalid capability strings. These enhancements improve the robustness of the capability granting and revoking logic, ensuring proper error propagation and validation of inputs. * fix: Wrap hasCapability calls in deny-by-default try-catch at remaining sites canAccessResource, files.js, and roles.js all had hasCapability inside outer try-catch blocks that returned 500 on DB failure instead of falling through to the regular ACL check. This contradicts the deny-by-default pattern used everywhere else. Also removes raw error.message from the roles.js 500 response to prevent internal host/connection info leaking to clients. * fix: Normalize user ID in canDeleteAccount before passing to hasCapability requireCapability normalizes req.user.id via _id?.toString() fallback, but canDeleteAccount passed raw req.user directly. If req.user.id is absent (some auth layers only populate _id), getUserPrincipals received undefined, silently returning empty principals and blocking the bypass. * fix: Harden systemGrant schema and type safety - Reject empty string tenantId in schema validator (was only blocking null; empty string silently orphaned documents) - Fix reverseImplications to use BaseSystemCapability[] instead of string[], preserving the narrow discriminated type - Document READ_ASSISTANTS as reserved/unenforced * test: Use fake timers for seedSystemGrants retry tests and add tenantId validation - Switch retry tests to jest.useFakeTimers() to eliminate 3+ seconds of real setTimeout delays per test run - Add regression test for empty-string tenantId rejection * docs: Add TODO(#12091) comments for tenant-scoped capability gaps In multi-tenant mode, platform-level grants (no tenantId) won't match tenant-scoped queries, breaking admin access. getUserPrincipals also returns cross-tenant group memberships. Both need fixes in #12091.
2026-03-07 13:56:32 -05:00
logger.error('[GET /roles/:roleName] Error:', error);
return res.status(500).send({ message: 'Failed to retrieve role' });
🗨️ feat: Prompts (#3131) * 🗨️ feat: Prompts (#7) * WIP: MERGE prompts/frontend (#1) * added schema for prompt and promptgroup, added model methods for prompts, added routes for prompts * * updated promptGroup Schema * updated model methods for prompts (get, add, delete) * slight fixes in prompt routes * * Created Files Management components * Created Vector Stores components * Added file management route in the routes folder * Completed UI for Files list, Compeleted UI for vector stores list, Completed UI for upload file modal, Completed UI for preview file, Completed UI for preview vector store * Fixed style and UI fixes for file dashboard, file list and vector stores list * added responsiveness classes for vector store page * fixed responsiveness of file page, dashboard page, and main page * fixed styling and responsiveness issues on dashboard page, file list page and vector store page * added queries and mutations for prompts and promptGroups, added relevant endpoints in data-provider, added relevant components prompts, added and updated relevant APIs * added types on mutation queries data service, updated prompt attributes * feature: Prompts and prompt groups management, added relevant APIs, added types for data service/queries/mutations, added relevant mutation and queries * chore: typing clarifications * added drop down on prompts mgmt dashboard * Fixes: fixed version switching issue on tags update or labels update, added cross button on create prompt group, fixed list updation on prompt group renaiming, added CSV upload button * Feature: Added oneliner and category attributes in prompt group, added schema for categories, added schema methods and route for categories * chore: typing and lint issues * chore: more type and linter fixes * chore: linting * chore: prompt controller and backend typing example; MOVE TO CONTROLLER DIRECTORY * chore: more type fixes * style: prompt name changes * chore: more type changes, and stateful prompt name change without flickering * fix: Return result of savePrompt in patchPrompt API endpoint * fix: navigation prompt queries; refactor: name 'prompt-groups' to just 'groups' * refactor: fetch prompt groups rewrite * refactor(prompts): query/mutation statefulness * refactor: remove `isActive` field * refactor: remove labels, consolidate logic * style: width, layout shift * refactor: improve hover toggle behavior and styling * refactor: add useParams hook to PromptListItem for dynamic rendering and add timeout ref for blur timeout * chore: hide upload button * refactor: import Button component from correct location in PromptSidePanel * style: prompt editor styling * style: fix more layout shifts * style: container scroll * refactor: Rename CreatePrompt component to CreatePromptForm * refactor: use react-hook-form * refactor: Add Prompts components and routes to Dashboard * style: skeletons for loading * fix: optimize makePromptProduction * refactor: consolidate variables * feat: create prompt form validation * refactor: Consolidate variables and update mutation hooks * style: minor touchups * chore: Update lucide-react npm dependency to version 0.394.0 and npm audit fix * refactor: add a new icon for the Prompts heading. * style: Update PromptsView heading to use h1 instead of h2 and other minor margin issues * chore: wording * refactor: Update PromptsView heading to use h1 instead of h2, consolidate variables, and add new icons * refactor: Prompts Button for Mobile * feature: added category field in prompt group, added relevant API and static data on BE to support FE UI for category in prompt group * chore: template for prompt cards --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * WIP: Prompts/frontend Continued (#2) * chore: loading style, remove unused component * feat: Add CategorySelector component for prompt group category selection * feat: add categories to create prompt * feat: prompt versions styling * feat: optimistic updates for prompt production state * refactor: optimize form state and show if prompt field is dirty with cross icon, also other styling changes * chore: remove unused code and localizations * fix: light mode styling * WIP: SidePanel Prompts * refactor: move to groups directory * refactor: rename GroupsSidePanel to GroupSidePanel and update imports * style: ListCard * refactor: isProduction changes * refactor: infinite query with productionPrompt * refactor: optimize snippets and prompts, and styling * refactor: Update getSnippet function to accept a length parameter * chore: localizations * feat: prompts navigation to chat and vice versa * fix: create prompt * feat: remember last selected category for creating prompts * fix(promptGroups): fix pagination and add usePromptGroupsNav hook * Prompts/frontend 3 (#3) * fix: stateful issues with prompt groups * style: improved layout * refactor: improve variable naming in Eng.ts * refactor: theme selector styling improvements * added prompt cards on chat new page, with dark mode, added API to fetch random prompts, added types for useQuery Slightly improved usePromptGroupNav logic to fetch updated result for pageSize, updated prompt cards view with darkmode and responsiveness fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page * WIP: Prompts/frontend (#4) * fix: optimize and fix paginated query * fix: remove unique constraint on names * refactor: button links and styling * style: menu border light mode * feat: Add Auto-Send Switch component for prompts groups * refactor(ChatView): use form context for submission text * chore: clear convo state on navigation to dashboard routes * chore: save prompt edit name on tab, remove console log * feat: basic prompt submission * refactor: move Auto-Send Switch * style(ListCard): border styling * feat: Add function to detect variables in text * feat: Add OriginalDialog component to UI library * chore(ui): Update SelectDropDown options list class to use text-xs size * refactor: submitMessage hook now includes submitPrompt, make compatible to document query selector * WIP: Variable Dialog * feat: variable submission working for both auto-send and non-autosend * feat: dashboard breadcrumbs and prompts/chat navigation * refactor: dashboard breadcrumb and dashboard link to chat navigation * refactor: Update VariableDialog and VariableForm styles * Prompts: Admin features (#5) * fix: link issue * fix: usePromptGroupsNav add missing dep. * style: dashbreadcrumb and sidepanel text color * temp fix: remove refetch on pageNumber change * fix: handle multiple variable replacement * WIP: create project schema and add project groups to fetch * feat: Add functionality to add prompt group IDs to a project * feat: Add caching for startup config in config route * chore: remove prompt landing * style: Update Skeleton component with additional background styling * chore: styling and types * WIP: SharePrompt first draft * feat(SharePrompt): form validation * feat: shared global indicators * refactor: prompt details * refactor: change NoPromptGroup directory * feat: preview prompt * feat: remove/add global prompts, add rbac-related enums * refactor: manage prompts location * WIP: first draft admin settings for prompts * feat: SystemRoles enum * refactor: update PromptDetails component styling * style: ellipsis custom class for showing more preview text * WIP: initial role schema and initialization * style: improved margins for single unordered lists * fix: use custom chat form context to prevent re-renders from FormProvider * feat: Role mutations for Prompt Permissions * feat: fetch user role * feat: update AdminSettings form default values from user role values * refactor: rename PromptPermissions to Permissions for general definitions * feat: initial role checks * feat: Add optional `bodyProps` parameter to generateCheckAccess middleware * refactor: UI access checks * Prompts: delete (#6) * Fixed delete prompt version API, fixed types and logic for prompt version deletion, updated prompt delete mutation logic * chore: Update return type of deletePrompt function in Prompt.js --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * chore: Update package-lock.json version to 0.7.4-rc1 and fast-xml-parser to 4.4.0 * feat: toast for saving admin settings, add timer no-access navigation * feat: always make prod * feat: Add localization to category labels in CategorySelector component * feat: Update category label localization in CategorySelector component * fix: Enable making prompt production in Prompt API --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * feat: Add helper fn for dark mode detection in ThemeProvider * style: surface-primary definition * fix(useHasAccess): utilize user.role and not just USER role * fix: empty category and role fetch * refactort: increase max height to options list and use label if no localization is found * fix: update CategorySelector to handle empty category value and improve localization * refactor: move prompts to own store/reactquery modules, add in filter WIP * refactor: Rename AutoSendSwitch to AutoSendPrompt * style: theming commit * style: fix slight coloring issue for convos in dark mode * style: better composition for prompts side panel * style: remove gray-750 and make it gray-850 * chore: adjust theming * feat: filter all prompt groups and properly remove prompts from projects * refactor: optimize delete prompt groups further * chore: localization * feat: Add uniqueProperty filtering to normalizeData function * WIP: filter prompts * chore: Update FilterPrompts component to include User icon in FilterItem * feat(FilterPrompts): set categories * feat: more system filters and show selected category icon * style: always make prod, flips switch to avoid mis-clicks * style: ui/ux loading/no prompts * chore: style FilterPrompts ChatView * fix: handle missing role edge case * style: special variables * feat: special variables * refactor: improve replaceSpecialVars function in prompts.ts * feat: simple/advanced editor modes * chore: bump versions * feat: localizations and hide production button on simple mode * fix: error connecting layout shift * fix: prompts CRUD for admins * fix: secure single group fetch * style: sidepanel styling * style(PromptName): bring edit button closer to name * style: mobile prompts header * style: mobile prompts header continued * style: align send prompts switch right * feat: description * Update special variables description in Eng.ts * feat: update/create/preview oneliner * fix: allow empty oneliner update * style: loading improvement and always make selected prompt Production if simple mode * fix: production index set and remove unused props * fix(ci): mock initializeRoles * fix: address #3128 * fix: address #3128 * feat: add deletion confirmation dialog * fix: mobile UI issues * style: prompt library UI update * style: focus, logcal tab order * style: Refactor SelectDropDown component to improve code readability and maintainability * chore: bump data-provider * chore: fix labels * refactor: confirm delete prompt version --------- Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
2024-06-20 20:24:32 -04:00
}
});
/**
* PUT /api/roles/:roleName/prompts
* Update prompt permissions for a specific role
*/
📜 feat: Implement System Grants for Capability-Based Authorization (#11896) * feat: Implement System Grants for Role-Based Capabilities - Added a new `systemGrant` model and associated methods to manage role-based capabilities within the application. - Introduced middleware functions `hasCapability` and `requireCapability` to check user permissions based on their roles. - Updated the database seeding process to include system grants for the ADMIN role, ensuring all necessary capabilities are assigned on startup. - Enhanced type definitions and schemas to support the new system grant functionality, improving overall type safety and clarity in the codebase. * test: Add unit tests for capabilities middleware and system grant methods - Introduced comprehensive unit tests for the capabilities middleware, including `hasCapability` and `requireCapability`, ensuring proper permission checks based on user roles. - Added tests for the `SystemGrant` methods, verifying the seeding of system grants, capability granting, and revocation processes. - Enhanced test coverage for edge cases, including idempotency of grant operations and handling of unexpected errors in middleware. - Utilized mocks for database interactions to isolate tests and improve reliability. * refactor: Transition to Capability-Based Access Control - Replaced role-based access checks with capability-based checks across various middleware and routes, enhancing permission management. - Introduced `hasCapability` and `requireCapability` functions to streamline capability verification for user actions. - Updated relevant routes and middleware to utilize the new capability system, ensuring consistent permission enforcement. - Enhanced type definitions and added tests for the new capability functions, improving overall code reliability and maintainability. * test: Enhance capability-based access tests for ADMIN role - Updated tests to reflect the new capability-based access control, specifically for the ADMIN role. - Modified test descriptions to clarify that users with the MANAGE_AGENTS capability can bypass permission checks. - Seeded capabilities for the ADMIN role in multiple test files to ensure consistent permission checks across different routes and middleware. - Improved overall test coverage for capability verification, ensuring robust permission management. * test: Update capability tests for MCP server access - Renamed test to reflect the correct capability for bypassing permission checks, changing from MANAGE_AGENTS to MANAGE_MCP_SERVERS. - Updated seeding of capabilities for the ADMIN role to align with the new capability structure. - Ensured consistency in capability definitions across tests and middleware for improved permission management. * feat: Add hasConfigCapability for enhanced config access control - Introduced `hasConfigCapability` function to check user permissions for managing or reading specific config sections. - Updated middleware to export the new capability function, ensuring consistent access control across the application. - Enhanced unit tests to cover various scenarios for the new capability, improving overall test coverage and reliability. * fix: Update tenantId filter in createSystemGrantMethods - Added a condition to set tenantId filter to { $exists: false } when tenantId is null, ensuring proper handling of cases where tenantId is not provided. - This change improves the robustness of the system grant methods by explicitly managing the absence of tenantId in the filter logic. * fix: account deletion capability check - Updated the `canDeleteAccount` middleware to ensure that the `hasManageUsers` capability check only occurs if a user is present, preventing potential errors when the user object is undefined. - This change improves the robustness of the account deletion logic by ensuring proper handling of user permissions. * refactor: Optimize seeding of system grants for ADMIN role - Replaced sequential capability granting with parallel execution using Promise.all in the seedSystemGrants function. - This change improves performance and efficiency during the initialization of system grants, ensuring all capabilities are granted concurrently. * refactor: Simplify systemGrantSchema index definition - Removed the sparse option from the unique index on principalType, principalId, capability, and tenantId in the systemGrantSchema. - This change streamlines the index definition, potentially improving query performance and clarity in the schema design. * refactor: Reorganize role capability check in roles route - Moved the capability check for reading roles to occur after parsing the roleName, improving code clarity and structure. - This change ensures that the authorization logic is consistently applied before fetching role details, enhancing overall permission management. * refactor: Remove unused ISystemGrant interface from systemCapabilities.ts - Deleted the ISystemGrant interface as it was no longer needed, streamlining the code and improving clarity. - This change helps reduce clutter in the file and focuses on relevant capabilities for the system. * refactor: Migrate SystemCapabilities to data-schemas - Replaced imports of SystemCapabilities from 'librechat-data-provider' with imports from '@librechat/data-schemas' across multiple files. - This change centralizes the management of system capabilities, improving code organization and maintainability. * refactor: Update account deletion middleware and capability checks - Modified the `canDeleteAccount` middleware to ensure that the account deletion permission is only granted to users with the `MANAGE_USERS` capability, improving security and clarity in permission management. - Enhanced error logging for unauthorized account deletion attempts, providing better insights into permission issues. - Updated the `capabilities.ts` file to ensure consistent handling of user authentication checks, improving robustness in capability verification. - Refined type definitions in `systemGrant.ts` and `systemGrantMethods.ts` to utilize the `PrincipalType` enum, enhancing type safety and code clarity. * refactor: Extract principal ID normalization into a separate function - Introduced `normalizePrincipalId` function to streamline the normalization of principal IDs based on their type, enhancing code clarity and reusability. - Updated references in `createSystemGrantMethods` to utilize the new normalization function, improving maintainability and reducing code duplication. * test: Add unit tests for principalId normalization in systemGrant - Introduced tests for the `grantCapability`, `revokeCapability`, and `getCapabilitiesForPrincipal` methods to verify correct handling of principalId normalization between string and ObjectId formats. - Enhanced the `capabilities.ts` middleware to utilize the `PrincipalType` enum for improved type safety. - Added a new utility function `normalizePrincipalId` to streamline principal ID normalization logic, ensuring consistent behavior across the application. * feat: Introduce capability implications and enhance system grant methods - Added `CapabilityImplications` to define relationships between broader and implied capabilities, allowing for more intuitive permission checks. - Updated `createSystemGrantMethods` to expand capability queries to include implied capabilities, improving authorization logic. - Enhanced `systemGrantSchema` to include an `expiresAt` field for future TTL enforcement of grants, and added validation to ensure `tenantId` is not set to null. - Documented authorization requirements for prompt group and prompt deletion methods to clarify access control expectations. * test: Add unit tests for canDeleteAccount middleware - Introduced unit tests for the `canDeleteAccount` middleware to verify account deletion permissions based on user roles and capabilities. - Covered scenarios for both allowed and blocked account deletions, including checks for ADMIN users with the `MANAGE_USERS` capability and handling of undefined user cases. - Enhanced test structure to ensure clarity and maintainability of permission checks in the middleware. * fix: Add principalType enum validation to SystemGrant schema Without enum validation, any string value was accepted for principalType and silently stored. Invalid documents would never match capability queries, creating phantom grants impossible to diagnose without raw DB inspection. All other ACL models in the codebase validate this field. * fix: Replace seedSystemGrants Promise.all with bulkWrite for concurrency safety When two server instances start simultaneously (K8s rolling deploy, PM2 cluster), both call seedSystemGrants. With Promise.all + findOneAndUpdate upsert, both instances may attempt to insert the same documents, causing E11000 duplicate key errors that crash server startup. bulkWrite with ordered:false handles concurrent upserts gracefully and reduces 17 individual round trips to a single network call. The returned documents (previously discarded) are no longer fetched. * perf: Add AsyncLocalStorage per-request cache for capability checks Every hasCapability call previously required 2 DB round trips (getUserPrincipals + SystemGrant.exists) — replacing what were O(1) string comparisons. Routes like patchPromptGroup triggered this twice, and hasConfigCapability's fallback path resolved principals twice. This adds a per-request AsyncLocalStorage cache that: - Caches resolved principals (same for all checks within one request) - Caches capability check results (same user+cap = same answer) - Automatically scoped to request lifetime (no stale grants) - Falls through to DB when no store exists (background jobs, tests) - Requires no signature changes to hasCapability The capabilityContextMiddleware is registered at the app level before all routes, initializing a fresh store per request. * fix: Add error handling for inline hasCapability calls canDeleteAccount, fetchAssistants, and validateAuthor all call hasCapability without try-catch. These were previously O(1) string comparisons that could never throw. Now they hit the database and can fail on connection timeout or transient errors. Wrap each call in try-catch, defaulting to deny (false) on error. This ensures a DB hiccup returns a clean 403 instead of an unhandled 500 with a stack trace. * test: Add canDeleteAccount DB-error resilience test Tests that hasCapability rejection (e.g., DB timeout) results in a clean 403 rather than an unhandled exception. Validates the error handling added in the previous commit. * refactor: Use barrel import for hasCapability in validateAuthor Import from ~/server/middleware barrel instead of directly from ~/server/middleware/roles/capabilities for consistency with other non-middleware consumers. Files within the middleware barrel itself must continue using direct imports to avoid circular requires. * refactor: Remove misleading pre('save') hook from SystemGrant schema The pre('save') hook normalized principalId for USER/GROUP principals, but the primary write path (grantCapability) uses findOneAndUpdate — which does not trigger save hooks. The normalization was already handled explicitly in grantCapability itself. The hook created a false impression of schema-level enforcement that only covered save()/create() paths. Replace with a comment documenting that all writes must go through grantCapability. * feat: Add READ_ASSISTANTS capability to complete manage/read pair Every other managed resource had a paired READ_X / MANAGE_X capability except assistants. This adds READ_ASSISTANTS and registers the MANAGE_ASSISTANTS → READ_ASSISTANTS implication in CapabilityImplications, enabling future read-only assistant visibility grants. * chore: Reorder systemGrant methods for clarity Moved hasCapabilityForPrincipals to a more logical position in the returned object of createSystemGrantMethods, improving code readability. This change also maintains the inclusion of seedSystemGrants in the export, ensuring all necessary methods are available. * fix: Wrap seedSystemGrants in try-catch to avoid blocking startup Seeding capabilities is idempotent and will succeed on the next restart. A transient DB error during seeding should not prevent the server from starting — log the error and continue. * refactor: Improve capability check efficiency and add audit logging Move hasCapability calls after cheap early-exits in validateAuthor and fetchAssistants so the DB check only runs when its result matters. Add logger.debug on every capability bypass grant across all 7 call sites for auditability, and log errors in catch blocks instead of silently swallowing them. * test: Add integration tests for AsyncLocalStorage capability caching Exercises the full vertical — ALS context, generateCapabilityCheck, real getUserPrincipals, real hasCapabilityForPrincipals, real MongoDB via MongoMemoryServer. Covers per-request caching, cross-context isolation, concurrent request isolation, negative caching, capability implications, tenant scoping, group-based grants, and requireCapability middleware. * test: Add systemGrant data-layer and ALS edge-case integration tests systemGrant.spec.ts (51 tests): Full integration tests for all systemGrant methods against real MongoDB — grant/revoke lifecycle, principalId normalization (string→ObjectId for USER/GROUP, string for ROLE), capability implications (both directions), tenant scoping, schema validation (null tenantId, invalid enum, required fields, unique compound index). capabilities.integration.spec.ts (27 tests): Adds ALS edge cases — missing context degrades gracefully with no caching (background jobs, child processes), nested middleware creates independent inner context, optional-chaining safety when store is undefined, mid-request grant changes are invisible due to result caching, requireCapability works without ALS, and interleaved concurrent contexts maintain isolation. * fix: Add worker thread guards to capability ALS usage Detect when hasCapability or capabilityContextMiddleware is called from a worker thread (where ALS context does not propagate from the parent). hasCapability logs a warn-once per factory instance; the middleware logs an error since mounting Express middleware in a worker is likely a misconfiguration. Both continue to function correctly — the guard is observability, not a hard block. * fix: Include tenantId in ALS principal cache key for tenant isolation The principal cache key was user.id:user.role, which would reuse cached principals across tenants for the same user within a request. When getUserPrincipals gains tenant-scoped group resolution, principals from tenant-a would incorrectly serve tenant-b checks. Changed to user.id:user.role:user.tenantId to prevent cross-tenant cache hits. Adds integration test proving separate principal lookups per tenantId. * test: Remove redundant mocked capabilities.spec.js The JS wrapper test (7 tests, all mocked) is a strict subset of capabilities.integration.spec.ts (28 tests, real MongoDB). Every scenario it covered — hasCapability true/false, tenantId passthrough, requireCapability 403/500, error handling — is tested with higher fidelity in the integration suite. * test: Replace mocked canDeleteAccount tests with real MongoDB integration Remove hasCapability mock — tests now exercise the full capability chain against real MongoDB (getUserPrincipals, hasCapabilityForPrincipals, SystemGrant collection). Only mocks remaining are logger and cache. Adds new coverage: admin role without grant is blocked, user-level grant bypasses deletion restriction, null user handling. * test: Add comprehensive tests for ACL entry management and user group methods Introduces new tests for `deleteAclEntries`, `bulkWriteAclEntries`, and `findPublicResourceIds` in `aclEntry.spec.ts`, ensuring proper functionality for deleting and bulk managing ACL entries. Additionally, enhances `userGroup.spec.ts` with tests for finding groups by ID and name pattern, including external ID matching and source filtering. These changes improve coverage and validate the integrity of ACL and user group operations against real MongoDB interactions. * refactor: Update capability checks and logging for better clarity and error handling Replaced `MANAGE_USERS` with `ACCESS_ADMIN` in the `canDeleteAccount` middleware and related tests to align with updated permission structure. Enhanced logging in various middleware functions to use `logger.warn` for capability check failures, providing clearer error messages. Additionally, refactored capability checks in the `patchPromptGroup` and `validateAuthor` functions to improve readability and maintainability. This commit also includes adjustments to the `systemGrant` methods to implement retry logic for transient failures during capability seeding, ensuring robustness in the face of database errors. * refactor: Enhance logging and retry logic in seedSystemGrants method Updated the logging format in the seedSystemGrants method to include error messages for better clarity. Improved the retry mechanism by explicitly mocking multiple failures in tests, ensuring robust error handling during transient database issues. Additionally, refined imports in the systemGrant schema for better type management. * refactor: Consolidate imports in canDeleteAccount middleware Merged logger and SystemCapabilities imports from the data-schemas module into a single line for improved readability and maintainability of the code. This change streamlines the import statements in the canDeleteAccount middleware. * test: Enhance systemGrant tests for error handling and capability validation Added tests to the systemGrant methods to handle various error scenarios, including E11000 race conditions, invalid ObjectId strings for USER and GROUP principals, and invalid capability strings. These enhancements improve the robustness of the capability granting and revoking logic, ensuring proper error propagation and validation of inputs. * fix: Wrap hasCapability calls in deny-by-default try-catch at remaining sites canAccessResource, files.js, and roles.js all had hasCapability inside outer try-catch blocks that returned 500 on DB failure instead of falling through to the regular ACL check. This contradicts the deny-by-default pattern used everywhere else. Also removes raw error.message from the roles.js 500 response to prevent internal host/connection info leaking to clients. * fix: Normalize user ID in canDeleteAccount before passing to hasCapability requireCapability normalizes req.user.id via _id?.toString() fallback, but canDeleteAccount passed raw req.user directly. If req.user.id is absent (some auth layers only populate _id), getUserPrincipals received undefined, silently returning empty principals and blocking the bypass. * fix: Harden systemGrant schema and type safety - Reject empty string tenantId in schema validator (was only blocking null; empty string silently orphaned documents) - Fix reverseImplications to use BaseSystemCapability[] instead of string[], preserving the narrow discriminated type - Document READ_ASSISTANTS as reserved/unenforced * test: Use fake timers for seedSystemGrants retry tests and add tenantId validation - Switch retry tests to jest.useFakeTimers() to eliminate 3+ seconds of real setTimeout delays per test run - Add regression test for empty-string tenantId rejection * docs: Add TODO(#12091) comments for tenant-scoped capability gaps In multi-tenant mode, platform-level grants (no tenantId) won't match tenant-scoped queries, breaking admin access. getUserPrincipals also returns cross-tenant group memberships. Both need fixes in #12091.
2026-03-07 13:56:32 -05:00
router.put('/:roleName/prompts', manageRoles, createPermissionUpdateHandler('prompts'));
🗨️ feat: Prompts (#3131) * 🗨️ feat: Prompts (#7) * WIP: MERGE prompts/frontend (#1) * added schema for prompt and promptgroup, added model methods for prompts, added routes for prompts * * updated promptGroup Schema * updated model methods for prompts (get, add, delete) * slight fixes in prompt routes * * Created Files Management components * Created Vector Stores components * Added file management route in the routes folder * Completed UI for Files list, Compeleted UI for vector stores list, Completed UI for upload file modal, Completed UI for preview file, Completed UI for preview vector store * Fixed style and UI fixes for file dashboard, file list and vector stores list * added responsiveness classes for vector store page * fixed responsiveness of file page, dashboard page, and main page * fixed styling and responsiveness issues on dashboard page, file list page and vector store page * added queries and mutations for prompts and promptGroups, added relevant endpoints in data-provider, added relevant components prompts, added and updated relevant APIs * added types on mutation queries data service, updated prompt attributes * feature: Prompts and prompt groups management, added relevant APIs, added types for data service/queries/mutations, added relevant mutation and queries * chore: typing clarifications * added drop down on prompts mgmt dashboard * Fixes: fixed version switching issue on tags update or labels update, added cross button on create prompt group, fixed list updation on prompt group renaiming, added CSV upload button * Feature: Added oneliner and category attributes in prompt group, added schema for categories, added schema methods and route for categories * chore: typing and lint issues * chore: more type and linter fixes * chore: linting * chore: prompt controller and backend typing example; MOVE TO CONTROLLER DIRECTORY * chore: more type fixes * style: prompt name changes * chore: more type changes, and stateful prompt name change without flickering * fix: Return result of savePrompt in patchPrompt API endpoint * fix: navigation prompt queries; refactor: name 'prompt-groups' to just 'groups' * refactor: fetch prompt groups rewrite * refactor(prompts): query/mutation statefulness * refactor: remove `isActive` field * refactor: remove labels, consolidate logic * style: width, layout shift * refactor: improve hover toggle behavior and styling * refactor: add useParams hook to PromptListItem for dynamic rendering and add timeout ref for blur timeout * chore: hide upload button * refactor: import Button component from correct location in PromptSidePanel * style: prompt editor styling * style: fix more layout shifts * style: container scroll * refactor: Rename CreatePrompt component to CreatePromptForm * refactor: use react-hook-form * refactor: Add Prompts components and routes to Dashboard * style: skeletons for loading * fix: optimize makePromptProduction * refactor: consolidate variables * feat: create prompt form validation * refactor: Consolidate variables and update mutation hooks * style: minor touchups * chore: Update lucide-react npm dependency to version 0.394.0 and npm audit fix * refactor: add a new icon for the Prompts heading. * style: Update PromptsView heading to use h1 instead of h2 and other minor margin issues * chore: wording * refactor: Update PromptsView heading to use h1 instead of h2, consolidate variables, and add new icons * refactor: Prompts Button for Mobile * feature: added category field in prompt group, added relevant API and static data on BE to support FE UI for category in prompt group * chore: template for prompt cards --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * WIP: Prompts/frontend Continued (#2) * chore: loading style, remove unused component * feat: Add CategorySelector component for prompt group category selection * feat: add categories to create prompt * feat: prompt versions styling * feat: optimistic updates for prompt production state * refactor: optimize form state and show if prompt field is dirty with cross icon, also other styling changes * chore: remove unused code and localizations * fix: light mode styling * WIP: SidePanel Prompts * refactor: move to groups directory * refactor: rename GroupsSidePanel to GroupSidePanel and update imports * style: ListCard * refactor: isProduction changes * refactor: infinite query with productionPrompt * refactor: optimize snippets and prompts, and styling * refactor: Update getSnippet function to accept a length parameter * chore: localizations * feat: prompts navigation to chat and vice versa * fix: create prompt * feat: remember last selected category for creating prompts * fix(promptGroups): fix pagination and add usePromptGroupsNav hook * Prompts/frontend 3 (#3) * fix: stateful issues with prompt groups * style: improved layout * refactor: improve variable naming in Eng.ts * refactor: theme selector styling improvements * added prompt cards on chat new page, with dark mode, added API to fetch random prompts, added types for useQuery Slightly improved usePromptGroupNav logic to fetch updated result for pageSize, updated prompt cards view with darkmode and responsiveness fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page * WIP: Prompts/frontend (#4) * fix: optimize and fix paginated query * fix: remove unique constraint on names * refactor: button links and styling * style: menu border light mode * feat: Add Auto-Send Switch component for prompts groups * refactor(ChatView): use form context for submission text * chore: clear convo state on navigation to dashboard routes * chore: save prompt edit name on tab, remove console log * feat: basic prompt submission * refactor: move Auto-Send Switch * style(ListCard): border styling * feat: Add function to detect variables in text * feat: Add OriginalDialog component to UI library * chore(ui): Update SelectDropDown options list class to use text-xs size * refactor: submitMessage hook now includes submitPrompt, make compatible to document query selector * WIP: Variable Dialog * feat: variable submission working for both auto-send and non-autosend * feat: dashboard breadcrumbs and prompts/chat navigation * refactor: dashboard breadcrumb and dashboard link to chat navigation * refactor: Update VariableDialog and VariableForm styles * Prompts: Admin features (#5) * fix: link issue * fix: usePromptGroupsNav add missing dep. * style: dashbreadcrumb and sidepanel text color * temp fix: remove refetch on pageNumber change * fix: handle multiple variable replacement * WIP: create project schema and add project groups to fetch * feat: Add functionality to add prompt group IDs to a project * feat: Add caching for startup config in config route * chore: remove prompt landing * style: Update Skeleton component with additional background styling * chore: styling and types * WIP: SharePrompt first draft * feat(SharePrompt): form validation * feat: shared global indicators * refactor: prompt details * refactor: change NoPromptGroup directory * feat: preview prompt * feat: remove/add global prompts, add rbac-related enums * refactor: manage prompts location * WIP: first draft admin settings for prompts * feat: SystemRoles enum * refactor: update PromptDetails component styling * style: ellipsis custom class for showing more preview text * WIP: initial role schema and initialization * style: improved margins for single unordered lists * fix: use custom chat form context to prevent re-renders from FormProvider * feat: Role mutations for Prompt Permissions * feat: fetch user role * feat: update AdminSettings form default values from user role values * refactor: rename PromptPermissions to Permissions for general definitions * feat: initial role checks * feat: Add optional `bodyProps` parameter to generateCheckAccess middleware * refactor: UI access checks * Prompts: delete (#6) * Fixed delete prompt version API, fixed types and logic for prompt version deletion, updated prompt delete mutation logic * chore: Update return type of deletePrompt function in Prompt.js --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * chore: Update package-lock.json version to 0.7.4-rc1 and fast-xml-parser to 4.4.0 * feat: toast for saving admin settings, add timer no-access navigation * feat: always make prod * feat: Add localization to category labels in CategorySelector component * feat: Update category label localization in CategorySelector component * fix: Enable making prompt production in Prompt API --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * feat: Add helper fn for dark mode detection in ThemeProvider * style: surface-primary definition * fix(useHasAccess): utilize user.role and not just USER role * fix: empty category and role fetch * refactort: increase max height to options list and use label if no localization is found * fix: update CategorySelector to handle empty category value and improve localization * refactor: move prompts to own store/reactquery modules, add in filter WIP * refactor: Rename AutoSendSwitch to AutoSendPrompt * style: theming commit * style: fix slight coloring issue for convos in dark mode * style: better composition for prompts side panel * style: remove gray-750 and make it gray-850 * chore: adjust theming * feat: filter all prompt groups and properly remove prompts from projects * refactor: optimize delete prompt groups further * chore: localization * feat: Add uniqueProperty filtering to normalizeData function * WIP: filter prompts * chore: Update FilterPrompts component to include User icon in FilterItem * feat(FilterPrompts): set categories * feat: more system filters and show selected category icon * style: always make prod, flips switch to avoid mis-clicks * style: ui/ux loading/no prompts * chore: style FilterPrompts ChatView * fix: handle missing role edge case * style: special variables * feat: special variables * refactor: improve replaceSpecialVars function in prompts.ts * feat: simple/advanced editor modes * chore: bump versions * feat: localizations and hide production button on simple mode * fix: error connecting layout shift * fix: prompts CRUD for admins * fix: secure single group fetch * style: sidepanel styling * style(PromptName): bring edit button closer to name * style: mobile prompts header * style: mobile prompts header continued * style: align send prompts switch right * feat: description * Update special variables description in Eng.ts * feat: update/create/preview oneliner * fix: allow empty oneliner update * style: loading improvement and always make selected prompt Production if simple mode * fix: production index set and remove unused props * fix(ci): mock initializeRoles * fix: address #3128 * fix: address #3128 * feat: add deletion confirmation dialog * fix: mobile UI issues * style: prompt library UI update * style: focus, logcal tab order * style: Refactor SelectDropDown component to improve code readability and maintainability * chore: bump data-provider * chore: fix labels * refactor: confirm delete prompt version --------- Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
2024-06-20 20:24:32 -04:00
🎉 feat: Code Interpreter API and Agents Release (#4860) * feat: Code Interpreter API & File Search Agent Uploads chore: add back code files wip: first pass, abstract key dialog refactor: influence checkbox on key changes refactor: update localization keys for 'execute code' to 'run code' wip: run code button refactor: add throwError parameter to loadAuthValues and getUserPluginAuthValue functions feat: first pass, API tool calling fix: handle missing toolId in callTool function and return 404 for non-existent tools feat: show code outputs fix: improve error handling in callTool function and log errors fix: handle potential null value for filepath in attachment destructuring fix: normalize language before rendering and prevent null return fix: add loading indicator in RunCode component while executing code feat: add support for conditional code execution in Markdown components feat: attachments refactor: remove bash fix: pass abort signal to graph/run refactor: debounce and rate limit tool call refactor: increase debounce delay for execute function feat: set code output attachments feat: image attachments refactor: apply message context refactor: pass `partIndex` feat: toolCall schema/model/methods feat: block indexing feat: get tool calls chore: imports chore: typing chore: condense type imports feat: get tool calls fix: block indexing chore: typing refactor: update tool calls mapping to support multiple results fix: add unique key to nav link for rendering wip: first pass, tool call results refactor: update query cache from successful tool call mutation style: improve result switcher styling chore: note on using \`.toObject()\` feat: add agent_id field to conversation schema chore: typing refactor: rename agentMap to agentsMap for consistency feat: Agent Name as chat input placeholder chore: bump agents 📦 chore: update @langchain dependencies to latest versions to match agents package 📦 chore: update @librechat/agents dependency to version 1.8.0 fix: Aborting agent stream removes sender; fix(bedrock): completion removes preset name label refactor: remove direct file parameter to use req.file, add `processAgentFileUpload` for image uploads feat: upload menu feat: prime message_file resources feat: implement conversation access validation in chat route refactor: remove file parameter from processFileUpload and use req.file instead feat: add savedMessageIds set to track saved message IDs in BaseClient, to prevent unnecessary double-write to db feat: prevent duplicate message saves by checking savedMessageIds in AgentController refactor: skip legacy RAG API handling for agents feat: add files field to convoSchema refactor: update request type annotations from Express.Request to ServerRequest in file processing functions feat: track conversation files fix: resendFiles, addPreviousAttachments handling feat: add ID validation for session_id and file_id in download route feat: entity_id for code file uploads/downloads fix: code file edge cases feat: delete related tool calls feat: add stream rate handling for LLM configuration feat: enhance system content with attached file information fix: improve error logging in resource priming function * WIP: PoC, sequential agents WIP: PoC Sequential Agents, first pass content data + bump agents package fix: package-lock WIP: PoC, o1 support, refactor bufferString feat: convertJsonSchemaToZod fix: form issues and schema defining erroneous model fix: max length issue on agent form instructions, limit conversation messages to sequential agents feat: add abort signal support to createRun function and AgentClient feat: PoC, hide prior sequential agent steps fix: update parameter naming from config to metadata in event handlers for clarity, add model to usage data refactor: use only last contentData, track model for usage data chore: bump agents package fix: content parts issue refactor: filter contentParts to include tool calls and relevant indices feat: show function calls refactor: filter context messages to exclude tool calls when no tools are available to the agent fix: ensure tool call content is not undefined in formatMessages feat: add agent_id field to conversationPreset schema feat: hide sequential agents feat: increase upload toast duration to 10 seconds * refactor: tool context handling & update Code API Key Dialog feat: toolContextMap chore: skipSpecs -> useSpecs ci: fix handleTools tests feat: API Key Dialog * feat: Agent Permissions Admin Controls feat: replace label with button for prompt permission toggle feat: update agent permissions feat: enable experimental agents and streamline capability configuration feat: implement access control for agents and enhance endpoint menu items feat: add welcome message for agent selection in localization feat: add agents permission to access control and update version to 0.7.57 * fix: update types in useAssistantListMap and useMentions hooks for better null handling * feat: mention agents * fix: agent tool resource race conditions when deleting agent tool resource files * feat: add error handling for code execution with user feedback * refactor: rename AdminControls to AdminSettings for clarity * style: add gap to button in AdminSettings for improved layout * refactor: separate agent query hooks and check access to enable fetching * fix: remove unused provider from agent initialization options, creates issue with custom endpoints * refactor: remove redundant/deprecated modelOptions from AgentClient processes * chore: update @librechat/agents to version 1.8.5 in package.json and package-lock.json * fix: minor styling issues + agent panel uniformity * fix: agent edge cases when set endpoint is no longer defined * refactor: remove unused cleanup function call from AppService * fix: update link in ApiKeyDialog to point to pricing page * fix: improve type handling and layout calculations in SidePanel component * fix: add missing localization string for agent selection in SidePanel * chore: form styling and localizations for upload filesearch/code interpreter * fix: model selection placeholder logic in AgentConfig component * style: agent capabilities * fix: add localization for provider selection and improve dropdown styling in ModelPanel * refactor: use gpt-4o-mini > gpt-3.5-turbo * fix: agents configuration for loadDefaultInterface and update related tests * feat: DALLE Agents support
2024-12-04 15:48:13 -05:00
/**
* PUT /api/roles/:roleName/agents
* Update agent permissions for a specific role
*/
📜 feat: Implement System Grants for Capability-Based Authorization (#11896) * feat: Implement System Grants for Role-Based Capabilities - Added a new `systemGrant` model and associated methods to manage role-based capabilities within the application. - Introduced middleware functions `hasCapability` and `requireCapability` to check user permissions based on their roles. - Updated the database seeding process to include system grants for the ADMIN role, ensuring all necessary capabilities are assigned on startup. - Enhanced type definitions and schemas to support the new system grant functionality, improving overall type safety and clarity in the codebase. * test: Add unit tests for capabilities middleware and system grant methods - Introduced comprehensive unit tests for the capabilities middleware, including `hasCapability` and `requireCapability`, ensuring proper permission checks based on user roles. - Added tests for the `SystemGrant` methods, verifying the seeding of system grants, capability granting, and revocation processes. - Enhanced test coverage for edge cases, including idempotency of grant operations and handling of unexpected errors in middleware. - Utilized mocks for database interactions to isolate tests and improve reliability. * refactor: Transition to Capability-Based Access Control - Replaced role-based access checks with capability-based checks across various middleware and routes, enhancing permission management. - Introduced `hasCapability` and `requireCapability` functions to streamline capability verification for user actions. - Updated relevant routes and middleware to utilize the new capability system, ensuring consistent permission enforcement. - Enhanced type definitions and added tests for the new capability functions, improving overall code reliability and maintainability. * test: Enhance capability-based access tests for ADMIN role - Updated tests to reflect the new capability-based access control, specifically for the ADMIN role. - Modified test descriptions to clarify that users with the MANAGE_AGENTS capability can bypass permission checks. - Seeded capabilities for the ADMIN role in multiple test files to ensure consistent permission checks across different routes and middleware. - Improved overall test coverage for capability verification, ensuring robust permission management. * test: Update capability tests for MCP server access - Renamed test to reflect the correct capability for bypassing permission checks, changing from MANAGE_AGENTS to MANAGE_MCP_SERVERS. - Updated seeding of capabilities for the ADMIN role to align with the new capability structure. - Ensured consistency in capability definitions across tests and middleware for improved permission management. * feat: Add hasConfigCapability for enhanced config access control - Introduced `hasConfigCapability` function to check user permissions for managing or reading specific config sections. - Updated middleware to export the new capability function, ensuring consistent access control across the application. - Enhanced unit tests to cover various scenarios for the new capability, improving overall test coverage and reliability. * fix: Update tenantId filter in createSystemGrantMethods - Added a condition to set tenantId filter to { $exists: false } when tenantId is null, ensuring proper handling of cases where tenantId is not provided. - This change improves the robustness of the system grant methods by explicitly managing the absence of tenantId in the filter logic. * fix: account deletion capability check - Updated the `canDeleteAccount` middleware to ensure that the `hasManageUsers` capability check only occurs if a user is present, preventing potential errors when the user object is undefined. - This change improves the robustness of the account deletion logic by ensuring proper handling of user permissions. * refactor: Optimize seeding of system grants for ADMIN role - Replaced sequential capability granting with parallel execution using Promise.all in the seedSystemGrants function. - This change improves performance and efficiency during the initialization of system grants, ensuring all capabilities are granted concurrently. * refactor: Simplify systemGrantSchema index definition - Removed the sparse option from the unique index on principalType, principalId, capability, and tenantId in the systemGrantSchema. - This change streamlines the index definition, potentially improving query performance and clarity in the schema design. * refactor: Reorganize role capability check in roles route - Moved the capability check for reading roles to occur after parsing the roleName, improving code clarity and structure. - This change ensures that the authorization logic is consistently applied before fetching role details, enhancing overall permission management. * refactor: Remove unused ISystemGrant interface from systemCapabilities.ts - Deleted the ISystemGrant interface as it was no longer needed, streamlining the code and improving clarity. - This change helps reduce clutter in the file and focuses on relevant capabilities for the system. * refactor: Migrate SystemCapabilities to data-schemas - Replaced imports of SystemCapabilities from 'librechat-data-provider' with imports from '@librechat/data-schemas' across multiple files. - This change centralizes the management of system capabilities, improving code organization and maintainability. * refactor: Update account deletion middleware and capability checks - Modified the `canDeleteAccount` middleware to ensure that the account deletion permission is only granted to users with the `MANAGE_USERS` capability, improving security and clarity in permission management. - Enhanced error logging for unauthorized account deletion attempts, providing better insights into permission issues. - Updated the `capabilities.ts` file to ensure consistent handling of user authentication checks, improving robustness in capability verification. - Refined type definitions in `systemGrant.ts` and `systemGrantMethods.ts` to utilize the `PrincipalType` enum, enhancing type safety and code clarity. * refactor: Extract principal ID normalization into a separate function - Introduced `normalizePrincipalId` function to streamline the normalization of principal IDs based on their type, enhancing code clarity and reusability. - Updated references in `createSystemGrantMethods` to utilize the new normalization function, improving maintainability and reducing code duplication. * test: Add unit tests for principalId normalization in systemGrant - Introduced tests for the `grantCapability`, `revokeCapability`, and `getCapabilitiesForPrincipal` methods to verify correct handling of principalId normalization between string and ObjectId formats. - Enhanced the `capabilities.ts` middleware to utilize the `PrincipalType` enum for improved type safety. - Added a new utility function `normalizePrincipalId` to streamline principal ID normalization logic, ensuring consistent behavior across the application. * feat: Introduce capability implications and enhance system grant methods - Added `CapabilityImplications` to define relationships between broader and implied capabilities, allowing for more intuitive permission checks. - Updated `createSystemGrantMethods` to expand capability queries to include implied capabilities, improving authorization logic. - Enhanced `systemGrantSchema` to include an `expiresAt` field for future TTL enforcement of grants, and added validation to ensure `tenantId` is not set to null. - Documented authorization requirements for prompt group and prompt deletion methods to clarify access control expectations. * test: Add unit tests for canDeleteAccount middleware - Introduced unit tests for the `canDeleteAccount` middleware to verify account deletion permissions based on user roles and capabilities. - Covered scenarios for both allowed and blocked account deletions, including checks for ADMIN users with the `MANAGE_USERS` capability and handling of undefined user cases. - Enhanced test structure to ensure clarity and maintainability of permission checks in the middleware. * fix: Add principalType enum validation to SystemGrant schema Without enum validation, any string value was accepted for principalType and silently stored. Invalid documents would never match capability queries, creating phantom grants impossible to diagnose without raw DB inspection. All other ACL models in the codebase validate this field. * fix: Replace seedSystemGrants Promise.all with bulkWrite for concurrency safety When two server instances start simultaneously (K8s rolling deploy, PM2 cluster), both call seedSystemGrants. With Promise.all + findOneAndUpdate upsert, both instances may attempt to insert the same documents, causing E11000 duplicate key errors that crash server startup. bulkWrite with ordered:false handles concurrent upserts gracefully and reduces 17 individual round trips to a single network call. The returned documents (previously discarded) are no longer fetched. * perf: Add AsyncLocalStorage per-request cache for capability checks Every hasCapability call previously required 2 DB round trips (getUserPrincipals + SystemGrant.exists) — replacing what were O(1) string comparisons. Routes like patchPromptGroup triggered this twice, and hasConfigCapability's fallback path resolved principals twice. This adds a per-request AsyncLocalStorage cache that: - Caches resolved principals (same for all checks within one request) - Caches capability check results (same user+cap = same answer) - Automatically scoped to request lifetime (no stale grants) - Falls through to DB when no store exists (background jobs, tests) - Requires no signature changes to hasCapability The capabilityContextMiddleware is registered at the app level before all routes, initializing a fresh store per request. * fix: Add error handling for inline hasCapability calls canDeleteAccount, fetchAssistants, and validateAuthor all call hasCapability without try-catch. These were previously O(1) string comparisons that could never throw. Now they hit the database and can fail on connection timeout or transient errors. Wrap each call in try-catch, defaulting to deny (false) on error. This ensures a DB hiccup returns a clean 403 instead of an unhandled 500 with a stack trace. * test: Add canDeleteAccount DB-error resilience test Tests that hasCapability rejection (e.g., DB timeout) results in a clean 403 rather than an unhandled exception. Validates the error handling added in the previous commit. * refactor: Use barrel import for hasCapability in validateAuthor Import from ~/server/middleware barrel instead of directly from ~/server/middleware/roles/capabilities for consistency with other non-middleware consumers. Files within the middleware barrel itself must continue using direct imports to avoid circular requires. * refactor: Remove misleading pre('save') hook from SystemGrant schema The pre('save') hook normalized principalId for USER/GROUP principals, but the primary write path (grantCapability) uses findOneAndUpdate — which does not trigger save hooks. The normalization was already handled explicitly in grantCapability itself. The hook created a false impression of schema-level enforcement that only covered save()/create() paths. Replace with a comment documenting that all writes must go through grantCapability. * feat: Add READ_ASSISTANTS capability to complete manage/read pair Every other managed resource had a paired READ_X / MANAGE_X capability except assistants. This adds READ_ASSISTANTS and registers the MANAGE_ASSISTANTS → READ_ASSISTANTS implication in CapabilityImplications, enabling future read-only assistant visibility grants. * chore: Reorder systemGrant methods for clarity Moved hasCapabilityForPrincipals to a more logical position in the returned object of createSystemGrantMethods, improving code readability. This change also maintains the inclusion of seedSystemGrants in the export, ensuring all necessary methods are available. * fix: Wrap seedSystemGrants in try-catch to avoid blocking startup Seeding capabilities is idempotent and will succeed on the next restart. A transient DB error during seeding should not prevent the server from starting — log the error and continue. * refactor: Improve capability check efficiency and add audit logging Move hasCapability calls after cheap early-exits in validateAuthor and fetchAssistants so the DB check only runs when its result matters. Add logger.debug on every capability bypass grant across all 7 call sites for auditability, and log errors in catch blocks instead of silently swallowing them. * test: Add integration tests for AsyncLocalStorage capability caching Exercises the full vertical — ALS context, generateCapabilityCheck, real getUserPrincipals, real hasCapabilityForPrincipals, real MongoDB via MongoMemoryServer. Covers per-request caching, cross-context isolation, concurrent request isolation, negative caching, capability implications, tenant scoping, group-based grants, and requireCapability middleware. * test: Add systemGrant data-layer and ALS edge-case integration tests systemGrant.spec.ts (51 tests): Full integration tests for all systemGrant methods against real MongoDB — grant/revoke lifecycle, principalId normalization (string→ObjectId for USER/GROUP, string for ROLE), capability implications (both directions), tenant scoping, schema validation (null tenantId, invalid enum, required fields, unique compound index). capabilities.integration.spec.ts (27 tests): Adds ALS edge cases — missing context degrades gracefully with no caching (background jobs, child processes), nested middleware creates independent inner context, optional-chaining safety when store is undefined, mid-request grant changes are invisible due to result caching, requireCapability works without ALS, and interleaved concurrent contexts maintain isolation. * fix: Add worker thread guards to capability ALS usage Detect when hasCapability or capabilityContextMiddleware is called from a worker thread (where ALS context does not propagate from the parent). hasCapability logs a warn-once per factory instance; the middleware logs an error since mounting Express middleware in a worker is likely a misconfiguration. Both continue to function correctly — the guard is observability, not a hard block. * fix: Include tenantId in ALS principal cache key for tenant isolation The principal cache key was user.id:user.role, which would reuse cached principals across tenants for the same user within a request. When getUserPrincipals gains tenant-scoped group resolution, principals from tenant-a would incorrectly serve tenant-b checks. Changed to user.id:user.role:user.tenantId to prevent cross-tenant cache hits. Adds integration test proving separate principal lookups per tenantId. * test: Remove redundant mocked capabilities.spec.js The JS wrapper test (7 tests, all mocked) is a strict subset of capabilities.integration.spec.ts (28 tests, real MongoDB). Every scenario it covered — hasCapability true/false, tenantId passthrough, requireCapability 403/500, error handling — is tested with higher fidelity in the integration suite. * test: Replace mocked canDeleteAccount tests with real MongoDB integration Remove hasCapability mock — tests now exercise the full capability chain against real MongoDB (getUserPrincipals, hasCapabilityForPrincipals, SystemGrant collection). Only mocks remaining are logger and cache. Adds new coverage: admin role without grant is blocked, user-level grant bypasses deletion restriction, null user handling. * test: Add comprehensive tests for ACL entry management and user group methods Introduces new tests for `deleteAclEntries`, `bulkWriteAclEntries`, and `findPublicResourceIds` in `aclEntry.spec.ts`, ensuring proper functionality for deleting and bulk managing ACL entries. Additionally, enhances `userGroup.spec.ts` with tests for finding groups by ID and name pattern, including external ID matching and source filtering. These changes improve coverage and validate the integrity of ACL and user group operations against real MongoDB interactions. * refactor: Update capability checks and logging for better clarity and error handling Replaced `MANAGE_USERS` with `ACCESS_ADMIN` in the `canDeleteAccount` middleware and related tests to align with updated permission structure. Enhanced logging in various middleware functions to use `logger.warn` for capability check failures, providing clearer error messages. Additionally, refactored capability checks in the `patchPromptGroup` and `validateAuthor` functions to improve readability and maintainability. This commit also includes adjustments to the `systemGrant` methods to implement retry logic for transient failures during capability seeding, ensuring robustness in the face of database errors. * refactor: Enhance logging and retry logic in seedSystemGrants method Updated the logging format in the seedSystemGrants method to include error messages for better clarity. Improved the retry mechanism by explicitly mocking multiple failures in tests, ensuring robust error handling during transient database issues. Additionally, refined imports in the systemGrant schema for better type management. * refactor: Consolidate imports in canDeleteAccount middleware Merged logger and SystemCapabilities imports from the data-schemas module into a single line for improved readability and maintainability of the code. This change streamlines the import statements in the canDeleteAccount middleware. * test: Enhance systemGrant tests for error handling and capability validation Added tests to the systemGrant methods to handle various error scenarios, including E11000 race conditions, invalid ObjectId strings for USER and GROUP principals, and invalid capability strings. These enhancements improve the robustness of the capability granting and revoking logic, ensuring proper error propagation and validation of inputs. * fix: Wrap hasCapability calls in deny-by-default try-catch at remaining sites canAccessResource, files.js, and roles.js all had hasCapability inside outer try-catch blocks that returned 500 on DB failure instead of falling through to the regular ACL check. This contradicts the deny-by-default pattern used everywhere else. Also removes raw error.message from the roles.js 500 response to prevent internal host/connection info leaking to clients. * fix: Normalize user ID in canDeleteAccount before passing to hasCapability requireCapability normalizes req.user.id via _id?.toString() fallback, but canDeleteAccount passed raw req.user directly. If req.user.id is absent (some auth layers only populate _id), getUserPrincipals received undefined, silently returning empty principals and blocking the bypass. * fix: Harden systemGrant schema and type safety - Reject empty string tenantId in schema validator (was only blocking null; empty string silently orphaned documents) - Fix reverseImplications to use BaseSystemCapability[] instead of string[], preserving the narrow discriminated type - Document READ_ASSISTANTS as reserved/unenforced * test: Use fake timers for seedSystemGrants retry tests and add tenantId validation - Switch retry tests to jest.useFakeTimers() to eliminate 3+ seconds of real setTimeout delays per test run - Add regression test for empty-string tenantId rejection * docs: Add TODO(#12091) comments for tenant-scoped capability gaps In multi-tenant mode, platform-level grants (no tenantId) won't match tenant-scoped queries, breaking admin access. getUserPrincipals also returns cross-tenant group memberships. Both need fixes in #12091.
2026-03-07 13:56:32 -05:00
router.put('/:roleName/agents', manageRoles, createPermissionUpdateHandler('agents'));
🎉 feat: Code Interpreter API and Agents Release (#4860) * feat: Code Interpreter API & File Search Agent Uploads chore: add back code files wip: first pass, abstract key dialog refactor: influence checkbox on key changes refactor: update localization keys for 'execute code' to 'run code' wip: run code button refactor: add throwError parameter to loadAuthValues and getUserPluginAuthValue functions feat: first pass, API tool calling fix: handle missing toolId in callTool function and return 404 for non-existent tools feat: show code outputs fix: improve error handling in callTool function and log errors fix: handle potential null value for filepath in attachment destructuring fix: normalize language before rendering and prevent null return fix: add loading indicator in RunCode component while executing code feat: add support for conditional code execution in Markdown components feat: attachments refactor: remove bash fix: pass abort signal to graph/run refactor: debounce and rate limit tool call refactor: increase debounce delay for execute function feat: set code output attachments feat: image attachments refactor: apply message context refactor: pass `partIndex` feat: toolCall schema/model/methods feat: block indexing feat: get tool calls chore: imports chore: typing chore: condense type imports feat: get tool calls fix: block indexing chore: typing refactor: update tool calls mapping to support multiple results fix: add unique key to nav link for rendering wip: first pass, tool call results refactor: update query cache from successful tool call mutation style: improve result switcher styling chore: note on using \`.toObject()\` feat: add agent_id field to conversation schema chore: typing refactor: rename agentMap to agentsMap for consistency feat: Agent Name as chat input placeholder chore: bump agents 📦 chore: update @langchain dependencies to latest versions to match agents package 📦 chore: update @librechat/agents dependency to version 1.8.0 fix: Aborting agent stream removes sender; fix(bedrock): completion removes preset name label refactor: remove direct file parameter to use req.file, add `processAgentFileUpload` for image uploads feat: upload menu feat: prime message_file resources feat: implement conversation access validation in chat route refactor: remove file parameter from processFileUpload and use req.file instead feat: add savedMessageIds set to track saved message IDs in BaseClient, to prevent unnecessary double-write to db feat: prevent duplicate message saves by checking savedMessageIds in AgentController refactor: skip legacy RAG API handling for agents feat: add files field to convoSchema refactor: update request type annotations from Express.Request to ServerRequest in file processing functions feat: track conversation files fix: resendFiles, addPreviousAttachments handling feat: add ID validation for session_id and file_id in download route feat: entity_id for code file uploads/downloads fix: code file edge cases feat: delete related tool calls feat: add stream rate handling for LLM configuration feat: enhance system content with attached file information fix: improve error logging in resource priming function * WIP: PoC, sequential agents WIP: PoC Sequential Agents, first pass content data + bump agents package fix: package-lock WIP: PoC, o1 support, refactor bufferString feat: convertJsonSchemaToZod fix: form issues and schema defining erroneous model fix: max length issue on agent form instructions, limit conversation messages to sequential agents feat: add abort signal support to createRun function and AgentClient feat: PoC, hide prior sequential agent steps fix: update parameter naming from config to metadata in event handlers for clarity, add model to usage data refactor: use only last contentData, track model for usage data chore: bump agents package fix: content parts issue refactor: filter contentParts to include tool calls and relevant indices feat: show function calls refactor: filter context messages to exclude tool calls when no tools are available to the agent fix: ensure tool call content is not undefined in formatMessages feat: add agent_id field to conversationPreset schema feat: hide sequential agents feat: increase upload toast duration to 10 seconds * refactor: tool context handling & update Code API Key Dialog feat: toolContextMap chore: skipSpecs -> useSpecs ci: fix handleTools tests feat: API Key Dialog * feat: Agent Permissions Admin Controls feat: replace label with button for prompt permission toggle feat: update agent permissions feat: enable experimental agents and streamline capability configuration feat: implement access control for agents and enhance endpoint menu items feat: add welcome message for agent selection in localization feat: add agents permission to access control and update version to 0.7.57 * fix: update types in useAssistantListMap and useMentions hooks for better null handling * feat: mention agents * fix: agent tool resource race conditions when deleting agent tool resource files * feat: add error handling for code execution with user feedback * refactor: rename AdminControls to AdminSettings for clarity * style: add gap to button in AdminSettings for improved layout * refactor: separate agent query hooks and check access to enable fetching * fix: remove unused provider from agent initialization options, creates issue with custom endpoints * refactor: remove redundant/deprecated modelOptions from AgentClient processes * chore: update @librechat/agents to version 1.8.5 in package.json and package-lock.json * fix: minor styling issues + agent panel uniformity * fix: agent edge cases when set endpoint is no longer defined * refactor: remove unused cleanup function call from AppService * fix: update link in ApiKeyDialog to point to pricing page * fix: improve type handling and layout calculations in SidePanel component * fix: add missing localization string for agent selection in SidePanel * chore: form styling and localizations for upload filesearch/code interpreter * fix: model selection placeholder logic in AgentConfig component * style: agent capabilities * fix: add localization for provider selection and improve dropdown styling in ModelPanel * refactor: use gpt-4o-mini > gpt-3.5-turbo * fix: agents configuration for loadDefaultInterface and update related tests * feat: DALLE Agents support
2024-12-04 15:48:13 -05:00
🧠 feat: User Memories for Conversational Context (#7760) * 🧠 feat: User Memories for Conversational Context chore: mcp typing, use `t` WIP: first pass, Memories UI - Added MemoryViewer component for displaying, editing, and deleting user memories. - Integrated data provider hooks for fetching, updating, and deleting memories. - Implemented pagination and loading states for better user experience. - Created unit tests for MemoryViewer to ensure functionality and interaction with data provider. - Updated translation files to include new UI strings related to memories. chore: move mcp-related files to own directory chore: rename librechat-mcp to librechat-api WIP: first pass, memory processing and data schemas chore: linting in fileSearch.js query description chore: rename librechat-api to @librechat/api across the project WIP: first pass, functional memory agent feat: add MemoryEditDialog and MemoryViewer components for managing user memories - Introduced MemoryEditDialog for editing memory entries with validation and toast notifications. - Updated MemoryViewer to support editing and deleting memories, including pagination and loading states. - Enhanced data provider to handle memory updates with optional original key for better management. - Added new localization strings for memory-related UI elements. feat: add memory permissions management - Implemented memory permissions in the backend, allowing roles to have specific permissions for using, creating, updating, and reading memories. - Added new API endpoints for updating memory permissions associated with roles. - Created a new AdminSettings component for managing memory permissions in the frontend. - Integrated memory permissions into the existing roles and permissions schemas. - Updated the interface to include memory settings and permissions. - Enhanced the MemoryViewer component to conditionally render admin settings based on user roles. - Added localization support for memory permissions in the translation files. feat: move AdminSettings component to a new position in MemoryViewer for better visibility refactor: clean up commented code in MemoryViewer component feat: enhance MemoryViewer with search functionality and improve MemoryEditDialog integration - Added a search input to filter memories in the MemoryViewer component. - Refactored MemoryEditDialog to accept children for better customization. - Updated MemoryViewer to utilize the new EditMemoryButton and DeleteMemoryButton components for editing and deleting memories. - Improved localization support by adding new strings for memory filtering and deletion confirmation. refactor: optimize memory filtering in MemoryViewer using match-sorter - Replaced manual filtering logic with match-sorter for improved search functionality. - Enhanced performance and readability of the filteredMemories computation. feat: enhance MemoryEditDialog with triggerRef and improve updateMemory mutation handling feat: implement access control for MemoryEditDialog and MemoryViewer components refactor: remove commented out code and create runMemory method refactor: rename role based files feat: implement access control for memory usage in AgentClient refactor: simplify checkVisionRequest method in AgentClient by removing commented-out code refactor: make `agents` dir in api package refactor: migrate Azure utilities to TypeScript and consolidate imports refactor: move sanitizeFilename function to a new file and update imports, add related tests refactor: update LLM configuration types and consolidate Azure options in the API package chore: linting chore: import order refactor: replace getLLMConfig with getOpenAIConfig and remove unused LLM configuration file chore: update winston-daily-rotate-file to version 5.0.0 and add object-hash dependency in package-lock.json refactor: move primeResources and optionalChainWithEmptyCheck functions to resources.ts and update imports refactor: move createRun function to a new run.ts file and update related imports fix: ensure safeAttachments is correctly typed as an array of TFile chore: add node-fetch dependency and refactor fetch-related functions into packages/api/utils, removing the old generators file refactor: enhance TEndpointOption type by using Pick to streamline endpoint fields and add new properties for model parameters and client options feat: implement initializeOpenAIOptions function and update OpenAI types for enhanced configuration handling fix: update types due to new TEndpointOption typing fix: ensure safe access to group parameters in initializeOpenAIOptions function fix: remove redundant API key validation comment in initializeOpenAIOptions function refactor: rename initializeOpenAIOptions to initializeOpenAI for consistency and update related documentation refactor: decouple req.body fields and tool loading from initializeAgentOptions chore: linting refactor: adjust column widths in MemoryViewer for improved layout refactor: simplify agent initialization by creating loadAgent function and removing unused code feat: add memory configuration loading and validation functions WIP: first pass, memory processing with config feat: implement memory callback and artifact handling feat: implement memory artifacts display and processing updates feat: add memory configuration options and schema validation for validKeys fix: update MemoryEditDialog and MemoryViewer to handle memory state and display improvements refactor: remove padding from BookmarkTable and MemoryViewer headers for consistent styling WIP: initial tokenLimit config and move Tokenizer to @librechat/api refactor: update mongoMeili plugin methods to use callback for better error handling feat: enhance memory management with token tracking and usage metrics - Added token counting for memory entries to enforce limits and provide usage statistics. - Updated memory retrieval and update routes to include total token usage and limit. - Enhanced MemoryEditDialog and MemoryViewer components to display memory usage and token information. - Refactored memory processing functions to handle token limits and provide feedback on memory capacity. feat: implement memory artifact handling in attachment handler - Enhanced useAttachmentHandler to process memory artifacts when receiving updates. - Introduced handleMemoryArtifact utility to manage memory updates and deletions. - Updated query client to reflect changes in memory state based on incoming data. refactor: restructure web search key extraction logic - Moved the logic for extracting API keys from the webSearchAuth configuration into a dedicated function, getWebSearchKeys. - Updated webSearchKeys to utilize the new function for improved clarity and maintainability. - Prevents build time errors feat: add personalization settings and memory preferences management - Introduced a new Personalization tab in settings to manage user memory preferences. - Implemented API endpoints and client-side logic for updating memory preferences. - Enhanced user interface components to reflect personalization options and memory usage. - Updated permissions to allow users to opt out of memory features. - Added localization support for new settings and messages related to personalization. style: personalization switch class feat: add PersonalizationIcon and align Side Panel UI feat: implement memory creation functionality - Added a new API endpoint for creating memory entries, including validation for key and value. - Introduced MemoryCreateDialog component for user interface to facilitate memory creation. - Integrated token limit checks to prevent exceeding user memory capacity. - Updated MemoryViewer to include a button for opening the memory creation dialog. - Enhanced localization support for new messages related to memory creation. feat: enhance message processing with configurable window size - Updated AgentClient to use a configurable message window size for processing messages. - Introduced messageWindowSize option in memory configuration schema with a default value of 5. - Improved logic for selecting messages to process based on the configured window size. chore: update librechat-data-provider version to 0.7.87 in package.json and package-lock.json chore: remove OpenAPIPlugin and its associated tests chore: remove MIGRATION_README.md as migration tasks are completed ci: fix backend tests chore: remove unused translation keys from localization file chore: remove problematic test file and unused var in AgentClient chore: remove unused import and import directly for JSDoc * feat: add api package build stage in Dockerfile for improved modularity * docs: reorder build steps in contributing guide for clarity
2025-06-07 18:52:22 -04:00
/**
* PUT /api/roles/:roleName/memories
* Update memory permissions for a specific role
*/
📜 feat: Implement System Grants for Capability-Based Authorization (#11896) * feat: Implement System Grants for Role-Based Capabilities - Added a new `systemGrant` model and associated methods to manage role-based capabilities within the application. - Introduced middleware functions `hasCapability` and `requireCapability` to check user permissions based on their roles. - Updated the database seeding process to include system grants for the ADMIN role, ensuring all necessary capabilities are assigned on startup. - Enhanced type definitions and schemas to support the new system grant functionality, improving overall type safety and clarity in the codebase. * test: Add unit tests for capabilities middleware and system grant methods - Introduced comprehensive unit tests for the capabilities middleware, including `hasCapability` and `requireCapability`, ensuring proper permission checks based on user roles. - Added tests for the `SystemGrant` methods, verifying the seeding of system grants, capability granting, and revocation processes. - Enhanced test coverage for edge cases, including idempotency of grant operations and handling of unexpected errors in middleware. - Utilized mocks for database interactions to isolate tests and improve reliability. * refactor: Transition to Capability-Based Access Control - Replaced role-based access checks with capability-based checks across various middleware and routes, enhancing permission management. - Introduced `hasCapability` and `requireCapability` functions to streamline capability verification for user actions. - Updated relevant routes and middleware to utilize the new capability system, ensuring consistent permission enforcement. - Enhanced type definitions and added tests for the new capability functions, improving overall code reliability and maintainability. * test: Enhance capability-based access tests for ADMIN role - Updated tests to reflect the new capability-based access control, specifically for the ADMIN role. - Modified test descriptions to clarify that users with the MANAGE_AGENTS capability can bypass permission checks. - Seeded capabilities for the ADMIN role in multiple test files to ensure consistent permission checks across different routes and middleware. - Improved overall test coverage for capability verification, ensuring robust permission management. * test: Update capability tests for MCP server access - Renamed test to reflect the correct capability for bypassing permission checks, changing from MANAGE_AGENTS to MANAGE_MCP_SERVERS. - Updated seeding of capabilities for the ADMIN role to align with the new capability structure. - Ensured consistency in capability definitions across tests and middleware for improved permission management. * feat: Add hasConfigCapability for enhanced config access control - Introduced `hasConfigCapability` function to check user permissions for managing or reading specific config sections. - Updated middleware to export the new capability function, ensuring consistent access control across the application. - Enhanced unit tests to cover various scenarios for the new capability, improving overall test coverage and reliability. * fix: Update tenantId filter in createSystemGrantMethods - Added a condition to set tenantId filter to { $exists: false } when tenantId is null, ensuring proper handling of cases where tenantId is not provided. - This change improves the robustness of the system grant methods by explicitly managing the absence of tenantId in the filter logic. * fix: account deletion capability check - Updated the `canDeleteAccount` middleware to ensure that the `hasManageUsers` capability check only occurs if a user is present, preventing potential errors when the user object is undefined. - This change improves the robustness of the account deletion logic by ensuring proper handling of user permissions. * refactor: Optimize seeding of system grants for ADMIN role - Replaced sequential capability granting with parallel execution using Promise.all in the seedSystemGrants function. - This change improves performance and efficiency during the initialization of system grants, ensuring all capabilities are granted concurrently. * refactor: Simplify systemGrantSchema index definition - Removed the sparse option from the unique index on principalType, principalId, capability, and tenantId in the systemGrantSchema. - This change streamlines the index definition, potentially improving query performance and clarity in the schema design. * refactor: Reorganize role capability check in roles route - Moved the capability check for reading roles to occur after parsing the roleName, improving code clarity and structure. - This change ensures that the authorization logic is consistently applied before fetching role details, enhancing overall permission management. * refactor: Remove unused ISystemGrant interface from systemCapabilities.ts - Deleted the ISystemGrant interface as it was no longer needed, streamlining the code and improving clarity. - This change helps reduce clutter in the file and focuses on relevant capabilities for the system. * refactor: Migrate SystemCapabilities to data-schemas - Replaced imports of SystemCapabilities from 'librechat-data-provider' with imports from '@librechat/data-schemas' across multiple files. - This change centralizes the management of system capabilities, improving code organization and maintainability. * refactor: Update account deletion middleware and capability checks - Modified the `canDeleteAccount` middleware to ensure that the account deletion permission is only granted to users with the `MANAGE_USERS` capability, improving security and clarity in permission management. - Enhanced error logging for unauthorized account deletion attempts, providing better insights into permission issues. - Updated the `capabilities.ts` file to ensure consistent handling of user authentication checks, improving robustness in capability verification. - Refined type definitions in `systemGrant.ts` and `systemGrantMethods.ts` to utilize the `PrincipalType` enum, enhancing type safety and code clarity. * refactor: Extract principal ID normalization into a separate function - Introduced `normalizePrincipalId` function to streamline the normalization of principal IDs based on their type, enhancing code clarity and reusability. - Updated references in `createSystemGrantMethods` to utilize the new normalization function, improving maintainability and reducing code duplication. * test: Add unit tests for principalId normalization in systemGrant - Introduced tests for the `grantCapability`, `revokeCapability`, and `getCapabilitiesForPrincipal` methods to verify correct handling of principalId normalization between string and ObjectId formats. - Enhanced the `capabilities.ts` middleware to utilize the `PrincipalType` enum for improved type safety. - Added a new utility function `normalizePrincipalId` to streamline principal ID normalization logic, ensuring consistent behavior across the application. * feat: Introduce capability implications and enhance system grant methods - Added `CapabilityImplications` to define relationships between broader and implied capabilities, allowing for more intuitive permission checks. - Updated `createSystemGrantMethods` to expand capability queries to include implied capabilities, improving authorization logic. - Enhanced `systemGrantSchema` to include an `expiresAt` field for future TTL enforcement of grants, and added validation to ensure `tenantId` is not set to null. - Documented authorization requirements for prompt group and prompt deletion methods to clarify access control expectations. * test: Add unit tests for canDeleteAccount middleware - Introduced unit tests for the `canDeleteAccount` middleware to verify account deletion permissions based on user roles and capabilities. - Covered scenarios for both allowed and blocked account deletions, including checks for ADMIN users with the `MANAGE_USERS` capability and handling of undefined user cases. - Enhanced test structure to ensure clarity and maintainability of permission checks in the middleware. * fix: Add principalType enum validation to SystemGrant schema Without enum validation, any string value was accepted for principalType and silently stored. Invalid documents would never match capability queries, creating phantom grants impossible to diagnose without raw DB inspection. All other ACL models in the codebase validate this field. * fix: Replace seedSystemGrants Promise.all with bulkWrite for concurrency safety When two server instances start simultaneously (K8s rolling deploy, PM2 cluster), both call seedSystemGrants. With Promise.all + findOneAndUpdate upsert, both instances may attempt to insert the same documents, causing E11000 duplicate key errors that crash server startup. bulkWrite with ordered:false handles concurrent upserts gracefully and reduces 17 individual round trips to a single network call. The returned documents (previously discarded) are no longer fetched. * perf: Add AsyncLocalStorage per-request cache for capability checks Every hasCapability call previously required 2 DB round trips (getUserPrincipals + SystemGrant.exists) — replacing what were O(1) string comparisons. Routes like patchPromptGroup triggered this twice, and hasConfigCapability's fallback path resolved principals twice. This adds a per-request AsyncLocalStorage cache that: - Caches resolved principals (same for all checks within one request) - Caches capability check results (same user+cap = same answer) - Automatically scoped to request lifetime (no stale grants) - Falls through to DB when no store exists (background jobs, tests) - Requires no signature changes to hasCapability The capabilityContextMiddleware is registered at the app level before all routes, initializing a fresh store per request. * fix: Add error handling for inline hasCapability calls canDeleteAccount, fetchAssistants, and validateAuthor all call hasCapability without try-catch. These were previously O(1) string comparisons that could never throw. Now they hit the database and can fail on connection timeout or transient errors. Wrap each call in try-catch, defaulting to deny (false) on error. This ensures a DB hiccup returns a clean 403 instead of an unhandled 500 with a stack trace. * test: Add canDeleteAccount DB-error resilience test Tests that hasCapability rejection (e.g., DB timeout) results in a clean 403 rather than an unhandled exception. Validates the error handling added in the previous commit. * refactor: Use barrel import for hasCapability in validateAuthor Import from ~/server/middleware barrel instead of directly from ~/server/middleware/roles/capabilities for consistency with other non-middleware consumers. Files within the middleware barrel itself must continue using direct imports to avoid circular requires. * refactor: Remove misleading pre('save') hook from SystemGrant schema The pre('save') hook normalized principalId for USER/GROUP principals, but the primary write path (grantCapability) uses findOneAndUpdate — which does not trigger save hooks. The normalization was already handled explicitly in grantCapability itself. The hook created a false impression of schema-level enforcement that only covered save()/create() paths. Replace with a comment documenting that all writes must go through grantCapability. * feat: Add READ_ASSISTANTS capability to complete manage/read pair Every other managed resource had a paired READ_X / MANAGE_X capability except assistants. This adds READ_ASSISTANTS and registers the MANAGE_ASSISTANTS → READ_ASSISTANTS implication in CapabilityImplications, enabling future read-only assistant visibility grants. * chore: Reorder systemGrant methods for clarity Moved hasCapabilityForPrincipals to a more logical position in the returned object of createSystemGrantMethods, improving code readability. This change also maintains the inclusion of seedSystemGrants in the export, ensuring all necessary methods are available. * fix: Wrap seedSystemGrants in try-catch to avoid blocking startup Seeding capabilities is idempotent and will succeed on the next restart. A transient DB error during seeding should not prevent the server from starting — log the error and continue. * refactor: Improve capability check efficiency and add audit logging Move hasCapability calls after cheap early-exits in validateAuthor and fetchAssistants so the DB check only runs when its result matters. Add logger.debug on every capability bypass grant across all 7 call sites for auditability, and log errors in catch blocks instead of silently swallowing them. * test: Add integration tests for AsyncLocalStorage capability caching Exercises the full vertical — ALS context, generateCapabilityCheck, real getUserPrincipals, real hasCapabilityForPrincipals, real MongoDB via MongoMemoryServer. Covers per-request caching, cross-context isolation, concurrent request isolation, negative caching, capability implications, tenant scoping, group-based grants, and requireCapability middleware. * test: Add systemGrant data-layer and ALS edge-case integration tests systemGrant.spec.ts (51 tests): Full integration tests for all systemGrant methods against real MongoDB — grant/revoke lifecycle, principalId normalization (string→ObjectId for USER/GROUP, string for ROLE), capability implications (both directions), tenant scoping, schema validation (null tenantId, invalid enum, required fields, unique compound index). capabilities.integration.spec.ts (27 tests): Adds ALS edge cases — missing context degrades gracefully with no caching (background jobs, child processes), nested middleware creates independent inner context, optional-chaining safety when store is undefined, mid-request grant changes are invisible due to result caching, requireCapability works without ALS, and interleaved concurrent contexts maintain isolation. * fix: Add worker thread guards to capability ALS usage Detect when hasCapability or capabilityContextMiddleware is called from a worker thread (where ALS context does not propagate from the parent). hasCapability logs a warn-once per factory instance; the middleware logs an error since mounting Express middleware in a worker is likely a misconfiguration. Both continue to function correctly — the guard is observability, not a hard block. * fix: Include tenantId in ALS principal cache key for tenant isolation The principal cache key was user.id:user.role, which would reuse cached principals across tenants for the same user within a request. When getUserPrincipals gains tenant-scoped group resolution, principals from tenant-a would incorrectly serve tenant-b checks. Changed to user.id:user.role:user.tenantId to prevent cross-tenant cache hits. Adds integration test proving separate principal lookups per tenantId. * test: Remove redundant mocked capabilities.spec.js The JS wrapper test (7 tests, all mocked) is a strict subset of capabilities.integration.spec.ts (28 tests, real MongoDB). Every scenario it covered — hasCapability true/false, tenantId passthrough, requireCapability 403/500, error handling — is tested with higher fidelity in the integration suite. * test: Replace mocked canDeleteAccount tests with real MongoDB integration Remove hasCapability mock — tests now exercise the full capability chain against real MongoDB (getUserPrincipals, hasCapabilityForPrincipals, SystemGrant collection). Only mocks remaining are logger and cache. Adds new coverage: admin role without grant is blocked, user-level grant bypasses deletion restriction, null user handling. * test: Add comprehensive tests for ACL entry management and user group methods Introduces new tests for `deleteAclEntries`, `bulkWriteAclEntries`, and `findPublicResourceIds` in `aclEntry.spec.ts`, ensuring proper functionality for deleting and bulk managing ACL entries. Additionally, enhances `userGroup.spec.ts` with tests for finding groups by ID and name pattern, including external ID matching and source filtering. These changes improve coverage and validate the integrity of ACL and user group operations against real MongoDB interactions. * refactor: Update capability checks and logging for better clarity and error handling Replaced `MANAGE_USERS` with `ACCESS_ADMIN` in the `canDeleteAccount` middleware and related tests to align with updated permission structure. Enhanced logging in various middleware functions to use `logger.warn` for capability check failures, providing clearer error messages. Additionally, refactored capability checks in the `patchPromptGroup` and `validateAuthor` functions to improve readability and maintainability. This commit also includes adjustments to the `systemGrant` methods to implement retry logic for transient failures during capability seeding, ensuring robustness in the face of database errors. * refactor: Enhance logging and retry logic in seedSystemGrants method Updated the logging format in the seedSystemGrants method to include error messages for better clarity. Improved the retry mechanism by explicitly mocking multiple failures in tests, ensuring robust error handling during transient database issues. Additionally, refined imports in the systemGrant schema for better type management. * refactor: Consolidate imports in canDeleteAccount middleware Merged logger and SystemCapabilities imports from the data-schemas module into a single line for improved readability and maintainability of the code. This change streamlines the import statements in the canDeleteAccount middleware. * test: Enhance systemGrant tests for error handling and capability validation Added tests to the systemGrant methods to handle various error scenarios, including E11000 race conditions, invalid ObjectId strings for USER and GROUP principals, and invalid capability strings. These enhancements improve the robustness of the capability granting and revoking logic, ensuring proper error propagation and validation of inputs. * fix: Wrap hasCapability calls in deny-by-default try-catch at remaining sites canAccessResource, files.js, and roles.js all had hasCapability inside outer try-catch blocks that returned 500 on DB failure instead of falling through to the regular ACL check. This contradicts the deny-by-default pattern used everywhere else. Also removes raw error.message from the roles.js 500 response to prevent internal host/connection info leaking to clients. * fix: Normalize user ID in canDeleteAccount before passing to hasCapability requireCapability normalizes req.user.id via _id?.toString() fallback, but canDeleteAccount passed raw req.user directly. If req.user.id is absent (some auth layers only populate _id), getUserPrincipals received undefined, silently returning empty principals and blocking the bypass. * fix: Harden systemGrant schema and type safety - Reject empty string tenantId in schema validator (was only blocking null; empty string silently orphaned documents) - Fix reverseImplications to use BaseSystemCapability[] instead of string[], preserving the narrow discriminated type - Document READ_ASSISTANTS as reserved/unenforced * test: Use fake timers for seedSystemGrants retry tests and add tenantId validation - Switch retry tests to jest.useFakeTimers() to eliminate 3+ seconds of real setTimeout delays per test run - Add regression test for empty-string tenantId rejection * docs: Add TODO(#12091) comments for tenant-scoped capability gaps In multi-tenant mode, platform-level grants (no tenantId) won't match tenant-scoped queries, breaking admin access. getUserPrincipals also returns cross-tenant group memberships. Both need fixes in #12091.
2026-03-07 13:56:32 -05:00
router.put('/:roleName/memories', manageRoles, createPermissionUpdateHandler('memories'));
🧠 feat: User Memories for Conversational Context (#7760) * 🧠 feat: User Memories for Conversational Context chore: mcp typing, use `t` WIP: first pass, Memories UI - Added MemoryViewer component for displaying, editing, and deleting user memories. - Integrated data provider hooks for fetching, updating, and deleting memories. - Implemented pagination and loading states for better user experience. - Created unit tests for MemoryViewer to ensure functionality and interaction with data provider. - Updated translation files to include new UI strings related to memories. chore: move mcp-related files to own directory chore: rename librechat-mcp to librechat-api WIP: first pass, memory processing and data schemas chore: linting in fileSearch.js query description chore: rename librechat-api to @librechat/api across the project WIP: first pass, functional memory agent feat: add MemoryEditDialog and MemoryViewer components for managing user memories - Introduced MemoryEditDialog for editing memory entries with validation and toast notifications. - Updated MemoryViewer to support editing and deleting memories, including pagination and loading states. - Enhanced data provider to handle memory updates with optional original key for better management. - Added new localization strings for memory-related UI elements. feat: add memory permissions management - Implemented memory permissions in the backend, allowing roles to have specific permissions for using, creating, updating, and reading memories. - Added new API endpoints for updating memory permissions associated with roles. - Created a new AdminSettings component for managing memory permissions in the frontend. - Integrated memory permissions into the existing roles and permissions schemas. - Updated the interface to include memory settings and permissions. - Enhanced the MemoryViewer component to conditionally render admin settings based on user roles. - Added localization support for memory permissions in the translation files. feat: move AdminSettings component to a new position in MemoryViewer for better visibility refactor: clean up commented code in MemoryViewer component feat: enhance MemoryViewer with search functionality and improve MemoryEditDialog integration - Added a search input to filter memories in the MemoryViewer component. - Refactored MemoryEditDialog to accept children for better customization. - Updated MemoryViewer to utilize the new EditMemoryButton and DeleteMemoryButton components for editing and deleting memories. - Improved localization support by adding new strings for memory filtering and deletion confirmation. refactor: optimize memory filtering in MemoryViewer using match-sorter - Replaced manual filtering logic with match-sorter for improved search functionality. - Enhanced performance and readability of the filteredMemories computation. feat: enhance MemoryEditDialog with triggerRef and improve updateMemory mutation handling feat: implement access control for MemoryEditDialog and MemoryViewer components refactor: remove commented out code and create runMemory method refactor: rename role based files feat: implement access control for memory usage in AgentClient refactor: simplify checkVisionRequest method in AgentClient by removing commented-out code refactor: make `agents` dir in api package refactor: migrate Azure utilities to TypeScript and consolidate imports refactor: move sanitizeFilename function to a new file and update imports, add related tests refactor: update LLM configuration types and consolidate Azure options in the API package chore: linting chore: import order refactor: replace getLLMConfig with getOpenAIConfig and remove unused LLM configuration file chore: update winston-daily-rotate-file to version 5.0.0 and add object-hash dependency in package-lock.json refactor: move primeResources and optionalChainWithEmptyCheck functions to resources.ts and update imports refactor: move createRun function to a new run.ts file and update related imports fix: ensure safeAttachments is correctly typed as an array of TFile chore: add node-fetch dependency and refactor fetch-related functions into packages/api/utils, removing the old generators file refactor: enhance TEndpointOption type by using Pick to streamline endpoint fields and add new properties for model parameters and client options feat: implement initializeOpenAIOptions function and update OpenAI types for enhanced configuration handling fix: update types due to new TEndpointOption typing fix: ensure safe access to group parameters in initializeOpenAIOptions function fix: remove redundant API key validation comment in initializeOpenAIOptions function refactor: rename initializeOpenAIOptions to initializeOpenAI for consistency and update related documentation refactor: decouple req.body fields and tool loading from initializeAgentOptions chore: linting refactor: adjust column widths in MemoryViewer for improved layout refactor: simplify agent initialization by creating loadAgent function and removing unused code feat: add memory configuration loading and validation functions WIP: first pass, memory processing with config feat: implement memory callback and artifact handling feat: implement memory artifacts display and processing updates feat: add memory configuration options and schema validation for validKeys fix: update MemoryEditDialog and MemoryViewer to handle memory state and display improvements refactor: remove padding from BookmarkTable and MemoryViewer headers for consistent styling WIP: initial tokenLimit config and move Tokenizer to @librechat/api refactor: update mongoMeili plugin methods to use callback for better error handling feat: enhance memory management with token tracking and usage metrics - Added token counting for memory entries to enforce limits and provide usage statistics. - Updated memory retrieval and update routes to include total token usage and limit. - Enhanced MemoryEditDialog and MemoryViewer components to display memory usage and token information. - Refactored memory processing functions to handle token limits and provide feedback on memory capacity. feat: implement memory artifact handling in attachment handler - Enhanced useAttachmentHandler to process memory artifacts when receiving updates. - Introduced handleMemoryArtifact utility to manage memory updates and deletions. - Updated query client to reflect changes in memory state based on incoming data. refactor: restructure web search key extraction logic - Moved the logic for extracting API keys from the webSearchAuth configuration into a dedicated function, getWebSearchKeys. - Updated webSearchKeys to utilize the new function for improved clarity and maintainability. - Prevents build time errors feat: add personalization settings and memory preferences management - Introduced a new Personalization tab in settings to manage user memory preferences. - Implemented API endpoints and client-side logic for updating memory preferences. - Enhanced user interface components to reflect personalization options and memory usage. - Updated permissions to allow users to opt out of memory features. - Added localization support for new settings and messages related to personalization. style: personalization switch class feat: add PersonalizationIcon and align Side Panel UI feat: implement memory creation functionality - Added a new API endpoint for creating memory entries, including validation for key and value. - Introduced MemoryCreateDialog component for user interface to facilitate memory creation. - Integrated token limit checks to prevent exceeding user memory capacity. - Updated MemoryViewer to include a button for opening the memory creation dialog. - Enhanced localization support for new messages related to memory creation. feat: enhance message processing with configurable window size - Updated AgentClient to use a configurable message window size for processing messages. - Introduced messageWindowSize option in memory configuration schema with a default value of 5. - Improved logic for selecting messages to process based on the configured window size. chore: update librechat-data-provider version to 0.7.87 in package.json and package-lock.json chore: remove OpenAPIPlugin and its associated tests chore: remove MIGRATION_README.md as migration tasks are completed ci: fix backend tests chore: remove unused translation keys from localization file chore: remove problematic test file and unused var in AgentClient chore: remove unused import and import directly for JSDoc * feat: add api package build stage in Dockerfile for improved modularity * docs: reorder build steps in contributing guide for clarity
2025-06-07 18:52:22 -04:00
/**
* PUT /api/roles/:roleName/people-picker
* Update people picker permissions for a specific role
*/
📜 feat: Implement System Grants for Capability-Based Authorization (#11896) * feat: Implement System Grants for Role-Based Capabilities - Added a new `systemGrant` model and associated methods to manage role-based capabilities within the application. - Introduced middleware functions `hasCapability` and `requireCapability` to check user permissions based on their roles. - Updated the database seeding process to include system grants for the ADMIN role, ensuring all necessary capabilities are assigned on startup. - Enhanced type definitions and schemas to support the new system grant functionality, improving overall type safety and clarity in the codebase. * test: Add unit tests for capabilities middleware and system grant methods - Introduced comprehensive unit tests for the capabilities middleware, including `hasCapability` and `requireCapability`, ensuring proper permission checks based on user roles. - Added tests for the `SystemGrant` methods, verifying the seeding of system grants, capability granting, and revocation processes. - Enhanced test coverage for edge cases, including idempotency of grant operations and handling of unexpected errors in middleware. - Utilized mocks for database interactions to isolate tests and improve reliability. * refactor: Transition to Capability-Based Access Control - Replaced role-based access checks with capability-based checks across various middleware and routes, enhancing permission management. - Introduced `hasCapability` and `requireCapability` functions to streamline capability verification for user actions. - Updated relevant routes and middleware to utilize the new capability system, ensuring consistent permission enforcement. - Enhanced type definitions and added tests for the new capability functions, improving overall code reliability and maintainability. * test: Enhance capability-based access tests for ADMIN role - Updated tests to reflect the new capability-based access control, specifically for the ADMIN role. - Modified test descriptions to clarify that users with the MANAGE_AGENTS capability can bypass permission checks. - Seeded capabilities for the ADMIN role in multiple test files to ensure consistent permission checks across different routes and middleware. - Improved overall test coverage for capability verification, ensuring robust permission management. * test: Update capability tests for MCP server access - Renamed test to reflect the correct capability for bypassing permission checks, changing from MANAGE_AGENTS to MANAGE_MCP_SERVERS. - Updated seeding of capabilities for the ADMIN role to align with the new capability structure. - Ensured consistency in capability definitions across tests and middleware for improved permission management. * feat: Add hasConfigCapability for enhanced config access control - Introduced `hasConfigCapability` function to check user permissions for managing or reading specific config sections. - Updated middleware to export the new capability function, ensuring consistent access control across the application. - Enhanced unit tests to cover various scenarios for the new capability, improving overall test coverage and reliability. * fix: Update tenantId filter in createSystemGrantMethods - Added a condition to set tenantId filter to { $exists: false } when tenantId is null, ensuring proper handling of cases where tenantId is not provided. - This change improves the robustness of the system grant methods by explicitly managing the absence of tenantId in the filter logic. * fix: account deletion capability check - Updated the `canDeleteAccount` middleware to ensure that the `hasManageUsers` capability check only occurs if a user is present, preventing potential errors when the user object is undefined. - This change improves the robustness of the account deletion logic by ensuring proper handling of user permissions. * refactor: Optimize seeding of system grants for ADMIN role - Replaced sequential capability granting with parallel execution using Promise.all in the seedSystemGrants function. - This change improves performance and efficiency during the initialization of system grants, ensuring all capabilities are granted concurrently. * refactor: Simplify systemGrantSchema index definition - Removed the sparse option from the unique index on principalType, principalId, capability, and tenantId in the systemGrantSchema. - This change streamlines the index definition, potentially improving query performance and clarity in the schema design. * refactor: Reorganize role capability check in roles route - Moved the capability check for reading roles to occur after parsing the roleName, improving code clarity and structure. - This change ensures that the authorization logic is consistently applied before fetching role details, enhancing overall permission management. * refactor: Remove unused ISystemGrant interface from systemCapabilities.ts - Deleted the ISystemGrant interface as it was no longer needed, streamlining the code and improving clarity. - This change helps reduce clutter in the file and focuses on relevant capabilities for the system. * refactor: Migrate SystemCapabilities to data-schemas - Replaced imports of SystemCapabilities from 'librechat-data-provider' with imports from '@librechat/data-schemas' across multiple files. - This change centralizes the management of system capabilities, improving code organization and maintainability. * refactor: Update account deletion middleware and capability checks - Modified the `canDeleteAccount` middleware to ensure that the account deletion permission is only granted to users with the `MANAGE_USERS` capability, improving security and clarity in permission management. - Enhanced error logging for unauthorized account deletion attempts, providing better insights into permission issues. - Updated the `capabilities.ts` file to ensure consistent handling of user authentication checks, improving robustness in capability verification. - Refined type definitions in `systemGrant.ts` and `systemGrantMethods.ts` to utilize the `PrincipalType` enum, enhancing type safety and code clarity. * refactor: Extract principal ID normalization into a separate function - Introduced `normalizePrincipalId` function to streamline the normalization of principal IDs based on their type, enhancing code clarity and reusability. - Updated references in `createSystemGrantMethods` to utilize the new normalization function, improving maintainability and reducing code duplication. * test: Add unit tests for principalId normalization in systemGrant - Introduced tests for the `grantCapability`, `revokeCapability`, and `getCapabilitiesForPrincipal` methods to verify correct handling of principalId normalization between string and ObjectId formats. - Enhanced the `capabilities.ts` middleware to utilize the `PrincipalType` enum for improved type safety. - Added a new utility function `normalizePrincipalId` to streamline principal ID normalization logic, ensuring consistent behavior across the application. * feat: Introduce capability implications and enhance system grant methods - Added `CapabilityImplications` to define relationships between broader and implied capabilities, allowing for more intuitive permission checks. - Updated `createSystemGrantMethods` to expand capability queries to include implied capabilities, improving authorization logic. - Enhanced `systemGrantSchema` to include an `expiresAt` field for future TTL enforcement of grants, and added validation to ensure `tenantId` is not set to null. - Documented authorization requirements for prompt group and prompt deletion methods to clarify access control expectations. * test: Add unit tests for canDeleteAccount middleware - Introduced unit tests for the `canDeleteAccount` middleware to verify account deletion permissions based on user roles and capabilities. - Covered scenarios for both allowed and blocked account deletions, including checks for ADMIN users with the `MANAGE_USERS` capability and handling of undefined user cases. - Enhanced test structure to ensure clarity and maintainability of permission checks in the middleware. * fix: Add principalType enum validation to SystemGrant schema Without enum validation, any string value was accepted for principalType and silently stored. Invalid documents would never match capability queries, creating phantom grants impossible to diagnose without raw DB inspection. All other ACL models in the codebase validate this field. * fix: Replace seedSystemGrants Promise.all with bulkWrite for concurrency safety When two server instances start simultaneously (K8s rolling deploy, PM2 cluster), both call seedSystemGrants. With Promise.all + findOneAndUpdate upsert, both instances may attempt to insert the same documents, causing E11000 duplicate key errors that crash server startup. bulkWrite with ordered:false handles concurrent upserts gracefully and reduces 17 individual round trips to a single network call. The returned documents (previously discarded) are no longer fetched. * perf: Add AsyncLocalStorage per-request cache for capability checks Every hasCapability call previously required 2 DB round trips (getUserPrincipals + SystemGrant.exists) — replacing what were O(1) string comparisons. Routes like patchPromptGroup triggered this twice, and hasConfigCapability's fallback path resolved principals twice. This adds a per-request AsyncLocalStorage cache that: - Caches resolved principals (same for all checks within one request) - Caches capability check results (same user+cap = same answer) - Automatically scoped to request lifetime (no stale grants) - Falls through to DB when no store exists (background jobs, tests) - Requires no signature changes to hasCapability The capabilityContextMiddleware is registered at the app level before all routes, initializing a fresh store per request. * fix: Add error handling for inline hasCapability calls canDeleteAccount, fetchAssistants, and validateAuthor all call hasCapability without try-catch. These were previously O(1) string comparisons that could never throw. Now they hit the database and can fail on connection timeout or transient errors. Wrap each call in try-catch, defaulting to deny (false) on error. This ensures a DB hiccup returns a clean 403 instead of an unhandled 500 with a stack trace. * test: Add canDeleteAccount DB-error resilience test Tests that hasCapability rejection (e.g., DB timeout) results in a clean 403 rather than an unhandled exception. Validates the error handling added in the previous commit. * refactor: Use barrel import for hasCapability in validateAuthor Import from ~/server/middleware barrel instead of directly from ~/server/middleware/roles/capabilities for consistency with other non-middleware consumers. Files within the middleware barrel itself must continue using direct imports to avoid circular requires. * refactor: Remove misleading pre('save') hook from SystemGrant schema The pre('save') hook normalized principalId for USER/GROUP principals, but the primary write path (grantCapability) uses findOneAndUpdate — which does not trigger save hooks. The normalization was already handled explicitly in grantCapability itself. The hook created a false impression of schema-level enforcement that only covered save()/create() paths. Replace with a comment documenting that all writes must go through grantCapability. * feat: Add READ_ASSISTANTS capability to complete manage/read pair Every other managed resource had a paired READ_X / MANAGE_X capability except assistants. This adds READ_ASSISTANTS and registers the MANAGE_ASSISTANTS → READ_ASSISTANTS implication in CapabilityImplications, enabling future read-only assistant visibility grants. * chore: Reorder systemGrant methods for clarity Moved hasCapabilityForPrincipals to a more logical position in the returned object of createSystemGrantMethods, improving code readability. This change also maintains the inclusion of seedSystemGrants in the export, ensuring all necessary methods are available. * fix: Wrap seedSystemGrants in try-catch to avoid blocking startup Seeding capabilities is idempotent and will succeed on the next restart. A transient DB error during seeding should not prevent the server from starting — log the error and continue. * refactor: Improve capability check efficiency and add audit logging Move hasCapability calls after cheap early-exits in validateAuthor and fetchAssistants so the DB check only runs when its result matters. Add logger.debug on every capability bypass grant across all 7 call sites for auditability, and log errors in catch blocks instead of silently swallowing them. * test: Add integration tests for AsyncLocalStorage capability caching Exercises the full vertical — ALS context, generateCapabilityCheck, real getUserPrincipals, real hasCapabilityForPrincipals, real MongoDB via MongoMemoryServer. Covers per-request caching, cross-context isolation, concurrent request isolation, negative caching, capability implications, tenant scoping, group-based grants, and requireCapability middleware. * test: Add systemGrant data-layer and ALS edge-case integration tests systemGrant.spec.ts (51 tests): Full integration tests for all systemGrant methods against real MongoDB — grant/revoke lifecycle, principalId normalization (string→ObjectId for USER/GROUP, string for ROLE), capability implications (both directions), tenant scoping, schema validation (null tenantId, invalid enum, required fields, unique compound index). capabilities.integration.spec.ts (27 tests): Adds ALS edge cases — missing context degrades gracefully with no caching (background jobs, child processes), nested middleware creates independent inner context, optional-chaining safety when store is undefined, mid-request grant changes are invisible due to result caching, requireCapability works without ALS, and interleaved concurrent contexts maintain isolation. * fix: Add worker thread guards to capability ALS usage Detect when hasCapability or capabilityContextMiddleware is called from a worker thread (where ALS context does not propagate from the parent). hasCapability logs a warn-once per factory instance; the middleware logs an error since mounting Express middleware in a worker is likely a misconfiguration. Both continue to function correctly — the guard is observability, not a hard block. * fix: Include tenantId in ALS principal cache key for tenant isolation The principal cache key was user.id:user.role, which would reuse cached principals across tenants for the same user within a request. When getUserPrincipals gains tenant-scoped group resolution, principals from tenant-a would incorrectly serve tenant-b checks. Changed to user.id:user.role:user.tenantId to prevent cross-tenant cache hits. Adds integration test proving separate principal lookups per tenantId. * test: Remove redundant mocked capabilities.spec.js The JS wrapper test (7 tests, all mocked) is a strict subset of capabilities.integration.spec.ts (28 tests, real MongoDB). Every scenario it covered — hasCapability true/false, tenantId passthrough, requireCapability 403/500, error handling — is tested with higher fidelity in the integration suite. * test: Replace mocked canDeleteAccount tests with real MongoDB integration Remove hasCapability mock — tests now exercise the full capability chain against real MongoDB (getUserPrincipals, hasCapabilityForPrincipals, SystemGrant collection). Only mocks remaining are logger and cache. Adds new coverage: admin role without grant is blocked, user-level grant bypasses deletion restriction, null user handling. * test: Add comprehensive tests for ACL entry management and user group methods Introduces new tests for `deleteAclEntries`, `bulkWriteAclEntries`, and `findPublicResourceIds` in `aclEntry.spec.ts`, ensuring proper functionality for deleting and bulk managing ACL entries. Additionally, enhances `userGroup.spec.ts` with tests for finding groups by ID and name pattern, including external ID matching and source filtering. These changes improve coverage and validate the integrity of ACL and user group operations against real MongoDB interactions. * refactor: Update capability checks and logging for better clarity and error handling Replaced `MANAGE_USERS` with `ACCESS_ADMIN` in the `canDeleteAccount` middleware and related tests to align with updated permission structure. Enhanced logging in various middleware functions to use `logger.warn` for capability check failures, providing clearer error messages. Additionally, refactored capability checks in the `patchPromptGroup` and `validateAuthor` functions to improve readability and maintainability. This commit also includes adjustments to the `systemGrant` methods to implement retry logic for transient failures during capability seeding, ensuring robustness in the face of database errors. * refactor: Enhance logging and retry logic in seedSystemGrants method Updated the logging format in the seedSystemGrants method to include error messages for better clarity. Improved the retry mechanism by explicitly mocking multiple failures in tests, ensuring robust error handling during transient database issues. Additionally, refined imports in the systemGrant schema for better type management. * refactor: Consolidate imports in canDeleteAccount middleware Merged logger and SystemCapabilities imports from the data-schemas module into a single line for improved readability and maintainability of the code. This change streamlines the import statements in the canDeleteAccount middleware. * test: Enhance systemGrant tests for error handling and capability validation Added tests to the systemGrant methods to handle various error scenarios, including E11000 race conditions, invalid ObjectId strings for USER and GROUP principals, and invalid capability strings. These enhancements improve the robustness of the capability granting and revoking logic, ensuring proper error propagation and validation of inputs. * fix: Wrap hasCapability calls in deny-by-default try-catch at remaining sites canAccessResource, files.js, and roles.js all had hasCapability inside outer try-catch blocks that returned 500 on DB failure instead of falling through to the regular ACL check. This contradicts the deny-by-default pattern used everywhere else. Also removes raw error.message from the roles.js 500 response to prevent internal host/connection info leaking to clients. * fix: Normalize user ID in canDeleteAccount before passing to hasCapability requireCapability normalizes req.user.id via _id?.toString() fallback, but canDeleteAccount passed raw req.user directly. If req.user.id is absent (some auth layers only populate _id), getUserPrincipals received undefined, silently returning empty principals and blocking the bypass. * fix: Harden systemGrant schema and type safety - Reject empty string tenantId in schema validator (was only blocking null; empty string silently orphaned documents) - Fix reverseImplications to use BaseSystemCapability[] instead of string[], preserving the narrow discriminated type - Document READ_ASSISTANTS as reserved/unenforced * test: Use fake timers for seedSystemGrants retry tests and add tenantId validation - Switch retry tests to jest.useFakeTimers() to eliminate 3+ seconds of real setTimeout delays per test run - Add regression test for empty-string tenantId rejection * docs: Add TODO(#12091) comments for tenant-scoped capability gaps In multi-tenant mode, platform-level grants (no tenantId) won't match tenant-scoped queries, breaking admin access. getUserPrincipals also returns cross-tenant group memberships. Both need fixes in #12091.
2026-03-07 13:56:32 -05:00
router.put('/:roleName/people-picker', manageRoles, createPermissionUpdateHandler('people-picker'));
🧠 feat: User Memories for Conversational Context (#7760) * 🧠 feat: User Memories for Conversational Context chore: mcp typing, use `t` WIP: first pass, Memories UI - Added MemoryViewer component for displaying, editing, and deleting user memories. - Integrated data provider hooks for fetching, updating, and deleting memories. - Implemented pagination and loading states for better user experience. - Created unit tests for MemoryViewer to ensure functionality and interaction with data provider. - Updated translation files to include new UI strings related to memories. chore: move mcp-related files to own directory chore: rename librechat-mcp to librechat-api WIP: first pass, memory processing and data schemas chore: linting in fileSearch.js query description chore: rename librechat-api to @librechat/api across the project WIP: first pass, functional memory agent feat: add MemoryEditDialog and MemoryViewer components for managing user memories - Introduced MemoryEditDialog for editing memory entries with validation and toast notifications. - Updated MemoryViewer to support editing and deleting memories, including pagination and loading states. - Enhanced data provider to handle memory updates with optional original key for better management. - Added new localization strings for memory-related UI elements. feat: add memory permissions management - Implemented memory permissions in the backend, allowing roles to have specific permissions for using, creating, updating, and reading memories. - Added new API endpoints for updating memory permissions associated with roles. - Created a new AdminSettings component for managing memory permissions in the frontend. - Integrated memory permissions into the existing roles and permissions schemas. - Updated the interface to include memory settings and permissions. - Enhanced the MemoryViewer component to conditionally render admin settings based on user roles. - Added localization support for memory permissions in the translation files. feat: move AdminSettings component to a new position in MemoryViewer for better visibility refactor: clean up commented code in MemoryViewer component feat: enhance MemoryViewer with search functionality and improve MemoryEditDialog integration - Added a search input to filter memories in the MemoryViewer component. - Refactored MemoryEditDialog to accept children for better customization. - Updated MemoryViewer to utilize the new EditMemoryButton and DeleteMemoryButton components for editing and deleting memories. - Improved localization support by adding new strings for memory filtering and deletion confirmation. refactor: optimize memory filtering in MemoryViewer using match-sorter - Replaced manual filtering logic with match-sorter for improved search functionality. - Enhanced performance and readability of the filteredMemories computation. feat: enhance MemoryEditDialog with triggerRef and improve updateMemory mutation handling feat: implement access control for MemoryEditDialog and MemoryViewer components refactor: remove commented out code and create runMemory method refactor: rename role based files feat: implement access control for memory usage in AgentClient refactor: simplify checkVisionRequest method in AgentClient by removing commented-out code refactor: make `agents` dir in api package refactor: migrate Azure utilities to TypeScript and consolidate imports refactor: move sanitizeFilename function to a new file and update imports, add related tests refactor: update LLM configuration types and consolidate Azure options in the API package chore: linting chore: import order refactor: replace getLLMConfig with getOpenAIConfig and remove unused LLM configuration file chore: update winston-daily-rotate-file to version 5.0.0 and add object-hash dependency in package-lock.json refactor: move primeResources and optionalChainWithEmptyCheck functions to resources.ts and update imports refactor: move createRun function to a new run.ts file and update related imports fix: ensure safeAttachments is correctly typed as an array of TFile chore: add node-fetch dependency and refactor fetch-related functions into packages/api/utils, removing the old generators file refactor: enhance TEndpointOption type by using Pick to streamline endpoint fields and add new properties for model parameters and client options feat: implement initializeOpenAIOptions function and update OpenAI types for enhanced configuration handling fix: update types due to new TEndpointOption typing fix: ensure safe access to group parameters in initializeOpenAIOptions function fix: remove redundant API key validation comment in initializeOpenAIOptions function refactor: rename initializeOpenAIOptions to initializeOpenAI for consistency and update related documentation refactor: decouple req.body fields and tool loading from initializeAgentOptions chore: linting refactor: adjust column widths in MemoryViewer for improved layout refactor: simplify agent initialization by creating loadAgent function and removing unused code feat: add memory configuration loading and validation functions WIP: first pass, memory processing with config feat: implement memory callback and artifact handling feat: implement memory artifacts display and processing updates feat: add memory configuration options and schema validation for validKeys fix: update MemoryEditDialog and MemoryViewer to handle memory state and display improvements refactor: remove padding from BookmarkTable and MemoryViewer headers for consistent styling WIP: initial tokenLimit config and move Tokenizer to @librechat/api refactor: update mongoMeili plugin methods to use callback for better error handling feat: enhance memory management with token tracking and usage metrics - Added token counting for memory entries to enforce limits and provide usage statistics. - Updated memory retrieval and update routes to include total token usage and limit. - Enhanced MemoryEditDialog and MemoryViewer components to display memory usage and token information. - Refactored memory processing functions to handle token limits and provide feedback on memory capacity. feat: implement memory artifact handling in attachment handler - Enhanced useAttachmentHandler to process memory artifacts when receiving updates. - Introduced handleMemoryArtifact utility to manage memory updates and deletions. - Updated query client to reflect changes in memory state based on incoming data. refactor: restructure web search key extraction logic - Moved the logic for extracting API keys from the webSearchAuth configuration into a dedicated function, getWebSearchKeys. - Updated webSearchKeys to utilize the new function for improved clarity and maintainability. - Prevents build time errors feat: add personalization settings and memory preferences management - Introduced a new Personalization tab in settings to manage user memory preferences. - Implemented API endpoints and client-side logic for updating memory preferences. - Enhanced user interface components to reflect personalization options and memory usage. - Updated permissions to allow users to opt out of memory features. - Added localization support for new settings and messages related to personalization. style: personalization switch class feat: add PersonalizationIcon and align Side Panel UI feat: implement memory creation functionality - Added a new API endpoint for creating memory entries, including validation for key and value. - Introduced MemoryCreateDialog component for user interface to facilitate memory creation. - Integrated token limit checks to prevent exceeding user memory capacity. - Updated MemoryViewer to include a button for opening the memory creation dialog. - Enhanced localization support for new messages related to memory creation. feat: enhance message processing with configurable window size - Updated AgentClient to use a configurable message window size for processing messages. - Introduced messageWindowSize option in memory configuration schema with a default value of 5. - Improved logic for selecting messages to process based on the configured window size. chore: update librechat-data-provider version to 0.7.87 in package.json and package-lock.json chore: remove OpenAPIPlugin and its associated tests chore: remove MIGRATION_README.md as migration tasks are completed ci: fix backend tests chore: remove unused translation keys from localization file chore: remove problematic test file and unused var in AgentClient chore: remove unused import and import directly for JSDoc * feat: add api package build stage in Dockerfile for improved modularity * docs: reorder build steps in contributing guide for clarity
2025-06-07 18:52:22 -04:00
🏗️ feat: Dynamic MCP Server Infrastructure with Access Control (#10787) * Feature: Dynamic MCP Server with Full UI Management * 🚦 feat: Add MCP Connection Status icons to MCPBuilder panel (#10805) * feature: Add MCP server connection status icons to MCPBuilder panel * refactor: Simplify MCPConfigDialog rendering in MCPBuilderPanel --------- Co-authored-by: Atef Bellaaj <slalom.bellaaj@external.daimlertruck.com> Co-authored-by: Danny Avila <danny@librechat.ai> * fix: address code review feedback for MCP server management - Fix OAuth secret preservation to avoid mutating input parameter by creating a merged config copy in ServerConfigsDB.update() - Improve error handling in getResourcePermissionsMap to propagate critical errors instead of silently returning empty Map - Extract duplicated MCP server filter logic by exposing selectableServers from useMCPServerManager hook and using it in MCPSelect component * test: Update PermissionService tests to throw errors on invalid resource types - Changed the test for handling invalid resource types to ensure it throws an error instead of returning an empty permissions map. - Updated the expectation to check for the specific error message when an invalid resource type is provided. * feat: Implement retry logic for MCP server creation to handle race conditions - Enhanced the createMCPServer method to include retry logic with exponential backoff for handling duplicate key errors during concurrent server creation. - Updated tests to verify that all concurrent requests succeed and that unique server names are generated. - Added a helper function to identify MongoDB duplicate key errors, improving error handling during server creation. * refactor: StatusIcon to use CircleCheck for connected status - Replaced the PlugZap icon with CircleCheck in the ConnectedStatusIcon component to better represent the connected state. - Ensured consistent icon usage across the component for improved visual clarity. * test: Update AccessControlService tests to throw errors on invalid resource types - Modified the test for invalid resource types to ensure it throws an error with a specific message instead of returning an empty permissions map. - This change enhances error handling and improves test coverage for the AccessControlService. * fix: Update error message for missing server name in MCP server retrieval - Changed the error message returned when the server name is not provided from 'MCP ID is required' to 'Server name is required' for better clarity and accuracy in the API response. --------- Co-authored-by: Atef Bellaaj <slalom.bellaaj@external.daimlertruck.com> Co-authored-by: Danny Avila <danny@librechat.ai>
2025-12-04 21:37:23 +01:00
/**
* PUT /api/roles/:roleName/mcp-servers
* Update MCP servers permissions for a specific role
*/
📜 feat: Implement System Grants for Capability-Based Authorization (#11896) * feat: Implement System Grants for Role-Based Capabilities - Added a new `systemGrant` model and associated methods to manage role-based capabilities within the application. - Introduced middleware functions `hasCapability` and `requireCapability` to check user permissions based on their roles. - Updated the database seeding process to include system grants for the ADMIN role, ensuring all necessary capabilities are assigned on startup. - Enhanced type definitions and schemas to support the new system grant functionality, improving overall type safety and clarity in the codebase. * test: Add unit tests for capabilities middleware and system grant methods - Introduced comprehensive unit tests for the capabilities middleware, including `hasCapability` and `requireCapability`, ensuring proper permission checks based on user roles. - Added tests for the `SystemGrant` methods, verifying the seeding of system grants, capability granting, and revocation processes. - Enhanced test coverage for edge cases, including idempotency of grant operations and handling of unexpected errors in middleware. - Utilized mocks for database interactions to isolate tests and improve reliability. * refactor: Transition to Capability-Based Access Control - Replaced role-based access checks with capability-based checks across various middleware and routes, enhancing permission management. - Introduced `hasCapability` and `requireCapability` functions to streamline capability verification for user actions. - Updated relevant routes and middleware to utilize the new capability system, ensuring consistent permission enforcement. - Enhanced type definitions and added tests for the new capability functions, improving overall code reliability and maintainability. * test: Enhance capability-based access tests for ADMIN role - Updated tests to reflect the new capability-based access control, specifically for the ADMIN role. - Modified test descriptions to clarify that users with the MANAGE_AGENTS capability can bypass permission checks. - Seeded capabilities for the ADMIN role in multiple test files to ensure consistent permission checks across different routes and middleware. - Improved overall test coverage for capability verification, ensuring robust permission management. * test: Update capability tests for MCP server access - Renamed test to reflect the correct capability for bypassing permission checks, changing from MANAGE_AGENTS to MANAGE_MCP_SERVERS. - Updated seeding of capabilities for the ADMIN role to align with the new capability structure. - Ensured consistency in capability definitions across tests and middleware for improved permission management. * feat: Add hasConfigCapability for enhanced config access control - Introduced `hasConfigCapability` function to check user permissions for managing or reading specific config sections. - Updated middleware to export the new capability function, ensuring consistent access control across the application. - Enhanced unit tests to cover various scenarios for the new capability, improving overall test coverage and reliability. * fix: Update tenantId filter in createSystemGrantMethods - Added a condition to set tenantId filter to { $exists: false } when tenantId is null, ensuring proper handling of cases where tenantId is not provided. - This change improves the robustness of the system grant methods by explicitly managing the absence of tenantId in the filter logic. * fix: account deletion capability check - Updated the `canDeleteAccount` middleware to ensure that the `hasManageUsers` capability check only occurs if a user is present, preventing potential errors when the user object is undefined. - This change improves the robustness of the account deletion logic by ensuring proper handling of user permissions. * refactor: Optimize seeding of system grants for ADMIN role - Replaced sequential capability granting with parallel execution using Promise.all in the seedSystemGrants function. - This change improves performance and efficiency during the initialization of system grants, ensuring all capabilities are granted concurrently. * refactor: Simplify systemGrantSchema index definition - Removed the sparse option from the unique index on principalType, principalId, capability, and tenantId in the systemGrantSchema. - This change streamlines the index definition, potentially improving query performance and clarity in the schema design. * refactor: Reorganize role capability check in roles route - Moved the capability check for reading roles to occur after parsing the roleName, improving code clarity and structure. - This change ensures that the authorization logic is consistently applied before fetching role details, enhancing overall permission management. * refactor: Remove unused ISystemGrant interface from systemCapabilities.ts - Deleted the ISystemGrant interface as it was no longer needed, streamlining the code and improving clarity. - This change helps reduce clutter in the file and focuses on relevant capabilities for the system. * refactor: Migrate SystemCapabilities to data-schemas - Replaced imports of SystemCapabilities from 'librechat-data-provider' with imports from '@librechat/data-schemas' across multiple files. - This change centralizes the management of system capabilities, improving code organization and maintainability. * refactor: Update account deletion middleware and capability checks - Modified the `canDeleteAccount` middleware to ensure that the account deletion permission is only granted to users with the `MANAGE_USERS` capability, improving security and clarity in permission management. - Enhanced error logging for unauthorized account deletion attempts, providing better insights into permission issues. - Updated the `capabilities.ts` file to ensure consistent handling of user authentication checks, improving robustness in capability verification. - Refined type definitions in `systemGrant.ts` and `systemGrantMethods.ts` to utilize the `PrincipalType` enum, enhancing type safety and code clarity. * refactor: Extract principal ID normalization into a separate function - Introduced `normalizePrincipalId` function to streamline the normalization of principal IDs based on their type, enhancing code clarity and reusability. - Updated references in `createSystemGrantMethods` to utilize the new normalization function, improving maintainability and reducing code duplication. * test: Add unit tests for principalId normalization in systemGrant - Introduced tests for the `grantCapability`, `revokeCapability`, and `getCapabilitiesForPrincipal` methods to verify correct handling of principalId normalization between string and ObjectId formats. - Enhanced the `capabilities.ts` middleware to utilize the `PrincipalType` enum for improved type safety. - Added a new utility function `normalizePrincipalId` to streamline principal ID normalization logic, ensuring consistent behavior across the application. * feat: Introduce capability implications and enhance system grant methods - Added `CapabilityImplications` to define relationships between broader and implied capabilities, allowing for more intuitive permission checks. - Updated `createSystemGrantMethods` to expand capability queries to include implied capabilities, improving authorization logic. - Enhanced `systemGrantSchema` to include an `expiresAt` field for future TTL enforcement of grants, and added validation to ensure `tenantId` is not set to null. - Documented authorization requirements for prompt group and prompt deletion methods to clarify access control expectations. * test: Add unit tests for canDeleteAccount middleware - Introduced unit tests for the `canDeleteAccount` middleware to verify account deletion permissions based on user roles and capabilities. - Covered scenarios for both allowed and blocked account deletions, including checks for ADMIN users with the `MANAGE_USERS` capability and handling of undefined user cases. - Enhanced test structure to ensure clarity and maintainability of permission checks in the middleware. * fix: Add principalType enum validation to SystemGrant schema Without enum validation, any string value was accepted for principalType and silently stored. Invalid documents would never match capability queries, creating phantom grants impossible to diagnose without raw DB inspection. All other ACL models in the codebase validate this field. * fix: Replace seedSystemGrants Promise.all with bulkWrite for concurrency safety When two server instances start simultaneously (K8s rolling deploy, PM2 cluster), both call seedSystemGrants. With Promise.all + findOneAndUpdate upsert, both instances may attempt to insert the same documents, causing E11000 duplicate key errors that crash server startup. bulkWrite with ordered:false handles concurrent upserts gracefully and reduces 17 individual round trips to a single network call. The returned documents (previously discarded) are no longer fetched. * perf: Add AsyncLocalStorage per-request cache for capability checks Every hasCapability call previously required 2 DB round trips (getUserPrincipals + SystemGrant.exists) — replacing what were O(1) string comparisons. Routes like patchPromptGroup triggered this twice, and hasConfigCapability's fallback path resolved principals twice. This adds a per-request AsyncLocalStorage cache that: - Caches resolved principals (same for all checks within one request) - Caches capability check results (same user+cap = same answer) - Automatically scoped to request lifetime (no stale grants) - Falls through to DB when no store exists (background jobs, tests) - Requires no signature changes to hasCapability The capabilityContextMiddleware is registered at the app level before all routes, initializing a fresh store per request. * fix: Add error handling for inline hasCapability calls canDeleteAccount, fetchAssistants, and validateAuthor all call hasCapability without try-catch. These were previously O(1) string comparisons that could never throw. Now they hit the database and can fail on connection timeout or transient errors. Wrap each call in try-catch, defaulting to deny (false) on error. This ensures a DB hiccup returns a clean 403 instead of an unhandled 500 with a stack trace. * test: Add canDeleteAccount DB-error resilience test Tests that hasCapability rejection (e.g., DB timeout) results in a clean 403 rather than an unhandled exception. Validates the error handling added in the previous commit. * refactor: Use barrel import for hasCapability in validateAuthor Import from ~/server/middleware barrel instead of directly from ~/server/middleware/roles/capabilities for consistency with other non-middleware consumers. Files within the middleware barrel itself must continue using direct imports to avoid circular requires. * refactor: Remove misleading pre('save') hook from SystemGrant schema The pre('save') hook normalized principalId for USER/GROUP principals, but the primary write path (grantCapability) uses findOneAndUpdate — which does not trigger save hooks. The normalization was already handled explicitly in grantCapability itself. The hook created a false impression of schema-level enforcement that only covered save()/create() paths. Replace with a comment documenting that all writes must go through grantCapability. * feat: Add READ_ASSISTANTS capability to complete manage/read pair Every other managed resource had a paired READ_X / MANAGE_X capability except assistants. This adds READ_ASSISTANTS and registers the MANAGE_ASSISTANTS → READ_ASSISTANTS implication in CapabilityImplications, enabling future read-only assistant visibility grants. * chore: Reorder systemGrant methods for clarity Moved hasCapabilityForPrincipals to a more logical position in the returned object of createSystemGrantMethods, improving code readability. This change also maintains the inclusion of seedSystemGrants in the export, ensuring all necessary methods are available. * fix: Wrap seedSystemGrants in try-catch to avoid blocking startup Seeding capabilities is idempotent and will succeed on the next restart. A transient DB error during seeding should not prevent the server from starting — log the error and continue. * refactor: Improve capability check efficiency and add audit logging Move hasCapability calls after cheap early-exits in validateAuthor and fetchAssistants so the DB check only runs when its result matters. Add logger.debug on every capability bypass grant across all 7 call sites for auditability, and log errors in catch blocks instead of silently swallowing them. * test: Add integration tests for AsyncLocalStorage capability caching Exercises the full vertical — ALS context, generateCapabilityCheck, real getUserPrincipals, real hasCapabilityForPrincipals, real MongoDB via MongoMemoryServer. Covers per-request caching, cross-context isolation, concurrent request isolation, negative caching, capability implications, tenant scoping, group-based grants, and requireCapability middleware. * test: Add systemGrant data-layer and ALS edge-case integration tests systemGrant.spec.ts (51 tests): Full integration tests for all systemGrant methods against real MongoDB — grant/revoke lifecycle, principalId normalization (string→ObjectId for USER/GROUP, string for ROLE), capability implications (both directions), tenant scoping, schema validation (null tenantId, invalid enum, required fields, unique compound index). capabilities.integration.spec.ts (27 tests): Adds ALS edge cases — missing context degrades gracefully with no caching (background jobs, child processes), nested middleware creates independent inner context, optional-chaining safety when store is undefined, mid-request grant changes are invisible due to result caching, requireCapability works without ALS, and interleaved concurrent contexts maintain isolation. * fix: Add worker thread guards to capability ALS usage Detect when hasCapability or capabilityContextMiddleware is called from a worker thread (where ALS context does not propagate from the parent). hasCapability logs a warn-once per factory instance; the middleware logs an error since mounting Express middleware in a worker is likely a misconfiguration. Both continue to function correctly — the guard is observability, not a hard block. * fix: Include tenantId in ALS principal cache key for tenant isolation The principal cache key was user.id:user.role, which would reuse cached principals across tenants for the same user within a request. When getUserPrincipals gains tenant-scoped group resolution, principals from tenant-a would incorrectly serve tenant-b checks. Changed to user.id:user.role:user.tenantId to prevent cross-tenant cache hits. Adds integration test proving separate principal lookups per tenantId. * test: Remove redundant mocked capabilities.spec.js The JS wrapper test (7 tests, all mocked) is a strict subset of capabilities.integration.spec.ts (28 tests, real MongoDB). Every scenario it covered — hasCapability true/false, tenantId passthrough, requireCapability 403/500, error handling — is tested with higher fidelity in the integration suite. * test: Replace mocked canDeleteAccount tests with real MongoDB integration Remove hasCapability mock — tests now exercise the full capability chain against real MongoDB (getUserPrincipals, hasCapabilityForPrincipals, SystemGrant collection). Only mocks remaining are logger and cache. Adds new coverage: admin role without grant is blocked, user-level grant bypasses deletion restriction, null user handling. * test: Add comprehensive tests for ACL entry management and user group methods Introduces new tests for `deleteAclEntries`, `bulkWriteAclEntries`, and `findPublicResourceIds` in `aclEntry.spec.ts`, ensuring proper functionality for deleting and bulk managing ACL entries. Additionally, enhances `userGroup.spec.ts` with tests for finding groups by ID and name pattern, including external ID matching and source filtering. These changes improve coverage and validate the integrity of ACL and user group operations against real MongoDB interactions. * refactor: Update capability checks and logging for better clarity and error handling Replaced `MANAGE_USERS` with `ACCESS_ADMIN` in the `canDeleteAccount` middleware and related tests to align with updated permission structure. Enhanced logging in various middleware functions to use `logger.warn` for capability check failures, providing clearer error messages. Additionally, refactored capability checks in the `patchPromptGroup` and `validateAuthor` functions to improve readability and maintainability. This commit also includes adjustments to the `systemGrant` methods to implement retry logic for transient failures during capability seeding, ensuring robustness in the face of database errors. * refactor: Enhance logging and retry logic in seedSystemGrants method Updated the logging format in the seedSystemGrants method to include error messages for better clarity. Improved the retry mechanism by explicitly mocking multiple failures in tests, ensuring robust error handling during transient database issues. Additionally, refined imports in the systemGrant schema for better type management. * refactor: Consolidate imports in canDeleteAccount middleware Merged logger and SystemCapabilities imports from the data-schemas module into a single line for improved readability and maintainability of the code. This change streamlines the import statements in the canDeleteAccount middleware. * test: Enhance systemGrant tests for error handling and capability validation Added tests to the systemGrant methods to handle various error scenarios, including E11000 race conditions, invalid ObjectId strings for USER and GROUP principals, and invalid capability strings. These enhancements improve the robustness of the capability granting and revoking logic, ensuring proper error propagation and validation of inputs. * fix: Wrap hasCapability calls in deny-by-default try-catch at remaining sites canAccessResource, files.js, and roles.js all had hasCapability inside outer try-catch blocks that returned 500 on DB failure instead of falling through to the regular ACL check. This contradicts the deny-by-default pattern used everywhere else. Also removes raw error.message from the roles.js 500 response to prevent internal host/connection info leaking to clients. * fix: Normalize user ID in canDeleteAccount before passing to hasCapability requireCapability normalizes req.user.id via _id?.toString() fallback, but canDeleteAccount passed raw req.user directly. If req.user.id is absent (some auth layers only populate _id), getUserPrincipals received undefined, silently returning empty principals and blocking the bypass. * fix: Harden systemGrant schema and type safety - Reject empty string tenantId in schema validator (was only blocking null; empty string silently orphaned documents) - Fix reverseImplications to use BaseSystemCapability[] instead of string[], preserving the narrow discriminated type - Document READ_ASSISTANTS as reserved/unenforced * test: Use fake timers for seedSystemGrants retry tests and add tenantId validation - Switch retry tests to jest.useFakeTimers() to eliminate 3+ seconds of real setTimeout delays per test run - Add regression test for empty-string tenantId rejection * docs: Add TODO(#12091) comments for tenant-scoped capability gaps In multi-tenant mode, platform-level grants (no tenantId) won't match tenant-scoped queries, breaking admin access. getUserPrincipals also returns cross-tenant group memberships. Both need fixes in #12091.
2026-03-07 13:56:32 -05:00
router.put('/:roleName/mcp-servers', manageRoles, createPermissionUpdateHandler('mcp-servers'));
🏗️ feat: Dynamic MCP Server Infrastructure with Access Control (#10787) * Feature: Dynamic MCP Server with Full UI Management * 🚦 feat: Add MCP Connection Status icons to MCPBuilder panel (#10805) * feature: Add MCP server connection status icons to MCPBuilder panel * refactor: Simplify MCPConfigDialog rendering in MCPBuilderPanel --------- Co-authored-by: Atef Bellaaj <slalom.bellaaj@external.daimlertruck.com> Co-authored-by: Danny Avila <danny@librechat.ai> * fix: address code review feedback for MCP server management - Fix OAuth secret preservation to avoid mutating input parameter by creating a merged config copy in ServerConfigsDB.update() - Improve error handling in getResourcePermissionsMap to propagate critical errors instead of silently returning empty Map - Extract duplicated MCP server filter logic by exposing selectableServers from useMCPServerManager hook and using it in MCPSelect component * test: Update PermissionService tests to throw errors on invalid resource types - Changed the test for handling invalid resource types to ensure it throws an error instead of returning an empty permissions map. - Updated the expectation to check for the specific error message when an invalid resource type is provided. * feat: Implement retry logic for MCP server creation to handle race conditions - Enhanced the createMCPServer method to include retry logic with exponential backoff for handling duplicate key errors during concurrent server creation. - Updated tests to verify that all concurrent requests succeed and that unique server names are generated. - Added a helper function to identify MongoDB duplicate key errors, improving error handling during server creation. * refactor: StatusIcon to use CircleCheck for connected status - Replaced the PlugZap icon with CircleCheck in the ConnectedStatusIcon component to better represent the connected state. - Ensured consistent icon usage across the component for improved visual clarity. * test: Update AccessControlService tests to throw errors on invalid resource types - Modified the test for invalid resource types to ensure it throws an error with a specific message instead of returning an empty permissions map. - This change enhances error handling and improves test coverage for the AccessControlService. * fix: Update error message for missing server name in MCP server retrieval - Changed the error message returned when the server name is not provided from 'MCP ID is required' to 'Server name is required' for better clarity and accuracy in the API response. --------- Co-authored-by: Atef Bellaaj <slalom.bellaaj@external.daimlertruck.com> Co-authored-by: Danny Avila <danny@librechat.ai>
2025-12-04 21:37:23 +01:00
/**
* PUT /api/roles/:roleName/marketplace
* Update marketplace permissions for a specific role
*/
📜 feat: Implement System Grants for Capability-Based Authorization (#11896) * feat: Implement System Grants for Role-Based Capabilities - Added a new `systemGrant` model and associated methods to manage role-based capabilities within the application. - Introduced middleware functions `hasCapability` and `requireCapability` to check user permissions based on their roles. - Updated the database seeding process to include system grants for the ADMIN role, ensuring all necessary capabilities are assigned on startup. - Enhanced type definitions and schemas to support the new system grant functionality, improving overall type safety and clarity in the codebase. * test: Add unit tests for capabilities middleware and system grant methods - Introduced comprehensive unit tests for the capabilities middleware, including `hasCapability` and `requireCapability`, ensuring proper permission checks based on user roles. - Added tests for the `SystemGrant` methods, verifying the seeding of system grants, capability granting, and revocation processes. - Enhanced test coverage for edge cases, including idempotency of grant operations and handling of unexpected errors in middleware. - Utilized mocks for database interactions to isolate tests and improve reliability. * refactor: Transition to Capability-Based Access Control - Replaced role-based access checks with capability-based checks across various middleware and routes, enhancing permission management. - Introduced `hasCapability` and `requireCapability` functions to streamline capability verification for user actions. - Updated relevant routes and middleware to utilize the new capability system, ensuring consistent permission enforcement. - Enhanced type definitions and added tests for the new capability functions, improving overall code reliability and maintainability. * test: Enhance capability-based access tests for ADMIN role - Updated tests to reflect the new capability-based access control, specifically for the ADMIN role. - Modified test descriptions to clarify that users with the MANAGE_AGENTS capability can bypass permission checks. - Seeded capabilities for the ADMIN role in multiple test files to ensure consistent permission checks across different routes and middleware. - Improved overall test coverage for capability verification, ensuring robust permission management. * test: Update capability tests for MCP server access - Renamed test to reflect the correct capability for bypassing permission checks, changing from MANAGE_AGENTS to MANAGE_MCP_SERVERS. - Updated seeding of capabilities for the ADMIN role to align with the new capability structure. - Ensured consistency in capability definitions across tests and middleware for improved permission management. * feat: Add hasConfigCapability for enhanced config access control - Introduced `hasConfigCapability` function to check user permissions for managing or reading specific config sections. - Updated middleware to export the new capability function, ensuring consistent access control across the application. - Enhanced unit tests to cover various scenarios for the new capability, improving overall test coverage and reliability. * fix: Update tenantId filter in createSystemGrantMethods - Added a condition to set tenantId filter to { $exists: false } when tenantId is null, ensuring proper handling of cases where tenantId is not provided. - This change improves the robustness of the system grant methods by explicitly managing the absence of tenantId in the filter logic. * fix: account deletion capability check - Updated the `canDeleteAccount` middleware to ensure that the `hasManageUsers` capability check only occurs if a user is present, preventing potential errors when the user object is undefined. - This change improves the robustness of the account deletion logic by ensuring proper handling of user permissions. * refactor: Optimize seeding of system grants for ADMIN role - Replaced sequential capability granting with parallel execution using Promise.all in the seedSystemGrants function. - This change improves performance and efficiency during the initialization of system grants, ensuring all capabilities are granted concurrently. * refactor: Simplify systemGrantSchema index definition - Removed the sparse option from the unique index on principalType, principalId, capability, and tenantId in the systemGrantSchema. - This change streamlines the index definition, potentially improving query performance and clarity in the schema design. * refactor: Reorganize role capability check in roles route - Moved the capability check for reading roles to occur after parsing the roleName, improving code clarity and structure. - This change ensures that the authorization logic is consistently applied before fetching role details, enhancing overall permission management. * refactor: Remove unused ISystemGrant interface from systemCapabilities.ts - Deleted the ISystemGrant interface as it was no longer needed, streamlining the code and improving clarity. - This change helps reduce clutter in the file and focuses on relevant capabilities for the system. * refactor: Migrate SystemCapabilities to data-schemas - Replaced imports of SystemCapabilities from 'librechat-data-provider' with imports from '@librechat/data-schemas' across multiple files. - This change centralizes the management of system capabilities, improving code organization and maintainability. * refactor: Update account deletion middleware and capability checks - Modified the `canDeleteAccount` middleware to ensure that the account deletion permission is only granted to users with the `MANAGE_USERS` capability, improving security and clarity in permission management. - Enhanced error logging for unauthorized account deletion attempts, providing better insights into permission issues. - Updated the `capabilities.ts` file to ensure consistent handling of user authentication checks, improving robustness in capability verification. - Refined type definitions in `systemGrant.ts` and `systemGrantMethods.ts` to utilize the `PrincipalType` enum, enhancing type safety and code clarity. * refactor: Extract principal ID normalization into a separate function - Introduced `normalizePrincipalId` function to streamline the normalization of principal IDs based on their type, enhancing code clarity and reusability. - Updated references in `createSystemGrantMethods` to utilize the new normalization function, improving maintainability and reducing code duplication. * test: Add unit tests for principalId normalization in systemGrant - Introduced tests for the `grantCapability`, `revokeCapability`, and `getCapabilitiesForPrincipal` methods to verify correct handling of principalId normalization between string and ObjectId formats. - Enhanced the `capabilities.ts` middleware to utilize the `PrincipalType` enum for improved type safety. - Added a new utility function `normalizePrincipalId` to streamline principal ID normalization logic, ensuring consistent behavior across the application. * feat: Introduce capability implications and enhance system grant methods - Added `CapabilityImplications` to define relationships between broader and implied capabilities, allowing for more intuitive permission checks. - Updated `createSystemGrantMethods` to expand capability queries to include implied capabilities, improving authorization logic. - Enhanced `systemGrantSchema` to include an `expiresAt` field for future TTL enforcement of grants, and added validation to ensure `tenantId` is not set to null. - Documented authorization requirements for prompt group and prompt deletion methods to clarify access control expectations. * test: Add unit tests for canDeleteAccount middleware - Introduced unit tests for the `canDeleteAccount` middleware to verify account deletion permissions based on user roles and capabilities. - Covered scenarios for both allowed and blocked account deletions, including checks for ADMIN users with the `MANAGE_USERS` capability and handling of undefined user cases. - Enhanced test structure to ensure clarity and maintainability of permission checks in the middleware. * fix: Add principalType enum validation to SystemGrant schema Without enum validation, any string value was accepted for principalType and silently stored. Invalid documents would never match capability queries, creating phantom grants impossible to diagnose without raw DB inspection. All other ACL models in the codebase validate this field. * fix: Replace seedSystemGrants Promise.all with bulkWrite for concurrency safety When two server instances start simultaneously (K8s rolling deploy, PM2 cluster), both call seedSystemGrants. With Promise.all + findOneAndUpdate upsert, both instances may attempt to insert the same documents, causing E11000 duplicate key errors that crash server startup. bulkWrite with ordered:false handles concurrent upserts gracefully and reduces 17 individual round trips to a single network call. The returned documents (previously discarded) are no longer fetched. * perf: Add AsyncLocalStorage per-request cache for capability checks Every hasCapability call previously required 2 DB round trips (getUserPrincipals + SystemGrant.exists) — replacing what were O(1) string comparisons. Routes like patchPromptGroup triggered this twice, and hasConfigCapability's fallback path resolved principals twice. This adds a per-request AsyncLocalStorage cache that: - Caches resolved principals (same for all checks within one request) - Caches capability check results (same user+cap = same answer) - Automatically scoped to request lifetime (no stale grants) - Falls through to DB when no store exists (background jobs, tests) - Requires no signature changes to hasCapability The capabilityContextMiddleware is registered at the app level before all routes, initializing a fresh store per request. * fix: Add error handling for inline hasCapability calls canDeleteAccount, fetchAssistants, and validateAuthor all call hasCapability without try-catch. These were previously O(1) string comparisons that could never throw. Now they hit the database and can fail on connection timeout or transient errors. Wrap each call in try-catch, defaulting to deny (false) on error. This ensures a DB hiccup returns a clean 403 instead of an unhandled 500 with a stack trace. * test: Add canDeleteAccount DB-error resilience test Tests that hasCapability rejection (e.g., DB timeout) results in a clean 403 rather than an unhandled exception. Validates the error handling added in the previous commit. * refactor: Use barrel import for hasCapability in validateAuthor Import from ~/server/middleware barrel instead of directly from ~/server/middleware/roles/capabilities for consistency with other non-middleware consumers. Files within the middleware barrel itself must continue using direct imports to avoid circular requires. * refactor: Remove misleading pre('save') hook from SystemGrant schema The pre('save') hook normalized principalId for USER/GROUP principals, but the primary write path (grantCapability) uses findOneAndUpdate — which does not trigger save hooks. The normalization was already handled explicitly in grantCapability itself. The hook created a false impression of schema-level enforcement that only covered save()/create() paths. Replace with a comment documenting that all writes must go through grantCapability. * feat: Add READ_ASSISTANTS capability to complete manage/read pair Every other managed resource had a paired READ_X / MANAGE_X capability except assistants. This adds READ_ASSISTANTS and registers the MANAGE_ASSISTANTS → READ_ASSISTANTS implication in CapabilityImplications, enabling future read-only assistant visibility grants. * chore: Reorder systemGrant methods for clarity Moved hasCapabilityForPrincipals to a more logical position in the returned object of createSystemGrantMethods, improving code readability. This change also maintains the inclusion of seedSystemGrants in the export, ensuring all necessary methods are available. * fix: Wrap seedSystemGrants in try-catch to avoid blocking startup Seeding capabilities is idempotent and will succeed on the next restart. A transient DB error during seeding should not prevent the server from starting — log the error and continue. * refactor: Improve capability check efficiency and add audit logging Move hasCapability calls after cheap early-exits in validateAuthor and fetchAssistants so the DB check only runs when its result matters. Add logger.debug on every capability bypass grant across all 7 call sites for auditability, and log errors in catch blocks instead of silently swallowing them. * test: Add integration tests for AsyncLocalStorage capability caching Exercises the full vertical — ALS context, generateCapabilityCheck, real getUserPrincipals, real hasCapabilityForPrincipals, real MongoDB via MongoMemoryServer. Covers per-request caching, cross-context isolation, concurrent request isolation, negative caching, capability implications, tenant scoping, group-based grants, and requireCapability middleware. * test: Add systemGrant data-layer and ALS edge-case integration tests systemGrant.spec.ts (51 tests): Full integration tests for all systemGrant methods against real MongoDB — grant/revoke lifecycle, principalId normalization (string→ObjectId for USER/GROUP, string for ROLE), capability implications (both directions), tenant scoping, schema validation (null tenantId, invalid enum, required fields, unique compound index). capabilities.integration.spec.ts (27 tests): Adds ALS edge cases — missing context degrades gracefully with no caching (background jobs, child processes), nested middleware creates independent inner context, optional-chaining safety when store is undefined, mid-request grant changes are invisible due to result caching, requireCapability works without ALS, and interleaved concurrent contexts maintain isolation. * fix: Add worker thread guards to capability ALS usage Detect when hasCapability or capabilityContextMiddleware is called from a worker thread (where ALS context does not propagate from the parent). hasCapability logs a warn-once per factory instance; the middleware logs an error since mounting Express middleware in a worker is likely a misconfiguration. Both continue to function correctly — the guard is observability, not a hard block. * fix: Include tenantId in ALS principal cache key for tenant isolation The principal cache key was user.id:user.role, which would reuse cached principals across tenants for the same user within a request. When getUserPrincipals gains tenant-scoped group resolution, principals from tenant-a would incorrectly serve tenant-b checks. Changed to user.id:user.role:user.tenantId to prevent cross-tenant cache hits. Adds integration test proving separate principal lookups per tenantId. * test: Remove redundant mocked capabilities.spec.js The JS wrapper test (7 tests, all mocked) is a strict subset of capabilities.integration.spec.ts (28 tests, real MongoDB). Every scenario it covered — hasCapability true/false, tenantId passthrough, requireCapability 403/500, error handling — is tested with higher fidelity in the integration suite. * test: Replace mocked canDeleteAccount tests with real MongoDB integration Remove hasCapability mock — tests now exercise the full capability chain against real MongoDB (getUserPrincipals, hasCapabilityForPrincipals, SystemGrant collection). Only mocks remaining are logger and cache. Adds new coverage: admin role without grant is blocked, user-level grant bypasses deletion restriction, null user handling. * test: Add comprehensive tests for ACL entry management and user group methods Introduces new tests for `deleteAclEntries`, `bulkWriteAclEntries`, and `findPublicResourceIds` in `aclEntry.spec.ts`, ensuring proper functionality for deleting and bulk managing ACL entries. Additionally, enhances `userGroup.spec.ts` with tests for finding groups by ID and name pattern, including external ID matching and source filtering. These changes improve coverage and validate the integrity of ACL and user group operations against real MongoDB interactions. * refactor: Update capability checks and logging for better clarity and error handling Replaced `MANAGE_USERS` with `ACCESS_ADMIN` in the `canDeleteAccount` middleware and related tests to align with updated permission structure. Enhanced logging in various middleware functions to use `logger.warn` for capability check failures, providing clearer error messages. Additionally, refactored capability checks in the `patchPromptGroup` and `validateAuthor` functions to improve readability and maintainability. This commit also includes adjustments to the `systemGrant` methods to implement retry logic for transient failures during capability seeding, ensuring robustness in the face of database errors. * refactor: Enhance logging and retry logic in seedSystemGrants method Updated the logging format in the seedSystemGrants method to include error messages for better clarity. Improved the retry mechanism by explicitly mocking multiple failures in tests, ensuring robust error handling during transient database issues. Additionally, refined imports in the systemGrant schema for better type management. * refactor: Consolidate imports in canDeleteAccount middleware Merged logger and SystemCapabilities imports from the data-schemas module into a single line for improved readability and maintainability of the code. This change streamlines the import statements in the canDeleteAccount middleware. * test: Enhance systemGrant tests for error handling and capability validation Added tests to the systemGrant methods to handle various error scenarios, including E11000 race conditions, invalid ObjectId strings for USER and GROUP principals, and invalid capability strings. These enhancements improve the robustness of the capability granting and revoking logic, ensuring proper error propagation and validation of inputs. * fix: Wrap hasCapability calls in deny-by-default try-catch at remaining sites canAccessResource, files.js, and roles.js all had hasCapability inside outer try-catch blocks that returned 500 on DB failure instead of falling through to the regular ACL check. This contradicts the deny-by-default pattern used everywhere else. Also removes raw error.message from the roles.js 500 response to prevent internal host/connection info leaking to clients. * fix: Normalize user ID in canDeleteAccount before passing to hasCapability requireCapability normalizes req.user.id via _id?.toString() fallback, but canDeleteAccount passed raw req.user directly. If req.user.id is absent (some auth layers only populate _id), getUserPrincipals received undefined, silently returning empty principals and blocking the bypass. * fix: Harden systemGrant schema and type safety - Reject empty string tenantId in schema validator (was only blocking null; empty string silently orphaned documents) - Fix reverseImplications to use BaseSystemCapability[] instead of string[], preserving the narrow discriminated type - Document READ_ASSISTANTS as reserved/unenforced * test: Use fake timers for seedSystemGrants retry tests and add tenantId validation - Switch retry tests to jest.useFakeTimers() to eliminate 3+ seconds of real setTimeout delays per test run - Add regression test for empty-string tenantId rejection * docs: Add TODO(#12091) comments for tenant-scoped capability gaps In multi-tenant mode, platform-level grants (no tenantId) won't match tenant-scoped queries, breaking admin access. getUserPrincipals also returns cross-tenant group memberships. Both need fixes in #12091.
2026-03-07 13:56:32 -05:00
router.put('/:roleName/marketplace', manageRoles, createPermissionUpdateHandler('marketplace'));
🛸 feat: Remote Agent Access with External API Support (#11503) * 🪪 feat: Microsoft Graph Access Token Placeholder for MCP Servers (#10867) * feat: MCP Graph Token env var * Addressing copilot remarks * Addressed Copilot review remarks * Fixed graphtokenservice mock in MCP test suite * fix: remove unnecessary type check and cast in resolveGraphTokensInRecord * ci: add Graph Token integration tests in MCPManager * refactor: update user type definitions to use Partial<IUser> in multiple functions * test: enhance MCP tests for graph token processing and user placeholder resolution - Added comprehensive tests to validate the interaction between preProcessGraphTokens and processMCPEnv. - Ensured correct resolution of graph tokens and user placeholders in various configurations. - Mocked OIDC utilities to facilitate testing of token extraction and validation. - Verified that original options remain unchanged after processing. * chore: import order * chore: imports --------- Co-authored-by: Danny Avila <danny@librechat.ai> * WIP: OpenAI-compatible API for LibreChat agents - Added OpenAIChatCompletionController for handling chat completions. - Introduced ListModelsController and GetModelController for listing and retrieving agent details. - Created routes for OpenAI API endpoints, including /v1/chat/completions and /v1/models. - Developed event handlers for streaming responses in OpenAI format. - Implemented request validation and error handling for API interactions. - Integrated content aggregation and response formatting to align with OpenAI specifications. This commit establishes a foundational API for interacting with LibreChat agents in a manner compatible with OpenAI's chat completion interface. * refactor: OpenAI-spec content aggregation for improved performance and clarity * fix: OpenAI chat completion controller with safe user handling for correct tool loading * refactor: Remove conversation ID from OpenAI response context and related handlers * refactor: OpenAI chat completion handling with streaming support - Introduced a lightweight tracker for streaming responses, allowing for efficient tracking of emitted content and usage metadata. - Updated the OpenAIChatCompletionController to utilize the new tracker, improving the handling of streaming and non-streaming responses. - Refactored event handlers to accommodate the new streaming logic, ensuring proper management of tool calls and content aggregation. - Adjusted response handling to streamline error reporting during streaming sessions. * WIP: Open Responses API with core service, types, and handlers - Added Open Responses API module with comprehensive types and enums. - Implemented core service for processing requests, including validation and input conversion. - Developed event handlers for streaming responses and non-streaming aggregation. - Established response building logic and error handling mechanisms. - Created detailed types for input and output content, ensuring compliance with Open Responses specification. * feat: Implement response storage and retrieval in Open Responses API - Added functionality to save user input messages and assistant responses to the database when the `store` flag is set to true. - Introduced a new endpoint to retrieve stored responses by ID, allowing users to access previous interactions. - Enhanced the response creation process to include database operations for conversation and message storage. - Implemented tests to validate the storage and retrieval of responses, ensuring correct behavior for both existing and non-existent response IDs. * refactor: Open Responses API with additional token tracking and validation - Added support for tracking cached tokens in response usage, improving token management. - Updated response structure to include new properties for top log probabilities and detailed usage metrics. - Enhanced tests to validate the presence and types of new properties in API responses, ensuring compliance with updated specifications. - Refactored response handling to accommodate new fields and improve overall clarity and performance. * refactor: Update reasoning event handlers and types for consistency - Renamed reasoning text events to simplify naming conventions, changing `emitReasoningTextDelta` to `emitReasoningDelta` and `emitReasoningTextDone` to `emitReasoningDone`. - Updated event types in the API to reflect the new naming, ensuring consistency across the codebase. - Added `logprobs` property to output events for enhanced tracking of log probabilities. * feat: Add validation for streaming events in Open Responses API tests * feat: Implement response.created event in Open Responses API - Added emitResponseCreated function to emit the response.created event as the first event in the streaming sequence, adhering to the Open Responses specification. - Updated createResponse function to emit response.created followed by response.in_progress. - Enhanced tests to validate the order of emitted events, ensuring response.created is triggered before response.in_progress. * feat: Responses API with attachment event handling - Introduced `createResponsesToolEndCallback` to handle attachment events in the Responses API, emitting `librechat:attachment` events as per the Open Responses extension specification. - Updated the `createResponse` function to utilize the new callback for processing tool outputs and emitting attachments during streaming. - Added helper functions for writing attachment events and defined types for attachment data, ensuring compatibility with the Open Responses protocol. - Enhanced tests to validate the integration of attachment events within the Responses API workflow. * WIP: remote agent auth * fix: Improve loading state handling in AgentApiKeys component - Updated the rendering logic to conditionally display loading spinner and API keys based on the loading state. - Removed unnecessary imports and streamlined the component for better readability. * refactor: Update API key access handling in routes - Replaced `checkAccess` with `generateCheckAccess` for improved access control. - Consolidated access checks into a single `checkApiKeyAccess` function, enhancing code readability and maintainability. - Streamlined route definitions for creating, listing, retrieving, and deleting API keys. * fix: Add permission handling for REMOTE_AGENT resource type * feat: Enhance permission handling for REMOTE_AGENT resources - Updated the deleteAgent and deleteUserAgents functions to handle permissions for both AGENT and REMOTE_AGENT resource types. - Introduced new functions to enrich REMOTE_AGENT principals and backfill permissions for AGENT owners. - Modified createAgentHandler and duplicateAgentHandler to grant permissions for REMOTE_AGENT alongside AGENT. - Added utility functions for retrieving effective permissions for REMOTE_AGENT resources, ensuring consistent access control across the application. * refactor: Rename and update roles for remote agent access - Changed role name from API User to Editor in translation files for clarity. - Updated default editor role ID from REMOTE_AGENT_USER to REMOTE_AGENT_EDITOR in resource configurations. - Adjusted role localization to reflect the new Editor role. - Modified access permissions to align with the updated role definitions across the application. * feat: Introduce remote agent permissions and update access handling - Added support for REMOTE_AGENTS in permission schemas, including use, create, share, and share_public permissions. - Updated the interface configuration to include remote agent settings. - Modified middleware and API key access checks to align with the new remote agent permission structure. - Enhanced role defaults to incorporate remote agent permissions, ensuring consistent access control across the application. * refactor: Update AgentApiKeys component and permissions handling - Refactored the AgentApiKeys component to improve structure and readability, including the introduction of ApiKeysContent for better separation of concerns. - Updated CreateKeyDialog to accept an onKeyCreated callback, enhancing its functionality. - Adjusted permission checks in Data component to use REMOTE_AGENTS and USE permissions, aligning with recent permission schema changes. - Enhanced loading state handling and dialog management for a smoother user experience. * refactor: Update remote agent access checks in API routes - Replaced existing access checks with `generateCheckAccess` for remote agents in the API keys and agents routes. - Introduced specific permission checks for creating, listing, retrieving, and deleting API keys, enhancing access control. - Improved code structure by consolidating permission handling for remote agents across multiple routes. * fix: Correct query parameters in ApiKeysContent component - Updated the useGetAgentApiKeysQuery call to include an object for the enabled parameter, ensuring proper functionality when the component is open. - This change improves the handling of API key retrieval based on the component's open state. * feat: Implement remote agents permissions and update API routes - Added new API route for updating remote agents permissions, enhancing role management capabilities. - Introduced remote agents permissions handling in the AgentApiKeys component, including a dedicated settings dialog. - Updated localization files to include new remote agents permission labels for better user experience. - Refactored data provider to support remote agents permissions updates, ensuring consistent access control across the application. * feat: Add remote agents permissions to role schema and interface - Introduced new permissions for REMOTE_AGENTS in the role schema, including USE, CREATE, SHARE, and SHARE_PUBLIC. - Updated the IRole interface to reflect the new remote agents permissions structure, enhancing role management capabilities. * feat: Add remote agents settings button to API keys dialog * feat: Update AgentFooter to include remote agent sharing permissions - Refactored access checks to incorporate permissions for sharing remote agents. - Enhanced conditional rendering logic to allow sharing by users with remote agent permissions. - Improved loading state handling for remote agent permissions, ensuring a smoother user experience. * refactor: Update API key creation access check and localization strings - Replaced the access check for creating API keys to use the existing remote agents access check. - Updated localization strings to correct the descriptions for remote agent permissions, ensuring clarity in user interface. * fix: resource permission mapping to include remote agents - Changed the resourceToPermissionMap to use a Partial<Record> for better flexibility. - Added mapping for REMOTE_AGENT permissions, enhancing the sharing capabilities for remote agents. * feat: Implement remote access checks for agent models - Enhanced ListModelsController and GetModelController to include checks for user permissions on remote agents. - Integrated findAccessibleResources to filter agents based on VIEW permission for REMOTE_AGENT. - Updated response handling to ensure users can only access agents they have permissions for, improving security and access control. * fix: Update user parameter type in processUserPlaceholders function - Changed the user parameter type in the processUserPlaceholders function from Partial<Partial<IUser>> to Partial<IUser> for improved type clarity and consistency. * refactor: Simplify integration test structure by removing conditional describe - Replaced conditional describeWithApiKey with a standard describe for all integration tests in responses.spec.js. - This change enhances test clarity and ensures all tests are executed consistently, regardless of the SKIP_INTEGRATION_TESTS flag. * test: Update AgentFooter tests to reflect new grant access dialog ID - Changed test IDs for the grant access dialog in AgentFooter tests to include the resource type, ensuring accurate identification in the test cases. - This update improves test clarity and aligns with recent changes in the component's implementation. * test: Enhance integration tests for Open Responses API - Updated integration tests in responses.spec.js to utilize an authRequest helper for consistent authorization handling across all test cases. - Introduced a test user and API key creation to improve test setup and ensure proper permission checks for remote agents. - Added checks for existing access roles and created necessary roles if they do not exist, enhancing test reliability and coverage. * feat: Extend accessRole schema to include remoteAgent resource type - Updated the accessRole schema to add 'remoteAgent' to the resourceType enum, enhancing the flexibility of role assignments and permissions management. * test: refactored test setup to create a minimal Express app for responses routes, enhancing test structure and maintainability. * test: Enhance abort.spec.js by mocking additional modules for improved test isolation - Updated the test setup in abort.spec.js to include actual implementations of '@librechat/data-schemas' and '@librechat/api' while maintaining mock functionality. - This change improves test reliability and ensures that the tests are more representative of the actual module behavior. * refactor: Update conversation ID generation to use UUID - Replaced the nanoid with uuidv4 for generating conversation IDs in the createResponse function, enhancing uniqueness and consistency in ID generation. * test: Add remote agent access roles to AccessRole model tests - Included additional access roles for remote agents (REMOTE_AGENT_EDITOR, REMOTE_AGENT_OWNER, REMOTE_AGENT_VIEWER) in the AccessRole model tests to ensure comprehensive coverage of role assignments and permissions management. * chore: Add deletion of user agent API keys in user deletion process - Updated the user deletion process in UserController and delete-user.js to include the removal of user agent API keys, ensuring comprehensive cleanup of user data upon account deletion. * test: Add remote agents permissions to permissions.spec.ts - Enhanced the permissions tests by including comprehensive permission settings for remote agents across various scenarios, ensuring accurate validation of access controls for remote agent roles. * chore: Update remote agents translations for clarity and consistency - Removed outdated remote agents translation entries and added revised entries to improve clarity on API key creation and sharing permissions for remote agents. This enhances user understanding of the available functionalities. * feat: Add indexing and TTL for agent API keys - Introduced an index on the `key` field for improved query performance. - Added a TTL index on the `expiresAt` field to enable automatic cleanup of expired API keys, ensuring efficient management of stored keys. * chore: Update API route documentation for clarity - Revised comments in the agents route file to clarify the handling of API key authentication. - Removed outdated endpoint listings to streamline the documentation and focus on current functionality. --------- Co-authored-by: Max Sanna <max@maxsanna.com>
2026-01-26 10:50:30 -05:00
/**
* PUT /api/roles/:roleName/remote-agents
* Update remote agents (API) permissions for a specific role
*/
📜 feat: Implement System Grants for Capability-Based Authorization (#11896) * feat: Implement System Grants for Role-Based Capabilities - Added a new `systemGrant` model and associated methods to manage role-based capabilities within the application. - Introduced middleware functions `hasCapability` and `requireCapability` to check user permissions based on their roles. - Updated the database seeding process to include system grants for the ADMIN role, ensuring all necessary capabilities are assigned on startup. - Enhanced type definitions and schemas to support the new system grant functionality, improving overall type safety and clarity in the codebase. * test: Add unit tests for capabilities middleware and system grant methods - Introduced comprehensive unit tests for the capabilities middleware, including `hasCapability` and `requireCapability`, ensuring proper permission checks based on user roles. - Added tests for the `SystemGrant` methods, verifying the seeding of system grants, capability granting, and revocation processes. - Enhanced test coverage for edge cases, including idempotency of grant operations and handling of unexpected errors in middleware. - Utilized mocks for database interactions to isolate tests and improve reliability. * refactor: Transition to Capability-Based Access Control - Replaced role-based access checks with capability-based checks across various middleware and routes, enhancing permission management. - Introduced `hasCapability` and `requireCapability` functions to streamline capability verification for user actions. - Updated relevant routes and middleware to utilize the new capability system, ensuring consistent permission enforcement. - Enhanced type definitions and added tests for the new capability functions, improving overall code reliability and maintainability. * test: Enhance capability-based access tests for ADMIN role - Updated tests to reflect the new capability-based access control, specifically for the ADMIN role. - Modified test descriptions to clarify that users with the MANAGE_AGENTS capability can bypass permission checks. - Seeded capabilities for the ADMIN role in multiple test files to ensure consistent permission checks across different routes and middleware. - Improved overall test coverage for capability verification, ensuring robust permission management. * test: Update capability tests for MCP server access - Renamed test to reflect the correct capability for bypassing permission checks, changing from MANAGE_AGENTS to MANAGE_MCP_SERVERS. - Updated seeding of capabilities for the ADMIN role to align with the new capability structure. - Ensured consistency in capability definitions across tests and middleware for improved permission management. * feat: Add hasConfigCapability for enhanced config access control - Introduced `hasConfigCapability` function to check user permissions for managing or reading specific config sections. - Updated middleware to export the new capability function, ensuring consistent access control across the application. - Enhanced unit tests to cover various scenarios for the new capability, improving overall test coverage and reliability. * fix: Update tenantId filter in createSystemGrantMethods - Added a condition to set tenantId filter to { $exists: false } when tenantId is null, ensuring proper handling of cases where tenantId is not provided. - This change improves the robustness of the system grant methods by explicitly managing the absence of tenantId in the filter logic. * fix: account deletion capability check - Updated the `canDeleteAccount` middleware to ensure that the `hasManageUsers` capability check only occurs if a user is present, preventing potential errors when the user object is undefined. - This change improves the robustness of the account deletion logic by ensuring proper handling of user permissions. * refactor: Optimize seeding of system grants for ADMIN role - Replaced sequential capability granting with parallel execution using Promise.all in the seedSystemGrants function. - This change improves performance and efficiency during the initialization of system grants, ensuring all capabilities are granted concurrently. * refactor: Simplify systemGrantSchema index definition - Removed the sparse option from the unique index on principalType, principalId, capability, and tenantId in the systemGrantSchema. - This change streamlines the index definition, potentially improving query performance and clarity in the schema design. * refactor: Reorganize role capability check in roles route - Moved the capability check for reading roles to occur after parsing the roleName, improving code clarity and structure. - This change ensures that the authorization logic is consistently applied before fetching role details, enhancing overall permission management. * refactor: Remove unused ISystemGrant interface from systemCapabilities.ts - Deleted the ISystemGrant interface as it was no longer needed, streamlining the code and improving clarity. - This change helps reduce clutter in the file and focuses on relevant capabilities for the system. * refactor: Migrate SystemCapabilities to data-schemas - Replaced imports of SystemCapabilities from 'librechat-data-provider' with imports from '@librechat/data-schemas' across multiple files. - This change centralizes the management of system capabilities, improving code organization and maintainability. * refactor: Update account deletion middleware and capability checks - Modified the `canDeleteAccount` middleware to ensure that the account deletion permission is only granted to users with the `MANAGE_USERS` capability, improving security and clarity in permission management. - Enhanced error logging for unauthorized account deletion attempts, providing better insights into permission issues. - Updated the `capabilities.ts` file to ensure consistent handling of user authentication checks, improving robustness in capability verification. - Refined type definitions in `systemGrant.ts` and `systemGrantMethods.ts` to utilize the `PrincipalType` enum, enhancing type safety and code clarity. * refactor: Extract principal ID normalization into a separate function - Introduced `normalizePrincipalId` function to streamline the normalization of principal IDs based on their type, enhancing code clarity and reusability. - Updated references in `createSystemGrantMethods` to utilize the new normalization function, improving maintainability and reducing code duplication. * test: Add unit tests for principalId normalization in systemGrant - Introduced tests for the `grantCapability`, `revokeCapability`, and `getCapabilitiesForPrincipal` methods to verify correct handling of principalId normalization between string and ObjectId formats. - Enhanced the `capabilities.ts` middleware to utilize the `PrincipalType` enum for improved type safety. - Added a new utility function `normalizePrincipalId` to streamline principal ID normalization logic, ensuring consistent behavior across the application. * feat: Introduce capability implications and enhance system grant methods - Added `CapabilityImplications` to define relationships between broader and implied capabilities, allowing for more intuitive permission checks. - Updated `createSystemGrantMethods` to expand capability queries to include implied capabilities, improving authorization logic. - Enhanced `systemGrantSchema` to include an `expiresAt` field for future TTL enforcement of grants, and added validation to ensure `tenantId` is not set to null. - Documented authorization requirements for prompt group and prompt deletion methods to clarify access control expectations. * test: Add unit tests for canDeleteAccount middleware - Introduced unit tests for the `canDeleteAccount` middleware to verify account deletion permissions based on user roles and capabilities. - Covered scenarios for both allowed and blocked account deletions, including checks for ADMIN users with the `MANAGE_USERS` capability and handling of undefined user cases. - Enhanced test structure to ensure clarity and maintainability of permission checks in the middleware. * fix: Add principalType enum validation to SystemGrant schema Without enum validation, any string value was accepted for principalType and silently stored. Invalid documents would never match capability queries, creating phantom grants impossible to diagnose without raw DB inspection. All other ACL models in the codebase validate this field. * fix: Replace seedSystemGrants Promise.all with bulkWrite for concurrency safety When two server instances start simultaneously (K8s rolling deploy, PM2 cluster), both call seedSystemGrants. With Promise.all + findOneAndUpdate upsert, both instances may attempt to insert the same documents, causing E11000 duplicate key errors that crash server startup. bulkWrite with ordered:false handles concurrent upserts gracefully and reduces 17 individual round trips to a single network call. The returned documents (previously discarded) are no longer fetched. * perf: Add AsyncLocalStorage per-request cache for capability checks Every hasCapability call previously required 2 DB round trips (getUserPrincipals + SystemGrant.exists) — replacing what were O(1) string comparisons. Routes like patchPromptGroup triggered this twice, and hasConfigCapability's fallback path resolved principals twice. This adds a per-request AsyncLocalStorage cache that: - Caches resolved principals (same for all checks within one request) - Caches capability check results (same user+cap = same answer) - Automatically scoped to request lifetime (no stale grants) - Falls through to DB when no store exists (background jobs, tests) - Requires no signature changes to hasCapability The capabilityContextMiddleware is registered at the app level before all routes, initializing a fresh store per request. * fix: Add error handling for inline hasCapability calls canDeleteAccount, fetchAssistants, and validateAuthor all call hasCapability without try-catch. These were previously O(1) string comparisons that could never throw. Now they hit the database and can fail on connection timeout or transient errors. Wrap each call in try-catch, defaulting to deny (false) on error. This ensures a DB hiccup returns a clean 403 instead of an unhandled 500 with a stack trace. * test: Add canDeleteAccount DB-error resilience test Tests that hasCapability rejection (e.g., DB timeout) results in a clean 403 rather than an unhandled exception. Validates the error handling added in the previous commit. * refactor: Use barrel import for hasCapability in validateAuthor Import from ~/server/middleware barrel instead of directly from ~/server/middleware/roles/capabilities for consistency with other non-middleware consumers. Files within the middleware barrel itself must continue using direct imports to avoid circular requires. * refactor: Remove misleading pre('save') hook from SystemGrant schema The pre('save') hook normalized principalId for USER/GROUP principals, but the primary write path (grantCapability) uses findOneAndUpdate — which does not trigger save hooks. The normalization was already handled explicitly in grantCapability itself. The hook created a false impression of schema-level enforcement that only covered save()/create() paths. Replace with a comment documenting that all writes must go through grantCapability. * feat: Add READ_ASSISTANTS capability to complete manage/read pair Every other managed resource had a paired READ_X / MANAGE_X capability except assistants. This adds READ_ASSISTANTS and registers the MANAGE_ASSISTANTS → READ_ASSISTANTS implication in CapabilityImplications, enabling future read-only assistant visibility grants. * chore: Reorder systemGrant methods for clarity Moved hasCapabilityForPrincipals to a more logical position in the returned object of createSystemGrantMethods, improving code readability. This change also maintains the inclusion of seedSystemGrants in the export, ensuring all necessary methods are available. * fix: Wrap seedSystemGrants in try-catch to avoid blocking startup Seeding capabilities is idempotent and will succeed on the next restart. A transient DB error during seeding should not prevent the server from starting — log the error and continue. * refactor: Improve capability check efficiency and add audit logging Move hasCapability calls after cheap early-exits in validateAuthor and fetchAssistants so the DB check only runs when its result matters. Add logger.debug on every capability bypass grant across all 7 call sites for auditability, and log errors in catch blocks instead of silently swallowing them. * test: Add integration tests for AsyncLocalStorage capability caching Exercises the full vertical — ALS context, generateCapabilityCheck, real getUserPrincipals, real hasCapabilityForPrincipals, real MongoDB via MongoMemoryServer. Covers per-request caching, cross-context isolation, concurrent request isolation, negative caching, capability implications, tenant scoping, group-based grants, and requireCapability middleware. * test: Add systemGrant data-layer and ALS edge-case integration tests systemGrant.spec.ts (51 tests): Full integration tests for all systemGrant methods against real MongoDB — grant/revoke lifecycle, principalId normalization (string→ObjectId for USER/GROUP, string for ROLE), capability implications (both directions), tenant scoping, schema validation (null tenantId, invalid enum, required fields, unique compound index). capabilities.integration.spec.ts (27 tests): Adds ALS edge cases — missing context degrades gracefully with no caching (background jobs, child processes), nested middleware creates independent inner context, optional-chaining safety when store is undefined, mid-request grant changes are invisible due to result caching, requireCapability works without ALS, and interleaved concurrent contexts maintain isolation. * fix: Add worker thread guards to capability ALS usage Detect when hasCapability or capabilityContextMiddleware is called from a worker thread (where ALS context does not propagate from the parent). hasCapability logs a warn-once per factory instance; the middleware logs an error since mounting Express middleware in a worker is likely a misconfiguration. Both continue to function correctly — the guard is observability, not a hard block. * fix: Include tenantId in ALS principal cache key for tenant isolation The principal cache key was user.id:user.role, which would reuse cached principals across tenants for the same user within a request. When getUserPrincipals gains tenant-scoped group resolution, principals from tenant-a would incorrectly serve tenant-b checks. Changed to user.id:user.role:user.tenantId to prevent cross-tenant cache hits. Adds integration test proving separate principal lookups per tenantId. * test: Remove redundant mocked capabilities.spec.js The JS wrapper test (7 tests, all mocked) is a strict subset of capabilities.integration.spec.ts (28 tests, real MongoDB). Every scenario it covered — hasCapability true/false, tenantId passthrough, requireCapability 403/500, error handling — is tested with higher fidelity in the integration suite. * test: Replace mocked canDeleteAccount tests with real MongoDB integration Remove hasCapability mock — tests now exercise the full capability chain against real MongoDB (getUserPrincipals, hasCapabilityForPrincipals, SystemGrant collection). Only mocks remaining are logger and cache. Adds new coverage: admin role without grant is blocked, user-level grant bypasses deletion restriction, null user handling. * test: Add comprehensive tests for ACL entry management and user group methods Introduces new tests for `deleteAclEntries`, `bulkWriteAclEntries`, and `findPublicResourceIds` in `aclEntry.spec.ts`, ensuring proper functionality for deleting and bulk managing ACL entries. Additionally, enhances `userGroup.spec.ts` with tests for finding groups by ID and name pattern, including external ID matching and source filtering. These changes improve coverage and validate the integrity of ACL and user group operations against real MongoDB interactions. * refactor: Update capability checks and logging for better clarity and error handling Replaced `MANAGE_USERS` with `ACCESS_ADMIN` in the `canDeleteAccount` middleware and related tests to align with updated permission structure. Enhanced logging in various middleware functions to use `logger.warn` for capability check failures, providing clearer error messages. Additionally, refactored capability checks in the `patchPromptGroup` and `validateAuthor` functions to improve readability and maintainability. This commit also includes adjustments to the `systemGrant` methods to implement retry logic for transient failures during capability seeding, ensuring robustness in the face of database errors. * refactor: Enhance logging and retry logic in seedSystemGrants method Updated the logging format in the seedSystemGrants method to include error messages for better clarity. Improved the retry mechanism by explicitly mocking multiple failures in tests, ensuring robust error handling during transient database issues. Additionally, refined imports in the systemGrant schema for better type management. * refactor: Consolidate imports in canDeleteAccount middleware Merged logger and SystemCapabilities imports from the data-schemas module into a single line for improved readability and maintainability of the code. This change streamlines the import statements in the canDeleteAccount middleware. * test: Enhance systemGrant tests for error handling and capability validation Added tests to the systemGrant methods to handle various error scenarios, including E11000 race conditions, invalid ObjectId strings for USER and GROUP principals, and invalid capability strings. These enhancements improve the robustness of the capability granting and revoking logic, ensuring proper error propagation and validation of inputs. * fix: Wrap hasCapability calls in deny-by-default try-catch at remaining sites canAccessResource, files.js, and roles.js all had hasCapability inside outer try-catch blocks that returned 500 on DB failure instead of falling through to the regular ACL check. This contradicts the deny-by-default pattern used everywhere else. Also removes raw error.message from the roles.js 500 response to prevent internal host/connection info leaking to clients. * fix: Normalize user ID in canDeleteAccount before passing to hasCapability requireCapability normalizes req.user.id via _id?.toString() fallback, but canDeleteAccount passed raw req.user directly. If req.user.id is absent (some auth layers only populate _id), getUserPrincipals received undefined, silently returning empty principals and blocking the bypass. * fix: Harden systemGrant schema and type safety - Reject empty string tenantId in schema validator (was only blocking null; empty string silently orphaned documents) - Fix reverseImplications to use BaseSystemCapability[] instead of string[], preserving the narrow discriminated type - Document READ_ASSISTANTS as reserved/unenforced * test: Use fake timers for seedSystemGrants retry tests and add tenantId validation - Switch retry tests to jest.useFakeTimers() to eliminate 3+ seconds of real setTimeout delays per test run - Add regression test for empty-string tenantId rejection * docs: Add TODO(#12091) comments for tenant-scoped capability gaps In multi-tenant mode, platform-level grants (no tenantId) won't match tenant-scoped queries, breaking admin access. getUserPrincipals also returns cross-tenant group memberships. Both need fixes in #12091.
2026-03-07 13:56:32 -05:00
router.put('/:roleName/remote-agents', manageRoles, createPermissionUpdateHandler('remote-agents'));
🛸 feat: Remote Agent Access with External API Support (#11503) * 🪪 feat: Microsoft Graph Access Token Placeholder for MCP Servers (#10867) * feat: MCP Graph Token env var * Addressing copilot remarks * Addressed Copilot review remarks * Fixed graphtokenservice mock in MCP test suite * fix: remove unnecessary type check and cast in resolveGraphTokensInRecord * ci: add Graph Token integration tests in MCPManager * refactor: update user type definitions to use Partial<IUser> in multiple functions * test: enhance MCP tests for graph token processing and user placeholder resolution - Added comprehensive tests to validate the interaction between preProcessGraphTokens and processMCPEnv. - Ensured correct resolution of graph tokens and user placeholders in various configurations. - Mocked OIDC utilities to facilitate testing of token extraction and validation. - Verified that original options remain unchanged after processing. * chore: import order * chore: imports --------- Co-authored-by: Danny Avila <danny@librechat.ai> * WIP: OpenAI-compatible API for LibreChat agents - Added OpenAIChatCompletionController for handling chat completions. - Introduced ListModelsController and GetModelController for listing and retrieving agent details. - Created routes for OpenAI API endpoints, including /v1/chat/completions and /v1/models. - Developed event handlers for streaming responses in OpenAI format. - Implemented request validation and error handling for API interactions. - Integrated content aggregation and response formatting to align with OpenAI specifications. This commit establishes a foundational API for interacting with LibreChat agents in a manner compatible with OpenAI's chat completion interface. * refactor: OpenAI-spec content aggregation for improved performance and clarity * fix: OpenAI chat completion controller with safe user handling for correct tool loading * refactor: Remove conversation ID from OpenAI response context and related handlers * refactor: OpenAI chat completion handling with streaming support - Introduced a lightweight tracker for streaming responses, allowing for efficient tracking of emitted content and usage metadata. - Updated the OpenAIChatCompletionController to utilize the new tracker, improving the handling of streaming and non-streaming responses. - Refactored event handlers to accommodate the new streaming logic, ensuring proper management of tool calls and content aggregation. - Adjusted response handling to streamline error reporting during streaming sessions. * WIP: Open Responses API with core service, types, and handlers - Added Open Responses API module with comprehensive types and enums. - Implemented core service for processing requests, including validation and input conversion. - Developed event handlers for streaming responses and non-streaming aggregation. - Established response building logic and error handling mechanisms. - Created detailed types for input and output content, ensuring compliance with Open Responses specification. * feat: Implement response storage and retrieval in Open Responses API - Added functionality to save user input messages and assistant responses to the database when the `store` flag is set to true. - Introduced a new endpoint to retrieve stored responses by ID, allowing users to access previous interactions. - Enhanced the response creation process to include database operations for conversation and message storage. - Implemented tests to validate the storage and retrieval of responses, ensuring correct behavior for both existing and non-existent response IDs. * refactor: Open Responses API with additional token tracking and validation - Added support for tracking cached tokens in response usage, improving token management. - Updated response structure to include new properties for top log probabilities and detailed usage metrics. - Enhanced tests to validate the presence and types of new properties in API responses, ensuring compliance with updated specifications. - Refactored response handling to accommodate new fields and improve overall clarity and performance. * refactor: Update reasoning event handlers and types for consistency - Renamed reasoning text events to simplify naming conventions, changing `emitReasoningTextDelta` to `emitReasoningDelta` and `emitReasoningTextDone` to `emitReasoningDone`. - Updated event types in the API to reflect the new naming, ensuring consistency across the codebase. - Added `logprobs` property to output events for enhanced tracking of log probabilities. * feat: Add validation for streaming events in Open Responses API tests * feat: Implement response.created event in Open Responses API - Added emitResponseCreated function to emit the response.created event as the first event in the streaming sequence, adhering to the Open Responses specification. - Updated createResponse function to emit response.created followed by response.in_progress. - Enhanced tests to validate the order of emitted events, ensuring response.created is triggered before response.in_progress. * feat: Responses API with attachment event handling - Introduced `createResponsesToolEndCallback` to handle attachment events in the Responses API, emitting `librechat:attachment` events as per the Open Responses extension specification. - Updated the `createResponse` function to utilize the new callback for processing tool outputs and emitting attachments during streaming. - Added helper functions for writing attachment events and defined types for attachment data, ensuring compatibility with the Open Responses protocol. - Enhanced tests to validate the integration of attachment events within the Responses API workflow. * WIP: remote agent auth * fix: Improve loading state handling in AgentApiKeys component - Updated the rendering logic to conditionally display loading spinner and API keys based on the loading state. - Removed unnecessary imports and streamlined the component for better readability. * refactor: Update API key access handling in routes - Replaced `checkAccess` with `generateCheckAccess` for improved access control. - Consolidated access checks into a single `checkApiKeyAccess` function, enhancing code readability and maintainability. - Streamlined route definitions for creating, listing, retrieving, and deleting API keys. * fix: Add permission handling for REMOTE_AGENT resource type * feat: Enhance permission handling for REMOTE_AGENT resources - Updated the deleteAgent and deleteUserAgents functions to handle permissions for both AGENT and REMOTE_AGENT resource types. - Introduced new functions to enrich REMOTE_AGENT principals and backfill permissions for AGENT owners. - Modified createAgentHandler and duplicateAgentHandler to grant permissions for REMOTE_AGENT alongside AGENT. - Added utility functions for retrieving effective permissions for REMOTE_AGENT resources, ensuring consistent access control across the application. * refactor: Rename and update roles for remote agent access - Changed role name from API User to Editor in translation files for clarity. - Updated default editor role ID from REMOTE_AGENT_USER to REMOTE_AGENT_EDITOR in resource configurations. - Adjusted role localization to reflect the new Editor role. - Modified access permissions to align with the updated role definitions across the application. * feat: Introduce remote agent permissions and update access handling - Added support for REMOTE_AGENTS in permission schemas, including use, create, share, and share_public permissions. - Updated the interface configuration to include remote agent settings. - Modified middleware and API key access checks to align with the new remote agent permission structure. - Enhanced role defaults to incorporate remote agent permissions, ensuring consistent access control across the application. * refactor: Update AgentApiKeys component and permissions handling - Refactored the AgentApiKeys component to improve structure and readability, including the introduction of ApiKeysContent for better separation of concerns. - Updated CreateKeyDialog to accept an onKeyCreated callback, enhancing its functionality. - Adjusted permission checks in Data component to use REMOTE_AGENTS and USE permissions, aligning with recent permission schema changes. - Enhanced loading state handling and dialog management for a smoother user experience. * refactor: Update remote agent access checks in API routes - Replaced existing access checks with `generateCheckAccess` for remote agents in the API keys and agents routes. - Introduced specific permission checks for creating, listing, retrieving, and deleting API keys, enhancing access control. - Improved code structure by consolidating permission handling for remote agents across multiple routes. * fix: Correct query parameters in ApiKeysContent component - Updated the useGetAgentApiKeysQuery call to include an object for the enabled parameter, ensuring proper functionality when the component is open. - This change improves the handling of API key retrieval based on the component's open state. * feat: Implement remote agents permissions and update API routes - Added new API route for updating remote agents permissions, enhancing role management capabilities. - Introduced remote agents permissions handling in the AgentApiKeys component, including a dedicated settings dialog. - Updated localization files to include new remote agents permission labels for better user experience. - Refactored data provider to support remote agents permissions updates, ensuring consistent access control across the application. * feat: Add remote agents permissions to role schema and interface - Introduced new permissions for REMOTE_AGENTS in the role schema, including USE, CREATE, SHARE, and SHARE_PUBLIC. - Updated the IRole interface to reflect the new remote agents permissions structure, enhancing role management capabilities. * feat: Add remote agents settings button to API keys dialog * feat: Update AgentFooter to include remote agent sharing permissions - Refactored access checks to incorporate permissions for sharing remote agents. - Enhanced conditional rendering logic to allow sharing by users with remote agent permissions. - Improved loading state handling for remote agent permissions, ensuring a smoother user experience. * refactor: Update API key creation access check and localization strings - Replaced the access check for creating API keys to use the existing remote agents access check. - Updated localization strings to correct the descriptions for remote agent permissions, ensuring clarity in user interface. * fix: resource permission mapping to include remote agents - Changed the resourceToPermissionMap to use a Partial<Record> for better flexibility. - Added mapping for REMOTE_AGENT permissions, enhancing the sharing capabilities for remote agents. * feat: Implement remote access checks for agent models - Enhanced ListModelsController and GetModelController to include checks for user permissions on remote agents. - Integrated findAccessibleResources to filter agents based on VIEW permission for REMOTE_AGENT. - Updated response handling to ensure users can only access agents they have permissions for, improving security and access control. * fix: Update user parameter type in processUserPlaceholders function - Changed the user parameter type in the processUserPlaceholders function from Partial<Partial<IUser>> to Partial<IUser> for improved type clarity and consistency. * refactor: Simplify integration test structure by removing conditional describe - Replaced conditional describeWithApiKey with a standard describe for all integration tests in responses.spec.js. - This change enhances test clarity and ensures all tests are executed consistently, regardless of the SKIP_INTEGRATION_TESTS flag. * test: Update AgentFooter tests to reflect new grant access dialog ID - Changed test IDs for the grant access dialog in AgentFooter tests to include the resource type, ensuring accurate identification in the test cases. - This update improves test clarity and aligns with recent changes in the component's implementation. * test: Enhance integration tests for Open Responses API - Updated integration tests in responses.spec.js to utilize an authRequest helper for consistent authorization handling across all test cases. - Introduced a test user and API key creation to improve test setup and ensure proper permission checks for remote agents. - Added checks for existing access roles and created necessary roles if they do not exist, enhancing test reliability and coverage. * feat: Extend accessRole schema to include remoteAgent resource type - Updated the accessRole schema to add 'remoteAgent' to the resourceType enum, enhancing the flexibility of role assignments and permissions management. * test: refactored test setup to create a minimal Express app for responses routes, enhancing test structure and maintainability. * test: Enhance abort.spec.js by mocking additional modules for improved test isolation - Updated the test setup in abort.spec.js to include actual implementations of '@librechat/data-schemas' and '@librechat/api' while maintaining mock functionality. - This change improves test reliability and ensures that the tests are more representative of the actual module behavior. * refactor: Update conversation ID generation to use UUID - Replaced the nanoid with uuidv4 for generating conversation IDs in the createResponse function, enhancing uniqueness and consistency in ID generation. * test: Add remote agent access roles to AccessRole model tests - Included additional access roles for remote agents (REMOTE_AGENT_EDITOR, REMOTE_AGENT_OWNER, REMOTE_AGENT_VIEWER) in the AccessRole model tests to ensure comprehensive coverage of role assignments and permissions management. * chore: Add deletion of user agent API keys in user deletion process - Updated the user deletion process in UserController and delete-user.js to include the removal of user agent API keys, ensuring comprehensive cleanup of user data upon account deletion. * test: Add remote agents permissions to permissions.spec.ts - Enhanced the permissions tests by including comprehensive permission settings for remote agents across various scenarios, ensuring accurate validation of access controls for remote agent roles. * chore: Update remote agents translations for clarity and consistency - Removed outdated remote agents translation entries and added revised entries to improve clarity on API key creation and sharing permissions for remote agents. This enhances user understanding of the available functionalities. * feat: Add indexing and TTL for agent API keys - Introduced an index on the `key` field for improved query performance. - Added a TTL index on the `expiresAt` field to enable automatic cleanup of expired API keys, ensuring efficient management of stored keys. * chore: Update API route documentation for clarity - Revised comments in the agents route file to clarify the handling of API key authentication. - Removed outdated endpoint listings to streamline the documentation and focus on current functionality. --------- Co-authored-by: Max Sanna <max@maxsanna.com>
2026-01-26 10:50:30 -05:00
🗨️ feat: Prompts (#3131) * 🗨️ feat: Prompts (#7) * WIP: MERGE prompts/frontend (#1) * added schema for prompt and promptgroup, added model methods for prompts, added routes for prompts * * updated promptGroup Schema * updated model methods for prompts (get, add, delete) * slight fixes in prompt routes * * Created Files Management components * Created Vector Stores components * Added file management route in the routes folder * Completed UI for Files list, Compeleted UI for vector stores list, Completed UI for upload file modal, Completed UI for preview file, Completed UI for preview vector store * Fixed style and UI fixes for file dashboard, file list and vector stores list * added responsiveness classes for vector store page * fixed responsiveness of file page, dashboard page, and main page * fixed styling and responsiveness issues on dashboard page, file list page and vector store page * added queries and mutations for prompts and promptGroups, added relevant endpoints in data-provider, added relevant components prompts, added and updated relevant APIs * added types on mutation queries data service, updated prompt attributes * feature: Prompts and prompt groups management, added relevant APIs, added types for data service/queries/mutations, added relevant mutation and queries * chore: typing clarifications * added drop down on prompts mgmt dashboard * Fixes: fixed version switching issue on tags update or labels update, added cross button on create prompt group, fixed list updation on prompt group renaiming, added CSV upload button * Feature: Added oneliner and category attributes in prompt group, added schema for categories, added schema methods and route for categories * chore: typing and lint issues * chore: more type and linter fixes * chore: linting * chore: prompt controller and backend typing example; MOVE TO CONTROLLER DIRECTORY * chore: more type fixes * style: prompt name changes * chore: more type changes, and stateful prompt name change without flickering * fix: Return result of savePrompt in patchPrompt API endpoint * fix: navigation prompt queries; refactor: name 'prompt-groups' to just 'groups' * refactor: fetch prompt groups rewrite * refactor(prompts): query/mutation statefulness * refactor: remove `isActive` field * refactor: remove labels, consolidate logic * style: width, layout shift * refactor: improve hover toggle behavior and styling * refactor: add useParams hook to PromptListItem for dynamic rendering and add timeout ref for blur timeout * chore: hide upload button * refactor: import Button component from correct location in PromptSidePanel * style: prompt editor styling * style: fix more layout shifts * style: container scroll * refactor: Rename CreatePrompt component to CreatePromptForm * refactor: use react-hook-form * refactor: Add Prompts components and routes to Dashboard * style: skeletons for loading * fix: optimize makePromptProduction * refactor: consolidate variables * feat: create prompt form validation * refactor: Consolidate variables and update mutation hooks * style: minor touchups * chore: Update lucide-react npm dependency to version 0.394.0 and npm audit fix * refactor: add a new icon for the Prompts heading. * style: Update PromptsView heading to use h1 instead of h2 and other minor margin issues * chore: wording * refactor: Update PromptsView heading to use h1 instead of h2, consolidate variables, and add new icons * refactor: Prompts Button for Mobile * feature: added category field in prompt group, added relevant API and static data on BE to support FE UI for category in prompt group * chore: template for prompt cards --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * WIP: Prompts/frontend Continued (#2) * chore: loading style, remove unused component * feat: Add CategorySelector component for prompt group category selection * feat: add categories to create prompt * feat: prompt versions styling * feat: optimistic updates for prompt production state * refactor: optimize form state and show if prompt field is dirty with cross icon, also other styling changes * chore: remove unused code and localizations * fix: light mode styling * WIP: SidePanel Prompts * refactor: move to groups directory * refactor: rename GroupsSidePanel to GroupSidePanel and update imports * style: ListCard * refactor: isProduction changes * refactor: infinite query with productionPrompt * refactor: optimize snippets and prompts, and styling * refactor: Update getSnippet function to accept a length parameter * chore: localizations * feat: prompts navigation to chat and vice versa * fix: create prompt * feat: remember last selected category for creating prompts * fix(promptGroups): fix pagination and add usePromptGroupsNav hook * Prompts/frontend 3 (#3) * fix: stateful issues with prompt groups * style: improved layout * refactor: improve variable naming in Eng.ts * refactor: theme selector styling improvements * added prompt cards on chat new page, with dark mode, added API to fetch random prompts, added types for useQuery Slightly improved usePromptGroupNav logic to fetch updated result for pageSize, updated prompt cards view with darkmode and responsiveness fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page fixed page size option buttons styling to match the theme added dark mode on create prompt page and prompt edit/preview page * WIP: Prompts/frontend (#4) * fix: optimize and fix paginated query * fix: remove unique constraint on names * refactor: button links and styling * style: menu border light mode * feat: Add Auto-Send Switch component for prompts groups * refactor(ChatView): use form context for submission text * chore: clear convo state on navigation to dashboard routes * chore: save prompt edit name on tab, remove console log * feat: basic prompt submission * refactor: move Auto-Send Switch * style(ListCard): border styling * feat: Add function to detect variables in text * feat: Add OriginalDialog component to UI library * chore(ui): Update SelectDropDown options list class to use text-xs size * refactor: submitMessage hook now includes submitPrompt, make compatible to document query selector * WIP: Variable Dialog * feat: variable submission working for both auto-send and non-autosend * feat: dashboard breadcrumbs and prompts/chat navigation * refactor: dashboard breadcrumb and dashboard link to chat navigation * refactor: Update VariableDialog and VariableForm styles * Prompts: Admin features (#5) * fix: link issue * fix: usePromptGroupsNav add missing dep. * style: dashbreadcrumb and sidepanel text color * temp fix: remove refetch on pageNumber change * fix: handle multiple variable replacement * WIP: create project schema and add project groups to fetch * feat: Add functionality to add prompt group IDs to a project * feat: Add caching for startup config in config route * chore: remove prompt landing * style: Update Skeleton component with additional background styling * chore: styling and types * WIP: SharePrompt first draft * feat(SharePrompt): form validation * feat: shared global indicators * refactor: prompt details * refactor: change NoPromptGroup directory * feat: preview prompt * feat: remove/add global prompts, add rbac-related enums * refactor: manage prompts location * WIP: first draft admin settings for prompts * feat: SystemRoles enum * refactor: update PromptDetails component styling * style: ellipsis custom class for showing more preview text * WIP: initial role schema and initialization * style: improved margins for single unordered lists * fix: use custom chat form context to prevent re-renders from FormProvider * feat: Role mutations for Prompt Permissions * feat: fetch user role * feat: update AdminSettings form default values from user role values * refactor: rename PromptPermissions to Permissions for general definitions * feat: initial role checks * feat: Add optional `bodyProps` parameter to generateCheckAccess middleware * refactor: UI access checks * Prompts: delete (#6) * Fixed delete prompt version API, fixed types and logic for prompt version deletion, updated prompt delete mutation logic * chore: Update return type of deletePrompt function in Prompt.js --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * chore: Update package-lock.json version to 0.7.4-rc1 and fast-xml-parser to 4.4.0 * feat: toast for saving admin settings, add timer no-access navigation * feat: always make prod * feat: Add localization to category labels in CategorySelector component * feat: Update category label localization in CategorySelector component * fix: Enable making prompt production in Prompt API --------- Co-authored-by: Fawadpot <contactfawada@gmail.com> * feat: Add helper fn for dark mode detection in ThemeProvider * style: surface-primary definition * fix(useHasAccess): utilize user.role and not just USER role * fix: empty category and role fetch * refactort: increase max height to options list and use label if no localization is found * fix: update CategorySelector to handle empty category value and improve localization * refactor: move prompts to own store/reactquery modules, add in filter WIP * refactor: Rename AutoSendSwitch to AutoSendPrompt * style: theming commit * style: fix slight coloring issue for convos in dark mode * style: better composition for prompts side panel * style: remove gray-750 and make it gray-850 * chore: adjust theming * feat: filter all prompt groups and properly remove prompts from projects * refactor: optimize delete prompt groups further * chore: localization * feat: Add uniqueProperty filtering to normalizeData function * WIP: filter prompts * chore: Update FilterPrompts component to include User icon in FilterItem * feat(FilterPrompts): set categories * feat: more system filters and show selected category icon * style: always make prod, flips switch to avoid mis-clicks * style: ui/ux loading/no prompts * chore: style FilterPrompts ChatView * fix: handle missing role edge case * style: special variables * feat: special variables * refactor: improve replaceSpecialVars function in prompts.ts * feat: simple/advanced editor modes * chore: bump versions * feat: localizations and hide production button on simple mode * fix: error connecting layout shift * fix: prompts CRUD for admins * fix: secure single group fetch * style: sidepanel styling * style(PromptName): bring edit button closer to name * style: mobile prompts header * style: mobile prompts header continued * style: align send prompts switch right * feat: description * Update special variables description in Eng.ts * feat: update/create/preview oneliner * fix: allow empty oneliner update * style: loading improvement and always make selected prompt Production if simple mode * fix: production index set and remove unused props * fix(ci): mock initializeRoles * fix: address #3128 * fix: address #3128 * feat: add deletion confirmation dialog * fix: mobile UI issues * style: prompt library UI update * style: focus, logcal tab order * style: Refactor SelectDropDown component to improve code readability and maintainability * chore: bump data-provider * chore: fix labels * refactor: confirm delete prompt version --------- Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
2024-06-20 20:24:32 -04:00
module.exports = router;