1105 commits

Author	SHA1	Message	Date
Marco Beretta	ccd049d8ce	📁 refactor: Prompts UI (#11570) ... * style: enhance prompts UI with new components and improved structure; add CreatePromptButton and AutoSendPrompt; refactor GroupSidePanel and PromptsAccordion * refactor(Prompts): move button components to buttons/ subdirectory * refactor(Prompts): move dialog components to dialogs/ subdirectory * refactor(Prompts): move display components to display/ subdirectory * refactor(Prompts): move editor components to editor/ subdirectory * refactor(Prompts): move field components to fields/ subdirectory * refactor(Prompts): move form components to forms/ subdirectory * refactor(Prompts): move layout components to layouts/ subdirectory * refactor(Prompts): move list components to lists/ subdirectory * refactor(Prompts): move sidebar components to sidebar/ subdirectory * refactor(Prompts): move utility components to utils/ subdirectory * refactor(Prompts): update main exports and external imports * refactor(Prompts): fix class name typo in AutoSendPrompt * refactor(Prompts): reorganize exports and imports order across components * refactor(Prompts): reorder exports for better organization and clarity * refactor(Buttons): enhance prompts accessibility with aria-labels and update translations * refactor(AdminSettings): reorganize imports and improve form structure for clarity * refactor(Dialogs): reorganize imports for consistency and clarity across DeleteVersion, SharePrompt, and VariableDialog components * refactor(Dialogs): enhance prompts accessibility with aria-labels * refactor(Display): enhance prompt components and accessibility features * refactor(.gitignore): add Playwright MCP directory * refactor(Preview): enhance prompt components, improve layout, and add accessibility features * refactor(Prompts): enhance variable handling, improve accessibility, and update UI components * refactor(Prompts): enhance loading state handling and improve accessibility in PromptName component * refactor(Prompts): streamline special variable handling, improve icon management, and enhance UI components * refactor(Prompts): update AdvancedSwitch component to use Radio for mode selection, enhance PromptName with tooltips, and improve layout in PromptForm * refactor(Prompts): enhance VersionCard and VersionBadge components for improved UI and accessibility, update loading state handling in VersionsPanel * refactor(Prompts): improve layout and styling of VersionCard component for better visual alignment and clarity * refactor(DeleteVersion): update text color for confirmation prompt in DeleteConfirmDialog * refactor(Prompts): add configurations for always make production and auto-send prompts, update localization strings for clarity * refactor(Prompts): enhance layout and styling in CategorySelector, CreatePromptForm, and List components for improved responsiveness and clarity * refactor(Prompts): enhance PromptDetailHeader and ChatGroupItem components, add shared prompt indication, and remove unused PromptMetadata component * refactor(Prompts): implement prompt group usage tracking, update sorting logic, and enhance related components * fix(Prompts): security, performance, and pagination fixes - Fix cursor pagination skipping/duplicating items by including numberOfGenerations in cursor condition to match sort order - Close NoSQL injection vector via otherFilters rest spread in GET /all, GET /groups, and buildPromptGroupFilter - Validate groupId as ObjectId before passing to query (GET /) - Add prompt body validation in addPromptToGroup (type + text) - Return 404 instead of 500 for missing group in POST /use - Combine data + count into single $facet aggregation - Add compound index {numberOfGenerations, updatedAt, _id} - Add index on prompt.author for deleteUserPrompts - Update useRecordPromptUsage to refresh client caches - Replace console.error with logger.error * refactor(PromptForm): remove console warning for unselected prompt in VersionsPanel * refactor(Prompts): improve error handling for groupId and streamline usage tracking * refactor(.gitignore): add CLAUDE.md to ignore list * refactor(Prompts): streamline prompt components by removing unused variables and enhancing props structure * refactor(Prompts): fix sort stability, keyboard handling, and remove dead code Add _id tiebreaker to prompt group sort pipelines for deterministic pagination ordering. Prevent default browser scroll on Space key in PromptEditor preview mode. Remove unused blurTimeoutRef and its onMutate callback from DashGroupItem. * refactor(Prompts): enhance groupId validation and improve prompt group aggregation handling * fix: aria-hidden, API fixes, accessibility improvements * fix: ACL author filter, mobile guard, semantic HTML, and add useFocusTrap hook - Remove author filter from patchPromptGroup so ACL-granted editors can update prompt groups (aligns with deletePromptGroupController) - Add missing group guard to mobile HeaderActions in PromptForm - Replace div with article in DashGroupItem, remove redundant stopPropagation and onClick on outer container - Add useFocusTrap hook for keyboard focus management - Add numberOfGenerations to default projection - Deduplicate ObjectId validation, remove console.warn, fix aria-labelledby, localize search announcements * refactor(Prompts): adjust UI and improve a11y * refactor(Prompts): reorder imports for consistency and clarity * refactor(Prompts): implement updateFieldsInPlace for efficient data updates and add related tests * refactor(Prompts): reorder imports to include updateFieldsInPlace for better organization * refactor(Prompts): enhance DashGroupItem with toast notifications for prompt updates and add click-to-edit functionality in PromptEditor * style: use self-closing TooltipAnchor in CreatePromptButton Replace ></TooltipAnchor> with /> for consistency with the rest of the Prompts directory. * fix(i18n): replace placeholder text for com_ui_global_group translation key The value was left as 'something needs to go here. was empty' which would be visible to users as an aria-label in DashGroupItem. * fix(DashGroupItem): sync rename input with group.name on external changes nameInputValue was initialized via useState(group.name) but never synced when group.name changed from a background refetch. Added useEffect that updates the input when the dialog is closed. * perf(useFocusTrap): store onEscape in ref to avoid listener churn onEscape was in the useEffect dependency array, causing the keydown listener to be torn down and re-attached on every render when callers passed an inline function. Now stored in a ref so the effect only re-runs when active or containerRef changes. * fix(a11y): replace role=button div with layered button overlay in ListCard The card used role='button' on a div that contained nested Button elements — an invalid ARIA pattern. Replaced with a hidden button at z-0 for the card action while child interactive elements sit at z-10, eliminating nested interactive element violations. * fix(PromptForm): reset selectionIndex on route change, guard auto-save, and fix a11y - Reset selectionIndex to 0 and isEditing to false when promptId changes, preventing out-of-bounds index when navigating between groups with different version counts. - Track selectedPrompt in a ref so the auto-save effect doesn't fire against a stale prompt when the selection changed mid-edit. - Stabilize useFocusTrap onEscape via useCallback to avoid unnecessary listener re-attachment. - Conditionally render mobile overlay instead of always-present button with aria-hidden/pointer-events toggling. * refactor: extract isValidObjectIdString to shared utility in data-schemas The same regex helper was duplicated in api/server/routes/prompts.js and packages/data-schemas/src/methods/prompt.ts. Moved to packages/data-schemas/src/utils/objectId.ts and imported from both consumers. Also removed a duplicate router.use block introduced during the extraction. * perf(updateFieldsInPlace): replace JSON deep clone with targeted spread Instead of JSON.parse(JSON.stringify(data)) which serializes the entire paginated data structure, use targeted immutable spreads that only copy the affected page and collection array. Returns the original data reference unchanged when the item is not found. * perf(VariablesDropdown): memoize items array and stabilize handleAddVariable The items array containing JSX elements was rebuilt on every render. Wrapped in useMemo keyed on usedVariables and localize. Also wrapped handleAddVariable in useCallback and memoized usedCount to avoid redundant array filtering. * perf(DashGroupItem): stabilize mutation callbacks via refs handleSaveRename and handleDelete had updateGroup/deleteGroup mutation objects in their useCallback dependency arrays. Since mutation objects are new references each render, the callbacks were recreated every render, defeating memoization. Now store mutation objects in refs and call via ref.current in the callbacks. * fix(security): validate groupId in incrementPromptGroupUsage The data-schema method passed the groupId string directly to findByIdAndUpdate without validation. If called from a different entrypoint without the route-level check, Mongoose would throw a CastError. Now validates with isValidObjectIdString before the DB call and throws a clean 'Invalid groupId' error. * fix(security): add rate limiter to prompt usage tracking endpoint POST /groups/:groupId/use had no rate limiting — a user could spam it to inflate numberOfGenerations, which controls sort order for all users. Added promptUsageLimiter (30 req/user/min) following the same pattern as toolCallLimiter. Also handle 'Invalid groupId' error from the data layer in the route error handler. * fix(updateFieldsInPlace): guard against undefined identifier value If updatedItem[identifierField] is null/undefined, findIndex could match unintended items where that field is also undefined. Added early return when the identifier value is nullish. * fix(a11y): use React useId for stable unique IDs in ListCard aria-describedby/id values were derived from prompt name which can contain spaces and special characters, producing invalid HTML IDs and potential collisions. Now uses React.useId() for guaranteed unique, valid IDs per component instance. * fix: Align prompts panel styling with other sidebar panels and fix test - Match FilterPrompts first row to Memory/Bookmark pattern (items-center gap-2) - Remove items-stretch override from PromptsAccordion - Add missing promptUsageLimiter mock to prompts route test * fix: Address code review findings for prompts refactor PR - Fix #5: Gate DeletePrompt in HeaderActions behind canDelete permission - Fix #8: BackToChat navigates to last conversation instead of /c/new - Fix #7: Restore useLiveAnnouncer for screen reader feedback on delete/rename - Fix #1: Use isPublic (set by API) instead of deprecated projectIds for globe icon - Fix #4: Optimistic cache update in useRecordPromptUsage instead of full invalidation - Fix #6: Add migration to drop superseded { createdAt, updatedAt } compound index - Fix #9: Single-pass reduce in PromptVariables instead of triple filter - Fix #10: Rename PromptLabelsForm internal component to avoid collision with PromptForm - Fix #14: Remove redundant aria-label from aria-hidden Checkbox in AutoSendPrompt * fix: Align prompts panel filter row element sizes with other panels - Override Dropdown trigger to size-9 (36px) to match FilterInput height - Set CreatePromptButton to size-9 shrink-0 bg-transparent matching Memory/Bookmark panel button pattern * fix(prompts): Shared Prompts filter ignores direct shares, only returns PUBLIC Folds fix from PR #11882 into the refactored codebase. Bug A: filterAccessibleIdsBySharedLogic now accepts ownedPromptGroupIds: - MY_PROMPTS: accessible intersect owned - SHARED_PROMPTS: (accessible union public) minus owned - ALL: accessible union public (deduplicated) Legacy fallback preserved when ownedPromptGroupIds is omitted. Bug B: getPromptGroup uses $lookup aggregation to populate productionPrompt, fixing empty text on direct URL navigation to shared prompts. Also adds getOwnedPromptGroupIds to data-schemas methods and passes it from both /all and /groups route handlers. * fix: Add missing canDelete to mobile HeaderActions, remove dead instanceProjectId prop - Pass canDelete to mobile HeaderActions row (was only on desktop) - Remove instanceProjectId prop from ChatGroupItem and DashGroupItem since global check now uses group.isPublic - Remove useGetStartupConfig from List.tsx (no longer needed) * fix: Use runtime ObjectId instead of type-only Types.ObjectId, fix i18next interpolation - getPromptGroup and getOwnedPromptGroupIds were using Types.ObjectId (imported as type-only), which is erased at compile time. Use the runtime ObjectId from mongoose.Types (already destructured at line 20). This fixes the 404s in PATCH /groups/:groupId tests. - Fix com_ui_prompt_deleted_group translation to use {{0}} (i18next double-brace syntax) instead of {0}. * chore: Fix translation key ordering, add sideEffects: false to data-provider - Reorder new translation keys to maintain alphabetical order: com_ui_click_to_edit, com_ui_labels, com_ui_live, com_ui_prompt_delete_confirm, com_ui_prompt_deleted_group, com_ui_prompt_details, com_ui_prompt_renamed, com_ui_prompt_update_error, com_ui_prompt_variables_list - Add "sideEffects": false to librechat-data-provider package.json to enable tree-shaking of unused exports (types, constants, pure functions) * fix: Reduce prompts panel spacing, align memory toggle with checkbox pattern - Remove unnecessary wrapper div around AutoSendPrompt in PromptsAccordion, reducing vertical space between the toggle and the first prompt item - Replace Memory panel's Switch toggle with Checkbox+Button pattern matching the prompts panel's AutoSendPrompt for visual consistency * fix: Reduce gap between AutoSendPrompt and first prompt item Change ChatGroupItem margin from my-2 to mb-2 to eliminate the doubled spacing (gap-2 from parent + top margin from first item). Restore wrapper div around AutoSendPrompt for right-alignment. * fix: Restore prompt name on empty save, remove dead bodyProps from checkGlobalPromptShare - PromptName: reset newName to name when save is cancelled due to empty or unchanged input, preventing blank title in read mode - checkGlobalPromptShare: remove dead bodyProps config — Permissions.SHARE was not in the permissions array so the bodyProps rule was never evaluated. Per-resource share checks are handled by canAccessPromptGroupResource. --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-03-22 16:56:22 -04:00
Danny Avila	04e65bb21a	🧹 chore: Move direct model usage from PermissionsController to data-schemas	2026-03-21 15:20:15 -04:00
Danny Avila	87a3b8221a	🧹 chore: Consolidate getSoleOwnedResourceIds into data-schemas and use db object in PermissionService ... Move getSoleOwnedResourceIds from PermissionService to data-schemas aclEntry methods, update PermissionService to use the db object pattern instead of destructured imports from ~/models, and update UserController + tests accordingly.	2026-03-21 14:28:57 -04:00
Danny Avila	7829fa9eca	🪄 refactor: Simplify MCP Tool Content Formatting to Unified String Output (#12352) ... * refactor: Simplify content formatting in MCP service and parser - Consolidated content handling in `formatToolContent` to return a plain-text string instead of an array for all providers, enhancing clarity and consistency. - Removed unnecessary checks for content array providers, streamlining the logic for handling text and image artifacts. - Updated related tests to reflect changes in expected output format, ensuring comprehensive coverage for the new implementation. * fix: Return empty string for image-only tool responses instead of '(No response)' When artifacts exist (images/UI resources) but no text content is present, return an empty string rather than the misleading '(No response)' fallback. Adds missing test assertions for image-only content and standardizes length checks to explicit `> 0` comparisons.	2026-03-21 14:28:56 -04:00
Danny Avila	b5c097e5c7	⚗️ feat: Agent Context Compaction/Summarization (#12287) ... * chore: imports/types Add summarization config and package-level summarize handler contracts Register summarize handlers across server controller paths Port cursor dual-read/dual-write summary support and UI status handling Selectively merge cursor branch files for BaseClient summary content block detection (last-summary-wins), dual-write persistence, summary block unit tests, and on_summarize_status SSE event handling with started/completed/failed branches. Co-authored-by: Cursor <cursoragent@cursor.com> refactor: type safety feat: add localization for summarization status messages refactor: optimize summary block detection in BaseClient Updated the logic for identifying existing summary content blocks to use a reverse loop for improved efficiency. Added a new test case to ensure the last summary content block is updated correctly when multiple summary blocks exist. chore: add runName to chainOptions in AgentClient refactor: streamline summarization configuration and handler integration Removed the deprecated summarizeNotConfigured function and replaced it with a more flexible createSummarizeFn. Updated the summarization handler setup across various controllers to utilize the new function, enhancing error handling and configuration resolution. Improved overall code clarity and maintainability by consolidating summarization logic. feat(summarization): add staged chunk-and-merge fallback feat(usage): track summarization usage separately from messages feat(summarization): resolve prompt from config in runtime fix(endpoints): use @librechat/api provider config loader refactor(agents): import getProviderConfig from @librechat/api chore: code order feat(app-config): auto-enable summarization when configured feat: summarization config refactor(summarization): streamline persist summary handling and enhance configuration validation Removed the deprecated createDeferredPersistSummary function and integrated a new createPersistSummary function for MongoDB persistence. Updated summarization handlers across various controllers to utilize the new persistence method. Enhanced validation for summarization configuration to ensure provider, model, and prompt are properly set, improving error handling and overall robustness. refactor(summarization): update event handling and remove legacy summarize handlers Replaced the deprecated summarization handlers with new event-driven handlers for summarization start and completion across multiple controllers. This change enhances the clarity of the summarization process and improves the integration of summarization events in the application. Additionally, removed unused summarization functions and streamlined the configuration loading process. refactor(summarization): standardize event names in handlers Updated event names in the summarization handlers to use constants from GraphEvents for consistency and clarity. This change improves maintainability and reduces the risk of errors related to string literals in event handling. feat(summarization): enhance usage tracking for summarization events Added logic to track summarization usage in multiple controllers by checking the current node type. If the node indicates a summarization task, the usage type is set accordingly. This change improves the granularity of usage data collected during summarization processes. feat(summarization): integrate SummarizationConfig into AppSummarizationConfig type Enhanced the AppSummarizationConfig type by extending it with the SummarizationConfig type from librechat-data-provider. This change improves type safety and consistency in the summarization configuration structure. test: add end-to-end tests for summarization functionality Introduced a comprehensive suite of end-to-end tests for the summarization feature, covering the full LibreChat pipeline from message creation to summarization. This includes a new setup file for environment configuration and a Jest configuration specifically for E2E tests. The tests utilize real API keys and ensure proper integration with the summarization process, enhancing overall test coverage and reliability. refactor(summarization): include initial summary in formatAgentMessages output Updated the formatAgentMessages function to return an initial summary alongside messages and index token count map. This change is reflected in multiple controllers and the corresponding tests, enhancing the summarization process by providing additional context for each agent's response. refactor: move hydrateMissingIndexTokenCounts to tokenMap utility Extracted the hydrateMissingIndexTokenCounts function from the AgentClient and related tests into a new tokenMap utility file. This change improves code organization and reusability, allowing for better management of token counting logic across the application. refactor(summarization): standardize step event handling and improve summary rendering Refactored the step event handling in the useStepHandler and related components to utilize constants for event names, enhancing consistency and maintainability. Additionally, improved the rendering logic in the Summary component to conditionally display the summary text based on its availability, providing a better user experience during the summarization process. feat(summarization): introduce baseContextTokens and reserveTokensRatio for improved context management Added baseContextTokens to the InitializedAgent type to calculate the context budget based on agentMaxContextNum and maxOutputTokensNum. Implemented reserveTokensRatio in the createRun function to allow configurable context token management. Updated related tests to validate these changes and ensure proper functionality. feat(summarization): add minReserveTokens, context pruning, and overflow recovery configurations Introduced new configuration options for summarization, including minReserveTokens, context pruning settings, and overflow recovery parameters. Updated the createRun function to accommodate these new options and added a comprehensive test suite to validate their functionality and integration within the summarization process. feat(summarization): add updatePrompt and reserveTokensRatio to summarization configuration Introduced an updatePrompt field for updating existing summaries with new messages, enhancing the flexibility of the summarization process. Additionally, added reserveTokensRatio to the configuration schema, allowing for improved management of token allocation during summarization. Updated related tests to validate these new features. feat(logging): add on_agent_log event handler for structured logging Implemented an on_agent_log event handler in both the agents' callbacks and responses to facilitate structured logging of agent activities. This enhancement allows for better tracking and debugging of agent interactions by logging messages with associated metadata. Updated the summarization process to ensure proper handling of log events. fix: remove duplicate IBalanceUpdate interface declaration perf(usage): single-pass partition of collectedUsage Replace two Array.filter() passes with a single for-of loop that partitions message vs. summarization usages in one iteration. fix(BaseClient): shallow-copy message content before mutating and preserve string content Avoid mutating the original message.content array in-place when appending a summary block. Also convert string content to a text content part instead of silently discarding it. fix(ui): fix Part.tsx indentation and useStepHandler summarize-complete handling - Fix SUMMARY else-if branch indentation in Part.tsx to match chain level - Guard ON_SUMMARIZE_COMPLETE with didFinalize flag to avoid unnecessary re-renders when no summarizing parts exist - Protect against undefined completeData.summary instead of unsafe spread fix(agents): use strict enabled check for summarization handlers Change summarizationConfig?.enabled !== false to === true so handlers are not registered when summarizationConfig is undefined. chore: fix initializeClient JSDoc and move DEFAULT_RESERVE_RATIO to module scope refactor(Summary): align collapse/expand behavior with Reasoning component - Single render path instead of separate streaming vs completed branches - Use useMessageContext for isSubmitting/isLatestMessage awareness so the "Summarizing..." label only shows during active streaming - Default to collapsed (matching Reasoning), user toggles to expand - Add proper aria attributes (aria-hidden, role, aria-controls, contentId) - Hide copy button while actively streaming feat(summarization): default to self-summarize using agent's own provider/model When no summarization config is provided (neither in librechat.yaml nor on the agent), automatically enable summarization using the agent's own provider and model. The agents package already provides default prompts, so no prompt configuration is needed. Also removes the dead resolveSummarizationLLMConfig in summarize.ts (and its spec) — run.ts buildAgentContext is the single source of truth for summarization config resolution. Removes the duplicate RuntimeSummarizationConfig local type in favor of the canonical SummarizationConfig from data-provider. chore: schema and type cleanup for summarization - Add trigger field to summarizationAgentOverrideSchema so per-agent trigger overrides in librechat.yaml are not silently stripped by Zod - Remove unused SummarizationStatus type from runs.ts - Make AppSummarizationConfig.enabled non-optional to reflect the invariant that loadSummarizationConfig always sets it refactor(responses): extract duplicated on_agent_log handler refactor(run): use agents package types for summarization config Import SummarizationConfig, ContextPruningConfig, and OverflowRecoveryConfig from @librechat/agents and use them to type-check the translation layer in buildAgentContext. This ensures the config object passed to the agent graph matches what it expects. - Use `satisfies AgentSummarizationConfig` on the config object - Cast contextPruningConfig and overflowRecoveryConfig to agents types - Properly narrow trigger fields from DeepPartial to required shape feat(config): add maxToolResultChars to base endpoint schema Add maxToolResultChars to baseEndpointSchema so it can be configured on any endpoint in librechat.yaml. Resolved during agent initialization using getProviderConfig's endpoint resolution: custom endpoint config takes precedence, then the provider-specific endpoint config, then the shared `all` config. Passed through to the agents package ToolNode, which uses it to cap tool result length before it enters the context window. When not configured, the agents package computes a sensible default from maxContextTokens. fix(summarization): forward agent model_parameters in self-summarize default When no explicit summarization config exists, the self-summarize default now forwards the agent's model_parameters as the summarization parameters. This ensures provider-specific settings (e.g. Bedrock region, credentials, endpoint host) are available when the agents package constructs the summarization LLM. fix(agents): register summarization handlers by default Change the enabled gate from === true to !== false so handlers register when no explicit summarization config exists. This aligns with the self-summarize default where summarization is always on unless explicitly disabled via enabled: false. refactor(summarization): let agents package inherit clientOptions for self-summarize Remove model_parameters forwarding from the self-summarize default. The agents package now reuses the agent's own clientOptions when the summarization provider matches the agent's provider, inheriting all provider-specific settings (region, credentials, proxy, etc.) automatically. refactor(summarization): use MessageContentComplex[] for summary content Unify summary content to always use MessageContentComplex[] arrays, matching the pattern used by on_message_delta. No more string | array unions — content is always an array of typed blocks ({ type: 'text', text: '...' } for text, { type: 'reasoning_content', ... } for reasoning). Agents package: - SummaryContentBlock.content: MessageContentComplex[] (was string) - tokenCount now optional (not sent on deltas) - Removed reasoning field — reasoning is now a content block type - streamAndCollect normalizes all chunks to content block arrays - Delta events pass content blocks directly LibreChat: - SummaryContentPart.content: Agents.MessageContentComplex[] - Updated Part.tsx, Summary.tsx, useStepHandler.ts, BaseClient.js - Summary.tsx derives display text from content blocks via useMemo - Aggregator uses simple array spread refactor(summarization): enhance summary handling and text extraction - Updated BaseClient.js to improve summary text extraction, accommodating both legacy and new content formats. - Modified summarization logic to ensure consistent handling of summary content across different message formats. - Adjusted test cases in summarization.e2e.spec.js to utilize the new summary text extraction method. - Refined SSE useStepHandler to initialize summary content as an array. - Updated configuration schema by removing unused minReserveTokens field. - Cleaned up SummaryContentPart type by removing rangeHash property. These changes streamline the summarization process and ensure compatibility with various content structures. refactor(summarization): streamline usage tracking and logging - Removed direct checks for summarization nodes in ModelEndHandler and replaced them with a dedicated markSummarizationUsage function for better readability and maintainability. - Updated OpenAIChatCompletionController and responses handlers to utilize the new markSummarizationUsage function for setting usage types. - Enhanced logging functionality by ensuring the logger correctly handles different log levels. - Introduced a new useCopyToClipboard hook in the Summary component to encapsulate clipboard copy logic, improving code reusability and clarity. These changes improve the overall structure and efficiency of the summarization handling and logging processes. refactor(summarization): update summary content block documentation - Removed outdated comment regarding the last summary content block in BaseClient.js. - Added a new comment to clarify the purpose of the findSummaryContentBlock method, ensuring consistency in documentation. These changes enhance code clarity and maintainability by providing accurate descriptions of the summarization logic. refactor(summarization): update summary content structure in tests - Modified the summarization content structure in e2e tests to use an array format for text, aligning with recent changes in summary handling. - Updated test descriptions to clarify the behavior of context token calculations, ensuring consistency and clarity in the tests. These changes enhance the accuracy and maintainability of the summarization tests by reflecting the updated content structure. refactor(summarization): remove legacy E2E test setup and configuration - Deleted the e2e-setup.js and jest.e2e.config.js files, which contained legacy configurations for E2E tests using real API keys. - Introduced a new summarization.e2e.ts file that implements comprehensive E2E backend integration tests for the summarization process, utilizing real AI providers and tracking summaries throughout the run. These changes streamline the testing framework by consolidating E2E tests into a single, more robust file while removing outdated configurations. refactor(summarization): enhance E2E tests and error handling - Added a cleanup step to force exit after all tests to manage Redis connections. - Updated the summarization model to 'claude-haiku-4-5-20251001' for consistency across tests. - Improved error handling in the processStream function to capture and return processing errors. - Enhanced logging for cross-run tests and tight context scenarios to provide better insights into test execution. These changes improve the reliability and clarity of the E2E tests for the summarization process. refactor(summarization): enhance test coverage for maxContextTokens behavior - Updated run-summarization.test.ts to include a new test case ensuring that maxContextTokens does not exceed user-defined limits, even when calculated ratios suggest otherwise. - Modified summarization.e2e.ts to replace legacy UsageMetadata type with a more appropriate type for collectedUsage, improving type safety and clarity in the test setup. These changes improve the robustness of the summarization tests by validating context token constraints and refining type definitions. feat(summarization): add comprehensive E2E tests for summarization process - Introduced a new summarization.e2e.test.ts file that implements extensive end-to-end integration tests for the summarization pipeline, covering the full flow from LibreChat to agents. - The tests utilize real AI providers and include functionality to track summaries during and between runs. - Added necessary cleanup steps to manage Redis connections post-tests and ensure proper exit. These changes enhance the testing framework by providing robust coverage for the summarization process, ensuring reliability and performance under real-world conditions. fix(service): import logger from winston configuration - Removed the import statement for logger from '@librechat/data-schemas' and replaced it with an import from '~/config/winston'. - This change ensures that the logger is correctly sourced from the updated configuration, improving consistency in logging practices across the application. refactor(summary): simplify Summary component and enhance token display - Removed the unused `meta` prop from the `SummaryButton` component to streamline its interface. - Updated the token display logic to use a localized string for better internationalization support. - Adjusted the rendering of the `meta` information to improve its visibility within the `Summary` component. These changes enhance the clarity and usability of the Summary component while ensuring better localization practices. feat(summarization): add maxInputTokens configuration for summarization - Introduced a new `maxInputTokens` property in the summarization configuration schema to control the amount of conversation context sent to the summarizer, with a default value of 10000. - Updated the `createRun` function to utilize the new `maxInputTokens` setting, allowing for more flexible summarization based on agent context. These changes enhance the summarization capabilities by providing better control over input token limits, improving the overall summarization process. refactor(summarization): simplify maxInputTokens logic in createRun function - Updated the logic for the `maxInputTokens` property in the `createRun` function to directly use the agent's base context tokens when the resolved summarization configuration does not specify a value. - This change streamlines the configuration process and enhances clarity in how input token limits are determined for summarization. These modifications improve the maintainability of the summarization configuration by reducing complexity in the token calculation logic. feat(summary): enhance Summary component to display meta information - Updated the SummaryContent component to accept an optional `meta` prop, allowing for additional contextual information to be displayed above the main content. - Adjusted the rendering logic in the Summary component to utilize the new `meta` prop, improving the visibility of supplementary details. These changes enhance the user experience by providing more context within the Summary component, making it clearer and more informative. refactor(summarization): standardize reserveRatio configuration in summarization logic - Replaced instances of `reserveTokensRatio` with `reserveRatio` in the `createRun` function and related tests to unify the terminology across the codebase. - Updated the summarization configuration schema to reflect this change, ensuring consistency in how the reserve ratio is defined and utilized. - Removed the per-agent override logic for summarization configuration, simplifying the overall structure and enhancing clarity. These modifications improve the maintainability and readability of the summarization logic by standardizing the configuration parameters. * fix: circular dependency of `~/models` * chore: update logging scope in agent log handlers Changed log scope from `[agentus:${data.scope}]` to `[agents:${data.scope}]` in both the callbacks and responses controllers to ensure consistent logging format across the application. * feat: calibration ratio * refactor(tests): update summarizationConfig tests to reflect changes in enabled property Modified tests to check for the new `summarizationEnabled` property instead of the deprecated `enabled` field in the summarization configuration. This change ensures that the tests accurately validate the current configuration structure and behavior of the agents. * feat(tests): add markSummarizationUsage mock for improved test coverage Introduced a mock for the markSummarizationUsage function in the responses unit tests to enhance the testing of summarization usage tracking. This addition supports better validation of summarization-related functionalities and ensures comprehensive test coverage for the agents' response handling. * refactor(tests): simplify event handler setup in createResponse tests Removed redundant mock implementations for event handlers in the createResponse unit tests, streamlining the setup process. This change enhances test clarity and maintainability while ensuring that the tests continue to validate the correct behavior of usage tracking during on_chat_model_end events. * refactor(agents): move calibration ratio capture to finally block Reorganized the logic for capturing the calibration ratio in the AgentClient class to ensure it is executed in the finally block. This change guarantees that the ratio is captured even if the run is aborted, enhancing the reliability of the response message persistence. Removed redundant code and improved clarity in the handling of context metadata. * refactor(agents): streamline bulk write logic in recordCollectedUsage function Removed redundant bulk write operations and consolidated document handling in the recordCollectedUsage function. The logic now combines all documents into a single bulk write operation, improving efficiency and reducing error handling complexity. Updated logging to provide consistent error messages for bulk write failures. * refactor(agents): enhance summarization configuration resolution in createRun function Streamlined the summarization configuration logic by introducing a base configuration and allowing for overrides from agent-specific settings. This change improves clarity and maintainability, ensuring that the summarization configuration is consistently applied while retaining flexibility for customization. Updated the handling of summarization parameters to ensure proper integration with the agent's model and provider settings. * refactor(agents): remove unused tokenCountMap and streamline calibration ratio handling Eliminated the unused tokenCountMap variable from the AgentClient class to enhance code clarity. Additionally, streamlined the logic for capturing the calibration ratio by using optional chaining and a fallback value, ensuring that context metadata is consistently defined. This change improves maintainability and reduces potential confusion in the codebase. * refactor(agents): extract agent log handler for improved clarity and reusability Refactored the agent log handling logic by extracting it into a dedicated function, `agentLogHandler`, enhancing code clarity and reusability across different modules. Updated the event handlers in both the OpenAI and responses controllers to utilize the new handler, ensuring consistent logging behavior throughout the application. * test: add summarization event tests for useStepHandler Implemented a series of tests for the summarization events in the useStepHandler hook. The tests cover scenarios for ON_SUMMARIZE_START, ON_SUMMARIZE_DELTA, and ON_SUMMARIZE_COMPLETE events, ensuring proper handling of summarization logic, including message accumulation and finalization. This addition enhances test coverage and validates the correct behavior of the summarization process within the application. * refactor(config): update summarizationTriggerSchema to use enum for type validation Changed the type of the `type` field in the summarizationTriggerSchema from a string to an enum with a single value 'token_count'. This modification enhances type safety and ensures that only valid types are accepted in the configuration, improving overall clarity and maintainability of the schema. * test(usage): add bulk write tests for message and summarization usage Implemented tests for the bulk write functionality in the recordCollectedUsage function, covering scenarios for combined message and summarization usage, summarization-only usage, and message-only usage. These tests ensure correct document handling and token rollup calculations, enhancing test coverage and validating the behavior of the usage tracking logic. * refactor(Chat): enhance clipboard copy functionality and type definitions in Summary component Updated the Summary component to improve the clipboard copy functionality by handling clipboard permission errors. Refactored type definitions for SummaryProps to use a more specific type, enhancing type safety. Adjusted the SummaryButton and FloatingSummaryBar components to accept isCopied and onCopy props, promoting better separation of concerns and reusability. * chore(translations): remove unused "Expand Summary" key from English translations Deleted the "Expand Summary" key from the English translation file to streamline the localization resources and improve clarity in the user interface. This change helps maintain an organized and efficient translation structure. * refactor: adjust token counting for Claude model to account for API discrepancies Implemented a correction factor for token counting when using the Claude model, addressing discrepancies between Anthropic's API and local tokenizer results. This change ensures accurate token counts by applying a scaling factor, improving the reliability of token-related functionalities. * refactor(agents): implement token count adjustment for Claude model messages Added a method to adjust token counts for messages processed by the Claude model, applying a correction factor to align with API expectations. This enhancement improves the accuracy of token counting, ensuring reliable functionality when interacting with the Claude model. * refactor(agents): token counting for media content in messages Introduced a new method to estimate token costs for image and document blocks in messages, improving the accuracy of token counting. This enhancement ensures that media content is properly accounted for, particularly for the Claude model, by integrating additional token estimation logic for various content types. Updated the token counting function to utilize this new method, enhancing overall reliability and functionality. * chore: fix missing import * fix(agents): clamp baseContextTokens and document reserve ratio change Prevent negative baseContextTokens when maxOutputTokens exceeds the context window (misconfigured models). Document the 10%→5% default reserve ratio reduction introduced alongside summarization. * fix(agents): include media tokens in hydrated token counts Add estimateMediaTokensForMessage to createTokenCounter so the hydration path (used by hydrateMissingIndexTokenCounts) matches the precomputed path in AgentClient.getTokenCountForMessage. Without this, messages containing images or documents were systematically undercounted during hydration, risking context window overflow. Add 34 unit tests covering all block-type branches of estimateMediaTokensForMessage. * fix(agents): include summarization output tokens in usage return value The returned output_tokens from recordCollectedUsage now reflects all billed LLM calls (message + summarization). Previously, summarization completions were billed but excluded from the returned metadata, causing a discrepancy between what users were charged and what the response message reported. * fix(tests): replace process.exit with proper Redis cleanup in e2e test The summarization E2E test used process.exit(0) to work around a Redis connection opened at import time, which killed the Jest runner and bypassed teardown. Use ioredisClient.quit() and keyvRedisClient.disconnect() for graceful cleanup instead. * fix(tests): update getConvo imports in OpenAI and response tests Refactor test files to import getConvo from the main models module instead of the Conversation submodule. This change ensures consistency across tests and simplifies the import structure, enhancing maintainability. * fix(clients): improve summary text validation in BaseClient Refactor the summary extraction logic to ensure that only non-empty summary texts are considered valid. This change enhances the robustness of the message processing by utilizing a dedicated method for summary text retrieval, improving overall reliability. * fix(config): replace z.any() with explicit union in summarization schema Model parameters (temperature, top_p, etc.) are constrained to primitive types rather than the policy-violating z.any(). * refactor(agents): deduplicate CLAUDE_TOKEN_CORRECTION constant Export from the TS source in packages/api and import in the JS client, eliminating the static class property that could drift out of sync. * refactor(agents): eliminate duplicate selfProvider in buildAgentContext selfProvider and provider were derived from the same expression with different type casts. Consolidated to a single provider variable. * refactor(agents): extract shared SSE handlers and restrict log levels - buildSummarizationHandlers() factory replaces triplicated handler blocks across responses.js and openai.js - agentLogHandlerObj exported from callbacks.js for consistent reuse - agentLogHandler restricted to an allowlist of safe log levels (debug, info, warn, error) instead of accepting arbitrary strings * fix(SSE): batch summarize deltas, add exhaustiveness check, conditional error announcement - ON_SUMMARIZE_DELTA coalesces rapid-fire renders via requestAnimationFrame instead of calling setMessages per chunk - Exhaustive never-check on TStepEvent catches unhandled variants at compile time when new StepEvents are added - ON_SUMMARIZE_COMPLETE error announcement only fires when a summary part was actually present and removed * feat(agents): persist instruction overhead in contextMeta and seed across runs Extend contextMeta with instructionOverhead and toolCount so the provider-observed instruction overhead is persisted on the response message and seeded into the pruner on subsequent runs. This enables the pruner to use a calibrated budget from the first call instead of waiting for a provider observation, preventing the ratio collapse caused by local tokenizer overestimating tool schema tokens. The seeded overhead is only used when encoding and tool count match between runs, ensuring stale values from different configurations are discarded. * test(agents): enhance OpenAI test mocks for summarization handlers Updated the OpenAI test suite to include additional mock implementations for summarization handlers, including buildSummarizationHandlers, markSummarizationUsage, and agentLogHandlerObj. This improves test coverage and ensures consistent behavior during testing. * fix(agents): address review findings for summarization v2 Cancel rAF on unmount to prevent stale Recoil writes from dead component context. Clear orphaned summarizing:true parts when ON_SUMMARIZE_COMPLETE arrives without a summary payload. Add null guard and safe spread to agentLogHandler. Handle Anthropic-format base64 image/* documents in estimateMediaTokensForMessage. Use role="region" for expandable summary content. Add .describe() to contextMeta Zod fields. Extract duplicate usage loop into helper. * refactor: simplify contextMeta to calibrationRatio + encoding only Remove instructionOverhead and toolCount from cross-run persistence — instruction tokens change too frequently between runs (prompt edits, tool changes) for a persisted seed to be reliable. The intra-run calibration in the pruner still self-corrects via provider observations. contextMeta now stores only the tokenizer-bias ratio and encoding, which are stable across instruction changes. * test(SSE): enhance useStepHandler tests for ON_SUMMARIZE_COMPLETE behavior Updated the test for ON_SUMMARIZE_COMPLETE to clarify that it finalizes the existing part with summarizing set to false when the summary is undefined. Added assertions to verify the correct behavior of message updates and the state of summary parts. * refactor(BaseClient): remove handleContextStrategy and truncateToolCallOutputs functions Eliminated the handleContextStrategy method from BaseClient to streamline message handling. Also removed the truncateToolCallOutputs function from the prompts module, simplifying the codebase and improving maintainability. * refactor: add AGENT_DEBUG_LOGGING option and refactor token count handling in BaseClient Introduced AGENT_DEBUG_LOGGING to .env.example for enhanced debugging capabilities. Refactored token count handling in BaseClient by removing the handleTokenCountMap method and simplifying token count updates. Updated AgentClient to log detailed token count recalculations and adjustments, improving traceability during message processing. * chore: update dependencies in package-lock.json and package.json files Bumped versions of several dependencies, including @librechat/agents to ^3.1.62 and various AWS SDK packages to their latest versions. This ensures compatibility and incorporates the latest features and fixes. * chore: imports order * refactor: extract summarization config resolution from buildAgentContext * refactor: rename and simplify summarization configuration shaping function * refactor: replace AgentClient token counting methods with single-pass pure utility Extract getTokenCount() and getTokenCountForMessage() from AgentClient into countFormattedMessageTokens(), a pure function in packages/api that handles text, tool_call, image, and document content types in one loop. - Decompose estimateMediaTokensForMessage into block-level helpers (estimateImageDataTokens, estimateImageBlockTokens, estimateDocumentBlockTokens) shared by both estimateMediaTokensForMessage and the new single-pass function - Remove redundant per-call getEncoding() resolution (closure captures once) - Remove deprecated gpt-3.5-turbo-0301 model branching - Drop this.getTokenCount guard from BaseClient.sendMessage * refactor: streamline token counting in createTokenCounter function Simplified the createTokenCounter function by removing the media token estimation and directly calculating the token count. This change enhances clarity and performance by consolidating the token counting logic into a single pass, while maintaining compatibility with Claude's token correction. * refactor: simplify summarization configuration types Removed the AppSummarizationConfig type and directly used SummarizationConfig in the AppConfig interface. This change streamlines the type definitions and enhances consistency across the codebase. * chore: import order * fix: summarization event handling in useStepHandler - Cancel pending summarizeDeltaRaf in clearStepMaps to prevent stale frames firing after map reset or component unmount - Move announcePolite('summarize_completed') inside the didFinalize guard so screen readers only announce when finalization actually occurs - Remove dead cleanup closure returned from stepHandler useCallback body that was never invoked by any caller * fix: estimate tokens for non-PDF/non-image base64 document blocks Previously estimateDocumentBlockTokens returned 0 for unrecognized MIME types (e.g. text/plain, application/json), silently underestimating context budget. Fall back to character-based heuristic or countTokens. * refactor: return cloned usage from markSummarizationUsage Avoid mutating LangChain's internal usage_metadata object by returning a shallow clone with the usage_type tag. Update all call sites in callbacks, openai, and responses controllers to use the returned value. * refactor: consolidate debug logging loops in buildMessages Merge the two sequential O(n) debug-logging passes over orderedMessages into a single pass inside the map callback where all data is available. * refactor: narrow SummaryContentPart.content type Replace broad Agents.MessageContentComplex[] with the specific Array<{ type: ContentTypes.TEXT; text: string }> that all producers and consumers already use, improving compile-time safety. * refactor: use single output array in recordCollectedUsage Have processUsageGroup append to a shared array instead of returning separate arrays that are spread into a third, reducing allocations. * refactor: use for...in in hydrateMissingIndexTokenCounts Replace Object.entries with for...in to avoid allocating an intermediate tuple array during token map hydration.	2026-03-21 14:28:56 -04:00
Danny Avila	67db0c1cb3	🗑️ chore: Remove Action Test Suite and Update Mock Implementations (#12268) ... - Deleted the Action test suite located in `api/models/Action.spec.js` to streamline the codebase. - Updated various test files to reflect changes in model mocks, consolidating mock implementations for user-related actions and enhancing clarity. - Improved consistency in test setups by aligning with the latest model updates and removing redundant mock definitions.	2026-03-21 14:28:55 -04:00
Danny Avila	dd72b7b17e	🔄 chore: Consolidate agent model imports across middleware and tests from rebase ... - Updated imports for `createAgent` and `getAgent` to streamline access from a unified `~/models` path. - Enhanced test files to reflect the new import structure, ensuring consistency and maintainability across the codebase. - Improved clarity by removing redundant imports and aligning with the latest model updates.	2026-03-21 14:28:55 -04:00
Atef Bellaaj	a0fed6173c	🗂️ refactor: Migrate S3 Storage to TypeScript in packages/api (#11947) ... * Migrate S3 storage module with unit and integration tests - Migrate S3 CRUD and image operations to packages/api/src/storage/s3/ - Add S3ImageService class with dependency injection - Add unit tests using aws-sdk-client-mock - Add integration tests with real s3 bucket (condition presence of AWS_TEST_BUCKET_NAME) * AI Review Findings Fixes * chore: tests and refactor S3 storage types - Added mock implementations for the 'sharp' library in various test files to improve image processing testing. - Updated type references in S3 storage files from MongoFile to TFile for consistency and type safety. - Refactored S3 CRUD operations to ensure proper handling of file types and improve code clarity. - Enhanced integration tests to validate S3 file operations and error handling more effectively. * chore: rename test file * Remove duplicate import of refreshS3Url * chore: imports order * fix: remove duplicate imports for S3 URL handling in UserController * fix: remove duplicate import of refreshS3FileUrls in files.js * test: Add mock implementations for 'sharp' and '@librechat/api' in UserController tests - Introduced mock functions for the 'sharp' library to facilitate image processing tests, including metadata retrieval and buffer conversion. - Enhanced mocking for '@librechat/api' to ensure consistent behavior in tests, particularly for the needsRefresh and getNewS3URL functions. --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-03-21 14:28:55 -04:00
Danny Avila	9e0592a236	📜 feat: Implement System Grants for Capability-Based Authorization (#11896) ... * feat: Implement System Grants for Role-Based Capabilities - Added a new `systemGrant` model and associated methods to manage role-based capabilities within the application. - Introduced middleware functions `hasCapability` and `requireCapability` to check user permissions based on their roles. - Updated the database seeding process to include system grants for the ADMIN role, ensuring all necessary capabilities are assigned on startup. - Enhanced type definitions and schemas to support the new system grant functionality, improving overall type safety and clarity in the codebase. * test: Add unit tests for capabilities middleware and system grant methods - Introduced comprehensive unit tests for the capabilities middleware, including `hasCapability` and `requireCapability`, ensuring proper permission checks based on user roles. - Added tests for the `SystemGrant` methods, verifying the seeding of system grants, capability granting, and revocation processes. - Enhanced test coverage for edge cases, including idempotency of grant operations and handling of unexpected errors in middleware. - Utilized mocks for database interactions to isolate tests and improve reliability. * refactor: Transition to Capability-Based Access Control - Replaced role-based access checks with capability-based checks across various middleware and routes, enhancing permission management. - Introduced `hasCapability` and `requireCapability` functions to streamline capability verification for user actions. - Updated relevant routes and middleware to utilize the new capability system, ensuring consistent permission enforcement. - Enhanced type definitions and added tests for the new capability functions, improving overall code reliability and maintainability. * test: Enhance capability-based access tests for ADMIN role - Updated tests to reflect the new capability-based access control, specifically for the ADMIN role. - Modified test descriptions to clarify that users with the MANAGE_AGENTS capability can bypass permission checks. - Seeded capabilities for the ADMIN role in multiple test files to ensure consistent permission checks across different routes and middleware. - Improved overall test coverage for capability verification, ensuring robust permission management. * test: Update capability tests for MCP server access - Renamed test to reflect the correct capability for bypassing permission checks, changing from MANAGE_AGENTS to MANAGE_MCP_SERVERS. - Updated seeding of capabilities for the ADMIN role to align with the new capability structure. - Ensured consistency in capability definitions across tests and middleware for improved permission management. * feat: Add hasConfigCapability for enhanced config access control - Introduced `hasConfigCapability` function to check user permissions for managing or reading specific config sections. - Updated middleware to export the new capability function, ensuring consistent access control across the application. - Enhanced unit tests to cover various scenarios for the new capability, improving overall test coverage and reliability. * fix: Update tenantId filter in createSystemGrantMethods - Added a condition to set tenantId filter to { $exists: false } when tenantId is null, ensuring proper handling of cases where tenantId is not provided. - This change improves the robustness of the system grant methods by explicitly managing the absence of tenantId in the filter logic. * fix: account deletion capability check - Updated the `canDeleteAccount` middleware to ensure that the `hasManageUsers` capability check only occurs if a user is present, preventing potential errors when the user object is undefined. - This change improves the robustness of the account deletion logic by ensuring proper handling of user permissions. * refactor: Optimize seeding of system grants for ADMIN role - Replaced sequential capability granting with parallel execution using Promise.all in the seedSystemGrants function. - This change improves performance and efficiency during the initialization of system grants, ensuring all capabilities are granted concurrently. * refactor: Simplify systemGrantSchema index definition - Removed the sparse option from the unique index on principalType, principalId, capability, and tenantId in the systemGrantSchema. - This change streamlines the index definition, potentially improving query performance and clarity in the schema design. * refactor: Reorganize role capability check in roles route - Moved the capability check for reading roles to occur after parsing the roleName, improving code clarity and structure. - This change ensures that the authorization logic is consistently applied before fetching role details, enhancing overall permission management. * refactor: Remove unused ISystemGrant interface from systemCapabilities.ts - Deleted the ISystemGrant interface as it was no longer needed, streamlining the code and improving clarity. - This change helps reduce clutter in the file and focuses on relevant capabilities for the system. * refactor: Migrate SystemCapabilities to data-schemas - Replaced imports of SystemCapabilities from 'librechat-data-provider' with imports from '@librechat/data-schemas' across multiple files. - This change centralizes the management of system capabilities, improving code organization and maintainability. * refactor: Update account deletion middleware and capability checks - Modified the `canDeleteAccount` middleware to ensure that the account deletion permission is only granted to users with the `MANAGE_USERS` capability, improving security and clarity in permission management. - Enhanced error logging for unauthorized account deletion attempts, providing better insights into permission issues. - Updated the `capabilities.ts` file to ensure consistent handling of user authentication checks, improving robustness in capability verification. - Refined type definitions in `systemGrant.ts` and `systemGrantMethods.ts` to utilize the `PrincipalType` enum, enhancing type safety and code clarity. * refactor: Extract principal ID normalization into a separate function - Introduced `normalizePrincipalId` function to streamline the normalization of principal IDs based on their type, enhancing code clarity and reusability. - Updated references in `createSystemGrantMethods` to utilize the new normalization function, improving maintainability and reducing code duplication. * test: Add unit tests for principalId normalization in systemGrant - Introduced tests for the `grantCapability`, `revokeCapability`, and `getCapabilitiesForPrincipal` methods to verify correct handling of principalId normalization between string and ObjectId formats. - Enhanced the `capabilities.ts` middleware to utilize the `PrincipalType` enum for improved type safety. - Added a new utility function `normalizePrincipalId` to streamline principal ID normalization logic, ensuring consistent behavior across the application. * feat: Introduce capability implications and enhance system grant methods - Added `CapabilityImplications` to define relationships between broader and implied capabilities, allowing for more intuitive permission checks. - Updated `createSystemGrantMethods` to expand capability queries to include implied capabilities, improving authorization logic. - Enhanced `systemGrantSchema` to include an `expiresAt` field for future TTL enforcement of grants, and added validation to ensure `tenantId` is not set to null. - Documented authorization requirements for prompt group and prompt deletion methods to clarify access control expectations. * test: Add unit tests for canDeleteAccount middleware - Introduced unit tests for the `canDeleteAccount` middleware to verify account deletion permissions based on user roles and capabilities. - Covered scenarios for both allowed and blocked account deletions, including checks for ADMIN users with the `MANAGE_USERS` capability and handling of undefined user cases. - Enhanced test structure to ensure clarity and maintainability of permission checks in the middleware. * fix: Add principalType enum validation to SystemGrant schema Without enum validation, any string value was accepted for principalType and silently stored. Invalid documents would never match capability queries, creating phantom grants impossible to diagnose without raw DB inspection. All other ACL models in the codebase validate this field. * fix: Replace seedSystemGrants Promise.all with bulkWrite for concurrency safety When two server instances start simultaneously (K8s rolling deploy, PM2 cluster), both call seedSystemGrants. With Promise.all + findOneAndUpdate upsert, both instances may attempt to insert the same documents, causing E11000 duplicate key errors that crash server startup. bulkWrite with ordered:false handles concurrent upserts gracefully and reduces 17 individual round trips to a single network call. The returned documents (previously discarded) are no longer fetched. * perf: Add AsyncLocalStorage per-request cache for capability checks Every hasCapability call previously required 2 DB round trips (getUserPrincipals + SystemGrant.exists) — replacing what were O(1) string comparisons. Routes like patchPromptGroup triggered this twice, and hasConfigCapability's fallback path resolved principals twice. This adds a per-request AsyncLocalStorage cache that: - Caches resolved principals (same for all checks within one request) - Caches capability check results (same user+cap = same answer) - Automatically scoped to request lifetime (no stale grants) - Falls through to DB when no store exists (background jobs, tests) - Requires no signature changes to hasCapability The capabilityContextMiddleware is registered at the app level before all routes, initializing a fresh store per request. * fix: Add error handling for inline hasCapability calls canDeleteAccount, fetchAssistants, and validateAuthor all call hasCapability without try-catch. These were previously O(1) string comparisons that could never throw. Now they hit the database and can fail on connection timeout or transient errors. Wrap each call in try-catch, defaulting to deny (false) on error. This ensures a DB hiccup returns a clean 403 instead of an unhandled 500 with a stack trace. * test: Add canDeleteAccount DB-error resilience test Tests that hasCapability rejection (e.g., DB timeout) results in a clean 403 rather than an unhandled exception. Validates the error handling added in the previous commit. * refactor: Use barrel import for hasCapability in validateAuthor Import from ~/server/middleware barrel instead of directly from ~/server/middleware/roles/capabilities for consistency with other non-middleware consumers. Files within the middleware barrel itself must continue using direct imports to avoid circular requires. * refactor: Remove misleading pre('save') hook from SystemGrant schema The pre('save') hook normalized principalId for USER/GROUP principals, but the primary write path (grantCapability) uses findOneAndUpdate — which does not trigger save hooks. The normalization was already handled explicitly in grantCapability itself. The hook created a false impression of schema-level enforcement that only covered save()/create() paths. Replace with a comment documenting that all writes must go through grantCapability. * feat: Add READ_ASSISTANTS capability to complete manage/read pair Every other managed resource had a paired READ_X / MANAGE_X capability except assistants. This adds READ_ASSISTANTS and registers the MANAGE_ASSISTANTS → READ_ASSISTANTS implication in CapabilityImplications, enabling future read-only assistant visibility grants. * chore: Reorder systemGrant methods for clarity Moved hasCapabilityForPrincipals to a more logical position in the returned object of createSystemGrantMethods, improving code readability. This change also maintains the inclusion of seedSystemGrants in the export, ensuring all necessary methods are available. * fix: Wrap seedSystemGrants in try-catch to avoid blocking startup Seeding capabilities is idempotent and will succeed on the next restart. A transient DB error during seeding should not prevent the server from starting — log the error and continue. * refactor: Improve capability check efficiency and add audit logging Move hasCapability calls after cheap early-exits in validateAuthor and fetchAssistants so the DB check only runs when its result matters. Add logger.debug on every capability bypass grant across all 7 call sites for auditability, and log errors in catch blocks instead of silently swallowing them. * test: Add integration tests for AsyncLocalStorage capability caching Exercises the full vertical — ALS context, generateCapabilityCheck, real getUserPrincipals, real hasCapabilityForPrincipals, real MongoDB via MongoMemoryServer. Covers per-request caching, cross-context isolation, concurrent request isolation, negative caching, capability implications, tenant scoping, group-based grants, and requireCapability middleware. * test: Add systemGrant data-layer and ALS edge-case integration tests systemGrant.spec.ts (51 tests): Full integration tests for all systemGrant methods against real MongoDB — grant/revoke lifecycle, principalId normalization (string→ObjectId for USER/GROUP, string for ROLE), capability implications (both directions), tenant scoping, schema validation (null tenantId, invalid enum, required fields, unique compound index). capabilities.integration.spec.ts (27 tests): Adds ALS edge cases — missing context degrades gracefully with no caching (background jobs, child processes), nested middleware creates independent inner context, optional-chaining safety when store is undefined, mid-request grant changes are invisible due to result caching, requireCapability works without ALS, and interleaved concurrent contexts maintain isolation. * fix: Add worker thread guards to capability ALS usage Detect when hasCapability or capabilityContextMiddleware is called from a worker thread (where ALS context does not propagate from the parent). hasCapability logs a warn-once per factory instance; the middleware logs an error since mounting Express middleware in a worker is likely a misconfiguration. Both continue to function correctly — the guard is observability, not a hard block. * fix: Include tenantId in ALS principal cache key for tenant isolation The principal cache key was user.id:user.role, which would reuse cached principals across tenants for the same user within a request. When getUserPrincipals gains tenant-scoped group resolution, principals from tenant-a would incorrectly serve tenant-b checks. Changed to user.id:user.role:user.tenantId to prevent cross-tenant cache hits. Adds integration test proving separate principal lookups per tenantId. * test: Remove redundant mocked capabilities.spec.js The JS wrapper test (7 tests, all mocked) is a strict subset of capabilities.integration.spec.ts (28 tests, real MongoDB). Every scenario it covered — hasCapability true/false, tenantId passthrough, requireCapability 403/500, error handling — is tested with higher fidelity in the integration suite. * test: Replace mocked canDeleteAccount tests with real MongoDB integration Remove hasCapability mock — tests now exercise the full capability chain against real MongoDB (getUserPrincipals, hasCapabilityForPrincipals, SystemGrant collection). Only mocks remaining are logger and cache. Adds new coverage: admin role without grant is blocked, user-level grant bypasses deletion restriction, null user handling. * test: Add comprehensive tests for ACL entry management and user group methods Introduces new tests for `deleteAclEntries`, `bulkWriteAclEntries`, and `findPublicResourceIds` in `aclEntry.spec.ts`, ensuring proper functionality for deleting and bulk managing ACL entries. Additionally, enhances `userGroup.spec.ts` with tests for finding groups by ID and name pattern, including external ID matching and source filtering. These changes improve coverage and validate the integrity of ACL and user group operations against real MongoDB interactions. * refactor: Update capability checks and logging for better clarity and error handling Replaced `MANAGE_USERS` with `ACCESS_ADMIN` in the `canDeleteAccount` middleware and related tests to align with updated permission structure. Enhanced logging in various middleware functions to use `logger.warn` for capability check failures, providing clearer error messages. Additionally, refactored capability checks in the `patchPromptGroup` and `validateAuthor` functions to improve readability and maintainability. This commit also includes adjustments to the `systemGrant` methods to implement retry logic for transient failures during capability seeding, ensuring robustness in the face of database errors. * refactor: Enhance logging and retry logic in seedSystemGrants method Updated the logging format in the seedSystemGrants method to include error messages for better clarity. Improved the retry mechanism by explicitly mocking multiple failures in tests, ensuring robust error handling during transient database issues. Additionally, refined imports in the systemGrant schema for better type management. * refactor: Consolidate imports in canDeleteAccount middleware Merged logger and SystemCapabilities imports from the data-schemas module into a single line for improved readability and maintainability of the code. This change streamlines the import statements in the canDeleteAccount middleware. * test: Enhance systemGrant tests for error handling and capability validation Added tests to the systemGrant methods to handle various error scenarios, including E11000 race conditions, invalid ObjectId strings for USER and GROUP principals, and invalid capability strings. These enhancements improve the robustness of the capability granting and revoking logic, ensuring proper error propagation and validation of inputs. * fix: Wrap hasCapability calls in deny-by-default try-catch at remaining sites canAccessResource, files.js, and roles.js all had hasCapability inside outer try-catch blocks that returned 500 on DB failure instead of falling through to the regular ACL check. This contradicts the deny-by-default pattern used everywhere else. Also removes raw error.message from the roles.js 500 response to prevent internal host/connection info leaking to clients. * fix: Normalize user ID in canDeleteAccount before passing to hasCapability requireCapability normalizes req.user.id via _id?.toString() fallback, but canDeleteAccount passed raw req.user directly. If req.user.id is absent (some auth layers only populate _id), getUserPrincipals received undefined, silently returning empty principals and blocking the bypass. * fix: Harden systemGrant schema and type safety - Reject empty string tenantId in schema validator (was only blocking null; empty string silently orphaned documents) - Fix reverseImplications to use BaseSystemCapability[] instead of string[], preserving the narrow discriminated type - Document READ_ASSISTANTS as reserved/unenforced * test: Use fake timers for seedSystemGrants retry tests and add tenantId validation - Switch retry tests to jest.useFakeTimers() to eliminate 3+ seconds of real setTimeout delays per test run - Add regression test for empty-string tenantId rejection * docs: Add TODO(#12091) comments for tenant-scoped capability gaps In multi-tenant mode, platform-level grants (no tenantId) won't match tenant-scoped queries, breaking admin access. getUserPrincipals also returns cross-tenant group memberships. Both need fixes in #12091.	2026-03-21 14:28:54 -04:00
Danny Avila	0412f05daf	🪢 chore: Consolidate Pricing and Tx Imports After tx.js Module Removal (#12086) ... * 🧹 chore: resolve imports due to rebase * chore: Update model mocks in unit tests for consistency - Consolidated model mock implementations across various test files to streamline setup and reduce redundancy. - Removed duplicate mock definitions for `getMultiplier` and `getCacheMultiplier`, ensuring a unified approach in `recordCollectedUsage.spec.js`, `openai.spec.js`, `responses.unit.spec.js`, and `abortMiddleware.spec.js`. - Enhanced clarity and maintainability of test files by aligning mock structures with the latest model updates. * fix: Safeguard token credit checks in transaction tests - Updated assertions in `transaction.spec.ts` to handle potential null values for `updatedBalance` by using optional chaining. - Enhanced robustness of tests related to token credit calculations, ensuring they correctly account for scenarios where the balance may not be found. * chore: transaction methods with bulk insert functionality - Introduced `bulkInsertTransactions` method in `transaction.ts` to facilitate batch insertion of transaction documents. - Updated test file `transactions.bulk-parity.spec.ts` to utilize new pricing function assignments and handle potential null values in calculations, improving test robustness. - Refactored pricing function initialization for clarity and consistency. * refactor: Enhance type definitions and introduce new utility functions for model matching - Added `findMatchingPattern` and `matchModelName` utility functions to improve model name matching logic in transaction methods. - Updated type definitions for `findMatchingPattern` to accept a more specific tokensMap structure, enhancing type safety. - Refactored `dbMethods` initialization in `transactions.bulk-parity.spec.ts` to include the new utility functions, improving test clarity and functionality. * refactor: Update database method imports and enhance transaction handling - Refactored `abortMiddleware.js` to utilize centralized database methods for message handling and conversation retrieval, improving code consistency. - Enhanced `bulkInsertTransactions` in `transaction.ts` to handle empty document arrays gracefully and added error logging for better debugging. - Updated type definitions in `transactions.ts` to enforce stricter typing for token types, enhancing type safety across transaction methods. - Improved test setup in `transactions.bulk-parity.spec.ts` by refining pricing function assignments and ensuring robust handling of potential null values. * refactor: Update database method references and improve transaction multiplier handling - Refactored `client.js` to update database method references for `bulkInsertTransactions` and `updateBalance`, ensuring consistency in method usage. - Enhanced transaction multiplier calculations in `transaction.spec.ts` to provide fallback values for write and read multipliers, improving robustness in cost calculations across structured token spending tests.	2026-03-21 14:28:53 -04:00
Danny Avila	8ba2bde5c1	📦 refactor: Consolidate DB models, encapsulating Mongoose usage in data-schemas (#11830) ... * chore: move database model methods to /packages/data-schemas * chore: add TypeScript ESLint rule to warn on unused variables * refactor: model imports to streamline access - Consolidated model imports across various files to improve code organization and reduce redundancy. - Updated imports for models such as Assistant, Message, Conversation, and others to a unified import path. - Adjusted middleware and service files to reflect the new import structure, ensuring functionality remains intact. - Enhanced test files to align with the new import paths, maintaining test coverage and integrity. * chore: migrate database models to packages/data-schemas and refactor all direct Mongoose Model usage outside of data-schemas * test: update agent model mocks in unit tests - Added `getAgent` mock to `client.test.js` to enhance test coverage for agent-related functionality. - Removed redundant `getAgent` and `getAgents` mocks from `openai.spec.js` and `responses.unit.spec.js` to streamline test setup and reduce duplication. - Ensured consistency in agent mock implementations across test files. * fix: update types in data-schemas * refactor: enhance type definitions in transaction and spending methods - Updated type definitions in `checkBalance.ts` to use specific request and response types. - Refined `spendTokens.ts` to utilize a new `SpendTxData` interface for better clarity and type safety. - Improved transaction handling in `transaction.ts` by introducing `TransactionResult` and `TxData` interfaces, ensuring consistent data structures across methods. - Adjusted unit tests in `transaction.spec.ts` to accommodate new type definitions and enhance robustness. * refactor: streamline model imports and enhance code organization - Consolidated model imports across various controllers and services to a unified import path, improving code clarity and reducing redundancy. - Updated multiple files to reflect the new import structure, ensuring all functionalities remain intact. - Enhanced overall code organization by removing duplicate import statements and optimizing the usage of model methods. * feat: implement loadAddedAgent and refactor agent loading logic - Introduced `loadAddedAgent` function to handle loading agents from added conversations, supporting multi-convo parallel execution. - Created a new `load.ts` file to encapsulate agent loading functionalities, including `loadEphemeralAgent` and `loadAgent`. - Updated the `index.ts` file to export the new `load` module instead of the deprecated `loadAgent`. - Enhanced type definitions and improved error handling in the agent loading process. - Adjusted unit tests to reflect changes in the agent loading structure and ensure comprehensive coverage. * refactor: enhance balance handling with new update interface - Introduced `IBalanceUpdate` interface to streamline balance update operations across the codebase. - Updated `upsertBalanceFields` method signatures in `balance.ts`, `transaction.ts`, and related tests to utilize the new interface for improved type safety. - Adjusted type imports in `balance.spec.ts` to include `IBalanceUpdate`, ensuring consistency in balance management functionalities. - Enhanced overall code clarity and maintainability by refining type definitions related to balance operations. * feat: add unit tests for loadAgent functionality and enhance agent loading logic - Introduced comprehensive unit tests for the `loadAgent` function, covering various scenarios including null and empty agent IDs, loading of ephemeral agents, and permission checks. - Enhanced the `initializeClient` function by moving `getConvoFiles` to the correct position in the database method exports, ensuring proper functionality. - Improved test coverage for agent loading, including handling of non-existent agents and user permissions. * chore: reorder memory method exports for consistency - Moved `deleteAllUserMemories` to the correct position in the exported memory methods, ensuring a consistent and logical order of method exports in `memory.ts`.	2026-03-21 14:28:53 -04:00
Danny Avila	58f128bee7	🗑️ chore: Remove Deprecated Project Model and Associated Fields (#11773) ... * chore: remove projects and projectIds usage * chore: empty line linting * chore: remove isCollaborative property across agent models and related tests - Removed the isCollaborative property from agent models, controllers, and tests, as it is deprecated in favor of ACL permissions. - Updated related validation schemas and data provider types to reflect this change. - Ensured all references to isCollaborative were stripped from the codebase to maintain consistency and clarity.	2026-03-21 14:28:53 -04:00
Danny Avila	38521381f4	🐘 feat: FerretDB Compatibility (#11769) ... * feat: replace unsupported MongoDB aggregation operators for FerretDB compatibility Replace $lookup, $unwind, $sample, $replaceRoot, and $addFields aggregation stages which are unsupported on FerretDB v2.x (postgres-documentdb backend). - Prompt.js: Replace $lookup/$unwind/$project pipelines with find().select().lean() + attachProductionPrompts() batch helper. Replace $group/$replaceRoot/$sample in getRandomPromptGroups with distinct() + Fisher-Yates shuffle. - Agent/Prompt migration scripts: Replace $lookup anti-join pattern with distinct() + $nin two-step queries for finding un-migrated resources. All replacement patterns verified against FerretDB v2.7.0. * fix: use $pullAll for simple array removals, fix memberIds type mismatches Replace $pull with $pullAll for exact-value scalar array removals. Both operators work on MongoDB and FerretDB, but $pullAll is more explicit for exact matching (no condition expressions). Fix critical type mismatch bugs where ObjectId values were used against String[] memberIds arrays in Group queries: - config/delete-user.js: use string uid instead of ObjectId user._id - e2e/setup/cleanupUser.ts: convert userId.toString() before query Harden PermissionService.bulkUpdateResourcePermissions abort handling to prevent crash when abortTransaction is called after commitTransaction. All changes verified against FerretDB v2.7.0 and MongoDB Memory Server. * fix: harden transaction support probe for FerretDB compatibility Commit the transaction before aborting in supportsTransactions probe, and wrap abortTransaction in try-catch to prevent crashes when abort is called after a successful commit (observed behavior on FerretDB). * feat: add FerretDB compatibility test suite, retry utilities, and CI config Add comprehensive FerretDB integration test suite covering: - $pullAll scalar array operations - $pull with subdocument conditions - $lookup replacement (find + manual join) - $sample replacement (distinct + Fisher-Yates) - $bit and $bitsAllSet operations - Migration anti-join pattern - Multi-tenancy (useDb, scaling, write amplification) - Sharding proof-of-concept - Production operations (backup/restore, schema migration, deadlock retry) Add production retryWithBackoff utility for deadlock recovery during concurrent index creation on FerretDB/DocumentDB backends. Add UserController.spec.js tests for deleteUserController (runs in CI). Configure jest and eslint to isolate FerretDB tests from CI pipelines: - packages/data-schemas/jest.config.mjs: ignore misc/ directory - eslint.config.mjs: ignore packages/data-schemas/misc/ Include Docker Compose config for local FerretDB v2.7 + postgres-documentdb, dedicated jest/tsconfig for the test files, and multi-tenancy findings doc. * style: brace formatting in aclEntry.ts modifyPermissionBits * refactor: reorganize retry utilities and update imports - Moved retryWithBackoff utility to a new file `retry.ts` for better structure. - Updated imports in `orgOperations.ferretdb.spec.ts` to reflect the new location of retry utilities. - Removed old import statement for retryWithBackoff from index.ts to streamline exports. * test: add $pullAll coverage for ConversationTag and PermissionService Add integration tests for deleteConversationTag verifying $pullAll removes tags from conversations correctly, and for syncUserEntraGroupMemberships verifying $pullAll removes user from non-matching Entra groups while preserving local group membership. ---------	2026-03-21 14:28:49 -04:00
Danny Avila	365a0dc0f6	🩺 refactor: Surface Descriptive OCR Error Messages to Client (#12344) ... * fix: pass along error message when OCR fails Right now, if OCR fails, it just says "Error processing file" which isn't very helpful. The `error.message` does has helpful information in it, but our filter wasn't including the right case to pass it along. Now it does! * fix: extract shared upload error filter, apply to images route The 'Unable to extract text from' error was only allowlisted in the files route but not the images route, which also calls processAgentFileUpload. Extract the duplicated error filter logic into a shared resolveUploadErrorMessage utility in packages/api so both routes stay in sync. --------- Co-authored-by: Dan Lew <daniel@mightyacorn.com>	2026-03-20 17:10:25 -04:00
mfish911	4e5ae28fa9	📡 feat: Support Unauthenticated SMTP Relays (#12322) ... * allow smtp server that does not have authentication * fix: align checkEmailConfig with optional SMTP credentials and add tests Remove EMAIL_USERNAME/EMAIL_PASSWORD requirements from the hasSMTPConfig predicate in checkEmailConfig() so the rest of the codebase (login, startup checks, invite-user) correctly recognizes unauthenticated SMTP as a valid email configuration. Add a warning when only one of the two credential env vars is set, in both sendEmail.js and checkEmailConfig(), to catch partial misconfigurations early. Add test coverage for both the transporter auth assembly in sendEmail.js and the checkEmailConfig predicate in packages/api. Document in .env.example that credentials are optional for unauthenticated SMTP relays. --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-03-20 13:07:39 -04:00
JooyoungChoi14	54fc9c2c99	♾️ fix: Permanent Ban Cache and Expired Ban Cleanup Defects (#12324) ... * fix: preserve ban data object in checkBan to prevent permanent cache The !! operator on line 108 coerces the ban data object to a boolean, losing the expiresAt property. This causes: 1. Number(true.expiresAt) = NaN → expired bans never cleaned from banLogs 2. banCache.set(key, true, NaN) → Keyv stores with expires: null (permanent) 3. IP-based cache entries persist indefinitely, blocking unrelated users Fix: replace isBanned (boolean) with banData (original object) so that expiresAt is accessible for TTL calculation and proper cache expiry. * fix: address checkBan cleanup defects exposed by ban data fix The prior commit correctly replaced boolean coercion with the ban data object, but activated previously-dead cleanup code with several defects: - IP-only expired bans fell through cleanup without returning next(), re-caching with negative TTL (permanent entry) and blocking the user - Redis deployments used cache-prefixed keys for banLogs.delete(), silently failing since bans are stored at raw keys - banCache.set() calls were fire-and-forget, silently dropping errors - No guard for missing/invalid expiresAt reproduced the NaN TTL bug on legacy ban records Consolidate expired-ban cleanup into a single block that always returns next(), use raw keys (req.ip, userId) for banLogs.delete(), add an expiresAt validity guard, await cache writes with error logging, and parallelize independent I/O with Promise.all. Add 25 tests covering all checkBan code paths including the specific regressions for IP-only cleanup, Redis key mismatch, missing expiresAt, and cache write failures. --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-03-20 12:47:51 -04:00
Airam Hernández Hernández	96f6976e00	🪂 fix: Automatic logout_hint Fallback for Oversized OpenID Token URLs (#12326) ... * fix: automatic logout_hint fallback for long OpenID tokens Implements OIDC RP-Initiated Logout cascading strategy to prevent errors when id_token_hint makes logout URL too long. Automatically detects URLs exceeding configurable length and falls back to logout_hint only when URL is too long, preserving previous behavior when token is missing. Adds OPENID_MAX_LOGOUT_URL_LENGTH environment variable. Comprehensive test coverage with 20 tests. Works with any OpenID provider. * fix: address review findings for OIDC logout URL length fallback - Replace two-boolean tri-state (useIdTokenHint/urlTooLong) with a single string discriminant ('use_token'|'too_long'|'no_token') for clarity - Fix misleading warning: differentiate 'url too long + no client_id' from 'no token + no client_id' so operators get actionable advice - Strict env var parsing: reject partial numeric strings like '500abc' that Number.parseInt silently accepted; use regex + Number() instead - Pre-compute projected URL length from base URL + token length (JWT chars are URL-safe), eliminating the set-then-delete mutation pattern - Extract parseMaxLogoutUrlLength helper for validation and early return - Add tests: invalid env values, url-too-long + missing OPENID_CLIENT_ID, boundary condition (exact max vs max+1), cookie-sourced long token - Remove redundant try/finally in 'respects custom limit' test - Use empty value in .env.example to signal optional config (default: 2000) --------- Co-authored-by: Airam Hernández Hernández <airam.hernandez@intelequia.com> Co-authored-by: Danny Avila <danny@librechat.ai>	2026-03-20 12:46:57 -04:00
Brad Russell	ecd6d76bc8	🚦 fix: ERR_ERL_INVALID_IP_ADDRESS and IPv6 Key Collisions in IP Rate Limiters (#12319) ... * fix: Add removePorts keyGenerator to all IP-based rate limiters Six IP-based rate limiters are missing the `keyGenerator: removePorts` option that is already used by the auth-related limiters (login, register, resetPassword, verifyEmail). Without it, reverse proxies that include ports in X-Forwarded-For headers cause ERR_ERL_INVALID_IP_ADDRESS errors from express-rate-limit. Fixes #12318 * fix: make removePorts IPv6-safe to prevent rate-limit key collisions The original regex `/:\d+[^:]*$/` treated the last colon-delimited segment of bare IPv6 addresses as a port, mangling valid IPs (e.g. `::1` → `::`, `2001:db8::1` → `2001:db8::`). Distinct IPv6 clients could collapse into the same rate-limit bucket. Use `net.isIP()` as a fast path for already-valid IPs, then match bracketed IPv6+port and IPv4+port explicitly. Bare IPv6 addresses are now returned unchanged. Also fixes pre-existing property ordering inconsistency in ttsLimiters.js userLimiterOptions (keyGenerator before store). * refactor: move removePorts to packages/api as TypeScript, fix import order - Move removePorts implementation to packages/api/src/utils/removePorts.ts with proper Express Request typing - Reduce api/server/utils/removePorts.js to a thin re-export from @librechat/api for backward compatibility - Consolidate removePorts import with limiterCache from @librechat/api in all 6 limiter files, fixing import order (package imports shortest to longest, local imports longest to shortest) - Remove narrating inline comments per code style guidelines --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-03-19 21:48:03 -04:00
Danny Avila	1ecff83b20	🪦 fix: ACL-Safe User Account Deletion for Agents, Prompts, and MCP Servers (#12314) ... * fix: use ACL ownership for prompt group cleanup on user deletion deleteUserPrompts previously called getAllPromptGroups with only an author filter, which defaults to searchShared=true and drops the author filter for shared/global project entries. This caused any user deleting their account to strip shared prompt group associations and ACL entries for other users. Replace the author-based query with ACL-based ownership lookup: - Find prompt groups where the user has OWNER permission (DELETE bit) - Only delete groups where the user is the sole owner - Preserve multi-owned groups and their ACL entries for other owners * fix: use ACL ownership for agent cleanup on user deletion deleteUserAgents used the deprecated author field to find and delete agents, then unconditionally removed all ACL entries for those agents. This could destroy ACL entries for agents shared with or co-owned by other users. Replace the author-based query with ACL-based ownership lookup: - Find agents where the user has OWNER permission (DELETE bit) - Only delete agents where the user is the sole owner - Preserve multi-owned agents and their ACL entries for other owners - Also clean up handoff edges referencing deleted agents * fix: add MCP server cleanup on user deletion User deletion had no cleanup for MCP servers, leaving solely-owned servers orphaned in the database with dangling ACL entries for other users. Add deleteUserMcpServers that follows the same ACL ownership pattern as prompt groups and agents: find servers with OWNER permission, check for sole ownership, and only delete those with no other owners. * style: fix prettier formatting in Prompt.spec.js * refactor: extract getSoleOwnedResourceIds to PermissionService The ACL sole-ownership detection algorithm was duplicated across deleteUserPrompts, deleteUserAgents, and deleteUserMcpServers. Centralizes the three-step pattern (find owned entries, find other owners, compute sole-owned set) into a single reusable utility. * refactor: use getSoleOwnedResourceIds in all deletion functions - Replace inline ACL queries with the centralized utility - Remove vestigial _req parameter from deleteUserPrompts - Use Promise.all for parallel project removal instead of sequential awaits - Disconnect live MCP sessions and invalidate tool cache before deleting sole-owned MCP server documents - Export deleteUserMcpServers for testability * test: improve deletion test coverage and quality - Move deleteUserPrompts call to beforeAll to eliminate execution-order dependency between tests - Standardize on test() instead of it() for consistency in Prompt.spec.js - Add assertion for deleting user's own ACL entry preservation on multi-owned agents - Add deleteUserMcpServers integration test suite with 6 tests covering sole-owner deletion, multi-owner preservation, session disconnect, cache invalidation, model-not-registered guard, and missing MCPManager - Add PermissionService mock to existing deleteUser.spec.js to fix import chain * fix: add legacy author-based fallback for unmigrated resources Resources created before the ACL system have author set but no AclEntry records. The sole-ownership detection returns empty for these, causing deleteUserPrompts, deleteUserAgents, and deleteUserMcpServers to silently skip them — permanently orphaning data on user deletion. Add a fallback that identifies author-owned resources with zero ACL entries (truly unmigrated) and includes them in the deletion set. This preserves the multi-owner safety of the ACL path while ensuring pre-ACL resources are still cleaned up regardless of migration status. * style: fix prettier formatting across all changed files * test: add resource type coverage guard for user deletion Ensures every ResourceType in the ACL system has a corresponding cleanup handler wired into deleteUserController. When a new ResourceType is added (e.g. WORKFLOW), this test fails immediately, preventing silent data orphaning on user account deletion. * style: fix import order in PermissionService destructure * test: add opt-out set and fix test lifecycle in coverage guard Add NO_USER_CLEANUP_NEEDED set for resource types that legitimately require no per-user deletion. Move fs.readFileSync into beforeAll so path errors surface as clean test failures instead of unhandled crashes.	2026-03-19 17:46:14 -04:00
Danny Avila	f380390408	🛡️ fix: Prevent loop in ChatGPT import on Cyclic Parent Graphs (#12313) ... Cap adjustTimestampsForOrdering to N passes and add cycle detection to findValidParent, preventing DoS via crafted ChatGPT export files with cyclic parentMessageId relationships. Add breakParentCycles to sever cyclic back-edges before saving, ensuring structurally valid message trees are persisted to the DB.	2026-03-19 17:15:12 -04:00
Danny Avila	39f5f83a8a	🔌 fix: Isolate Code-Server HTTP Agents to Prevent Socket Pool Contamination (#12311) ... * 🔧 fix: Isolate HTTP agents for code-server axios requests Prevents socket hang up after 5s on Node 19+ when code executor has file attachments. follow-redirects (axios dep) leaks `socket.destroy` as a timeout listener on TCP sockets; with Node 19+ defaulting to keepAlive: true, tainted sockets re-enter the global pool and destroy active node-fetch requests in CodeExecutor after the idle timeout. Uses dedicated http/https agents with keepAlive: false for all axios calls targeting CODE_BASEURL in crud.js and process.js. Closes #12298 * ♻️ refactor: Extract code-server HTTP agents to shared module - Move duplicated agent construction from crud.js and process.js into a shared agents.js module to eliminate DRY violation - Switch process.js from raw `require('axios')` to `createAxiosInstance()` for proxy configuration parity with crud.js - Fix import ordering in process.js (agent constants no longer split imports) - Add 120s timeout to uploadCodeEnvFile (was the only code-server call without a timeout) * ✅ test: Add regression tests for code-server socket isolation - Add crud.spec.js covering getCodeOutputDownloadStream and uploadCodeEnvFile (agent options, timeout, URL, error handling) - Add socket pool isolation tests to process.spec.js asserting keepAlive:false agents are forwarded to axios - Update process.spec.js mocks for createAxiosInstance() migration * ♻️ refactor: Move code-server agents to packages/api Relocate agents.js from api/server/services/Files/Code/ to packages/api/src/utils/code.ts per workspace conventions. Consumers now import codeServerHttpAgent/codeServerHttpsAgent from @librechat/api.	2026-03-19 16:16:57 -04:00
Danny Avila	93952f06b4	🧯 fix: Remove Revoked Agents from User Favorites (#12296) ... * 🧯 fix: Remove revoked agents from user favorites When agent access is revoked, the agent remained in the user's favorites causing repeated 403 errors on page load. Backend now cleans up favorites on permission revocation; frontend treats 403 like 404 and auto-removes stale agent references. * 🧪 fix: Address review findings for stale agent favorites cleanup - Guard cleanup effect with ref to prevent infinite loop on mutation failure (Finding 1) - Use validated results.revoked instead of raw request payload for revokedUserIds (Finding 3) - Stabilize staleAgentIds memo with string key to avoid spurious re-evaluation during drag-drop (Finding 5) - Add JSDoc with param types to removeRevokedAgentFromFavorites (Finding 7) - Return promise from removeRevokedAgentFromFavorites for testability - Add 7 backend tests covering revocation cleanup paths - Add 3 frontend tests for 403 handling and stale cleanup persistence	2026-03-19 15:15:10 -04:00
Danny Avila	2f09d29c71	🛂 fix: Validate types Query Param in People Picker Access Middleware (#12276) ... * 🛂 fix: Validate `types` query param in people picker access middleware checkPeoplePickerAccess only inspected `req.query.type` (singular), allowing callers to bypass type-specific permission checks by using the `types` (plural) parameter accepted by the controller. Now both `type` and `types` are collected and each requested principal type is validated against the caller's role permissions. * 🛂 refactor: Hoist valid types constant, improve logging, and add edge-case tests - Hoist VALID_PRINCIPAL_TYPES to module-level Set to avoid per-request allocation - Include both `type` and `types` in error log for debuggability - Restore detailed JSDoc documenting per-type permission requirements - Add missing .json() assertion on partial-denial test - Add edge-case tests: all-invalid types, empty string types, PrincipalType.PUBLIC * 🏷️ fix: Align TPrincipalSearchParams with actual controller API The stale type used `type` (singular) but the controller and all callers use `types` (plural array). Aligns with PrincipalSearchParams in types/queries.ts.	2026-03-17 02:46:11 -04:00
Danny Avila	9a64791e3e	🪢 fix: Action Domain Encoding Collision for HTTPS URLs (#12271) ... * fix: strip protocol from domain before encoding in `domainParser` All https:// (and http://) domains produced the same 10-char base64 prefix due to ENCODED_DOMAIN_LENGTH truncation, causing tool name collisions for agents with multiple actions. Strip the protocol before encoding so the base64 key is derived from the hostname. Add `legacyDomainEncode` to preserve the old encoding logic for backward-compatible matching of existing stored actions. * fix: backward-compatible tool matching in ToolService Update `getActionToolDefinitions` to match stored tools against both new and legacy domain encodings. Update `loadActionToolsForExecution` to resolve model-called tool names via a `normalizedToDomain` map that includes both encoding variants, with legacy fallback for request builder lookup. * fix: action route save/delete domain encoding issues Save routes now remove old tools matching either new or legacy domain encoding, preventing stale entries when an action's encoding changes on update. Delete routes no longer re-encode the already-encoded domain extracted from the stored actions array, which was producing incorrect keys and leaving orphaned tools. * test: comprehensive coverage for action domain encoding Rewrite ActionService tests to cover real matching patterns used by ToolService and action routes. Tests verify encode/decode round-trips, protocol stripping, backward-compatible tool name matching at both definition and execution phases, save-route cleanup of old/new encodings, delete-route domain extraction, and the collision fix for multi-action agents. * fix: add legacy domain compat to all execution paths, make legacyDomainEncode sync CRITICAL: processRequiredActions (assistants path) was not updated with legacy domain matching — existing assistants with https:// domain actions would silently fail post-deployment because domainMap only had new encoding. MAJOR: loadAgentTools definitionsOnly=false path had the same issue. Both now use a normalizedToDomain map with legacy+new entries and extract function names via the matched key (not the canonical domain). Also: make legacyDomainEncode synchronous (no async operations), store legacyNormalized in processedActionSets to eliminate recomputation in the per-tool fallback, and hoist domainSeparatorRegex to module level. * refactor: clarify domain variable naming and tool-filter helpers in action routes Rename shadowed 'domain' to 'encodedDomain' to separate raw URL from encoded key in both agent and assistant save routes. Rename shouldRemoveTool to shouldRemoveAgentTool / shouldRemoveAssistantTool to make the distinct data-shape guards explicit. Remove await on now-synchronous legacyDomainEncode. * test: expand coverage for all review findings - Add validateAndUpdateTool tests (protocol-stripping match logic) - Restore unicode domain encode/decode/round-trip tests - Add processRequiredActions matching pattern tests (assistants path) - Add legacy guard skip test for short bare hostnames - Add pre-normalized Set test for definition-phase optimization - Fix corrupt-cache test to assert typeof instead of toBeDefined - Verify legacyDomainEncode is synchronous (not a Promise) - Remove all await on legacyDomainEncode (now sync) 58 tests, up from 44. * fix: address follow-up review findings A-E A: Fix stale JSDoc @returns {Promise<string>} on now-synchronous legacyDomainEncode — changed to @returns {string}. B: Rename normalizedToDomain to domainLookupMap in processRequiredActions and loadAgentTools where keys are raw encoded domains (not normalized), avoiding confusion with loadActionToolsForExecution where keys ARE normalized. C: Pre-normalize actionToolNames into a Set<string> in getActionToolDefinitions, replacing O(signatures × tools) per-check .some() + .replace() with O(1) Set.has() lookups. D: Remove stripProtocol from ActionService exports — it is a one-line internal helper. Spec tests for it removed; behavior is fully covered by domainParser protocol-stripping tests. E: Fix pre-existing bug where processRequiredActions re-loaded action sets on every missing-tool iteration. The guard !actionSets.length always re-triggered because actionSets was reassigned to a plain object (whose .length is undefined). Replaced with a null-check on a dedicated actionSetsData variable. * fix: strip path and query from domain URLs in stripProtocol URLs like 'https://api.example.com/v1/endpoint?foo=bar' previously retained the path after protocol stripping, contaminating the encoded domain key. Now strips everything after the first '/' following the host, using string indexing instead of URL parsing to avoid punycode normalization of unicode hostnames. Closes Copilot review comments 1, 2, and 5.	2026-03-17 01:38:51 -04:00
Danny Avila	d17ac8f06d	🔏 fix: Remove Federated Tokens from OpenID Refresh Response (#12264) ... * 🔒 fix: Remove OpenID federated tokens from refresh endpoint response The refresh controller was attaching federatedTokens (including the refresh_token) to the user object returned in the JSON response, exposing HttpOnly-protected tokens to client-side JavaScript. The tokens are already stored server-side by setOpenIDAuthTokens and re-attached by the JWT strategy on authenticated requests. * 🔒 fix: Strip sensitive fields from OpenID refresh response user object The OpenID refresh path returned the raw findOpenIDUser result without field projection, unlike the non-OpenID path which excludes password, __v, totpSecret, and backupCodes via getUserById projection. Destructure out sensitive fields before serializing. Also strengthens the regression test: uses not.toHaveProperty for true property-absence checks (expect.anything() misses null/undefined), adds positive shape assertion, and DRYs up duplicated mock user setup.	2026-03-16 09:23:46 -04:00
Danny Avila	381ed8539b	🪪 fix: Enforce Conversation Ownership Checks in Remote Agent Controllers (#12263) ... * 🔒 fix: Validate conversation ownership in remote agent API endpoints Add user-scoped ownership checks for client-supplied conversation IDs in OpenAI-compatible and Open Responses controllers to prevent cross-tenant file/message loading via IDOR. * 🔒 fix: Harden ownership checks against type confusion and unhandled errors - Add typeof string validation before getConvo to block NoSQL operator injection (e.g. { "$gt": "" }) bypassing the ownership check - Move ownership checks inside try/catch so DB errors produce structured JSON error responses instead of unhandled promise rejections - Add string type validation for conversation_id and previous_response_id in the upstream TS request validators (defense-in-depth) * 🧪 test: Add coverage for conversation ownership validation in remote agent APIs - Fix broken getConvo mock in openai.spec.js (was missing entirely) - Add tests for: owned conversation, unowned (404), non-string type (400), absent conversation_id (skipped), and DB error (500) — both controllers	2026-03-16 09:19:48 -04:00
Danny Avila	acd07e8085	🗝️ fix: Exempt Admin-Trusted Domains from MCP OAuth Validation (#12255) ... * fix: exempt allowedDomains from MCP OAuth SSRF checks (#12254) The SSRF guard in validateOAuthUrl was context-blind — it blocked private/internal OAuth endpoints even for admin-trusted MCP servers listed in mcpSettings.allowedDomains. Add isHostnameAllowed() to domain.ts and skip SSRF checks in validateOAuthUrl when the OAuth endpoint hostname matches an allowed domain. * refactor: thread allowedDomains through MCP connection stack Pass allowedDomains from MCPServersRegistry through BasicConnectionOptions, MCPConnectionFactory, and into MCPOAuthHandler method calls so the OAuth layer can exempt admin-trusted domains from SSRF validation. * test: add allowedDomains bypass tests and fix registry mocks Add isHostnameAllowed unit tests (exact, wildcard, case-insensitive, private IPs). Add MCPOAuthSecurity tests covering the allowedDomains bypass for initiateOAuthFlow, refreshOAuthTokens, and revokeOAuthToken. Update registry mocks to include getAllowedDomains. * fix: enforce protocol/port constraints in OAuth allowedDomains bypass Replace isHostnameAllowed (hostname-only check) with isOAuthUrlAllowed which parses the full OAuth URL and matches against allowedDomains entries including protocol and explicit port constraints — mirroring isDomainAllowedCore's allowlist logic. Prevents a port-scoped entry like 'https://auth.internal:8443' from also exempting other ports. * test: cover auto-discovery and branch-3 refresh paths with allowedDomains Add three new integration tests using a real OAuth test server: - auto-discovered OAuth endpoints allowed when server IP is in allowedDomains - auto-discovered endpoints rejected when allowedDomains doesn't match - refreshOAuthTokens branch 3 (no clientInfo/config) with allowedDomains bypass Also rename describe block from ephemeral issue number to durable name. * docs: explain intentional absence of allowedDomains in completeOAuthFlow Prevents future contributors from assuming a missing parameter during security audits — URLs are pre-validated during initiateOAuthFlow. * test: update initiateOAuthFlow assertion for allowedDomains parameter * perf: avoid redundant URL parse for admin-trusted OAuth endpoints Move isOAuthUrlAllowed check before the hostname extraction so admin-trusted URLs short-circuit with a single URL parse instead of two. The hostname extraction (new URL) is now deferred to the SSRF-check path where it's actually needed.	2026-03-15 23:03:12 -04:00
Danny Avila	8e8fb01d18	🧱 fix: Enforce Agent Access Control on Context and OCR File Loading (#12253) ... * 🔏 fix: Apply agent access control filtering to context/OCR resource loading The context/OCR file path in primeResources fetched files by file_id without applying filterFilesByAgentAccess, unlike the file_search and execute_code paths. Add filterFiles dependency injection to primeResources and invoke it after getFiles to enforce consistent access control. * fix: Wire filterFilesByAgentAccess into all agent initialization callers Pass the filterFilesByAgentAccess function from the JS layer into the TS initializeAgent → primeResources chain via dependency injection, covering primary, handoff, added-convo, and memory agent init paths. * test: Add access control filtering tests for primeResources Cover filterFiles invocation with context/OCR files, verify filtering rejects inaccessible files, and confirm graceful fallback when filterFiles, userId, or agentId are absent. * fix: Guard filterFilesByAgentAccess against ephemeral agent IDs Ephemeral agents have no DB document, so getAgent returns null and the access map defaults to all-false, silently blocking all non-owned files. Short-circuit with isEphemeralAgentId to preserve the pass-through behavior for inline-built agents (memory, tool agents). * fix: Clean up resources.ts and JS caller import order Remove redundant optional chain on req.user.role inside user-guarded block, update primeResources JSDoc with filterFiles and agentId params, and reorder JS imports to longest-to-shortest per project conventions. * test: Strengthen OCR assertion and add filterFiles error-path test Use toHaveBeenCalledWith for the OCR filtering test to verify exact arguments after the OCR→context merge step. Add test for filterFiles rejection to verify graceful degradation (logs error, returns original tool_resources). * fix: Correct import order in addedConvo.js and initialize.js Sort by total line length descending: loadAddedAgent (91) before filterFilesByAgentAccess (84), loadAgentTools (91) before filterFilesByAgentAccess (84). * test: Add unit tests for filterFilesByAgentAccess and hasAccessToFilesViaAgent Cover every branch in permissions.js: ephemeral agent guard, missing userId/agentId/files early returns, all-owned short-circuit, mixed owned + non-owned with VIEW/no-VIEW, agent-not-found fail-closed, author path scoped to attached files, EDIT gate on delete, DB error fail-closed, and agent with no tool_resources. * test: Cover file.user undefined/null in permissions spec Files with no user field fall into the non-owned path and get run through hasAccessToFilesViaAgent. Add two cases: attached file with no user field is returned, unattached file with no user field is excluded.	2026-03-15 23:02:36 -04:00
Danny Avila	6f87b49df8	🛂 fix: Enforce Actions Capability Gate Across All Event-Driven Tool Loading Paths (#12252) ... * fix: gate action tools by actions capability in all code paths Extract resolveAgentCapabilities helper to eliminate 3x-duplicated capability resolution. Apply early action-tool filtering in both loadToolDefinitionsWrapper and loadAgentTools non-definitions path. Gate loadActionToolsForExecution in loadToolsForExecution behind an actionsEnabled parameter with a cache-based fallback. Replace the late capability guard in loadAgentTools with a hasActionTools check to avoid unnecessary loadActionSets DB calls and duplicate warnings. * fix: thread actionsEnabled through InitializedAgent type Add actionsEnabled to the loadTools callback return type, InitializedAgent, and the initializeAgent destructuring/return so callers can forward the resolved value to loadToolsForExecution without redundant getEndpointsConfig cache lookups. * fix: pass actionsEnabled from callers to loadToolsForExecution Thread actionsEnabled through the agentToolContexts map in initialize.js (primary and handoff agents) and through primaryConfig in the openai.js and responses.js controllers, avoiding per-tool-call capability re-resolution on the hot path. * test: add regression tests for action capability gating Test the real exported functions (resolveAgentCapabilities, loadAgentTools, loadToolsForExecution) with mocked dependencies instead of shadow re-implementations. Covers definition filtering, execution gating, actionsEnabled param forwarding, and fallback capability resolution. * test: use Constants.EPHEMERAL_AGENT_ID in ephemeral fallback test Replaces a string guess with the canonical constant to avoid fragility if the ephemeral detection heuristic changes. * fix: populate agentToolContexts for addedConvo parallel agents After processAddedConvo returns, backfill agentToolContexts for any agents in agentConfigs not already present, so ON_TOOL_EXECUTE for added-convo agents receives actionsEnabled instead of falling back to a per-call cache lookup.	2026-03-15 23:01:36 -04:00
Danny Avila	a26eeea592	🔏 fix: Enforce MCP Server Authorization on Agent Tool Persistence (#12250) ... * 🛡️ fix: Validate MCP tool authorization on agent create/update Agent creation and update accepted arbitrary MCP tool strings without verifying the user has access to the referenced MCP servers. This allowed a user to embed unauthorized server names in tool identifiers (e.g. "anything_mcp_<victimServer>"), causing mcpServerNames to be stored on the agent and granting consumeOnly access via hasAccessViaAgent(). Adds filterAuthorizedTools() that checks MCP tool strings against the user's accessible server configs (via getAllServerConfigs) before persisting. Applied to create, update, and duplicate agent paths. * 🛡️ fix: Harden MCP tool authorization and add test coverage Addresses review findings on the MCP agent tool authorization fix: - Wrap getMCPServersRegistry() in try/catch so uninitialized registry gracefully filters all MCP tools instead of causing a 500 (DoS risk) - Guard revertAgentVersionHandler: filter unauthorized MCP tools after reverting to a previous version snapshot - Preserve existing MCP tools on collaborative updates: only validate newly added tools, preventing silent stripping of tools the editing user lacks direct access to - Add audit logging (logger.warn) when MCP tools are rejected - Refactor to single-pass lazy-fetch (registry queried only on first MCP tool encountered) - Export filterAuthorizedTools for direct unit testing - Add 18 tests covering: authorized/unauthorized/mixed tools, registry unavailable fallback, create/update/duplicate/revert handler paths, collaborative update preservation, and mcpServerNames persistence * test: Add duplicate handler test, use Constants.mcp_delimiter, DB assertions - N1: Add duplicateAgentHandler integration test verifying unauthorized MCP tools are stripped from the cloned agent and mcpServerNames are correctly persisted in the database - N2: Replace all hardcoded '_mcp_' delimiter literals with Constants.mcp_delimiter to prevent silent false-positive tests if the delimiter value ever changes - N3: Add DB state assertion to the revert-with-strip test confirming persisted tools match the response after unauthorized tools are removed * fix: Enforce exact 2-segment format for MCP tool keys Reject MCP tool keys with multiple delimiters to prevent authorization/execution mismatch when `.pop()` vs `split[1]` extract different server names from the same key. * fix: Preserve existing MCP tools when registry is unavailable When the MCP registry is uninitialized (e.g. server restart), existing tools already persisted on the agent are preserved instead of silently stripped. New MCP tools are still rejected when the registry cannot verify them. Applies to duplicate and revert handlers via existingTools param; update handler already preserves existing tools via its diff logic.	2026-03-15 20:08:34 -04:00
Danny Avila	ad08df4db6	🔏 fix: Scope Agent-Author File Access to Attached Files Only (#12251) ... * 🛡️ fix: Scope agent-author file access to attached files only The hasAccessToFilesViaAgent helper short-circuited for agent authors, granting access to all requested file IDs without verifying they were attached to the agent's tool_resources. This enabled an IDOR where any agent author could delete arbitrary files by supplying their agent_id alongside unrelated file IDs. Now both the author and non-author paths check file IDs against the agent's tool_resources before granting access. * chore: Use Object.values/for...of and add JSDoc in getAttachedFileIds * test: Add boundary cases for agent file access authorization - Agent with no tool_resources denies all access (fail-closed) - Files across multiple resource types are all reachable - Author + isDelete: true still scopes to attached files only	2026-03-15 18:54:34 -04:00
Danny Avila	bcf45519bd	🪪 fix: Enforce VIEW ACL on Agent Edge References at Write and Runtime (#12246) ... * 🛡️ fix: Enforce ACL checks on handoff edge and added-convo agent loading Edge-linked agents and added-convo agents were fetched by ID via getAgent without verifying the requesting user's access permissions. This allowed an authenticated user to reference another user's private agent in edges or addedConvo and have it initialized at runtime. Add checkPermission(VIEW) gate in processAgent before initializing any handoff agent, and in processAddedConvo for non-ephemeral added agents. Unauthorized agents are logged and added to skippedAgentIds so orphaned-edge filtering removes them cleanly. * 🛡️ fix: Validate edge agent access at agent create/update time Reject agent create/update requests that reference agents in edges the requesting user cannot VIEW. This provides early feedback and prevents storing unauthorized agent references as defense-in-depth alongside the runtime ACL gate in processAgent. Add collectEdgeAgentIds utility to extract all unique agent IDs from an edge array, and validateEdgeAgentAccess helper in the v1 handler. * 🧪 test: Improve ACL gate test coverage and correctness - Add processAgent ACL gate tests for initializeClient (skip/allow handoff agents) - Fix addedConvo.spec.js to mock loadAddedAgent directly instead of getAgent - Seed permMap with ownedAgent VIEW bits in v1.spec.js update-403 test * 🧹 chore: Remove redundant addedConvo ACL gate (now in middleware) PR #12243 moved the addedConvo agent ACL check upstream into canAccessAgentFromBody middleware, making the runtime check in processAddedConvo and its spec redundant. * 🧪 test: Rewrite processAgent ACL test with real DB and minimal mocking Replace heavy mock-based test (12 mocks, Providers.XAI crash) with MongoMemoryServer-backed integration test that exercises real getAgent, checkPermission, and AclEntry — only external I/O (initializeAgent, ToolService, AgentClient) remains mocked. Load edge utilities directly from packages/api/src/agents/edges to sidestep the config.ts barrel. * 🧪 fix: Use requireActual spread for @librechat/agents and @librechat/api mocks The Providers.XAI crash was caused by mocking @librechat/agents with a minimal replacement object, breaking the @librechat/api initialization chain. Match the established pattern from client.test.js and recordCollectedUsage.spec.js: spread jest.requireActual for both packages, overriding only the functions under test.	2026-03-15 18:08:57 -04:00
Danny Avila	8dc6d60750	🛡️ fix: Enforce MULTI_CONVO and agent ACL checks on addedConvo (#12243) ... * 🛡️ fix: Enforce MULTI_CONVO and agent ACL checks on addedConvo addedConvo.agent_id was passed through to loadAddedAgent without any permission check, enabling an authenticated user to load and execute another user's private agent via the parallel multi-convo feature. The middleware now chains a checkAddedConvoAccess gate after the primary agent check: when req.body.addedConvo is present it verifies the user has MULTI_CONVO:USE role permission, and when the addedConvo agent_id is a real (non-ephemeral) agent it runs the same canAccessResource ACL check used for the primary agent. * refactor: Harden addedConvo middleware and avoid duplicate agent fetch - Convert checkAddedConvoAccess to curried factory matching Express middleware signature: (requiredPermission) => (req, res, next) - Call checkPermission directly for the addedConvo agent instead of routing through canAccessResource's tempReq pattern; this avoids orphaning the resolved agent document and enables caching it on req.resolvedAddedAgent for downstream loadAddedAgent - Update loadAddedAgent to use req.resolvedAddedAgent when available, eliminating a duplicate getAgent DB call per chat request - Validate addedConvo is a plain object and agent_id is a string before passing to isEphemeralAgentId (prevents TypeError on object injection, returns 400-equivalent early exit instead of 500) - Fix JSDoc: "VIEW access" → "same permission as primary agent", add @param/@returns to helpers, restore @example on factory - Fix redundant return await in resolveAgentIdFromBody * test: Add canAccessAgentFromBody spec covering IDOR fix 26 integration tests using MongoMemoryServer with real models, ACL entries, and PermissionService — no mocks for core logic. Covered paths: - Factory validation (requiredPermission type check) - Primary agent: missing agent_id, ephemeral, non-agents endpoint - addedConvo absent / invalid shape (string, array, object injection) - MULTI_CONVO:USE gate: denied, missing role, ADMIN bypass - Agent resource ACL: no ACL → 403, insufficient bits → 403, nonexistent agent → 404, valid ACL → next + cached on req - End-to-end: both real agents, primary denied short-circuits, ephemeral primary + real addedConvo	2026-03-15 17:12:45 -04:00
Danny Avila	0c27ad2d55	🛡️ refactor: Scope Action Mutations by Parent Resource Ownership (#12237) ... * 🛡️ fix: Scope action mutations by parent resource ownership Prevent cross-tenant action overwrites by validating that an existing action's agent_id/assistant_id matches the URL parameter before allowing updates or deletes. Without this, a user with EDIT access on their own agent could reference a foreign action_id to hijack another agent's action record. * 🛡️ fix: Harden action ownership checks and scope write filters - Remove && short-circuit that bypassed the guard when agent_id or assistant_id was falsy (e.g. assistant-owned actions have no agent_id, so the check was skipped entirely on the agents route). - Include agent_id / assistant_id in the updateAction and deleteAction query filters so the DB write itself enforces ownership atomically. - Log a warning when deleteAction returns null (silent no-op from data-integrity mismatch). * 📝 docs: Update Action model JSDoc to reflect scoped query params * ✅ test: Add Action ownership scoping tests Cover update, delete, and cross-type protection scenarios using MongoMemoryServer to verify that scoped query filters (agent_id, assistant_id) prevent cross-tenant overwrites and deletions at the database level. * 🛡️ fix: Scope updateAction filter in agent duplication handler * 🐛 fix: Use action metadata domain instead of action_id when duplicating agent actions The duplicate handler was splitting `action.action_id` by `actionDelimiter` to extract the domain, but `action_id` is a bare nanoid that doesn't contain the delimiter. This produced malformed entries in the duplicated agent's actions array (nanoid_action_newNanoid instead of domain_action_newNanoid). The domain is available on `action.metadata.domain`. * ✅ test: Add integration tests for agent duplication action handling Uses MongoMemoryServer with real Agent and Action models to verify: - Duplicated actions use metadata.domain (not action_id) for the agent actions array entries - Sensitive metadata fields are stripped from duplicated actions - Original action documents are not modified	2026-03-15 10:19:29 -04:00
Danny Avila	7bc793b18d	🌊 fix: Prevent Buffered Event Duplication on SSE Resume Connections (#12225) ... * fix: skipBufferReplay for job resume connections - Introduced a new option `skipBufferReplay` in the `subscribe` method of `GenerationJobManagerClass` to prevent duplication of events when resuming a connection. - Updated the logic to conditionally skip replaying buffered events if a sync event has already been sent, enhancing the efficiency of event handling during reconnections. - Added integration tests to verify the correct behavior of the new option, ensuring that no buffered events are replayed when `skipBufferReplay` is true, while still allowing for normal replay behavior when false. * refactor: Update GenerationJobManager to handle sync events more efficiently - Modified the `subscribe` method to utilize a new `skipBufferReplay` option, allowing for the prevention of duplicate events during resume connections. - Enhanced the logic in the `chat/stream` route to conditionally skip replaying buffered events if a sync event has already been sent, improving event handling efficiency. - Updated integration tests to verify the correct behavior of the new option, ensuring that no buffered events are replayed when `skipBufferReplay` is true, while maintaining normal replay behavior when false. * test: Enhance GenerationJobManager integration tests for Redis mode - Updated integration tests to conditionally run based on the USE_REDIS environment variable, allowing for better control over Redis-related tests. - Refactored test descriptions to utilize a dynamic `describeRedis` function, improving clarity and organization of tests related to Redis functionality. - Removed redundant checks for Redis availability within individual tests, streamlining the test logic and enhancing readability. * fix: sync handler state for new messages on resume The sync event's else branch (new response message) was missing resetContentHandler() and syncStepMessage() calls, leaving stale handler state that caused subsequent deltas to build on partial content instead of the synced aggregatedContent. * feat: atomic subscribeWithResume to close resume event gap Replaces separate getResumeState() + subscribe() calls with a single subscribeWithResume() that atomically drains earlyEventBuffer between the resume snapshot and the subscribe. In in-memory mode, drained events are returned as pendingEvents for the client to replay after sync. In Redis mode, pendingEvents is empty since chunks are already persisted. The route handler now uses the atomic method for resume connections and extracted shared SSE write helpers to reduce duplication. The client replays any pendingEvents through the existing step/content handlers after applying aggregatedContent from the sync payload. * fix: only capture gap events in subscribeWithResume, not pre-snapshot buffer The previous implementation drained the entire earlyEventBuffer into pendingEvents, but pre-snapshot events are already reflected in aggregatedContent. Replaying them re-introduced the duplication bug through a different vector. Now records buffer length before getResumeState() and slices from that index, so only events arriving during the async gap are returned as pendingEvents. Also: - Handle pendingEvents when resumeState is null (replay directly) - Hoist duplicate test helpers to shared scope - Remove redundant writableEnded guard in onDone	2026-03-14 10:54:26 -04:00
Danny Avila	f67bbb2bc5	🧹 fix: Sanitize Artifact Filenames in Code Execution Output (#12222) ... * fix: sanitize artifact filenames to prevent path traversal in code output * test: Mock sanitizeFilename function in process.spec.js to return the original filename - Added a mock implementation for the `sanitizeFilename` function in the `process.spec.js` test file to return the original filename, ensuring that tests can run without altering the filename during the testing process. * fix: use path.relative for traversal check, sanitize all filenames, add security logging - Replace startsWith with path.relative pattern in saveLocalBuffer, consistent with deleteLocalFile and getLocalFileStream in the same file - Hoist sanitizeFilename call before the image/non-image branch so both code paths store the sanitized name in MongoDB - Log a warning when sanitizeFilename mutates a filename (potential traversal) - Log a specific warning when saveLocalBuffer throws a traversal error, so security events are distinguishable from generic network errors in the catch * test: improve traversal test coverage and remove mock reimplementation - Remove partial sanitizeFilename reimplementation from process-traversal tests; use controlled mock returns to verify processCodeOutput wiring instead - Add test for image branch sanitization - Use mkdtempSync for test isolation in crud-traversal to avoid parallel worker collisions - Add prefix-collision bypass test case (../user10/evil vs user1 directory) * fix: use path.relative in isValidPath to prevent prefix-collision bypass Pre-existing startsWith check without path separator had the same class of prefix-collision vulnerability fixed in saveLocalBuffer.	2026-03-14 03:09:26 -04:00
Danny Avila	35a35dc2e9	📏 refactor: Add File Size Limits to Conversation Imports (#12221) ... * fix: add file size limits to conversation import multer instance * fix: address review findings for conversation import file size limits * fix: use local jest.mock for data-schemas instead of global moduleNameMapper The global @librechat/data-schemas mock in jest.config.js only provided logger, breaking all tests that depend on createModels from the same package. Replace with a virtual jest.mock scoped to the import spec file. * fix: move import to top of file, pre-compute upload middleware, assert logger.warn in tests * refactor: move resolveImportMaxFileSize to packages/api New backend logic belongs in packages/api as TypeScript. Delete the api/server/utils/import/limits.js wrapper and import directly from @librechat/api in convos.js and importConversations.js. Resolver unit tests move to packages/api; the api/ spec retains only multer behavior tests. * chore: rename importLimits to import * fix: stale type reference and mock isolation in import tests Update typeof import path from '../importLimits' to '../import' after the rename. Clear mockLogger.warn in beforeEach to prevent cross-test accumulation. * fix: add resolveImportMaxFileSize to @librechat/api mock in convos.spec.js * fix: resolve jest.mock hoisting issue in import tests jest.mock factories are hoisted above const declarations, so the mockLogger reference was undefined at factory evaluation time. Use a direct import of the mocked logger module instead. * fix: remove virtual flag from data-schemas mock for CI compatibility virtual: true prevents the mock from intercepting the real module in CI where @librechat/data-schemas is built, causing import.ts to use the real logger while the test asserts against the mock.	2026-03-14 03:06:29 -04:00
Danny Avila	c6982dc180	🛡️ fix: Agent Permission Check on Image Upload Route (#12219) ... * fix: add agent permission check to image upload route * refactor: remove unused SystemRoles import and format test file for clarity * fix: address review findings for image upload agent permission check * refactor: move agent upload auth logic to TypeScript in packages/api Extract pure authorization logic from agentPermCheck.js into checkAgentUploadAuth() in packages/api/src/files/agentUploadAuth.ts. The function returns a structured result ({ allowed, status, error }) instead of writing HTTP responses directly, eliminating the dual responsibility and confusing sentinel return value. The JS wrapper in /api is now a thin adapter that translates the result to HTTP. * test: rewrite image upload permission tests as integration tests Replace mock-heavy images-agent-perm.spec.js with integration tests using MongoMemoryServer, real models, and real PermissionService. Follows the established pattern in files.agents.test.js. Moves test to sibling location (images.agents.test.js) matching backend convention. Adds temp file cleanup assertions on 403/404 responses and covers message_file exemption paths (boolean true, string "true", false). * fix: widen AgentUploadAuthDeps types to accept ObjectId from Mongoose The injected getAgent returns Mongoose documents where _id and author are Types.ObjectId at runtime, not string. Widen the DI interface to accept string | Types.ObjectId for _id, author, and resourceId so the contract accurately reflects real callers. * chore: move agent upload auth into files/agents/ subdirectory * refactor: delete agentPermCheck.js wrapper, move verifyAgentUploadPermission to packages/api The /api-only dependencies (getAgent, checkPermission) are now passed as object-field params from the route call sites. Both images.js and files.js import verifyAgentUploadPermission from @librechat/api and inject the deps directly, eliminating the intermediate JS wrapper. * style: fix import type ordering in agent upload auth * fix: prevent token TTL race in MCPTokenStorage.storeTokens When expires_in is provided, use it directly instead of round-tripping through Date arithmetic. The previous code computed accessTokenExpiry as a Date, then after an async encryptV2 call, recomputed expiresIn by subtracting Date.now(). On loaded CI runners the elapsed time caused Math.floor to truncate to 0, triggering the 1-year fallback and making the token appear permanently valid — so refresh never fired.	2026-03-14 02:57:56 -04:00
Danny Avila	71a3b48504	🔑 fix: Require OTP Verification for 2FA Re-Enrollment and Backup Code Regeneration (#12223) ... * fix: require OTP verification for 2FA re-enrollment and backup code regeneration * fix: require OTP verification for account deletion when 2FA is enabled * refactor: Improve code formatting and readability in TwoFactorController and UserController - Reformatted code in TwoFactorController and UserController for better readability by aligning parameters and breaking long lines. - Updated test cases in deleteUser.spec.js and TwoFactorController.spec.js to enhance clarity by formatting object parameters consistently. * refactor: Consolidate OTP and backup code verification logic in TwoFactorController and UserController - Introduced a new `verifyOTPOrBackupCode` function to streamline the verification process for TOTP tokens and backup codes across multiple controllers. - Updated the `enable2FA`, `disable2FA`, and `deleteUserController` methods to utilize the new verification function, enhancing code reusability and readability. - Adjusted related tests to reflect the changes in verification logic, ensuring consistent behavior across different scenarios. - Improved error handling and response messages for verification failures, providing clearer feedback to users. * chore: linting * refactor: Update BackupCodesItem component to enhance OTP verification logic - Consolidated OTP input handling by moving the 2FA verification UI logic to a more consistent location within the component. - Improved the state management for OTP readiness, ensuring the regenerate button is only enabled when the OTP is ready. - Cleaned up imports by removing redundant type imports, enhancing code clarity and maintainability. * chore: lint * fix: stage 2FA re-enrollment in pending fields to prevent disarmament window enable2FA now writes to pendingTotpSecret/pendingBackupCodes instead of overwriting the live fields. confirm2FA performs the atomic swap only after the new TOTP code is verified. If the user abandons mid-flow, their existing 2FA remains active and intact.	2026-03-14 01:51:31 -04:00
Danny Avila	189cdf581d	🔐 fix: Add User Filter to Message Deletion (#12220) ... * fix: add user filter to message deletion to prevent IDOR * refactor: streamline DELETE request syntax in messages-delete test - Simplified the DELETE request syntax in the messages-delete.spec.js test file by combining multiple lines into a single line for improved readability. This change enhances the clarity of the test code without altering its functionality. * fix: address review findings for message deletion IDOR fix * fix: add user filter to message deletion in conversation tests - Included a user filter in the message deletion test to ensure proper handling of user-specific deletions, enhancing the accuracy of the test case and preventing potential IDOR vulnerabilities. * chore: lint	2026-03-13 23:42:37 -04:00
Danny Avila	ca79a03135	🚦 fix: Add Rate Limiting to Conversation Duplicate Endpoint (#12218) ... * fix: add rate limiting to conversation duplicate endpoint * chore: linter * fix: address review findings for conversation duplicate rate limiting * refactor: streamline test mocks for conversation routes - Consolidated mock implementations into a dedicated `convos-route-mocks.js` file to enhance maintainability and readability of test files. - Updated tests in `convos-duplicate-ratelimit.spec.js` and `convos.spec.js` to utilize the new mock structure, improving clarity and reducing redundancy. - Enhanced the `duplicateConversation` function to accept an optional title parameter for better flexibility in conversation duplication. * chore: rename files	2026-03-13 23:40:44 -04:00
Danny Avila	fa9e1b228a	🪪 fix: MCP API Responses and OAuth Validation (#12217) ... * 🔒 fix: Validate MCP Configs in Server Responses * 🔒 fix: Enhance OAuth URL Validation in MCPOAuthHandler - Introduced validation for OAuth URLs to ensure they do not target private or internal addresses, enhancing security against SSRF attacks. - Updated the OAuth flow to validate both authorization and token URLs before use, ensuring compliance with security standards. - Refactored redirect URI handling to streamline the OAuth client registration process. - Added comprehensive error handling for invalid URLs, improving robustness in OAuth interactions. * 🔒 feat: Implement Permission Checks for MCP Server Management - Added permission checkers for MCP server usage and creation, enhancing access control. - Updated routes for reinitializing MCP servers and retrieving authentication values to include these permission checks, ensuring only authorized users can access these functionalities. - Refactored existing permission logic to improve clarity and maintainability. * 🔒 fix: Enhance MCP Server Response Validation and Redaction - Updated MCP route tests to use `toMatchObject` for better validation of server response structures, ensuring consistency in expected properties. - Refactored the `redactServerSecrets` function to streamline the removal of sensitive information, ensuring that user-sourced API keys are properly redacted while retaining their source. - Improved OAuth security tests to validate rejection of private URLs across multiple endpoints, enhancing protection against SSRF vulnerabilities. - Added comprehensive tests for the `redactServerSecrets` function to ensure proper handling of various server configurations, reinforcing security measures. * chore: eslint * 🔒 fix: Enhance OAuth Server URL Validation in MCPOAuthHandler - Added validation for discovered authorization server URLs to ensure they meet security standards. - Improved logging to provide clearer insights when an authorization server is found from resource metadata. - Refactored the handling of authorization server URLs to enhance robustness against potential security vulnerabilities. * 🔒 test: Bypass SSRF validation for MCP OAuth Flow tests - Mocked SSRF validation functions to allow tests to use real local HTTP servers, facilitating more accurate testing of the MCP OAuth flow. - Updated test setup to ensure compatibility with the new mocking strategy, enhancing the reliability of the tests. * 🔒 fix: Add Validation for OAuth Metadata Endpoints in MCPOAuthHandler - Implemented checks for the presence and validity of registration and token endpoints in the OAuth metadata, enhancing security by ensuring that these URLs are properly validated before use. - Improved error handling and logging to provide better insights during the OAuth metadata processing, reinforcing the robustness of the OAuth flow. * 🔒 refactor: Simplify MCP Auth Values Endpoint Logic - Removed redundant permission checks for accessing the MCP server resource in the auth-values endpoint, streamlining the request handling process. - Consolidated error handling and response structure for improved clarity and maintainability. - Enhanced logging for better insights during the authentication value checks, reinforcing the robustness of the endpoint. * 🔒 test: Refactor LeaderElection Integration Tests for Improved Cleanup - Moved Redis key cleanup to the beforeEach hook to ensure a clean state before each test. - Enhanced afterEach logic to handle instance resignations and Redis key deletion more robustly, improving test reliability and maintainability.	2026-03-13 23:18:56 -04:00
Danny Avila	f32907cd36	🔏 fix: MCP Server URL Schema Validation (#12204) ... * fix: MCP server configuration validation and schema - Added tests to reject URLs containing environment variable references for SSE, streamable-http, and websocket types in the MCP routes. - Introduced a new schema in the data provider to ensure user input URLs do not resolve environment variables, enhancing security against potential leaks. - Updated existing MCP server user input schema to utilize the new validation logic, ensuring consistent handling of user-supplied URLs across the application. * fix: MCP URL validation to reject env variable references - Updated tests to ensure that URLs for SSE, streamable-http, and websocket types containing environment variable patterns are rejected, improving security against potential leaks. - Refactored the MCP server user input schema to enforce stricter validation rules, preventing the resolution of environment variables in user-supplied URLs. - Introduced new test cases for various URL types to validate the rejection logic, ensuring consistent handling across the application. * test: Enhance MCPServerUserInputSchema tests for environment variable handling - Introduced new test cases to validate the prevention of environment variable exfiltration through user input URLs in the MCPServerUserInputSchema. - Updated existing tests to confirm that URLs containing environment variable patterns are correctly resolved or rejected, improving security against potential leaks. - Refactored test structure to better organize environment variable handling scenarios, ensuring comprehensive coverage of edge cases.	2026-03-12 23:19:31 -04:00
Danny Avila	9a5d7eaa4e	⚡ refactor: Replace tiktoken with ai-tokenizer (#12175) ... * chore: Update dependencies by adding ai-tokenizer and removing tiktoken - Added ai-tokenizer version 1.0.6 to package.json and package-lock.json across multiple packages. - Removed tiktoken version 1.0.15 from package.json and package-lock.json in the same locations, streamlining dependency management. * refactor: replace js-tiktoken with ai-tokenizer - Added support for 'claude' encoding in the AgentClient class to improve model compatibility. - Updated Tokenizer class to utilize 'ai-tokenizer' for both 'o200k_base' and 'claude' encodings, replacing the previous 'tiktoken' dependency. - Refactored tests to reflect changes in tokenizer behavior and ensure accurate token counting for both encoding types. - Removed deprecated references to 'tiktoken' and adjusted related tests for improved clarity and functionality. * chore: remove tiktoken mocks from DALLE3 tests - Eliminated mock implementations of 'tiktoken' from DALLE3-related test files to streamline test setup and align with recent dependency updates. - Adjusted related test structures to ensure compatibility with the new tokenizer implementation. * chore: Add distinct encoding support for Anthropic Claude models - Introduced a new method `getEncoding` in the AgentClient class to handle the specific BPE tokenizer for Claude models, ensuring compatibility with the distinct encoding requirements. - Updated documentation to clarify the encoding logic for Claude and other models. * docs: Update return type documentation for getEncoding method in AgentClient - Clarified the return type of the getEncoding method to specify that it can return an EncodingName or undefined, enhancing code readability and type safety. * refactor: Tokenizer class and error handling - Exported the EncodingName type for broader usage. - Renamed encodingMap to encodingData for clarity. - Improved error handling in getTokenCount method to ensure recovery attempts are logged and return 0 on failure. - Updated countTokens function documentation to specify the use of 'o200k_base' encoding. * refactor: Simplify encoding documentation and export type - Updated the getEncoding method documentation to clarify the default behavior for non-Anthropic Claude models. - Exported the EncodingName type separately from the Tokenizer module for improved clarity and usage. * test: Update text processing tests for token limits - Adjusted test cases to handle smaller text sizes, changing scenarios from ~120k tokens to ~20k tokens for both the real tokenizer and countTokens functions. - Updated token limits in tests to reflect new constraints, ensuring tests accurately assess performance and call reduction. - Enhanced console log messages for clarity regarding token counts and reductions in the updated scenarios. * refactor: Update Tokenizer imports and exports - Moved Tokenizer and countTokens exports to the tokenizer module for better organization. - Adjusted imports in memory.ts to reflect the new structure, ensuring consistent usage across the codebase. - Updated memory.test.ts to mock the Tokenizer from the correct module path, enhancing test accuracy. * refactor: Tokenizer initialization and error handling - Introduced an async `initEncoding` method to preload tokenizers, improving performance and accuracy in token counting. - Updated `getTokenCount` to handle uninitialized tokenizers more gracefully, ensuring proper recovery and logging on errors. - Removed deprecated synchronous tokenizer retrieval, streamlining the overall tokenizer management process. * test: Enhance tokenizer tests with initialization and encoding checks - Added `beforeAll` hooks to initialize tokenizers for 'o200k_base' and 'claude' encodings before running tests, ensuring proper setup. - Updated tests to validate the loading of encodings and the correctness of token counts for both 'o200k_base' and 'claude'. - Improved test structure to deduplicate concurrent initialization calls, enhancing performance and reliability.	2026-03-10 23:14:52 -04:00
Danny Avila	fcb344da47	🛂 fix: MCP OAuth Race Conditions, CSRF Fallback, and Token Expiry Handling (#12171) ... * fix: Implement race conditions in MCP OAuth flow - Added connection mutex to coalesce concurrent `getUserConnection` calls, preventing multiple simultaneous attempts. - Enhanced flow state management to retry once when a flow state is missing, improving resilience against race conditions. - Introduced `ReauthenticationRequiredError` for better error handling when access tokens are expired or missing. - Updated tests to cover new race condition scenarios and ensure proper handling of OAuth flows. * fix: Stale PENDING flow detection and OAuth URL re-issuance PENDING flows in handleOAuthRequired now check createdAt age — flows older than 2 minutes are treated as stale and replaced instead of joined. Fixes the case where a leftover PENDING flow from a previous session blocks new OAuth initiation. authorizationUrl is now stored in MCPOAuthFlowMetadata so that when a second caller joins an active PENDING flow (e.g., the SSE-emitting path in ToolService), it can re-issue the URL to the user via oauthStart. * fix: CSRF fallback via active PENDING flow in OAuth callback When the OAuth callback arrives without CSRF or session cookies (common in the chat/SSE flow where cookies can't be set on streaming responses), fall back to validating that a PENDING flow exists for the flowId. This is safe because the flow was created server-side after JWT authentication and the authorization code is PKCE-protected. * test: Extract shared OAuth test server helpers Move MockKeyv, getFreePort, trackSockets, and createOAuthMCPServer into a shared helpers/oauthTestServer module. Enhance the test server with refresh token support, token rotation, metadata discovery, and dynamic client registration endpoints. Add InMemoryTokenStore for token storage tests. Refactor MCPOAuthRaceCondition.test.ts to import from shared helpers. * test: Add comprehensive MCP OAuth test modules MCPOAuthTokenStorage — 21 tests for storeTokens/getTokens with InMemoryTokenStore: encrypt/decrypt round-trips, expiry calculation, refresh callback wiring, ReauthenticationRequiredError paths. MCPOAuthFlow — 10 tests against real HTTP server: token refresh with stored client info, refresh token rotation, metadata discovery, dynamic client registration, full store/retrieve/expire/refresh lifecycle. MCPOAuthConnectionEvents — 5 tests for MCPConnection OAuth event cycle with real OAuth-gated MCP server: oauthRequired emission on 401, oauthHandled reconnection, oauthFailed rejection, token expiry detection. MCPOAuthTokenExpiry — 12 tests for the token expiry edge case: refresh success/failure paths, ReauthenticationRequiredError, PENDING flow CSRF fallback, authorizationUrl metadata storage, full re-auth cycle after refresh failure, concurrent expired token coalescing, stale PENDING flow detection. * test: Enhance MCP OAuth connection tests with cooldown reset Added a `beforeEach` hook to clear the cooldown for `MCPConnection` before each test, ensuring a clean state. Updated the race condition handling in the tests to properly clear the timeout, improving reliability in the event data retrieval process. * refactor: PENDING flow management and state recovery in MCP OAuth - Introduced a constant `PENDING_STALE_MS` to define the age threshold for PENDING flows, improving the handling of stale flows. - Updated the logic in `MCPConnectionFactory` and `FlowStateManager` to check the age of PENDING flows before joining or reusing them. - Modified the `completeFlow` method to return false when the flow state is deleted, ensuring graceful handling of race conditions. - Enhanced tests to validate the new behavior and ensure robustness against state recovery issues. * refactor: MCP OAuth flow management and testing - Updated the `completeFlow` method to log warnings when a tool flow state is not found during completion, improving error handling. - Introduced a new `normalizeExpiresAt` function to standardize expiration timestamp handling across the application. - Refactored token expiration checks in `MCPConnectionFactory` to utilize the new normalization function, ensuring consistent behavior. - Added a comprehensive test suite for OAuth callback CSRF fallback logic, validating the handling of PENDING flows and their staleness. - Enhanced existing tests to cover new expiration normalization logic and ensure robust flow state management. * test: Add CSRF fallback tests for active PENDING flows in MCP OAuth - Introduced new tests to validate CSRF fallback behavior when a fresh PENDING flow exists without cookies, ensuring successful OAuth callback handling. - Added scenarios to reject requests when no PENDING flow exists, when only a COMPLETED flow is present, and when a PENDING flow is stale, enhancing the robustness of flow state management. - Improved overall test coverage for OAuth callback logic, reinforcing the handling of CSRF validation failures. * chore: imports order * refactor: Update UserConnectionManager to conditionally manage pending connections - Modified the logic in `UserConnectionManager` to only set pending connections if `forceNew` is false, preventing unnecessary overwrites. - Adjusted the cleanup process to ensure pending connections are only deleted when not forced, enhancing connection management efficiency. * refactor: MCP OAuth flow state management - Introduced a new method `storeStateMapping` in `MCPOAuthHandler` to securely map the OAuth state parameter to the flow ID, improving callback resolution and security against forgery. - Updated the OAuth initiation and callback handling in `mcp.js` to utilize the new state mapping functionality, ensuring robust flow management. - Refactored `MCPConnectionFactory` to store state mappings during flow initialization, enhancing the integrity of the OAuth process. - Adjusted comments to clarify the purpose of state parameters in authorization URLs, reinforcing code readability. * refactor: MCPConnection with OAuth recovery handling - Added `oauthRecovery` flag to manage OAuth recovery state during connection attempts. - Introduced `decrementCycleCount` method to reduce the circuit breaker's cycle count upon successful reconnection after OAuth recovery. - Updated connection logic to reset the `oauthRecovery` flag after handling OAuth, improving state management and connection reliability. * chore: Add debug logging for OAuth recovery cycle count decrement - Introduced a debug log statement in the `MCPConnection` class to track the decrement of the cycle count after a successful reconnection during OAuth recovery. - This enhancement improves observability and aids in troubleshooting connection issues related to OAuth recovery. * test: Add OAuth recovery cycle management tests - Introduced new tests for the OAuth recovery cycle in `MCPConnection`, validating the decrement of cycle counts after successful reconnections. - Added scenarios to ensure that the cycle count is not decremented on OAuth failures, enhancing the robustness of connection management. - Improved test coverage for OAuth reconnect scenarios, ensuring reliable behavior under various conditions. * feat: Implement circuit breaker configuration in MCP - Added circuit breaker settings to `.env.example` for max cycles, cycle window, and cooldown duration. - Refactored `MCPConnection` to utilize the new configuration values from `mcpConfig`, enhancing circuit breaker management. - Improved code maintainability by centralizing circuit breaker parameters in the configuration file. * refactor: Update decrementCycleCount method for circuit breaker management - Changed the visibility of the `decrementCycleCount` method in `MCPConnection` from private to public static, allowing it to be called with a server name parameter. - Updated calls to `decrementCycleCount` in `MCPConnectionFactory` to use the new static method, improving clarity and consistency in circuit breaker management during connection failures and OAuth recovery. - Enhanced the handling of circuit breaker state by ensuring the method checks for the existence of the circuit breaker before decrementing the cycle count. * refactor: cycle count decrement on tool listing failure - Added a call to `MCPConnection.decrementCycleCount` in the `MCPConnectionFactory` to handle cases where unauthenticated tool listing fails, improving circuit breaker management. - This change ensures that the cycle count is decremented appropriately, maintaining the integrity of the connection recovery process. * refactor: Update circuit breaker configuration and logic - Enhanced circuit breaker settings in `.env.example` to include new parameters for failed rounds and backoff strategies. - Refactored `MCPConnection` to utilize the updated configuration values from `mcpConfig`, improving circuit breaker management. - Updated tests to reflect changes in circuit breaker logic, ensuring accurate validation of connection behavior under rapid reconnect scenarios. * feat: Implement state mapping deletion in MCP flow management - Added a new method `deleteStateMapping` in `MCPOAuthHandler` to remove orphaned state mappings when a flow is replaced, preventing old authorization URLs from resolving after a flow restart. - Updated `MCPConnectionFactory` to call `deleteStateMapping` during flow cleanup, ensuring proper management of OAuth states. - Enhanced test coverage for state mapping functionality to validate the new deletion logic.	2026-03-10 21:15:01 -04:00
Danny Avila	6167ce6e57	🧪 chore: MCP Reconnect Storm Follow-Up Fixes and Integration Tests (#12172) ... * 🧪 test: Add reconnection storm regression tests for MCPConnection Introduced a comprehensive test suite for reconnection storm scenarios, validating circuit breaker, throttling, cooldown, and timeout fixes. The tests utilize real MCP SDK transports and a StreamableHTTP server to ensure accurate behavior under rapid connect/disconnect cycles and error handling for SSE 400/405 responses. This enhances the reliability of the MCPConnection by ensuring proper handling of reconnection logic and circuit breaker functionality. * 🔧 fix: Update createUnavailableToolStub to return structured response Modified the `createUnavailableToolStub` function to return an array containing the unavailable message and a null value, enhancing the response structure. Additionally, added a debug log to skip tool creation when the result is null, improving the handling of reconnection scenarios in the MCP service. * 🧪 test: Enhance MCP tool creation tests for cache and throttle interactions Added new test cases for the `createMCPTool` function to validate the caching behavior when tools are unavailable or throttled. The tests ensure that tools are correctly cached as missing and prevent unnecessary reconnects across different users, improving the reliability of the MCP service under concurrent usage scenarios. Additionally, introduced a test for the `createMCPTools` function to verify that it returns an empty array when reconnect is throttled, ensuring proper handling of throttling logic. * 📝 docs: Update AGENTS.md with testing philosophy and guidelines Expanded the testing section in AGENTS.md to emphasize the importance of using real logic over mocks, advocating for the use of spies and real dependencies in tests. Added specific recommendations for testing with MongoDB and MCP SDK, highlighting the need to mock only uncontrollable external services. This update aims to improve testing practices and encourage more robust test implementations. * 🧪 test: Enhance reconnection storm tests with socket tracking and SSE handling Updated the reconnection storm test suite to include a new socket tracking mechanism for better resource management during tests. Improved the handling of SSE 400/405 responses by ensuring they are processed in the same branch as 404 errors, preventing unhandled cases. This enhances the reliability of the MCPConnection under rapid reconnect scenarios and ensures proper error handling. * 🔧 fix: Implement cache eviction for stale reconnect attempts and missing tools Added an `evictStale` function to manage the size of the `lastReconnectAttempts` and `missingToolCache` maps, ensuring they do not exceed a maximum cache size. This enhancement improves resource management by removing outdated entries based on a specified time-to-live (TTL), thereby optimizing the MCP service's performance during reconnection scenarios.	2026-03-10 17:44:13 -04:00
matt burnett	ad5c51f62b	⛈️ fix: MCP Reconnection Storm Prevention with Circuit Breaker, Backoff, and Tool Stubs (#12162) ... * fix: MCP reconnection stability - circuit breaker, throttling, and cooldown retry * Comment and logging cleanup * fix broken tests	2026-03-10 14:21:36 -04:00
Danny Avila	32cadb1cc5	🩹 fix: MCP Server Recovery from Startup Inspection Failures (#12145) ... * feat: MCP server reinitialization recovery mechanism - Added functionality to store a stub configuration for MCP servers that fail inspection at startup, allowing for recovery via reinitialization. - Introduced `reinspectServer` method in `MCPServersRegistry` to handle reinspection of previously failed servers. - Enhanced `MCPServersInitializer` to log and manage server initialization failures, ensuring proper handling of inspection failures. - Added integration tests to verify the recovery process for unreachable MCP servers, ensuring that stub configurations are stored and can be reinitialized successfully. - Updated type definitions to include `inspectionFailed` flag in server configurations for better state management. * fix: MCP server handling for inspection failures - Updated `reinitMCPServer` to return a structured response when the server is unreachable, providing clearer feedback on the failure. - Modified `ConnectionsRepository` to prevent connections to servers marked as inspection failed, improving error handling. - Adjusted `MCPServersRegistry` methods to ensure proper management of server states, including throwing errors for non-failed servers during reinspection. - Enhanced integration tests to validate the behavior of the system when dealing with unreachable MCP servers and inspection failures, ensuring robust recovery mechanisms. * fix: Clear all cached server configurations in MCPServersRegistry - Added a comment to clarify the necessity of clearing all cached server configurations when updating a server's configuration, as the cache is keyed by userId without a reverse index for enumeration. * fix: Update integration test for file_tools_server inspection handling - Modified the test to verify that the `file_tools_server` is stored as a stub when inspection fails, ensuring it can be reinitialized correctly. - Adjusted expectations to confirm that the `inspectionFailed` flag is set to true for the stub configuration, enhancing the robustness of the recovery mechanism. * test: Add unit tests for reinspecting servers in MCPServersRegistry - Introduced tests for the `reinspectServer` method to validate error handling when called on a healthy server and when the server does not exist. - Ensured that appropriate exceptions are thrown for both scenarios, enhancing the robustness of server state management. * test: Add integration test for concurrent reinspectServer calls - Introduced a new test to validate that multiple concurrent calls to reinspectServer do not crash or corrupt the server state. - Ensured that at least one call succeeds and any failures are due to the server not being in a failed state, enhancing the reliability of the reinitialization process. * test: Enhance integration test for concurrent MCP server reinitialization - Added a new test to validate that concurrent calls to reinitialize the MCP server do not crash or corrupt the server state. - Ensured that at least one call succeeds and that failures are handled gracefully, improving the reliability of the reinitialization process. - Reset MCPManager instance after each test to maintain a clean state for subsequent tests.	2026-03-08 21:49:04 -04:00
Lionel Ringenbach	6d0938be64	🔒 refactor: Set ALLOW_SHARED_LINKS_PUBLIC to false by Default (#12100) ... * fix: default ALLOW_SHARED_LINKS_PUBLIC to false for security Shared links were publicly accessible by default when ALLOW_SHARED_LINKS_PUBLIC was not explicitly set, which could lead to unintentional data exposure. Users may assume their authentication settings protect shared links when they do not. This changes the default behavior so shared links require JWT authentication unless ALLOW_SHARED_LINKS_PUBLIC is explicitly set to true. * Document ALLOW_SHARED_LINKS_PUBLIC in .env.example Add comment explaining ALLOW_SHARED_LINKS_PUBLIC setting. --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Danny Avila <danacordially@gmail.com>	2026-03-06 19:05:56 -05:00
Peter Nancarrow	14bcab60b3	🧬 feat: Allow Agent Editors to Duplicate Agents (#12041) ... * feat: allow editors to duplicate agents * fix: Update permissions for duplicating agents and enhance visibility in AgentFooter - Changed required permission for duplicating agents from VIEW to EDIT in the API route. - Updated AgentFooter component to display the duplicate button for admins and users with EDIT permission, improving access control. - Added tests to ensure the duplicate button visibility logic works correctly based on user roles and permissions. * test: Update AgentFooter tests to reflect permission changes - Adjusted tests in AgentFooter.spec.tsx to verify UI behavior based on user permissions. - Updated expectations for the visibility of the grant access dialog and duplicate button, ensuring they align with the new permission logic. --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-03-03 20:45:02 -05:00