* 🪪 feat: Microsoft Graph Access Token Placeholder for MCP Servers (#10867)
* feat: MCP Graph Token env var
* Addressing copilot remarks
* Addressed Copilot review remarks
* Fixed graphtokenservice mock in MCP test suite
* fix: remove unnecessary type check and cast in resolveGraphTokensInRecord
* ci: add Graph Token integration tests in MCPManager
* refactor: update user type definitions to use Partial<IUser> in multiple functions
* test: enhance MCP tests for graph token processing and user placeholder resolution
- Added comprehensive tests to validate the interaction between preProcessGraphTokens and processMCPEnv.
- Ensured correct resolution of graph tokens and user placeholders in various configurations.
- Mocked OIDC utilities to facilitate testing of token extraction and validation.
- Verified that original options remain unchanged after processing.
* chore: import order
* chore: imports
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
* WIP: OpenAI-compatible API for LibreChat agents
- Added OpenAIChatCompletionController for handling chat completions.
- Introduced ListModelsController and GetModelController for listing and retrieving agent details.
- Created routes for OpenAI API endpoints, including /v1/chat/completions and /v1/models.
- Developed event handlers for streaming responses in OpenAI format.
- Implemented request validation and error handling for API interactions.
- Integrated content aggregation and response formatting to align with OpenAI specifications.
This commit establishes a foundational API for interacting with LibreChat agents in a manner compatible with OpenAI's chat completion interface.
* refactor: OpenAI-spec content aggregation for improved performance and clarity
* fix: OpenAI chat completion controller with safe user handling for correct tool loading
* refactor: Remove conversation ID from OpenAI response context and related handlers
* refactor: OpenAI chat completion handling with streaming support
- Introduced a lightweight tracker for streaming responses, allowing for efficient tracking of emitted content and usage metadata.
- Updated the OpenAIChatCompletionController to utilize the new tracker, improving the handling of streaming and non-streaming responses.
- Refactored event handlers to accommodate the new streaming logic, ensuring proper management of tool calls and content aggregation.
- Adjusted response handling to streamline error reporting during streaming sessions.
* WIP: Open Responses API with core service, types, and handlers
- Added Open Responses API module with comprehensive types and enums.
- Implemented core service for processing requests, including validation and input conversion.
- Developed event handlers for streaming responses and non-streaming aggregation.
- Established response building logic and error handling mechanisms.
- Created detailed types for input and output content, ensuring compliance with Open Responses specification.
* feat: Implement response storage and retrieval in Open Responses API
- Added functionality to save user input messages and assistant responses to the database when the `store` flag is set to true.
- Introduced a new endpoint to retrieve stored responses by ID, allowing users to access previous interactions.
- Enhanced the response creation process to include database operations for conversation and message storage.
- Implemented tests to validate the storage and retrieval of responses, ensuring correct behavior for both existing and non-existent response IDs.
* refactor: Open Responses API with additional token tracking and validation
- Added support for tracking cached tokens in response usage, improving token management.
- Updated response structure to include new properties for top log probabilities and detailed usage metrics.
- Enhanced tests to validate the presence and types of new properties in API responses, ensuring compliance with updated specifications.
- Refactored response handling to accommodate new fields and improve overall clarity and performance.
* refactor: Update reasoning event handlers and types for consistency
- Renamed reasoning text events to simplify naming conventions, changing `emitReasoningTextDelta` to `emitReasoningDelta` and `emitReasoningTextDone` to `emitReasoningDone`.
- Updated event types in the API to reflect the new naming, ensuring consistency across the codebase.
- Added `logprobs` property to output events for enhanced tracking of log probabilities.
* feat: Add validation for streaming events in Open Responses API tests
* feat: Implement response.created event in Open Responses API
- Added emitResponseCreated function to emit the response.created event as the first event in the streaming sequence, adhering to the Open Responses specification.
- Updated createResponse function to emit response.created followed by response.in_progress.
- Enhanced tests to validate the order of emitted events, ensuring response.created is triggered before response.in_progress.
* feat: Responses API with attachment event handling
- Introduced `createResponsesToolEndCallback` to handle attachment events in the Responses API, emitting `librechat:attachment` events as per the Open Responses extension specification.
- Updated the `createResponse` function to utilize the new callback for processing tool outputs and emitting attachments during streaming.
- Added helper functions for writing attachment events and defined types for attachment data, ensuring compatibility with the Open Responses protocol.
- Enhanced tests to validate the integration of attachment events within the Responses API workflow.
* WIP: remote agent auth
* fix: Improve loading state handling in AgentApiKeys component
- Updated the rendering logic to conditionally display loading spinner and API keys based on the loading state.
- Removed unnecessary imports and streamlined the component for better readability.
* refactor: Update API key access handling in routes
- Replaced `checkAccess` with `generateCheckAccess` for improved access control.
- Consolidated access checks into a single `checkApiKeyAccess` function, enhancing code readability and maintainability.
- Streamlined route definitions for creating, listing, retrieving, and deleting API keys.
* fix: Add permission handling for REMOTE_AGENT resource type
* feat: Enhance permission handling for REMOTE_AGENT resources
- Updated the deleteAgent and deleteUserAgents functions to handle permissions for both AGENT and REMOTE_AGENT resource types.
- Introduced new functions to enrich REMOTE_AGENT principals and backfill permissions for AGENT owners.
- Modified createAgentHandler and duplicateAgentHandler to grant permissions for REMOTE_AGENT alongside AGENT.
- Added utility functions for retrieving effective permissions for REMOTE_AGENT resources, ensuring consistent access control across the application.
* refactor: Rename and update roles for remote agent access
- Changed role name from API User to Editor in translation files for clarity.
- Updated default editor role ID from REMOTE_AGENT_USER to REMOTE_AGENT_EDITOR in resource configurations.
- Adjusted role localization to reflect the new Editor role.
- Modified access permissions to align with the updated role definitions across the application.
* feat: Introduce remote agent permissions and update access handling
- Added support for REMOTE_AGENTS in permission schemas, including use, create, share, and share_public permissions.
- Updated the interface configuration to include remote agent settings.
- Modified middleware and API key access checks to align with the new remote agent permission structure.
- Enhanced role defaults to incorporate remote agent permissions, ensuring consistent access control across the application.
* refactor: Update AgentApiKeys component and permissions handling
- Refactored the AgentApiKeys component to improve structure and readability, including the introduction of ApiKeysContent for better separation of concerns.
- Updated CreateKeyDialog to accept an onKeyCreated callback, enhancing its functionality.
- Adjusted permission checks in Data component to use REMOTE_AGENTS and USE permissions, aligning with recent permission schema changes.
- Enhanced loading state handling and dialog management for a smoother user experience.
* refactor: Update remote agent access checks in API routes
- Replaced existing access checks with `generateCheckAccess` for remote agents in the API keys and agents routes.
- Introduced specific permission checks for creating, listing, retrieving, and deleting API keys, enhancing access control.
- Improved code structure by consolidating permission handling for remote agents across multiple routes.
* fix: Correct query parameters in ApiKeysContent component
- Updated the useGetAgentApiKeysQuery call to include an object for the enabled parameter, ensuring proper functionality when the component is open.
- This change improves the handling of API key retrieval based on the component's open state.
* feat: Implement remote agents permissions and update API routes
- Added new API route for updating remote agents permissions, enhancing role management capabilities.
- Introduced remote agents permissions handling in the AgentApiKeys component, including a dedicated settings dialog.
- Updated localization files to include new remote agents permission labels for better user experience.
- Refactored data provider to support remote agents permissions updates, ensuring consistent access control across the application.
* feat: Add remote agents permissions to role schema and interface
- Introduced new permissions for REMOTE_AGENTS in the role schema, including USE, CREATE, SHARE, and SHARE_PUBLIC.
- Updated the IRole interface to reflect the new remote agents permissions structure, enhancing role management capabilities.
* feat: Add remote agents settings button to API keys dialog
* feat: Update AgentFooter to include remote agent sharing permissions
- Refactored access checks to incorporate permissions for sharing remote agents.
- Enhanced conditional rendering logic to allow sharing by users with remote agent permissions.
- Improved loading state handling for remote agent permissions, ensuring a smoother user experience.
* refactor: Update API key creation access check and localization strings
- Replaced the access check for creating API keys to use the existing remote agents access check.
- Updated localization strings to correct the descriptions for remote agent permissions, ensuring clarity in user interface.
* fix: resource permission mapping to include remote agents
- Changed the resourceToPermissionMap to use a Partial<Record> for better flexibility.
- Added mapping for REMOTE_AGENT permissions, enhancing the sharing capabilities for remote agents.
* feat: Implement remote access checks for agent models
- Enhanced ListModelsController and GetModelController to include checks for user permissions on remote agents.
- Integrated findAccessibleResources to filter agents based on VIEW permission for REMOTE_AGENT.
- Updated response handling to ensure users can only access agents they have permissions for, improving security and access control.
* fix: Update user parameter type in processUserPlaceholders function
- Changed the user parameter type in the processUserPlaceholders function from Partial<Partial<IUser>> to Partial<IUser> for improved type clarity and consistency.
* refactor: Simplify integration test structure by removing conditional describe
- Replaced conditional describeWithApiKey with a standard describe for all integration tests in responses.spec.js.
- This change enhances test clarity and ensures all tests are executed consistently, regardless of the SKIP_INTEGRATION_TESTS flag.
* test: Update AgentFooter tests to reflect new grant access dialog ID
- Changed test IDs for the grant access dialog in AgentFooter tests to include the resource type, ensuring accurate identification in the test cases.
- This update improves test clarity and aligns with recent changes in the component's implementation.
* test: Enhance integration tests for Open Responses API
- Updated integration tests in responses.spec.js to utilize an authRequest helper for consistent authorization handling across all test cases.
- Introduced a test user and API key creation to improve test setup and ensure proper permission checks for remote agents.
- Added checks for existing access roles and created necessary roles if they do not exist, enhancing test reliability and coverage.
* feat: Extend accessRole schema to include remoteAgent resource type
- Updated the accessRole schema to add 'remoteAgent' to the resourceType enum, enhancing the flexibility of role assignments and permissions management.
* test: refactored test setup to create a minimal Express app for responses routes, enhancing test structure and maintainability.
* test: Enhance abort.spec.js by mocking additional modules for improved test isolation
- Updated the test setup in abort.spec.js to include actual implementations of '@librechat/data-schemas' and '@librechat/api' while maintaining mock functionality.
- This change improves test reliability and ensures that the tests are more representative of the actual module behavior.
* refactor: Update conversation ID generation to use UUID
- Replaced the nanoid with uuidv4 for generating conversation IDs in the createResponse function, enhancing uniqueness and consistency in ID generation.
* test: Add remote agent access roles to AccessRole model tests
- Included additional access roles for remote agents (REMOTE_AGENT_EDITOR, REMOTE_AGENT_OWNER, REMOTE_AGENT_VIEWER) in the AccessRole model tests to ensure comprehensive coverage of role assignments and permissions management.
* chore: Add deletion of user agent API keys in user deletion process
- Updated the user deletion process in UserController and delete-user.js to include the removal of user agent API keys, ensuring comprehensive cleanup of user data upon account deletion.
* test: Add remote agents permissions to permissions.spec.ts
- Enhanced the permissions tests by including comprehensive permission settings for remote agents across various scenarios, ensuring accurate validation of access controls for remote agent roles.
* chore: Update remote agents translations for clarity and consistency
- Removed outdated remote agents translation entries and added revised entries to improve clarity on API key creation and sharing permissions for remote agents. This enhances user understanding of the available functionalities.
* feat: Add indexing and TTL for agent API keys
- Introduced an index on the `key` field for improved query performance.
- Added a TTL index on the `expiresAt` field to enable automatic cleanup of expired API keys, ensuring efficient management of stored keys.
* chore: Update API route documentation for clarity
- Revised comments in the agents route file to clarify the handling of API key authentication.
- Removed outdated endpoint listings to streamline the documentation and focus on current functionality.
---------
Co-authored-by: Max Sanna <max@maxsanna.com>
* feat: MCP Graph Token env var
* Addressing copilot remarks
* Addressed Copilot review remarks
* Fixed graphtokenservice mock in MCP test suite
* fix: remove unnecessary type check and cast in resolveGraphTokensInRecord
* ci: add Graph Token integration tests in MCPManager
* refactor: update user type definitions to use Partial<IUser> in multiple functions
* test: enhance MCP tests for graph token processing and user placeholder resolution
- Added comprehensive tests to validate the interaction between preProcessGraphTokens and processMCPEnv.
- Ensured correct resolution of graph tokens and user placeholders in various configurations.
- Mocked OIDC utilities to facilitate testing of token extraction and validation.
- Verified that original options remain unchanged after processing.
* chore: import order
* chore: imports
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
* feat: add support for inferenceProfiles mapping
* fix: remove friendly name since api requires actual model id for validation alongside inference profile
* docs: more generic description in docs
* chore: address comments
* chore: update peer dependency versions in package.json
- Bump @aws-sdk/client-bedrock-runtime from ^3.941.0 to ^3.970.0
- Update @librechat/agents from ^3.0.78 to ^3.0.79
* fix: update @librechat/agents dependency to version 3.0.80
* test: add unit tests for inference profile configuration in initializeBedrock function
- Introduced tests to validate the applicationInferenceProfile setting based on model configuration.
- Ensured correct handling of environment variables and fallback scenarios for inference profile ARNs.
- Added cases for empty inferenceProfiles and absence of bedrock config to confirm expected behavior.
* fix: update bedrock endpoint schema reference in config
- Changed the bedrock endpoint reference from baseEndpointSchema to bedrockEndpointSchema for improved clarity and accuracy in configuration.
* test: add unit tests for Bedrock endpoint configuration
- Introduced tests to validate the configuration of Bedrock endpoints with models and inference profiles.
- Added scenarios for both complete and minimal configurations to ensure expected behavior.
- Enhanced coverage for the handling of inference profiles without a models array.
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
* refactor: process code output files for re-use (WIP)
* feat: file attachment handling with additional metadata for downloads
* refactor: Update directory path logic for local file saving based on basePath
* refactor: file attachment handling to support TFile type and improve data merging logic
* feat: thread filtering of code-generated files
- Introduced parentMessageId parameter in addedConvo and initialize functions to enhance thread management.
- Updated related methods to utilize parentMessageId for retrieving messages and filtering code-generated files by conversation threads.
- Enhanced type definitions to include parentMessageId in relevant interfaces for better clarity and usage.
* chore: imports/params ordering
* feat: update file model to use messageId for filtering and processing
- Changed references from 'message' to 'messageId' in file-related methods for consistency.
- Added messageId field to the file schema and updated related types.
- Enhanced file processing logic to accommodate the new messageId structure.
* feat: enhance file retrieval methods to support user-uploaded execute_code files
- Added a new method `getUserCodeFiles` to retrieve user-uploaded execute_code files, excluding code-generated files.
- Updated existing file retrieval methods to improve filtering logic and handle edge cases.
- Enhanced thread data extraction to collect both message IDs and file IDs efficiently.
- Integrated `getUserCodeFiles` into relevant endpoints for better file management in conversations.
* chore: update @librechat/agents package version to 3.0.78 in package-lock.json and related package.json files
* refactor: file processing and retrieval logic
- Added a fallback mechanism for download URLs when files exceed size limits or cannot be processed locally.
- Implemented a deduplication strategy for code-generated files based on conversationId and filename to optimize storage.
- Updated file retrieval methods to ensure proper filtering by messageIds, preventing orphaned files from being included.
- Introduced comprehensive tests for new thread data extraction functionality, covering edge cases and performance considerations.
* fix: improve file retrieval tests and handling of optional properties
- Updated tests to safely access optional properties using non-null assertions.
- Modified test descriptions for clarity regarding the exclusion of execute_code files.
- Ensured that the retrieval logic correctly reflects the expected outcomes for file queries.
* test: add comprehensive unit tests for processCodeOutput functionality
- Introduced a new test suite for the processCodeOutput function, covering various scenarios including file retrieval, creation, and processing for both image and non-image files.
- Implemented mocks for dependencies such as axios, logger, and file models to isolate tests and ensure reliable outcomes.
- Validated behavior for existing files, new file creation, and error handling, including size limits and fallback mechanisms.
- Enhanced test coverage for metadata handling and usage increment logic, ensuring robust verification of file processing outcomes.
* test: enhance file size limit enforcement in processCodeOutput tests
- Introduced a configurable file size limit for tests to improve flexibility and coverage.
- Mocked the `librechat-data-provider` to allow dynamic adjustment of file size limits during tests.
- Updated the file size limit enforcement test to validate behavior when files exceed specified limits, ensuring proper fallback to download URLs.
- Reset file size limit after tests to maintain isolation for subsequent test cases.
* feat: implement admin authentication with OpenID & Local Auth proxy support
* feat: implement admin OAuth exchange flow with caching support
- Added caching for admin OAuth exchange codes with a short TTL.
- Introduced new endpoints for generating and exchanging admin OAuth codes.
- Updated relevant controllers and routes to handle admin panel redirects and token exchanges.
- Enhanced logging for better traceability of OAuth operations.
* refactor: enhance OpenID strategy mock to support multiple verify callbacks
- Updated the OpenID strategy mock to store and retrieve verify callbacks by strategy name.
- Improved backward compatibility by maintaining a method to get the last registered callback.
- Adjusted tests to utilize the new callback retrieval methods, ensuring clarity in the verification process for the 'openid' strategy.
* refactor: reorder import statements for better organization
* refactor: admin OAuth flow with improved URL handling and validation
- Added a utility function to retrieve the admin panel URL, defaulting to a local development URL if not set in the environment.
- Updated the OAuth exchange endpoint to include validation for the authorization code format.
- Refactored the admin panel redirect logic to handle URL parsing more robustly, ensuring accurate origin comparisons.
- Removed redundant local URL definitions from the codebase for better maintainability.
* refactor: remove deprecated requireAdmin middleware and migrate to TypeScript
- Deleted the old requireAdmin middleware file and its references in the middleware index.
- Introduced a new TypeScript version of the requireAdmin middleware with enhanced error handling and logging.
- Updated routes to utilize the new requireAdmin middleware, ensuring consistent access control for admin routes.
* feat: add requireAdmin middleware for admin role verification
- Introduced requireAdmin middleware to enforce admin role checks for authenticated users.
- Implemented comprehensive error handling and logging for unauthorized access attempts.
- Added unit tests to validate middleware functionality and ensure proper behavior for different user roles.
- Updated middleware index to include the new requireAdmin export.
* WIP: code ptc
* refactor: tool classification and calling logic
* 🔧 fix: Update @librechat/agents dependency to version 3.0.68
* chore: import order and correct renamed tool name for tool search
* refactor: streamline tool classification logic for local and programmatic tools
* feat: add per-tool configuration options for agents, including deferred loading and allowed callers
- Introduced `tool_options` in agent forms to manage tool behavior.
- Updated tool classification logic to prioritize agent-level configurations.
- Enhanced UI components to support tool deferral functionality.
- Added localization strings for new tool options and actions.
* feat: enhance agent schema with per-tool options for configuration
- Added `tool_options` schema to support per-tool configurations, including `defer_loading` and `allowed_callers`.
- Updated agent data model to incorporate new tool options, ensuring flexibility in tool behavior management.
- Modified type definitions to reflect the new `tool_options` structure for agents.
* feat: add tool_options parameter to loadTools and initializeAgent for enhanced agent configuration
* chore: update @librechat/agents dependency to version 3.0.71 and enhance agent tool loading logic
- Updated the @librechat/agents package to version 3.0.71 across multiple files.
- Added support for handling deferred loading of tools in agent initialization and execution processes.
- Improved the extraction of discovered tools from message history to optimize tool loading behavior.
* chore: update @librechat/agents dependency to version 3.0.72
* chore: update @librechat/agents dependency to version 3.0.75
* refactor: simplify tool defer loading logic in MCPTool component
- Removed local state management for deferred tools, relying on form state instead.
- Updated related functions to directly use form values for checking and toggling defer loading.
- Cleaned up code by eliminating unnecessary optimistic updates and local state dependencies.
* chore: remove deprecated localization strings for tool deferral in translation.json
- Eliminated unused strings related to deferred loading descriptions in the English translation file.
- Streamlined localization to reflect recent changes in tool loading logic.
* refactor: improve tool defer loading handling in MCPTool component
- Enhanced the logic for managing deferred loading of tools by simplifying the update process for tool options.
- Ensured that the state reflects the correct loading behavior based on the new deferred loading conditions.
- Cleaned up the code to remove unnecessary complexity in handling tool options.
* refactor: update agent mocks in callbacks test to use actual implementations
- Modified the agent mocks in the callbacks test to include actual implementations from the @librechat/agents module.
- This change enhances the accuracy of the tests by ensuring they reflect the real behavior of the agent functions.
* chore: Update version from v0.8.2-rc3 to v0.8.2 across multiple files
* chore: Update package versions for @librechat/api to 1.7.22 and @librechat/client to 0.4.51
* chore: Bump version of librechat-data-provider from 0.8.230 to 0.8.231
* chore: Bump version of @librechat/data-schemas to 0.0.35
* chore: bump config version to 1.3.2
* chore: bump config version to 1.3.3
* docs: Update README to include new features for resumable streams and enhanced platform capabilities
* fix: `useMemory` in AgentClient for PrelimAgent Assignment
* Updated the useMemory method in AgentClient to handle prelimAgent assignment based on memory configuration.
* Added logic to return early if prelimAgent is undefined, improving flow control.
* Introduced comprehensive unit tests to validate behavior for various memory configurations, including scenarios for matching and differing agent IDs, as well as handling of ephemeral agents.
* Mocked necessary dependencies in tests to ensure isolation and reliability of the new functionality.
* fix: Update temperature handling for Bedrock and Anthropic providers in memory management
* fix: Replace hardcoded provider strings with constants in memory agent tests
* fix: Replace hardcoded provider string with constant in allowedProviders for AgentClient
* fix: memory agent tests to use actual Providers and GraphEvents constants
* 🔒 feat: Sanitize Placeholders in User-provider MCP Server Config Headers
* Implemented sanitization for dangerous credential placeholders in headers to prevent credential exfiltration when MCP servers are shared.
* Added tests to verify that dangerous placeholders are stripped from headers during both add and update operations, while safe placeholders are preserved.
* Refactored ServerConfigsDB to include a new sanitizeCredentialPlaceholders function for header processing.
* ci: tests for preserving credential placeholders in YAML configs
* Introduced new tests to ensure that LIBRECHAT_OPENID and LIBRECHAT_USER placeholders are preserved in admin configuration headers when added to the cache.
* Validated that the expected placeholders remain intact during retrieval, enhancing the integrity of configuration management.
* 🧑🏫 fix: Multi-Agent Instructions Handling
* Refactored AgentClient to streamline the process of building messages by applying shared run context and agent-specific instructions.
* Introduced new utility functions in context.ts for extracting MCP server names, fetching MCP instructions, and building combined agent instructions.
* Updated the Agent type to make instructions optional, allowing for more flexible agent configurations.
* Improved the handling of context application to agents, ensuring that all relevant information is correctly integrated before execution.
* chore: Update EphemeralAgent Type in Context
* Enhanced the context.ts file by importing the TEphemeralAgent type from librechat-data-provider.
* Updated the applyContextToAgent function to use TEphemeralAgent for the ephemeralAgent parameter, improving type safety and clarity in agent context handling.
* ci: Update Agent Instructions in Tests for Clarity
* Revised test assertions in AgentClient to clarify the source of agent instructions, ensuring they are explicitly referenced as coming from agent configuration rather than build options.
* Updated comments in tests to enhance understanding of the expected behavior regarding base agent instructions and their handling in various scenarios.
* ci: Unit Tests for Agent Context Utilities
* Introduced comprehensive unit tests for agent context utilities, including functions for extracting MCP servers, fetching MCP instructions, and building agent instructions.
* Enhanced test coverage to ensure correct behavior across various scenarios, including handling of empty tools, mixed tool types, and error cases.
* Improved type definitions for AgentWithTools to clarify the structure and requirements for agent context operations.
Fixes ByteString conversion errors when user names contain Unicode
characters > 255 (e.g., ć, đ, ł, š, ž) in MCP server headers.
- Add encodeHeaderValue() function to Base64 encode extended Unicode
- Update processUserPlaceholders() to encode name/username/email in headers
- Update processSingleValue() with isHeader parameter
- Apply encoding in processMCPEnv() and resolveHeaders()
Tested locally with MCP server using user name 'Đorđe' (contains đ=272).
Headers are correctly encoded as base64, preventing ByteString errors.
Co-authored-by: kenzaelk98 <kenzaelk98@leoninestudios.com>
Co-authored-by: heptapod <164861708+leondape@users.noreply.github.com>
* fix: Token Spending Logic for Multi-Agents on Abort Scenarios
* Implemented logic to skip token spending if a conversation is aborted, preventing double-spending.
* Introduced `spendCollectedUsage` function to handle token spending for multiple models during aborts, ensuring accurate accounting for parallel agents.
* Updated `GenerationJobManager` to store and retrieve collected usage data for improved abort handling.
* Added comprehensive tests for the new functionality, covering various scenarios including cache token handling and parallel agent usage.
* fix: Memory Context Handling for Multi-Agents
* Refactored `buildMessages` method to pass memory context to parallel agents, ensuring they share the same user context.
* Improved handling of memory context when no existing instructions are present for parallel agents.
* Added comprehensive tests to verify memory context propagation and behavior under various scenarios, including cases with no memory available and empty agent configurations.
* Enhanced logging for better traceability of memory context additions to agents.
* chore: Memory Context Documentation for Parallel Agents
* Updated documentation in the `AgentClient` class to clarify the in-place mutation of agentConfig objects when passing memory context to parallel agents.
* Added notes on the implications of mutating objects directly to ensure all parallel agents receive the correct memory context before execution.
* chore: UsageMetadata Interface docs for Token Spending
* Expanded the UsageMetadata interface to support both OpenAI and Anthropic cache token formats.
* Added detailed documentation for cache token properties, including mutually exclusive fields for different model types.
* Improved clarity on how to access cache token details for accurate token spending tracking.
* fix: Enhance Token Spending Logic in Abort Middleware
* Refactored `spendCollectedUsage` function to utilize Promise.all for concurrent token spending, improving performance and ensuring all operations complete before clearing the collectedUsage array.
* Added documentation to clarify the importance of clearing the collectedUsage array to prevent double-spending in abort scenarios.
* Updated tests to verify the correct behavior of the spending logic and the clearing of the array after spending operations.
* 🔧 fix: Update @librechat/agents version to 3.0.771 in package.json and package-lock.json
* 🔧 fix: Update @librechat/agents version to 3.0.772 in package.json and package-lock.json
* 🔧 fix: Update @librechat/agents version to 3.0.774 in package.json and package-lock.json
* 🪵 refactor: Preserve job error state for late stream subscribers
* 🔧 fix: Enhance error handling for late subscribers in GenerationJobManager
- Implemented a cleanup strategy for error jobs to prevent immediate deletion, allowing late clients to receive error messages.
- Updated job status handling to prioritize error notifications over completion events.
- Added integration tests to verify error preservation and proper notification to late subscribers, including scenarios with Redis support.
* feat: Bedrock provider support in memory processing
- Introduced support for the Bedrock provider in the memory processing logic.
- Updated the handling of instructions to ensure they are included in user messages for Bedrock, while maintaining the standard approach for other providers.
- Added tests to verify the correct behavior for both Bedrock and non-Bedrock providers regarding instruction handling.
* refactor: Bedrock memory processing logic
- Improved handling of the first message in Bedrock memory processing to ensure proper content is used.
- Added logging for cases where the first message content is not a string.
- Adjusted the processed messages to include the original content or fallback to a new HumanMessage if no messages are present.
* feat: Enhance Bedrock configuration handling in memory processing
- Added logic to set the temperature to 1 when using the Bedrock provider with thinking enabled.
- Ensured compatibility with additional model request fields for improved memory processing.
* Refresh all S3 avatars for this user's accessible agent set, not the first page
* Cleaner debug messages
* Log errors as errors
* refactor: avatar refresh logic to process agents in batches and improve error handling. Introduced new utility functions for refreshing S3 avatars and updating agent records. Updated tests to cover various scenarios including cache hits, user ownership checks, and error handling. Added constants for maximum refresh limits.
* refactor: update avatar refresh logic to allow users with VIEW access to refresh avatars for all accessible agents. Removed checks for agent ownership and author presence, and updated related tests to reflect new behavior.
* chore: Remove YouTube toolkit due to #11331
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
* 🗑️ refactor: Remove YouTube API integration and related configurations as it's broken and should be integrated via MCP instead. Currently there seems not to be a single MCP out there with working get_transcript methods for months. API seems to have changed and there are no maintainers on these projects. We will work out something soon an MCP solution
- Deleted YouTube API key and related configurations from .env.example.
- Removed YouTube tools and their references from the API client, including the manifest and structured files.
- Updated package.json to remove YouTube-related dependencies.
- Cleaned up toolkit exports by removing YouTube toolkit references.
* chore: revert package removal to properly remove packages
* 🗑️ refactor: Remove YouTube API and related dependencies due to integration issues
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
* 🔧 refactor: Improve token calculation in AgentClient.recordCollectedUsage
- Updated the token calculation logic to sum output tokens directly from all entries, addressing issues with negative values in parallel execution scenarios.
- Added comments for clarity on the usage of input tokens and output tokens.
- Introduced a new test file for comprehensive testing of the recordCollectedUsage function, covering various execution scenarios including sequential and parallel processing, cache token handling, and model fallback logic.
* 🔧 refactor: Anthropic `promptCache` handling in LLM configuration
* 🔧 test: Add comprehensive test for cache token handling in recordCollectedUsage
- Introduced a new test case to validate the handling of cache tokens across multiple tool calls in the recordCollectedUsage function.
- Ensured correct calculations for input and output tokens, including scenarios with cache creation and reading.
- Verified the expected interactions with token spending methods to enhance the robustness of the token management logic.
* 🔧 fix: MCP server ID resolver in access permissions (#11315)
- Replaced `findMCPServerById` with `findMCPServerByObjectId` in access permissions route and corresponding tests for improved clarity and consistency in resource identification.
* 🔧 refactor: Update MCP server resource access methods to use server name
- Replaced instances of `findMCPServerById` with `findMCPServerByServerName` across middleware, database, and test files for improved clarity and consistency in resource identification.
- Updated related comments and test cases to reflect the change in method usage.
* chore: Increase timeout for Redis update in GenerationJobManager integration tests
- Updated the timeout duration from 50ms to 200ms in the GenerationJobManager integration tests to ensure reliable verification of final event data in Redis after emitting the done event.
* 🔧 refactor: permission handling for public sharing
- Updated permission keys from SHARED_GLOBAL to SHARE across various files for consistency.
- Added public access configuration in librechat.example.yaml.
- Adjusted related tests and components to reflect the new permission structure.
* chore: Update default SHARE permission to false
* fix: Update SHARE permissions in tests and implementation
- Added SHARE permission handling for user and admin roles in permissions.spec.ts and permissions.ts.
- Updated expected permissions in tests to reflect new SHARE permission values for various permission types.
* fix: Handle undefined values in PeoplePickerAdminSettings component
- Updated the checked and value props of the Switch component to handle undefined values gracefully by defaulting to false. This ensures consistent behavior when the field value is not set.
* feat: Add CREATE permission handling for prompts and agents
- Introduced CREATE permission for user and admin roles in permissions.spec.ts and permissions.ts.
- Updated expected permissions in tests to include CREATE permission for various permission types.
* 🔧 refactor: Enhance permission handling for sharing dialog usability
* refactor: public sharing permissions for resources
- Added middleware to check SHARE_PUBLIC permissions for agents, prompts, and MCP servers.
- Updated interface configuration in librechat.example.yaml to include public sharing options.
- Enhanced components and hooks to support public sharing functionality.
- Adjusted tests to validate new permission handling for public sharing across various resource types.
* refactor: update Share2Icon styling in GenericGrantAccessDialog
* refactor: update Share2Icon size in GenericGrantAccessDialog for consistency
* refactor: improve layout and styling of Share2Icon in GenericGrantAccessDialog
* refactor: update Share2Icon size in GenericGrantAccessDialog for improved consistency
* chore: remove redundant public sharing option from People Picker
* refactor: add SHARE_PUBLIC permission handling in updateInterfacePermissions tests
* ✨ v0.8.2-rc2
* chore: Update package versions in bun.lock and package-lock.json
- Bumped versions for @librechat/api (1.7.10 to 1.7.20), @librechat/client (0.4.3 to 0.4.4), librechat-data-provider (0.8.210 to 0.8.220), and @librechat/data-schemas (0.0.32 to 0.0.33) in relevant files.
* 🔒 fix: Enhance SSRF Protection and Domain Handling in MCP Server Configuration
- Updated the `extractMCPServerDomain` function to return the full origin (protocol://hostname:port) for improved protocol/port matching against allowed domains.
- Enhanced tests for `isMCPDomainAllowed` to validate domain access for internal hostnames and .local TLDs, ensuring proper SSRF protection.
- Added detailed comments in the configuration file to clarify security measures regarding allowed domains and internal target access.
* refactor: Domain Validation for WebSocket Protocols in Action and MCP Handling
- Added comprehensive tests to validate handling of WebSocket URLs in `isActionDomainAllowed` and `isMCPDomainAllowed` functions, ensuring that WebSocket protocols are rejected for OpenAPI Actions while allowed for MCP.
- Updated domain validation logic to support HTTP, HTTPS, WS, and WSS protocols, enhancing security and compliance with specifications.
- Refactored `parseDomainSpec` to improve protocol recognition and validation, ensuring robust handling of domain specifications.
- Introduced detailed comments to clarify the purpose and security implications of domain validation functions.
* feat: Implement cross-replica support in GenerationJobManager for Redis mode
- Enhanced GenerationJobManager to support cross-replica scenarios by lazily creating runtime states from Redis when jobs exist but are not present in local memory.
- Added functionality to persist `syncSent` and `finalEvent` states to Redis for consistency across replicas.
- Implemented abort signal handling to allow replicas to receive and respond to abort requests from other instances, ensuring proper job termination.
- Updated tests to validate cross-replica behavior, including job retrieval, subscription, and abort signal propagation.
This update improves the robustness and reliability of job management in distributed environments.
* fix: Enhance error handling and implement abort signal for cross-replica jobs in GenerationJobManager
- Added error handling for Redis job updates in GenerationJobManager to log failures when persisting `syncSent` and `finalEvent` states.
- Implemented a listener for cross-replica abort signals, ensuring that lazily-initialized jobs can respond to abort requests from other replicas.
- Introduced a new integration test to validate the handling of abort signals for lazily-initialized jobs across replicas.
These changes improve the reliability and robustness of job management in distributed environments.
* feat: add support for designating custom endpoints to use with memory tool
* test: add tests for header resolution in processMemory
* chore: address comments
When users don't explicitly set `maxOutputTokens` in Google/Gemini endpoint
settings, LibreChat was sending `"maxOutputTokens": ""` instead of omitting
the field. This caused issues with Gemini SDK and AI reverse proxies that
expect numeric types.
Changes:
- Add `removeEmptyStrings=true` to `removeNullishValues()` in googleSchema
- Add `removeEmptyStrings=true` to `removeNullishValues()` in compactGoogleSchema
- Add `removeEmptyStrings=true` to `removeNullishValues()` in getGoogleConfig()
- Create comprehensive test suite for Google endpoint (45 tests)
The fix ensures empty strings for numeric fields like maxOutputTokens,
temperature, topP, and topK are properly removed from the request payload.
Closes#11187
* feat: Add Bedrock Guardrails support
* Update packages/data-provider/src/schemas.ts
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Prevent user override of guardrails
* refactor: Bedrock initialization and schema to handle guardrailConfig more effectively
- Updated the initializeBedrock function to apply guardrailConfig conditionally, ensuring it is set only when available.
- Removed guardrailConfig from bedrockInputSchema and bedrockInputParser to streamline input handling.
- Excluded guardrailConfig from tConversationSchema to simplify the schema definition.
These changes enhance the clarity and functionality of the Bedrock initialization process.
* test: Add unit tests for Bedrock initialization
- Introduced comprehensive tests for the initializeBedrock function, covering various configurations including environment variables, user-provided credentials, and guardrail configurations.
- Ensured proper handling of proxy settings and session tokens.
- Validated return structure and edge cases for credentials management.
These tests enhance the reliability and maintainability of the Bedrock initialization process.
---------
Co-authored-by: David Neale <david.neale@admiralfinancialservices.co.uk>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Danny Avila <danny@librechat.ai>
* Added fully functioning Agent Tool supporting Google's Nano Banana
* 🔧 refactor: Update Google credentials handling in GeminiImageGen.js
* Refactored the credentials path to follow a consistent pattern with other Google service integrations, allowing for an environment variable override.
* Updated documentation in README-GeminiNanoBanana.md to reflect the new credentials handling approach and removed references to hardcoded paths.
* 🛠️ refactor: Remove unnecessary whitespace in handleTools.js
* 🔧 feat: Update Gemini Image Generation Tool
- Bump @google/genai package version to ^1.19.0 for improved functionality.
- Refactor GeminiImageGen to createGeminiImageTool for better clarity and consistency.
- Enhance manifest.json for Gemini Image Tools with updated descriptions and icon.
- Add SVG icon for Gemini Image Tools.
- Implement progress tracking for Gemini image generation in the UI.
- Introduce new toolkit and context handling for image generation tools.
This update improves the Gemini image generation capabilities and user experience.
* 🗑️ chore: Remove outdated Gemini image generation PNG and update SVG icon
- Deleted the obsolete PNG file for Gemini image generation.
- Updated the SVG icon with a new design featuring a gradient and shadow effect, enhancing visual appeal and consistency.
* fix: ESLint formatting and unused variable in GeminiImageGen
* fix: Update default model to gemini-2.5-flash-image
* ✨ feat: Enhance Gemini Image Generation Configuration
- Updated .env.example to include new environment variables for Google Cloud region, service account configuration, and Gemini API key options.
- Modified GeminiImageGen.js to support both user-provided API keys and Vertex AI service accounts, improving flexibility in client initialization.
- Updated manifest.json to reflect changes in authentication methods for the Gemini Image Tools.
- Bumped @google/genai package version to 1.19.0 in package-lock.json for compatibility with new features.
* 🔧 fix: Format Default Service Key Path in GeminiImageGen.js
- Adjusted the return statement in getDefaultServiceKeyPath function for improved readability by formatting it across multiple lines. This change enhances code clarity without altering functionality.
* ✨ feat: Enhance Gemini Image Generation with Token Usage Tracking
- Added `recordTokenUsage` function to track token usage for balance management.
- Integrated token recording into the image generation process.
- Updated Gemini image generation tool to accept optional `aspectRatio` and `imageSize` parameters for improved image customization.
- Updated token values for new Gemini models in the transaction model.
- Improved documentation for image generation tool descriptions and parameters.
* ✨ feat: Add new Gemini models for image generation token limits
- Introduced token limits for 'gemini-3-pro-image' and 'gemini-2.5-flash-image' models.
- Updated token values to enhance the Gemini image generation capabilities.
* 🔧 fix: Update Google Service Key Path for Consistency in Initialization (#11001)
* 🔧 refactor: Update GeminiImageGen for improved file handling and path resolution
- Changed the default service key path to use process.cwd() for better compatibility.
- Replaced synchronous file system operations with asynchronous promises for mkdir and writeFile, enhancing performance and error handling.
- Added error handling for credential file access to prevent crashes when the file does not exist.
* 🔧 refactor: Update GeminiImageGen to streamline API key handling
- Refactored API key checks to improve clarity and consistency.
- Removed redundant checks for user-provided keys, enhancing code readability.
- Ensured proper logging for API key usage across different configurations.
* 🔧 fix: Update GeminiImageGen to handle imageSize support conditionally
- Added a check to ensure imageSize is only applied if the gemini model does not include 'gemini-2.5-flash-image', improving compatibility.
- Enhanced the logic for setting imageConfig to prevent potential issues with unsupported configurations.
* 🔧 refactor: Simplify local storage condition in createGeminiImageTool function
* 🔧 feat: Enhance image format handling in GeminiImageGen with conversion support
* 🔧 refactor: Streamline API key initialization in GeminiImageGen
- Simplified the handling of API keys by removing redundant checks for user-provided keys.
- Updated logging to reflect the new priority order for API key usage, enhancing clarity and consistency.
- Improved code readability by consolidating key retrieval logic.
---------
Co-authored-by: Dev Bhanushali <dev.bhanushali@hingehealth.com>
Co-authored-by: Danny Avila <danny@librechat.ai>
* fix: Tool Resources Dropped between Agent Handoffs
* fix: agent deletion process to remove handoff edges
- Added logic to the `deleteAgent` function to remove references to the deleted agent from other agents' handoff edges.
- Implemented error handling to log any issues encountered during the edge removal process.
- Introduced a new test case to verify that handoff edges are correctly removed when an agent is deleted, ensuring data integrity across agent relationships.
* fix: Improve agent loading process by handling orphaned references
- Added logic to track and log agents that fail to load during initialization, preventing errors from interrupting the process.
- Introduced a Set to store skipped agent IDs and updated edge filtering to exclude these orphaned references, enhancing data integrity in agent relationships.
* chore: Update @librechat/agents to version 3.0.62
* feat: Enhance agent initialization with edge collection and filtering
- Introduced new functions for edge collection and filtering orphaned edges, improving the agent loading process.
- Refactored the `initializeClient` function to utilize breadth-first search (BFS) for discovering connected agents, enabling transitive handoffs.
- Added a new module for edge-related utilities, including deduplication and participant extraction, to streamline edge management.
- Updated the agent configuration handling to ensure proper edge processing and integrity during initialization.
* refactor: primary agent ID selection for multi-agent conversations
- Added a new function `findPrimaryAgentId` to determine the primary agent ID from a set of agent IDs based on suffix rules.
- Updated `createMultiAgentMapper` to filter messages by primary agent for parallel agents and handle handoffs appropriately.
- Enhanced message processing logic to ensure correct inclusion of agent content based on group and agent ID presence.
- Improved documentation to clarify the distinctions between parallel execution and handoff scenarios.
* feat: Implement primary agent ID selection for multi-agent content filtering
* chore: Update @librechat/agents to version 3.0.63 in package.json and package-lock.json
* chore: Update @librechat/agents to version 3.0.64 in package.json and package-lock.json
* chore: Update @librechat/agents to version 3.0.65 in package.json and package-lock.json
* feat: Add optional agent name to run creation for improved identification
* chore: Update @librechat/agents to version 3.0.66 in package.json and package-lock.json
* test: Add unit tests for edge utilities including key generation, participant extraction, and orphaned edge filtering
- Implemented tests for `getEdgeKey`, `getEdgeParticipants`, `filterOrphanedEdges`, and `createEdgeCollector` functions.
- Ensured comprehensive coverage for various edge cases, including handling of arrays and default values.
- Verified correct behavior of edge filtering based on skipped agents and deduplication of edges.
* feat: Implement concurrent request handling in ResumableAgentController
- Introduced a new concurrency management system by adding `checkAndIncrementPendingRequest` and `decrementPendingRequest` functions to manage user request limits.
- Replaced the previous `concurrentLimiter` middleware with a more integrated approach directly within the `ResumableAgentController`.
- Enhanced violation logging and request denial for users exceeding their concurrent request limits.
- Removed the obsolete `concurrentLimiter` middleware file and updated related imports across the codebase.
* refactor: Simplify error handling in ResumableAgentController and enhance SSE error management
- Removed the `denyRequest` middleware and replaced it with a direct response for concurrent request violations in the ResumableAgentController.
- Improved error handling in the `useResumableSSE` hook to differentiate between network errors and other error types, ensuring more informative error responses are sent to the error handler.
* test: Enhance MCP server configuration tests with new mocks and improved logging
- Added mocks for MCP server registry and manager in `index.spec.js` to facilitate testing of server configurations.
- Updated debug logging in `initializeMCPs.spec.js` to simplify messages regarding server configurations, improving clarity in test outputs.
* refactor: Enhance concurrency management in request handling
- Updated `checkAndIncrementPendingRequest` and `decrementPendingRequest` functions to utilize Redis for atomic request counting, improving concurrency control.
- Added error handling for Redis operations to ensure requests can proceed even during Redis failures.
- Streamlined cache key generation for both Redis and in-memory fallback, enhancing clarity and performance in managing pending requests.
- Improved comments and documentation for better understanding of the concurrency logic and its implications.
* refactor: Improve atomicity in Redis operations for pending request management
- Updated `checkAndIncrementPendingRequest` to utilize Redis pipelines for atomic INCR and EXPIRE operations, enhancing concurrency control and preventing edge cases.
- Added error handling for pipeline execution failures to ensure robust request management.
- Improved comments for clarity on the concurrency logic and its implications.
* 🔧 chore: Update package-lock.json and package.json to include google-auth-library
- Added google-auth-library version 9.15.1 to both package.json and package-lock.json for improved authentication support.
- Cleaned up package-lock.json by removing unused AWS SDK dependencies to streamline the project.
* 🔧 chore: Update express and body-parser dependencies in package.json and package-lock.json
- Upgraded express from version 5.1.0 to 5.2.1 for improved performance and features.
- Updated body-parser from version 2.2.0 to 2.2.1 to address potential issues and enhance functionality.
- Added depd package for better deprecation warnings management.
* 🔧 chore: Update qs dependency in package-lock.json
- Upgraded qs from version 6.14.0 to 6.14.1 to incorporate the latest improvements and fixes.
* feat: Add Anthropic Vertex AI Support
* Remove changes from the unused AnthropicClient class
* Add @anthropic-ai/vertex-sdk as peerDependency to packages/api
* Clean up Vertex AI credentials handling
* feat: websearch header
* feat: add prompt caching support for Anthropic Vertex AI
- Support both OpenAI format (input_token_details) and Anthropic format (cache_*_input_tokens) for token usage tracking
- Filter out unsupported anthropic-beta header values for Vertex AI (prompt-caching, max-tokens, output-128k, token-efficient-tools, context-1m)
* ✨ feat: Add Vertex AI support for Anthropic models
- Introduced configuration options for running Anthropic models via Google Cloud Vertex AI in the YAML file.
- Updated ModelService to prioritize Vertex AI models from the configuration.
- Enhanced endpoint configuration to enable Anthropic endpoint when Vertex AI is configured.
- Implemented validation and processing for Vertex AI credentials and options.
- Added new types and schemas for Vertex AI configuration in the data provider.
- Created utility functions for loading and validating Vertex AI credentials and configurations.
- Updated various services to integrate Vertex AI options into the Anthropic client setup.
* 🔒 fix: Improve error handling for missing credentials in LLM configuration
- Updated the `getLLMConfig` function to throw a specific error message when credentials are missing, enhancing clarity for users.
- Refactored the `parseCredentials` function to handle plain API key strings more gracefully, returning them wrapped in an object if JSON parsing fails.
* 🔧 refactor: Clean up code formatting and improve readability
- Updated the `setOptions` method in `AgentClient` to use a parameter name for clarity.
- Refactored error handling in `loadDefaultModels` for better readability.
- Removed unnecessary blank lines in `initialize.js`, `endpoints.ts`, and `vertex.ts` to streamline the code.
- Enhanced formatting in `validateVertexConfig` for improved consistency and clarity.
* 🔧 refactor: Enhance Vertex AI Model Configuration and Integration
- Updated the YAML configuration to support visible model names and deployment mappings for Vertex AI.
- Refactored the `loadDefaultModels` function to utilize the new model name structure.
- Improved the `initializeClient` function to pass full Vertex AI configuration, including model mappings.
- Added utility functions to map visible model names to deployment names, enhancing the integration of Vertex AI models.
- Updated various services and types to accommodate the new model configuration schema and improve overall clarity and functionality.
* 🔧 chore: Update @anthropic-ai/sdk dependency to version 0.71.0 in package.json and package-lock.json
* refactor: Change clientOptions declaration from let to const in initialize.ts for better code clarity
* chore: repository cleanup
* 🌊 feat: Resumable LLM Streams with Horizontal Scaling (#10926)
* ✨ feat: Implement Resumable Generation Jobs with SSE Support
- Introduced GenerationJobManager to handle resumable LLM generation jobs independently of HTTP connections.
- Added support for subscribing to ongoing generation jobs via SSE, allowing clients to reconnect and receive updates without losing progress.
- Enhanced existing agent controllers and routes to integrate resumable functionality, including job creation, completion, and error handling.
- Updated client-side hooks to manage adaptive SSE streams, switching between standard and resumable modes based on user settings.
- Added UI components and settings for enabling/disabling resumable streams, improving user experience during unstable connections.
* WIP: resuming
* WIP: resumable stream
* feat: Enhance Stream Management with Abort Functionality
- Updated the abort endpoint to support aborting ongoing generation streams using either streamId or conversationId.
- Introduced a new mutation hook `useAbortStreamMutation` for client-side integration.
- Added `useStreamStatus` query to monitor stream status and facilitate resuming conversations.
- Enhanced `useChatHelpers` to incorporate abort functionality when stopping generation.
- Improved `useResumableSSE` to handle stream errors and token refresh seamlessly.
- Updated `useResumeOnLoad` to check for active streams and resume conversations appropriately.
* fix: Update query parameter handling in useChatHelpers
- Refactored the logic for determining the query parameter used in fetching messages to prioritize paramId from the URL, falling back to conversationId only if paramId is not available. This change ensures consistency with the ChatView component's expectations.
* fix: improve syncing when switching conversations
* fix: Prevent memory leaks in useResumableSSE by clearing handler maps on stream completion and cleanup
* fix: Improve content type mismatch handling in useStepHandler
- Enhanced the condition for detecting content type mismatches to include additional checks, ensuring more robust validation of content types before processing updates.
* fix: Allow dynamic content creation in useChatFunctions
- Updated the initial response handling to avoid pre-initializing content types, enabling dynamic creation of content parts based on incoming delta events. This change supports various content types such as think and text.
* fix: Refine response message handling in useStepHandler
- Updated logic to determine the appropriate response message based on the last message's origin, ensuring correct message replacement or appending based on user interaction. This change enhances the accuracy of message updates in the chat flow.
* refactor: Enhance GenerationJobManager with In-Memory Implementations
- Introduced InMemoryJobStore, InMemoryEventTransport, and InMemoryContentState for improved job management and event handling.
- Updated GenerationJobManager to utilize these new implementations, allowing for better separation of concerns and easier maintenance.
- Enhanced job metadata handling to support user messages and response IDs for resumable functionality.
- Improved cleanup and state management processes to prevent memory leaks and ensure efficient resource usage.
* refactor: Enhance GenerationJobManager with improved subscriber handling
- Updated RuntimeJobState to include allSubscribersLeftHandlers for managing client disconnections without affecting subscriber count.
- Refined createJob and subscribe methods to ensure generation starts only when the first real client connects.
- Added detailed documentation for methods and properties to clarify the synchronization of job generation with client readiness.
- Improved logging for subscriber checks and event handling to facilitate debugging and monitoring.
* chore: Adjust timeout for subscriber readiness in ResumableAgentController
- Reduced the timeout duration from 5000ms to 2500ms in the startGeneration function to improve responsiveness when waiting for subscriber readiness. This change aims to enhance the efficiency of the agent's background generation process.
* refactor: Update GenerationJobManager documentation and structure
- Enhanced the documentation for GenerationJobManager to clarify the architecture and pluggable service design.
- Updated comments to reflect the potential for Redis integration and the need for async refactoring.
- Improved the structure of the GenerationJob facade to emphasize the unified API while allowing for implementation swapping without affecting consumer code.
* refactor: Convert GenerationJobManager methods to async for improved performance
- Updated methods in GenerationJobManager and InMemoryJobStore to be asynchronous, enhancing the handling of job creation, retrieval, and management.
- Adjusted the ResumableAgentController and related routes to await job operations, ensuring proper flow and error handling.
- Increased timeout duration in ResumableAgentController's startGeneration function to 3500ms for better subscriber readiness management.
* refactor: Simplify initial response handling in useChatFunctions
- Removed unnecessary pre-initialization of content types in the initial response, allowing for dynamic content creation based on incoming delta events. This change enhances flexibility in handling various content types in the chat flow.
* refactor: Clarify content handling logic in useStepHandler
- Updated comments to better explain the handling of initialContent and existingContent in edit and resume scenarios.
- Simplified the logic for merging content, ensuring that initialContent is used directly when available, improving clarity and maintainability.
* refactor: Improve message handling logic in useStepHandler
- Enhanced the logic for managing messages in multi-tab scenarios, ensuring that the most up-to-date message history is utilized.
- Removed existing response placeholders and ensured user messages are included, improving the accuracy of message updates in the chat flow.
* fix: remove unnecessary content length logging in the chat stream response, simplifying the debug message while retaining essential information about run steps. This change enhances clarity in logging without losing critical context.
* refactor: Integrate streamId handling for improved resumable functionality for attachments
- Added streamId parameter to various functions to support resumable mode in tool loading and memory processing.
- Updated related methods to ensure proper handling of attachments and responses based on the presence of streamId, enhancing the overall streaming experience.
- Improved logging and attachment management to accommodate both standard and resumable modes.
* refactor: Streamline abort handling and integrate GenerationJobManager for improved job management
- Removed the abortControllers middleware and integrated abort handling directly into GenerationJobManager.
- Updated abortMessage function to utilize GenerationJobManager for aborting jobs by conversation ID, enhancing clarity and efficiency.
- Simplified cleanup processes and improved error handling during abort operations.
- Enhanced metadata management for jobs, including endpoint and model information, to facilitate better tracking and resource management.
* refactor: Unify streamId and conversationId handling for improved job management
- Updated ResumableAgentController and AgentController to generate conversationId upfront, ensuring it matches streamId for consistency.
- Simplified job creation and metadata management by removing redundant conversationId updates from callbacks.
- Refactored abortMiddleware and related methods to utilize the unified streamId/conversationId approach, enhancing clarity in job handling.
- Removed deprecated methods from GenerationJobManager and InMemoryJobStore, streamlining the codebase and improving maintainability.
* refactor: Enhance resumable SSE handling with improved UI state management and error recovery
- Added UI state restoration on successful SSE connection to indicate ongoing submission.
- Implemented detailed error handling for network failures, including retry logic with exponential backoff.
- Introduced abort event handling to reset UI state on intentional stream closure.
- Enhanced debugging capabilities for testing reconnection and clean close scenarios.
- Updated generation function to retry on network errors, improving resilience during submission processes.
* refactor: Consolidate content state management into IJobStore for improved job handling
- Removed InMemoryContentState and integrated its functionality into InMemoryJobStore, streamlining content state management.
- Updated GenerationJobManager to utilize jobStore for content state operations, enhancing clarity and reducing redundancy.
- Introduced RedisJobStore for horizontal scaling, allowing for efficient job management and content reconstruction from chunks.
- Updated IJobStore interface to reflect changes in content state handling, ensuring consistency across implementations.
* feat: Introduce Redis-backed stream services for enhanced job management
- Added createStreamServices function to configure job store and event transport, supporting both Redis and in-memory options.
- Updated GenerationJobManager to allow configuration with custom job stores and event transports, improving flexibility for different deployment scenarios.
- Refactored IJobStore interface to support asynchronous content retrieval, ensuring compatibility with Redis implementations.
- Implemented RedisEventTransport for real-time event delivery across instances, enhancing scalability and responsiveness.
- Updated InMemoryJobStore to align with new async patterns for content and run step retrieval, ensuring consistent behavior across storage options.
* refactor: Remove redundant debug logging in GenerationJobManager and RedisEventTransport
- Eliminated unnecessary debug statements in GenerationJobManager related to subscriber actions and job updates, enhancing log clarity.
- Removed debug logging in RedisEventTransport for subscription and subscriber disconnection events, streamlining the logging output.
- Cleaned up debug messages in RedisJobStore to focus on essential information, improving overall logging efficiency.
* refactor: Enhance job state management and TTL configuration in RedisJobStore
- Updated the RedisJobStore to allow customizable TTL values for job states, improving flexibility in job management.
- Refactored the handling of job expiration and cleanup processes to align with new TTL configurations.
- Simplified the response structure in the chat status endpoint by consolidating state retrieval, enhancing clarity and performance.
- Improved comments and documentation for better understanding of the changes made.
* refactor: cleanupOnComplete option to GenerationJobManager for flexible resource management
- Introduced a new configuration option, cleanupOnComplete, allowing immediate cleanup of event transport and job resources upon job completion.
- Updated completeJob and abortJob methods to respect the cleanupOnComplete setting, enhancing memory management.
- Improved cleanup logic in the cleanup method to handle orphaned resources effectively.
- Enhanced documentation and comments for better clarity on the new functionality.
* refactor: Update TTL configuration for completed jobs in InMemoryJobStore
- Changed the TTL for completed jobs from 5 minutes to 0, allowing for immediate cleanup.
- Enhanced cleanup logic to respect the new TTL setting, improving resource management.
- Updated comments for clarity on the behavior of the TTL configuration.
* refactor: Enhance RedisJobStore with local graph caching for improved performance
- Introduced a local cache for graph references using WeakRef to optimize reconnects for the same instance.
- Updated job deletion and cleanup methods to manage the local cache effectively, ensuring stale entries are removed.
- Enhanced content retrieval methods to prioritize local cache access, reducing Redis round-trips for same-instance reconnects.
- Improved documentation and comments for clarity on the caching mechanism and its benefits.
* feat: Add integration tests for GenerationJobManager, RedisEventTransport, and RedisJobStore, add Redis Cluster support
- Introduced comprehensive integration tests for GenerationJobManager, covering both in-memory and Redis modes to ensure consistent job management and event handling.
- Added tests for RedisEventTransport to validate pub/sub functionality, including cross-instance event delivery and error handling.
- Implemented integration tests for RedisJobStore, focusing on multi-instance job access, content reconstruction from chunks, and consumer group behavior.
- Enhanced test setup and teardown processes to ensure a clean environment for each test run, improving reliability and maintainability.
* fix: Improve error handling in GenerationJobManager for allSubscribersLeft handlers
- Enhanced the error handling logic when retrieving content parts for allSubscribersLeft handlers, ensuring that any failures are logged appropriately.
- Updated the promise chain to catch errors from getContentParts, improving robustness and clarity in error reporting.
* ci: Improve Redis client disconnection handling in integration tests
- Updated the afterAll cleanup logic in integration tests for GenerationJobManager, RedisEventTransport, and RedisJobStore to use `quit()` for graceful disconnection of the Redis client.
- Added fallback to `disconnect()` if `quit()` fails, enhancing robustness in resource management during test teardown.
- Improved comments for clarity on the disconnection process and error handling.
* refactor: Enhance GenerationJobManager and event transports for improved resource management
- Updated GenerationJobManager to prevent immediate cleanup of eventTransport upon job completion, allowing final events to transmit fully before cleanup.
- Added orphaned stream cleanup logic in GenerationJobManager to handle streams without corresponding jobs.
- Introduced getTrackedStreamIds method in both InMemoryEventTransport and RedisEventTransport for better management of orphaned streams.
- Improved comments for clarity on resource management and cleanup processes.
* refactor: Update GenerationJobManager and ResumableAgentController for improved event handling
- Modified GenerationJobManager to resolve readyPromise immediately, eliminating startup latency and allowing early event buffering for late subscribers.
- Enhanced event handling logic to replay buffered events when the first subscriber connects, ensuring no events are lost due to race conditions.
- Updated comments for clarity on the new event synchronization mechanism and its benefits in both Redis and in-memory modes.
* fix: Update cache integration test command for stream to ensure proper execution
- Modified the test command for cache integration related to streams by adding the --forceExit flag to prevent hanging tests.
- This change enhances the reliability of the test suite by ensuring all tests complete as expected.
* feat: Add active job management for user and show progress in conversation list
- Implemented a new endpoint to retrieve active generation job IDs for the current user, enhancing user experience by allowing visibility of ongoing tasks.
- Integrated active job tracking in the Conversations component, displaying generation indicators based on active jobs.
- Optimized job management in the GenerationJobManager and InMemoryJobStore to support user-specific job queries, ensuring efficient resource handling and cleanup.
- Updated relevant components and hooks to utilize the new active jobs feature, improving overall application responsiveness and user feedback.
* feat: Implement active job tracking by user in RedisJobStore
- Added functionality to retrieve active job IDs for a specific user, enhancing user experience by allowing visibility of ongoing tasks.
- Implemented self-healing cleanup for stale job entries, ensuring accurate tracking of active jobs.
- Updated job creation, update, and deletion methods to manage user-specific job sets effectively.
- Enhanced integration tests to validate the new user-specific job management features.
* refactor: Simplify job deletion logic by removing user job cleanup from InMemoryJobStore and RedisJobStore
* WIP: Add backend inspect script for easier debugging in production
* refactor: title generation logic
- Changed the title generation endpoint from POST to GET, allowing for more efficient retrieval of titles based on conversation ID.
- Implemented exponential backoff for title fetching retries, improving responsiveness and reducing server load.
- Introduced a queuing mechanism for title generation, ensuring titles are generated only after job completion.
- Updated relevant components and hooks to utilize the new title generation logic, enhancing user experience and application performance.
* feat: Enhance updateConvoInAllQueries to support moving conversations to the top
* chore: temp. remove added multi convo
* refactor: Update active jobs query integration for optimistic updates on abort
- Introduced a new interface for active jobs response to standardize data handling.
- Updated query keys for active jobs to ensure consistency across components.
- Enhanced job management logic in hooks to properly reflect active job states, improving overall application responsiveness.
* refactor: useResumableStreamToggle hook to manage resumable streams for legacy/assistants endpoints
- Introduced a new hook, useResumableStreamToggle, to automatically toggle resumable streams off for assistants endpoints and restore the previous value when switching away.
- Updated ChatView component to utilize the new hook, enhancing the handling of streaming behavior based on endpoint type.
- Refactored imports in ChatView for better organization.
* refactor: streamline conversation title generation handling
- Removed unused type definition for TGenTitleMutation in mutations.ts to clean up the codebase.
- Integrated queueTitleGeneration call in useEventHandlers to trigger title generation for new conversations, enhancing the responsiveness of the application.
* feat: Add USE_REDIS_STREAMS configuration for stream job storage
- Introduced USE_REDIS_STREAMS to control Redis usage for resumable stream job storage, defaulting to true if USE_REDIS is enabled but not explicitly set.
- Updated cacheConfig to include USE_REDIS_STREAMS and modified createStreamServices to utilize this new configuration.
- Enhanced unit tests to validate the behavior of USE_REDIS_STREAMS under various environment settings, ensuring correct defaults and overrides.
* fix: title generation queue management for assistants
- Introduced a queueListeners mechanism to notify changes in the title generation queue, improving responsiveness for non-resumable streams.
- Updated the useTitleGeneration hook to track queue changes with a queueVersion state, ensuring accurate updates when jobs complete.
- Refactored the queueTitleGeneration function to trigger listeners upon adding new conversation IDs, enhancing the overall title generation flow.
* refactor: streamline agent controller and remove legacy resumable handling
- Updated the AgentController to route all requests to ResumableAgentController, simplifying the logic.
- Deprecated the legacy non-resumable path, providing a clear migration path for future use.
- Adjusted setHeaders middleware to remove unnecessary checks for resumable mode.
- Cleaned up the useResumableSSE hook to eliminate redundant query parameters, enhancing clarity and performance.
* feat: Add USE_REDIS_STREAMS configuration to .env.example
- Updated .env.example to include USE_REDIS_STREAMS setting, allowing control over Redis usage for resumable LLM streams.
- Provided additional context on the behavior of USE_REDIS_STREAMS when not explicitly set, enhancing clarity for configuration management.
* refactor: remove unused setHeaders middleware from chat route
- Eliminated the setHeaders middleware from the chat route, streamlining the request handling process.
- This change contributes to cleaner code and improved performance by reducing unnecessary middleware checks.
* fix: Add streamId parameter for resumable stream handling across services (actions, mcp oauth)
* fix(flow): add immediate abort handling and fix intervalId initialization
- Add immediate abort handler that responds instantly to abort signal
- Declare intervalId before cleanup function to prevent 'Cannot access before initialization' error
- Consolidate cleanup logic into single function to avoid duplicate cleanup
- Properly remove abort event listener on cleanup
* fix(mcp): clean up OAuth flows on abort and simplify flow handling
- Add abort handler in reconnectServer to clean up mcp_oauth and mcp_get_tokens flows
- Update createAbortHandler to clean up both flow types on tool call abort
- Pass abort signal to createFlow in returnOnOAuth path
- Simplify handleOAuthRequired to always cancel existing flows and start fresh
- This ensures user always gets a new OAuth URL instead of waiting for stale flows
* fix(agents): handle 'new' conversationId and improve abort reliability
- Treat 'new' as placeholder that needs UUID in request controller
- Send JSON response immediately before tool loading for faster SSE connection
- Use job's abort controller instead of prelimAbortController
- Emit errors to stream if headers already sent
- Skip 'new' as valid ID in abort endpoint
- Add fallback to find active jobs by userId when conversationId is 'new'
* fix(stream): detect early abort and prevent navigation to non-existent conversation
- Abort controller on job completion to signal pending operations
- Detect early abort (no content, no responseMessageId) in abortJob
- Set conversation and responseMessage to null for early aborts
- Add earlyAbort flag to final event for frontend detection
- Remove unused text field from AbortResult interface
- Frontend handles earlyAbort by staying on/navigating to new chat
* test(mcp): update test to expect signal parameter in createFlow
* 🔧 refactor: Update Vertex AI Configuration Handling
- Simplified the logic for enabling Vertex AI in the Anthropic initialization process, ensuring it defaults to enabled unless explicitly set to false.
- Adjusted the Vertex AI schema to make the 'enabled' property optional, defaulting to true when the configuration is present.
- Updated related comments and documentation for clarity on the configuration behavior.
* 🔧 chore: Update Anthropic Configuration and Logging Enhancements
- Changed the default region for Anthropic Vertex AI from 'global' to 'us-east5' in the .env.example file for better regional alignment.
- Added debug logging to handle non-JSON credentials in the Anthropic client, improving error visibility during credential parsing.
- Updated the service key path resolution in the Vertex AI client to use the current working directory, enhancing flexibility in file location.
---------
Co-authored-by: Ziyan <5621658+Ziyann@users.noreply.github.com>
Co-authored-by: Aron Gates <aron@muonspace.com>
Co-authored-by: Danny Avila <danny@librechat.ai>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Addresses Server-Side Request Forgery vulnerability that allowed authenticated
attackers to interact with arbitrary internal/external HTTP services via the
Actions feature, including the internal RAG API, localhost services, and cloud
metadata endpoints.
## Security Changes
### 1. SSRF Target Blocklist (when allowedDomains is empty)
- Block localhost and loopback addresses (127.0.0.0/8, ::1)
- Block private IP ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16)
- Block link-local/cloud metadata IPs (169.254.0.0/16)
- Block common internal Docker/Kubernetes service names (rag_api, redis, mongo, etc.)
- Block .internal and .local TLDs
Admins can still explicitly allow internal targets by adding them to
`actions.allowedDomains` in librechat.yaml.
### 2. Protocol and Port Restrictions
Enhanced `allowedDomains` to support protocol and port constraints:
- `example.com` - any protocol, any port (existing behavior)
- `https://example.com` - HTTPS only, any port
- `https://api.example.com:8443` - HTTPS only, port 8443 only
This allows admins to restrict actions to specific protocols/ports, preventing
attackers from accessing unintended services on allowed domains.
### 3. Redirect-based SSRF Prevention
Disabled automatic redirect following in action HTTP requests (`maxRedirects: 0`).
This prevents attackers from bypassing domain restrictions by:
1. Pointing action to allowed external domain
2. External domain redirects to internal service (e.g., 127.0.0.1)
3. Server follows redirect and accesses internal service
## Files Changed
- packages/api/src/auth/domain.ts: Added isSSRFTarget(), parseDomainSpec(),
updated isActionDomainAllowed() with protocol/port matching
- packages/api/src/auth/domain.spec.ts: Added tests for SSRF protection,
protocol restrictions, and port restrictions
- packages/data-provider/src/actions.ts: Added maxRedirects: 0 to axios config
## Configuration Example
# librechat.yaml
actions:
allowedDomains:
- "https://api.example.com" # HTTPS only
- "https://api.example.com:8443" # HTTPS + specific port
- "http://localhost:3000" # Admin override for local devRef: SBA-ADV-20251205-02
CVSSv3: 9.1 (Critical) - CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:L/A:L
* 🛜 refactor: SSE Connection Handling in MCP
- Introduced a longer initial handshake timeout for SSE connections to improve reliability through proxies.
- Added standardized request headers for SSE connections to ensure proper handling.
- Implemented a function to extract meaningful error messages from SSE transport errors, addressing common issues like timeouts, connection resets, and DNS failures.
- Enhanced error logging with detailed context to aid in debugging and provide insights into connection issues.
This update aims to improve the robustness and user experience of SSE connections in the MCP.
* chore: Update SSE Connection Headers Documentation
- Clarified the documentation for SSE connection headers by removing unnecessary details and emphasizing the headers that are intentionally excluded.
- Improved readability and conciseness of the comments regarding HTTP/2 connection management.
This change aims to enhance the clarity of the code documentation for better understanding and maintenance.
* refactor: Improved the handling of SSE transport errors by implementing a function to extract meaningful error messages, addressing various scenarios such as timeouts, connection resets, and DNS failures.
- Added comprehensive unit tests for the new error handling function, ensuring robust detection and reporting of common SSE error cases.
- Updated comments in the connection handling code to clarify the merging of headers, emphasizing user-defined overrides.
This update aims to improve the reliability and clarity of error reporting in SSE connections, enhancing the overall user experience.
* refactor: Enhance SSE error message extraction for improved timeout detection
- Updated the `extractSSEErrorMessage` function to include case-insensitive matching for various timeout patterns, including 'ESOCKETTIMEDOUT', 'timed out', 'timeout after', and 'request timeout'.
- Added unit tests to ensure accurate detection of these timeout messages and prevent false positives in unrelated contexts.
- Improved comments for clarity on the timeout detection logic.
This change aims to enhance the reliability of error handling in SSE connections, providing clearer feedback for timeout scenarios.
* chore: bump `@modelcontextprotocol/sdk` to v1.25.1
* chore: update dependencies in package-lock.json and package.json
- Bump versions of several `@smithy` packages to improve compatibility and performance.
- Add `@aws-sdk/client-bedrock-runtime` and `@smithy/node-http-handler` as peer dependencies.
- Update `https-proxy-agent` to version 7.0.6 for enhanced security and functionality.
- Remove outdated dependencies to streamline the project.
* fix: update httpsAgent type in fetchModels function
- Changed the type of httpsAgent from HttpsProxyAgent to HttpsProxyAgent<string> for improved type safety and clarity in the fetchModels function.
* fix: only show upload to provider for azureOpenAi when use responses api is true
* fix: model_parameters not available on first load so Upload Image incorrectly shown - now we query if not populated
* test: update tests for new azureOpenAI Responses API logic
* chore: correct order of headers in OAuth request to ensure proper content type is set
* fix: add useResponsesApi prop to AttachFileMenu and DragDropModal components
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
* Added video upload support for OpenRouter
- Added VIDEO_URL content type to support video_url message format
- Implemented OpenRouter video encoding using base64 data URLs
- Extended encodeAndFormatVideos() to handle OpenRouter provider
- Updated UI to accept video uploads for OpenRouter (mp4, webm, mpeg, mov)
- Fixed case-sensitivity in provider detection for agents
- Made isDocumentSupportedProvider() and isOpenAILikeProvider() case-insensitive
Videos are now converted to data:video/mp4;base64,... format compatible
with OpenRouter's API requirements per their documentation.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* refactor: change multimodal and google_multimodal to more transparent variable names of image_document and image_document_video_audio
(also google_multimodal doesn't apply as much since we are adding support for video and audio uploads for open router)
* fix: revert .toLowerCase change to isOpenAILikeProvider and isDocumentSupportedProvider which broke upload to provider detection for openAI endpoints
* wip: add audio support to openrouter
* fix: filetypes now properly parsed and sent rather than destructured mimetypes for openrouter
* refactor: Omit to Exclude for ESLint
* feat: update DragDropModal for new openrouter support
* fix: special case openrouter for lower case provider
(currently getting issues with the provider coming in as 'OpenRouter' and our enum being 'openrouter') This will probably require a larger refactor later to handle case insensitivity for all providers, but that will have to be thoroughly tested in its own isolated PR
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Dustin Healy <54083382+dustinhealy@users.noreply.github.com>
* chore: Add mathjs dependency
* refactor: Replace eval with mathjs for safer expression evaluation and improve session expiry handling to not environment variables from data-schemas package
* test: Add integration tests for math function with environment variable expressions
* refactor: Update test description for clarity on expiresIn behavior
* refactor: Update test cases to clarify default expiration behavior for token generation
* refactor: Improve error handling in math function for clearer evaluation errors
