Commit graph

12 commits

Author SHA1 Message Date
Danny Avila
01e9b196bc
🤖 feat: Streamline Endpoints to Agent Framework (#8013)
* refactor(buildEndpointOption): Improve error logging in middleware, consolidate `isAgents` builder logic, remove adding `modelsConfig` to `endpointOption`

* refactor: parameter extraction and organization in agent services, minimize redundancy of shared fields across objects, make clear distinction of parameters processed uniquely by LibreChat vs LLM Provider Configs

* refactor(createPayload): streamline all endpoints to agent route

* fix: add `modelLabel` to response sender options for agent initialization

* chore: correct log message context in EditController abort controller cleanup

* chore: remove unused abortRequest hook

* chore: remove unused addToCache module and its dependencies

* refactor: remove AskController and related routes, update endpoint URLs (now all streamlined to agents route)

* chore: remove unused bedrock route and its related imports

* refactor: simplify response sender logic for Google endpoint

* chore: add `modelDisplayLabel` handling for agents endpoint

* feat: add file search capability to ephemeral agents, update code interpreter selection based of file upload, consolidate main upload menu for all endpoints

* feat: implement useToolToggle hook for managing tool toggle state, refactor CodeInterpreter and WebSearch components to utilize new hook

* feat: add ToolsDropdown component to BadgeRow for enhanced tool options

* feat: introduce BadgeRowContext and BadgeRowProvider for managing conversation state, refactor related components to utilize context

* feat: implement useMCPSelect hook for managing MCP selection state, refactor MCPSelect component to utilize new hook

* feat: enhance BadgeRowContext with MCPSelect and tool toggle functionality, refactor related components to utilize updated context and hooks

* refactor: streamline useToolToggle hook by integrating setEphemeralAgent directly into toggle logic and removing redundant setValue function

* refactor: consolidate codeApiKeyForm and searchApiKeyForm from CodeInterpreter and WebSearch to utilize new context properties

* refactor: update CheckboxButton to support controlled state and enhance ToolsDropdown with permission-based toggles for web search and code interpreter

* refactor: conditionally render CheckboxButton in CodeInterpreter and WebSearch components for improved UI responsiveness

* chore: add jotai dependency to package.json and package-lock.json

* chore: update brace-expansion package to version 2.0.2 in package-lock.json due to CVE-2025-5889

* Revert "chore: add jotai dependency to package.json and package-lock.json"

This reverts commit 69b6997396.

* refactor: add pinning functionality to CodeInterpreter and WebSearch components, and enhance ToolsDropdown with pin toggle for web search and code interpreter

* chore: move MCPIcon to correct location, remove duplicate

* fix: update MCP import to use type-only import from librechat-data-provider

* feat: implement MCPSubMenu component and integrate pinning functionality into ToolsDropdown

* fix: cycling to submenu by using parent menu context

* feat: add FileSearch component and integrate it into BadgeRow and ToolsDropdown

* chore: import order

* chore: remove agent specific logic that would block functionality for streamlined endpoints

* chore: linting for `createContextHandlers`

* chore: ensure ToolsDropdown doesn't show up for agents

* chore: ensure tool resource is selected when dragged to UI

* chore: update file search behavior to simulate legacy functionality

* feat: ToolDialogs with multiple trigger references, add settings to tool dropdown

* refactor: simplify web search and code interpreter settings checks

* chore: simplify local storage key for pinned state in useToolToggle

* refactor: reinstate agent check in AttachFileChat component, as individual providers will ahve different file configurations

* ci: increase timeout for MongoDB connection in Agent tests
2025-06-23 09:59:05 -04:00
Danny Avila
ec7370dfe9
🪐 feat: MCP OAuth 2.0 Discovery Support (#7924)
* chore: Update @modelcontextprotocol/sdk to version 1.12.3 in package.json and package-lock.json

- Bump version of @modelcontextprotocol/sdk to 1.12.3 to incorporate recent updates.
- Update dependencies for ajv and cross-spawn to their latest versions.
- Add ajv as a new dependency in the sdk module.
- Include json-schema-traverse as a new dependency in the sdk module.

* feat: @librechat/auth

* feat: Add crypto module exports to auth package

- Introduced a new crypto module by creating index.ts in the crypto directory.
- Updated the main index.ts of the auth package to export from the new crypto module.

* feat: Update package dependencies and build scripts for auth package

- Added @librechat/auth as a dependency in package.json and package-lock.json.
- Updated build scripts to include the auth package in both frontend and bun build processes.
- Removed unused mongoose and openid-client dependencies from package-lock.json for cleaner dependency management.

* refactor: Migrate crypto utility functions to @librechat/auth

- Replaced local crypto utility imports with the new @librechat/auth package across multiple files.
- Removed the obsolete crypto.js file and its exports.
- Updated relevant services and models to utilize the new encryption and decryption methods from @librechat/auth.

* feat: Enhance OAuth token handling and update dependencies in auth package

* chore: Remove Token model and TokenService due to restructuring of OAuth handling

- Deleted the Token.js model and TokenService.js, which were responsible for managing OAuth tokens.
- This change is part of a broader refactor to streamline OAuth token management and improve code organization.

* refactor: imports from '@librechat/auth' to '@librechat/api' and add OAuth token handling functionality

* refactor: Simplify logger usage in MCP and FlowStateManager classes

* chore: fix imports

* feat: Add OAuth configuration schema to MCP with token exchange method support

* feat: FIRST PASS Implement MCP OAuth flow with token management and error handling

- Added a new route for handling OAuth callbacks and token retrieval.
- Integrated OAuth token storage and retrieval mechanisms.
- Enhanced MCP connection to support automatic OAuth flow initiation on 401 errors.
- Implemented dynamic client registration and metadata discovery for OAuth.
- Updated MCPManager to manage OAuth tokens and handle authentication requirements.
- Introduced comprehensive logging for OAuth processes and error handling.

* refactor: Update MCPConnection and MCPManager to utilize new URL handling

- Added a `url` property to MCPConnection for better URL management.
- Refactored MCPManager to use the new `url` property instead of a deprecated method for OAuth handling.
- Changed logging from info to debug level for flow manager and token methods initialization.
- Improved comments for clarity on existing tokens and OAuth event listener setup.

* refactor: Improve connection timeout error messages in MCPConnection and MCPManager and use initTimeout for connection

- Updated the connection timeout error messages to include the duration of the timeout.
- Introduced a configurable `connectTimeout` variable in both MCPConnection and MCPManager for better flexibility.

* chore: cleanup MCP OAuth Token exchange handling; fix: erroneous use of flowsCache and remove verbose logs

* refactor: Update MCPManager and MCPTokenStorage to use TokenMethods for token management

- Removed direct token storage handling in MCPManager and replaced it with TokenMethods for better abstraction.
- Refactored MCPTokenStorage methods to accept parameters for token operations, enhancing flexibility and readability.
- Improved logging messages related to token persistence and retrieval processes.

* refactor: Update MCP OAuth handling to use static methods and improve flow management

- Refactored MCPOAuthHandler to utilize static methods for initiating and completing OAuth flows, enhancing clarity and reducing instance dependencies.
- Updated MCPManager to pass flowManager explicitly to OAuth handling methods, improving flexibility in flow state management.
- Enhanced comments and logging for better understanding of OAuth processes and flow state retrieval.

* refactor: Integrate token methods into createMCPTool for enhanced token management

* refactor: Change logging from info to debug level in MCPOAuthHandler for improved log management

* chore: clean up logging

* feat: first pass, auth URL from MCP OAuth flow

* chore: Improve logging format for OAuth authentication URL display

* chore: cleanup mcp manager comments

* feat: add connection reconnection logic in MCPManager

* refactor: reorganize token storage handling in MCP

- Moved token storage logic from MCPManager to a new MCPTokenStorage class for better separation of concerns.
- Updated imports to reflect the new token storage structure.
- Enhanced methods for storing, retrieving, updating, and deleting OAuth tokens, improving overall token management.

* chore: update comment for SYSTEM_USER_ID in MCPManager for clarity

* feat: implement refresh token functionality in MCP

- Added refresh token handling in MCPManager to support token renewal for both app-level and user-specific connections.
- Introduced a refreshTokens function to facilitate token refresh logic.
- Enhanced MCPTokenStorage to manage client information and refresh token processes.
- Updated logging for better traceability during token operations.

* chore: cleanup @librechat/auth

* feat: implement MCP server initialization in a separate service

- Added a new service to handle the initialization of MCP servers, improving code organization and readability.
- Refactored the server startup logic to utilize the new initializeMCP function.
- Removed redundant MCP initialization code from the main server file.

* fix: don't log auth url for user connections

* feat: enhance OAuth flow with success and error handling components

- Updated OAuth callback routes to redirect to new success and error pages instead of sending status messages.
- Introduced `OAuthSuccess` and `OAuthError` components to provide user feedback during authentication.
- Added localization support for success and error messages in the translation files.
- Implemented countdown functionality in the success component for a better user experience.

* fix: refresh token handling for user connections, add missing URL and methods

- add standard enum for system user id and helper for determining app-lvel vs. user-level connections

* refactor: update token handling in MCPManager and MCPTokenStorage

* fix: improve error logging in OAuth authentication handler

* fix: concurrency issues for both login url emission and concurrency of oauth flows for shared flows (same user, same server, multiple calls for same server)

* fix: properly fail shared flows for concurrent server calls and prevent duplication of tokens

* chore: remove unused auth package directory from update configuration

* ci: fix mocks in samlStrategy tests

* ci: add mcpConfig to AppService test setup

* chore: remove obsolete MCP OAuth implementation documentation

* fix: update build script for API to use correct command

* chore: bump version of @librechat/api to 1.2.4

* fix: update abort signal handling in createMCPTool function

* fix: add optional clientInfo parameter to refreshTokensFunction metadata

* refactor: replace app.locals.availableTools with getCachedTools in multiple services and controllers for improved tool management

* fix: concurrent refresh token handling issue

* refactor: add signal parameter to getUserConnection method for improved abort handling

* chore: JSDoc typing for `loadEphemeralAgent`

* refactor: update isConnectionActive method to use destructured parameters for improved readability

* feat: implement caching for MCP tools to handle app-level disconnects for loading list of tools

* ci: fix agent test
2025-06-17 13:50:33 -04:00
Danny Avila
13c7ceb918
📋 fix: Agent Resource Deduplication & Sharing Duplicate False Positive (#7835)
* fix: `primeResources` to Prevent Duplicate Files Across Sources

- Added multiple test cases to ensure that the `primeResources` function correctly handles duplicate files from OCR and attachments, including scenarios with shared files, files without IDs, and duplicates within attachments.
- Implemented logic to categorize files into appropriate tool resources while preventing duplicates across different categories.
- Enhanced error handling and ensured that unique files are returned in the final attachments array.

* fix: Update ToolService to handle single OCR tool case (no loaded tool necessary)

* refactor: Add skipVersioning option to updateAgent for isolated updates

- for now, mainly concerns sharing/unsharing of agents

* chore: Update translation for shared agent message in UI
2025-06-11 14:17:48 -04:00
matt burnett
53df6a1a71
🔄 fix: Update Agent Versioning to Include agent_ids (#7762)
* Removed agent_ids exclusion from version comparison in the Agent model.
* Added tests to ensure agent_ids changes trigger new version creation and handle duplicates correctly.
* Enhanced existing tests to validate agent_ids alongside other fields and preserve history.
2025-06-06 16:43:39 -04:00
matt burnett
32bab33499
🔄 fix: Handle Legacy Agent Version Creation (#7718)
* Simplify version creation logic in updateAgent function

* Add comprehensive tests for agent functionality including version history, action metadata generation, and loading agents

- Introduced tests for generating consistent hashes for action metadata.
- Implemented tests for loading agents with various scenarios including null and non-existent IDs.
- Added edge case tests for agent creation, updates, and error handling.
- Ensured proper handling of ephemeral agents and their associated functionalities.

* Enhance tests for Agent model functionality

- Updated test structure for Agent resource file operations, improving organization and readability.
- Added comprehensive tests for handling concurrent file additions and removals, ensuring data integrity.
- Implemented edge case tests for agent creation and resource file management, including scenarios with non-existent agents.
- Enhanced error handling in tests to cover various failure scenarios, ensuring robustness in agent operations.

* optimize tests
2025-06-03 22:04:13 -04:00
Danny Avila
a2fc7d312a
🏗️ refactor: Extract DB layers to data-schemas for shared use (#7650)
* refactor: move model definitions and database-related methods to packages/data-schemas

* ci: update tests due to new DB structure

fix: disable mocking `librechat-data-provider`

feat: Add schema exports to data-schemas package

- Introduced a new schema module that exports various schemas including action, agent, and user schemas.
- Updated index.ts to include the new schema exports for better modularity and organization.

ci: fix appleStrategy tests

fix: Agent.spec.js

ci: refactor handleTools tests to use MongoMemoryServer for in-memory database

fix: getLogStores imports

ci: update banViolation tests to use MongoMemoryServer and improve session mocking

test: refactor samlStrategy tests to improve mock configurations and user handling

ci: fix crypto mock in handleText tests for improved accuracy

ci: refactor spendTokens tests to improve model imports and setup

ci: refactor Message model tests to use MongoMemoryServer and improve database interactions

* refactor: streamline IMessage interface and move feedback properties to types/message.ts

* refactor: use exported initializeRoles from `data-schemas`, remove api workspace version (this serves as an example of future migrations that still need to happen)

* refactor: update model imports to use destructuring from `~/db/models` for consistency and clarity

* refactor: remove unused mongoose imports from model files for cleaner code

* refactor: remove unused mongoose imports from Share, Prompt, and Transaction model files for cleaner code

* refactor: remove unused import in Transaction model for cleaner code

* ci: update deploy workflow to reference new Docker Dev Branch Images Build and add new workflow for building Docker images on dev branch

* chore: cleanup imports
2025-05-30 22:18:13 -04:00
Danny Avila
442976c74f
🔧 fix: Agent Versioning with Action Hashing and OAuth Redirect (#7627)
* 🔧 chore: Update navigateFallbackDenylist in Vite config to include API routes

* 🔧 fix: Update redirect_uri in createActionTool to use DOMAIN_SERVER instead of DOMAIN_CLIENT

* 🔧 feat: Enhance Agent Versioning with Action Metadata Hashing

- Added support for generating a hash of action metadata to detect changes and manage agent versioning.
- Updated `updateAgent` function to include an optional `forceVersion` parameter for version creation.
- Modified `isDuplicateVersion` to compare action metadata hashes.
- Updated related tests to validate new versioning behavior with action changes.
- Refactored agent update logic to ensure proper tracking of user updates and version history.
2025-05-29 10:30:35 -04:00
matt burnett
cede5d120c
👤 feat: Enhance Agent Versioning to Track User Updates (#7523)
* feat: Enhance agent update functionality to track user updates

- Updated `updateAgent` function to accept an `updatingUserId` parameter for tracking who made changes.
- Modified agent versioning to include `updatedBy` field for better audit trails.
- Adjusted related functions and tests to ensure proper handling of user updates and version history.
- Enhanced tests to verify correct tracking of `updatedBy` during agent updates and restorations.

* fix: Refactor import tests for improved readability and consistency

- Adjusted formatting in `importChatGptConvo` test to enhance clarity.
- Updated expected output string in `processAssistantMessage` test to use double quotes for consistency.
- Modified processing time expectation in `processAssistantMessage` test to allow for CI environment variability.
2025-05-23 20:47:14 -04:00
matt burnett
d47d827ed9
feat: Agent Version History and Management (#7455)
*  feat: Enhance agent update functionality to save current state in versions array

- Updated the `updateAgent` function to push the current agent's state into a new `versions` array when an agent is updated.
- Modified the agent schema to include a `versions` field for storing historical states of agents.

*  feat: Add comprehensive CRUD operations for agents in tests

- Introduced a new test suite for CRUD operations on agents, including create, read, update, and delete functionalities.
- Implemented tests for listing agents by author and updating agent projects.
- Enhanced the agent model to support version history tracking during updates.
- Ensured proper environment variable management during tests.

*  feat: Introduce version tracking for agents and enhance UI components

- Added a `version` property to the agent model to track the number of versions.
- Updated the `getAgentHandler` to include the agent's version in the response.
- Introduced a new `VersionButton` component for navigating to the version panel.
- Created a `VersionPanel` component for displaying version-related information.
- Updated the UI to conditionally render the version button and panel based on the active state.
- Added localization for the new version-related UI elements.

*  i18n: Add "version" translation key across multiple languages

- Introduced the "com_ui_agent_version" translation key in various language files to support version tracking for agents.
- Updated Arabic, Czech, German, English, Spanish, Estonian, Persian, Finnish, French, Hebrew, Hungarian, Indonesian, Italian, Japanese, Korean, Dutch, Polish, Portuguese (Brazil and Portugal), Russian, Swedish, Thai, Turkish, Vietnamese, and Chinese (Simplified and Traditional) translations.

*  feat: Update AgentFooter to conditionally render AdminSettings

- Modified the logic for displaying buttons in the AgentFooter component to only show them when the active panel is the builder.
- Ensured that AdminSettings is displayed only when the user has an admin role and the buttons are visible.

*  feat: Enhance AgentPanelSwitch and VersionPanel for improved agent capabilities

- Updated AgentPanelSwitch to include a new VersionPanel for displaying version-related information.
- Enhanced agentsConfig logic to properly handle agent capabilities.
- Modified VersionPanel to improve structure and localization support.
- Integrated createAgent mutation for future agent creation functionality.

*  feat: Enhance VersionPanel to display agent version history and loading states

- Integrated version fetching logic in VersionPanel to retrieve and display agent version history.
- Added loading and error handling states to improve user experience.
- Updated agent schema to use mixed types for versions, allowing for more flexible version data structures.
- Introduced localization support for version-related UI elements.

*  feat: Update VersionPanel and AgentPanelSwitch to enhance agent selection and version display

- Modified AgentPanelSwitch to pass selectedAgentId to VersionPanel for improved agent context.
- Enhanced VersionPanel to handle multiple timestamp formats and display appropriate messages when no agent is selected.
- Improved structure and readability of the VersionPanel component by adding a helper function for timestamp retrieval.

*  feat: Refactor VersionPanel to utilize localization and improve timestamp handling

- Replaced hardcoded text constants with localization support for various UI elements in VersionPanel.
- Enhanced the timestamp retrieval function to handle errors gracefully and utilize localized messages for unknown dates.
- Improved user feedback by displaying localized messages for agent selection, version errors, and empty states.

*  refactor: Clean up VersionPanel by removing unused code and improving timestamp handling

*  feat: Implement agent version reverting functionality

- Added `revertAgentVersion` method in the Agent model to allow reverting to a previous version of an agent.
- Introduced `revertAgentVersionHandler` in the agents controller to handle requests for reverting agent versions.
- Updated API routes to include a new endpoint for reverting agent versions.
- Enhanced the VersionPanel component to support version restoration with user confirmation and feedback.
- Added localization support for success and error messages related to version restoration.

*  i18n: Add localization for agent version restoration messages

* Simplify VersionPanel by removing unused parameters and enhancing agent ID handling

* Refactor Agent model and VersionPanel component to streamline version data handling

* Update version handling in Agent model and VersionPanel

- Enhanced the Agent model to include an `updatedAt` timestamp when pushing new versions.
- Improved the VersionPanel component to sort versions by the `updatedAt` timestamp for better display order.
- Added a new localization entry for indicating the active version of an agent.

*  i18n: Add localization for active agent version across multiple languages

*  feat: Introduce version management components for agent history

- Added `isActiveVersion` utility to determine the active version of an agent based on various criteria.
- Implemented `VersionContent` and `VersionItem` components to display agent version history, including loading and error states.
- Enhanced `VersionPanel` to integrate new components and manage version context effectively.
- Added comprehensive tests for version management functionalities to ensure reliability and correctness.

* Add unit tests for AgentFooter component

* cleanup

* Enhance agent version update handling and add unit tests for update operators

- Updated the `updateAgent` function to properly handle various update operators ($push, $pull, $addToSet) while maintaining version history.
- Modified unit tests to validate the correct behavior of agent updates, including versioning and tool management.

* Enhance version comparison logic and update tests for artifacts handling

- Modified the `isActiveVersion` utility to include artifacts in the version comparison criteria.
- Updated the `VersionPanel` component to support artifacts in the agent state.
- Added new unit tests to validate artifacts matching scenarios and edge cases in the `isActiveVersion` function.

* Implement duplicate version detection in agent updates and enhance error handling

- Added `isDuplicateVersion` function to check for identical versions during agent updates, excluding certain fields.
- Updated `updateAgent` function to throw an error if a duplicate version is detected, with detailed error information.
- Enhanced the `updateAgentHandler` to return appropriate responses for duplicate version errors.
- Modified client-side error handling to display user-friendly messages for duplicate version scenarios.
- Added comprehensive unit tests to validate duplicate version detection and error handling across various update scenarios.

* Update version title localization to include version number across multiple languages

- Modified the `com_ui_agent_version_title` translation key to include a placeholder for the version number in various language files.
- Enhanced the `VersionItem` component to utilize the updated localization for displaying version titles dynamically.

* Enhance agent version handling and add revert functionality

- Updated the `isDuplicateVersion` function to improve version comparison logic, including special handling for `projectIds` and arrays of objects.
- Modified the `updateAgent` function to streamline version updates and removed unnecessary checks for test environments.
- Introduced a new `revertAgentVersion` function to allow reverting agents to specific versions, with detailed documentation.
- Enhanced unit tests to validate duplicate version detection and revert functionality, ensuring robust error handling and version management.

* fix CI issues

* cleanup

* Revert all non-English translations

* clean up tests
2025-05-23 17:40:39 -04:00
Danny Avila
851938e7a6
🔧 fix: Agent Resource Form, Convo Menu Style, Ensure Draft Clears on Submission (#6925)
*  style: Adjust z-index for popover UI and update className in ConvoOptions

*  feat: Add 'spec' field to conversation query selection

* 🛠️ fix: add back conversationId to use Constants.PENDING_CONVO in useSSE hook on submission to allow text drafts to clear

*  chore: add .clineignore to .gitignore for Cline configuration

*  refactor: memoize FileSearchCheckbox component for performance optimization

* fix: agent resource management by adding tool_resource to agent's tools if missing
2025-04-16 18:14:34 -04:00
Danny Avila
cff392e578
🔧 fix: Agent Capability Checks & DocumentDB Compatibility for Agent Resource Removal (#6726)
* fix: tool capability checks in loadAgentTools function

* fix: enhance atomicity in removing agent resource files and add concurrency tests, improve documentdb compatibility
2025-04-04 10:33:53 -04:00
Danny Avila
964a74c73b
🛠 refactor: Ensure File Deletions, File Naming, and Agent Resource Updates (#5928)
* refactor: Improve error logging for file upload and processing functions to prevent verbosity

* refactor: Add uploads directory to Docker Compose to persist file uploads

* refactor: `addAgentResourceFile` to handle edge case of non-existing `tool_resource` array

* refactor: Remove version specification from deploy-compose.yml

* refactor: Prefix filenames with file_id to ensure uniqueness in file uploads

* refactor: Enhance error handling in deleteVectors to log warnings for non-404 errors

* refactor: Limit file search results to top 5 based on relevance score

* 🌍 i18n: Update translation.json with latest translations

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-02-17 19:37:03 -05:00