Commit graph

84 commits

Author SHA1 Message Date
Danny Avila
b5ab32c5ae
🎯 refactor: Centralize Agent Model Handling Across Conversation Lifecycle (#10956)
* refactor: Implement clearModelForNonEphemeralAgent utility for improved agent handling

- Introduced clearModelForNonEphemeralAgent function to manage model state for non-ephemeral agents across various components.
- Updated ModelSelectorContext to initialize model based on agent type.
- Enhanced useNavigateToConvo, useQueryParams, and useSelectMention hooks to clear model for non-ephemeral agents.
- Refactored buildDefaultConvo and endpoints utility to ensure proper handling of agent_id and model state.
- Improved overall conversation logic and state management for better performance and reliability.

* refactor: Enhance useNewConvo hook to improve agent conversation handling

- Added logic to skip access checks for existing agent conversations, utilizing localStorage to restore conversations after refresh.
- Improved handling of default endpoints for agents based on user access and existing conversation state, ensuring more reliable conversation initialization.

* refactor: Update ChatRoute and useAuthRedirect to include user roles

- Enhanced ChatRoute to utilize user roles for improved conversation state management.
- Modified useAuthRedirect to return user roles alongside authentication status, ensuring roles are available for conditional logic in ChatRoute.
- Adjusted conversation initialization logic to depend on the loaded roles, enhancing the reliability of the conversation setup.

* refactor: Update BaseClient to handle non-ephemeral agents in conversation logic

- Added a check for non-ephemeral agents in BaseClient, modifying the exceptions set to include 'model' when applicable.
- Enhanced conversation handling to improve flexibility based on agent type.

* test: Add mock for clearModelForNonEphemeralAgent in useQueryParams tests

- Introduced a mock for clearModelForNonEphemeralAgent to enhance testing of query parameters related to non-ephemeral agents.
- This addition supports improved test coverage and ensures proper handling of model state in relevant scenarios.

* refactor: Simplify mocks in useQueryParams tests for improved clarity

- Updated the mocking strategy for utilities in useQueryParams tests to use actual implementations where possible, while still suppressing test output for the logger.
- Enhanced the mock for tQueryParamsSchema to minimize complexity and avoid unnecessary validation during tests, improving test reliability and maintainability.

* refactor: Enhance agent identification logic in BaseClient for improved clarity

* chore: Import Constants in families.ts for enhanced functionality
2025-12-13 08:29:15 -05:00
Danny Avila
04a4a2aa44
🧵 refactor: Migrate Endpoint Initialization to TypeScript (#10794)
* refactor: move endpoint initialization methods to typescript

* refactor: move agent init to packages/api

- Introduced `initialize.ts` for agent initialization, including file processing and tool loading.
- Updated `resources.ts` to allow optional appConfig parameter.
- Enhanced endpoint configuration handling in various initialization files to support model parameters.
- Added new artifacts and prompts for React component generation.
- Refactored existing code to improve type safety and maintainability.

* refactor: streamline endpoint initialization and enhance type safety

- Updated initialization functions across various endpoints to use a consistent request structure, replacing `unknown` types with `ServerResponse`.
- Simplified request handling by directly extracting keys from the request body.
- Improved type safety by ensuring user IDs are safely accessed with optional chaining.
- Removed unnecessary parameters and streamlined model options handling for better clarity and maintainability.

* refactor: moved ModelService and extractBaseURL to packages/api

- Added comprehensive tests for the models fetching functionality, covering scenarios for OpenAI, Anthropic, Google, and Ollama models.
- Updated existing endpoint index to include the new models module.
- Enhanced utility functions for URL extraction and model data processing.
- Improved type safety and error handling across the models fetching logic.

* refactor: consolidate utility functions and remove unused files

- Merged `deriveBaseURL` and `extractBaseURL` into the `@librechat/api` module for better organization.
- Removed redundant utility files and their associated tests to streamline the codebase.
- Updated imports across various client files to utilize the new consolidated functions.
- Enhanced overall maintainability by reducing the number of utility modules.

* refactor: replace ModelService references with direct imports from @librechat/api and remove ModelService file

* refactor: move encrypt/decrypt methods and key db methods to data-schemas, use `getProviderConfig` from `@librechat/api`

* chore: remove unused 'res' from options in AgentClient

* refactor: file model imports and methods

- Updated imports in various controllers and services to use the unified file model from '~/models' instead of '~/models/File'.
- Consolidated file-related methods into a new file methods module in the data-schemas package.
- Added comprehensive tests for file methods including creation, retrieval, updating, and deletion.
- Enhanced the initializeAgent function to accept dependency injection for file-related methods.
- Improved error handling and logging in file methods.

* refactor: streamline database method references in agent initialization

* refactor: enhance file method tests and update type references to IMongoFile

* refactor: consolidate database method imports in agent client and initialization

* chore: remove redundant import of initializeAgent from @librechat/api

* refactor: move checkUserKeyExpiry utility to @librechat/api and update references across endpoints

* refactor: move updateUserPlugins logic to user.ts and simplify UserController

* refactor: update imports for user key management and remove UserService

* refactor: remove unused Anthropics and Bedrock endpoint files and clean up imports

* refactor: consolidate and update encryption imports across various files to use @librechat/data-schemas

* chore: update file model mock to use unified import from '~/models'

* chore: import order

* refactor: remove migrated to TS agent.js file and its associated logic from the endpoints

* chore: add reusable function to extract imports from source code in unused-packages workflow

* chore: enhance unused-packages workflow to include @librechat/api dependencies and improve dependency extraction

* chore: improve dependency extraction in unused-packages workflow with enhanced error handling and debugging output

* chore: add detailed debugging output to unused-packages workflow for better visibility into unused dependencies and exclusion lists

* chore: refine subpath handling in unused-packages workflow to correctly process scoped and non-scoped package imports

* chore: clean up unused debug output in unused-packages workflow and reorganize type imports in initialize.ts
2025-12-11 16:37:16 -05:00
Danny Avila
8bdc808074
refactor: Optimize & Standardize Tokenizer Usage (#10777)
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Waiting to run
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
* refactor: Token Limit Processing with Enhanced Efficiency

- Added a new test suite for `processTextWithTokenLimit`, ensuring comprehensive coverage of various scenarios including under, at, and exceeding token limits.
- Refactored the `processTextWithTokenLimit` function to utilize a ratio-based estimation method, significantly reducing the number of token counting function calls compared to the previous binary search approach.
- Improved handling of edge cases and variable token density, ensuring accurate truncation and performance across diverse text inputs.
- Included direct comparisons with the old implementation to validate correctness and efficiency improvements.

* refactor: Remove Tokenizer Route and Related References

- Deleted the tokenizer route from the server and removed its references from the routes index and server files, streamlining the API structure.
- This change simplifies the routing configuration by eliminating unused endpoints.

* refactor: Migrate countTokens Utility to API Module

- Removed the local countTokens utility and integrated it into the @librechat/api module for centralized access.
- Updated various files to reference the new countTokens import from the API module, ensuring consistent usage across the application.
- Cleaned up unused references and imports related to the previous countTokens implementation.

* refactor: Centralize escapeRegExp Utility in API Module

- Moved the escapeRegExp function from local utility files to the @librechat/api module for consistent usage across the application.
- Updated imports in various files to reference the new centralized escapeRegExp function, ensuring cleaner code and reducing redundancy.
- Removed duplicate implementations of escapeRegExp from multiple files, streamlining the codebase.

* refactor: Enhance Token Counting Flexibility in Text Processing

- Updated the `processTextWithTokenLimit` function to accept both synchronous and asynchronous token counting functions, improving its versatility.
- Introduced a new `TokenCountFn` type to define the token counting function signature.
- Added comprehensive tests to validate the behavior of `processTextWithTokenLimit` with both sync and async token counting functions, ensuring consistent results.
- Implemented a wrapper to track call counts for the `countTokens` function, optimizing performance and reducing unnecessary calls.
- Enhanced existing tests to compare the performance of the new implementation against the old one, demonstrating significant improvements in efficiency.

* chore: documentation for Truncation Safety Buffer in Token Processing

- Added a safety buffer multiplier to the character position estimates during text truncation to prevent overshooting token limits.
- Updated the `processTextWithTokenLimit` function to utilize the new `TRUNCATION_SAFETY_BUFFER` constant, enhancing the accuracy of token limit processing.
- Improved documentation to clarify the rationale behind the buffer and its impact on performance and efficiency in token counting.
2025-12-02 12:22:04 -05:00
Zihao Zhou
5b8f0cba04
📄 refactor: Add Provider Fallback for Media Encoding using Client Endpoint (#10656)
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Waiting to run
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
When using direct endpoints (e.g., Google) instead of Agents, `this.options.agent` is undefined, causing provider detection to fail. This resulted in "Unknown content type document" errors for Google/Gemini PDF uploads.

Added `?? this.options.endpoint` fallback in addDocuments(), addVideos(), and addAudios() methods to ensure correct provider detection for all endpoint types.
2025-11-25 17:07:37 -05:00
Danny Avila
c0cb48256e
🤖 refactor: Improve Agent Handoff Context Tracking (#10553)
* chore: update @librechat/agents dependency to version 3.0.18

* refactor: add optional metadata field to message schema and types

* chore: update @librechat/agents to v3.0.19

* refactor: update return type of sendCompletion method to include metadata

* chore: linting

* chore: update @librechat/agents dependency to v3.0.20

* refactor: implement agent labeling for conversation history in multi-agent scenarios

* refactor: improve error handling for capturing agent ID map in AgentClient

* refactor: clear agentIdMap and related properties during client disposal to prevent memory leaks

* chore: update sendCompletion method for FakeClient to return an object with completion and metadata fields
2025-11-17 16:57:51 -05:00
Danny Avila
2524d33362
📂 refactor: Cleanup File Filtering Logic, Improve Validation (#10414)
* feat: add filterFilesByEndpointConfig to filter disabled file processing by provider

* chore: explicit define of endpointFileConfig for better debugging

* refactor: move `normalizeEndpointName` to data-provider as used app-wide

* chore: remove overrideEndpoint from useFileHandling

* refactor: improve endpoint file config selection

* refactor: update filterFilesByEndpointConfig to accept structured parameters and improve endpoint file config handling

* refactor: replace defaultFileConfig with getEndpointFileConfig for improved file configuration handling across components

* test: add comprehensive unit tests for getEndpointFileConfig to validate endpoint configuration handling

* refactor: streamline agent endpoint assignment and improve file filtering logic

* feat: add error handling for disabled file uploads in endpoint configuration

* refactor: update encodeAndFormat functions to accept structured parameters for provider and endpoint

* refactor: streamline requestFiles handling in initializeAgent function

* fix: getEndpointFileConfig partial config merging scenarios

* refactor: enhance mergeWithDefault function to support document-supported providers with comprehensive MIME types

* refactor: user-configured default file config in getEndpointFileConfig

* fix: prevent file handling when endpoint is disabled and file is dragged to chat

* refactor: move `getEndpointField` to `data-provider` and update usage across components and hooks

* fix: prioritize endpointType based on agent.endpoint in file filtering logic

* fix: prioritize agent.endpoint in file filtering logic and remove unnecessary endpointType defaulting
2025-11-10 19:05:30 -05:00
Danny Avila
07d0abc9fd
🖼️ fix: Extract File Context & Persist Attachments (#10069)
- problem: `addImageUrls` had a side effect that was being leveraged before to populate both the `ocr` message field, now `fileContext`, and `client.options.attachments`, which would record the user's uploaded message attachments to the user message when saved to the database and returned at the end of the request lifecycle
- solution: created dedicated handling for file context, and made sure to populate `allFiles` with non-provider attachments
2025-10-10 05:35:37 -04:00
Danny Avila
bcd97aad2f
📎 feat: Direct Provider Attachment Support for Multimodal Content (#9994)
Some checks failed
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Has been cancelled
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Has been cancelled
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Has been cancelled
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Has been cancelled
* 📎 feat: Direct Provider Attachment Support for Multimodal Content

* 📑 feat: Anthropic Direct Provider Upload (#9072)

* feat: implement Anthropic native PDF support with document preservation

- Add comprehensive debug logging throughout PDF processing pipeline
- Refactor attachment processing to separate image and document handling
- Create distinct addImageURLs(), addDocuments(), and processAttachments() methods
- Fix critical bugs in stream handling and parameter passing
- Add streamToBuffer utility for proper stream-to-buffer conversion
- Remove api/agents submodule from repository

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: remove out of scope formatting changes

* fix: stop duplication of file in chat on end of response stream

* chore: bring back file search and ocr options

* chore: localize upload to provider string in file menu

* refactor: change createMenuItems args to fit new pattern introduced by anthropic-native-pdf-support

* feat: add cache point for pdfs processed by anthropic endpoint since they are unlikely to change and should benefit from caching

* feat: combine Upload Image into Upload to Provider since they both perform direct upload and change provider upload icon to reflect multimodal upload

* feat: add citations support according to docs

* refactor: remove redundant 'document' check since documents are handled properly by formatMessage in the agents repo now

* refactor: change upload logic so anthropic endpoint isn't exempted from normal upload path using Agents for consistency with the rest of the upload logic

* fix: include width and height in return from uploadLocalFile so images are correctly identified when going through an AgentUpload in addImageURLs

* chore: remove client specific handling since the direct provider stuff is handled by the agent client

* feat: handle documents in AgentClient so no need for change to agents repo

* chore: removed unused changes

* chore: remove auto generated comments from OG commit

* feat: add logic for agents to use direct to provider uploads if supported (currently just anthropic)

* fix: reintroduce role check to fix render error because of undefined value for Content Part

* fix: actually fix render bug by using proper isCreatedByUser check and making sure our mutation of formattedMessage.content is consistent

---------

Co-authored-by: Andres Restrepo <andres@thelinuxkid.com>
Co-authored-by: Claude <noreply@anthropic.com>

📁 feat: Send Attachments Directly to Provider (OpenAI) (#9098)

* refactor: change references from direct upload to direct attach to better reflect functionality

since we are just using base64 encoding strategy now rather than Files/File API for sending our attachments directly to the provider, the upload nomenclature no longer makes sense. direct_attach better describes the different methods of sending attachments to providers anyways even if we later introduce direct upload support

* feat: add upload to provider option for openai (and agent) ui

* chore: move anthropic pdf validator over to packages/api

* feat: simple pdf validation according to openai docs

* feat: add provider agnostic validatePdf logic to start handling multiple endpoints

* feat: add handling for openai specific documentPart formatting

* refactor: move require statement to proper place at top of file

* chore: add in openAI endpoint for the rest of the document handling logic

* feat: add direct attach support for azureOpenAI endpoint and agents

* feat: add pdf validation for azureOpenAI endpoint

* refactor: unify all the endpoint checks with isDocumentSupportedEndpoint

* refactor: consolidate Upload to Provider vs Upload image logic for clarity

* refactor: remove anthropic from anthropic_multimodal fileType since we support multiple providers now

🗂️ feat: Send Attachments Directly to Provider (Google) (#9100)

* feat: add validation for google PDFs and add google endpoint as a document supporting endpoint

* feat: add proper pdf formatting for google endpoints (requires PR #14 in agents)

* feat: add multimodal support for google endpoint attachments

* feat: add audio file svg

* fix: refactor attachments logic so multi-attachment messages work properly

* feat: add video file svg

* fix: allows for followup questions of uploaded multimodal attachments

* fix: remove incorrect final message filtering that was breaking Attachment component rendering

fix: manualy rename 'documents' to 'Documents' in git since it wasn't picked up due to case insensitivity in dir name

fix: add logic so filepicker for a google agent has proper filetype filtering

🛫 refactor: Move Encoding Logic to packages/api (#9182)

* refactor: move audio encode over to TS

* refactor: audio encoding now functional in LC again

* refactor: move video encode over to TS

* refactor: move document encode over to TS

* refactor: video encoding now functional in LC again

* refactor: document encoding now functional in LC again

* fix: extend file type options in AttachFileMenu to include 'google_multimodal' and update dependency array to include agent?.provider

* feat: only accept pdfs if responses api is enabled for openai convos

chore: address ESLint comments

chore: add missing audio mimetype

* fix: type safety for message content parts and improve null handling

* chore: reorder AttachFileMenuProps for consistency and clarity

* chore: import order in AttachFileMenu

* fix: improve null handling for text parts in parseTextParts function

* fix: remove no longer used unsupported capability error message for file uploads

* fix: OpenAI Direct File Attachment Format

* fix: update encodeAndFormatDocuments to support  OpenAI responses API and enhance document result types

* refactor: broaden providers supported for documents

* feat: enhance DragDrop context and modal to support document uploads based on provider capabilities

* fix: reorder import statements for consistency in video encoding module

---------

Co-authored-by: Dustin Healy <54083382+dustinhealy@users.noreply.github.com>
2025-10-06 17:30:16 -04:00
Danny Avila
9a210971f5
🛜 refactor: Streamline App Config Usage (#9234)
* WIP: app.locals refactoring

WIP: appConfig

fix: update memory configuration retrieval to use getAppConfig based on user role

fix: update comment for AppConfig interface to clarify purpose

🏷️ refactor: Update tests to use getAppConfig for endpoint configurations

ci: Update AppService tests to initialize app config instead of app.locals

ci: Integrate getAppConfig into remaining tests

refactor: Update multer storage destination to use promise-based getAppConfig and improve error handling in tests

refactor: Rename initializeAppConfig to setAppConfig and update related tests

ci: Mock getAppConfig in various tests to provide default configurations

refactor: Update convertMCPToolsToPlugins to use mcpManager for server configuration and adjust related tests

chore: rename `Config/getAppConfig` -> `Config/app`

fix: streamline OpenAI image tools configuration by removing direct appConfig dependency and using function parameters

chore: correct parameter documentation for imageOutputType in ToolService.js

refactor: remove `getCustomConfig` dependency in config route

refactor: update domain validation to use appConfig for allowed domains

refactor: use appConfig registration property

chore: remove app parameter from AppService invocation

refactor: update AppConfig interface to correct registration and turnstile configurations

refactor: remove getCustomConfig dependency and use getAppConfig in PluginController, multer, and MCP services

refactor: replace getCustomConfig with getAppConfig in STTService, TTSService, and related files

refactor: replace getCustomConfig with getAppConfig in Conversation and Message models, update tempChatRetention functions to use AppConfig type

refactor: update getAppConfig calls in Conversation and Message models to include user role for temporary chat expiration

ci: update related tests

refactor: update getAppConfig call in getCustomConfigSpeech to include user role

fix: update appConfig usage to access allowedDomains from actions instead of registration

refactor: enhance AppConfig to include fileStrategies and update related file strategy logic

refactor: update imports to use normalizeEndpointName from @librechat/api and remove redundant definitions

chore: remove deprecated unused RunManager

refactor: get balance config primarily from appConfig

refactor: remove customConfig dependency for appConfig and streamline loadConfigModels logic

refactor: remove getCustomConfig usage and use app config in file citations

refactor: consolidate endpoint loading logic into loadEndpoints function

refactor: update appConfig access to use endpoints structure across various services

refactor: implement custom endpoints configuration and streamline endpoint loading logic

refactor: update getAppConfig call to include user role parameter

refactor: streamline endpoint configuration and enhance appConfig usage across services

refactor: replace getMCPAuthMap with getUserMCPAuthMap and remove unused getCustomConfig file

refactor: add type annotation for loadedEndpoints in loadEndpoints function

refactor: move /services/Files/images/parse to TS API

chore: add missing FILE_CITATIONS permission to IRole interface

refactor: restructure toolkits to TS API

refactor: separate manifest logic into its own module

refactor: consolidate tool loading logic into a new tools module for startup logic

refactor: move interface config logic to TS API

refactor: migrate checkEmailConfig to TypeScript and update imports

refactor: add FunctionTool interface and availableTools to AppConfig

refactor: decouple caching and DB operations from AppService, make part of consolidated `getAppConfig`

WIP: fix tests

* fix: rebase conflicts

* refactor: remove app.locals references

* refactor: replace getBalanceConfig with getAppConfig in various strategies and middleware

* refactor: replace appConfig?.balance with getBalanceConfig in various controllers and clients

* test: add balance configuration to titleConvo method in AgentClient tests

* chore: remove unused `openai-chat-tokens` package

* chore: remove unused imports in initializeMCPs.js

* refactor: update balance configuration to use getAppConfig instead of getBalanceConfig

* refactor: integrate configMiddleware for centralized configuration handling

* refactor: optimize email domain validation by removing unnecessary async calls

* refactor: simplify multer storage configuration by removing async calls

* refactor: reorder imports for better readability in user.js

* refactor: replace getAppConfig calls with req.config for improved performance

* chore: replace getAppConfig calls with req.config in tests for centralized configuration handling

* chore: remove unused override config

* refactor: add configMiddleware to endpoint route and replace getAppConfig with req.config

* chore: remove customConfig parameter from TTSService constructor

* refactor: pass appConfig from request to processFileCitations for improved configuration handling

* refactor: remove configMiddleware from endpoint route and retrieve appConfig directly in getEndpointsConfig if not in `req.config`

* test: add mockAppConfig to processFileCitations tests for improved configuration handling

* fix: pass req.config to hasCustomUserVars and call without await after synchronous refactor

* fix: type safety in useExportConversation

* refactor: retrieve appConfig using getAppConfig in PluginController and remove configMiddleware from plugins route, to avoid always retrieving when plugins are cached

* chore: change `MongoUser` typedef to `IUser`

* fix: Add `user` and `config` fields to ServerRequest and update JSDoc type annotations from Express.Request to ServerRequest

* fix: remove unused setAppConfig mock from Server configuration tests
2025-08-26 12:10:18 -04:00
Danny Avila
e0ebb7097e
fix: AbortSignal Cleanup Logic for New Chats (#9177)
* chore: import paths for isEnabled and logger in title.js

*  fix: `AbortSignal` Cleanup Logic for New Chats

* test: Add `isNewConvo` parameter to onStart expectation in BaseClient tests
2025-08-20 14:56:07 -04:00
Danny Avila
d7d02766ea
🏷️ feat: Request Placeholders for Custom Endpoint & MCP Headers (#9095)
* feat: Add conversation ID support to custom endpoint headers

- Add LIBRECHAT_CONVERSATION_ID to customUserVars when provided
- Pass conversation ID to header resolution for dynamic headers
- Add comprehensive test coverage

Enables custom endpoints to access conversation context using {{LIBRECHAT_CONVERSATION_ID}} placeholder.

* fix: filter out unresolved placeholders from headers (thanks @MrunmayS)

* feat: add support for request body placeholders in custom endpoint headers

- Add {{LIBRECHAT_BODY_*}} placeholders for conversationId, parentMessageId, messageId
- Update tests to reflect new body placeholder functionality

* refactor resolveHeaders

* style: minor styling cleanup

* fix: type error in unit test

* feat: add body to other endpoints

* feat: add body for mcp tool calls

* chore: remove changes that unnecessarily increase scope after clarification of requirements

* refactor: move http.ts to packages/api and have RequestBody intersect with Express request body

* refactor: processMCPEnv now uses single object argument pattern

* refactor: update processMCPEnv to use 'options' parameter and align types across MCP connection classes

* feat: enhance MCP connection handling with dynamic request headers to pass request body fields

---------

Co-authored-by: Gopal Sharma <gopalsharma@gopal.sharma1>
Co-authored-by: s10gopal <36487439+s10gopal@users.noreply.github.com>
Co-authored-by: Dustin Healy <dustinhealy1@gmail.com>
2025-08-16 20:45:55 -04:00
Danny Avila
35d8ef50f4
🪙 fix: Use Fallback Token Transaction if No Collected Usage (#8503) 2025-07-16 17:58:15 -04:00
Danny Avila
e370a87ebe
♻️ fix: Correct Message ID Assignment Logic (#8439)
* fix: Add `isRegenerate` flag to chat payload to avoid saving temporary response IDs

* fix: Remove unused `isResubmission` flag

* ci: Add tests for responseMessageId regeneration logic in BaseClient
2025-07-14 00:57:20 -04:00
Danny Avila
434289fe92
🔀 feat: Save & Submit Message Content Parts (#8171)
* 🐛 fix: Enhance provider validation and error handling in getProviderConfig function

* WIP: edit text part

* refactor: Allow updating of both TEXT and THINK content types in message updates

* WIP: first pass, save & submit

* chore: remove legacy generation user message field

* feat: merge edited content

* fix: update placeholder and description for bedrock setting

* fix: remove unsupported warning message for AI resubmission
2025-07-01 15:43:10 -04:00
Danny Avila
01e9b196bc
🤖 feat: Streamline Endpoints to Agent Framework (#8013)
* refactor(buildEndpointOption): Improve error logging in middleware, consolidate `isAgents` builder logic, remove adding `modelsConfig` to `endpointOption`

* refactor: parameter extraction and organization in agent services, minimize redundancy of shared fields across objects, make clear distinction of parameters processed uniquely by LibreChat vs LLM Provider Configs

* refactor(createPayload): streamline all endpoints to agent route

* fix: add `modelLabel` to response sender options for agent initialization

* chore: correct log message context in EditController abort controller cleanup

* chore: remove unused abortRequest hook

* chore: remove unused addToCache module and its dependencies

* refactor: remove AskController and related routes, update endpoint URLs (now all streamlined to agents route)

* chore: remove unused bedrock route and its related imports

* refactor: simplify response sender logic for Google endpoint

* chore: add `modelDisplayLabel` handling for agents endpoint

* feat: add file search capability to ephemeral agents, update code interpreter selection based of file upload, consolidate main upload menu for all endpoints

* feat: implement useToolToggle hook for managing tool toggle state, refactor CodeInterpreter and WebSearch components to utilize new hook

* feat: add ToolsDropdown component to BadgeRow for enhanced tool options

* feat: introduce BadgeRowContext and BadgeRowProvider for managing conversation state, refactor related components to utilize context

* feat: implement useMCPSelect hook for managing MCP selection state, refactor MCPSelect component to utilize new hook

* feat: enhance BadgeRowContext with MCPSelect and tool toggle functionality, refactor related components to utilize updated context and hooks

* refactor: streamline useToolToggle hook by integrating setEphemeralAgent directly into toggle logic and removing redundant setValue function

* refactor: consolidate codeApiKeyForm and searchApiKeyForm from CodeInterpreter and WebSearch to utilize new context properties

* refactor: update CheckboxButton to support controlled state and enhance ToolsDropdown with permission-based toggles for web search and code interpreter

* refactor: conditionally render CheckboxButton in CodeInterpreter and WebSearch components for improved UI responsiveness

* chore: add jotai dependency to package.json and package-lock.json

* chore: update brace-expansion package to version 2.0.2 in package-lock.json due to CVE-2025-5889

* Revert "chore: add jotai dependency to package.json and package-lock.json"

This reverts commit 69b6997396.

* refactor: add pinning functionality to CodeInterpreter and WebSearch components, and enhance ToolsDropdown with pin toggle for web search and code interpreter

* chore: move MCPIcon to correct location, remove duplicate

* fix: update MCP import to use type-only import from librechat-data-provider

* feat: implement MCPSubMenu component and integrate pinning functionality into ToolsDropdown

* fix: cycling to submenu by using parent menu context

* feat: add FileSearch component and integrate it into BadgeRow and ToolsDropdown

* chore: import order

* chore: remove agent specific logic that would block functionality for streamlined endpoints

* chore: linting for `createContextHandlers`

* chore: ensure ToolsDropdown doesn't show up for agents

* chore: ensure tool resource is selected when dragged to UI

* chore: update file search behavior to simulate legacy functionality

* feat: ToolDialogs with multiple trigger references, add settings to tool dropdown

* refactor: simplify web search and code interpreter settings checks

* chore: simplify local storage key for pinned state in useToolToggle

* refactor: reinstate agent check in AttachFileChat component, as individual providers will ahve different file configurations

* ci: increase timeout for MongoDB connection in Agent tests
2025-06-23 09:59:05 -04:00
Danny Avila
66093b1eb3
💬 refactor: MCP Chat Visibility Option, Google Rates, Remove OpenAPI Plugins (#7286)
* fix: Update Gemini 2.5 Pro Preview Model Name in Token Values

* refactor: Update DeleteButton to close menu when deletion is successful

* refactor: Add unmountOnHide prop to DropdownPopup in multiple components

* chore: linting

* chore: linting

* feat: Add `chatMenu` option for MCP Servers to control visibility in MCPSelect dropdown

* refactor: Update loadManifestTools to return combined tool manifest with MCP tools first

* chore: remove deprecated openapi plugins

* chore: linting

* chore(AgentClient): linting, remove unnecessary `checkVisionRequest` logger

* refactor(AuthService): change logoutUser logging from error to debug level

* chore: new Gemini models token values and rates

* chore(AskController): linting
2025-05-08 12:12:36 -04:00
Danny Avila
37964975c1
🤖 refactor: Improve Agents Memory Usage, Bump Keyv, Grok 3 (#6850)
* chore: remove unused redis file

* chore: bump keyv dependencies, and update related imports

* refactor: Implement IoRedis client for rate limiting across middleware, as node-redis via keyv not compatible

* fix: Set max listeners to expected amount

* WIP: memory improvements

* refactor: Simplify getAbortData assignment in createAbortController

* refactor: Update getAbortData to use WeakRef for content management

* WIP: memory improvements in agent chat requests

* refactor: Enhance memory management with finalization registry and cleanup functions

* refactor: Simplify domainParser calls by removing unnecessary request parameter

* refactor: Update parameter types for action tools and agent loading functions to use minimal configs

* refactor: Simplify domainParser tests by removing unnecessary request parameter

* refactor: Simplify domainParser call by removing unnecessary request parameter

* refactor: Enhance client disposal by nullifying additional properties to improve memory management

* refactor: Improve title generation by adding abort controller and timeout handling, consolidate request cleanup

* refactor: Update checkIdleConnections to skip current user when checking for idle connections if passed

* refactor: Update createMCPTool to derive userId from config and handle abort signals

* refactor: Introduce createTokenCounter function and update tokenCounter usage; enhance disposeClient to reset Graph values

* refactor: Update getMCPManager to accept userId parameter for improved idle connection handling

* refactor: Extract logToolError function for improved error handling in AgentClient

* refactor: Update disposeClient to clear handlerRegistry and graphRunnable references in client.run

* refactor: Extract createHandleNewToken function to streamline token handling in initializeClient

* chore: bump @librechat/agents

* refactor: Improve timeout handling in addTitle function for better error management

* refactor: Introduce createFetch instead of using class method

* refactor: Enhance client disposal and request data handling in AskController and EditController

* refactor: Update import statements for AnthropicClient and OpenAIClient to use specific paths

* refactor: Use WeakRef for response handling in SplitStreamHandler to prevent memory leaks

* refactor: Simplify client disposal and rename getReqData to processReqData in AskController and EditController

* refactor: Improve logging structure and parameter handling in OpenAIClient

* refactor: Remove unused GraphEvents and improve stream event handling in AnthropicClient and OpenAIClient

* refactor: Simplify client initialization in AskController and EditController

* refactor: Remove unused mock functions and implement in-memory store for KeyvMongo

* chore: Update dependencies in package-lock.json to latest versions

* refactor: Await token usage recording in OpenAIClient to ensure proper async handling

* refactor: Remove handleAbort route from multiple endpoints and enhance client disposal logic

* refactor: Enhance abort controller logic by managing abortKey more effectively

* refactor: Add newConversation handling in useEventHandlers for improved conversation management

* fix: dropparams

* refactor: Use optional chaining for safer access to request properties in BaseClient

* refactor: Move client disposal and request data processing logic to cleanup module for better organization

* refactor: Remove aborted request check from addTitle function for cleaner logic

* feat: Add Grok 3 model pricing and update tests for new models

* chore: Remove trace warnings and inspect flags from backend start script used for debugging

* refactor: Replace user identifier handling with userId for consistency across controllers, use UserId in clientRegistry

* refactor: Enhance client disposal logic to prevent memory leaks by clearing additional references

* chore: Update @librechat/agents to version 2.4.14 in package.json and package-lock.json
2025-04-12 18:46:36 -04:00
Danny Avila
910c73359b
🔦 feat: MCP Support for Non-Agent Endpoints (#6775)
* wip: mcp select

* refactor: Update useAvailableToolsQuery to support generic data types

* feat: Enhance MCPSelect to dynamically load server options and improve MultiSelect component styling

* WIP: ephemeral agents

* wip: Add null check for MCPSelect and improve MultiSelect focus handling

* feat: Pass conversationId prop to MCPSelect in BadgeRow to optimize badge rendering

* feat: useApplyNewAgentTemplate hook to manage ephemeral agent upon conversation creation

* WIP: eph. agent payload

* refactor(OpenAIClient): streamline message processing by replacing content handling with parseTextParts function

* feat: enhance applyAgentTemplate function to accept source conversation ID for improved template application

* feat(parsers): add skipReasoning parameter to parseTextParts for conditional reasoning handling

* WIP: first pass, ephemeral agent backend processing

* chore: import order

* feat: update loadEphemeralAgent and loadAgent functions to accept model_parameters for enhanced agent configuration

* feat: add showMCPServers prop to BadgeRow for conditional rendering of MCPSelect, fix react rule violation

* feat: enhance MCPSelect with localized placeholder and custom icon, add renderSelectedValues callback

* feat: simplify message processing in AnthropicClient by replacing content handling with parseTextParts function

* feat: implement useLocalStorage hook for managing MCP values and update MCPSelect to utilize it

* chore: remove chatGPTBrowserSchema from endpoint schemas and update types for improved schema management

* chore: remove compactChatGPTSchema from endpoint schemas and update types for better schema management

* refactor: rename schemas for clarity and improve schema management

* feat: extend model detection to include 'codestral' alongside 'mistral'

* feat: add endpointType parameter to buildOptions and initializeClient functions

* fix: update condition for handling completion in BaseClient to include agents client

* refactor: simplify payload parsing logic in AgentClient and remove unused providerParsers

* refactor: change useSetRecoilState to useRecoilState for better state management in MCPSelect component

* refactor: streamline chat route handlers by consolidating middleware and improving endpoint structure

* style: update MCPSelect and MultiSelect components for improved layout in mobile view

* v0.7.790

* feat: add getMessageMapMethod to process message text and content in GoogleClient

* chore: include LAST_MCP_ key prefix in clearLocalStorage function for proper teardown on logout
2025-04-07 19:16:56 -04:00
Marco Beretta
7f29f2f676
🎨 feat: UI Refresh for Enhanced UX (#6346)
*  feat: Add Expand Chat functionality and improve UI components

*  feat: Introduce Chat Badges feature with editing capabilities and UI enhancements

*  feat: re-implement file attachment functionality with new components and improved UI

*  feat: Enhance BadgeRow component with drag-and-drop functionality and add animations for better user experience

*  feat: Add useChatBadges hook and enhance Badge component with animations and toggle functionality

* feat: Improve Add/Delete Badges + style and bug fixes

*  feat: Refactor EditBadges component and optimize useChatBadges hook for improved performance and readability

*  feat: Add type definition for LucideIcon in EditBadges component

* refactor: Clean up BadgeRow component by removing outdated comment and improving code readability

* refactor: Rename app-icon class to badge-icon for consistency and improve badge styling

* feat: Add Center Chat Input toggle and update related components for improved UI/UX

* refactor: Simplify ChatView and MessagesView components for improved readability and performance

* refactor: Improve layout and positioning of scroll button in MessagesView component

* refactor: Adjust scroll button position in MessagesView component for better visibility

* refactor: Remove redundant background class from Badge component for cleaner styling

* feat: disable chat badges

* refactor: adjust positioning of scroll button and popover for improved layout

* refactor: simplify class names in ChatForm and RemoveFile components for cleaner code

* refactor: move Switcher to HeaderOptions from SidePanel

* fix(Landing): duplicate description

* feat: add SplitText component for animated text display and update Landing component to use it

* feat(Chat): add ConversationStarters component and integrate it into ChatView; remove ConvoStarter component

* feat(Chat): enhance Message component layout and styling for improved readability

* feat(ControlCombobox, Select): enhance styling and add animation for improved UI experience

* feat(Chat): update Header and HeaderNewChat components for improved layout and styling

* feat(Chat): add ModelDropdown (now includes both endpoint and model) and refactor Menu components for improved UI

* feat(ModelDropdown): add Agent Select; removed old AgentSwitcher components

* feat(ModelDropdown): add settings button for user key configuration

* fix(ModelDropdown): the model dropdown wasn't opening automatically when opening the endpoint one

* refactor(Chat): remove unused EndpointsMenu and related components to streamline codebase

* feat: enhance greeting message and improve accessibility fro ModelDropdown

* refactor(Endpoints): add new hooks and components for endpoint management

* feat(Endpoint): add support for modelSpecs

* feat(Endpoints): add mobile support

* fix: type issues

* fix(modelSpec): type issue

* fix(EndpointMenuDropdown): double overflow scroller in mobile model list

* fix: search model on mobile

* refactor: Endpoint/Model/modelSpec dropdown

* refactor: reorganize imports in Endpoint components

* refactor: remove unused translation keys from English locale

* BREAKING: moving to ariakit with new CustomMenu

* refactor: remove unnecessary comments

* refactor: remove EndpointItem, ModelDropdownButton, SpecIcon, and SpecItem components

* 🔧 fix: AI Icon bump when regenerating message

* wip: chat UI refactoring, fix issues

* chore: add recent update to useAutoSave

* feat: add access control for agent permissions in useMentions hook

* refactor: streamline ModelSelector by removing unused endpoints logic

* refactor: enhance ModelSelector and context by integrating endpointsConfig and improving type usage

* feat: update ModelSelectorContext to utilize conversation data for initial state

* feat: add selector effects for synced endpoint handling

* feat: add guard clause for conversation endpoint in useSelectorEffects hook

* fix: safely call onSelectMention and add autofocus to mention input

* chore: typing

* refactor: ModelSelector to streamline key dialog handling and improve endpoint rendering

* refactor: extract SettingsButton component for cleaner endpoint item rendering

* wip: first pass, expand set api key

* wip: first pass, expanding set key

* refactor: update EndpointItem styles for improved layout and hover effects

* refactor: adjust padding in EndpointItem for improved layout consistency

* refactor: update preset structure in useSelectMention to include spec as null

* refactor: rename setKeyDialogOpen to onOpenChange for clarity and consistency, bring focus back to button that opened dialog

* feat: add SpecIcon component for dynamic model spec icons in menu, adjust icon styling

* refactor: update getSelectedIcon to accept additional parameters and improve icon rendering logic

* fix: adjust padding in MessageRender for improved layout

* refactor: remove inline style for menu width in CustomMenu component

* refactor: enhance layout and styling in ModelSpecItem component for better responsiveness

* refactor: update getDefaultModelSpec to accept startupConfig and improve model spec retrieval logic

* refactor: improve key management and default values in ModelSelector and related components

* refactor: adjust menu width and improve responsiveness in CustomMenu and EndpointItem components

* refactor: enhance focus styles and responsiveness in EndpointItem component

* refactor: improve layout and spacing in Header and ModelSelector components for better responsiveness

* refactor: adjust button styles for consistency and improved layout in AddMultiConvo and PresetsMenu components

* fix: initial fix of assistant names

* fix: assistants handling

* chore: update version of librechat-data-provider to 0.7.75 and add 'spec' to excludedKeys

* fix: improve endpoint filtering logic based on interface configuration and access rights

* fix: remove unused HeaderOptions import and set spec to null in presets and mentions

* fix: ensure currentExample is always an object when updating examples

* fix: update interfaceConfig checks to ensure modelSelect is considered for rendering components

* fix: update model selection logic to consider interface configuration when prioritizing model specs

* fix: add missing localizations

* fix: remove unused agent and assistant selection translations

* fix: implement debounced state updates for selected values in useSelectorEffects

* style: minor style changes related to the ModelSelector

* fix: adjust maximum height for popover and set fixed height for model item

* fix: update placeholders for model and endpoint search inputs

* fix: refactor MessageRender and ContentRender components to better match each other

* fix: remove convo fallback for iconURL in MessageRender and ContentRender components

* fix: update handling of spec, iconURL, and modelLabel in conversation presets, to allow better interchangeability

* fix: replace chatGptLabel with modelLabel in OpenAI settings configuration (fully deprecate chatGptLabel)

* fix: remove console log for assistantNames in useEndpoints hook

* refactor: add cleanInput and cleanOutput options to default conversation handling

* chore: update bun.lockb

* fix: set default value for showIconInHeader in getSelectedIcon function

* refactor: enhance error handling in message processing when latest message has existing content blocks

* chore: allow import/no-cycle for messages

* fix: adjust flex properties in BookmarkMenu for better layout

* feat: support both 'prompt' and 'q' as query parameters in useQueryParams hook

* feat: re-enable Badges components

* refactor: disable edit badge component

* chore: rename assistantMap to assistantsMap for consistency

* chore: rename assistantMap to assistantsMap for consistency in Mention component

* feat: set staleTime for various queries to improve data freshness

* feat: add spec field to tQueryParamsSchema for model specification

* feat: enhance useQueryParams to handle model specs

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2025-03-25 18:50:58 -04:00
Danny Avila
842b68fc32
🏗️ fix: Agents Token Spend Race Conditions, Add Auto-refill Tx, Add Relevant Tests (#6480)
* 🏗️ refactor: Improve spendTokens logic to handle zero completion tokens and enhance test coverage

* 🏗️ test: Add tests to ensure balance does not go below zero when spending tokens

* 🏗️ fix: Ensure proper continuation in AgentClient when handling errors

* fix: spend token race conditions

* 🏗️ test: Add test for handling multiple concurrent transactions with high balance

* fix: Handle Omni models prompt prefix handling for user messages with array content in OpenAIClient

* refactor: Update checkBalance import paths to use new balanceMethods module

* refactor: Update checkBalance imports and implement updateBalance function for atomic balance updates

* fix: import from replace method

* feat: Add createAutoRefillTransaction method to handle non-balance updating transactions

* refactor: Move auto-refill logic to balanceMethods and enhance checkBalance functionality

* feat: Implement logging for auto-refill transactions in balance checks

* refactor: Remove logRefill calls from multiple client and handler files

* refactor: Move balance checking and auto-refill logic to balanceMethods for improved structure

* refactor: Simplify balance check calls by removing unnecessary balanceRecord assignments

* fix: Prevent negative rawAmount in spendTokens when promptTokens is zero

* fix: Update balanceMethods to use Balance model for findOneAndUpdate

* chore: import order

* refactor: remove unused txMethods file to streamline codebase

* feat: enhance updateBalance and createAutoRefillTransaction methods to support additional parameters for improved balance management
2025-03-22 17:54:25 -04:00
Ruben Talstra
3a62a2633d
💵 feat: Add Automatic Balance Refill (#6452)
* 🚀 feat: Add automatic refill settings to balance schema

* 🚀 feat: Refactor balance feature to use global interface configuration

* 🚀 feat: Implement auto-refill functionality for balance management

* 🚀 feat: Enhance auto-refill logic and configuration for balance management

* 🚀 chore: Bump version to 0.7.74 in package.json and package-lock.json

* 🚀 chore: Bump version to 0.0.5 in package.json and package-lock.json

* 🚀 docs: Update comment for balance settings in librechat.example.yaml

* chore: space in `.env.example`

* 🚀 feat: Implement balance configuration loading and refactor related components

* 🚀 test: Refactor tests to use custom config for balance feature

* 🚀 fix: Update balance response handling in Transaction.js to use Balance model

* 🚀 test: Update AppService tests to include balance configuration in mock setup

* 🚀 test: Enhance AppService tests with complete balance configuration scenarios

* 🚀 refactor: Rename balanceConfig to balance and update related tests for clarity

* 🚀 refactor: Remove loadDefaultBalance and update balance handling in AppService

* 🚀 test: Update AppService tests to reflect new balance structure and defaults

* 🚀 test: Mock getCustomConfig in BaseClient tests to control balance configuration

* 🚀 test: Add get method to mockCache in OpenAIClient tests for improved cache handling

* 🚀 test: Mock getCustomConfig in OpenAIClient tests to control balance configuration

* 🚀 test: Remove mock for getCustomConfig in OpenAIClient tests to streamline configuration handling

* 🚀 fix: Update balance configuration reference in config.js for consistency

* refactor: Add getBalanceConfig function to retrieve balance configuration

* chore: Comment out example balance settings in librechat.example.yaml

* refactor: Replace getCustomConfig with getBalanceConfig for balance handling

* fix: tests

* refactor: Replace getBalanceConfig call with balance from request locals

* refactor: Update balance handling to use environment variables for configuration

* refactor: Replace getBalanceConfig calls with balance from request locals

* refactor: Simplify balance configuration logic in getBalanceConfig

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2025-03-21 17:48:11 -04:00
Danny Avila
efb616d600
🔧 fix: Update Token Calculations/Mapping, MCP env Initialization (#6406)
* fix: Enhance MCP initialization to process environment variables

* fix: only build tokenCountMap with messages that are being used in the payload

* fix: Adjust maxContextTokens calculation to account for maxOutputTokens

* refactor: Make processMCPEnv optional in MCPManager initialization

* chore: Bump version of librechat-data-provider to 0.7.73
2025-03-18 23:16:45 -04:00
Danny Avila
d6a17784dc
🔗 feat: Agent Chain (Mixture-of-Agents) (#6374)
* wip: first pass, dropdown for selecting sequential agents

* refactor: Improve agent selection logic and enhance performance in SequentialAgents component

* wip: seq. agents working ideas

* wip: sequential agents style change

* refactor: move agent form options/submission outside of AgentConfig

* refactor: prevent repeating code

* refactor: simplify current agent display in SequentialAgents component

* feat: persist  form value handling in AgentSelect component for agent_ids

* feat: first pass, sequential agnets agent update

* feat: enhance message display with agent updates and empty text handling

* chore: update Icon component to use EModelEndpoint for agent endpoints

* feat: update content type checks in BaseClient to use constants for better readability

* feat: adjust max context tokens calculation to use 90% of the model's max tokens

* feat: first pass, agent run message pruning

* chore: increase max listeners for abort controller to prevent memory leaks

* feat: enhance runAgent function to include current index count map for improved token tracking

* chore: update @librechat/agents dependency to version 2.2.5

* feat: update icons and style of SequentialAgents component for improved UI consistency

* feat: add AdvancedButton and AdvancedPanel components for enhanced agent settings navigation, update styling for agent form

* chore: adjust minimum height of AdvancedPanel component for better layout consistency

* chore: update @librechat/agents dependency to version 2.2.6

* feat: enhance message formatting by incorporating tool set into agent message processing, in order to allow better mix/matching of agents (as tool calls for tools not found in set will be stringified)

* refactor: reorder components in AgentConfig for improved readability and maintainability

* refactor: enhance layout of AgentUpdate component for improved visual structure

* feat: add DeepSeek provider to Bedrock settings and schemas

* feat: enhance link styling in mobile.css for better visibility and accessibility

* fix: update banner model import in update banner script; export Banner model

* refactor: `duplicateAgentHandler` to include tool_resources only for OCR context files

* feat: add 'qwen-vl' to visionModels for enhanced model support

* fix: change image format from JPEG to PNG in DALLE3 response

* feat: reorganize Advanced components and add localizations

* refactor: simplify JSX structure in AgentChain component to defer container styling to parent

* feat: add FormInput component for reusable input handling

* feat: make agent recursion limit configurable from builder

* feat: add support for agent capabilities chain in AdvancedPanel and update data-provider version

* feat: add maxRecursionLimit configuration for agents and update related documentation

* fix: update CONFIG_VERSION to 1.2.3 in data provider configuration

* feat: replace recursion limit input with MaxAgentSteps component and enhance input handling

* feat: enhance AgentChain component with hover card for additional information and update related labels

* fix: pass request and response objects to `createActionTool` when using assistant actions to prevent auth error

* feat: update AgentChain component layout to include agent count display

* feat: increase default max listeners and implement capability check function for agent chain

* fix: update link styles in mobile.css for better visibility in dark mode

* chore: temp. remove agents package while bumping shared packages

* chore: update @langchain/google-genai package to version 0.1.11

* chore: update @langchain/google-vertexai package to version 0.2.2

* chore: add @librechat/agents package at version 2.2.8

* feat: add deepseek.r1 model with token rate and context values for bedrock
2025-03-17 16:43:44 -04:00
Danny Avila
ded3cd8876
🔍 feat: Mistral OCR API / Upload Files as Text (#6274)
* refactor: move `loadAuthValues` to `~/services/Tools/credentials`

* feat: add createAxiosInstance function to configure axios with proxy support

* WIP: First pass mistral ocr

* refactor: replace getConvoFiles with getToolFiles for improved file retrieval logic

* refactor: improve document formatting in encodeAndFormat function

* refactor: remove unused resendFiles parameter from buildOptions function (this option comes from the agent config)

* fix: update getFiles call to include files with `text` property as well

* refactor: move file handling to `initializeAgentOptions`

* refactor: enhance addImageURLs method to handle OCR text and improve message formatting

* refactor: update message formatting to handle OCR text in various content types

* refactor: remove unused resendFiles property from compactAgentsSchema

* fix: add error handling for Mistral OCR document upload and logging

* refactor: integrate OCR capability into file upload options and configuration

* refactor: skip processing for text source files in delete request, as they are directly tied to database

* feat: add metadata field to ExtendedFile type and update PanelColumns and PanelTable components for localization and metadata handling

* fix: source icon styling

* wip: first pass, frontend file context agent resources

* refactor: add hover card with contextual information for File Context (OCR) in FileContext component

* feat: enhance file processing by integrating file retrieval for OCR resources in agent initialization

* feat: implement OCR config; fix: agent resource deletion for ocr files

* feat: enhance agent initialization by adding OCR capability check in resource priming

* ci: fix `~/config` module mock

* ci: add OCR property expectation in AppService tests

* refactor: simplify OCR config loading by removing environment variable extraction, to be done when OCR is actually performed

* ci: add unit test to ensure environment variable references are not parsed in OCR config

* refactor: disable base64 image inclusion in OCR request

* refactor: enhance OCR configuration handling by validating environment variables and providing defaults

* refactor: use file stream from disk for mistral ocr api
2025-03-10 17:23:46 -04:00
Danny Avila
be280004cf
🔧 refactor: Improve Params Handling, Remove Legacy Items, & Update Configs (#6074)
* chore: include all assets for service worker, remove unused tsconfig.node.json, eslint ignore vite config

* chore: exclude image files from service worker caching

* refactor: simplify googleSchema transformation and error handling

* fix: max output tokens cap for 3.7 models

* fix: skip index fixing in CI, development, and test environments

* ci: add maxOutputTokens handling tests for Claude models

* refactor: drop top_k and top_p parameters for claude-3.7 in AnthropicClient and add tests for new behavior

* refactor: conditionally include top_k and top_p parameters for non-claude-3.7 models

* ci: add unit tests for getLLMConfig function with various model options

* chore: remove all OPENROUTER_API_KEY legacy logic

* refactor: optimize stream chunk handling

* feat: reset model parameters button

* refactor: remove unused examples field from convoSchema and presetSchema

* chore: update librechat-data-provider version to 0.7.6993

* refactor: move excludedKeys set to data-provider for better reusability

* feat: enhance saveMessageToDatabase to handle unset fields and fetched conversation state

* feat: add 'iconURL' and 'greeting' to excludedKeys in data provider config

* fix: add optional chaining to user ID retrieval in getConvo call
2025-02-26 15:02:03 -05:00
Danny Avila
63afb317c6
🚀 fix: Resolve Google Client Issues, CDN Screenshots, Update Models (#5703)
* 🤖 refactor: streamline model selection logic for title model in GoogleClient

* refactor: add options for empty object schemas in convertJsonSchemaToZod

* refactor: add utility function to check for empty object schemas in convertJsonSchemaToZod

* fix: Google MCP Tool errors, and remove Object Unescaping as Google fixed this

* fix: google safetySettings

* feat: add safety settings exclusion via GOOGLE_EXCLUDE_SAFETY_SETTINGS environment variable

* fix: rename environment variable for console JSON string length

* fix: disable portal for dropdown in ExportModal component

* fix: screenshot functionality to use image placeholder for remote images

* feat: add visionMode property to BaseClient and initialize in GoogleClient to fix resendFiles issue

* fix: enhance formatMessages to include image URLs in message content for Vertex AI

* fix: safety settings for titleChatCompletion

* fix: remove deprecated model assignment in GoogleClient and streamline title model retrieval

* fix: remove unused image preloading logic in ScreenshotContext

* chore: update default google models to latest models shared by vertex ai and gen ai

* refactor: enhance Google error messaging

* fix: update token values and model limits for Gemini models

* ci: fix model matching

* chore: bump version of librechat-data-provider to 0.7.699
2025-02-06 18:13:18 -05:00
Danny Avila
591a019766
🏄‍♂️ refactor: Optimize Reasoning UI & Token Streaming (#5546)
*  feat: Implement Show Thinking feature; refactor: testing thinking render optimizations

*  feat: Refactor Thinking component styles and enhance Markdown rendering

* chore: add back removed code, revert type changes

* chore: Add back resetCounter effect to Markdown component for improved code block indexing

* chore: bump @librechat/agents and google langchain packages

* WIP: reasoning type updates

* WIP: first pass, reasoning content blocks

* chore: revert code

* chore: bump @librechat/agents

* refactor: optimize reasoning tag handling

* style: ul indent padding

* feat: add Reasoning component to handle reasoning display

* feat: first pass, content reasoning part styling

* refactor: add content placeholder for endpoints using new stream handler

* refactor: only cache messages when requesting stream audio

* fix: circular dep.

* fix: add default param

* refactor: tts, only request after message stream, fix chrome autoplay

* style: update label for submitting state and add localization for 'Thinking...'

* fix: improve global audio pause logic and reset active run ID

* fix: handle artifact edge cases

* fix: remove unnecessary console log from artifact update test

* feat: add support for continued message handling with new streaming method

---------

Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
2025-01-29 19:46:58 -05:00
Danny Avila
4110209494
♻️ fix: Prevent Instructions from Removal when nearing Max Context (#5516)
* refactor: getMessagesWithinTokenLimit to accept params object

* refactor: always include instructions in payload if provided

* ci: remove obsolete test

* refactor: update logoutUser to accept request object and handle session destruction

* test: enhance getMessagesWithinTokenLimit tests for instruction handling
2025-01-27 20:37:38 -05:00
Danny Avila
d6b4d83b68
🔥 feat: deepseek-reasoner Thought Streaming (#5379)
* 🔧 refactor: Remove unused penalties and enhance reasoning token handling in OpenAIClient

* 🔧 refactor: `addInstructions` default to adding instructions at index 0, flag for legacy behavior

* chore: remove long placeholder

* chore: update localization strings across multiple languages

* ci: adjust tests for new `addInstructions` behavior
2025-01-20 18:21:18 -05:00
Danny Avila
b35a8b78e2
🔧 refactor: Improve Agent Context & Minor Fixes (#5349)
* refactor: Improve Context for Agents

* 🔧 fix: Safeguard against undefined properties in OpenAIClient response handling

* refactor: log error before re-throwing for original stack trace

* refactor: remove toolResource state from useFileHandling, allow svg files

* refactor: prevent verbose logs from axios errors when using actions

* refactor: add silent method recordTokenUsage in AgentClient

* refactor: streamline token count assignment in BaseClient

* refactor: enhance safety settings handling for Gemini 2.0 model

* fix: capabilities structure in MCPConnection

* refactor: simplify civic integrity threshold handling in GoogleClient and llm

* refactor: update token count retrieval method in BaseClient tests

* ci: fix test for svg
2025-01-17 12:55:48 -05:00
Danny Avila
6c9a468b8e
🐛 fix: Artifacts Type Error, Tool Token Counts, and Agent Chat Import (#5142)
* fix: message import functionality to support content field

* fix: handle tool calls token counts in context window management

* fix: handle potential undefined size in FilePreview component
2024-12-30 13:01:47 -05:00
Danny Avila
24cad6bbd4
🤖 feat: Support Google Agents, fix Various Provider Configurations (#5126)
* feat: Refactor ModelEndHandler to collect usage metadata only if it exists

* feat: google tool end handling, custom anthropic class for better token ux

* refactor: differentiate between client <> request options

* feat: initial support for google agents

* feat: only cache messages with non-empty text

* feat: Cache non-empty messages in chatV2 controller

* fix: anthropic llm client options llmConfig

* refactor: streamline client options handling in LLM configuration

* fix: VertexAI Agent Auth & Tool Handling

* fix: additional fields for llmConfig, however customHeaders are not supported by langchain, requires PR

* feat: set default location for vertexai LLM configuration

* fix: outdated OpenAI Client options for getLLMConfig

* chore: agent provider options typing

* chore: add note about currently unsupported customHeaders in langchain GenAI client

* fix: skip transaction creation when rawAmount is NaN
2024-12-28 17:15:03 -05:00
Danny Avila
1a815f5e19
🎉 feat: Code Interpreter API and Agents Release (#4860)
* feat: Code Interpreter API & File Search Agent Uploads

chore: add back code files

wip: first pass, abstract key dialog

refactor: influence checkbox on key changes

refactor: update localization keys for 'execute code' to 'run code'

wip: run code button

refactor: add throwError parameter to loadAuthValues and getUserPluginAuthValue functions

feat: first pass, API tool calling

fix: handle missing toolId in callTool function and return 404 for non-existent tools

feat: show code outputs

fix: improve error handling in callTool function and log errors

fix: handle potential null value for filepath in attachment destructuring

fix: normalize language before rendering and prevent null return

fix: add loading indicator in RunCode component while executing code

feat: add support for conditional code execution in Markdown components

feat: attachments

refactor: remove bash

fix: pass abort signal to graph/run

refactor: debounce and rate limit tool call

refactor: increase debounce delay for execute function

feat: set code output attachments

feat: image attachments

refactor: apply message context

refactor: pass `partIndex`

feat: toolCall schema/model/methods

feat: block indexing

feat: get tool calls

chore: imports

chore: typing

chore: condense type imports

feat: get tool calls

fix: block indexing

chore: typing

refactor: update tool calls mapping to support multiple results

fix: add unique key to nav link for rendering

wip: first pass, tool call results

refactor: update query cache from successful tool call mutation

style: improve result switcher styling

chore: note on using \`.toObject()\`

feat: add agent_id field to conversation schema

chore: typing

refactor: rename agentMap to agentsMap for consistency

feat: Agent Name as chat input placeholder

chore: bump agents

📦 chore: update @langchain dependencies to latest versions to match agents package

📦 chore: update @librechat/agents dependency to version 1.8.0

fix: Aborting agent stream removes sender; fix(bedrock): completion removes preset name label

refactor: remove direct file parameter to use req.file, add `processAgentFileUpload` for image uploads

feat: upload menu

feat: prime message_file resources

feat: implement conversation access validation in chat route

refactor: remove file parameter from processFileUpload and use req.file instead

feat: add savedMessageIds set to track saved message IDs in BaseClient, to prevent unnecessary double-write to db

feat: prevent duplicate message saves by checking savedMessageIds in AgentController

refactor: skip legacy RAG API handling for agents

feat: add files field to convoSchema

refactor: update request type annotations from Express.Request to ServerRequest in file processing functions

feat: track conversation files

fix: resendFiles, addPreviousAttachments handling

feat: add ID validation for session_id and file_id in download route

feat: entity_id for code file uploads/downloads

fix: code file edge cases

feat: delete related tool calls

feat: add stream rate handling for LLM configuration

feat: enhance system content with attached file information

fix: improve error logging in resource priming function

* WIP: PoC, sequential agents

WIP: PoC Sequential Agents, first pass content data + bump agents package

fix: package-lock

WIP: PoC, o1 support, refactor bufferString

feat: convertJsonSchemaToZod

fix: form issues and schema defining erroneous model

fix: max length issue on agent form instructions, limit conversation messages to sequential agents

feat: add abort signal support to createRun function and AgentClient

feat: PoC, hide prior sequential agent steps

fix: update parameter naming from config to metadata in event handlers for clarity, add model to usage data

refactor: use only last contentData, track model for usage data

chore: bump agents package

fix: content parts issue

refactor: filter contentParts to include tool calls and relevant indices

feat: show function calls

refactor: filter context messages to exclude tool calls when no tools are available to the agent

fix: ensure tool call content is not undefined in formatMessages

feat: add agent_id field to conversationPreset schema

feat: hide sequential agents

feat: increase upload toast duration to 10 seconds

* refactor: tool context handling & update Code API Key Dialog

feat: toolContextMap

chore: skipSpecs -> useSpecs

ci: fix handleTools tests

feat: API Key Dialog

* feat: Agent Permissions Admin Controls

feat: replace label with button for prompt permission toggle

feat: update agent permissions

feat: enable experimental agents and streamline capability configuration

feat: implement access control for agents and enhance endpoint menu items

feat: add welcome message for agent selection in localization

feat: add agents permission to access control and update version to 0.7.57

* fix: update types in useAssistantListMap and useMentions hooks for better null handling

* feat: mention agents

* fix: agent tool resource race conditions when deleting agent tool resource files

* feat: add error handling for code execution with user feedback

* refactor: rename AdminControls to AdminSettings for clarity

* style: add gap to button in AdminSettings for improved layout

* refactor: separate agent query hooks and check access to enable fetching

* fix: remove unused provider from agent initialization options, creates issue with custom endpoints

* refactor: remove redundant/deprecated modelOptions from AgentClient processes

* chore: update @librechat/agents to version 1.8.5 in package.json and package-lock.json

* fix: minor styling issues + agent panel uniformity

* fix: agent edge cases when set endpoint is no longer defined

* refactor: remove unused cleanup function call from AppService

* fix: update link in ApiKeyDialog to point to pricing page

* fix: improve type handling and layout calculations in SidePanel component

* fix: add missing localization string for agent selection in SidePanel

* chore: form styling and localizations for upload filesearch/code interpreter

* fix: model selection placeholder logic in AgentConfig component

* style: agent capabilities

* fix: add localization for provider selection and improve dropdown styling in ModelPanel

* refactor: use gpt-4o-mini > gpt-3.5-turbo

* fix: agents configuration for loadDefaultInterface and update related tests

* feat: DALLE Agents support
2024-12-04 15:48:13 -05:00
Danny Avila
ad74350036
🚧 chore: merge latest dev build (#4288)
* fix: agent initialization, add `collectedUsage` handling

* style: improve side panel styling

* refactor(loadAgent): Optimize order agent project ID retrieval

* feat: code execution

* fix: typing issues

* feat: ExecuteCode content part

* refactor: use local state for default collapsed state of analysis content parts

* fix: code parsing in ExecuteCode component

* chore: bump agents package, export loadAuthValues

* refactor: Update handleTools.js to use EnvVar for code execution tool authentication

* WIP

* feat: download code outputs

* fix(useEventHandlers): type issues

* feat: backend handling for code outputs

* Refactor: Remove console.log statement in Part.tsx

* refactor: add attachments to TMessage/messageSchema

* WIP: prelim handling for code outputs

* feat: attachments rendering

* refactor: improve attachments rendering

* fix: attachments, nullish edge case, handle attachments from event stream, bump agents package

* fix filename download

* fix: tool assignment for 'run code' on agent creation

* fix: image handling by adding attachments

* refactor: prevent agent creation without provider/model

* refactor: remove unnecessary space in agent creation success message

* refactor: select first model if selecting provider from empty on form

* fix: Agent avatar bug

* fix: `defaultAgentFormValues` causing boolean typing issue and typeerror

* fix: capabilities counting as tools, causing duplication of them

* fix: formatted messages edge case where consecutive content text type parts with the latter having tool_call_ids would cause consecutive AI messages to be created. furthermore, content could not be an array for tool_use messages (anthropic limitation)

* chore: bump @librechat/agents dependency to version 1.6.9

* feat: bedrock agents

* feat: new Agents icon

* feat: agent titling

* feat: agent landing

* refactor: allow sharing agent globally only if user is admin or author

* feat: initial AgentPanelSkeleton

* feat: AgentPanelSkeleton

* feat: collaborative agents

* chore: add potential authorName as part of schema

* chore: Remove unnecessary console.log statement

* WIP: agent model parameters

* chore: ToolsDialog typing and tool related localization chnages

* refactor: update tool instance type (latest langchain class), and rename google tool to 'google' proper

* chore: add back tools

* feat: Agent knowledge files upload

* refactor: better verbiage for disabled knowledge

* chore: debug logs for file deletions

* chore: debug logs for file deletions

* feat: upload/delete agent knowledge/file-search files

* feat: file search UI for agents

* feat: first pass, file search tool

* chore: update default agent capabilities and info
2024-09-30 17:17:57 -04:00
Danny Avila
8dc5b320bc
📊 refactor: use Parameters from Side Panel for OpenAI, Anthropic, and Custom endpoints (#4092)
* feat: openai parameters

* refactor: anthropic/bedrock params, add preset params for openai, and add azure params

* refactor: use 'compact' schemas for anthropic/openai

* refactor: ensure custom endpoints are properly recognized as valid param endpoints

* refactor: update paramEndpoints check in BaseClient.js

* chore: optimize logging by omitting modelsConfig

* refactor: update label casing in baseDefinitions combobox items

* fix: remove 'stop' model options when using o1 series models

* refactor(AnthropicClient): remove default `stop` value

* refactor: reset params on parameters change

* refactor: remove unused default parameter value map introduced in prior commit

* fix: 'min' typo for 'max' value

* refactor: preset settings

* refactor: replace dropdown for image detail with slider; remove `preventDelayedUpdate` condition from DynamicSlider

* fix: localizations for freq./pres. penalty

* Refactor maxOutputTokens to use coerceNumber in tConversationSchema

* refactor(AnthropicClient): use `getModelMaxOutputTokens`
2024-09-17 22:25:54 -04:00
Danny Avila
45b42830a5
🚀 feat: o1 (#4019)
* feat: o1 default response sender string

* feat: add o1 models to default openai models list, add `no_system_messages` error type; refactor: use error type as localization key

* refactor(MessageEndpointIcon): differentiate openAI icon model color for o1 models

* refactor(AnthropicClient): use new input/output tokens keys; add prompt caching for claude-3-opus

* refactor(BaseClient): to use new input/output tokens keys; update typedefs

* feat: initial o1 model handling, including token cost complexity

* EXPERIMENTAL: special handling for o1 model with custom instructions
2024-09-12 18:15:43 -04:00
Danny Avila
341e086d70
🛠️ fix: Completion Edge Cases & Improve Error Handling UX (#3968)
* fix: edge cases concerning completion response as an array

* refactor: improve invalid request error UX
2024-09-09 20:58:15 -04:00
Danny Avila
d59b62174f
🪨 feat: AWS Bedrock support (#3935)
* feat: Add BedrockIcon component to SVG library

* feat: EModelEndpoint.bedrock

* feat: first pass, bedrock chat. note: AgentClient is returning `agents` as conversation.endpoint

* fix: declare endpoint in initialization step

* chore: Update @librechat/agents dependency to version 1.4.5

* feat: backend content aggregation for agents/bedrock

* feat: abort agent requests

* feat: AWS Bedrock icons

* WIP: agent provider schema parsing

* chore: Update EditIcon props type

* refactor(useGenerationsByLatest): make agents and bedrock editable

* refactor: non-assistant message content, parts

* fix: Bedrock response `sender`

* fix: use endpointOption.model_parameters not endpointOption.modelOptions

* fix: types for step handler

* refactor: Update Agents.ToolCallDelta type

* refactor: Remove unnecessary assignment of parentMessageId in AskController

* refactor: remove unnecessary assignment of parentMessageId (agent request handler)

* fix(bedrock/agents): message regeneration

* refactor: dynamic form elements using react-hook-form Controllers

* fix: agent icons/labels for messages

* fix: agent actions

* fix: use of new dynamic tags causing application crash

* refactor: dynamic settings touch-ups

* refactor: update Slider component to allow custom track class name

* refactor: update DynamicSlider component styles

* refactor: use Constants value for GLOBAL_PROJECT_NAME (enum)

* feat: agent share global methods/controllers

* fix: agents query

* fix: `getResponseModel`

* fix: share prompt a11y issue

* refactor: update SharePrompt dialog theme styles

* refactor: explicit typing for SharePrompt

* feat: add agent roles/permissions

* chore: update @librechat/agents dependency to version 1.4.7 for tool_call_ids edge case

* fix(Anthropic): messages.X.content.Y.tool_use.input: Input should be a valid dictionary

* fix: handle text parts with tool_call_ids and empty text

* fix: role initialization

* refactor: don't make instructions required

* refactor: improve typing of Text part

* fix: setShowStopButton for agents route

* chore: remove params for now

* fix: add streamBuffer and streamRate to help prevent 'Overloaded' errors from Anthropic API

* refactor: remove console.log statement in ContentRender component

* chore: typing, rename Context to Delete Button

* chore(DeleteButton): logging

* refactor(Action): make accessible

* style(Action): improve a11y again

* refactor: remove use/mention of mongoose sessions

* feat: first pass, sharing agents

* feat: visual indicator for global agent, remove author when serving to non-author

* wip: params

* chore: fix typing issues

* fix(schemas): typing

* refactor: improve accessibility of ListCard component and fix console React warning

* wip: reset templates for non-legacy new convos

* Revert "wip: params"

This reverts commit f8067e91d4.

* Revert "refactor: dynamic form elements using react-hook-form Controllers"

This reverts commit 2150c4815d.

* fix(Parameters): types and parameter effect update to only update local state to parameters

* refactor: optimize useDebouncedInput hook for better performance

* feat: first pass, anthropic bedrock params

* chore: paramEndpoints check for endpointType too

* fix: maxTokens to use coerceNumber.optional(),

* feat: extra chat model params

* chore: reduce code repetition

* refactor: improve preset title handling in SaveAsPresetDialog component

* refactor: improve preset handling in HeaderOptions component

* chore: improve typing, replace legacy dialog for SaveAsPresetDialog

* feat: save as preset from parameters panel

* fix: multi-search in select dropdown when using Option type

* refactor: update default showDefault value to false in Dynamic components

* feat: Bedrock presets settings

* chore: config, fix agents schema, update config version

* refactor: update AWS region variable name in bedrock options endpoint to BEDROCK_AWS_DEFAULT_REGION

* refactor: update baseEndpointSchema in config.ts to include baseURL property

* refactor: update createRun function to include req parameter and set streamRate based on provider

* feat: availableRegions via config

* refactor: remove unused demo agent controller file

* WIP: title

* Update @librechat/agents to version 1.5.0

* chore: addTitle.js to handle empty responseText

* feat: support images and titles

* feat: context token updates

* Refactor BaseClient test to use expect.objectContaining

* refactor: add model select, remove header options params, move side panel params below prompts

* chore: update models list, catch title error

* feat: model service for bedrock models (env)

* chore: Remove verbose debug log in AgentClient class following stream

* feat(bedrock): track token spend; fix: token rates, value key mapping for AWS models

* refactor: handle streamRate in `handleLLMNewToken` callback

* chore: AWS Bedrock example config in `.env.example`

* refactor: Rename bedrockMeta to bedrockGeneral in settings.ts and use for AI21 and Amazon Bedrock providers

* refactor: Update `.env.example` with AWS Bedrock model IDs URL and additional notes

* feat: titleModel support for bedrock

* refactor: Update `.env.example` with additional notes for AWS Bedrock model IDs
2024-09-09 12:06:59 -04:00
Danny Avila
a0291ed155
🚧 chore: merge latest dev build to main repo (#3844)
* agents - phase 1 (#30)

* chore: copy assistant files

* feat: frontend and data-provider

* feat: backend get endpoint test

* fix(MessageEndpointIcon): switched to AgentName and AgentAvatar

* fix: small fixes

* fix: agent endpoint config

* fix: show Agent Builder

* chore: install agentus

* chore: initial scaffolding for agents

* fix: updated Assistant logic to Agent Logic for some Agent components

* WIP first pass, demo of agent package

* WIP: initial backend infra for agents

* fix: agent list error

* wip: agents routing

* chore: Refactor useSSE hook to handle different data events

* wip: correctly emit events

* chore: Update @librechat/agentus npm dependency to version 1.0.9

* remove comment

* first pass: streaming agent text

* chore: Remove @librechat/agentus root-level workspace npm dependency

* feat: Agent Schema and Model

* fix: content handling fixes

* fix: content message save

* WIP: new content data

* fix: run step issue with tool calls

* chore: Update @librechat/agentus npm dependency to version 1.1.5

* feat: update controller and agent routes

* wip: initial backend tool and tool error handling support

* wip: tool chunks

* chore: Update @librechat/agentus npm dependency to version 1.1.7

* chore: update tool_call typing, add test conditions and logs

* fix: create agent

* fix: create agent

* first pass: render completed content parts

* fix: remove logging, fix step handler typing

* chore: Update @librechat/agentus npm dependency to version 1.1.9

* refactor: cleanup maps on unmount

* chore: Update BaseClient.js to safely count tokens for string, number, and boolean values

* fix: support subsequent messages with tool_calls

* chore: export order

* fix: select agent

* fix: tool call types and handling

* chore: switch to anthropic for testing

* fix: AgentSelect

* refactor: experimental: OpenAIClient to use array for intermediateReply

* fix(useSSE): revert old condition for streaming legacy client tokens

* fix: lint

* revert `agent_id` to `id`

* chore: update localization keys for agent-related components

* feat: zod schema handling for actions

* refactor(actions): if no params, no zodSchema

* chore: Update @librechat/agentus npm dependency to version 1.2.1

* feat: first pass, actions

* refactor: empty schema for actions without params

* feat: Update createRun function to accept additional options

* fix: message payload formatting; feat: add more client options

* fix: ToolCall component rendering when action has no args but has output

* refactor(ToolCall): allow non-stringy args

* WIP: first pass, correctly formatted tool_calls between providers

* refactor: Remove duplicate import of 'roles' module

* refactor: Exclude 'vite.config.ts' from TypeScript compilation

* refactor: fix agent related types
> - no need to use endpoint/model fields for identifying agent metadata
> - add `provider` distinction for agent-configured 'endpoint'
- no need for agent-endpoint map
- reduce complexity of tools as functions into tools as string[]
- fix types related to above changes
- reduce unnecessary variables for queries/mutations and corresponding react-query keys

* refactor: Add tools and tool_kwargs fields to agent schema

* refactor: Remove unused code and update dependencies

* refactor: Update updateAgentHandler to use req.body directly

* refactor: Update AgentSelect component to use localized hooks

* refactor: Update agent schema to include tools and provider fields

* refactor(AgentPanel): add scrollbar gutter, add provider field to form, fix agent schema required values

* refactor: Update AgentSwitcher component to use selectedAgentId instead of selectedAgent

* refactor: Update AgentPanel component to include alternateName import and defaultAgentFormValues

* refactor(SelectDropDown): allow setting value as option while still supporting legacy usage (string values only)

* refactor: SelectDropdown changes - Only necessary when the available values are objects with label/value fields and the selected value is expected to be a string.

* refactor: TypeError issues and handle provider as option

* feat: Add placeholder for provider selection in AgentPanel component

* refactor: Update agent schema to include author and provider fields

* fix: show expected 'create agent' placeholder when creating agent

* chore: fix localization strings, hide capabilities form for now

* chore: typing

* refactor: import order and use compact agents schema for now

* chore: typing

* refactor: Update AgentForm type to use AgentCapabilities

* fix agent form agent selection issues

* feat: responsive agent selection

* fix: Handle cancelled fetch in useSelectAgent hook

* fix: reset agent form on accordion close/open

* feat: Add agent_id to default conversation for agents endpoint

* feat: agents endpoint request handling

* refactor: reset conversation model on agent select

* refactor: add `additional_instructions` to conversation schema, organize other fields

* chore: casing

* chore: types

* refactor(loadAgentTools): explicitly pass agent_id, do not pass `model` to loadAgentTools for now, load action sets by agent_id

* WIP: initial draft of real agent client initialization

* WIP: first pass, anthropic agent requests

* feat: remember last selected agent

* feat: openai and azure connected

* fix: prioritize agent model for runs unless an explicit override model is passed from client

* feat: Agent Actions

* fix: save agent id to convo

* feat: model panel (#29)

* feat: model panel

* bring back comments

* fix: method still null

* fix: AgentPanel FormContext

* feat: add more parameters

* fix: style issues; refactor: Agent Controller

* fix: cherry-pick

* fix: Update AgentAvatar component to use AssistantIcon instead of BrainCircuit

* feat: OGDialog for delete agent; feat(assistant): update Agent types, introduced `model_parameters`

* feat: icon and general `model_parameters` update

* feat: use react-hook-form better

* fix: agent builder form reset issue when switching panels

* refactor: modularize agent builder form

---------

Co-authored-by: Danny Avila <danny@librechat.ai>

* fix: AgentPanel and ModelPanel type issues and use `useFormContext` and `watch` instead of `methods` directly and `useWatch`.

* fix: tool call issues due to invalid input (anthropic) of empty string

* fix: handle empty text in Part component

---------

Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>

* refactor: remove form ModelPanel and fixed nested ternary expressions in AgentConfig

* fix: Model Parameters not saved correctly

* refactor: remove console log

* feat: avatar upload and get for Agents (#36)

Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>

* chore: update to public package

* fix: typing, optional chaining

* fix: cursor not showing for content parts

* chore: conditionally enable agents

* ci: fix azure test

* ci: fix frontend tests, fix eslint api

* refactor: Remove unused errorContentPart variable

* continue of the agent message PR (#40)

* last fixes

* fix: agentMap

* pr merge test  (#41)

* fix: model icon not fetching correctly

* remove console logs

* feat: agent name

* refactor: pass documentsMap as a prop to allow re-render of assistant form

* refactor: pass documentsMap as a prop to allow re-render of assistant form

* chore: Bump version to 0.7.419

* fix: TypeError: Cannot read properties of undefined (reading 'id')

* refactor: update AgentSwitcher component to use ControlCombobox instead of Combobox

---------

Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
2024-08-31 16:33:51 -04:00
Danny Avila
7536e649d4
🚨 feat: Implement INPUT_LENGTH Error Type (#3866)
* feat: CONTEXT_LENGTH error type

* chore: rename error type

* chore: import order
2024-08-30 15:01:29 -04:00
Danny Avila
a45b384bbc
💾 feat: Anthropic Prompt Caching (#3670)
* wip: initial cache control implementation, add typing for transactions handling

* feat: first pass of Anthropic Prompt Caching

* feat: standardize stream usage as pass in when calculating token counts

* feat: Add getCacheMultiplier function to calculate cache multiplier for different valueKeys and cacheTypes

* chore: imports order

* refactor: token usage recording in AnthropicClient, no need to "correct" as we have the correct amount

* feat: more accurate token counting using stream usage data

* feat: Improve token counting accuracy with stream usage data

* refactor: ensure more accurate than not token estimations if custom instructions or files are not being resent with every request

* refactor: cleanup updateUserMessageTokenCount to allow transactions to be as accurate as possible even if we shouldn't update user message token counts

* ci: fix tests
2024-08-17 03:24:09 -04:00
Danny Avila
2ad097647c
fix: Wait for Initial Message Save & Correct Latest Message (#3399)
* chore: assistants, unsupported assistant, better logging

* chore: remove unnecessary logger in validateAssistant middleware

* fix: resolve initial conversation save/promise before saving response

* chore: Import and organize dependencies in Speech component

* fix: conversation statefulness
- Latest Message (at index 0) should not be reset if existing convo
- add debugging context for clearAllLatestMessages
- Added logging concerning latest Message updates (dev mode only)
- update latest message Set logic, also checks for change in conversation Id
- consolidated latest message helpers to client/src/utils/messages.ts
2024-07-20 01:51:59 -04:00
Danny Avila
f6125ccd59
🌿 fix: Message Route Improvements pt. 2 (#3379)
* fix: edge case with debugTraverse function in parsers.js

* chore: Update error message in audio stream processing

* refactor: Add ONE_MINUTE and THIRTY_SECONDS options to Time enum

* fix: logging
2024-07-18 13:56:19 -04:00
Danny Avila
5d40d0a37a
⚙️ feat: Adjust Rate of Stream Progress (#3244)
* chore: bump data-provider and add MESSAGES CacheKey

* refactor: avoid saving messages while streaming, save partial text to cache instead

* fix(ci): processChunks

* chore: logging aborted request to debug

* feat: set stream rate for token processing

* chore: specify default stream rate

* fix(ci): Update AppService.js to use optional chaining for endpointLocals assignment

* refactor: abstract the error handler

* feat: streamRate for assistants; refactor: update default rate for token

* refactor: update error handling in assistants/errors.js

* refactor: update error handling in assistants/errors.js
2024-07-17 10:47:17 -04:00
Danny Avila
d5d188eebf
🔐 fix: Enhance Message & Image Access Security (#3363)
* chore: slight refactor

* fix: prevent message updates unless explicitly owned

* refactor: rethrow errors, update deleteMessagesSince (not used), add basic tests

* fix: Add path normalization and validation to image request middleware

* fix: image validation path security
2024-07-17 09:51:03 -04:00
Danny Avila
a2fd975cd5
🚤 refactor: Optimize Request Lifecycle Speeds (#3222)
* refactor: optimize backend operations for client requests

* fix: message styling

* refactor: Improve handleKeyUp logic in StreamRunManager.js and handleText.js

* refactor: Improve handleKeyUp logic in StreamRunManager.js and handleText.js

* fix: clear new convo messages on clear all convos

* fix: forgot to pass userId to getConvo

* refactor: update getPartialText to send basePayload.text
2024-06-28 08:44:47 -04:00
Danny Avila
156c52e293
🌿 feat: Multi-response Streaming (#3191)
* chore: comment back handlePlusCommand

* chore: ignore .git dir

* refactor: pass newConversation to `useSelectMention`

refactor: pass newConversation to Mention component

refactor: useChatFunctions for modular use of `ask` and `regenerate`

refactor: set latest message only for the first index in useChatFunctions

refactor: pass setLatestMessage to useChatFunctions

refactor: Pass setSubmission to useChatFunctions for submission handling

refactor: consolidate event handlers to separate hook from useSSE

WIP: additional response handlers

feat: responsive added convo, clears on new chat/navigating to chat, assistants excluded

feat: Add conversationByKeySelector to select any conversation by index

WIP: handle second submission with messages paired to root

* style: surface-primary-contrast

* refactor: remove unnecessary console.log statement in useChatFunctions

* refactor: Consolidate imports in ChatForm and Input hooks

* refactor: compositional usage of useSSE for multiple streams

* WIP: set latest 'multi' message

* WIP: first pass, added response streaming

* pass: performant multi-message stream

* fix: styling and message render

* second pass: modular, performant multi-stream

* fix: align parentMessageId of multiMessage

* refactor: move resetting latestMultiMessage

* chore: update footer text in Chat component

* fix: stop button styling

* fix: handle abortMessage request for multi-response

* clear messages but bug with latest message reset present

* fix: add delay for additional message generation

* fix: access LAST_CONVO_SETUP by index

* style: add div to prevent layout shift before hover buttons render

* chore: Update Message component styling for card messages

* chore: move hook use order

* fix: abort middleware using unsent field from req.body

* feat: support multi-response stream from initial message

* refactor: buildTree function to improve readability and remove unused code

* feat: add logger for frontend dev

* refactor: use depth to track if message is really last in its branch

* fix(buildTree): default export

* fix: share parent message Id and avoid duplication error for multi-response streams

* fix: prevent addedConvo reset to response convo

* feat: allow setting multi message as latest message to control which to respond to

* chore: wrap setSiblingIdxRev with useCallback

* chore: styling and allow editing messages

* style: styling fixes

* feat: Add "AddMultiConvo" component to Chat Header

* feat: prevent clearing added convos on endpoint, preset, mention, or modelSpec switch

* fix: message styling fixes, mainly related to code blocks

* fix: stop button visibility logic

* fix: Handle edge case in abortMiddleware for non-existant `abortControllers`

* refactor: optimize/memoize icons

* chore(GoogleClient): change info to debug logs

* style: active message styling

* style: prevent layout shift due to placeholder row

* chore: remove unused code

* fix: Update BaseClient to handle optional request body properties

* fix(ci): `onStart` now accepts 2 args, the 2nd being responseMessageId

* chore: bump data-provider
2024-06-25 03:02:38 -04:00
Danny Avila
0294cfc881
v0.7.3 (#3067)
* refactor: revert BaseClient to use node-fetch instead of undici

* chore: bump version

* chore: update npm dependencies to latest versions

* chore: fix custom footer
2024-06-15 12:17:10 -04:00
Danny Avila
4416f69a9b
🚀 refactor: Use Undici Instead of Node-Fetch, prevent Event Close, add Index (#3052)
* feat: Add index to conversationId field in messageSchema

* refactor: prevent immediate event close on error

* refactor: use undici instead of node-fetch in non-Bun environment
2024-06-13 13:38:15 -04:00
Danny Avila
40685f6eb4
🚀 refactor: Enhance Custom Endpoints, Message Logic, and Payload Handling (#2895)
* chore: use node-fetch for OpenAIClient fetch key for non-crashing usage of AbortController in Bun runtime

* chore: variable order

* fix(useSSE): prevent finalHandler call in abortConversation to update messages/conversation after user navigated away

* chore: params order

* refactor: organize intermediate message logic and ensure correct variables are passed

* fix: Add stt and tts routes before upload limiters, prevent bans

* fix(abortRun): temp fix to delete unfinished messages to avoid message thread parent relationship issues

* refactor: Update AnthropicClient to use node-fetch for fetch key and add proxy support

* fix(gptPlugins): ensure parentMessageId/messageId relationship is maintained

* feat(BaseClient): custom fetch function to analyze/edit payloads just before sending (also prevents abortController crash on Bun runtime)

* feat: `directEndpoint` and `titleMessageRole` custom endpoint options

* chore: Bump version to 0.6.6 in data-provider package.json
2024-05-28 14:52:12 -04:00