Commit graph

95 commits

Author SHA1 Message Date
Danny Avila
5f2d1c5dc9
👁️ feat: Azure Mistral OCR Strategy (#7888)
* 👁️ feat: Add Azure Mistral OCR strategy and endpoint integration

This commit introduces a new OCR strategy named 'azure_mistral_ocr', allowing the use of a Mistral OCR endpoint deployed on Azure. The configuration, schemas, and file upload strategies have been updated to support this integration, enabling seamless OCR processing via Azure-hosted Mistral services.

* 🗑️ chore: Clean up .gitignore by removing commented-out uncommon directory name

* chore: remove unused vars

* refactor: Move createAxiosInstance to packages/api/utils and update imports

- Removed the createAxiosInstance function from the config module and relocated it to a new utils module for better organization.
- Updated import paths in relevant files to reflect the new location of createAxiosInstance.
- Added tests for createAxiosInstance to ensure proper functionality and proxy configuration handling.

* chore: move axios helpers to packages/api

- Added logAxiosError function to @librechat/api for centralized error logging.
- Updated imports across various files to use the new logAxiosError function.
- Removed the old axios.js utility file as it is no longer needed.

* chore: Update Jest moduleNameMapper for improved path resolution

- Added a new mapping for '~/' to resolve module paths in Jest configuration, enhancing import handling for the project.

* feat: Implement Mistral OCR API integration in TS

* chore: Update MistralOCR tests based on new imports

* fix: Enhance MistralOCR configuration handling and tests

- Introduced helper functions for resolving configuration values from environment variables or hardcoded settings.
- Updated the uploadMistralOCR and uploadAzureMistralOCR functions to utilize the new configuration resolution logic.
- Improved test cases to ensure correct behavior when mixing environment variables and hardcoded values.
- Mocked file upload and signed URL responses in tests to validate functionality without external dependencies.

* feat: Enhance MistralOCR functionality with improved configuration and error handling

- Introduced helper functions for loading authentication configuration and resolving values from environment variables.
- Updated uploadMistralOCR and uploadAzureMistralOCR functions to utilize the new configuration logic.
- Added utility functions for processing OCR results and creating error messages.
- Improved document type determination and result aggregation for better OCR processing.

* refactor: Reorganize OCR type imports in Mistral CRUD file

- Moved OCRResult, OCRResultPage, and OCRImage imports to a more logical grouping for better readability and maintainability.

* feat: Add file exports to API and create files index

* chore: Update OCR types for enhanced structure and clarity

- Redesigned OCRImage interface to include mandatory fields and improved naming conventions.
- Added PageDimensions interface for better representation of page metrics.
- Updated OCRResultPage to include dimensions and mandatory images array.
- Refined OCRResult to include document annotation and usage information.

* refactor: use TS counterpart of uploadOCR methods

* ci: Update MistralOCR tests to reflect new OCR result structure

* chore: Bump version of @librechat/api to 1.2.3 in package.json and package-lock.json

* chore: Update CONFIG_VERSION to 1.2.8

* chore: remove unused sendEvent function from config module (now imported from '@librechat/api')

* chore: remove MistralOCR service files and tests (now in '@librechat/api')

* ci: update logger import in ModelService tests to use @librechat/data-schemas

---------

Co-authored-by: arthurolivierfortin <arthurolivier.fortin@gmail.com>
2025-06-13 15:14:57 -04:00
Danny Avila
a57224c1d5
🧑‍💻 fix: Agents Config Defaults and Avatar Uploads Across File Strategies (#7814)
* fix: avatar processing across storage services, uniqueness by agent ID, prevent overwriting user avatar

* fix: sanitize file paths in deleteLocalFile function to prevent invalid path errors

* fix: correct spelling of 'agentsEndpointSchema' in agents.js and config.ts

* fix: default app.locals agents configuration setup and add agent endpoint schema default
2025-06-10 09:53:15 -04:00
Danny Avila
29ef91b4dd
🧠 feat: User Memories for Conversational Context (#7760)
* 🧠 feat: User Memories for Conversational Context

chore: mcp typing, use `t`

WIP: first pass, Memories UI

- Added MemoryViewer component for displaying, editing, and deleting user memories.
- Integrated data provider hooks for fetching, updating, and deleting memories.
- Implemented pagination and loading states for better user experience.
- Created unit tests for MemoryViewer to ensure functionality and interaction with data provider.
- Updated translation files to include new UI strings related to memories.

chore: move mcp-related files to own directory

chore: rename librechat-mcp to librechat-api

WIP: first pass, memory processing and data schemas

chore: linting in fileSearch.js query description

chore: rename librechat-api to @librechat/api across the project

WIP: first pass, functional memory agent

feat: add MemoryEditDialog and MemoryViewer components for managing user memories

- Introduced MemoryEditDialog for editing memory entries with validation and toast notifications.
- Updated MemoryViewer to support editing and deleting memories, including pagination and loading states.
- Enhanced data provider to handle memory updates with optional original key for better management.
- Added new localization strings for memory-related UI elements.

feat: add memory permissions management

- Implemented memory permissions in the backend, allowing roles to have specific permissions for using, creating, updating, and reading memories.
- Added new API endpoints for updating memory permissions associated with roles.
- Created a new AdminSettings component for managing memory permissions in the frontend.
- Integrated memory permissions into the existing roles and permissions schemas.
- Updated the interface to include memory settings and permissions.
- Enhanced the MemoryViewer component to conditionally render admin settings based on user roles.
- Added localization support for memory permissions in the translation files.

feat: move AdminSettings component to a new position in MemoryViewer for better visibility

refactor: clean up commented code in MemoryViewer component

feat: enhance MemoryViewer with search functionality and improve MemoryEditDialog integration

- Added a search input to filter memories in the MemoryViewer component.
- Refactored MemoryEditDialog to accept children for better customization.
- Updated MemoryViewer to utilize the new EditMemoryButton and DeleteMemoryButton components for editing and deleting memories.
- Improved localization support by adding new strings for memory filtering and deletion confirmation.

refactor: optimize memory filtering in MemoryViewer using match-sorter

- Replaced manual filtering logic with match-sorter for improved search functionality.
- Enhanced performance and readability of the filteredMemories computation.

feat: enhance MemoryEditDialog with triggerRef and improve updateMemory mutation handling

feat: implement access control for MemoryEditDialog and MemoryViewer components

refactor: remove commented out code and create runMemory method

refactor: rename role based files

feat: implement access control for memory usage in AgentClient

refactor: simplify checkVisionRequest method in AgentClient by removing commented-out code

refactor: make `agents` dir in api package

refactor: migrate Azure utilities to TypeScript and consolidate imports

refactor: move sanitizeFilename function to a new file and update imports, add related tests

refactor: update LLM configuration types and consolidate Azure options in the API package

chore: linting

chore: import order

refactor: replace getLLMConfig with getOpenAIConfig and remove unused LLM configuration file

chore: update winston-daily-rotate-file to version 5.0.0 and add object-hash dependency in package-lock.json

refactor: move primeResources and optionalChainWithEmptyCheck functions to resources.ts and update imports

refactor: move createRun function to a new run.ts file and update related imports

fix: ensure safeAttachments is correctly typed as an array of TFile

chore: add node-fetch dependency and refactor fetch-related functions into packages/api/utils, removing the old generators file

refactor: enhance TEndpointOption type by using Pick to streamline endpoint fields and add new properties for model parameters and client options

feat: implement initializeOpenAIOptions function and update OpenAI types for enhanced configuration handling

fix: update types due to new TEndpointOption typing

fix: ensure safe access to group parameters in initializeOpenAIOptions function

fix: remove redundant API key validation comment in initializeOpenAIOptions function

refactor: rename initializeOpenAIOptions to initializeOpenAI for consistency and update related documentation

refactor: decouple req.body fields and tool loading from initializeAgentOptions

chore: linting

refactor: adjust column widths in MemoryViewer for improved layout

refactor: simplify agent initialization by creating loadAgent function and removing unused code

feat: add memory configuration loading and validation functions

WIP: first pass, memory processing with config

feat: implement memory callback and artifact handling

feat: implement memory artifacts display and processing updates

feat: add memory configuration options and schema validation for validKeys

fix: update MemoryEditDialog and MemoryViewer to handle memory state and display improvements

refactor: remove padding from BookmarkTable and MemoryViewer headers for consistent styling

WIP: initial tokenLimit config and move Tokenizer to @librechat/api

refactor: update mongoMeili plugin methods to use callback for better error handling

feat: enhance memory management with token tracking and usage metrics

- Added token counting for memory entries to enforce limits and provide usage statistics.
- Updated memory retrieval and update routes to include total token usage and limit.
- Enhanced MemoryEditDialog and MemoryViewer components to display memory usage and token information.
- Refactored memory processing functions to handle token limits and provide feedback on memory capacity.

feat: implement memory artifact handling in attachment handler

- Enhanced useAttachmentHandler to process memory artifacts when receiving updates.
- Introduced handleMemoryArtifact utility to manage memory updates and deletions.
- Updated query client to reflect changes in memory state based on incoming data.

refactor: restructure web search key extraction logic

- Moved the logic for extracting API keys from the webSearchAuth configuration into a dedicated function, getWebSearchKeys.
- Updated webSearchKeys to utilize the new function for improved clarity and maintainability.
- Prevents build time errors

feat: add personalization settings and memory preferences management

- Introduced a new Personalization tab in settings to manage user memory preferences.
- Implemented API endpoints and client-side logic for updating memory preferences.
- Enhanced user interface components to reflect personalization options and memory usage.
- Updated permissions to allow users to opt out of memory features.
- Added localization support for new settings and messages related to personalization.

style: personalization switch class

feat: add PersonalizationIcon and align Side Panel UI

feat: implement memory creation functionality

- Added a new API endpoint for creating memory entries, including validation for key and value.
- Introduced MemoryCreateDialog component for user interface to facilitate memory creation.
- Integrated token limit checks to prevent exceeding user memory capacity.
- Updated MemoryViewer to include a button for opening the memory creation dialog.
- Enhanced localization support for new messages related to memory creation.

feat: enhance message processing with configurable window size

- Updated AgentClient to use a configurable message window size for processing messages.
- Introduced messageWindowSize option in memory configuration schema with a default value of 5.
- Improved logic for selecting messages to process based on the configured window size.

chore: update librechat-data-provider version to 0.7.87 in package.json and package-lock.json

chore: remove OpenAPIPlugin and its associated tests

chore: remove MIGRATION_README.md as migration tasks are completed

ci: fix backend tests

chore: remove unused translation keys from localization file

chore: remove problematic test file and unused var in AgentClient

chore: remove unused import and import directly for JSDoc

* feat: add api package build stage in Dockerfile for improved modularity

* docs: reorder build steps in contributing guide for clarity
2025-06-07 18:52:22 -04:00
Marco Beretta
442b149d55
🖼️ feat: Avatar GIF Support & Dynamic Extensions (#7657) 2025-06-02 07:51:38 -04:00
Danny Avila
f9d40784f0
🔧 fix: Dev Deployment, Mistral OCR Error, and UI Consistency (#7668)
* 🔧 fix: Update ProgressText and ToolCall components for improved error handling and localization

* 🔧 chore: Format ESLint configuration for improved readability and remove unused rule

* 🔧 refactor: Simplify ProgressText component logic for better readability and maintainability

* 🔧 refactor: Update ProgressText and ToolCall components for improved layout consistency

* 🔧 refactor: Simplify icon rendering in TTS components and enhance button rendering logic in HoverButtons

* 🔧 refactor: Update placeholder logic in VariableForm component to simply use variable name

* fix: .docx. .pptx Mistral OCR Error with `image_limit=0`

* chore: Update deploy workflow to include conditions for successful dev branch deployment and streamline deployment steps

* ci: Set image_limit to 0 in MistralOCR service tests for consistent behavior
2025-06-01 17:48:19 -04:00
Danny Avila
a2fc7d312a
🏗️ refactor: Extract DB layers to data-schemas for shared use (#7650)
* refactor: move model definitions and database-related methods to packages/data-schemas

* ci: update tests due to new DB structure

fix: disable mocking `librechat-data-provider`

feat: Add schema exports to data-schemas package

- Introduced a new schema module that exports various schemas including action, agent, and user schemas.
- Updated index.ts to include the new schema exports for better modularity and organization.

ci: fix appleStrategy tests

fix: Agent.spec.js

ci: refactor handleTools tests to use MongoMemoryServer for in-memory database

fix: getLogStores imports

ci: update banViolation tests to use MongoMemoryServer and improve session mocking

test: refactor samlStrategy tests to improve mock configurations and user handling

ci: fix crypto mock in handleText tests for improved accuracy

ci: refactor spendTokens tests to improve model imports and setup

ci: refactor Message model tests to use MongoMemoryServer and improve database interactions

* refactor: streamline IMessage interface and move feedback properties to types/message.ts

* refactor: use exported initializeRoles from `data-schemas`, remove api workspace version (this serves as an example of future migrations that still need to happen)

* refactor: update model imports to use destructuring from `~/db/models` for consistency and clarity

* refactor: remove unused mongoose imports from model files for cleaner code

* refactor: remove unused mongoose imports from Share, Prompt, and Transaction model files for cleaner code

* refactor: remove unused import in Transaction model for cleaner code

* ci: update deploy workflow to reference new Docker Dev Branch Images Build and add new workflow for building Docker images on dev branch

* chore: cleanup imports
2025-05-30 22:18:13 -04:00
Danny Avila
2f462c9b3c
🔧 refactor: Centralize Default Agent Capabilities and Better Logging (#7598)
* refactor: Simplify grid column calculation in SourcesGroup component

* refactor: Centralize default agent capabilities and simplify capability assignment

* Edge case: use defined/fallback capabilities for ephemeral agents when the "agents" endpoint is not enabled

* refactor: consolidate gemini 2 vision check

* feat: enhance capability check logging for agents

* chore: update librechat-data-provider version to 0.7.86

* refactor: import default agent capabilities for enhanced capability management

* chore: standardize quotes in error message check for consistency

* fix: improve error logging both client and api-side for mistral ocr upload errors

* ci: update error handling in MistralOCR tests to use specific error message
2025-05-27 15:48:43 -04:00
Danny Avila
0dbbf7de04
🔎 feat: Native Web Search with Citation References (#7516)
* WIP: search tool integration

* WIP: Add web search capabilities and API key management to agent actions

* WIP: web search capability to agent configuration and selection

* WIP: Add web search capability to backend agent configuration

* WIP: add web search option to default agent form values

* WIP: add attachments for web search

* feat: add plugin for processing web search citations

* WIP: first pass, Citation UI

* chore: remove console.log

* feat: Add AnimatedTabs component for tabbed UI functionality

* refactor: AnimatedTabs component with CSS animations and stable ID generation

* WIP example content

* feat: SearchContext for managing search results apart from MessageContext

* feat: Enhance AnimatedTabs with underline animation and state management

* WIP: first pass, Implement dynamic tab functionality in Sources component with search results integration

* fix: Update class names for improved styling in Sources and AnimatedTabs components

* feat: Improve styling and layout in Sources component with enhanced button and item designs

* feat: Refactor Sources component to integrate OGDialog for source display and improve layout

* style: Update background color in SourceItem and SourcesGroup components for improved visibility

* refactor: Sources component to enhance SourceItem structure and improve favicon handling

* style: Adjust font size of domain text in SourceItem for better readability

* feat: Add localization for citation source and details in CompositeCitation component

* style: add theming to Citation components

* feat: Enhance SourceItem component with dialog support and improved hovercard functionality

* feat: Add localization for sources tab and image alt text in Sources component

* style: Replace divs with spans for better semantic structure in CompositeCitation and Citation components

* refactor: Sources component to use useMemo for tab generation and improve performance

* chore: bump @librechat/agents to v2.4.318

* chore: update search result types

* fix: search results retrieval in ContentParts component, re-render attachments when expected

* feat: update sources style/types to use latest search result structure

* style: enhance Dialog (expanded) SourceItem component with link wrapping and improved styling

* style: update ImageItem component styling for improved title visibility

* refactor: remove SourceItemBase component and adjust SourceItem layout for improved styling

* chore: linting twcss order

* fix: prevent FileAttachment from rendering search attachments

* fix: append underscore to responseMessageId for unique identification to prevent mapping of previous latest message's attachments

* chore: remove unused parameter 'useSpecs' from loadTools function

* chore: twcss order

* WIP: WebSearch Tool UI

* refactor: add limit parameter to StackedFavicons for customizable source display

* refactor: optimize search results memoization by making more granular and separate conerns

* refactor: integrated StackedFavicons to WebSearch mid-run

* chore: bump @librechat/agents to expose handleToolCallChunks

* chore: use typedefs from dedicated file instead of defining them in AgentClient module

* WIP: first pass, search progress results

* refactor: move createOnSearchResults function to a dedicated search module

* chore: bump @librechat/agents to v2.4.320

* WIP: first pass, search results processed UX

* refactor: consolidate context variables in createOnSearchResults function

* chore: bump @librechat/agents to v2.4.321

* feat: add guidelines for web search tool response formatting in loadTools function

* feat: add isLast prop to Part component and update WebSearch logic for improved state handling

* style: update Hovercard styles for improved UI consistency

* feat: export FaviconImage component for improved accessibility in other modules

* refactor: export getCleanDomain function and use FaviconImage in Citation component for improved source representation

* refactor: implement SourceHovercard component for consistency and DRY compliance

* fix: replace <p> with <span> for snippet and title in SourceItem and SourceHovercard for consistency

* style: `not-prose`

* style: remove 'not-prose' class for consistency in SourceItem, Citation, and SourceHovercard components, adjust style classes

* refactor: `imageUrl` on hover and prevent duplicate sources

* refactor: enhance SourcesGroup dialog layout and improve source item presentation

* refactor: reorganize Web Components, save in same directory

* feat: add 'news' refType to refTypeMap for citation sources

* style: adjust Hovercard width for improved layout

* refactor: update tool usage guidelines for improved clarity and execution

* chore: linting

* feat: add Web Search badge with initial permissions and local storage logic

* feat: add webSearch support to interface and permissions schemas

* feat: implement Web Search API key management and localization updates

* feat: refactor Web Search API key handling and integrate new search API key form

* fix: remove unnecessary visibility state from FileAttachment component

* feat: update WebSearch component to use Globe icon and localized search label

* feat: enhance ApiKeyDialog with dropdown for reranker selection and update translations

* feat: implement dropdown menus for engine, scraper, and reranker selection in ApiKeyDialog

* chore: linting and add unknown instead of `any` type

* feat: refactor ApiKeyDialog and useAuthSearchTool for improved API key management

* refactor: update ocrSchema to use template literals for default apiKey and baseURL

* feat: add web search configuration and utility functions for environment variable extraction

* fix: ensure filepath is defined before checking its prefix in useAttachmentHandler

* feat: enhance web search functionality with improved configuration and environment variable extraction for authFields

* fix: update auth type in TPluginAction and TUpdateUserPlugins to use Partial<Record<string, string>>

* feat: implement web search authentication verification and enhance webSearchAuth structure

* feat: enhance ephemeral agent handling with new web search capability and type definition

* feat: enhance isEphemeralAgent function to include web search selection

* feat: refactor verifyWebSearchAuth to improve key handling and authentication checks

* feat: implement loadWebSearchAuth function for improved web search authentication handling

* feat: enhance web search authentication with new configuration options and refactor related types

* refactor: rename search engine to search provider and update related localization keys

* feat: update verifyWebSearchAuth to handle multiple authentication types and improve error handling

* feat: update ApiKeyDialog to accept authTypes prop and remove isUserProvided check

* feat: add tests for extractWebSearchEnvVars and loadWebSearchAuth functions

* feat: enhance loadWebSearchAuth to support specific service checks for providers, scrapers, and rerankers

* fix: update web search configuration key and adjust auth result handling in loadTools function

* feat: add new progress key for repeated web searching and update localization

* chore: bump @librechat/agents to 2.4.322

* feat: enhance loadTools function to include ISO time and improve search tool logging

* feat: update StackedFavicons to handle negative start index and improve citation attribution styling and text

* chore: update .gitignore to categorize AI-related files

* fix: mobile responsiveness of sources/citations hovercards

* feat: enhance source display with improved line clamping for better readability

* chore: bump @librechat/agents to v2.4.33

* feat: add handling for image sources in references mapping

* chore: bump librechat-data-provider version to 0.7.84

* chore: bump @librechat/agents version to 2.4.34

* fix: update auth handling to support multiple auth types in tools and allow key configuration in agent panel

* chore: remove redundant agent attribution text from search form

* fix: web search auth uninstall

* refactor: convert CheckboxButton to a forwardRef component and update setValue callback signature

* feat: add triggerRef prop to ApiKeyDialog components for improved dialog control

* feat: integrate triggerRef in CodeInterpreter and WebSearch components for enhanced dialog management

* feat: enhance ApiKeyDialog with additional links for Firecrawl and Jina API key guidance

* feat: implement web search configuration handling in ApiKeyDialog and add tests for dropdown visibility

* fix: update webSearchConfig reference in config route for correct payload assignment

* feat: update ApiKeyDialog to conditionally render sections based on authTypes and modify loadWebSearchAuth to correctly categorize authentication types

* feat: refactor ApiKeyDialog and related tests to use SearchCategories and RerankerTypes enums and remove nested ternaries

* refactor: move ThinkingButton rendering to improve layout consistency in ContentParts

* feat: integrate search context into Markdown component to conditionally include unicodeCitation plugin

* chore: bump @librechat/agents to v2.4.35

* chore: remove unused 18n key

* ci: add WEB_SEARCH permission testing and update AppService tests for new webSearch configuration

* ci: add more comprehensive tests for loadWebSearchAuth to validate authentication handling and authTypes structure

* chore: remove debugging console log from web.spec.ts to clean up test output
2025-05-23 17:40:40 -04:00
Danny Avila
c8f5f5131e
🔧 fix: File Deletion for Azure Assistants API (#7466) 2025-05-20 08:37:39 -04:00
Marco Beretta
c79ee32006
🖼️ feat: Tool Call and Loading UI Refresh, Image Resize Config (#7086)
*  feat: Enhance Spinner component with customizable properties and improved animation

* 🔧 fix: Replace Loader with Spinner in RunCode component and update FilePreview to use Spinner for progress indication

*  feat: Refactor icons in CodeProgress and CancelledIcon components; enhance animation and styling in ExecuteCode and ProgressText components

*  feat: Refactor attachment handling in ExecuteCode component; replace individual attachment rendering with AttachmentGroup for improved structure

*  feat: Refactor dialog components for improved accessibility and styling; integrate Skeleton loading state in Image component

*  feat: Refactor ToolCall component to use ToolCallInfo for better structure; replace ToolPopover with AttachmentGroup; enhance ProgressText with error handling and improved UI elements

* 🔧 fix: Remove unnecessary whitespace in ProgressText

* 🔧 fix: Remove unnecessary margin from AgentFooter and AgentPanel components; clean up SidePanel imports

*  feat: Enhance ToolCall and ToolCallInfo components with improved styling; update translations and add warning text color to Tailwind config

* 🔧 fix: Update import statement for useLocalize in ToolCallInfo component; fix: chatform transition

*  feat: Refactor ToolCall and ToolCallInfo components for improved structure and styling; add optimized code block for better output display

*  feat: Implement OpenAI image generation component; add progress tracking and localization for user feedback

* 🔧 fix: Adjust base duration values for image generation; optimize timing for quality settings

* chore: remove unnecessary space

*  feat: Enhance OpenAI image generation with editing capabilities; update localization for progress feedback

*  feat: Add download functionality to images; enhance DialogImage component with download button

*  feat: Enhance image resizing functionality; support custom percentage and pixel dimensions in resizeImageBuffer
2025-05-19 19:23:11 -04:00
Danny Avila
c0ebb434a6
🎨 feat: OpenAI Image Tools (GPT-Image-1) (#7079)
* wip: OpenAI Image Generation Tool with customizable options

* WIP: First pass OpenAI Image Generation Tool and integrate into existing tools

* 🔀 fix: Comment out unused validation for image generation tool parameters

* 🔀 refactor: Update primeResources function parameters for better destructuring

* feat: Add image_edit resource to EToolResources and update AgentToolResources interface

* feat: Enhance file retrieval with tool resource filtering for image editing

* refactor: add OpenAI Image Tools for generation and editing, refactor related components, pass current request image attachments as tool resources for editing

* refactor: Remove commented-out code and clean up API key retrieval in createOpenAIImageTools function

* fix: show message attachments in shared links

* fix: Correct parent message retrieval logic for regenerated messages in useChatFunctions

* fix: Update primeResources to utilize requestFileSet for image file processing

* refactor: Improve description for image generation tool and clarify usage conditions, only provide edit tool if there are images available to edit

* chore: Update OpenAI Image Tools icon to use local asset

* refactor: Update image generation tool description and logic to prioritize editing tool when files are uploaded

* refactor: Enhance image tool descriptions to clarify usage conditions and note potential unavailability of uploaded images

* refactor: Update useAttachmentHandler to accept queryClient to update query cache with newly created file

* refactor: Add customizable descriptions and prompts for OpenAI image generation and editing tools

* chore: Update comments to use JSDoc style for better clarity and consistency

* refactor: Rename config variable to clientConfig for clarity and update signal handling in image generation

* refactor: Update axios request configuration to include derived signal and baseURL for improved request handling

* refactor: Update baseURL environment variable for OpenAI image generation tool configuration

* refactor: Enhance axios request configuration with conditional headers and improved clientConfig setup

* chore: Update comments for clarity and remove unnecessary lines in OpenAI image tools

* refactor: Update description for image generation without files to clarify user instructions

* refactor: Simplify target parent message logic for regeneration and resubmission cases

* chore: Remove backticks from error messages in image generation and editing functions

* refactor: Rename toolResources to toolResourceSet for clarity in file retrieval functions

* chore: Remove redundant comments and clean up TODOs in OpenAI image tools

* refactor: Rename fileStrategy to appFileStrategy for clarity and improve error handling in image processing

* chore: Update react-resizable-panels to version 2.1.8 in package.json and package-lock.json

* chore: Ensure required validation for logs and Code of Conduct agreement in bug report template

* fix: Update ArtifactPreview to use startupConfig and currentCode from memoized props to prevent unnecessary re-renders

* fix: improve robustness of `save & submit` when used from a user-message with existing attachments

* fix: add null check for artifact index in CodeEditor to prevent errors, trigger re-render on artifact ID change

* fix: standardize default values for artifact properties in Artifact component, avoiding prematurely setting an "empty/default" artifact

* fix: reset current artifact ID before setting a new one in ArtifactButton to ensure correct state management

* chore: rename `setArtifactId` variable to `setCurrentArtifactId`  for consistency

* chore: update type annotations in File and S3 CRUD functions for consistency

* refactor: improve image handling in OpenAI tools by using image_id references and enhance tool context for image editing

* fix: update image_ids schema in image_edit_oai to enforce presence and provide clear guidelines for usage

* fix: enhance file fetching logic to ensure user-specific and dimension-validated results

* chore: add details on image generation and editing capabilities with various models
2025-04-26 04:30:58 -04:00
Danny Avila
d32f34e5d7
📝 fix: Mistral OCR Image Support and Azure Agent Titles (#6901)
* fix: azure title model

* refactor: typing for uploadMistralOCR

* fix: update conversation ID handling in useSSE for better state management, only use PENDING_CONVO for new conversations

* fix: streamline conversation ID handling in useSSE for simplicity, only needs state update to prevent draft from applying

* fix: update performOCR and tests to support document and image URLs with appropriate types
2025-04-15 18:03:56 -04:00
Danny Avila
339882eea4
💾 refactor: Enhance Memory In Image Encodings & Client Disposal (#6852)
* 💾 chore: Clear Additional Properties in `disposeClient`

* refactor: stream handling and base64 conversion in encode.js to better free memory
2025-04-12 20:53:38 -04:00
Danny Avila
953e9732d9
🔧 fix: Chat Middleware, Zod Conversion, Auto-Save and S3 URL Refresh (#6720)
* 🔧 feat: Add configurable S3 URL refresh expiry time

* fix: Set default width and height for URLIcon component in case container style results in NaN

* refactor: Enhance auto-save functionality with debounced restore methods

* feat: Add support for additionalProperties in JSON schema conversion to Zod

* test: Add tests for additionalProperties handling in JSON schema to Zod conversion

* chore: Reorder import statements for better readability in ask route

* fix: Handle additional successful response status code (200) in SSE error handler

* fix: add missing rate limiting middleware for bedrock and agent chat routes

* fix: update moderation middleware to check feature flag before processing requests

* fix: add moderation middleware to chat routes for text moderation

* Revert "refactor: Enhance auto-save functionality with debounced restore methods"

This reverts commit d2e4134d1f.

* refactor: Move base64 encoding/decoding functions to top-level scope and optimize input handling
2025-04-03 20:42:56 -04:00
Danny Avila
c4f1da26b3
🔄 fix: Avatar & Error Handling Enhancements (#6687)
* fix: Ensure safe access to agent capabilities in AgentConfig

* fix: don't show agent builder if agents endpoint is not enabled

* fix: Improve error logging for MCP tool calls

* fix: Enhance error message for MCP tool failures

* feat: Add optional spec and iconURL properties to TEndpointOption type

* chore: Update condition to use constant for new conversation parameter

* feat: Enhance abort error handling with additional endpoint options to properly render error message fields

* fix: Throw error instead of returning message for failed MCP tool calls

* refactor: separate logic to generate new S3 URLs for expired links

* feat: Implement S3 URL refresh for user avatars with error handling

* fix: authcontext error in chats where agent chain is used

* refactor: streamline balance configuration logic in getBalanceConfig function

* fix: enhance icon resolution logic in SpecIcon component

* fix: allow null values for spec and iconURL in TEndpointOption type

* fix: update balance check to allow null tokenCredits
2025-04-02 18:44:13 -04:00
Danny Avila
3c91f7b0b7
🚀 feat: Enhance S3 URL Expiry with Refresh; fix: S3 File Deletion (#6647)
* refactor: Improve error logging in image fetching to base64 conversion

* fix: Add error handling for custom endpoint configuration retrieval

* fix: Update audio stream processing to parse text parts from complex message content

* chore: import order in streamAudio

* fix: S3 file deletion and optimize file upload

* feat: Implement S3 URL refresh mechanism and add cache for expiry check intervals

* feat: Add S3 URL refresh functionality for agent avatars

* chore: remove unnecessary console.log in MultiMessage component

* chore: update version of librechat-data-provider to 0.7.77
2025-03-31 18:40:06 -04:00
Ruben Talstra
bc039cea29
🔧 fix: Azure Blob Integration and File Source References (#6575)
* 🔧 fix: Update file source references to include 'azure_blob' for correct service initialization

* 🔧 fix: Add Azure Blob Storage Emulator entries to .gitignore

* fix: Update file source references to include 'azure_blob' for correct service initialization

* fix: Refactor Azure Blob Storage functions to use environment variables for access control and container name, fix deletion improper logging and improper params

* fix: Add basePath determination for agent file uploads based on MIME type

* fix: Implement file streaming to Azure Blob Storage to optimize memory usage during uploads (non-images)

* fix: Update SourceIcon to include 'azure_blob' class and adjust model setting in useSelectorEffects for assistants

* chore: import order

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2025-03-31 13:44:20 -04:00
Danny Avila
c381fc3ff0
🔧 fix: Ensure continuation in Image processing on base64 encoding from Blob Storage (#6619) 2025-03-29 14:48:35 -04:00
Danny Avila
ea2cbc55a7
🔧 fix: S3 Download Stream with Key Extraction and Blob Storage Encoding for Vision (#6557) 2025-03-26 15:04:01 -04:00
Danny Avila
299cabd6ed
🔧 refactor: Consolidate Logging, Model Selection & Actions Optimizations, Minor Fixes (#6553)
* 🔧 feat: Enhance logging configuration for production and debug environments

* 🔒 feat: Implement encryption and decryption functions for sensitive values in ActionService with URL encoding/decoding

* refactor: optimize action service for agent tools

* refactor: optimize action processing for Assistants API

* fix: handle case where agent is not found in loadAgent function

* refactor: improve error handling in API calls by throwing new Error with logAxiosError output

* chore: bump @librechat/agents to 2.3.95, fixes "Invalid tool call structure: No preceding AIMessage with tool_call_ids"

* refactor: enhance error logging in logAxiosError function to include response status

* refactor: remove unused useModelSelection hook from Endpoint

* refactor: add support for assistants in useSelectorEffects hook

* refactor: replace string easing with imported easings in Landing component

* chore: remove duplicate translation

* refactor: update model selection logic and improve localization for UI elements

* refactor: replace endpoint value checks with helper functions for agents and assistants

* refactor: optimize display value logic and utilize useMemo for performance improvements

* refactor: clean up imports and optimize display/icon value logic in endpoint components, fix spec selection

* refactor: enhance error logging in axios utility to include stack traces for better debugging

* refactor: update logging configuration to use DEBUG_LOGGING and streamline log level handling

* refactor: adjust className for export menu button to improve layout consistency and remove unused title prop from ShareButton

* refactor: update import path for logAxiosError utility to improve module organization and clarity

* refactor: implement debounced search value setter in ModelSelectorContext for improved performance
2025-03-26 14:10:52 -04:00
Marco Beretta
20f353630e
🗣️ feat: add support for gpt-4o-transcribe models (#6483) 2025-03-23 11:26:06 -04:00
Danny Avila
bc88ac846d
🏃‍♂️ refactor: Improve Agent Run Context & Misc. Changes (#6448)
* chore: bump Model Context Protocol SDK dependencies

* fix: correct indentation in MCPConnection class

* refactor: enhance SSE transport with abort controller and add error handling for empty results

* chore: remove outdated Model Context Protocol SDK dependency

* chore: update @modelcontextprotocol/sdk dependency to version 1.7.0

* chore: add debugging comments for PingRequest handling in MCPConnection class

* refactor: update callTool method to accept structured arguments and options

* refactor: simplify maxContextTokens calculation in initializeAgentOptions

* chore: update @babel/runtime dependency to version 7.26.10

* chore: update @librechat/agents dependency to version 2.2.9

* chore: update @librechat/agents dependency to version 2.3.6

* refactor: imports and prevent s3 initialization if strategy not configured

* refactor: mark redis as non-experimental

* refactor: add missing `maxContextTokens` for OpenAI parameters

* refactor: improve log message for Redis initialization

* chore: update @librechat/agents dependency to version 2.3.8

* refactor: extend `streamBuffer` condition to include BEDROCK provider as easily gets throttled by AWS

* refactor: filter out 'think' parts from message content in Anthropic and OpenAI clients
2025-03-20 22:56:57 -04:00
Ruben Talstra
692fba51d8
🚀 feat: Add support for custom AWS endpoint in S3 initialization (#6431) 2025-03-20 09:00:59 -04:00
Ruben Talstra
0a4a16d1f7
🚀 feat: Integrate Azure Blob Storage for file handling and image uploads (#6153)
* 🚀 feat: Integrate Azure Blob Storage for file handling and image uploads

* 🐼 refactor: Correct module import case for Azure in strategies.js

* 🚀 feat: Add Azure support in SourceIcon component

* 🚀 feat: Enhance Azure Blob Service initialization with Managed Identity support

* 🐼 refactor: Remove unused Azure dependencies from package.json and package-lock.json

* 🐼 refactor: Remove unused Azure dependencies from package.json and package-lock.json

* 🐼 refactor: Remove unused Azure dependencies from package.json and package-lock.json

* 🚀 feat: Add Azure SDK dependencies for identity and storage blob

* 🔧 fix: Reorganize imports in strategies.js for better clarity

* 🔧 fix: Correct comment formatting in strategies.js for consistency

* 🔧 fix: Improve comment formatting in strategies.js for consistency
2025-03-19 10:45:52 -04:00
Ruben Talstra
8f68e8be81
🚀 feat: S3 Integration for File handling and Image uploads (#6142)
* French Translation Update

* French Translation Update

* test

* Add fileStrategy S3 Config

* update s3 crud.js

* 🔧 chore: downgrade dotenv to version 16.0.3 and add aws-sdk to package-lock.json

* 🔧 chore: remove aws-sdk from package.json

* 🚀 feat: Integrate AWS SDK for S3 with enhanced upload and retrieval functionalities

* 🚀 feat: Implement S3 integration for file upload and retrieval functionalities

* 🚀 feat: Enhance S3 initialization to support default credentials and improved error handling

---------

Co-authored-by: Gael Martins <gael.martins@acolad.com>
2025-03-19 02:04:45 -04:00
Danny Avila
d6a17784dc
🔗 feat: Agent Chain (Mixture-of-Agents) (#6374)
* wip: first pass, dropdown for selecting sequential agents

* refactor: Improve agent selection logic and enhance performance in SequentialAgents component

* wip: seq. agents working ideas

* wip: sequential agents style change

* refactor: move agent form options/submission outside of AgentConfig

* refactor: prevent repeating code

* refactor: simplify current agent display in SequentialAgents component

* feat: persist  form value handling in AgentSelect component for agent_ids

* feat: first pass, sequential agnets agent update

* feat: enhance message display with agent updates and empty text handling

* chore: update Icon component to use EModelEndpoint for agent endpoints

* feat: update content type checks in BaseClient to use constants for better readability

* feat: adjust max context tokens calculation to use 90% of the model's max tokens

* feat: first pass, agent run message pruning

* chore: increase max listeners for abort controller to prevent memory leaks

* feat: enhance runAgent function to include current index count map for improved token tracking

* chore: update @librechat/agents dependency to version 2.2.5

* feat: update icons and style of SequentialAgents component for improved UI consistency

* feat: add AdvancedButton and AdvancedPanel components for enhanced agent settings navigation, update styling for agent form

* chore: adjust minimum height of AdvancedPanel component for better layout consistency

* chore: update @librechat/agents dependency to version 2.2.6

* feat: enhance message formatting by incorporating tool set into agent message processing, in order to allow better mix/matching of agents (as tool calls for tools not found in set will be stringified)

* refactor: reorder components in AgentConfig for improved readability and maintainability

* refactor: enhance layout of AgentUpdate component for improved visual structure

* feat: add DeepSeek provider to Bedrock settings and schemas

* feat: enhance link styling in mobile.css for better visibility and accessibility

* fix: update banner model import in update banner script; export Banner model

* refactor: `duplicateAgentHandler` to include tool_resources only for OCR context files

* feat: add 'qwen-vl' to visionModels for enhanced model support

* fix: change image format from JPEG to PNG in DALLE3 response

* feat: reorganize Advanced components and add localizations

* refactor: simplify JSX structure in AgentChain component to defer container styling to parent

* feat: add FormInput component for reusable input handling

* feat: make agent recursion limit configurable from builder

* feat: add support for agent capabilities chain in AdvancedPanel and update data-provider version

* feat: add maxRecursionLimit configuration for agents and update related documentation

* fix: update CONFIG_VERSION to 1.2.3 in data provider configuration

* feat: replace recursion limit input with MaxAgentSteps component and enhance input handling

* feat: enhance AgentChain component with hover card for additional information and update related labels

* fix: pass request and response objects to `createActionTool` when using assistant actions to prevent auth error

* feat: update AgentChain component layout to include agent count display

* feat: increase default max listeners and implement capability check function for agent chain

* fix: update link styles in mobile.css for better visibility in dark mode

* chore: temp. remove agents package while bumping shared packages

* chore: update @langchain/google-genai package to version 0.1.11

* chore: update @langchain/google-vertexai package to version 0.2.2

* chore: add @librechat/agents package at version 2.2.8

* feat: add deepseek.r1 model with token rate and context values for bedrock
2025-03-17 16:43:44 -04:00
Danny Avila
cf03731cc8
🔧 fix: Axios Proxy Usage And Bump mongoose (#6298)
* fix: bump mongoose to fix nested schema errors

* fix: Enhance Axios instance creation with improved proxy handling and error logging

* fix: Refactor Axios instance creation and remove proxy handling from file upload functions

* fix: Update proxy configuration in Axios instance creation and add unit tests
2025-03-11 14:44:54 -04:00
Danny Avila
ded3cd8876
🔍 feat: Mistral OCR API / Upload Files as Text (#6274)
* refactor: move `loadAuthValues` to `~/services/Tools/credentials`

* feat: add createAxiosInstance function to configure axios with proxy support

* WIP: First pass mistral ocr

* refactor: replace getConvoFiles with getToolFiles for improved file retrieval logic

* refactor: improve document formatting in encodeAndFormat function

* refactor: remove unused resendFiles parameter from buildOptions function (this option comes from the agent config)

* fix: update getFiles call to include files with `text` property as well

* refactor: move file handling to `initializeAgentOptions`

* refactor: enhance addImageURLs method to handle OCR text and improve message formatting

* refactor: update message formatting to handle OCR text in various content types

* refactor: remove unused resendFiles property from compactAgentsSchema

* fix: add error handling for Mistral OCR document upload and logging

* refactor: integrate OCR capability into file upload options and configuration

* refactor: skip processing for text source files in delete request, as they are directly tied to database

* feat: add metadata field to ExtendedFile type and update PanelColumns and PanelTable components for localization and metadata handling

* fix: source icon styling

* wip: first pass, frontend file context agent resources

* refactor: add hover card with contextual information for File Context (OCR) in FileContext component

* feat: enhance file processing by integrating file retrieval for OCR resources in agent initialization

* feat: implement OCR config; fix: agent resource deletion for ocr files

* feat: enhance agent initialization by adding OCR capability check in resource priming

* ci: fix `~/config` module mock

* ci: add OCR property expectation in AppService tests

* refactor: simplify OCR config loading by removing environment variable extraction, to be done when OCR is actually performed

* ci: add unit test to ensure environment variable references are not parsed in OCR config

* refactor: disable base64 image inclusion in OCR request

* refactor: enhance OCR configuration handling by validating environment variables and providing defaults

* refactor: use file stream from disk for mistral ocr api
2025-03-10 17:23:46 -04:00
Danny Avila
8cb7f34f86
🚀 feat: Add Code API Proxy Support and Update MCP SDK (#6203)
* chore: bump mcp sdk

* feat: Add proxy support for file download and upload in Code Environment CRUD operations

* chore: remove unused files

* chore: change output format from CommonJS to ES module in server rollup config
2025-03-06 12:47:59 -05:00
Danny Avila
fdb3cf3f58
🔧 fix: Resizable Panel Unmount Error & Code Env. File Re-Upload (#5947)
* 🔧 refactor: handle full path for code env. file re-upload

* fix: update react-resizable-panels to version 2.1.7 to resolve error thrown on unmount of artifacts; ref: https://github.com/bvaughn/react-resizable-panels/issues/372

* refactor: replace promptPrefix with systemMessage in GoogleClient for improved clarity, and to prevent saving LibreChat feature-specific instructions to the user's custom instructions
2025-02-19 14:53:22 -05:00
Danny Avila
ecddffa7b2
🐛 fix: RAG Results Sorted By Distance (#5931)
* refactor: Extract file unlinking logic into a separate function and don't throw error

* fix: RAG results are actually in distance, not score
2025-02-18 08:14:19 -05:00
Danny Avila
964a74c73b
🛠 refactor: Ensure File Deletions, File Naming, and Agent Resource Updates (#5928)
* refactor: Improve error logging for file upload and processing functions to prevent verbosity

* refactor: Add uploads directory to Docker Compose to persist file uploads

* refactor: `addAgentResourceFile` to handle edge case of non-existing `tool_resource` array

* refactor: Remove version specification from deploy-compose.yml

* refactor: Prefix filenames with file_id to ensure uniqueness in file uploads

* refactor: Enhance error handling in deleteVectors to log warnings for non-404 errors

* refactor: Limit file search results to top 5 based on relevance score

* 🌍 i18n: Update translation.json with latest translations

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-02-17 19:37:03 -05:00
Danny Avila
591a019766
🏄‍♂️ refactor: Optimize Reasoning UI & Token Streaming (#5546)
*  feat: Implement Show Thinking feature; refactor: testing thinking render optimizations

*  feat: Refactor Thinking component styles and enhance Markdown rendering

* chore: add back removed code, revert type changes

* chore: Add back resetCounter effect to Markdown component for improved code block indexing

* chore: bump @librechat/agents and google langchain packages

* WIP: reasoning type updates

* WIP: first pass, reasoning content blocks

* chore: revert code

* chore: bump @librechat/agents

* refactor: optimize reasoning tag handling

* style: ul indent padding

* feat: add Reasoning component to handle reasoning display

* feat: first pass, content reasoning part styling

* refactor: add content placeholder for endpoints using new stream handler

* refactor: only cache messages when requesting stream audio

* fix: circular dep.

* fix: add default param

* refactor: tts, only request after message stream, fix chrome autoplay

* style: update label for submitting state and add localization for 'Thinking...'

* fix: improve global audio pause logic and reset active run ID

* fix: handle artifact edge cases

* fix: remove unnecessary console log from artifact update test

* feat: add support for continued message handling with new streaming method

---------

Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
2025-01-29 19:46:58 -05:00
Danny Avila
bf0a84e45a
®️ feat: Support Rscript for Code Interpreter & recursionLimit for Agents (#5170)
* chore: bump @librechat/agents to v1.9.8 for rscript support

* chore: fix @langchain/google-genai dep., match agents

* chore: fix @langchain/google-vertexai to v0.1.5, match with agents

* chore: bump @librechat/agents to v1.9.9

* chore: update @librechat/agents to v1.9.91 and @langchain/google-vertexai to v0.1.6

* chore: increase MAX_FILE_SIZE to 150MB for file uploads

* chore: bump @librechat/agents to v1.9.92

* feat: support `recursionLimit` for agents

* chore: update configuration version to 1.2.1 in librechat.yaml and config.ts

* feat: add R language SVG icon to the assets and include it in ApiKeyDialog

* feat: add support for new vision model 'o1' and exclude 'o1-mini'
2025-01-03 16:50:00 -05:00
Danny Avila
3fbbcb1cfe
refactor: Integrate Capabilities into Agent File Uploads and Tool Handling (#5048)
* refactor: support drag/drop files for agents, handle undefined tool_resource edge cases

* refactor: consolidate endpoints config logic to dedicated getter

* refactor: Enhance agent tools loading logic to respect capabilities and filter tools accordingly

* refactor: Integrate endpoint capabilities into file upload dropdown for dynamic resource handling

* refactor: Implement capability checks for agent file upload operations

* fix: non-image tool_resource check
2024-12-19 13:04:48 -05:00
Danny Avila
d3cafeee96
🔍 feat: Add Entity ID Support for File Search Shared Resources (#5028) 2024-12-17 22:11:18 -05:00
Danny Avila
e391347b9e
🔧 feat: Initial MCP Support (Tools) (#5015)
* 📝 chore: Add comment to clarify purpose of check_updates.sh script

* feat: mcp package

* feat: add librechat-mcp package and update dependencies

* feat: refactor MCPConnectionSingleton to handle transport initialization and connection management

* feat: change private methods to public in MCPConnectionSingleton for improved accessibility

* feat: filesystem demo

* chore: everything demo and move everything under mcp workspace

* chore: move ts-node to mcp workspace

* feat: mcp examples

* feat: working sse MCP example

* refactor: rename MCPConnectionSingleton to MCPConnection for clarity

* refactor: replace MCPConnectionSingleton with MCPConnection for consistency

* refactor: manager/connections

* refactor: update MCPConnection to use type definitions from mcp types

* refactor: update MCPManager to use winston logger and enhance server initialization

* refactor: share logger between connections and manager

* refactor: add schema definitions and update MCPManager to accept logger parameter

* feat: map available MCP tools

* feat: load manifest tools

* feat: add MCP tools delimiter constant and update plugin key generation

* feat: call MCP tools

* feat: update librechat-data-provider version to 0.7.63 and enhance StdioOptionsSchema with additional properties

* refactor: simplify typing

* chore: update types/packages

* feat: MCP Tool Content parsing

* chore: update dependencies and improve package configurations

* feat: add 'mcp' directory to package and update configurations

* refactor: return CONTENT_AND_ARTIFACT format for MCP callTool

* chore: bump @librechat/agents

* WIP: MCP artifacts

* chore: bump @librechat/agents to v1.8.7

* fix: ensure filename has extension when saving base64 image

* fix: move base64 buffer conversion before filename extension check

* chore: update backend review workflow to install MCP package

* fix: use correct `mime` method

* fix: enhance file metadata with message and tool call IDs in image saving process

* fix: refactor ToolCall component to handle MCP tool calls and improve domain extraction

* fix: update ToolItem component for default isInstalled value and improve localization in ToolSelectDialog

* fix: update ToolItem component to use consistent text color for tool description

* style: add theming to ToolSelectDialog

* fix: improve domain extraction logic in ToolCall component

* refactor: conversation item theming, fix rename UI bug, optimize props, add missing types

* feat: enhance MCP options schema with base options (iconPath to start) and make transport type optional, infer based on other option fields

* fix: improve reconnection logic with parallel init and exponential backoff and enhance transport debug logging

* refactor: improve logging format

* refactor: improve logging of available tools by displaying tool names

* refactor: improve reconnection/connection logic

* feat: add MCP package build process to Dockerfile

* feat: add fallback icon for tools without an image in ToolItem component

* feat: Assistants Support for MCP Tools

* fix(build): configure rollup to use output.dir for dynamic imports

* chore: update @librechat/agents to version 1.8.8 and add @langchain/anthropic dependency

* fix: update CONFIG_VERSION to 1.2.0
2024-12-17 13:12:57 -05:00
Danny Avila
1a815f5e19
🎉 feat: Code Interpreter API and Agents Release (#4860)
* feat: Code Interpreter API & File Search Agent Uploads

chore: add back code files

wip: first pass, abstract key dialog

refactor: influence checkbox on key changes

refactor: update localization keys for 'execute code' to 'run code'

wip: run code button

refactor: add throwError parameter to loadAuthValues and getUserPluginAuthValue functions

feat: first pass, API tool calling

fix: handle missing toolId in callTool function and return 404 for non-existent tools

feat: show code outputs

fix: improve error handling in callTool function and log errors

fix: handle potential null value for filepath in attachment destructuring

fix: normalize language before rendering and prevent null return

fix: add loading indicator in RunCode component while executing code

feat: add support for conditional code execution in Markdown components

feat: attachments

refactor: remove bash

fix: pass abort signal to graph/run

refactor: debounce and rate limit tool call

refactor: increase debounce delay for execute function

feat: set code output attachments

feat: image attachments

refactor: apply message context

refactor: pass `partIndex`

feat: toolCall schema/model/methods

feat: block indexing

feat: get tool calls

chore: imports

chore: typing

chore: condense type imports

feat: get tool calls

fix: block indexing

chore: typing

refactor: update tool calls mapping to support multiple results

fix: add unique key to nav link for rendering

wip: first pass, tool call results

refactor: update query cache from successful tool call mutation

style: improve result switcher styling

chore: note on using \`.toObject()\`

feat: add agent_id field to conversation schema

chore: typing

refactor: rename agentMap to agentsMap for consistency

feat: Agent Name as chat input placeholder

chore: bump agents

📦 chore: update @langchain dependencies to latest versions to match agents package

📦 chore: update @librechat/agents dependency to version 1.8.0

fix: Aborting agent stream removes sender; fix(bedrock): completion removes preset name label

refactor: remove direct file parameter to use req.file, add `processAgentFileUpload` for image uploads

feat: upload menu

feat: prime message_file resources

feat: implement conversation access validation in chat route

refactor: remove file parameter from processFileUpload and use req.file instead

feat: add savedMessageIds set to track saved message IDs in BaseClient, to prevent unnecessary double-write to db

feat: prevent duplicate message saves by checking savedMessageIds in AgentController

refactor: skip legacy RAG API handling for agents

feat: add files field to convoSchema

refactor: update request type annotations from Express.Request to ServerRequest in file processing functions

feat: track conversation files

fix: resendFiles, addPreviousAttachments handling

feat: add ID validation for session_id and file_id in download route

feat: entity_id for code file uploads/downloads

fix: code file edge cases

feat: delete related tool calls

feat: add stream rate handling for LLM configuration

feat: enhance system content with attached file information

fix: improve error logging in resource priming function

* WIP: PoC, sequential agents

WIP: PoC Sequential Agents, first pass content data + bump agents package

fix: package-lock

WIP: PoC, o1 support, refactor bufferString

feat: convertJsonSchemaToZod

fix: form issues and schema defining erroneous model

fix: max length issue on agent form instructions, limit conversation messages to sequential agents

feat: add abort signal support to createRun function and AgentClient

feat: PoC, hide prior sequential agent steps

fix: update parameter naming from config to metadata in event handlers for clarity, add model to usage data

refactor: use only last contentData, track model for usage data

chore: bump agents package

fix: content parts issue

refactor: filter contentParts to include tool calls and relevant indices

feat: show function calls

refactor: filter context messages to exclude tool calls when no tools are available to the agent

fix: ensure tool call content is not undefined in formatMessages

feat: add agent_id field to conversationPreset schema

feat: hide sequential agents

feat: increase upload toast duration to 10 seconds

* refactor: tool context handling & update Code API Key Dialog

feat: toolContextMap

chore: skipSpecs -> useSpecs

ci: fix handleTools tests

feat: API Key Dialog

* feat: Agent Permissions Admin Controls

feat: replace label with button for prompt permission toggle

feat: update agent permissions

feat: enable experimental agents and streamline capability configuration

feat: implement access control for agents and enhance endpoint menu items

feat: add welcome message for agent selection in localization

feat: add agents permission to access control and update version to 0.7.57

* fix: update types in useAssistantListMap and useMentions hooks for better null handling

* feat: mention agents

* fix: agent tool resource race conditions when deleting agent tool resource files

* feat: add error handling for code execution with user feedback

* refactor: rename AdminControls to AdminSettings for clarity

* style: add gap to button in AdminSettings for improved layout

* refactor: separate agent query hooks and check access to enable fetching

* fix: remove unused provider from agent initialization options, creates issue with custom endpoints

* refactor: remove redundant/deprecated modelOptions from AgentClient processes

* chore: update @librechat/agents to version 1.8.5 in package.json and package-lock.json

* fix: minor styling issues + agent panel uniformity

* fix: agent edge cases when set endpoint is no longer defined

* refactor: remove unused cleanup function call from AppService

* fix: update link in ApiKeyDialog to point to pricing page

* fix: improve type handling and layout calculations in SidePanel component

* fix: add missing localization string for agent selection in SidePanel

* chore: form styling and localizations for upload filesearch/code interpreter

* fix: model selection placeholder logic in AgentConfig component

* style: agent capabilities

* fix: add localization for provider selection and improve dropdown styling in ModelPanel

* refactor: use gpt-4o-mini > gpt-3.5-turbo

* fix: agents configuration for loadDefaultInterface and update related tests

* feat: DALLE Agents support
2024-12-04 15:48:13 -05:00
Marco Beretta
affcebd48c
⚙️ fix: update Azure OpenAI STT/TTS env handling (#4859) 2024-12-04 11:44:00 -05:00
Danny Avila
d012da0065
🛡️ fix: Enhance File Upload Security & Error Handling (#4705)
* fix: sanitize filename in multer storage callback

* fix: ensure temporary image upload file is deleted after processing

* fix: prevent cleanup flag from being set to false before actually deleted

* refactor: user avatar, typing, use 'file' for formData instead of 'input', add disk storage, use localization

* fix: update Avatar component to include image dimensions in formData and refactor editor reference type

* fix: refactor avatar upload handling to use fs for file reading and enhance file validation

* fix: ensure temporary image upload file is deleted after processing

* fix: refactor avatar upload routes and handlers for agents and assistants, improve file handling and validation

* fix: improve audio file validation and cleanup

* fix: add filename sanitization utility and integrate it into multer storage configuration

* fix: update group project ID check for null and refactor delete prompt group response type

* fix: invalid access control for deleting prompt groups

* fix: add error handling and logging to checkBan middleware

* fix: catch conversation parsing errors

* chore: revert unnecessary height and width parameters from avatar upload

* chore: update librechat-data-provider version to 0.7.55

* style: ensure KaTeX can spread across visible space
2024-11-12 16:41:04 -05:00
Danny Avila
49ee88b6e8
⚙️ fix: File Config Handling (#4664)
* chore: typing

* refactor: create file filter from custom fileConfig, if provided

* refactor: use logger utility to avoid overly verbose axios error logs when using RAG_API

* fix(useFileHandling): use memoization/callbacks to make sure the appropriate fileConfig is used; refactor: move endpoint to first field applied to formdata

* chore: update librechat-data-provider version to 0.7.54

* chore: revert type change
2024-11-07 11:11:20 -05:00
Danny Avila
2e519f9b57
🤖 feat: Custom Endpoint Agents (experimental) (#4627)
* wip: first pass, custom endpoint agents

* chore: imports

* chore: consolidate exports

* fix: imports

* feat: convert message.content array to strings for legacy format handling (deepseek/groq)

* refactor: normalize ollama endpoint name

* refactor: update mocking in isDomainAllowed.spec.js

* refactor: update deepseekModels in tokens.js and tokens.spec.js
2024-11-04 12:59:04 -05:00
Danny Avila
95011ce349
🚧 WIP: Merge Dev Build (#4611)
* refactor: Agent CodeFiles, abortUpload WIP

* feat: code environment file upload

* refactor: useLazyEffect

* refactor:
- Add `watch` from `useFormContext` to check if code execution is enabled
- Disable file upload button if `agent_id` is not selected or code execution is disabled

* WIP: primeCodeFiles; refactor: rename sessionId to session_id for uniformity

* Refactor: Rename session_id to sessionId for uniformity in AuthService.js

* chore: bump @librechat/agents to version 1.7.1

* WIP: prime code files

* refactor: Update code env file upload method to use read stream

* feat: reupload code env file if no longer active

* refactor: isAssistantTool -> isEntityTool + address type issues

* feat: execute code tool hook

* refactor: Rename isPluginAuthenticated to checkPluginAuth in PluginController.js

* refactor: Update PluginController.js to use AuthType constant for comparison

* feat: verify tool authentication (execute_code)

* feat: enter librechat_code_api_key

* refactor: Remove unused imports in BookmarkForm.tsx

* feat: authenticate code tool

* refactor: Update Action.tsx to conditionally render the key and revoke key buttons

* refactor(Code/Action): prevent uncheck-able 'Run Code' capability when key is revoked

* refactor(Code/Action): Update Action.tsx to conditionally render the key and revoke key buttons

* fix: agent file upload edge cases

* chore: bump @librechat/agents

* fix: custom endpoint providerValue icon

* feat: ollama meta modal token values + context

* feat: ollama agents

* refactor: Update token models for Ollama models

* chore: Comment out CodeForm

* refactor: Update token models for Ollama and Meta models
2024-11-01 18:36:39 -04:00
Danny Avila
ebe3e7f796
🖼️ fix: Avatar Handling for Agents and Assistants (#4507)
The changes include:
- In the agent controller:
  - Removed the parsing of the avatar metadata from the request body.
  - Fetched the avatar data from the agent object using the agent ID.
  - Updated the error logging when fetching the agent.
  - Updated the deleteFileByFilter function to include the user ID when deleting the old avatar file.

- In the assistant controller:
  - Removed the parsing of the metadata from the request body.
  - Fetched the metadata from the assistant object using the assistant ID.
  - Updated the error logging when fetching the assistant.
  - Updated the deleteFileByFilter function to include the user ID when deleting the old avatar file.
2024-10-22 14:53:45 -04:00
Danny Avila
ec922986a9
🤖 fix: Address Minor Agent Issues (#4483)
* fix(Agents): remove test code in openAI/llm.js

* refactor: add use of enums in encodeAndFormat

* fix: image attachment payload formatting for agents

* chore: imports
2024-10-21 09:41:04 -04:00
Danny Avila
ad74350036
🚧 chore: merge latest dev build (#4288)
* fix: agent initialization, add `collectedUsage` handling

* style: improve side panel styling

* refactor(loadAgent): Optimize order agent project ID retrieval

* feat: code execution

* fix: typing issues

* feat: ExecuteCode content part

* refactor: use local state for default collapsed state of analysis content parts

* fix: code parsing in ExecuteCode component

* chore: bump agents package, export loadAuthValues

* refactor: Update handleTools.js to use EnvVar for code execution tool authentication

* WIP

* feat: download code outputs

* fix(useEventHandlers): type issues

* feat: backend handling for code outputs

* Refactor: Remove console.log statement in Part.tsx

* refactor: add attachments to TMessage/messageSchema

* WIP: prelim handling for code outputs

* feat: attachments rendering

* refactor: improve attachments rendering

* fix: attachments, nullish edge case, handle attachments from event stream, bump agents package

* fix filename download

* fix: tool assignment for 'run code' on agent creation

* fix: image handling by adding attachments

* refactor: prevent agent creation without provider/model

* refactor: remove unnecessary space in agent creation success message

* refactor: select first model if selecting provider from empty on form

* fix: Agent avatar bug

* fix: `defaultAgentFormValues` causing boolean typing issue and typeerror

* fix: capabilities counting as tools, causing duplication of them

* fix: formatted messages edge case where consecutive content text type parts with the latter having tool_call_ids would cause consecutive AI messages to be created. furthermore, content could not be an array for tool_use messages (anthropic limitation)

* chore: bump @librechat/agents dependency to version 1.6.9

* feat: bedrock agents

* feat: new Agents icon

* feat: agent titling

* feat: agent landing

* refactor: allow sharing agent globally only if user is admin or author

* feat: initial AgentPanelSkeleton

* feat: AgentPanelSkeleton

* feat: collaborative agents

* chore: add potential authorName as part of schema

* chore: Remove unnecessary console.log statement

* WIP: agent model parameters

* chore: ToolsDialog typing and tool related localization chnages

* refactor: update tool instance type (latest langchain class), and rename google tool to 'google' proper

* chore: add back tools

* feat: Agent knowledge files upload

* refactor: better verbiage for disabled knowledge

* chore: debug logs for file deletions

* chore: debug logs for file deletions

* feat: upload/delete agent knowledge/file-search files

* feat: file search UI for agents

* feat: first pass, file search tool

* chore: update default agent capabilities and info
2024-09-30 17:17:57 -04:00
Danny Avila
94d1afee84
🛡️ chore: address several npm vulnerabilities (#4151)
* chore: bump express to 4.21.0 to address CVE-2024-45590 and CVE-2024-43796

* chore: npm audit fix

* chore: uninstall unused `ws` dependency

* chore: bump nodemailer to 6.9.15

* chore: bump mongoose to v7.3.3

* chore: bump lint-staged for micromatch upgrade

* chore: bump axios to 1.7.7

* chore: npm audit fix for mongodb/mongoose vulns
2024-09-19 20:28:32 -04:00
Danny Avila
d59b62174f
🪨 feat: AWS Bedrock support (#3935)
* feat: Add BedrockIcon component to SVG library

* feat: EModelEndpoint.bedrock

* feat: first pass, bedrock chat. note: AgentClient is returning `agents` as conversation.endpoint

* fix: declare endpoint in initialization step

* chore: Update @librechat/agents dependency to version 1.4.5

* feat: backend content aggregation for agents/bedrock

* feat: abort agent requests

* feat: AWS Bedrock icons

* WIP: agent provider schema parsing

* chore: Update EditIcon props type

* refactor(useGenerationsByLatest): make agents and bedrock editable

* refactor: non-assistant message content, parts

* fix: Bedrock response `sender`

* fix: use endpointOption.model_parameters not endpointOption.modelOptions

* fix: types for step handler

* refactor: Update Agents.ToolCallDelta type

* refactor: Remove unnecessary assignment of parentMessageId in AskController

* refactor: remove unnecessary assignment of parentMessageId (agent request handler)

* fix(bedrock/agents): message regeneration

* refactor: dynamic form elements using react-hook-form Controllers

* fix: agent icons/labels for messages

* fix: agent actions

* fix: use of new dynamic tags causing application crash

* refactor: dynamic settings touch-ups

* refactor: update Slider component to allow custom track class name

* refactor: update DynamicSlider component styles

* refactor: use Constants value for GLOBAL_PROJECT_NAME (enum)

* feat: agent share global methods/controllers

* fix: agents query

* fix: `getResponseModel`

* fix: share prompt a11y issue

* refactor: update SharePrompt dialog theme styles

* refactor: explicit typing for SharePrompt

* feat: add agent roles/permissions

* chore: update @librechat/agents dependency to version 1.4.7 for tool_call_ids edge case

* fix(Anthropic): messages.X.content.Y.tool_use.input: Input should be a valid dictionary

* fix: handle text parts with tool_call_ids and empty text

* fix: role initialization

* refactor: don't make instructions required

* refactor: improve typing of Text part

* fix: setShowStopButton for agents route

* chore: remove params for now

* fix: add streamBuffer and streamRate to help prevent 'Overloaded' errors from Anthropic API

* refactor: remove console.log statement in ContentRender component

* chore: typing, rename Context to Delete Button

* chore(DeleteButton): logging

* refactor(Action): make accessible

* style(Action): improve a11y again

* refactor: remove use/mention of mongoose sessions

* feat: first pass, sharing agents

* feat: visual indicator for global agent, remove author when serving to non-author

* wip: params

* chore: fix typing issues

* fix(schemas): typing

* refactor: improve accessibility of ListCard component and fix console React warning

* wip: reset templates for non-legacy new convos

* Revert "wip: params"

This reverts commit f8067e91d4.

* Revert "refactor: dynamic form elements using react-hook-form Controllers"

This reverts commit 2150c4815d.

* fix(Parameters): types and parameter effect update to only update local state to parameters

* refactor: optimize useDebouncedInput hook for better performance

* feat: first pass, anthropic bedrock params

* chore: paramEndpoints check for endpointType too

* fix: maxTokens to use coerceNumber.optional(),

* feat: extra chat model params

* chore: reduce code repetition

* refactor: improve preset title handling in SaveAsPresetDialog component

* refactor: improve preset handling in HeaderOptions component

* chore: improve typing, replace legacy dialog for SaveAsPresetDialog

* feat: save as preset from parameters panel

* fix: multi-search in select dropdown when using Option type

* refactor: update default showDefault value to false in Dynamic components

* feat: Bedrock presets settings

* chore: config, fix agents schema, update config version

* refactor: update AWS region variable name in bedrock options endpoint to BEDROCK_AWS_DEFAULT_REGION

* refactor: update baseEndpointSchema in config.ts to include baseURL property

* refactor: update createRun function to include req parameter and set streamRate based on provider

* feat: availableRegions via config

* refactor: remove unused demo agent controller file

* WIP: title

* Update @librechat/agents to version 1.5.0

* chore: addTitle.js to handle empty responseText

* feat: support images and titles

* feat: context token updates

* Refactor BaseClient test to use expect.objectContaining

* refactor: add model select, remove header options params, move side panel params below prompts

* chore: update models list, catch title error

* feat: model service for bedrock models (env)

* chore: Remove verbose debug log in AgentClient class following stream

* feat(bedrock): track token spend; fix: token rates, value key mapping for AWS models

* refactor: handle streamRate in `handleLLMNewToken` callback

* chore: AWS Bedrock example config in `.env.example`

* refactor: Rename bedrockMeta to bedrockGeneral in settings.ts and use for AI21 and Amazon Bedrock providers

* refactor: Update `.env.example` with AWS Bedrock model IDs URL and additional notes

* feat: titleModel support for bedrock

* refactor: Update `.env.example` with additional notes for AWS Bedrock model IDs
2024-09-09 12:06:59 -04:00
Max Sanna
63b80c3067
🗣️ fix: Azure OpenAI STT (#3731)
* Fix for Azure OpenAI STT

* chore(STTService): imports order

---------

Co-authored-by: Danny Avila <danacordially@gmail.com>
2024-08-30 15:11:15 -04:00
Marco Beretta
62881fee54
🔧 fix: handle missing custom config speech (#3790)
* feat: Update speech settings retrieval logic to handle missing custom configuration

This commit updates the logic in the Speech component and the getCustomConfigSpeech function to handle the case where the custom configuration is missing. Previously, if no custom configuration was found, an error would occur. Now, the code checks for the presence of the custom configuration and returns a message indicating that no custom configuration was found. This improves the robustness of the application and provides a better user experience.

* refactor: changed response message when no custom config is found
2024-08-27 06:09:04 -04:00