* feat: Added "document parser" OCR strategy
The document parser uses libraries to parse the text out of known document types.
This lets LibreChat handle some complex document types without having to use a
secondary service (like Mistral or standing up a RAG API server).
To enable the document parser, set the ocr strategy to "document_parser" in
librechat.yaml.
We now support:
- PDFs using pdfjs
- DOCX using mammoth
- XLS/XLSX using SheetJS
(The associated packages were also added to the project.)
* fix: applied Copilot code review suggestions
- Properly calculate length of text based on UTF8.
- Avoid issues with loading / blocking PDF parsing.
* fix: improved docs on parseDocument()
* chore: move to packages/api for TS support
* refactor: make document processing the default ocr strategy
- Introduced support for additional document types in the OCR strategy, including PDF, DOCX, and XLS/XLSX.
- Updated the file upload handling to dynamically select the appropriate parsing strategy based on the file type.
- Refactored the document parsing functions to use asynchronous imports for improved performance and maintainability.
* test: add unit tests for processAgentFileUpload functionality
- Introduced a new test suite for the processAgentFileUpload function in process.spec.js.
- Implemented various test cases to validate OCR strategy selection based on file types, including PDF, DOCX, XLSX, and XLS.
- Mocked dependencies to ensure isolated testing of file upload handling and strategy selection logic.
- Enhanced coverage for scenarios involving OCR capability checks and default strategy fallbacks.
* chore: update pdfjs-dist version and enhance document parsing tests
- Bumped pdfjs-dist dependency to version 5.4.624 in both api and packages/api.
- Refactored document parsing tests to use 'originalname' instead of 'filename' for file objects.
- Added a new test case for parsing XLS files to improve coverage of document types supported by the parser.
- Introduced a sample XLS file for testing purposes.
* feat: enforce text size limit and improve OCR fallback handling in processAgentFileUpload
- Added a check to ensure extracted text does not exceed the 15MB storage limit, throwing an error if it does.
- Refactored the OCR handling logic to improve fallback behavior when the configured OCR fails, ensuring a more robust document processing flow.
- Enhanced unit tests to cover scenarios for oversized text and fallback mechanisms, ensuring proper error handling and functionality.
* fix: correct OCR URL construction in performOCR function
- Updated the OCR URL construction to ensure it correctly appends '/ocr' to the base URL if not already present, improving the reliability of the OCR request.
---------
Co-authored-by: Dan Lew <daniel@mightyacorn.com>
* refactor: move endpoint initialization methods to typescript
* refactor: move agent init to packages/api
- Introduced `initialize.ts` for agent initialization, including file processing and tool loading.
- Updated `resources.ts` to allow optional appConfig parameter.
- Enhanced endpoint configuration handling in various initialization files to support model parameters.
- Added new artifacts and prompts for React component generation.
- Refactored existing code to improve type safety and maintainability.
* refactor: streamline endpoint initialization and enhance type safety
- Updated initialization functions across various endpoints to use a consistent request structure, replacing `unknown` types with `ServerResponse`.
- Simplified request handling by directly extracting keys from the request body.
- Improved type safety by ensuring user IDs are safely accessed with optional chaining.
- Removed unnecessary parameters and streamlined model options handling for better clarity and maintainability.
* refactor: moved ModelService and extractBaseURL to packages/api
- Added comprehensive tests for the models fetching functionality, covering scenarios for OpenAI, Anthropic, Google, and Ollama models.
- Updated existing endpoint index to include the new models module.
- Enhanced utility functions for URL extraction and model data processing.
- Improved type safety and error handling across the models fetching logic.
* refactor: consolidate utility functions and remove unused files
- Merged `deriveBaseURL` and `extractBaseURL` into the `@librechat/api` module for better organization.
- Removed redundant utility files and their associated tests to streamline the codebase.
- Updated imports across various client files to utilize the new consolidated functions.
- Enhanced overall maintainability by reducing the number of utility modules.
* refactor: replace ModelService references with direct imports from @librechat/api and remove ModelService file
* refactor: move encrypt/decrypt methods and key db methods to data-schemas, use `getProviderConfig` from `@librechat/api`
* chore: remove unused 'res' from options in AgentClient
* refactor: file model imports and methods
- Updated imports in various controllers and services to use the unified file model from '~/models' instead of '~/models/File'.
- Consolidated file-related methods into a new file methods module in the data-schemas package.
- Added comprehensive tests for file methods including creation, retrieval, updating, and deletion.
- Enhanced the initializeAgent function to accept dependency injection for file-related methods.
- Improved error handling and logging in file methods.
* refactor: streamline database method references in agent initialization
* refactor: enhance file method tests and update type references to IMongoFile
* refactor: consolidate database method imports in agent client and initialization
* chore: remove redundant import of initializeAgent from @librechat/api
* refactor: move checkUserKeyExpiry utility to @librechat/api and update references across endpoints
* refactor: move updateUserPlugins logic to user.ts and simplify UserController
* refactor: update imports for user key management and remove UserService
* refactor: remove unused Anthropics and Bedrock endpoint files and clean up imports
* refactor: consolidate and update encryption imports across various files to use @librechat/data-schemas
* chore: update file model mock to use unified import from '~/models'
* chore: import order
* refactor: remove migrated to TS agent.js file and its associated logic from the endpoints
* chore: add reusable function to extract imports from source code in unused-packages workflow
* chore: enhance unused-packages workflow to include @librechat/api dependencies and improve dependency extraction
* chore: improve dependency extraction in unused-packages workflow with enhanced error handling and debugging output
* chore: add detailed debugging output to unused-packages workflow for better visibility into unused dependencies and exclusion lists
* chore: refine subpath handling in unused-packages workflow to correctly process scoped and non-scoped package imports
* chore: clean up unused debug output in unused-packages workflow and reorganize type imports in initialize.ts
* feat: add filterFilesByEndpointConfig to filter disabled file processing by provider
* chore: explicit define of endpointFileConfig for better debugging
* refactor: move `normalizeEndpointName` to data-provider as used app-wide
* chore: remove overrideEndpoint from useFileHandling
* refactor: improve endpoint file config selection
* refactor: update filterFilesByEndpointConfig to accept structured parameters and improve endpoint file config handling
* refactor: replace defaultFileConfig with getEndpointFileConfig for improved file configuration handling across components
* test: add comprehensive unit tests for getEndpointFileConfig to validate endpoint configuration handling
* refactor: streamline agent endpoint assignment and improve file filtering logic
* feat: add error handling for disabled file uploads in endpoint configuration
* refactor: update encodeAndFormat functions to accept structured parameters for provider and endpoint
* refactor: streamline requestFiles handling in initializeAgent function
* fix: getEndpointFileConfig partial config merging scenarios
* refactor: enhance mergeWithDefault function to support document-supported providers with comprehensive MIME types
* refactor: user-configured default file config in getEndpointFileConfig
* fix: prevent file handling when endpoint is disabled and file is dragged to chat
* refactor: move `getEndpointField` to `data-provider` and update usage across components and hooks
* fix: prioritize endpointType based on agent.endpoint in file filtering logic
* fix: prioritize agent.endpoint in file filtering logic and remove unnecessary endpointType defaulting
* 📎 feat: Direct Provider Attachment Support for Multimodal Content
* 📑 feat: Anthropic Direct Provider Upload (#9072)
* feat: implement Anthropic native PDF support with document preservation
- Add comprehensive debug logging throughout PDF processing pipeline
- Refactor attachment processing to separate image and document handling
- Create distinct addImageURLs(), addDocuments(), and processAttachments() methods
- Fix critical bugs in stream handling and parameter passing
- Add streamToBuffer utility for proper stream-to-buffer conversion
- Remove api/agents submodule from repository
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* chore: remove out of scope formatting changes
* fix: stop duplication of file in chat on end of response stream
* chore: bring back file search and ocr options
* chore: localize upload to provider string in file menu
* refactor: change createMenuItems args to fit new pattern introduced by anthropic-native-pdf-support
* feat: add cache point for pdfs processed by anthropic endpoint since they are unlikely to change and should benefit from caching
* feat: combine Upload Image into Upload to Provider since they both perform direct upload and change provider upload icon to reflect multimodal upload
* feat: add citations support according to docs
* refactor: remove redundant 'document' check since documents are handled properly by formatMessage in the agents repo now
* refactor: change upload logic so anthropic endpoint isn't exempted from normal upload path using Agents for consistency with the rest of the upload logic
* fix: include width and height in return from uploadLocalFile so images are correctly identified when going through an AgentUpload in addImageURLs
* chore: remove client specific handling since the direct provider stuff is handled by the agent client
* feat: handle documents in AgentClient so no need for change to agents repo
* chore: removed unused changes
* chore: remove auto generated comments from OG commit
* feat: add logic for agents to use direct to provider uploads if supported (currently just anthropic)
* fix: reintroduce role check to fix render error because of undefined value for Content Part
* fix: actually fix render bug by using proper isCreatedByUser check and making sure our mutation of formattedMessage.content is consistent
---------
Co-authored-by: Andres Restrepo <andres@thelinuxkid.com>
Co-authored-by: Claude <noreply@anthropic.com>
📁 feat: Send Attachments Directly to Provider (OpenAI) (#9098)
* refactor: change references from direct upload to direct attach to better reflect functionality
since we are just using base64 encoding strategy now rather than Files/File API for sending our attachments directly to the provider, the upload nomenclature no longer makes sense. direct_attach better describes the different methods of sending attachments to providers anyways even if we later introduce direct upload support
* feat: add upload to provider option for openai (and agent) ui
* chore: move anthropic pdf validator over to packages/api
* feat: simple pdf validation according to openai docs
* feat: add provider agnostic validatePdf logic to start handling multiple endpoints
* feat: add handling for openai specific documentPart formatting
* refactor: move require statement to proper place at top of file
* chore: add in openAI endpoint for the rest of the document handling logic
* feat: add direct attach support for azureOpenAI endpoint and agents
* feat: add pdf validation for azureOpenAI endpoint
* refactor: unify all the endpoint checks with isDocumentSupportedEndpoint
* refactor: consolidate Upload to Provider vs Upload image logic for clarity
* refactor: remove anthropic from anthropic_multimodal fileType since we support multiple providers now
🗂️ feat: Send Attachments Directly to Provider (Google) (#9100)
* feat: add validation for google PDFs and add google endpoint as a document supporting endpoint
* feat: add proper pdf formatting for google endpoints (requires PR #14 in agents)
* feat: add multimodal support for google endpoint attachments
* feat: add audio file svg
* fix: refactor attachments logic so multi-attachment messages work properly
* feat: add video file svg
* fix: allows for followup questions of uploaded multimodal attachments
* fix: remove incorrect final message filtering that was breaking Attachment component rendering
fix: manualy rename 'documents' to 'Documents' in git since it wasn't picked up due to case insensitivity in dir name
fix: add logic so filepicker for a google agent has proper filetype filtering
🛫 refactor: Move Encoding Logic to packages/api (#9182)
* refactor: move audio encode over to TS
* refactor: audio encoding now functional in LC again
* refactor: move video encode over to TS
* refactor: move document encode over to TS
* refactor: video encoding now functional in LC again
* refactor: document encoding now functional in LC again
* fix: extend file type options in AttachFileMenu to include 'google_multimodal' and update dependency array to include agent?.provider
* feat: only accept pdfs if responses api is enabled for openai convos
chore: address ESLint comments
chore: add missing audio mimetype
* fix: type safety for message content parts and improve null handling
* chore: reorder AttachFileMenuProps for consistency and clarity
* chore: import order in AttachFileMenu
* fix: improve null handling for text parts in parseTextParts function
* fix: remove no longer used unsupported capability error message for file uploads
* fix: OpenAI Direct File Attachment Format
* fix: update encodeAndFormatDocuments to support OpenAI responses API and enhance document result types
* refactor: broaden providers supported for documents
* feat: enhance DragDrop context and modal to support document uploads based on provider capabilities
* fix: reorder import statements for consistency in video encoding module
---------
Co-authored-by: Dustin Healy <54083382+dustinhealy@users.noreply.github.com>
* refactor: move `loadOCRConfig` from `packages/data-provider` to `packages/api` and return `undefined` if not explicitly configured
* fix: loadOCRConfig import from @librechat/api
* refactor: update defaultTextMimeTypes to support virtually all file types for text parsing
* fix: improve OCR capability check and error message for unsupported file types
* ci: remove unnecessary ocr expectation from AppService test
* WIP: conversion of `ocr` to `context`
* refactor: make `primeResources` backwards-compatible for `ocr` tool_resources
* refactor: Convert legacy `ocr` tool resource to `context` in agent updates
- Implemented conversion logic to replace `ocr` with `context` in both incoming updates and existing agent data.
- Merged file IDs and files from `ocr` into `context` while ensuring deduplication.
- Updated tools array to reflect the change from `ocr` to `context`.
* refactor: Enhance context file handling in agent processing
- Updated the logic for managing context files by consolidating file IDs from both `ocr` and `context` resources.
- Improved backwards compatibility by ensuring that context files are correctly populated and handled.
- Simplified the iteration over context files for better readability and maintainability.
* refactor: Enhance tool_resources handling in primeResources
- Added tests to verify the deletion behavior of tool_resources fields, ensuring original objects remain unchanged.
- Implemented logic to delete `ocr` and `context` fields after fetching and re-categorizing files.
- Preserved context field when the context capability is disabled, ensuring correct behavior in various scenarios.
* refactor: Replace `ocrEnabled` with `contextEnabled` in AgentConfig
* refactor: Adjust legacy tool handling order for improved clarity
* refactor: Implement OCR to context conversion functions and remove original conversion logic in update agent handling
* refactor: Move contextEnabled declaration to maintain consistent order in capabilities
* refactor: Update localization keys for file context to improve clarity and accuracy
* chore: Update localization key for file context information to improve clarity
* 🪶 feat: Add Support for Uploading Plaintext Files
feat: delineate between OCR and text handling in fileConfig field of config file
- also adds support for passing in mimetypes as just plain file extensions
feat: add showLabel bool to support future synthetic component DynamicDropdownInput
feat: add new combination dropdown-input component in params panel to support file type token limits
refactor: move hovercard to side to align with other hovercards
chore: clean up autogenerated comments
feat: add delineation to file upload path between text and ocr configured filetypes
feat: add token limit checks during file upload
refactor: move textParsing out of ocrEnabled logic
refactor: clean up types for filetype config
refactor: finish decoupling DynamicDropdownInput from fileTokenLimits
fix: move image token cost function into file to fix circular dependency causing unittest to fail and remove unused var for linter
chore: remove out of scope code following review
refactor: make fileTokenLimit conform to existing styles
chore: remove unused localization string
chore: undo changes to DynamicInput and other strays
feat: add fileTokenLimit to all provider config panels
fix: move textParsing back into ocr tool_resource block for now so that it doesn't interfere with other upload types
* 📤 feat: Add RAG API Endpoint Support for Text Parsing (#8849)
* feat: implement RAG API integration for text parsing with fallback to native parsing
* chore: remove TODO now that placeholder and fllback are implemented
* ✈️ refactor: Migrate Text Parsing to TS (#8892)
* refactor: move generateShortLivedToken to packages/api
* refactor: move textParsing logic into packages/api
* refactor: reduce nesting and dry code with createTextFile
* fix: add proper source handling
* fix: mock new parseText and parseTextNative functions in jest file
* ci: add test coverage for textParser
* 💬 feat: Add Audio File Support to Upload as Text (#8893)
* feat: add STT support for Upload as Text
* refactor: move processAudioFile to packages/api
* refactor: move textParsing from utils to files
* fix: remove audio/mp3 from unsupported mimetypes test since it is now supported
* ✂️ feat: Configurable File Token Limits and Truncation (#8911)
* feat: add configurable fileTokenLimit default value
* fix: add stt to fileConfig merge logic
* fix: add fileTokenLimit to mergeFileConfig logic so configurable value is actually respected from yaml
* feat: add token limiting to parsed text files
* fix: add extraction logic and update tests so fileTokenLimit isnt sent to LLM providers
* fix: address comments
* refactor: rename textTokenLimiter.ts to text.ts
* chore: update form-data package to address CVE-2025-7783 and update package-lock
* feat: use default supported mime types for ocr on frontend file validation
* fix: should be using logger.debug not console.debug
* fix: mock existsSync in text.spec.ts
* fix: mock logger rather than every one of its function calls
* fix: reorganize imports and streamline file upload processing logic
* refactor: update createTextFile function to use destructured parameters and improve readability
* chore: update file validation to use EToolResources for improved type safety
* chore: update import path for types in audio processing module
* fix: update file configuration access and replace console.debug with logger.debug for improved logging
---------
Co-authored-by: Dustin Healy <dustinhealy1@gmail.com>
Co-authored-by: Dustin Healy <54083382+dustinhealy@users.noreply.github.com>
* WIP: app.locals refactoring
WIP: appConfig
fix: update memory configuration retrieval to use getAppConfig based on user role
fix: update comment for AppConfig interface to clarify purpose
🏷️ refactor: Update tests to use getAppConfig for endpoint configurations
ci: Update AppService tests to initialize app config instead of app.locals
ci: Integrate getAppConfig into remaining tests
refactor: Update multer storage destination to use promise-based getAppConfig and improve error handling in tests
refactor: Rename initializeAppConfig to setAppConfig and update related tests
ci: Mock getAppConfig in various tests to provide default configurations
refactor: Update convertMCPToolsToPlugins to use mcpManager for server configuration and adjust related tests
chore: rename `Config/getAppConfig` -> `Config/app`
fix: streamline OpenAI image tools configuration by removing direct appConfig dependency and using function parameters
chore: correct parameter documentation for imageOutputType in ToolService.js
refactor: remove `getCustomConfig` dependency in config route
refactor: update domain validation to use appConfig for allowed domains
refactor: use appConfig registration property
chore: remove app parameter from AppService invocation
refactor: update AppConfig interface to correct registration and turnstile configurations
refactor: remove getCustomConfig dependency and use getAppConfig in PluginController, multer, and MCP services
refactor: replace getCustomConfig with getAppConfig in STTService, TTSService, and related files
refactor: replace getCustomConfig with getAppConfig in Conversation and Message models, update tempChatRetention functions to use AppConfig type
refactor: update getAppConfig calls in Conversation and Message models to include user role for temporary chat expiration
ci: update related tests
refactor: update getAppConfig call in getCustomConfigSpeech to include user role
fix: update appConfig usage to access allowedDomains from actions instead of registration
refactor: enhance AppConfig to include fileStrategies and update related file strategy logic
refactor: update imports to use normalizeEndpointName from @librechat/api and remove redundant definitions
chore: remove deprecated unused RunManager
refactor: get balance config primarily from appConfig
refactor: remove customConfig dependency for appConfig and streamline loadConfigModels logic
refactor: remove getCustomConfig usage and use app config in file citations
refactor: consolidate endpoint loading logic into loadEndpoints function
refactor: update appConfig access to use endpoints structure across various services
refactor: implement custom endpoints configuration and streamline endpoint loading logic
refactor: update getAppConfig call to include user role parameter
refactor: streamline endpoint configuration and enhance appConfig usage across services
refactor: replace getMCPAuthMap with getUserMCPAuthMap and remove unused getCustomConfig file
refactor: add type annotation for loadedEndpoints in loadEndpoints function
refactor: move /services/Files/images/parse to TS API
chore: add missing FILE_CITATIONS permission to IRole interface
refactor: restructure toolkits to TS API
refactor: separate manifest logic into its own module
refactor: consolidate tool loading logic into a new tools module for startup logic
refactor: move interface config logic to TS API
refactor: migrate checkEmailConfig to TypeScript and update imports
refactor: add FunctionTool interface and availableTools to AppConfig
refactor: decouple caching and DB operations from AppService, make part of consolidated `getAppConfig`
WIP: fix tests
* fix: rebase conflicts
* refactor: remove app.locals references
* refactor: replace getBalanceConfig with getAppConfig in various strategies and middleware
* refactor: replace appConfig?.balance with getBalanceConfig in various controllers and clients
* test: add balance configuration to titleConvo method in AgentClient tests
* chore: remove unused `openai-chat-tokens` package
* chore: remove unused imports in initializeMCPs.js
* refactor: update balance configuration to use getAppConfig instead of getBalanceConfig
* refactor: integrate configMiddleware for centralized configuration handling
* refactor: optimize email domain validation by removing unnecessary async calls
* refactor: simplify multer storage configuration by removing async calls
* refactor: reorder imports for better readability in user.js
* refactor: replace getAppConfig calls with req.config for improved performance
* chore: replace getAppConfig calls with req.config in tests for centralized configuration handling
* chore: remove unused override config
* refactor: add configMiddleware to endpoint route and replace getAppConfig with req.config
* chore: remove customConfig parameter from TTSService constructor
* refactor: pass appConfig from request to processFileCitations for improved configuration handling
* refactor: remove configMiddleware from endpoint route and retrieve appConfig directly in getEndpointsConfig if not in `req.config`
* test: add mockAppConfig to processFileCitations tests for improved configuration handling
* fix: pass req.config to hasCustomUserVars and call without await after synchronous refactor
* fix: type safety in useExportConversation
* refactor: retrieve appConfig using getAppConfig in PluginController and remove configMiddleware from plugins route, to avoid always retrieving when plugins are cached
* chore: change `MongoUser` typedef to `IUser`
* fix: Add `user` and `config` fields to ServerRequest and update JSDoc type annotations from Express.Request to ServerRequest
* fix: remove unused setAppConfig mock from Server configuration tests
* 👁️ feat: Add Azure Mistral OCR strategy and endpoint integration
This commit introduces a new OCR strategy named 'azure_mistral_ocr', allowing the use of a Mistral OCR endpoint deployed on Azure. The configuration, schemas, and file upload strategies have been updated to support this integration, enabling seamless OCR processing via Azure-hosted Mistral services.
* 🗑️ chore: Clean up .gitignore by removing commented-out uncommon directory name
* chore: remove unused vars
* refactor: Move createAxiosInstance to packages/api/utils and update imports
- Removed the createAxiosInstance function from the config module and relocated it to a new utils module for better organization.
- Updated import paths in relevant files to reflect the new location of createAxiosInstance.
- Added tests for createAxiosInstance to ensure proper functionality and proxy configuration handling.
* chore: move axios helpers to packages/api
- Added logAxiosError function to @librechat/api for centralized error logging.
- Updated imports across various files to use the new logAxiosError function.
- Removed the old axios.js utility file as it is no longer needed.
* chore: Update Jest moduleNameMapper for improved path resolution
- Added a new mapping for '~/' to resolve module paths in Jest configuration, enhancing import handling for the project.
* feat: Implement Mistral OCR API integration in TS
* chore: Update MistralOCR tests based on new imports
* fix: Enhance MistralOCR configuration handling and tests
- Introduced helper functions for resolving configuration values from environment variables or hardcoded settings.
- Updated the uploadMistralOCR and uploadAzureMistralOCR functions to utilize the new configuration resolution logic.
- Improved test cases to ensure correct behavior when mixing environment variables and hardcoded values.
- Mocked file upload and signed URL responses in tests to validate functionality without external dependencies.
* feat: Enhance MistralOCR functionality with improved configuration and error handling
- Introduced helper functions for loading authentication configuration and resolving values from environment variables.
- Updated uploadMistralOCR and uploadAzureMistralOCR functions to utilize the new configuration logic.
- Added utility functions for processing OCR results and creating error messages.
- Improved document type determination and result aggregation for better OCR processing.
* refactor: Reorganize OCR type imports in Mistral CRUD file
- Moved OCRResult, OCRResultPage, and OCRImage imports to a more logical grouping for better readability and maintainability.
* feat: Add file exports to API and create files index
* chore: Update OCR types for enhanced structure and clarity
- Redesigned OCRImage interface to include mandatory fields and improved naming conventions.
- Added PageDimensions interface for better representation of page metrics.
- Updated OCRResultPage to include dimensions and mandatory images array.
- Refined OCRResult to include document annotation and usage information.
* refactor: use TS counterpart of uploadOCR methods
* ci: Update MistralOCR tests to reflect new OCR result structure
* chore: Bump version of @librechat/api to 1.2.3 in package.json and package-lock.json
* chore: Update CONFIG_VERSION to 1.2.8
* chore: remove unused sendEvent function from config module (now imported from '@librechat/api')
* chore: remove MistralOCR service files and tests (now in '@librechat/api')
* ci: update logger import in ModelService tests to use @librechat/data-schemas
---------
Co-authored-by: arthurolivierfortin <arthurolivier.fortin@gmail.com>
* ✨ feat: Enhance Spinner component with customizable properties and improved animation
* 🔧 fix: Replace Loader with Spinner in RunCode component and update FilePreview to use Spinner for progress indication
* ✨ feat: Refactor icons in CodeProgress and CancelledIcon components; enhance animation and styling in ExecuteCode and ProgressText components
* ✨ feat: Refactor attachment handling in ExecuteCode component; replace individual attachment rendering with AttachmentGroup for improved structure
* ✨ feat: Refactor dialog components for improved accessibility and styling; integrate Skeleton loading state in Image component
* ✨ feat: Refactor ToolCall component to use ToolCallInfo for better structure; replace ToolPopover with AttachmentGroup; enhance ProgressText with error handling and improved UI elements
* 🔧 fix: Remove unnecessary whitespace in ProgressText
* 🔧 fix: Remove unnecessary margin from AgentFooter and AgentPanel components; clean up SidePanel imports
* ✨ feat: Enhance ToolCall and ToolCallInfo components with improved styling; update translations and add warning text color to Tailwind config
* 🔧 fix: Update import statement for useLocalize in ToolCallInfo component; fix: chatform transition
* ✨ feat: Refactor ToolCall and ToolCallInfo components for improved structure and styling; add optimized code block for better output display
* ✨ feat: Implement OpenAI image generation component; add progress tracking and localization for user feedback
* 🔧 fix: Adjust base duration values for image generation; optimize timing for quality settings
* chore: remove unnecessary space
* ✨ feat: Enhance OpenAI image generation with editing capabilities; update localization for progress feedback
* ✨ feat: Add download functionality to images; enhance DialogImage component with download button
* ✨ feat: Enhance image resizing functionality; support custom percentage and pixel dimensions in resizeImageBuffer
* fix: azure title model
* refactor: typing for uploadMistralOCR
* fix: update conversation ID handling in useSSE for better state management, only use PENDING_CONVO for new conversations
* fix: streamline conversation ID handling in useSSE for simplicity, only needs state update to prevent draft from applying
* fix: update performOCR and tests to support document and image URLs with appropriate types
* 🔧 fix: Update file source references to include 'azure_blob' for correct service initialization
* 🔧 fix: Add Azure Blob Storage Emulator entries to .gitignore
* fix: Update file source references to include 'azure_blob' for correct service initialization
* fix: Refactor Azure Blob Storage functions to use environment variables for access control and container name, fix deletion improper logging and improper params
* fix: Add basePath determination for agent file uploads based on MIME type
* fix: Implement file streaming to Azure Blob Storage to optimize memory usage during uploads (non-images)
* fix: Update SourceIcon to include 'azure_blob' class and adjust model setting in useSelectorEffects for assistants
* chore: import order
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
* wip: first pass, dropdown for selecting sequential agents
* refactor: Improve agent selection logic and enhance performance in SequentialAgents component
* wip: seq. agents working ideas
* wip: sequential agents style change
* refactor: move agent form options/submission outside of AgentConfig
* refactor: prevent repeating code
* refactor: simplify current agent display in SequentialAgents component
* feat: persist form value handling in AgentSelect component for agent_ids
* feat: first pass, sequential agnets agent update
* feat: enhance message display with agent updates and empty text handling
* chore: update Icon component to use EModelEndpoint for agent endpoints
* feat: update content type checks in BaseClient to use constants for better readability
* feat: adjust max context tokens calculation to use 90% of the model's max tokens
* feat: first pass, agent run message pruning
* chore: increase max listeners for abort controller to prevent memory leaks
* feat: enhance runAgent function to include current index count map for improved token tracking
* chore: update @librechat/agents dependency to version 2.2.5
* feat: update icons and style of SequentialAgents component for improved UI consistency
* feat: add AdvancedButton and AdvancedPanel components for enhanced agent settings navigation, update styling for agent form
* chore: adjust minimum height of AdvancedPanel component for better layout consistency
* chore: update @librechat/agents dependency to version 2.2.6
* feat: enhance message formatting by incorporating tool set into agent message processing, in order to allow better mix/matching of agents (as tool calls for tools not found in set will be stringified)
* refactor: reorder components in AgentConfig for improved readability and maintainability
* refactor: enhance layout of AgentUpdate component for improved visual structure
* feat: add DeepSeek provider to Bedrock settings and schemas
* feat: enhance link styling in mobile.css for better visibility and accessibility
* fix: update banner model import in update banner script; export Banner model
* refactor: `duplicateAgentHandler` to include tool_resources only for OCR context files
* feat: add 'qwen-vl' to visionModels for enhanced model support
* fix: change image format from JPEG to PNG in DALLE3 response
* feat: reorganize Advanced components and add localizations
* refactor: simplify JSX structure in AgentChain component to defer container styling to parent
* feat: add FormInput component for reusable input handling
* feat: make agent recursion limit configurable from builder
* feat: add support for agent capabilities chain in AdvancedPanel and update data-provider version
* feat: add maxRecursionLimit configuration for agents and update related documentation
* fix: update CONFIG_VERSION to 1.2.3 in data provider configuration
* feat: replace recursion limit input with MaxAgentSteps component and enhance input handling
* feat: enhance AgentChain component with hover card for additional information and update related labels
* fix: pass request and response objects to `createActionTool` when using assistant actions to prevent auth error
* feat: update AgentChain component layout to include agent count display
* feat: increase default max listeners and implement capability check function for agent chain
* fix: update link styles in mobile.css for better visibility in dark mode
* chore: temp. remove agents package while bumping shared packages
* chore: update @langchain/google-genai package to version 0.1.11
* chore: update @langchain/google-vertexai package to version 0.2.2
* chore: add @librechat/agents package at version 2.2.8
* feat: add deepseek.r1 model with token rate and context values for bedrock
* refactor: move `loadAuthValues` to `~/services/Tools/credentials`
* feat: add createAxiosInstance function to configure axios with proxy support
* WIP: First pass mistral ocr
* refactor: replace getConvoFiles with getToolFiles for improved file retrieval logic
* refactor: improve document formatting in encodeAndFormat function
* refactor: remove unused resendFiles parameter from buildOptions function (this option comes from the agent config)
* fix: update getFiles call to include files with `text` property as well
* refactor: move file handling to `initializeAgentOptions`
* refactor: enhance addImageURLs method to handle OCR text and improve message formatting
* refactor: update message formatting to handle OCR text in various content types
* refactor: remove unused resendFiles property from compactAgentsSchema
* fix: add error handling for Mistral OCR document upload and logging
* refactor: integrate OCR capability into file upload options and configuration
* refactor: skip processing for text source files in delete request, as they are directly tied to database
* feat: add metadata field to ExtendedFile type and update PanelColumns and PanelTable components for localization and metadata handling
* fix: source icon styling
* wip: first pass, frontend file context agent resources
* refactor: add hover card with contextual information for File Context (OCR) in FileContext component
* feat: enhance file processing by integrating file retrieval for OCR resources in agent initialization
* feat: implement OCR config; fix: agent resource deletion for ocr files
* feat: enhance agent initialization by adding OCR capability check in resource priming
* ci: fix `~/config` module mock
* ci: add OCR property expectation in AppService tests
* refactor: simplify OCR config loading by removing environment variable extraction, to be done when OCR is actually performed
* ci: add unit test to ensure environment variable references are not parsed in OCR config
* refactor: disable base64 image inclusion in OCR request
* refactor: enhance OCR configuration handling by validating environment variables and providing defaults
* refactor: use file stream from disk for mistral ocr api
* refactor: Improve error logging for file upload and processing functions to prevent verbosity
* refactor: Add uploads directory to Docker Compose to persist file uploads
* refactor: `addAgentResourceFile` to handle edge case of non-existing `tool_resource` array
* refactor: Remove version specification from deploy-compose.yml
* refactor: Prefix filenames with file_id to ensure uniqueness in file uploads
* refactor: Enhance error handling in deleteVectors to log warnings for non-404 errors
* refactor: Limit file search results to top 5 based on relevance score
* 🌍 i18n: Update translation.json with latest translations
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* 📝 chore: Add comment to clarify purpose of check_updates.sh script
* feat: mcp package
* feat: add librechat-mcp package and update dependencies
* feat: refactor MCPConnectionSingleton to handle transport initialization and connection management
* feat: change private methods to public in MCPConnectionSingleton for improved accessibility
* feat: filesystem demo
* chore: everything demo and move everything under mcp workspace
* chore: move ts-node to mcp workspace
* feat: mcp examples
* feat: working sse MCP example
* refactor: rename MCPConnectionSingleton to MCPConnection for clarity
* refactor: replace MCPConnectionSingleton with MCPConnection for consistency
* refactor: manager/connections
* refactor: update MCPConnection to use type definitions from mcp types
* refactor: update MCPManager to use winston logger and enhance server initialization
* refactor: share logger between connections and manager
* refactor: add schema definitions and update MCPManager to accept logger parameter
* feat: map available MCP tools
* feat: load manifest tools
* feat: add MCP tools delimiter constant and update plugin key generation
* feat: call MCP tools
* feat: update librechat-data-provider version to 0.7.63 and enhance StdioOptionsSchema with additional properties
* refactor: simplify typing
* chore: update types/packages
* feat: MCP Tool Content parsing
* chore: update dependencies and improve package configurations
* feat: add 'mcp' directory to package and update configurations
* refactor: return CONTENT_AND_ARTIFACT format for MCP callTool
* chore: bump @librechat/agents
* WIP: MCP artifacts
* chore: bump @librechat/agents to v1.8.7
* fix: ensure filename has extension when saving base64 image
* fix: move base64 buffer conversion before filename extension check
* chore: update backend review workflow to install MCP package
* fix: use correct `mime` method
* fix: enhance file metadata with message and tool call IDs in image saving process
* fix: refactor ToolCall component to handle MCP tool calls and improve domain extraction
* fix: update ToolItem component for default isInstalled value and improve localization in ToolSelectDialog
* fix: update ToolItem component to use consistent text color for tool description
* style: add theming to ToolSelectDialog
* fix: improve domain extraction logic in ToolCall component
* refactor: conversation item theming, fix rename UI bug, optimize props, add missing types
* feat: enhance MCP options schema with base options (iconPath to start) and make transport type optional, infer based on other option fields
* fix: improve reconnection logic with parallel init and exponential backoff and enhance transport debug logging
* refactor: improve logging format
* refactor: improve logging of available tools by displaying tool names
* refactor: improve reconnection/connection logic
* feat: add MCP package build process to Dockerfile
* feat: add fallback icon for tools without an image in ToolItem component
* feat: Assistants Support for MCP Tools
* fix(build): configure rollup to use output.dir for dynamic imports
* chore: update @librechat/agents to version 1.8.8 and add @langchain/anthropic dependency
* fix: update CONFIG_VERSION to 1.2.0
* feat: Code Interpreter API & File Search Agent Uploads
chore: add back code files
wip: first pass, abstract key dialog
refactor: influence checkbox on key changes
refactor: update localization keys for 'execute code' to 'run code'
wip: run code button
refactor: add throwError parameter to loadAuthValues and getUserPluginAuthValue functions
feat: first pass, API tool calling
fix: handle missing toolId in callTool function and return 404 for non-existent tools
feat: show code outputs
fix: improve error handling in callTool function and log errors
fix: handle potential null value for filepath in attachment destructuring
fix: normalize language before rendering and prevent null return
fix: add loading indicator in RunCode component while executing code
feat: add support for conditional code execution in Markdown components
feat: attachments
refactor: remove bash
fix: pass abort signal to graph/run
refactor: debounce and rate limit tool call
refactor: increase debounce delay for execute function
feat: set code output attachments
feat: image attachments
refactor: apply message context
refactor: pass `partIndex`
feat: toolCall schema/model/methods
feat: block indexing
feat: get tool calls
chore: imports
chore: typing
chore: condense type imports
feat: get tool calls
fix: block indexing
chore: typing
refactor: update tool calls mapping to support multiple results
fix: add unique key to nav link for rendering
wip: first pass, tool call results
refactor: update query cache from successful tool call mutation
style: improve result switcher styling
chore: note on using \`.toObject()\`
feat: add agent_id field to conversation schema
chore: typing
refactor: rename agentMap to agentsMap for consistency
feat: Agent Name as chat input placeholder
chore: bump agents
📦 chore: update @langchain dependencies to latest versions to match agents package
📦 chore: update @librechat/agents dependency to version 1.8.0
fix: Aborting agent stream removes sender; fix(bedrock): completion removes preset name label
refactor: remove direct file parameter to use req.file, add `processAgentFileUpload` for image uploads
feat: upload menu
feat: prime message_file resources
feat: implement conversation access validation in chat route
refactor: remove file parameter from processFileUpload and use req.file instead
feat: add savedMessageIds set to track saved message IDs in BaseClient, to prevent unnecessary double-write to db
feat: prevent duplicate message saves by checking savedMessageIds in AgentController
refactor: skip legacy RAG API handling for agents
feat: add files field to convoSchema
refactor: update request type annotations from Express.Request to ServerRequest in file processing functions
feat: track conversation files
fix: resendFiles, addPreviousAttachments handling
feat: add ID validation for session_id and file_id in download route
feat: entity_id for code file uploads/downloads
fix: code file edge cases
feat: delete related tool calls
feat: add stream rate handling for LLM configuration
feat: enhance system content with attached file information
fix: improve error logging in resource priming function
* WIP: PoC, sequential agents
WIP: PoC Sequential Agents, first pass content data + bump agents package
fix: package-lock
WIP: PoC, o1 support, refactor bufferString
feat: convertJsonSchemaToZod
fix: form issues and schema defining erroneous model
fix: max length issue on agent form instructions, limit conversation messages to sequential agents
feat: add abort signal support to createRun function and AgentClient
feat: PoC, hide prior sequential agent steps
fix: update parameter naming from config to metadata in event handlers for clarity, add model to usage data
refactor: use only last contentData, track model for usage data
chore: bump agents package
fix: content parts issue
refactor: filter contentParts to include tool calls and relevant indices
feat: show function calls
refactor: filter context messages to exclude tool calls when no tools are available to the agent
fix: ensure tool call content is not undefined in formatMessages
feat: add agent_id field to conversationPreset schema
feat: hide sequential agents
feat: increase upload toast duration to 10 seconds
* refactor: tool context handling & update Code API Key Dialog
feat: toolContextMap
chore: skipSpecs -> useSpecs
ci: fix handleTools tests
feat: API Key Dialog
* feat: Agent Permissions Admin Controls
feat: replace label with button for prompt permission toggle
feat: update agent permissions
feat: enable experimental agents and streamline capability configuration
feat: implement access control for agents and enhance endpoint menu items
feat: add welcome message for agent selection in localization
feat: add agents permission to access control and update version to 0.7.57
* fix: update types in useAssistantListMap and useMentions hooks for better null handling
* feat: mention agents
* fix: agent tool resource race conditions when deleting agent tool resource files
* feat: add error handling for code execution with user feedback
* refactor: rename AdminControls to AdminSettings for clarity
* style: add gap to button in AdminSettings for improved layout
* refactor: separate agent query hooks and check access to enable fetching
* fix: remove unused provider from agent initialization options, creates issue with custom endpoints
* refactor: remove redundant/deprecated modelOptions from AgentClient processes
* chore: update @librechat/agents to version 1.8.5 in package.json and package-lock.json
* fix: minor styling issues + agent panel uniformity
* fix: agent edge cases when set endpoint is no longer defined
* refactor: remove unused cleanup function call from AppService
* fix: update link in ApiKeyDialog to point to pricing page
* fix: improve type handling and layout calculations in SidePanel component
* fix: add missing localization string for agent selection in SidePanel
* chore: form styling and localizations for upload filesearch/code interpreter
* fix: model selection placeholder logic in AgentConfig component
* style: agent capabilities
* fix: add localization for provider selection and improve dropdown styling in ModelPanel
* refactor: use gpt-4o-mini > gpt-3.5-turbo
* fix: agents configuration for loadDefaultInterface and update related tests
* feat: DALLE Agents support
* fix: sanitize filename in multer storage callback
* fix: ensure temporary image upload file is deleted after processing
* fix: prevent cleanup flag from being set to false before actually deleted
* refactor: user avatar, typing, use 'file' for formData instead of 'input', add disk storage, use localization
* fix: update Avatar component to include image dimensions in formData and refactor editor reference type
* fix: refactor avatar upload handling to use fs for file reading and enhance file validation
* fix: ensure temporary image upload file is deleted after processing
* fix: refactor avatar upload routes and handlers for agents and assistants, improve file handling and validation
* fix: improve audio file validation and cleanup
* fix: add filename sanitization utility and integrate it into multer storage configuration
* fix: update group project ID check for null and refactor delete prompt group response type
* fix: invalid access control for deleting prompt groups
* fix: add error handling and logging to checkBan middleware
* fix: catch conversation parsing errors
* chore: revert unnecessary height and width parameters from avatar upload
* chore: update librechat-data-provider version to 0.7.55
* style: ensure KaTeX can spread across visible space
* fix: agent initialization, add `collectedUsage` handling
* style: improve side panel styling
* refactor(loadAgent): Optimize order agent project ID retrieval
* feat: code execution
* fix: typing issues
* feat: ExecuteCode content part
* refactor: use local state for default collapsed state of analysis content parts
* fix: code parsing in ExecuteCode component
* chore: bump agents package, export loadAuthValues
* refactor: Update handleTools.js to use EnvVar for code execution tool authentication
* WIP
* feat: download code outputs
* fix(useEventHandlers): type issues
* feat: backend handling for code outputs
* Refactor: Remove console.log statement in Part.tsx
* refactor: add attachments to TMessage/messageSchema
* WIP: prelim handling for code outputs
* feat: attachments rendering
* refactor: improve attachments rendering
* fix: attachments, nullish edge case, handle attachments from event stream, bump agents package
* fix filename download
* fix: tool assignment for 'run code' on agent creation
* fix: image handling by adding attachments
* refactor: prevent agent creation without provider/model
* refactor: remove unnecessary space in agent creation success message
* refactor: select first model if selecting provider from empty on form
* fix: Agent avatar bug
* fix: `defaultAgentFormValues` causing boolean typing issue and typeerror
* fix: capabilities counting as tools, causing duplication of them
* fix: formatted messages edge case where consecutive content text type parts with the latter having tool_call_ids would cause consecutive AI messages to be created. furthermore, content could not be an array for tool_use messages (anthropic limitation)
* chore: bump @librechat/agents dependency to version 1.6.9
* feat: bedrock agents
* feat: new Agents icon
* feat: agent titling
* feat: agent landing
* refactor: allow sharing agent globally only if user is admin or author
* feat: initial AgentPanelSkeleton
* feat: AgentPanelSkeleton
* feat: collaborative agents
* chore: add potential authorName as part of schema
* chore: Remove unnecessary console.log statement
* WIP: agent model parameters
* chore: ToolsDialog typing and tool related localization chnages
* refactor: update tool instance type (latest langchain class), and rename google tool to 'google' proper
* chore: add back tools
* feat: Agent knowledge files upload
* refactor: better verbiage for disabled knowledge
* chore: debug logs for file deletions
* chore: debug logs for file deletions
* feat: upload/delete agent knowledge/file-search files
* feat: file search UI for agents
* feat: first pass, file search tool
* chore: update default agent capabilities and info
* 🤖 Assistants V2 Support: Part 1
- Separated Azure Assistants to its own endpoint
- File Search / Vector Store integration is incomplete, but can toggle and use storage from playground
- Code Interpreter resource files can be added but not deleted
- GPT-4o is supported
- Many improvements to the Assistants Endpoint overall
data-provider v2 changes
copy existing route as v1
chore: rename new endpoint to reduce comparison operations and add new azure filesource
api: add azureAssistants part 1
force use of version for assistants/assistantsAzure
chore: switch name back to azureAssistants
refactor type version: string | number
Ensure assistants endpoints have version set
fix: isArchived type issue in ConversationListParams
refactor: update assistants mutations/queries with endpoint/version definitions, update Assistants Map structure
chore: FilePreview component ExtendedFile type assertion
feat: isAssistantsEndpoint helper
chore: remove unused useGenerations
chore(buildTree): type issue
chore(Advanced): type issue (unused component, maybe in future)
first pass for multi-assistant endpoint rewrite
fix(listAssistants): pass params correctly
feat: list separate assistants by endpoint
fix(useTextarea): access assistantMap correctly
fix: assistant endpoint switching, resetting ID
fix: broken during rewrite, selecting assistant mention
fix: set/invalidate assistants endpoint query data correctly
feat: Fix issue with assistant ID not being reset correctly
getOpenAIClient helper function
feat: add toast for assistant deletion
fix: assistants delete right after create issue for azure
fix: assistant patching
refactor: actions to use getOpenAIClient
refactor: consolidate logic into helpers file
fix: issue where conversation data was not initially available
v1 chat support
refactor(spendTokens): only early return if completionTokens isNaN
fix(OpenAIClient): ensure spendTokens has all necessary params
refactor: route/controller logic
fix(assistants/initializeClient): use defaultHeaders field
fix: sanitize default operation id
chore: bump openai package
first pass v2 action service
feat: retroactive domain parsing for actions added via v1
feat: delete db records of actions/assistants on openai assistant deletion
chore: remove vision tools from v2 assistants
feat: v2 upload and delete assistant vision images
WIP first pass, thread attachments
fix: show assistant vision files (save local/firebase copy)
v2 image continue
fix: annotations
fix: refine annotations
show analyze as error if is no longer submitting before progress reaches 1 and show file_search as retrieval tool
fix: abort run, undefined endpoint issue
refactor: consolidate capabilities logic and anticipate versioning
frontend version 2 changes
fix: query selection and filter
add endpoint to unknown filepath
add file ids to resource, deleting in progress
enable/disable file search
remove version log
* 🤖 Assistants V2 Support: Part 2
🎹 fix: Autocompletion Chrome Bug on Action API Key Input
chore: remove `useOriginNavigate`
chore: set correct OpenAI Storage Source
fix: azure file deletions, instantiate clients by source for deletion
update code interpret files info
feat: deleteResourceFileId
chore: increase poll interval as azure easily rate limits
fix: openai file deletions, TODO: evaluate rejected deletion settled promises to determine which to delete from db records
file source icons
update table file filters
chore: file search info and versioning
fix: retrieval update with necessary tool_resources if specified
fix(useMentions): add optional chaining in case listMap value is undefined
fix: force assistant avatar roundedness
fix: azure assistants, check correct flag
chore: bump data-provider
* fix: merge conflict
* ci: fix backend tests due to new updates
* chore: update .env.example
* meilisearch improvements
* localization updates
* chore: update comparisons
* feat: add additional metadata: endpoint, author ID
* chore: azureAssistants ENDPOINTS exclusion warning
* WIP: gemini-1.5 support
* feat: extended vertex ai support
* fix: handle possibly undefined modelName
* fix: gpt-4-turbo-preview invalid vision model
* feat: specify `fileConfig.imageOutputType` and make PNG default image conversion type
* feat: better truncation for errors including base64 strings
* fix: gemini inlineData formatting
* feat: RAG augmented prompt for gemini-1.5
* feat: gemini-1.5 rates and token window
* chore: adjust tokens, update docs, update vision Models
* chore: add back `ChatGoogleVertexAI` for chat models via vertex ai
* refactor: ask/edit controllers to not use `unfinished` field for google endpoint
* chore: remove comment
* chore(ci): fix AppService test
* chore: remove comment
* refactor(GoogleSearch): use `GOOGLE_SEARCH_API_KEY` instead, issue warning for old variable
* chore: bump data-provider to 0.5.4
* chore: update docs
* fix: condition for gemini-1.5 using generative ai lib
* chore: update docs
* ci: add additional AppService test for `imageOutputType`
* refactor: optimize new config value `imageOutputType`
* chore: bump CONFIG_VERSION
* fix(assistants): avatar upload
* fix(processMessages): properly handle assistant file citations and add sources list
* feat: improve file download UX by making any downloaded files accessible within the app post-download
* refactor(processOpenAIImageOutput): correctly handle two different outputs for images since OpenAI generates a file in their storage, shares filepath for image rendering
* refactor: create `addFileToCache` helper to use across frontend
* refactor: add ImageFile parts to cache on processing content stream
* WIP: basic route for file downloads and file strategy for generating readablestream to pipe as res
* chore(DALLE3): add typing for OpenAI client
* chore: add `CONSOLE_JSON` notes to dotenv.md
* WIP: first pass OpenAI Assistants File Output handling
* feat: first pass assistants output file download from openai
* chore: yml vs. yaml variation to .gitignore for `librechat.yml`
* refactor(retrieveAndProcessFile): remove redundancies
* fix(syncMessages): explicit sort of apiMessages to fix message order on abort
* chore: add logs for warnings and errors, show toast on frontend
* chore: add logger where console was still being used
* feat: new vector file processing strategy
* chore: remove unused client files
* chore: remove more unused client files
* chore: remove more unused client files and move used to new dir
* chore(DataIcon): add className
* WIP: Model Endpoint Settings Update, draft additional context settings
* feat: improve parsing for augmented prompt, add full context option
* chore: remove volume mounting from rag.yml as no longer necessary
* chore: bump openai to 4.29.0 and npm audit fix
* chore: remove unnecessary stream field from ContentData
* feat: new enum and types for AssistantStreamEvent
* refactor(AssistantService): remove stream field and add conversationId to text ContentData
> - return `finalMessage` and `text` on run completion
> - move `processMessages` to services/Threads to avoid circular dependencies with new stream handling
> - refactor(processMessages/retrieveAndProcessFile): add new `client` field to differentiate new RunClient type
* WIP: new assistants stream handling
* chore: stores messages to StreamRunManager
* chore: add additional typedefs
* fix: pass req and openai to StreamRunManager
* fix(AssistantService): pass openai as client to `retrieveAndProcessFile`
* WIP: streaming tool i/o, handle in_progress and completed run steps
* feat(assistants): process required actions with streaming enabled
* chore: condense early return check for useSSE useEffect
* chore: remove unnecessary comments and only handle completed tool calls when not function
* feat: add TTL for assistants run abort cacheKey
* feat: abort stream runs
* fix(assistants): render streaming cursor
* fix(assistants): hide edit icon as functionality is not supported
* fix(textArea): handle pasting edge cases; first, when onChange events wouldn't fire; second, when textarea wouldn't resize
* chore: memoize Conversations
* chore(useTextarea): reverse args order
* fix: load default capabilities when an azure is configured to support assistants, but `assistants` endpoint is not configured
* fix(AssistantSelect): update form assistant model on assistant form select
* fix(actions): handle azure strict validation for function names to fix crud for actions
* chore: remove content data debug log as it fires in rapid succession
* feat: improve UX for assistant errors mid-request
* feat: add tool call localizations and replace any domain separators from azure action names
* refactor(chat): error out tool calls without outputs during handleError
* fix(ToolService): handle domain separators allowing Azure use of actions
* refactor(StreamRunManager): types and throw Error if tool submission fails
* refactor: re-purpose `resendImages` as `resendFiles`
* refactor: re-purpose `resendImages` as `resendFiles`
* feat: upload general files
* feat: embed file during upload
* feat: delete file embeddings on file deletion
* chore(fileConfig): add epub+zip type
* feat(encodeAndFormat): handle non-image files
* feat(createContextHandlers): build context prompt from file attachments and successful RAG
* fix: prevent non-temp files as well as embedded files to be deleted on new conversation
* fix: remove temp_file_id on usage, prevent non-temp files as well as embedded files to be deleted on new conversation
* fix: prevent non-temp files as well as embedded files to be deleted on new conversation
* feat(OpenAI/Anthropic/Google): basic RAG support
* fix: delete `resendFiles` only when true (Default)
* refactor(RAG): update endpoints and pass JWT
* fix(resendFiles): default values
* fix(context/processFile): query unique ids only
* feat: rag-api.yaml
* feat: file upload improved ux for longer uploads
* chore: await embed call and catch embedding errors
* refactor: store augmentedPrompt in Client
* refactor(processFileUpload): throw error if not assistant file upload
* fix(useFileHandling): handle markdown empty mimetype issue
* chore: necessary compose file changes
* chore: rename dir from `assistant` to plural
* feat: `assistants` field for azure config, spread options in AppService
* refactor: rename constructAzureURL param for azure as `azureOptions`
* chore: bump openai and bun
* chore(loadDefaultModels): change naming of assistant -> assistants
* feat: load azure settings with currect baseURL for assistants' initializeClient
* refactor: add `assistants` flags to groups and model configs, add mapGroupToAzureConfig
* feat(loadConfigEndpoints): initialize assistants endpoint if azure flag `assistants` is enabled
* feat(AppService): determine assistant models on startup, throw Error if none
* refactor(useDeleteAssistantMutation): send model along with assistant id for delete mutations
* feat: support listing and deleting assistants with azure
* feat: add model query to assistant avatar upload
* feat: add azure support for retrieveRun method
* refactor: update OpenAIClient initialization
* chore: update README
* fix(ci): tests passing
* refactor(uploadOpenAIFile): improve logging and use more efficient REST API method
* refactor(useFileHandling): add model to metadata to target Azure region compatible with current model
* chore(files): add azure naming pattern for valid file id recognition
* fix(assistants): initialize openai with first available assistant model if none provided
* refactor(uploadOpenAIFile): add content type for azure, initialize formdata before azure options
* refactor(sleep): move sleep function out of Runs and into `~/server/utils`
* fix(azureOpenAI/assistants): make sure to only overwrite models with assistant models if `assistants` flag is enabled
* refactor(uploadOpenAIFile): revert to old method
* chore(uploadOpenAIFile): use enum for file purpose
* docs: azureOpenAI update guide with more info, examples
* feat: enable/disable assistant capabilities and specify retrieval models
* refactor: optional chain conditional statement in loadConfigModels.js
* docs: add assistants examples
* chore: update librechat.example.yaml
* docs(azure): update note of file upload behavior in Azure OpenAI Assistants
* chore: update docs and add descriptive message about assistant errors
* fix: prevent message submission with invalid assistant or if files loading
* style: update Landing icon & text when assistant is not selected
* chore: bump librechat-data-provider to 0.4.8
* fix(assistants/azure): assign req.body.model for proper azure init to abort runs
* chore: bump anthropic SDK
* chore: update anthropic config settings (fileSupport, default models)
* feat: anthropic multi modal formatting
* refactor: update vision models and use endpoint specific max long side resizing
* feat(anthropic): multimodal messages, retry logic, and messages payload
* chore: add more safety to trimming content due to whitespace error for assistant messages
* feat(anthropic): token accounting and resending multiple images in progress
* chore: bump data-provider
* feat(anthropic): resendImages feature
* chore: optimize Edit/Ask controllers, switch model back to req model
* fix: false positive of invalid model
* refactor(validateVisionModel): use object as arg, pass in additional/available models
* refactor(validateModel): use helper function, `getModelsConfig`
* feat: add modelsConfig to endpointOption so it gets passed to all clients, use for properly validating vision models
* refactor: initialize default vision model and make sure it's available before assigning it
* refactor(useSSE): avoid resetting model if user selected a new model between request and response
* feat: show rate in transaction logging
* fix: return tokenCountMap regardless of payload shape
* feat: assistant name/icon in Landing & Header
* feat: assistname in textarea placeholder, and use `Assistant` as default name
* feat: display non-image files in user messages
* fix: only render files if files.length is > 0
* refactor(config -> file-config): move file related configuration values to separate module, add excel types
* chore: spreadsheet file rendering
* fix(Landing): dark mode style for Assistant Name
* refactor: move progress incrementing to own hook, start smaller, cap near limit \(1\)
* refactor(useContentHandler): add empty Text part if last part was completed tool or image
* chore: add accordion trigger border styling for dark mode
* feat: Assistant Builder model selection
* chore: use Spinner when Assistant is mutating
* fix(get/assistants): return correct response object `AssistantListResponse`
* refactor(Spinner): pass size as prop
* refactor: make assistant crud mutations optimistic, add types for options
* chore: remove assistants route and view
* chore: move assistant builder components to separate directory
* feat(ContextButton): delete Assistant via context button/dialog, add localization
* refactor: conditionally show use and context menu buttons, add localization for create assistant
* feat: save side panel states to localStorage
* style(SidePanel): improve avatar menu and assistant select styling for dark mode
* refactor: make NavToggle reusable for either side (left or right), add SidePanel Toggle with ability to close it completely
* fix: resize handle and navToggle behavior
* fix(/avatar/:assistant_id): await `deleteFile` and assign unique name to uploaded image
* WIP: file UI components from PR #576
* refactor(OpenAIMinimalIcon): pass className
* feat: formatDate helper fn
* feat: DataTableColumnHeader
* feat: add row selection, formatted row values, number of rows selected
* WIP: add files to Side panel temporarily
* feat: `LB_QueueAsyncCall`: Leaky Bucket queue for external APIs, use in `processDeleteRequest`
* fix(TFile): correct `source` type with `FileSources`
* fix(useFileHandling): use `continue` instead of return when iterating multiple files, add file type to extendedFile
* chore: add generic setter type
* refactor(processDeleteRequest): settle promises to prevent rejections from processing deletions, log errors
* feat: `useFileDeletion` to reuse file deletion logic
* refactor(useFileDeletion): make `setFiles` an optional param and use object as param
* feat: useDeleteFilesFromTable
* feat: use real `files` data and add deletion action to data table
* fix(Table): make headers sticky
* feat: add dynamic filtering for columns; only show to user Host or OpenAI storage type
* style(DropdownMenu): replace `slate` with `gray`
* style(DataTable): apply dark mode themes and other misc styling
* style(Columns): add color to OpenAI Storage option
* refactor(FileContainer): make file preview reusable
* refactor(Images): make image preview reusable
* refactor(FilePreview): make file prop optional for FileIcon and FilePreview, fix relative style
* feat(Columns): add file/image previews, set a minimum size to show for file size in bytes
* WIP: File Panel with real files and formatted
* feat: open files dialog from panel
* style: file data table mobile and general column styling fixes
* refactor(api/files): return files sorted by the most recently updated
* refactor: provide fileMap through context to prevent re-selecting files to map in different areas; remove unused imports commented out in PanelColumns
* refactor(ExtendFile): make File type optional, add `attached` to prevent attached files from being deleted on remove, make Message.files a partial TFile type
* feat: attach files through file panel
* refactor(useFileHandling): move files to the start of cache list when uploaded
* refactor(useDeleteFilesMutation): delete files from cache when successfully deleted from server
* fix(FileRow): handle possible edge case of duplication due to attaching recently uploaded file
* style(SidePanel): make resize grip border transparent, remove unnecessary styling on close sidepanel button
* feat: action utilities and tests
* refactor(actions): add `ValidationResult` type and change wording for no server URL found
* refactor(actions): check for empty server URL
* fix(data-provider): revert tsconfig to fix type issue resolution
* feat(client): first pass of actions input for assistants
* refactor(FunctionSignature): change method to output object instead of string
* refactor(models/Assistant): add actions field to schema, use searchParams object for methods, and add `getAssistant`
* feat: post actions input first pass
- create new Action document
- add actions to Assistant DB document
- create /action/:assistant_id POST route
- pass more props down from PanelSwitcher, derive assistant_id from switcher
- move privacy policy to ActionInput
- reset data on input change/validation
- add `useUpdateAction`
- conform FunctionSignature type to FunctionTool
- add action, assistant doc, update hook related types
* refactor: optimize assistant/actions relationship
- past domain in metadata as hostname and not a URL
- include domain in tool name
- add `getActions` for actions retrieval by user
- add `getAssistants` for assistant docs retrieval by user
- add `assistant_id` to Action schema
- move actions to own module as a subroute to `api/assistants`
- add `useGetActionsQuery` and `useGetAssistantDocsQuery` hooks
- fix Action type def
* feat: show assistant actions in assistant builder
* feat: switch to actions on action click, editing action styling
* fix: add Assistant state for builder panel to allow immediate selection of newly created assistants as well as retaining the current assistant when switching to a different panel within the builder
* refactor(SidePanel/NavToggle): offset less from right when SidePanel is completely collapsed
* chore: rename `processActions` -> `processRequiredActions`
* chore: rename Assistant API Action to RequiredAction
* refactor(actions): avoid nesting actual API params under generic `requestBody` to optimize LLM token usage
* fix(handleTools): avoid calling `validTool` if not defined, add optional param to skip the loading of specs, which throws an error in the context of assistants
* WIP: working first pass of toolCalls generated from openapi specs
* WIP: first pass ToolCall styling
* feat: programmatic iv encryption/decryption helpers
* fix: correct ActionAuth types/enums, and define type for AuthForm
* feat: encryption/decryption helpers for Action AuthMetadata
* refactor(getActions): remove sensitive fields from query response
* refactor(POST/actions): encrypt and remove sensitive fields from mutation response
* fix(ActionService): change ESM import to CJS
* feat: frontend auth handling for actions + optimistic update on action update/creation
* refactor(actions): use the correct variables and types for setAuth method
* refactor: POST /:assistant_id action can now handle updating an existing action, add `saved_auth_fields` to determine when user explicitly saves new auth creds. only send auth metadata if user explicitly saved fields
* refactor(createActionTool): catch errors and send back meaningful error message, add flag to `getActions` to determine whether to retrieve sensitive values or not
* refactor(ToolService): add `action` property to ToolCall PartMetadata to determine if the tool call was an action, fix parsing function name issue with actionDelimiter
* fix(ActionRequest): use URL class to correctly join endpoint parts for `execute` call
* feat: delete assistant actions
* refactor: conditionally show Available actions
* refactor: show `retrieval` and `code_interpreter` as Capabilities, swap `Switch` for `Checkbox`
* chore: remove shadow-stroke from messages
* WIP: first pass of Assistants Knowledge attachments
* refactor: remove AssistantsProvider in favor of FormProvider, fix selectedAssistant re-render bug, map Assistant file_ids to files via fileMap, initialize Knowledge component with mapped files if any exist
* fix: prevent deleting files on assistant file upload
* chore: remove console.log
* refactor(useUploadFileMutation): update files and assistants cache on upload
* chore: disable oauth option as not supported yet
* feat: cancel assistant runs
* refactor: initialize OpenAI client with helper function, resolve all related circular dependencies
* fix(DALL-E): initialization
* fix(process): openai client initialization
* fix: select an existing Assistant when the active one is deleted
* chore: allow attaching files for assistant endpoint, send back relevant OpenAI error message when uploading, deconstruct openAI initialization correctly, add `message_file` to formData when a file is attached to the message but not the assistant
* fix: add assistant_id on newConvo
* fix(initializeClient): import fix
* chore: swap setAssistant for setOption in useEffect
* fix(DALL-E): add processFileURL to loadTools call
* chore: add customConfig to debug logs
* feat: delete threads on convo delete
* chore: replace Assistants icon
* chore: remove console.dir() in `abortRun`
* feat(AssistantService): accumulate text values from run in openai.responseText
* feat: titling for assistants endpoint
* chore: move panel file components to appropriate directory, add file checks for attaching files, change icon for Attach Files
* refactor: add localizations to tools, plugins, add condition for adding/remove user plugins so tool selections don't affect this value
* chore: disable `import from url` action for now
* chore: remove textMimeTypes from default fileConfig for now
* fix: catch tool errors and send as outputs with error messages
* fix: React warning about button as descendant of button
* style: retrieval and cancelled icon
* WIP: pass isSubmitting to Parts, use InProgressCall to display cancelled tool calls correctly, show domain/function name
* fix(meilisearch): fix `postSaveHook` issue where indexing expects a mongo document, and join all text content parts for meili indexing
* ci: fix dall-e tests
* ci: fix client tests
* fix: button types in actions panel
* fix: plugin auth form persisting across tool selections
* fix(ci): update AppService spec with `loadAndFormatTools`
* fix(clearConvos): add id check earlier on
* refactor(AssistantAvatar): set previewURL dynamically when emtadata.avatar changes
* feat(assistants): addTitle cache setting
* fix(useSSE): resolve rebase conflicts
* fix: delete mutation
* style(SidePanel): make grip visible on active and hover, invisible otherwise
* ci: add data-provider tests to workflow, also update eslint/tsconfig to recognize specs, and add `text/csv` to fileConfig
* fix: handle edge case where auth object is undefined, and log errors
* refactor(actions): resolve schemas, add tests for resolving refs, import specs from separate file for tests
* chore: remove comment
* fix(ActionsInput): re-render bug when initializing states with action fields
* fix(patch/assistant): filter undefined tools
* chore: add logging for errors in assistants routes
* fix(updateAssistant): map actions to functions to avoid overwriting
* fix(actions): properly handle GET paths
* fix(convos): unhandled delete thread exception
* refactor(AssistantService): pass both thread_id and conversationId when sending intermediate assistant messages, remove `mapMessagesToSteps` from AssistantService
* refactor(useSSE): replace all messages with runMessages and pass latestMessageId to abortRun; fix(checkMessageGaps): include tool calls when syncing messages
* refactor(assistants/chat): invoke `createOnTextProgress` after thread creation
* chore: add typing
* style: sidepanel styling
* style: action tool call domain styling
* feat(assistants): default models, limit retrieval to certain models, add env variables to to env.example
* feat: assistants api key in EndpointService
* refactor: set assistant model to conversation on assistant switch
* refactor: set assistant model to conversation on assistant select from panel
* fix(retrieveAndProcessFile): catch attempt to download file with `assistant` purpose which is not allowed; add logging
* feat: retrieval styling, handling, and logging
* chore: rename ASSISTANTS_REVERSE_PROXY to ASSISTANTS_BASE_URL
* feat: FileContext for file metadata
* feat: context file mgmt and filtering
* style(Select): hover/rounded changes
* refactor: explicit conversation switch, endpoint dependent, through `useSelectAssistant`, which does not create new chat if current endpoint is assistant endpoint
* fix(AssistantAvatar): make empty previewURL if no avatar present
* refactor: side panel mobile styling
* style: merge tool and action section, optimize mobile styling for action/tool buttons
* fix: localStorage issues
* fix(useSelectAssistant): invoke react query hook directly in select hook as Map was not being updated in time
* style: light mode fixes
* fix: prevent sidepanel nav styling from shifting layout up
* refactor: change default layout (collapsed by default)
* style: mobile optimization of DataTable
* style: datatable
* feat: client-side hide right-side panel
* chore(useNewConvo): add partial typing for preset
* fix(useSelectAssistant): pass correct model name by using template as preset
* WIP: assistant presets
* refactor(ToolService): add native solution for `TavilySearchResults` and log tool output errors
* refactor: organize imports and use native TavilySearchResults
* fix(TavilySearchResults): stringify result
* fix(ToolCall): show tool call outputs when not an action
* chore: rename Prompt Prefix to custom instructions (in user facing text only)
* refactor(EditPresetDialog): Optimize setting title by debouncing, reset preset on dialog close to avoid state mixture
* feat: add `presetOverride` to overwrite active conversation settings when saving a Preset (relevant for client side updates only)
* feat: Assistant preset settings (client-side)
* fix(Switcher): only set assistant_id and model if current endpoint is Assistants
* feat: use `useDebouncedInput` for updating conversation settings, starting with EditPresetDialog title setting and Assistant instructions setting
* feat(Assistants): add instructions field to settings
* feat(chat/assistants): pass conversation settings to run body
* wip: begin localization and only allow actions if the assistant is created
* refactor(AssistantsPanel): knowledge localization, allow tools on creation
* feat: experimental: allow 'priming' values before assistant is created, that would normally require an assistant_id to be defined
* chore: trim console logs and make more meaningful
* chore: toast messages
* fix(ci): date test
* feat: create file when uploading Assistant Avatar
* feat: file upload rate limiting from custom config with dynamic file route initialization
* refactor: use file upload limiters on post routes only
* refactor(fileConfig): add endpoints field for endpoint specific fileconfigs, add mergeConfig function, add tests
* refactor: fileConfig route, dynamic multer instances used on all '/' and '/images' POST routes, data service and query hook
* feat: supportedMimeTypesSchema, test for array of regex
* feat: configurable file config limits
* chore: clarify assistants file knowledge prereq.
* chore(useTextarea): default to localized 'Assistant' if assistant name is empty
* feat: configurable file limits and toggle file upload per endpoint
* fix(useUploadFileMutation): prevent updating assistant.files cache if file upload is a message_file attachment
* fix(AssistantSelect): set last selected assistant only when timeout successfully runs
* refactor(queries): disable assistant queries if assistants endpoint is not enabled
* chore(Switcher): add localization
* chore: pluralize `assistant` for `EModelEndpoint key and value
* feat: show/hide assistant UI components based on endpoint availability; librechat.yaml config for disabling builder section and setting polling/timeout intervals
* fix(compactEndpointSchemas): use EModelEndpoint for schema access
* feat(runAssistant): use configured values from `librechat.yaml` for `pollIntervalMs` and `timeout`
* fix: naming issue
* wip: revert landing
* 🎉 happy birthday LibreChat (#1768)
* happy birthday LibreChat
* Refactor endpoint condition in Landing component
* Update birthday message in Eng.tsx
* fix(/config): avoid nesting ternaries
* refactor(/config): check birthday
---------
Co-authored-by: Danny Avila <messagedaniel@protonmail.com>
* fix: landing
* fix: landing
* fix(useMessageHelpers): hardcoded check to use EModelEndpoint instead
* fix(ci): convo test revert to main
* fix(assistants/chat): fix issue where assistant_id was being saved as model for convo
* chore: added logging, promises racing to prevent longer timeouts, explicit setting of maxRetries and timeouts, robust catching of invalid abortRun params
* refactor: use recoil state for `showStopButton` and only show for assistants endpoint after syncing conversation data
* refactor: optimize abortRun strategy using localStorage, refactor `abortConversation` to use async/await and await the result, refactor how the abortKey cache is set for runs
* fix(checkMessageGaps): assign `assistant_id` to synced messages if defined; prevents UI from showing blank assistant for cancelled messages
* refactor: re-order sequence of chat route, only allow aborting messages after run is created, cancel abortRun if there was a cancelling error (likely due already cancelled in chat route), and add extra logging
* chore(typedefs): add httpAgent type to OpenAIClient
* refactor: use custom implementation of retrieving run with axios to allow for timing out run query
* fix(waitForRun): handle timed out run retrieval query
* refactor: update preset conditions:
- presets will retain settings when a different endpoint is selected; for existing convos, either when modular or is assistant switch
- no longer use `navigateToConvo` on preset select
* fix: temporary calculator hack as expects string input when invoked
* fix: cancel abortRun only when cancelling error is a result of the run already being cancelled
* chore: remove use of `fileMaxSizeMB` and total counterpart (redundant)
* docs: custom config documentation update
* docs: assistants api setup and dotenv, new custom config fields
* refactor(Switcher): make Assistant switcher sticky in SidePanel
* chore(useSSE): remove console log of data and message index
* refactor(AssistantPanel): button styling and add secondary select button to bottom of panel
* refactor(OpenAIClient): allow passing conversationId to RunManager through titleConvo and initializeLLM to properly record title context tokens used in cases where conversationId was not defined by the client
* feat(assistants): token tracking for assistant runs
* chore(spendTokens): improve logging
* feat: support/exclude specific assistant Ids
* chore: add update `librechat.example.yaml`, optimize `AppService` handling, new tests for `AppService`, optimize missing/outdate config logging
* chore: mount docker logs to root of project
* chore: condense axios errors
* chore: bump vite
* chore: vite hot reload fix using latest version
* chore(getOpenAIModels): sort instruct models to the end of models list
* fix(assistants): user provided key
* fix(assistants): user provided key, invalidate more queries on revoke
---------
Co-authored-by: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
* chore: fix `endpoint` typescript issues and typo in console info message
* feat(api): files GET endpoint and save only file_id references to messages
* refactor(client): `useGetFiles` query hook, update file types, optimistic update of filesQuery on file upload
* refactor(buildTree): update to use params object and accept fileMap
* feat: map files to messages; refactor(ChatView): messages only available after files are fetched
* fix: fetch files only when authenticated
* feat(api): AppService
- rename app.locals.configs to app.locals.paths
- load custom config use fileStrategy from yaml config in app.locals
* refactor: separate Firebase and Local strategies, call based on config
* refactor: modularize file strategies and employ with use of DALL-E
* refactor(librechat.yaml): add fileStrategy field
* feat: add source to MongoFile schema, as well as BatchFile, and ExtendedFile types
* feat: employ file strategies for upload/delete files
* refactor(deleteFirebaseFile): add user id validation for firebase file deletion
* chore(deleteFirebaseFile): update jsdocs
* feat: employ strategies for vision requests
* fix(client): handle messages with deleted files
* fix(client): ensure `filesToDelete` always saves/sends `file.source`
* feat(openAI): configurable `resendImages` and `imageDetail`
* refactor(getTokenCountForMessage): recursive process only when array of Objects and only their values (not keys) aside from `image_url` types
* feat(OpenAIClient): calculateImageTokenCost
* chore: remove comment
* refactor(uploadAvatar): employ fileStrategy for avatars, from social logins or user upload
* docs: update docs on how to configure fileStrategy
* fix(ci): mock winston and winston related modules, update DALLE3.spec.js with changes made
* refactor(redis): change terminal message to reflect current development state
* fix(DALL-E-2): pass fileStrategy to dall-e
* feat: add timer duration to showToast, show toast for preset selection
* refactor: replace old /chat/ route with /c/. e2e tests will fail here
* refactor: move typedefs to root of /api/ and add a few to assistant types in TS
* refactor: reorganize data-provider imports, fix dependency cycle, strategize new plan to separate react dependent packages
* feat: add dataService for uploading images
* feat(data-provider): add mutation keys
* feat: file resizing and upload
* WIP: initial API image handling
* fix: catch JSON.parse of localStorage tools
* chore: experimental: use module-alias for absolute imports
* refactor: change temp_file_id strategy
* fix: updating files state by using Map and defining react query callbacks in a way that keeps them during component unmount, initial delete handling
* feat: properly handle file deletion
* refactor: unexpose complete filepath and resize from server for higher fidelity
* fix: make sure resized height, width is saved, catch bad requests
* refactor: use absolute imports
* fix: prevent setOptions from being called more than once for OpenAIClient, made note to fix for PluginsClient
* refactor: import supportsFiles and models vars from schemas
* fix: correctly replace temp file id
* refactor(BaseClient): use absolute imports, pass message 'opts' to buildMessages method, count tokens for nested objects/arrays
* feat: add validateVisionModel to determine if model has vision capabilities
* chore(checkBalance): update jsdoc
* feat: formatVisionMessage: change message content format dependent on role and image_urls passed
* refactor: add usage to File schema, make create and updateFile, correctly set and remove TTL
* feat: working vision support
TODO: file size, type, amount validations, making sure they are styled right, and making sure you can add images from the clipboard/dragging
* feat: clipboard support for uploading images
* feat: handle files on drop to screen, refactor top level view code to Presentation component so the useDragHelpers hook has ChatContext
* fix(Images): replace uploaded images in place
* feat: add filepath validation to protect sensitive files
* fix: ensure correct file_ids are push and not the Map key values
* fix(ToastContext): type issue
* feat: add basic file validation
* fix(useDragHelpers): correct context issue with `files` dependency
* refactor: consolidate setErrors logic to setError
* feat: add dialog Image overlay on image click
* fix: close endpoints menu on click
* chore: set detail to auto, make note for configuration
* fix: react warning (button desc. of button)
* refactor: optimize filepath handling, pass file_ids to images for easier re-use
* refactor: optimize image file handling, allow re-using files in regen, pass more file metadata in messages
* feat: lazy loading images including use of upload preview
* fix: SetKeyDialog closing, stopPropagation on Dialog content click
* style(EndpointMenuItem): tighten up the style, fix dark theme showing in lightmode, make menu more ux friendly
* style: change maxheight of all settings textareas to 138px from 300px
* style: better styling for textarea and enclosing buttons
* refactor(PresetItems): swap back edit and delete icons
* feat: make textarea placeholder dynamic to endpoint
* style: show user hover buttons only on hover when message is streaming
* fix: ordered list not going past 9, fix css
* feat: add User/AI labels; style: hide loading spinner
* feat: add back custom footer, change original footer text
* feat: dynamic landing icons based on endpoint
* chore: comment out assistants route
* fix: autoScroll to newest on /c/ view
* fix: Export Conversation on new UI
* style: match message style of official more closely
* ci: fix api jest unit tests, comment out e2e tests for now as they will fail until addressed
* feat: more file validation and use blob in preview field, not filepath, to fix temp deletion
* feat: filefilter for multer
* feat: better AI labels based on custom name, model, and endpoint instead of `ChatGPT`
