* 👁️ feat: Add Azure Mistral OCR strategy and endpoint integration
This commit introduces a new OCR strategy named 'azure_mistral_ocr', allowing the use of a Mistral OCR endpoint deployed on Azure. The configuration, schemas, and file upload strategies have been updated to support this integration, enabling seamless OCR processing via Azure-hosted Mistral services.
* 🗑️ chore: Clean up .gitignore by removing commented-out uncommon directory name
* chore: remove unused vars
* refactor: Move createAxiosInstance to packages/api/utils and update imports
- Removed the createAxiosInstance function from the config module and relocated it to a new utils module for better organization.
- Updated import paths in relevant files to reflect the new location of createAxiosInstance.
- Added tests for createAxiosInstance to ensure proper functionality and proxy configuration handling.
* chore: move axios helpers to packages/api
- Added logAxiosError function to @librechat/api for centralized error logging.
- Updated imports across various files to use the new logAxiosError function.
- Removed the old axios.js utility file as it is no longer needed.
* chore: Update Jest moduleNameMapper for improved path resolution
- Added a new mapping for '~/' to resolve module paths in Jest configuration, enhancing import handling for the project.
* feat: Implement Mistral OCR API integration in TS
* chore: Update MistralOCR tests based on new imports
* fix: Enhance MistralOCR configuration handling and tests
- Introduced helper functions for resolving configuration values from environment variables or hardcoded settings.
- Updated the uploadMistralOCR and uploadAzureMistralOCR functions to utilize the new configuration resolution logic.
- Improved test cases to ensure correct behavior when mixing environment variables and hardcoded values.
- Mocked file upload and signed URL responses in tests to validate functionality without external dependencies.
* feat: Enhance MistralOCR functionality with improved configuration and error handling
- Introduced helper functions for loading authentication configuration and resolving values from environment variables.
- Updated uploadMistralOCR and uploadAzureMistralOCR functions to utilize the new configuration logic.
- Added utility functions for processing OCR results and creating error messages.
- Improved document type determination and result aggregation for better OCR processing.
* refactor: Reorganize OCR type imports in Mistral CRUD file
- Moved OCRResult, OCRResultPage, and OCRImage imports to a more logical grouping for better readability and maintainability.
* feat: Add file exports to API and create files index
* chore: Update OCR types for enhanced structure and clarity
- Redesigned OCRImage interface to include mandatory fields and improved naming conventions.
- Added PageDimensions interface for better representation of page metrics.
- Updated OCRResultPage to include dimensions and mandatory images array.
- Refined OCRResult to include document annotation and usage information.
* refactor: use TS counterpart of uploadOCR methods
* ci: Update MistralOCR tests to reflect new OCR result structure
* chore: Bump version of @librechat/api to 1.2.3 in package.json and package-lock.json
* chore: Update CONFIG_VERSION to 1.2.8
* chore: remove unused sendEvent function from config module (now imported from '@librechat/api')
* chore: remove MistralOCR service files and tests (now in '@librechat/api')
* ci: update logger import in ModelService tests to use @librechat/data-schemas
---------
Co-authored-by: arthurolivierfortin <arthurolivier.fortin@gmail.com>
* refactor: move `loadAuthValues` to `~/services/Tools/credentials`
* feat: add createAxiosInstance function to configure axios with proxy support
* WIP: First pass mistral ocr
* refactor: replace getConvoFiles with getToolFiles for improved file retrieval logic
* refactor: improve document formatting in encodeAndFormat function
* refactor: remove unused resendFiles parameter from buildOptions function (this option comes from the agent config)
* fix: update getFiles call to include files with `text` property as well
* refactor: move file handling to `initializeAgentOptions`
* refactor: enhance addImageURLs method to handle OCR text and improve message formatting
* refactor: update message formatting to handle OCR text in various content types
* refactor: remove unused resendFiles property from compactAgentsSchema
* fix: add error handling for Mistral OCR document upload and logging
* refactor: integrate OCR capability into file upload options and configuration
* refactor: skip processing for text source files in delete request, as they are directly tied to database
* feat: add metadata field to ExtendedFile type and update PanelColumns and PanelTable components for localization and metadata handling
* fix: source icon styling
* wip: first pass, frontend file context agent resources
* refactor: add hover card with contextual information for File Context (OCR) in FileContext component
* feat: enhance file processing by integrating file retrieval for OCR resources in agent initialization
* feat: implement OCR config; fix: agent resource deletion for ocr files
* feat: enhance agent initialization by adding OCR capability check in resource priming
* ci: fix `~/config` module mock
* ci: add OCR property expectation in AppService tests
* refactor: simplify OCR config loading by removing environment variable extraction, to be done when OCR is actually performed
* ci: add unit test to ensure environment variable references are not parsed in OCR config
* refactor: disable base64 image inclusion in OCR request
* refactor: enhance OCR configuration handling by validating environment variables and providing defaults
* refactor: use file stream from disk for mistral ocr api
* chore: fix `endpoint` typescript issues and typo in console info message
* feat(api): files GET endpoint and save only file_id references to messages
* refactor(client): `useGetFiles` query hook, update file types, optimistic update of filesQuery on file upload
* refactor(buildTree): update to use params object and accept fileMap
* feat: map files to messages; refactor(ChatView): messages only available after files are fetched
* fix: fetch files only when authenticated
* feat(api): AppService
- rename app.locals.configs to app.locals.paths
- load custom config use fileStrategy from yaml config in app.locals
* refactor: separate Firebase and Local strategies, call based on config
* refactor: modularize file strategies and employ with use of DALL-E
* refactor(librechat.yaml): add fileStrategy field
* feat: add source to MongoFile schema, as well as BatchFile, and ExtendedFile types
* feat: employ file strategies for upload/delete files
* refactor(deleteFirebaseFile): add user id validation for firebase file deletion
* chore(deleteFirebaseFile): update jsdocs
* feat: employ strategies for vision requests
* fix(client): handle messages with deleted files
* fix(client): ensure `filesToDelete` always saves/sends `file.source`
* feat(openAI): configurable `resendImages` and `imageDetail`
* refactor(getTokenCountForMessage): recursive process only when array of Objects and only their values (not keys) aside from `image_url` types
* feat(OpenAIClient): calculateImageTokenCost
* chore: remove comment
* refactor(uploadAvatar): employ fileStrategy for avatars, from social logins or user upload
* docs: update docs on how to configure fileStrategy
* fix(ci): mock winston and winston related modules, update DALLE3.spec.js with changes made
* refactor(redis): change terminal message to reflect current development state
* fix(DALL-E-2): pass fileStrategy to dall-e
* WIP: initial logging changes
add several transports in ~/config/winston
omit messages in logs, truncate long strings
add short blurb in dotenv for debug logging
GoogleClient: using logger
OpenAIClient: using logger, handleOpenAIErrors
Adding typedef for payload message
bumped winston and using winston-daily-rotate-file
moved config for server paths to ~/config dir
Added `DEBUG_LOGGING=true` to .env.example
* WIP: Refactor logging statements in code
* WIP: Refactor logging statements and import configurations
* WIP: Refactor logging statements and import configurations
* refactor: broadcast Redis initialization message with `info` not `debug`
* refactor: complete Refactor logging statements and import configurations
* chore: delete unused tools
* fix: circular dependencies due to accessing logger
* refactor(handleText): handle booleans and write tests
* refactor: redact sensitive values, better formatting
* chore: improve log formatting, avoid passing strings to 2nd arg
* fix(ci): fix jest tests due to logger changes
* refactor(getAvailablePluginsController): cache plugins as they are static and avoids async addOpenAPISpecs call every time
* chore: update docs
* chore: update docs
* chore: create separate meiliSync logger, clean up logs to avoid being unnecessarily verbose
* chore: spread objects where they are commonly logged to allow string truncation
* chore: improve error log formatting
