7 commits

Author	SHA1	Message	Date
Danny Avila	b93d60c416	🎞️ refactor: Image Rendering with Preview Caching and Layout Reservation (#12114) ... * refactor: Update Image Component to Remove Lazy Loading and Enhance Rendering - Removed the react-lazy-load-image-component dependency from the Image component, simplifying the image loading process. - Updated the Image component to use a standard <img> tag with async decoding for improved performance and user experience. - Adjusted related tests to reflect changes in image rendering behavior and ensure proper functionality without lazy loading. * refactor: Enhance Image Handling and Caching Across Components - Introduced a new previewCache utility for managing local blob preview URLs, improving image loading efficiency. - Updated the Image component and related parts (FileRow, Files, Part, ImageAttachment, LogContent) to utilize cached previews, enhancing rendering performance and user experience. - Added width and height properties to the Image component for better layout management and consistency across different usages. - Improved file handling logic in useFileHandling to cache previews during file uploads, ensuring quick access to image data. - Enhanced overall code clarity and maintainability by streamlining image rendering logic and reducing redundancy. * refactor: Enhance OpenAIImageGen Component with Image Dimensions - Added width and height properties to the OpenAIImageGen component for improved image rendering and layout management. - Updated the Image component usage within OpenAIImageGen to utilize the new dimensions, enhancing visual consistency and performance. - Improved code clarity by destructuring additional properties from the attachment object, streamlining the component's logic. * refactor: Implement Image Size Caching in DialogImage Component - Introduced an imageSizeCache to store and retrieve image sizes, enhancing performance by reducing redundant fetch requests. - Updated the getImageSize function to first check the cache before making network requests, improving efficiency in image handling. - Added decoding attribute to the image element for optimized rendering behavior. * refactor: Enhance UserAvatar Component with Avatar Caching and Error Handling - Introduced avatar caching logic to optimize avatar resolution based on user ID and avatar source, improving performance and reducing redundant image loads. - Implemented error handling for failed image loads, allowing for fallback to a default avatar when necessary. - Updated UserAvatar props to streamline the interface by removing the user object and directly accepting avatar-related properties. - Enhanced overall code clarity and maintainability by refactoring the component structure and logic. * fix: Layout Shift in Message and Placeholder Components for Consistent Height Management - Adjusted the height of the PlaceholderRow and related message components to ensure consistent rendering with a minimum height of 31px. - Updated the MessageParts and ContentRender components to utilize a minimum height for better layout stability. - Enhanced overall code clarity by standardizing the structure of message-related components. * tests: Update FileRow Component to Prefer Cached Previews for Image Rendering - Modified the image URL selection logic in the FileRow component to prioritize cached previews over file paths when uploads are complete, enhancing rendering performance and user experience. - Updated related tests to reflect changes in image URL handling, ensuring accurate assertions for both preview and file path scenarios. - Introduced a fallback mechanism to use file paths when no preview exists, improving robustness in file handling. * fix: Image cache lifecycle and dialog decoding - Add deletePreview/clearPreviewCache to previewCache.ts for blob URL cleanup - Wire deletePreview into useFileDeletion to revoke blobs on file delete - Move dimensionCache.set into useMemo to avoid side effects during render - Extract IMAGE_MAX_W_PX constant (512) to document coupling with max-w-lg - Export _resetImageCaches for test isolation - Change DialogImage decoding from "sync" to "async" to avoid blocking main thread * fix: Avatar cache invalidation and cleanup - Include avatarSrc in cache invalidation to prevent stale avatars - Remove unused username parameter from resolveAvatar - Skip caching when userId is empty to prevent cache key collisions * test: Fix test isolation and type safety - Reset module-level dimensionCache/paintedUrls in beforeEach via _resetImageCaches - Replace any[] with typed mock signature in cn mock for both test files * chore: Code quality improvements from review - Use barrel imports for previewCache in Files.tsx and Part.tsx - Single Map.get with truthy check instead of has+get in useEventHandlers - Add JSDoc comments explaining EmptyText margin removal and PlaceholderRow height - Fix FileRow toast showing "Deleting file" when file isn't actually deleted (progress < 1) * fix: Address remaining review findings (R1-R3) - Add deletePreview calls to deleteFiles batch path to prevent blob URL leaks - Change useFileDeletion import from deep path to barrel (~/utils) - Change useMemo to useEffect for dimensionCache.set (side effect, not derived value) * fix: Address audit comments 2, 5, and 7 - Fix files preservation to distinguish null (missing) from [] (empty) in finalHandler - Add auto-revoke on overwrite in cachePreview to prevent leaked blobs - Add removePreviewEntry for key transfer without revoke - Clean up stale temp_file_id cache entry after promotion to permanent file_id	2026-03-06 19:09:52 -05:00
Carolina	3a73907daa	📐 fix: Replace JS Image Scaling with CSS Viewport Constraints (#12089) ... * fix: remove scaleImage function that stretched vertical images * chore: lint * refactor: Simplify Image Component Usage Across Chat Parts - Removed height and width props from the Image component in various parts (Files, Part, ImageAttachment, LogContent) to streamline image rendering. - Introduced a constant for maximum image height in the Image component for consistent styling. - Updated related components to utilize the new simplified Image component structure, enhancing maintainability and reducing redundancy. * refactor: Simplify LogContent and Enhance Image Component Tests - Removed height and width properties from the ImageAttachment type in LogContent for cleaner code. - Updated the image rendering logic to rely solely on the filepath, improving clarity. - Enhanced the Image component tests with additional assertions for rendering behavior and accessibility. - Introduced new tests for OpenAIImageGen to validate image preloading and progress handling, ensuring robust functionality. --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-03-06 16:42:23 -05:00
Danny Avila	75c02a1a18	🗂️ feat: Better Persistence for Code Execution Files Between Sessions (#11362) ... * refactor: process code output files for re-use (WIP) * feat: file attachment handling with additional metadata for downloads * refactor: Update directory path logic for local file saving based on basePath * refactor: file attachment handling to support TFile type and improve data merging logic * feat: thread filtering of code-generated files - Introduced parentMessageId parameter in addedConvo and initialize functions to enhance thread management. - Updated related methods to utilize parentMessageId for retrieving messages and filtering code-generated files by conversation threads. - Enhanced type definitions to include parentMessageId in relevant interfaces for better clarity and usage. * chore: imports/params ordering * feat: update file model to use messageId for filtering and processing - Changed references from 'message' to 'messageId' in file-related methods for consistency. - Added messageId field to the file schema and updated related types. - Enhanced file processing logic to accommodate the new messageId structure. * feat: enhance file retrieval methods to support user-uploaded execute_code files - Added a new method `getUserCodeFiles` to retrieve user-uploaded execute_code files, excluding code-generated files. - Updated existing file retrieval methods to improve filtering logic and handle edge cases. - Enhanced thread data extraction to collect both message IDs and file IDs efficiently. - Integrated `getUserCodeFiles` into relevant endpoints for better file management in conversations. * chore: update @librechat/agents package version to 3.0.78 in package-lock.json and related package.json files * refactor: file processing and retrieval logic - Added a fallback mechanism for download URLs when files exceed size limits or cannot be processed locally. - Implemented a deduplication strategy for code-generated files based on conversationId and filename to optimize storage. - Updated file retrieval methods to ensure proper filtering by messageIds, preventing orphaned files from being included. - Introduced comprehensive tests for new thread data extraction functionality, covering edge cases and performance considerations. * fix: improve file retrieval tests and handling of optional properties - Updated tests to safely access optional properties using non-null assertions. - Modified test descriptions for clarity regarding the exclusion of execute_code files. - Ensured that the retrieval logic correctly reflects the expected outcomes for file queries. * test: add comprehensive unit tests for processCodeOutput functionality - Introduced a new test suite for the processCodeOutput function, covering various scenarios including file retrieval, creation, and processing for both image and non-image files. - Implemented mocks for dependencies such as axios, logger, and file models to isolate tests and ensure reliable outcomes. - Validated behavior for existing files, new file creation, and error handling, including size limits and fallback mechanisms. - Enhanced test coverage for metadata handling and usage increment logic, ensuring robust verification of file processing outcomes. * test: enhance file size limit enforcement in processCodeOutput tests - Introduced a configurable file size limit for tests to improve flexibility and coverage. - Mocked the `librechat-data-provider` to allow dynamic adjustment of file size limits during tests. - Updated the file size limit enforcement test to validate behavior when files exceed specified limits, ensuring proper fallback to download URLs. - Reset file size limit after tests to maintain isolation for subsequent test cases.	2026-01-28 17:44:32 -05:00
Danny Avila	bcd97aad2f	📎 feat: Direct Provider Attachment Support for Multimodal Content (#9994) ... * 📎 feat: Direct Provider Attachment Support for Multimodal Content * 📑 feat: Anthropic Direct Provider Upload (#9072) * feat: implement Anthropic native PDF support with document preservation - Add comprehensive debug logging throughout PDF processing pipeline - Refactor attachment processing to separate image and document handling - Create distinct addImageURLs(), addDocuments(), and processAttachments() methods - Fix critical bugs in stream handling and parameter passing - Add streamToBuffer utility for proper stream-to-buffer conversion - Remove api/agents submodule from repository 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * chore: remove out of scope formatting changes * fix: stop duplication of file in chat on end of response stream * chore: bring back file search and ocr options * chore: localize upload to provider string in file menu * refactor: change createMenuItems args to fit new pattern introduced by anthropic-native-pdf-support * feat: add cache point for pdfs processed by anthropic endpoint since they are unlikely to change and should benefit from caching * feat: combine Upload Image into Upload to Provider since they both perform direct upload and change provider upload icon to reflect multimodal upload * feat: add citations support according to docs * refactor: remove redundant 'document' check since documents are handled properly by formatMessage in the agents repo now * refactor: change upload logic so anthropic endpoint isn't exempted from normal upload path using Agents for consistency with the rest of the upload logic * fix: include width and height in return from uploadLocalFile so images are correctly identified when going through an AgentUpload in addImageURLs * chore: remove client specific handling since the direct provider stuff is handled by the agent client * feat: handle documents in AgentClient so no need for change to agents repo * chore: removed unused changes * chore: remove auto generated comments from OG commit * feat: add logic for agents to use direct to provider uploads if supported (currently just anthropic) * fix: reintroduce role check to fix render error because of undefined value for Content Part * fix: actually fix render bug by using proper isCreatedByUser check and making sure our mutation of formattedMessage.content is consistent --------- Co-authored-by: Andres Restrepo <andres@thelinuxkid.com> Co-authored-by: Claude <noreply@anthropic.com> 📁 feat: Send Attachments Directly to Provider (OpenAI) (#9098) * refactor: change references from direct upload to direct attach to better reflect functionality since we are just using base64 encoding strategy now rather than Files/File API for sending our attachments directly to the provider, the upload nomenclature no longer makes sense. direct_attach better describes the different methods of sending attachments to providers anyways even if we later introduce direct upload support * feat: add upload to provider option for openai (and agent) ui * chore: move anthropic pdf validator over to packages/api * feat: simple pdf validation according to openai docs * feat: add provider agnostic validatePdf logic to start handling multiple endpoints * feat: add handling for openai specific documentPart formatting * refactor: move require statement to proper place at top of file * chore: add in openAI endpoint for the rest of the document handling logic * feat: add direct attach support for azureOpenAI endpoint and agents * feat: add pdf validation for azureOpenAI endpoint * refactor: unify all the endpoint checks with isDocumentSupportedEndpoint * refactor: consolidate Upload to Provider vs Upload image logic for clarity * refactor: remove anthropic from anthropic_multimodal fileType since we support multiple providers now 🗂️ feat: Send Attachments Directly to Provider (Google) (#9100) * feat: add validation for google PDFs and add google endpoint as a document supporting endpoint * feat: add proper pdf formatting for google endpoints (requires PR #14 in agents) * feat: add multimodal support for google endpoint attachments * feat: add audio file svg * fix: refactor attachments logic so multi-attachment messages work properly * feat: add video file svg * fix: allows for followup questions of uploaded multimodal attachments * fix: remove incorrect final message filtering that was breaking Attachment component rendering fix: manualy rename 'documents' to 'Documents' in git since it wasn't picked up due to case insensitivity in dir name fix: add logic so filepicker for a google agent has proper filetype filtering 🛫 refactor: Move Encoding Logic to packages/api (#9182) * refactor: move audio encode over to TS * refactor: audio encoding now functional in LC again * refactor: move video encode over to TS * refactor: move document encode over to TS * refactor: video encoding now functional in LC again * refactor: document encoding now functional in LC again * fix: extend file type options in AttachFileMenu to include 'google_multimodal' and update dependency array to include agent?.provider * feat: only accept pdfs if responses api is enabled for openai convos chore: address ESLint comments chore: add missing audio mimetype * fix: type safety for message content parts and improve null handling * chore: reorder AttachFileMenuProps for consistency and clarity * chore: import order in AttachFileMenu * fix: improve null handling for text parts in parseTextParts function * fix: remove no longer used unsupported capability error message for file uploads * fix: OpenAI Direct File Attachment Format * fix: update encodeAndFormatDocuments to support OpenAI responses API and enhance document result types * refactor: broaden providers supported for documents * feat: enhance DragDrop context and modal to support document uploads based on provider capabilities * fix: reorder import statements for consistency in video encoding module --------- Co-authored-by: Dustin Healy <54083382+dustinhealy@users.noreply.github.com>	2025-10-06 17:30:16 -04:00
Ruben Talstra	aae413cc71	🌎 i18n: React-i18next & i18next Integration (#5720) ... * better i18n support an internationalization-framework. * removed unused package * auto sort for translation.json * fixed tests with the new locales function * added new CI actions from locize * to use locize a mention in the README.md * to use locize a mention in the README.md * updated README.md and added TRANSLATION.md to the repo * updated TRANSLATION.md badges * updated README.md to go to the TRANSLATION.md when clicking on the Translation Progress badge * updated TRANSLATION.md and added a new issue template. * updated TRANSLATION.md and added a new issue template. * updated issue template to add the iso code link. * updated the new GitHub actions for `locize` * updated label for new issue template --> i18n * fixed type issue * Fix eslint * Fix eslint with key-spacing spacing * fix: error type * fix: handle undefined values in SortFilterHeader component * fix: typing in Image component * fix: handle optional promptGroup in PromptCard component * fix: update localize function to accept string type and remove unnecessary JSX element * fix: update localize function to enforce TranslationKeys type for better type safety * fix: improve type safety and handle null values in Assistants component * fix: enhance null checks for fileId in FilesListView component * fix: localize 'Go back' button text in FilesListView component * fix: update aria-label for menu buttons and add translation for 'Close Menu' * docs: add Reasoning UI section for Chain-of-Thought AI models in README * fix: enhance type safety by adding type for message in MultiMessage component * fix: improve null checks and optional chaining in useAutoSave hook * fix: improve handling of optional properties in cleanupPreset function * fix: ensure isFetchingNextPage defaults to false and improve null checks for messages in Search component * fix: enhance type safety and null checks in useBuildMessageTree hook --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2025-02-09 12:05:31 -05:00
Danny Avila	1a815f5e19	🎉 feat: Code Interpreter API and Agents Release (#4860) ... * feat: Code Interpreter API & File Search Agent Uploads chore: add back code files wip: first pass, abstract key dialog refactor: influence checkbox on key changes refactor: update localization keys for 'execute code' to 'run code' wip: run code button refactor: add throwError parameter to loadAuthValues and getUserPluginAuthValue functions feat: first pass, API tool calling fix: handle missing toolId in callTool function and return 404 for non-existent tools feat: show code outputs fix: improve error handling in callTool function and log errors fix: handle potential null value for filepath in attachment destructuring fix: normalize language before rendering and prevent null return fix: add loading indicator in RunCode component while executing code feat: add support for conditional code execution in Markdown components feat: attachments refactor: remove bash fix: pass abort signal to graph/run refactor: debounce and rate limit tool call refactor: increase debounce delay for execute function feat: set code output attachments feat: image attachments refactor: apply message context refactor: pass `partIndex` feat: toolCall schema/model/methods feat: block indexing feat: get tool calls chore: imports chore: typing chore: condense type imports feat: get tool calls fix: block indexing chore: typing refactor: update tool calls mapping to support multiple results fix: add unique key to nav link for rendering wip: first pass, tool call results refactor: update query cache from successful tool call mutation style: improve result switcher styling chore: note on using \`.toObject()\` feat: add agent_id field to conversation schema chore: typing refactor: rename agentMap to agentsMap for consistency feat: Agent Name as chat input placeholder chore: bump agents 📦 chore: update @langchain dependencies to latest versions to match agents package 📦 chore: update @librechat/agents dependency to version 1.8.0 fix: Aborting agent stream removes sender; fix(bedrock): completion removes preset name label refactor: remove direct file parameter to use req.file, add `processAgentFileUpload` for image uploads feat: upload menu feat: prime message_file resources feat: implement conversation access validation in chat route refactor: remove file parameter from processFileUpload and use req.file instead feat: add savedMessageIds set to track saved message IDs in BaseClient, to prevent unnecessary double-write to db feat: prevent duplicate message saves by checking savedMessageIds in AgentController refactor: skip legacy RAG API handling for agents feat: add files field to convoSchema refactor: update request type annotations from Express.Request to ServerRequest in file processing functions feat: track conversation files fix: resendFiles, addPreviousAttachments handling feat: add ID validation for session_id and file_id in download route feat: entity_id for code file uploads/downloads fix: code file edge cases feat: delete related tool calls feat: add stream rate handling for LLM configuration feat: enhance system content with attached file information fix: improve error logging in resource priming function * WIP: PoC, sequential agents WIP: PoC Sequential Agents, first pass content data + bump agents package fix: package-lock WIP: PoC, o1 support, refactor bufferString feat: convertJsonSchemaToZod fix: form issues and schema defining erroneous model fix: max length issue on agent form instructions, limit conversation messages to sequential agents feat: add abort signal support to createRun function and AgentClient feat: PoC, hide prior sequential agent steps fix: update parameter naming from config to metadata in event handlers for clarity, add model to usage data refactor: use only last contentData, track model for usage data chore: bump agents package fix: content parts issue refactor: filter contentParts to include tool calls and relevant indices feat: show function calls refactor: filter context messages to exclude tool calls when no tools are available to the agent fix: ensure tool call content is not undefined in formatMessages feat: add agent_id field to conversationPreset schema feat: hide sequential agents feat: increase upload toast duration to 10 seconds * refactor: tool context handling & update Code API Key Dialog feat: toolContextMap chore: skipSpecs -> useSpecs ci: fix handleTools tests feat: API Key Dialog * feat: Agent Permissions Admin Controls feat: replace label with button for prompt permission toggle feat: update agent permissions feat: enable experimental agents and streamline capability configuration feat: implement access control for agents and enhance endpoint menu items feat: add welcome message for agent selection in localization feat: add agents permission to access control and update version to 0.7.57 * fix: update types in useAssistantListMap and useMentions hooks for better null handling * feat: mention agents * fix: agent tool resource race conditions when deleting agent tool resource files * feat: add error handling for code execution with user feedback * refactor: rename AdminControls to AdminSettings for clarity * style: add gap to button in AdminSettings for improved layout * refactor: separate agent query hooks and check access to enable fetching * fix: remove unused provider from agent initialization options, creates issue with custom endpoints * refactor: remove redundant/deprecated modelOptions from AgentClient processes * chore: update @librechat/agents to version 1.8.5 in package.json and package-lock.json * fix: minor styling issues + agent panel uniformity * fix: agent edge cases when set endpoint is no longer defined * refactor: remove unused cleanup function call from AppService * fix: update link in ApiKeyDialog to point to pricing page * fix: improve type handling and layout calculations in SidePanel component * fix: add missing localization string for agent selection in SidePanel * chore: form styling and localizations for upload filesearch/code interpreter * fix: model selection placeholder logic in AgentConfig component * style: agent capabilities * fix: add localization for provider selection and improve dropdown styling in ModelPanel * refactor: use gpt-4o-mini > gpt-3.5-turbo * fix: agents configuration for loadDefaultInterface and update related tests * feat: DALLE Agents support	2024-12-04 15:48:13 -05:00
Danny Avila	ad74350036	🚧 chore: merge latest dev build (#4288) ... * fix: agent initialization, add `collectedUsage` handling * style: improve side panel styling * refactor(loadAgent): Optimize order agent project ID retrieval * feat: code execution * fix: typing issues * feat: ExecuteCode content part * refactor: use local state for default collapsed state of analysis content parts * fix: code parsing in ExecuteCode component * chore: bump agents package, export loadAuthValues * refactor: Update handleTools.js to use EnvVar for code execution tool authentication * WIP * feat: download code outputs * fix(useEventHandlers): type issues * feat: backend handling for code outputs * Refactor: Remove console.log statement in Part.tsx * refactor: add attachments to TMessage/messageSchema * WIP: prelim handling for code outputs * feat: attachments rendering * refactor: improve attachments rendering * fix: attachments, nullish edge case, handle attachments from event stream, bump agents package * fix filename download * fix: tool assignment for 'run code' on agent creation * fix: image handling by adding attachments * refactor: prevent agent creation without provider/model * refactor: remove unnecessary space in agent creation success message * refactor: select first model if selecting provider from empty on form * fix: Agent avatar bug * fix: `defaultAgentFormValues` causing boolean typing issue and typeerror * fix: capabilities counting as tools, causing duplication of them * fix: formatted messages edge case where consecutive content text type parts with the latter having tool_call_ids would cause consecutive AI messages to be created. furthermore, content could not be an array for tool_use messages (anthropic limitation) * chore: bump @librechat/agents dependency to version 1.6.9 * feat: bedrock agents * feat: new Agents icon * feat: agent titling * feat: agent landing * refactor: allow sharing agent globally only if user is admin or author * feat: initial AgentPanelSkeleton * feat: AgentPanelSkeleton * feat: collaborative agents * chore: add potential authorName as part of schema * chore: Remove unnecessary console.log statement * WIP: agent model parameters * chore: ToolsDialog typing and tool related localization chnages * refactor: update tool instance type (latest langchain class), and rename google tool to 'google' proper * chore: add back tools * feat: Agent knowledge files upload * refactor: better verbiage for disabled knowledge * chore: debug logs for file deletions * chore: debug logs for file deletions * feat: upload/delete agent knowledge/file-search files * feat: file search UI for agents * feat: first pass, file search tool * chore: update default agent capabilities and info	2024-09-30 17:17:57 -04:00