* Added fully functioning Agent Tool supporting Google's Nano Banana
* 🔧 refactor: Update Google credentials handling in GeminiImageGen.js
* Refactored the credentials path to follow a consistent pattern with other Google service integrations, allowing for an environment variable override.
* Updated documentation in README-GeminiNanoBanana.md to reflect the new credentials handling approach and removed references to hardcoded paths.
* 🛠️ refactor: Remove unnecessary whitespace in handleTools.js
* 🔧 feat: Update Gemini Image Generation Tool
- Bump @google/genai package version to ^1.19.0 for improved functionality.
- Refactor GeminiImageGen to createGeminiImageTool for better clarity and consistency.
- Enhance manifest.json for Gemini Image Tools with updated descriptions and icon.
- Add SVG icon for Gemini Image Tools.
- Implement progress tracking for Gemini image generation in the UI.
- Introduce new toolkit and context handling for image generation tools.
This update improves the Gemini image generation capabilities and user experience.
* 🗑️ chore: Remove outdated Gemini image generation PNG and update SVG icon
- Deleted the obsolete PNG file for Gemini image generation.
- Updated the SVG icon with a new design featuring a gradient and shadow effect, enhancing visual appeal and consistency.
* fix: ESLint formatting and unused variable in GeminiImageGen
* fix: Update default model to gemini-2.5-flash-image
* ✨ feat: Enhance Gemini Image Generation Configuration
- Updated .env.example to include new environment variables for Google Cloud region, service account configuration, and Gemini API key options.
- Modified GeminiImageGen.js to support both user-provided API keys and Vertex AI service accounts, improving flexibility in client initialization.
- Updated manifest.json to reflect changes in authentication methods for the Gemini Image Tools.
- Bumped @google/genai package version to 1.19.0 in package-lock.json for compatibility with new features.
* 🔧 fix: Format Default Service Key Path in GeminiImageGen.js
- Adjusted the return statement in getDefaultServiceKeyPath function for improved readability by formatting it across multiple lines. This change enhances code clarity without altering functionality.
* ✨ feat: Enhance Gemini Image Generation with Token Usage Tracking
- Added `recordTokenUsage` function to track token usage for balance management.
- Integrated token recording into the image generation process.
- Updated Gemini image generation tool to accept optional `aspectRatio` and `imageSize` parameters for improved image customization.
- Updated token values for new Gemini models in the transaction model.
- Improved documentation for image generation tool descriptions and parameters.
* ✨ feat: Add new Gemini models for image generation token limits
- Introduced token limits for 'gemini-3-pro-image' and 'gemini-2.5-flash-image' models.
- Updated token values to enhance the Gemini image generation capabilities.
* 🔧 fix: Update Google Service Key Path for Consistency in Initialization (#11001)
* 🔧 refactor: Update GeminiImageGen for improved file handling and path resolution
- Changed the default service key path to use process.cwd() for better compatibility.
- Replaced synchronous file system operations with asynchronous promises for mkdir and writeFile, enhancing performance and error handling.
- Added error handling for credential file access to prevent crashes when the file does not exist.
* 🔧 refactor: Update GeminiImageGen to streamline API key handling
- Refactored API key checks to improve clarity and consistency.
- Removed redundant checks for user-provided keys, enhancing code readability.
- Ensured proper logging for API key usage across different configurations.
* 🔧 fix: Update GeminiImageGen to handle imageSize support conditionally
- Added a check to ensure imageSize is only applied if the gemini model does not include 'gemini-2.5-flash-image', improving compatibility.
- Enhanced the logic for setting imageConfig to prevent potential issues with unsupported configurations.
* 🔧 refactor: Simplify local storage condition in createGeminiImageTool function
* 🔧 feat: Enhance image format handling in GeminiImageGen with conversion support
* 🔧 refactor: Streamline API key initialization in GeminiImageGen
- Simplified the handling of API keys by removing redundant checks for user-provided keys.
- Updated logging to reflect the new priority order for API key usage, enhancing clarity and consistency.
- Improved code readability by consolidating key retrieval logic.
---------
Co-authored-by: Dev Bhanushali <dev.bhanushali@hingehealth.com>
Co-authored-by: Danny Avila <danny@librechat.ai>
* 🔧 refactor: Update z-index values for popover components
- Reduced z-index from 50 to 40 across various popover components including Artifacts, ArtifactsSubMenu, MCPSubMenu, CustomMenu, and others to ensure consistent layering and improve UI behavior.
- Adjusted related CSS styles in Dropdown.css and DropdownMenu.tsx to align with the new z-index values, enhancing overall component visibility and interaction.
* chore: remove string template for className concatenation in CustomMenu component
- Improved the readability of the className prop in the CustomMenu component by restructuring the concatenation of class names. This change enhances maintainability and clarity in the styling logic.
* refactor: Simplify button visibility logic in SiblingHeader component
- Updated the button rendering logic in the SiblingHeader component to improve clarity and maintainability. The button is now always rendered, with its visibility controlled by the disabled state based on messageId, agentId, and submission status, enhancing user experience during interactions.
* refactor: Update shift key handling in Conversation and ConvoOptions components
- Modified the handling of the `isShiftHeld` state in both the Conversation and ConvoOptions components to improve clarity and functionality. The logic now ensures that the shift key state is accurately reflected based on the active conversation status, enhancing user interaction during conversations.
- Cleaned up imports in ConvoOptions by removing the unused `useShiftKey` hook, streamlining the component's dependencies.
* refactor: Improve Escape key handling in OriginalDialog component
- Updated the Escape key handling logic to prevent closing the dialog when a tooltip or dropdown menu has focus. This change enhances accessibility by ensuring compliance with WCAG standards for dismissable tooltips.
- Simplified the focus checking mechanism by directly assessing the active element within dropdown menus and tooltips, improving code clarity and maintainability.
* chore: imports
* refactor: Enhance Escape key handling in OriginalDialog component
- Updated the Escape key handling logic to prevent closing the dialog when a trigger with an open popover is focused. This change improves accessibility and user experience by ensuring that the dialog remains open during interactions with popovers, dropdowns, and listboxes.
- Simplified the focus checking mechanism to include additional roles, enhancing the clarity and maintainability of the code.
* refactor: Add dropdownClassName prop to FilterPrompts component
- Enhanced the FilterPrompts component by introducing a new dropdownClassName prop, allowing for customizable styling of the dropdown element.
- Updated the PromptsView component to utilize the new prop, improving the flexibility of the FilterPrompts integration within the UI.
* refactor: Clean up imports and remove unused code in DashBreadcrumb component
- Streamlined the DashBreadcrumb component by removing commented-out imports and unused code, enhancing clarity and maintainability.
- Adjusted the import order for better organization and readability.
* refactor: Update z-index handling in Dropdown component
- Removed the z-index property from Dropdown.css to streamline styling.
- Adjusted the className in Dropdown.tsx to include a new z-40 class for consistent z-index management, enhancing UI layering and interaction.
* refactor: Enhance file type acceptance in AttachFileMenu and useDragHelpers
- Updated the AttachFileMenu component to accept additional image formats (.heif, .heic) alongside existing types, improving file upload flexibility.
- Modified the useDragHelpers hook to utilize inferMimeType for better file type detection, ensuring accurate handling of dragged files.
* refactor: Enhance FloatingThinkingBar with copy functionality
- Added a copy button to the FloatingThinkingBar component, allowing users to copy thoughts to the clipboard.
- Updated the tooltip descriptions for the expand/collapse and copy actions to improve user experience.
- Cleaned up imports and adjusted prop types for better clarity and maintainability.
* refactor: Enhance RunCode component with icon-only mode
- Updated the RunCode component to accept an `iconOnly` prop, allowing for a simplified button display that shows only the icon when desired.
- Adjusted the button rendering logic to improve user experience and maintainability.
- Cleaned up imports and ensured consistent styling in the FloatingCodeBar component.
This commit fixes the messages view height auto-expansion issue introduced
in PR #11142 and improves the UX of both Thinking and CodeBlock components.
## Bug Fix
The ThinkingFooter component added in PR #11142 was causing the messages
view height to automatically expand. The footer was placed inside the
CSS grid's overflow-hidden container, but its presence affected the
grid height calculation, causing layout issues during streaming.
## Solution: FloatingThinkingBar
Replaced ThinkingFooter with a new FloatingThinkingBar component that:
- Uses absolute positioning (bottom-right) like CodeBlock's FloatingCodeBar
- Only appears on hover/focus, not affecting layout
- Shows expand/collapse button with dynamic icon based on state
- Uses TooltipAnchor for accessible tooltips
- Supports keyboard navigation by showing when any element in the
container is focused (top bar buttons or content)
## Changes
### Thinking.tsx
- Added FloatingThinkingBar component with hover/focus visibility
- Updated ThinkingContent with additional bottom padding (pb-10)
- Added containerRef and hover/focus event handlers on outer container
- Removed ThinkingFooter component (replaced by FloatingThinkingBar)
### Reasoning.tsx
- Integrated FloatingThinkingBar with same hover/focus pattern
- Added containerRef and event handlers on outer container
- Supports keyboard navigation through entire component
### CodeBlock.tsx
- Updated FloatingCodeBar to show icons only (removed text labels)
- Added TooltipAnchor wrapper for copy button with localized tooltips
- Improved accessibility with proper aria-label and aria-hidden
### Localization
- Added com_ui_expand_thoughts: "Expand Thoughts"
## Accessibility
- Full keyboard navigation support: tabbing through ThinkingButton,
copy button, and FloatingThinkingBar
- TooltipAnchor provides hover tooltips for icon-only buttons
- Proper aria-label attributes on all interactive elements
- tabIndex management based on visibility state
* 🌊 feat: Implement multi-conversation feature with added conversation context and payload adjustments
* refactor: Replace isSubmittingFamily with isSubmitting across message components for consistency
* feat: Add loadAddedAgent and processAddedConvo for multi-conversation agent execution
* refactor: Update ContentRender usage to conditionally render PlaceholderRow based on isLast and isSubmitting
* WIP: first pass, sibling index
* feat: Enhance multi-conversation support with agent tracking and display improvements
* refactor: Introduce isEphemeralAgentId utility and update related logic for agent handling
* refactor: Implement createDualMessageContent utility for sibling message display and enhance useStepHandler for added conversations
* refactor: duplicate tools for added agent if ephemeral and primary agent is also ephemeral
* chore: remove deprecated multimessage rendering
* refactor: enhance dual message content creation and agent handling for parallel rendering
* refactor: streamline message rendering and submission handling by removing unused state and optimizing conditional logic
* refactor: adjust content handling in parallel mode to utilize existing content for improved agent display
* refactor: update @librechat/agents dependency to version 3.0.53
* refactor: update @langchain/core and @librechat/agents dependencies to latest versions
* refactor: remove deprecated @langchain/core dependency from package.json
* chore: remove unused SearchToolConfig and GetSourcesParams types from web.ts
* refactor: remove unused message properties from Message component
* refactor: enhance parallel content handling with groupId support in ContentParts and useStepHandler
* refactor: implement parallel content styling in Message, MessageRender, and ContentRender components. use explicit model name
* refactor: improve agent ID handling in createDualMessageContent for dual message display
* refactor: simplify title generation in AddedConvo by removing unused sender and preset logic
* refactor: replace string interpolation with cn utility for className in HoverButtons component
* refactor: enhance agent ID handling by adding suffix management for parallel agents and updating related components
* refactor: enhance column ordering in ContentParts by sorting agents with suffix management
* refactor: update @librechat/agents dependency to version 3.0.55
* feat: implement parallel content rendering with metadata support
- Added `ParallelContentRenderer` and `ParallelColumns` components for rendering messages in parallel based on groupId and agentId.
- Introduced `contentMetadataMap` to store metadata for each content part, allowing efficient parallel content detection.
- Updated `Message` and `ContentRender` components to utilize the new metadata structure for rendering.
- Modified `useStepHandler` to manage content indices and metadata during message processing.
- Enhanced `IJobStore` interface and its implementations to support storing and retrieving content metadata.
- Updated data schemas to include `contentMetadataMap` for messages, enabling multi-agent and parallel execution scenarios.
* refactor: update @librechat/agents dependency to version 3.0.56
* refactor: remove unused EPHEMERAL_AGENT_ID constant and simplify agent ID check
* refactor: enhance multi-agent message processing and primary agent determination
* refactor: implement branch message functionality for parallel responses
* refactor: integrate added conversation retrieval into message editing and regeneration processes
* refactor: remove unused isCard and isMultiMessage props from MessageRender and ContentRender components
* refactor: update @librechat/agents dependency to version 3.0.60
* refactor: replace usage of EPHEMERAL_AGENT_ID constant with isEphemeralAgentId function for improved clarity and consistency
* refactor: standardize agent ID format in tests for consistency
* chore: move addedConvo property to the correct position in payload construction
* refactor: rename agent_id values in loadAgent tests for clarity
* chore: reorder props in ContentParts component for improved readability
* refactor: rename variable 'content' to 'result' for clarity in RedisJobStore tests
* refactor: streamline useMessageActions by removing duplicate handleFeedback assignment
* chore: revert placeholder rendering logic MessageRender and ContentRender components to original
* refactor: implement useContentMetadata hook for optimized content metadata handling
* refactor: remove contentMetadataMap and related logic from the codebase and revert back to agentId/groupId in content parts
- Eliminated contentMetadataMap from various components and services, simplifying the handling of message content.
- Updated functions to directly access agentId and groupId from content parts instead of relying on a separate metadata map.
- Adjusted related hooks and components to reflect the removal of contentMetadataMap, ensuring consistent handling of message content.
- Updated tests and documentation to align with the new structure of message content handling.
* refactor: remove logging from groupParallelContent function to clean up output
* refactor: remove model parameter from TBranchMessageRequest type for simplification
* refactor: enhance branch message creation by stripping metadata for standalone content
* chore: streamline branch message creation by simplifying content filtering and removing unnecessary metadata checks
* refactor: include attachments in branch message creation for improved content handling
* refactor: streamline agent content processing by consolidating primary agent identification and filtering logic
* refactor: simplify multi-agent message processing by creating a dedicated mapping method and enhancing content filtering
* refactor: remove unused parameter from loadEphemeralAgent function for cleaner code
* refactor: update groupId handling in metadata to only set when provided by the server
* 🔱 fix: Fork Menu Accessibility Improvements (#10910)
* feat: more accessible aria-label for fork button
* fix: alignment between text and checkbox
* feat: add text change on focus for parity with on hover for keyboard accessibility
* 🤔 fix: Programmatic Expansion State for Thinking Button (#10912)
* 🙋♂️ feat: Accessible Default User Icon Colors (#10909)
* fix: downshift values for all non-compliant default bg-colors for user icons to achieve 4.5:1 contrast threshold minimums with text
* 🚪 feat: Open Sidebar Label Accessibility (#10893)
* feat: more accessible labelling on open / close sidebar
If you've got a screen reader that is reading out the whole page,
each icon button (i.e., `<button><SVG></button>`) will have both
the button's aria-label read out as well as the title from the
SVG (which is usually just "image").
Since we are pretty good about setting aria-labels, we should instead
use `aria-hidden="true"` on these images, since they are not useful
to be read out.
I don't consider this a comprehensive review of all icons in the app,
but I knocked out all the low hanging fruit in this commit.
* feat: Integrate logger for MessageIcon component
* feat: Enhance artifact sharing functionality with updated path checks and read-only state management
* feat: Refactor Thinking and Reasoning components for improved structure and styling
* feat: Enhance artifact sharing with context value management and responsive layout
* feat: Enhance ShareView with theme and language management features
* feat: Improve ThinkingButton accessibility and styling for better user interaction
* feat: Introduce isArtifactRoute utility for route validation in Artifact components
* feat: Add latest message text extraction in SharedView for improved message display
* feat: Update locale handling in SharedView for dynamic date formatting
* feat: Refactor ArtifactsContext and SharedView for improved context handling and styling adjustments
* feat: Enhance artifact panel size management with local storage integration
* chore: imports
* refactor: move ShareArtifactsContainer out of ShareView
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
* feat: Add support for agent handoffs with edges in agent forms and schemas
chore: Mark `agent_ids` field as deprecated in favor of edges across various schemas and types
chore: Update dependencies for @langchain/core and @librechat/agents to latest versions
chore: Update peer dependency for @librechat/agents to version 3.0.0-rc2 in package.json
chore: Update @librechat/agents dependency to version 3.0.0-rc3 in package.json and package-lock.json
feat: first pass, multi-agent handoffs
fix: update output type to ToolMessage in memory handling functions
fix: improve type checking for graphConfig in createRun function
refactor: remove unused content filtering logic in AgentClient
chore: update @librechat/agents dependency to version 3.0.0-rc4 in package.json and package-lock.json
fix: update @langchain/core peer dependency version to ^0.3.72 in package.json and package-lock.json
fix: update @librechat/agents dependency to version 3.0.0-rc6 in package.json and package-lock.json; refactor stream rate handling in various endpoints
feat: Agent handoff UI
chore: update @librechat/agents dependency to version 3.0.0-rc8 in package.json and package-lock.json
fix: improve hasInfo condition and adjust UI element classes in AgentHandoff component
refactor: remove current fixed agent display from AgentHandoffs component due to redundancy
feat: enhance AgentHandoffs UI with localized beta label and improved layout
chore: update @librechat/agents dependency to version 3.0.0-rc10 in package.json and package-lock.json
feat: add `createSequentialChainEdges` function to add back agent chaining via multi-agents
feat: update `createSequentialChainEdges` call to only provide conversation context between agents
feat: deprecate Agent Chain functionality and update related methods for improved clarity
* chore: update @librechat/agents dependency to version 3.0.0-rc11 in package.json and package-lock.json
* refactor: remove unused addCacheControl function and related imports and import from @librechat/agents
* chore: remove unused i18n keys
* refactor: remove unused format export from index.ts
* chore: update @librechat/agents to v3.0.0-rc13
* chore: remove BEDROCK_LEGACY provider from Providers enum
* chore: update @librechat/agents to version 3.0.2 in package.json
* refactor: Reasoning components with independent toggle buttons
- Refactored ThinkingButton to remove unnecessary state and props.
- Updated ContentParts to simplify content rendering and remove hover handling.
- Improved Reasoning component to include independent toggle functionality for each THINK part.
- Adjusted styles for better layout consistency and user experience.
* refactor: isolate hover effects for Reasoning
- Updated ThinkingButton to improve hover effects and layout consistency.
- Refactored Reasoning component to include a new wrapper class for better styling.
- Adjusted icon visibility and transitions for a smoother user experience.
* fix: Prevent rendering of empty messages in Chat component
- Added a check to skip rendering if the message text is only whitespace, improving the user interface by avoiding empty containers.
* chore: Replace div with fragment in Thinking component for cleaner markup
* chore: move Thinking component to Content Parts directory
* refactor: prevent rendering of whitespace-only text in Part component only for edge cases
* ✨ feat: Refactor error handling and improve loading states in MessageContent component
* ✨ feat: Enhance Thinking and ContentParts components with improved hover functionality and clipboard support
* fix: Adjust padding in Thinking and ContentParts components for consistent layout
* ✨ feat: Add response label and improve message editing UI with contextual indicators
* ✨ feat: Add isEditing prop to Feedback and Fork components for improved editing state handling
* refactor: Remove isEditing prop from Feedback and Fork components for cleaner state management
* refactor: Migrate state management from Recoil to Jotai for font size and show thinking features
* refactor: Separate ToggleSwitch into RecoilToggle and JotaiToggle components for improved clarity and state management
* refactor: Remove unnecessary comments in ToggleSwitch and MessageContent components for cleaner code
* chore: reorder import statements in Thinking.tsx
* chore: reorder import statement in EditTextPart.tsx
* chore: reorder import statement
* chore: Reorganize imports in ToggleSwitch.tsx
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
* chore: add i18n localization comment for AlwaysMakeProd component
* feat: enhance accessibility by adding aria-label and aria-labelledby to Switch component
* feat: add aria-labels for accessibility in Agent and Assistant avatar buttons
* fix: add switch aria-labels for accessibility in various components
* feat: add aria-labels and localization keys for accessibility in DataTable, DataTableColumnHeader, and OGDialogTemplate components
* chore: refactor out nested ternary
* feat: add aria-label to DataTable filter button for My Files modal
* feat: add aria-labels for Buttons and localization strings
* feat: add aria-labels to Checkboxes in Agent Builder
* feat: enhance accessibility by adding aria-label and aria-labelledby to Checkbox component
* feat: add aria-label to FileSearchCheckbox in Agent Builder
* feat: add aria-label to Prompts text input area
* feat: enhance accessibility by adding aria-label and aria-labelledby to TextAreaAutosize component
* feat: remove improper role: "list" prop from List in Conversations.tsx to enhance accessibility and stop aria rules conflicting within react-virtualized component
* feat: enhance accessibility by allowing tab navigation and adding ring highlights for conversation title editing accept/reject buttons
* feat: add aria-label to Copy Link button in the conversation share modal
* feat: add title to QR code svg in conversation share modal to describe the image content
* feat: enhance accessibility by making Agent Avatar upload keyboard navigable and round out highlight border on focus
* feat: enhance accessibility by adding aria attributes around alerting users with screen readers to invalid email address inputs in the Agent Builder
* feat: add aria-labels to buttons in Advanced panel of Agent Builder
* feat: enhance accessibility by making FileUpload and Clear All buttons in PresetItems keyboard navigable
* feat: enchance accessiblity by indexing view and delete button aria-labels in shared links management modal to their specific chat titles
* feat: add border highlighting on focus for AnimatedSearchInput
* feat: add category description to aria-labels for prompts in ListCard
* feat: add proper scoping to rows and columns in table headers
* feat: add localized aria-labelling to EditTextPart's TextAreaAutosize component and base dynamic paramters panel components and their supporting translation keys
* feat: add localized aria-labels and aria-labelledBy to Checkbox components without them
* feat: add localized aria-labeledBy for endpoint settings Sliders
* feat: add localized aria-labels for TextareaAutosize components
* chore: remove unused i18n string
* feat: add localized aria-label for BookmarkForm Checkbox
* fix: add stopPropagation onKeyDown for Preview and Edit menu items in prompts that was causing the prompts to inadvertently be sent when triggered with keyboard navigation when Auto-send Prompts was toggled on
* fix: switch TableCell to TableHead for title cells according to harvard issue #789
* fix: add more descriptive localization key for file filter button in DataTable
* chore: remove self-explanatory code comment from RenameForm
* fix: remove stray bg-yellow highlight that was left in during debugging
* fix: add aria-label to model configurator panel back button
* fix: undo incorrect hoist of tool name split for aria-label and span in MCPInput
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
* 📎 feat: Direct Provider Attachment Support for Multimodal Content
* 📑 feat: Anthropic Direct Provider Upload (#9072)
* feat: implement Anthropic native PDF support with document preservation
- Add comprehensive debug logging throughout PDF processing pipeline
- Refactor attachment processing to separate image and document handling
- Create distinct addImageURLs(), addDocuments(), and processAttachments() methods
- Fix critical bugs in stream handling and parameter passing
- Add streamToBuffer utility for proper stream-to-buffer conversion
- Remove api/agents submodule from repository
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* chore: remove out of scope formatting changes
* fix: stop duplication of file in chat on end of response stream
* chore: bring back file search and ocr options
* chore: localize upload to provider string in file menu
* refactor: change createMenuItems args to fit new pattern introduced by anthropic-native-pdf-support
* feat: add cache point for pdfs processed by anthropic endpoint since they are unlikely to change and should benefit from caching
* feat: combine Upload Image into Upload to Provider since they both perform direct upload and change provider upload icon to reflect multimodal upload
* feat: add citations support according to docs
* refactor: remove redundant 'document' check since documents are handled properly by formatMessage in the agents repo now
* refactor: change upload logic so anthropic endpoint isn't exempted from normal upload path using Agents for consistency with the rest of the upload logic
* fix: include width and height in return from uploadLocalFile so images are correctly identified when going through an AgentUpload in addImageURLs
* chore: remove client specific handling since the direct provider stuff is handled by the agent client
* feat: handle documents in AgentClient so no need for change to agents repo
* chore: removed unused changes
* chore: remove auto generated comments from OG commit
* feat: add logic for agents to use direct to provider uploads if supported (currently just anthropic)
* fix: reintroduce role check to fix render error because of undefined value for Content Part
* fix: actually fix render bug by using proper isCreatedByUser check and making sure our mutation of formattedMessage.content is consistent
---------
Co-authored-by: Andres Restrepo <andres@thelinuxkid.com>
Co-authored-by: Claude <noreply@anthropic.com>
📁 feat: Send Attachments Directly to Provider (OpenAI) (#9098)
* refactor: change references from direct upload to direct attach to better reflect functionality
since we are just using base64 encoding strategy now rather than Files/File API for sending our attachments directly to the provider, the upload nomenclature no longer makes sense. direct_attach better describes the different methods of sending attachments to providers anyways even if we later introduce direct upload support
* feat: add upload to provider option for openai (and agent) ui
* chore: move anthropic pdf validator over to packages/api
* feat: simple pdf validation according to openai docs
* feat: add provider agnostic validatePdf logic to start handling multiple endpoints
* feat: add handling for openai specific documentPart formatting
* refactor: move require statement to proper place at top of file
* chore: add in openAI endpoint for the rest of the document handling logic
* feat: add direct attach support for azureOpenAI endpoint and agents
* feat: add pdf validation for azureOpenAI endpoint
* refactor: unify all the endpoint checks with isDocumentSupportedEndpoint
* refactor: consolidate Upload to Provider vs Upload image logic for clarity
* refactor: remove anthropic from anthropic_multimodal fileType since we support multiple providers now
🗂️ feat: Send Attachments Directly to Provider (Google) (#9100)
* feat: add validation for google PDFs and add google endpoint as a document supporting endpoint
* feat: add proper pdf formatting for google endpoints (requires PR #14 in agents)
* feat: add multimodal support for google endpoint attachments
* feat: add audio file svg
* fix: refactor attachments logic so multi-attachment messages work properly
* feat: add video file svg
* fix: allows for followup questions of uploaded multimodal attachments
* fix: remove incorrect final message filtering that was breaking Attachment component rendering
fix: manualy rename 'documents' to 'Documents' in git since it wasn't picked up due to case insensitivity in dir name
fix: add logic so filepicker for a google agent has proper filetype filtering
🛫 refactor: Move Encoding Logic to packages/api (#9182)
* refactor: move audio encode over to TS
* refactor: audio encoding now functional in LC again
* refactor: move video encode over to TS
* refactor: move document encode over to TS
* refactor: video encoding now functional in LC again
* refactor: document encoding now functional in LC again
* fix: extend file type options in AttachFileMenu to include 'google_multimodal' and update dependency array to include agent?.provider
* feat: only accept pdfs if responses api is enabled for openai convos
chore: address ESLint comments
chore: add missing audio mimetype
* fix: type safety for message content parts and improve null handling
* chore: reorder AttachFileMenuProps for consistency and clarity
* chore: import order in AttachFileMenu
* fix: improve null handling for text parts in parseTextParts function
* fix: remove no longer used unsupported capability error message for file uploads
* fix: OpenAI Direct File Attachment Format
* fix: update encodeAndFormatDocuments to support OpenAI responses API and enhance document result types
* refactor: broaden providers supported for documents
* feat: enhance DragDrop context and modal to support document uploads based on provider capabilities
* fix: reorder import statements for consistency in video encoding module
---------
Co-authored-by: Dustin Healy <54083382+dustinhealy@users.noreply.github.com>
* chore: Update documentation for formatToolContent function, remove JSDoc types and duplicate comments
* chore: fix type errors due to attachment.filename in Attachment component
* feat: init @librechat/client
* feat: Add common types and interfaces for accessibility, agents, artifacts, assistants, and tools
* feat: Add jotai as a peer dependency
* fix build client package
* feat: cleanup unused types from common/index.ts
- Remove 104 unused type exports from packages/client/src/common/index.ts
- Keep only 7 actually used exports (93% reduction)
- Add cleanup script with enhanced import pattern detection
- Support both named imports and namespace imports (* as t)
- Create automatic backups and comprehensive documentation
- Maintain type safety with build verification
- No breaking changes to existing code
Kept exports:
- TShowToast, Option, OptionWithIcon, DropdownValueSetter
- MentionOption, NotificationSeverity, MenuItemProps
Scripts: cleanup-common-types-safe.js, README-CLEANUP.md
* fix: cleanup
* fix: package; refactor: tsconfig
* feat: add back `recoil`
* fix: move dependencies to peerDependencies in client package
* feat: add @librechat/client as a dependency in package.json and package-lock.json
* feat: update client package configuration and dependencies
- Added new dependencies for Rollup plugins and updated existing ones in package.json and package-lock.json.
- Introduced a new Rollup configuration file for building the client package.
- Refactored build scripts to include a dedicated build command for the client.
- Updated TypeScript configuration for improved module resolution and type declaration output.
- Integrated a Toast component from the client package into the main App component.
* feat: enhance Rollup configuration for client package
- Updated terser plugin settings to preserve directives like 'use client'.
- Added custom warning handler to ignore "use client" directive warnings during the build process.
* chore: rename package/client build script command
* feat: update client package dependencies and Rollup configuration
- Added rollup-plugin-postcss to package.json and updated package-lock.json.
- Enhanced Rollup configuration to include postcss plugin for CSS handling.
- Updated index.ts to export all components from the components directory for better modularity.
* feat: add client package directory to update configuration
- Included the 'client' package directory in the update.js configuration to ensure it is recognized during updates.
* feat: export Toast component in client package
- Added export for the Toast component in index.ts to enhance modularity and accessibility of components.
* feat: /client transition to @librechat/client
* chore: fixed formatting issues
* fix: update peer dependencies in @librechat/client to prevent bundling them
* fix: correct useSprings implementation in SplitText component
* fix: circular dependencies in DataTable
* fix: add remaining peer dependencies and match actual versions previously used in `client/package.json`
* fix: correct frontend:ci script to include client package build
* chore: enhance unused package detection for @librechat/client and improve dependency extraction
* fix: add missing peer dependency for @radix-ui/react-collapsible
* chore: include "packages/client" in unused i18next keys detection
* test: update AgentFooter tests to use document.querySelector for spinner checks
test: mock window.matchMedia in setupTests.js for consistent test environment
* feat: add react-hook-form dependency and update FormInput component to use its types
* chore: linting
* refactor: remove unused defaultSelectedValues prop from MCPSelect and MultiSelect components
* chore: linting
* feat: update GitHub Actions workflow to publish @librechat/client
* chore: update GitHub Actions workflow to install and build data-provider and client dependencies
* chore: add missing @testing-library/react dependency to client package
* chore: update tsconfig.json to exclude additional test files
* chore: fix build issues, resolve latest LC changes
* chore: move MCP components outside of `~/components/ui`
* feat: implement dynamic theme system with environment variable support and Tailwind CSS integration
* chore: remove unnecessary logging of sttExternal and ttsExternal in Speech component
* chore: squashed cleanup commits
chore: move @tanstack/react-virtual to dependencies and remove recoil from package.json
chore: move dependencies to peerDependencies in package.json
feat: update package.json and rollup.config.js to include jotai and enhance bundling configuration
feat: update package.json and rollup.config.js to include jotai and enhance bundling configuration
refactor: reorganize exports in index.ts for improved clarity
refactor: remove unused types and interfaces from common files
refactor: update peer dependencies and improve component typings
- Removed duplicate peer dependencies from package.json and organized them.
- Updated rollup.config.js to disable TypeScript checking during the build process.
- Modified AnimatedTabs component to use React.ReactNode for label and content types, and added TypeScript workarounds for compatibility.
- Enhanced Label and Separator components to accept an optional className prop and improved prop spreading.
- Updated Slider component to include an optional className prop and refined prop handling for better type safety.
refactor: clean up client workflow and update package dependencies
refactor: update package dependencies and improve PostCSS and Rollup configurations
chore: bump version to 0.1.2 in package.json
chore: bump client version to 0.1.2 in package-lock.json
chore: bump client version to 0.1.3 and update dependencies
chore: bump client version to 0.1.4 and update @react-spring dependencies
chore: update package version to 0.1.5 and adjust peer dependencies
- Bump version in package.json from 0.1.4 to 0.1.5.
- Update peer dependency for @tanstack/react-query to allow version 5.0.0.
- Add @tanstack/react-table and @tanstack/react-virtual as dependencies.
- Update various dependencies to their latest compatible versions.
- Simplify postcss.config.js by removing unnecessary options.
- Clean up rollup.config.js by removing ignored PostCSS warnings.
- Update CheckboxButton component to cast icon as React JSX element.
- Adjust Combobox component's class names for better styling.
- Change DropdownPopup component to use React's namespace import.
- Modify InputOTP component to use 'any' type for OTPInputContext.
- Ensure displayLabel and value in ModelParameters are converted to strings.
- Update MultiSearch component's placeholder to ensure it's a string.
- Cast selectIcon in MultiSelect as React JSX element for consistency.
- Update OGDialogTemplate to cast selectText as React JSX element.
- Initialize animationRef in PixelCard with undefined for clarity.
- Add TypeScript ignore comments in Select and SelectDropDown components for Radix UI type conflicts.
- Ensure title in SelectDropDown is a string and adjust rendering of options.
- Update useLocalize hook to cast options as any for compatibility.
refactor: code structure; chore: translations cleanup
chore: remove unused imports and clean up code in NewChat component
refactor: enhance Menu component to support custom render functions for menu items
style: update itemClassName in ToolsDropdown for improved UI consistency
fix: merge conflicts
chore: update @radix-ui/react-accordion to version 1.2.11
* refactor: remove unnecessary TypeScript type assertions in AnimatedTabs, Label, Separator, and Slider components
* feat: enhance theme system with localStorage persistence and new theme atoms
* chore: bump version of @librechat/client to 0.1.7
* chore: fix ci/cd warnings/errors related to linting and unused localization keys
* chore: update dependencies for class-variance-authority, clsx, and match-sorter
* chore: bump @librechat/client to v0.1.8
* feat: add utility colors for theme customization and remove unused tailwindConfig
* v0.1.9
---------
Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
* 🔧 fix: Handle optional arguments in `useParseArgs` and improve tool call condition
* chore: Remove math plugins from `MarkdownLite`
* ✨ feat: Add Error Boundary to Markdown Component for Enhanced Error Handling
- Introduced `MarkdownErrorBoundary` to catch and display errors during Markdown rendering.
- Updated the `Markdown` component to utilize the new error boundary, improving user experience by handling rendering issues gracefully.
* Revert "chore: Remove math plugins from `MarkdownLite`"
This reverts commit d393099d52.
* ✨ feat: Introduce MarkdownErrorBoundary for improved error handling in Markdown components
* refactor: include most markdown elements in error boundary fallback, aside from problematic plugins
* 🐛 fix: Enhance provider validation and error handling in getProviderConfig function
* WIP: edit text part
* refactor: Allow updating of both TEXT and THINK content types in message updates
* WIP: first pass, save & submit
* chore: remove legacy generation user message field
* feat: merge edited content
* fix: update placeholder and description for bedrock setting
* fix: remove unsupported warning message for AI resubmission
* refactor: Markdown LaTeX processing
- Added micromark-extension-llm-math as a dependency in package.json and package-lock.json.
- Updated Vite configuration to alias micromark-extension-math.
- Modified Markdown components to use singleDollarTextMath: false for improved LaTeX rendering.
- Refactored latex utility functions to enhance LaTeX processing and escaping mechanisms.
* chore: linting of `EditTextPart`
* fix: handle key up to initiate edit of latest user message by adding id prop to Edit Message HoverButton
* chore: linting in Artifact component
* refactor: enhance LaTeX preprocessing functionality
- Updated `preprocessLaTeX` to improve handling of currency and LaTeX expressions.
- Introduced optimized regex patterns for better performance.
- Added support for escaping mhchem commands and handling code blocks.
- Enhanced tests for various LaTeX scenarios, including currency and special characters.
- Refactored existing tests to align with new preprocessing logic.
* chore: filter out false positives in unused packages workflow
- Added a grep command to exclude the micromark-extension-llm-math package from the list of unused dependencies in the GitHub Actions workflow.
* WIP: search tool integration
* WIP: Add web search capabilities and API key management to agent actions
* WIP: web search capability to agent configuration and selection
* WIP: Add web search capability to backend agent configuration
* WIP: add web search option to default agent form values
* WIP: add attachments for web search
* feat: add plugin for processing web search citations
* WIP: first pass, Citation UI
* chore: remove console.log
* feat: Add AnimatedTabs component for tabbed UI functionality
* refactor: AnimatedTabs component with CSS animations and stable ID generation
* WIP example content
* feat: SearchContext for managing search results apart from MessageContext
* feat: Enhance AnimatedTabs with underline animation and state management
* WIP: first pass, Implement dynamic tab functionality in Sources component with search results integration
* fix: Update class names for improved styling in Sources and AnimatedTabs components
* feat: Improve styling and layout in Sources component with enhanced button and item designs
* feat: Refactor Sources component to integrate OGDialog for source display and improve layout
* style: Update background color in SourceItem and SourcesGroup components for improved visibility
* refactor: Sources component to enhance SourceItem structure and improve favicon handling
* style: Adjust font size of domain text in SourceItem for better readability
* feat: Add localization for citation source and details in CompositeCitation component
* style: add theming to Citation components
* feat: Enhance SourceItem component with dialog support and improved hovercard functionality
* feat: Add localization for sources tab and image alt text in Sources component
* style: Replace divs with spans for better semantic structure in CompositeCitation and Citation components
* refactor: Sources component to use useMemo for tab generation and improve performance
* chore: bump @librechat/agents to v2.4.318
* chore: update search result types
* fix: search results retrieval in ContentParts component, re-render attachments when expected
* feat: update sources style/types to use latest search result structure
* style: enhance Dialog (expanded) SourceItem component with link wrapping and improved styling
* style: update ImageItem component styling for improved title visibility
* refactor: remove SourceItemBase component and adjust SourceItem layout for improved styling
* chore: linting twcss order
* fix: prevent FileAttachment from rendering search attachments
* fix: append underscore to responseMessageId for unique identification to prevent mapping of previous latest message's attachments
* chore: remove unused parameter 'useSpecs' from loadTools function
* chore: twcss order
* WIP: WebSearch Tool UI
* refactor: add limit parameter to StackedFavicons for customizable source display
* refactor: optimize search results memoization by making more granular and separate conerns
* refactor: integrated StackedFavicons to WebSearch mid-run
* chore: bump @librechat/agents to expose handleToolCallChunks
* chore: use typedefs from dedicated file instead of defining them in AgentClient module
* WIP: first pass, search progress results
* refactor: move createOnSearchResults function to a dedicated search module
* chore: bump @librechat/agents to v2.4.320
* WIP: first pass, search results processed UX
* refactor: consolidate context variables in createOnSearchResults function
* chore: bump @librechat/agents to v2.4.321
* feat: add guidelines for web search tool response formatting in loadTools function
* feat: add isLast prop to Part component and update WebSearch logic for improved state handling
* style: update Hovercard styles for improved UI consistency
* feat: export FaviconImage component for improved accessibility in other modules
* refactor: export getCleanDomain function and use FaviconImage in Citation component for improved source representation
* refactor: implement SourceHovercard component for consistency and DRY compliance
* fix: replace <p> with <span> for snippet and title in SourceItem and SourceHovercard for consistency
* style: `not-prose`
* style: remove 'not-prose' class for consistency in SourceItem, Citation, and SourceHovercard components, adjust style classes
* refactor: `imageUrl` on hover and prevent duplicate sources
* refactor: enhance SourcesGroup dialog layout and improve source item presentation
* refactor: reorganize Web Components, save in same directory
* feat: add 'news' refType to refTypeMap for citation sources
* style: adjust Hovercard width for improved layout
* refactor: update tool usage guidelines for improved clarity and execution
* chore: linting
* feat: add Web Search badge with initial permissions and local storage logic
* feat: add webSearch support to interface and permissions schemas
* feat: implement Web Search API key management and localization updates
* feat: refactor Web Search API key handling and integrate new search API key form
* fix: remove unnecessary visibility state from FileAttachment component
* feat: update WebSearch component to use Globe icon and localized search label
* feat: enhance ApiKeyDialog with dropdown for reranker selection and update translations
* feat: implement dropdown menus for engine, scraper, and reranker selection in ApiKeyDialog
* chore: linting and add unknown instead of `any` type
* feat: refactor ApiKeyDialog and useAuthSearchTool for improved API key management
* refactor: update ocrSchema to use template literals for default apiKey and baseURL
* feat: add web search configuration and utility functions for environment variable extraction
* fix: ensure filepath is defined before checking its prefix in useAttachmentHandler
* feat: enhance web search functionality with improved configuration and environment variable extraction for authFields
* fix: update auth type in TPluginAction and TUpdateUserPlugins to use Partial<Record<string, string>>
* feat: implement web search authentication verification and enhance webSearchAuth structure
* feat: enhance ephemeral agent handling with new web search capability and type definition
* feat: enhance isEphemeralAgent function to include web search selection
* feat: refactor verifyWebSearchAuth to improve key handling and authentication checks
* feat: implement loadWebSearchAuth function for improved web search authentication handling
* feat: enhance web search authentication with new configuration options and refactor related types
* refactor: rename search engine to search provider and update related localization keys
* feat: update verifyWebSearchAuth to handle multiple authentication types and improve error handling
* feat: update ApiKeyDialog to accept authTypes prop and remove isUserProvided check
* feat: add tests for extractWebSearchEnvVars and loadWebSearchAuth functions
* feat: enhance loadWebSearchAuth to support specific service checks for providers, scrapers, and rerankers
* fix: update web search configuration key and adjust auth result handling in loadTools function
* feat: add new progress key for repeated web searching and update localization
* chore: bump @librechat/agents to 2.4.322
* feat: enhance loadTools function to include ISO time and improve search tool logging
* feat: update StackedFavicons to handle negative start index and improve citation attribution styling and text
* chore: update .gitignore to categorize AI-related files
* fix: mobile responsiveness of sources/citations hovercards
* feat: enhance source display with improved line clamping for better readability
* chore: bump @librechat/agents to v2.4.33
* feat: add handling for image sources in references mapping
* chore: bump librechat-data-provider version to 0.7.84
* chore: bump @librechat/agents version to 2.4.34
* fix: update auth handling to support multiple auth types in tools and allow key configuration in agent panel
* chore: remove redundant agent attribution text from search form
* fix: web search auth uninstall
* refactor: convert CheckboxButton to a forwardRef component and update setValue callback signature
* feat: add triggerRef prop to ApiKeyDialog components for improved dialog control
* feat: integrate triggerRef in CodeInterpreter and WebSearch components for enhanced dialog management
* feat: enhance ApiKeyDialog with additional links for Firecrawl and Jina API key guidance
* feat: implement web search configuration handling in ApiKeyDialog and add tests for dropdown visibility
* fix: update webSearchConfig reference in config route for correct payload assignment
* feat: update ApiKeyDialog to conditionally render sections based on authTypes and modify loadWebSearchAuth to correctly categorize authentication types
* feat: refactor ApiKeyDialog and related tests to use SearchCategories and RerankerTypes enums and remove nested ternaries
* refactor: move ThinkingButton rendering to improve layout consistency in ContentParts
* feat: integrate search context into Markdown component to conditionally include unicodeCitation plugin
* chore: bump @librechat/agents to v2.4.35
* chore: remove unused 18n key
* ci: add WEB_SEARCH permission testing and update AppService tests for new webSearch configuration
* ci: add more comprehensive tests for loadWebSearchAuth to validate authentication handling and authTypes structure
* chore: remove debugging console log from web.spec.ts to clean up test output
* refactor: optional attachment properties for `FileAttachment`
* refactor: update ActionButton to use localized text
* chore: localize text in DataTableFile, add missing translation, imports order, and linting
* chore: linting in DataTable
* fix: integrate Recoil state management for file deletion in DataTableFile
* fix: integrate Recoil state management for file deletion in DataTable
* fix: add temp_file_id to BatchFile type and update deleteFiles logic to properly remove files that are mapped to temp_file_id
* chore: linting
* 🔧 fix: Correctly parse dimensions for image width and height in OpenAIImageGen component
* style: overlay class for DialogImage component to improve visibility
* style: Update transition timing function for PixelCard component to rely on style props
* ✨ feat: Enhance Spinner component with customizable properties and improved animation
* 🔧 fix: Replace Loader with Spinner in RunCode component and update FilePreview to use Spinner for progress indication
* ✨ feat: Refactor icons in CodeProgress and CancelledIcon components; enhance animation and styling in ExecuteCode and ProgressText components
* ✨ feat: Refactor attachment handling in ExecuteCode component; replace individual attachment rendering with AttachmentGroup for improved structure
* ✨ feat: Refactor dialog components for improved accessibility and styling; integrate Skeleton loading state in Image component
* ✨ feat: Refactor ToolCall component to use ToolCallInfo for better structure; replace ToolPopover with AttachmentGroup; enhance ProgressText with error handling and improved UI elements
* 🔧 fix: Remove unnecessary whitespace in ProgressText
* 🔧 fix: Remove unnecessary margin from AgentFooter and AgentPanel components; clean up SidePanel imports
* ✨ feat: Enhance ToolCall and ToolCallInfo components with improved styling; update translations and add warning text color to Tailwind config
* 🔧 fix: Update import statement for useLocalize in ToolCallInfo component; fix: chatform transition
* ✨ feat: Refactor ToolCall and ToolCallInfo components for improved structure and styling; add optimized code block for better output display
* ✨ feat: Implement OpenAI image generation component; add progress tracking and localization for user feedback
* 🔧 fix: Adjust base duration values for image generation; optimize timing for quality settings
* chore: remove unnecessary space
* ✨ feat: Enhance OpenAI image generation with editing capabilities; update localization for progress feedback
* ✨ feat: Add download functionality to images; enhance DialogImage component with download button
* ✨ feat: Enhance image resizing functionality; support custom percentage and pixel dimensions in resizeImageBuffer
* fix: Reset artifacts on unmount and remove useIdChangeEffect hook
* feat: Replace SVG icons with Lucide icons for improved consistency
* fix: Refactor artifact reset logic on unmount and conversation change
* refactor: Rename artifactsVisible to artifactsVisibility for consistency
* feat: Replace custom SVG icons with Lucide icons for improved consistency
* feat: Add visibleArtifacts atom for managing visibility state
* feat: Implement debounced visibility state management for artifacts
* refactor: Add useIdChangeEffect hook to reset visible artifacts on conversation ID change
* refactor: Remove unnecessary dependency from useMemo in TextPart component
* refactor: Enhance artifact visibility management by incorporating location checks for search path
* refactor: Improve transition effects for artifact visibility in Artifacts component
* chore: Remove preprocessCodeArtifacts function and related tests
* fix: Update regex for detecting enclosed artifacts in latest message
* refactor: Update artifact visibility checks to be more generic (not just search)
* chore: Enhance artifact visibility logging
* refactor: Extract closeArtifacts function to improve button click handling
* refactor: remove nested logic from use artifacts effect
* refactor: Update regex for detecting enclosed artifacts to handle new line variations
* refactor: Enhance FileContainer with customizable button and container styles, onClick button handling, and type override
* refactor: Update file type handling to support partial file objects
* refactor: Extract download handling into a custom hook for improved reusability
* refactor: Replace LogContent with Stdout component and enhance Attachment rendering for added visibility
* feat: Update @librechat/agents to version 2.4.1 for referencing generated files in subsequent code interpreter uses
* feat: Add support for tab-separated values (TSV) in mime type handling and improve error logging for regex patterns
* chore: Update @librechat/agents to version 2.4.11 for better `session_id` instructions when wanting to persist files between executions
* chore: Update @librechat/agents to version 2.4.12 for improved functionality
* fix: Enhance argument parsing in useParseArgs to support JSON input and improve code extraction
* refactor: Update input handling in useAutoSave to require more than one character before saving to local storage
* fix: Ensure safe access to agent capabilities in AgentConfig
* fix: don't show agent builder if agents endpoint is not enabled
* fix: Improve error logging for MCP tool calls
* fix: Enhance error message for MCP tool failures
* feat: Add optional spec and iconURL properties to TEndpointOption type
* chore: Update condition to use constant for new conversation parameter
* feat: Enhance abort error handling with additional endpoint options to properly render error message fields
* fix: Throw error instead of returning message for failed MCP tool calls
* refactor: separate logic to generate new S3 URLs for expired links
* feat: Implement S3 URL refresh for user avatars with error handling
* fix: authcontext error in chats where agent chain is used
* refactor: streamline balance configuration logic in getBalanceConfig function
* fix: enhance icon resolution logic in SpecIcon component
* fix: allow null values for spec and iconURL in TEndpointOption type
* fix: update balance check to allow null tokenCredits
* wip: first pass, dropdown for selecting sequential agents
* refactor: Improve agent selection logic and enhance performance in SequentialAgents component
* wip: seq. agents working ideas
* wip: sequential agents style change
* refactor: move agent form options/submission outside of AgentConfig
* refactor: prevent repeating code
* refactor: simplify current agent display in SequentialAgents component
* feat: persist form value handling in AgentSelect component for agent_ids
* feat: first pass, sequential agnets agent update
* feat: enhance message display with agent updates and empty text handling
* chore: update Icon component to use EModelEndpoint for agent endpoints
* feat: update content type checks in BaseClient to use constants for better readability
* feat: adjust max context tokens calculation to use 90% of the model's max tokens
* feat: first pass, agent run message pruning
* chore: increase max listeners for abort controller to prevent memory leaks
* feat: enhance runAgent function to include current index count map for improved token tracking
* chore: update @librechat/agents dependency to version 2.2.5
* feat: update icons and style of SequentialAgents component for improved UI consistency
* feat: add AdvancedButton and AdvancedPanel components for enhanced agent settings navigation, update styling for agent form
* chore: adjust minimum height of AdvancedPanel component for better layout consistency
* chore: update @librechat/agents dependency to version 2.2.6
* feat: enhance message formatting by incorporating tool set into agent message processing, in order to allow better mix/matching of agents (as tool calls for tools not found in set will be stringified)
* refactor: reorder components in AgentConfig for improved readability and maintainability
* refactor: enhance layout of AgentUpdate component for improved visual structure
* feat: add DeepSeek provider to Bedrock settings and schemas
* feat: enhance link styling in mobile.css for better visibility and accessibility
* fix: update banner model import in update banner script; export Banner model
* refactor: `duplicateAgentHandler` to include tool_resources only for OCR context files
* feat: add 'qwen-vl' to visionModels for enhanced model support
* fix: change image format from JPEG to PNG in DALLE3 response
* feat: reorganize Advanced components and add localizations
* refactor: simplify JSX structure in AgentChain component to defer container styling to parent
* feat: add FormInput component for reusable input handling
* feat: make agent recursion limit configurable from builder
* feat: add support for agent capabilities chain in AdvancedPanel and update data-provider version
* feat: add maxRecursionLimit configuration for agents and update related documentation
* fix: update CONFIG_VERSION to 1.2.3 in data provider configuration
* feat: replace recursion limit input with MaxAgentSteps component and enhance input handling
* feat: enhance AgentChain component with hover card for additional information and update related labels
* fix: pass request and response objects to `createActionTool` when using assistant actions to prevent auth error
* feat: update AgentChain component layout to include agent count display
* feat: increase default max listeners and implement capability check function for agent chain
* fix: update link styles in mobile.css for better visibility in dark mode
* chore: temp. remove agents package while bumping shared packages
* chore: update @langchain/google-genai package to version 0.1.11
* chore: update @langchain/google-vertexai package to version 0.2.2
* chore: add @librechat/agents package at version 2.2.8
* feat: add deepseek.r1 model with token rate and context values for bedrock
* 🔧 fix: Log warning for aborted operations in AgentClient
* ci: Remove unused saveMessageToDatabase mock in FakeClient initialization
* ci: test actual implementation of saveMessageToDatabase
* refactor: Change log level from warning to error for aborted operations in AgentClient
* refactor: Add className prop to Image component for customizable styling, use theme selectors
* feat: FLUX Image Generation tool
* refactor: remove artifacts toggle
* refactor: allow hiding side panel while allowing artifacts view
* chore: rename SidePanelGroup to SidePanel for clarity
* Revert "refactor: remove artifacts toggle"
This reverts commit f884c2cfcd.
* feat: add artifacts capability to agent configuration
* refactor: conditionally set artifacts mode based on endpoint type
* feat: Artifacts Capability for Agents
* refactor: enhance getStreamText method to handle intermediate replies and add `stream_options` for openai/azure
* feat: localize progress text and improve UX in CodeAnalyze and ExecuteCode components for expanding analysis
* better i18n support an internationalization-framework.
* removed unused package
* auto sort for translation.json
* fixed tests with the new locales function
* added new CI actions from locize
* to use locize a mention in the README.md
* to use locize a mention in the README.md
* updated README.md and added TRANSLATION.md to the repo
* updated TRANSLATION.md badges
* updated README.md to go to the TRANSLATION.md when clicking on the Translation Progress badge
* updated TRANSLATION.md and added a new issue template.
* updated TRANSLATION.md and added a new issue template.
* updated issue template to add the iso code link.
* updated the new GitHub actions for `locize`
* updated label for new issue template --> i18n
* fixed type issue
* Fix eslint
* Fix eslint with key-spacing spacing
* fix: error type
* fix: handle undefined values in SortFilterHeader component
* fix: typing in Image component
* fix: handle optional promptGroup in PromptCard component
* fix: update localize function to accept string type and remove unnecessary JSX element
* fix: update localize function to enforce TranslationKeys type for better type safety
* fix: improve type safety and handle null values in Assistants component
* fix: enhance null checks for fileId in FilesListView component
* fix: localize 'Go back' button text in FilesListView component
* fix: update aria-label for menu buttons and add translation for 'Close Menu'
* docs: add Reasoning UI section for Chain-of-Thought AI models in README
* fix: enhance type safety by adding type for message in MultiMessage component
* fix: improve null checks and optional chaining in useAutoSave hook
* fix: improve handling of optional properties in cleanupPreset function
* fix: ensure isFetchingNextPage defaults to false and improve null checks for messages in Search component
* fix: enhance type safety and null checks in useBuildMessageTree hook
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
* adding youtube tool
* refactor: use short `url` param instead of `videoUrl`
* refactor: move API key retrieval to a separate credentials module
* refactor: remove unnecessary `isEdited` message property
* refactor: remove unnecessary `isEdited` message property pt. 2
* refactor: YouTube Tool with new `tool()` generator, handle tools already created by new `tool` generator
* fix: only reset request data for multi-convo messages
* refactor: enhance YouTube tool by adding transcript parsing and returning structured JSON responses
* refactor: update transcript parsing to handle raw response and clean up text output
* feat: support toolkits and refactor YouTube tool as a toolkit for better LLM usage
* refactor: remove unused OpenAPI specs and streamline tools transformation in loadAsyncEndpoints
* refactor: implement manifestToolMap for better tool management and streamline authentication handling
* feat: support toolkits for assistants
* refactor: rename loadedTools to toolDefinitions for clarity in PluginController and assistant controllers
* feat: complete support of toolkits for assistants
---------
Co-authored-by: Danilo Pejakovic <danilo.pejakovic@leoninestudios.com>
* ✨ feat: Implement Show Thinking feature; refactor: testing thinking render optimizations
* ✨ feat: Refactor Thinking component styles and enhance Markdown rendering
* chore: add back removed code, revert type changes
* chore: Add back resetCounter effect to Markdown component for improved code block indexing
* chore: bump @librechat/agents and google langchain packages
* WIP: reasoning type updates
* WIP: first pass, reasoning content blocks
* chore: revert code
* chore: bump @librechat/agents
* refactor: optimize reasoning tag handling
* style: ul indent padding
* feat: add Reasoning component to handle reasoning display
* feat: first pass, content reasoning part styling
* refactor: add content placeholder for endpoints using new stream handler
* refactor: only cache messages when requesting stream audio
* fix: circular dep.
* fix: add default param
* refactor: tts, only request after message stream, fix chrome autoplay
* style: update label for submitting state and add localization for 'Thinking...'
* fix: improve global audio pause logic and reset active run ID
* fix: handle artifact edge cases
* fix: remove unnecessary console log from artifact update test
* feat: add support for continued message handling with new streaming method
---------
Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
* feat: Code Interpreter API & File Search Agent Uploads
chore: add back code files
wip: first pass, abstract key dialog
refactor: influence checkbox on key changes
refactor: update localization keys for 'execute code' to 'run code'
wip: run code button
refactor: add throwError parameter to loadAuthValues and getUserPluginAuthValue functions
feat: first pass, API tool calling
fix: handle missing toolId in callTool function and return 404 for non-existent tools
feat: show code outputs
fix: improve error handling in callTool function and log errors
fix: handle potential null value for filepath in attachment destructuring
fix: normalize language before rendering and prevent null return
fix: add loading indicator in RunCode component while executing code
feat: add support for conditional code execution in Markdown components
feat: attachments
refactor: remove bash
fix: pass abort signal to graph/run
refactor: debounce and rate limit tool call
refactor: increase debounce delay for execute function
feat: set code output attachments
feat: image attachments
refactor: apply message context
refactor: pass `partIndex`
feat: toolCall schema/model/methods
feat: block indexing
feat: get tool calls
chore: imports
chore: typing
chore: condense type imports
feat: get tool calls
fix: block indexing
chore: typing
refactor: update tool calls mapping to support multiple results
fix: add unique key to nav link for rendering
wip: first pass, tool call results
refactor: update query cache from successful tool call mutation
style: improve result switcher styling
chore: note on using \`.toObject()\`
feat: add agent_id field to conversation schema
chore: typing
refactor: rename agentMap to agentsMap for consistency
feat: Agent Name as chat input placeholder
chore: bump agents
📦 chore: update @langchain dependencies to latest versions to match agents package
📦 chore: update @librechat/agents dependency to version 1.8.0
fix: Aborting agent stream removes sender; fix(bedrock): completion removes preset name label
refactor: remove direct file parameter to use req.file, add `processAgentFileUpload` for image uploads
feat: upload menu
feat: prime message_file resources
feat: implement conversation access validation in chat route
refactor: remove file parameter from processFileUpload and use req.file instead
feat: add savedMessageIds set to track saved message IDs in BaseClient, to prevent unnecessary double-write to db
feat: prevent duplicate message saves by checking savedMessageIds in AgentController
refactor: skip legacy RAG API handling for agents
feat: add files field to convoSchema
refactor: update request type annotations from Express.Request to ServerRequest in file processing functions
feat: track conversation files
fix: resendFiles, addPreviousAttachments handling
feat: add ID validation for session_id and file_id in download route
feat: entity_id for code file uploads/downloads
fix: code file edge cases
feat: delete related tool calls
feat: add stream rate handling for LLM configuration
feat: enhance system content with attached file information
fix: improve error logging in resource priming function
* WIP: PoC, sequential agents
WIP: PoC Sequential Agents, first pass content data + bump agents package
fix: package-lock
WIP: PoC, o1 support, refactor bufferString
feat: convertJsonSchemaToZod
fix: form issues and schema defining erroneous model
fix: max length issue on agent form instructions, limit conversation messages to sequential agents
feat: add abort signal support to createRun function and AgentClient
feat: PoC, hide prior sequential agent steps
fix: update parameter naming from config to metadata in event handlers for clarity, add model to usage data
refactor: use only last contentData, track model for usage data
chore: bump agents package
fix: content parts issue
refactor: filter contentParts to include tool calls and relevant indices
feat: show function calls
refactor: filter context messages to exclude tool calls when no tools are available to the agent
fix: ensure tool call content is not undefined in formatMessages
feat: add agent_id field to conversationPreset schema
feat: hide sequential agents
feat: increase upload toast duration to 10 seconds
* refactor: tool context handling & update Code API Key Dialog
feat: toolContextMap
chore: skipSpecs -> useSpecs
ci: fix handleTools tests
feat: API Key Dialog
* feat: Agent Permissions Admin Controls
feat: replace label with button for prompt permission toggle
feat: update agent permissions
feat: enable experimental agents and streamline capability configuration
feat: implement access control for agents and enhance endpoint menu items
feat: add welcome message for agent selection in localization
feat: add agents permission to access control and update version to 0.7.57
* fix: update types in useAssistantListMap and useMentions hooks for better null handling
* feat: mention agents
* fix: agent tool resource race conditions when deleting agent tool resource files
* feat: add error handling for code execution with user feedback
* refactor: rename AdminControls to AdminSettings for clarity
* style: add gap to button in AdminSettings for improved layout
* refactor: separate agent query hooks and check access to enable fetching
* fix: remove unused provider from agent initialization options, creates issue with custom endpoints
* refactor: remove redundant/deprecated modelOptions from AgentClient processes
* chore: update @librechat/agents to version 1.8.5 in package.json and package-lock.json
* fix: minor styling issues + agent panel uniformity
* fix: agent edge cases when set endpoint is no longer defined
* refactor: remove unused cleanup function call from AppService
* fix: update link in ApiKeyDialog to point to pricing page
* fix: improve type handling and layout calculations in SidePanel component
* fix: add missing localization string for agent selection in SidePanel
* chore: form styling and localizations for upload filesearch/code interpreter
* fix: model selection placeholder logic in AgentConfig component
* style: agent capabilities
* fix: add localization for provider selection and improve dropdown styling in ModelPanel
* refactor: use gpt-4o-mini > gpt-3.5-turbo
* fix: agents configuration for loadDefaultInterface and update related tests
* feat: DALLE Agents support
* fix: agent initialization, add `collectedUsage` handling
* style: improve side panel styling
* refactor(loadAgent): Optimize order agent project ID retrieval
* feat: code execution
* fix: typing issues
* feat: ExecuteCode content part
* refactor: use local state for default collapsed state of analysis content parts
* fix: code parsing in ExecuteCode component
* chore: bump agents package, export loadAuthValues
* refactor: Update handleTools.js to use EnvVar for code execution tool authentication
* WIP
* feat: download code outputs
* fix(useEventHandlers): type issues
* feat: backend handling for code outputs
* Refactor: Remove console.log statement in Part.tsx
* refactor: add attachments to TMessage/messageSchema
* WIP: prelim handling for code outputs
* feat: attachments rendering
* refactor: improve attachments rendering
* fix: attachments, nullish edge case, handle attachments from event stream, bump agents package
* fix filename download
* fix: tool assignment for 'run code' on agent creation
* fix: image handling by adding attachments
* refactor: prevent agent creation without provider/model
* refactor: remove unnecessary space in agent creation success message
* refactor: select first model if selecting provider from empty on form
* fix: Agent avatar bug
* fix: `defaultAgentFormValues` causing boolean typing issue and typeerror
* fix: capabilities counting as tools, causing duplication of them
* fix: formatted messages edge case where consecutive content text type parts with the latter having tool_call_ids would cause consecutive AI messages to be created. furthermore, content could not be an array for tool_use messages (anthropic limitation)
* chore: bump @librechat/agents dependency to version 1.6.9
* feat: bedrock agents
* feat: new Agents icon
* feat: agent titling
* feat: agent landing
* refactor: allow sharing agent globally only if user is admin or author
* feat: initial AgentPanelSkeleton
* feat: AgentPanelSkeleton
* feat: collaborative agents
* chore: add potential authorName as part of schema
* chore: Remove unnecessary console.log statement
* WIP: agent model parameters
* chore: ToolsDialog typing and tool related localization chnages
* refactor: update tool instance type (latest langchain class), and rename google tool to 'google' proper
* chore: add back tools
* feat: Agent knowledge files upload
* refactor: better verbiage for disabled knowledge
* chore: debug logs for file deletions
* chore: debug logs for file deletions
* feat: upload/delete agent knowledge/file-search files
* feat: file search UI for agents
* feat: first pass, file search tool
* chore: update default agent capabilities and info
* refactor(EditMessage): avoid manipulation of native paste handling, leverage react-hook-form for textarea changes
* style: apply better theming for MinimalIcon
* fix(useVoicesQuery/useCustomConfigSpeechQuery): make sure to only try request once per render
* feat: edit message content parts
* fix(useCopyToClipboard): handle both assistants and agents content blocks
* refactor: remove save & submit and update text content correctly
* chore(.env.example/config): exclude unsupported bedrock models
* feat: artifacts for aws bedrock
* fix: export options for bedrock conversations
* feat: Add BedrockIcon component to SVG library
* feat: EModelEndpoint.bedrock
* feat: first pass, bedrock chat. note: AgentClient is returning `agents` as conversation.endpoint
* fix: declare endpoint in initialization step
* chore: Update @librechat/agents dependency to version 1.4.5
* feat: backend content aggregation for agents/bedrock
* feat: abort agent requests
* feat: AWS Bedrock icons
* WIP: agent provider schema parsing
* chore: Update EditIcon props type
* refactor(useGenerationsByLatest): make agents and bedrock editable
* refactor: non-assistant message content, parts
* fix: Bedrock response `sender`
* fix: use endpointOption.model_parameters not endpointOption.modelOptions
* fix: types for step handler
* refactor: Update Agents.ToolCallDelta type
* refactor: Remove unnecessary assignment of parentMessageId in AskController
* refactor: remove unnecessary assignment of parentMessageId (agent request handler)
* fix(bedrock/agents): message regeneration
* refactor: dynamic form elements using react-hook-form Controllers
* fix: agent icons/labels for messages
* fix: agent actions
* fix: use of new dynamic tags causing application crash
* refactor: dynamic settings touch-ups
* refactor: update Slider component to allow custom track class name
* refactor: update DynamicSlider component styles
* refactor: use Constants value for GLOBAL_PROJECT_NAME (enum)
* feat: agent share global methods/controllers
* fix: agents query
* fix: `getResponseModel`
* fix: share prompt a11y issue
* refactor: update SharePrompt dialog theme styles
* refactor: explicit typing for SharePrompt
* feat: add agent roles/permissions
* chore: update @librechat/agents dependency to version 1.4.7 for tool_call_ids edge case
* fix(Anthropic): messages.X.content.Y.tool_use.input: Input should be a valid dictionary
* fix: handle text parts with tool_call_ids and empty text
* fix: role initialization
* refactor: don't make instructions required
* refactor: improve typing of Text part
* fix: setShowStopButton for agents route
* chore: remove params for now
* fix: add streamBuffer and streamRate to help prevent 'Overloaded' errors from Anthropic API
* refactor: remove console.log statement in ContentRender component
* chore: typing, rename Context to Delete Button
* chore(DeleteButton): logging
* refactor(Action): make accessible
* style(Action): improve a11y again
* refactor: remove use/mention of mongoose sessions
* feat: first pass, sharing agents
* feat: visual indicator for global agent, remove author when serving to non-author
* wip: params
* chore: fix typing issues
* fix(schemas): typing
* refactor: improve accessibility of ListCard component and fix console React warning
* wip: reset templates for non-legacy new convos
* Revert "wip: params"
This reverts commit f8067e91d4.
* Revert "refactor: dynamic form elements using react-hook-form Controllers"
This reverts commit 2150c4815d.
* fix(Parameters): types and parameter effect update to only update local state to parameters
* refactor: optimize useDebouncedInput hook for better performance
* feat: first pass, anthropic bedrock params
* chore: paramEndpoints check for endpointType too
* fix: maxTokens to use coerceNumber.optional(),
* feat: extra chat model params
* chore: reduce code repetition
* refactor: improve preset title handling in SaveAsPresetDialog component
* refactor: improve preset handling in HeaderOptions component
* chore: improve typing, replace legacy dialog for SaveAsPresetDialog
* feat: save as preset from parameters panel
* fix: multi-search in select dropdown when using Option type
* refactor: update default showDefault value to false in Dynamic components
* feat: Bedrock presets settings
* chore: config, fix agents schema, update config version
* refactor: update AWS region variable name in bedrock options endpoint to BEDROCK_AWS_DEFAULT_REGION
* refactor: update baseEndpointSchema in config.ts to include baseURL property
* refactor: update createRun function to include req parameter and set streamRate based on provider
* feat: availableRegions via config
* refactor: remove unused demo agent controller file
* WIP: title
* Update @librechat/agents to version 1.5.0
* chore: addTitle.js to handle empty responseText
* feat: support images and titles
* feat: context token updates
* Refactor BaseClient test to use expect.objectContaining
* refactor: add model select, remove header options params, move side panel params below prompts
* chore: update models list, catch title error
* feat: model service for bedrock models (env)
* chore: Remove verbose debug log in AgentClient class following stream
* feat(bedrock): track token spend; fix: token rates, value key mapping for AWS models
* refactor: handle streamRate in `handleLLMNewToken` callback
* chore: AWS Bedrock example config in `.env.example`
* refactor: Rename bedrockMeta to bedrockGeneral in settings.ts and use for AI21 and Amazon Bedrock providers
* refactor: Update `.env.example` with AWS Bedrock model IDs URL and additional notes
* feat: titleModel support for bedrock
* refactor: Update `.env.example` with additional notes for AWS Bedrock model IDs
