mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-04-03 22:37:20 +02:00
209 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Danny Avila
|
083042e56c
|
🪝 fix: Safe Hook Fallbacks for Tool-Call Components in Search Route (#12423)
* fix: add useOptionalMessagesOperations hook for context-safe message operations Add a variant of useMessagesOperations that returns no-op functions when MessagesViewProvider is absent instead of throwing, enabling shared components to render safely outside the chat route. * fix: use optional message operations in ToolCallInfo and UIResourceCarousel Switch ToolCallInfo and UIResourceCarousel from useMessagesOperations to useOptionalMessagesOperations so they no longer crash when rendered in the /search route, which lacks MessagesViewProvider. * fix: update test mocks to use useOptionalMessagesOperations * fix: consolidate noops and narrow useMemo dependency in useOptionalMessagesOperations - Replace three noop variants (noopAsync, noopReturn, noop) with a single `const noop = () => undefined` that correctly returns void/undefined - Destructure individual fields from context before the useMemo so the dependency array tracks stable operation references, not the full context object (avoiding unnecessary re-renders on unrelated state changes) - Add useOptionalMessagesConversation for components that need conversation data outside MessagesViewProvider * fix: use optional hooks in MCPUIResource components to prevent search crash MCPUIResource and MCPUIResourceCarousel render inside Markdown prose and can appear in the /search route. Switch them from the strict useMessagesOperations/useMessagesConversation hooks to the optional variants that return safe defaults when MessagesViewProvider is absent. * test: update test mocks for optional hook renames * fix: update ToolCallInfo and UIResourceCarousel test mocks to useOptionalMessagesOperations * fix: use optional message operations in useConversationUIResources useConversationUIResources internally called the strict useMessagesOperations(), which still threw when MCPUIResource rendered outside MessagesViewProvider. Switch to useOptionalMessagesOperations so the entire MCPUIResource render chain is safe in the /search route. * style: fix import order per project conventions * fix: replace as-unknown-as casts with typed NOOP_OPS stubs - Define OptionalMessagesOps type and NOOP_OPS constant with properly typed no-op functions, eliminating all `as unknown as T` casts - Use conversationId directly from useOptionalMessagesConversation instead of re-deriving it from conversation object - Update JSDoc to reflect search route support * test: add no-provider regression tests for optional message hooks Verify useOptionalMessagesOperations and useOptionalMessagesConversation return safe defaults when rendered outside MessagesViewProvider, covering the core crash path this PR fixes. |
||
|
Danny Avila
|
8e2721011e
|
🔑 fix: Robust MCP OAuth Detection in Tool-Call Flow (#12418)
* fix(api): add buildOAuthToolCallName utility for MCP OAuth flows
Extract a shared utility that builds the synthetic tool-call name
used during MCP OAuth flows (oauth_mcp_{normalizedServerName}).
Uses startsWith on the raw serverName (not the normalized form) to
guard against double-wrapping, so names that merely normalize to
start with oauth_mcp_ (e.g., oauth@mcp@server) are correctly
prefixed while genuinely pre-wrapped names are left as-is.
Add 8 unit tests covering normal names, pre-wrapped names, _mcp_
substrings, special characters, non-ASCII, and empty string inputs.
* fix(backend): use buildOAuthToolCallName in MCP OAuth flows
Replace inline tool-call name construction in both reconnectServer
(MCP.js) and createOAuthEmitter (ToolService.js) with the shared
buildOAuthToolCallName utility. Remove unused normalizeServerName
import from ToolService.js. Fix import ordering in both files.
This ensures the oauth_mcp_ prefix is consistently applied so the
client correctly identifies MCP OAuth flows and binds the CSRF
cookie to the right server.
* fix(client): robust MCP OAuth detection and split handling in ToolCall
- Fix split() destructuring to preserve tail segments for server names
containing _mcp_ (e.g., foo_mcp_bar no longer truncated to foo).
- Add auth URL redirect_uri fallback: when the tool-call name lacks
the _mcp_ delimiter, parse redirect_uri for the MCP callback path.
Set function_name to the extracted server name so progress text
shows the server, not the raw tool-call ID.
- Display server name instead of literal "oauth" as function_name,
gated on auth presence to avoid misidentifying real tools named
"oauth".
- Consolidate three independent new URL(auth) parses into a single
parsedAuthUrl useMemo shared across detection, actionId, and
authDomain hooks.
- Replace any type on ProgressText test mock with structural type.
- Add 8 tests covering delimiter detection, multi-segment names,
function_name display, redirect_uri fallback, normalized _mcp_
server names, and non-MCP action auth exclusion.
* chore: fix import order in utils.test.ts
* fix(client): drop auth gate on OAuth displayName so completed flows show server name
The createOAuthEnd handler re-emits the toolCall delta without auth,
so auth is cleared on the client after OAuth completes. Gating
displayName on `func === 'oauth' && auth` caused completed OAuth
steps to render "Completed oauth" instead of "Completed my-server".
Remove the `&& auth` gate — within the MCP delimiter branch the
func="oauth" check alone is sufficient. Also remove `auth` from the
useMemo dep array since only `parsedAuthUrl` is referenced. Update
the test to assert correct post-completion display.
|
||
|
Marco Beretta
|
0c66823c26
|
🧩 feat: Redesign Tool Call UI with Contextual Icons, Smart Grouping, and Rich Output Rendering (#12163)
* feat: redesign tool call UI with type-specific icons, smart grouping, and rich output rendering
Replace the generic spinner/checkmark tool call UI with a modern, Cursor-inspired design:
- Add per-tool-type icons (Plug for MCP, Terminal for code, Globe for web search, etc.)
- Group 2+ consecutive tool calls into collapsible "Used N tools" sections
- Stack unique tool icons in grouped headers with overlapping circle design
- Replace raw JSON output with intelligent renderers (table, error, text)
- Restructure ToolCallInfo: output first, parameters collapsible at bottom
- Add shared useExpandCollapse hook for consistent animations
- Add CodeWindowHeader for ExecuteCode windowed view
- Remove FinishedIcon (purple checkmark) entirely
* feat: display custom MCP server icons in tool calls
Add useMCPIconMap hook to resolve MCP server names to their configured
icon paths. ToolIcon and StackedToolIcons now accept custom icon URLs,
showing actual server logos (e.g., Home Assistant, GitHub) instead of
the generic Plug icon for MCP tool calls.
* refactor: unify container styling across code blocks, mermaid, and tool output
Replace hardcoded gray colors with theme tokens throughout:
- CodeBlock: bg-gray-900/700 -> bg-surface-secondary/tertiary + border-border-light
- Mermaid dialog: bg-gray-700 -> bg-surface-secondary, text-gray-200 -> text-text-secondary
- Mermaid containers: rounded-xl -> rounded-lg, remove shadow-md for consistency
- ResultSwitcher: bg-gray-700 -> bg-surface-secondary with border separator
- RunCode: hover:bg-gray-700 -> hover:bg-surface-hover
- ErrorOutput: add border for visual consistency
- MermaidHeader/CodeWindowHeader: consistent focus outlines using border-heavy
* refactor: simplify tool output to plain text, remove custom renderers
Remove over-engineered tool output system (TableOutput, ErrorOutput,
detectOutputType) in favor of simple text extraction. Tool output now
extracts the text content from MCP content blocks and displays it as
clean readable text — no tables, no error styling, no JSON formatting.
Parameters only show key-value badges for simple objects; complex JSON
is hidden instead of dumped raw. Matches Cursor-style simplicity.
* fix: handle error messages and format JSON arrays in tool output
- Strip verbose MCP error prefixes (Error: [MCP][server][tool] tool call
failed: Error POSTing...) and show just the meaningful error message
- Display errors in red text
- Format uniform JSON arrays as readable lists (name — path) instead
of raw JSON dumps
- Format plain JSON objects as key: value lines
* feat: improve JSON display in tool call output and parameters
- Replace flat formatObject with recursive formatValue for proper
indented display of nested JSON structures
- Add ComplexInput component for tool parameters with nested objects,
arrays, or long strings (previously hidden)
- Broaden hasParams check to show parameters for all object types
- Add font-mono to output renderer for better alignment
* feat: add localization keys for tool errors, web search, and code UI
* refactor: move Mermaid components into dedicated directory module
* refactor: extract CodeBar, FloatingCodeBar, and copy utilities from CodeBlock
* feat: replace manual SVG icons with @icons-pack/react-simple-icons
Supports 50+ programming languages with tree-shaken brand icons
instead of hand-crafted SVGs for 19 languages.
* refactor: simplify code execution UI with persistent code toggle
* refactor: use useExpandCollapse hook in Thinking and Reasoning
* feat: improve tool call error states, subtitles, and group summaries
* feat: redesign web search with inline source display
* feat: improve agent handoff with keyboard accessibility
* feat: reorganize exports order in hooks and utils
* refactor: unify CopyCodeButton with animated icon transitions and iconOnly support
* feat: add run code state machine with animated success/error feedback
* refactor: improve ResultSwitcher with lucide icons and accessibility
* refactor: update CopyButton component
* refactor: replace CopyCodeButton with CopyButton component across multiple files
* test: add ImageGen test stubs
* test: add RetrievalCall test stubs
* feat: merge ImageGen with ToolIcon and localized progress text
* feat: modernize RetrievalCall with ToolIcon and collapsible output
* test: add getToolIconType action delimiter tests
* test: add ImageGen collapsible output tests
* feat: add action ToolIcon type with Zap icon
* fix: replace AgentHandoff div with semantic button
* feat: add aria-live regions to tool components
* feat: redesign execute_code tool UI with syntax highlighting and language icons
- Remove filename labels (script.py, main.rs) and line counter from CodeWindowHeader
- Replace generic FileCode icon with language-specific LangIcon
- Add syntax highlighting via highlight.js to code blocks
- Add SquareTerminal icon to ExecuteCode progress text
- Use shared CopyButton component in CodeWindowHeader
- Remove active:scale-95 press animation from CopyButton and RunCode
* feat: dynamic tool status text sizing based on markdown font-size variable
- Add tool-status-text CSS class using calc(0.9 * --markdown-font-size)
- Update progress-text-wrapper to use dynamic sizing instead of base size
- Apply tool-status-text to WebSearch, ToolCallGroup, AgentHandoff, ImageGen
- Replace hardcoded text-sm/text-xs with dynamic class across all tools
- Animate chevron rotation in ProgressText and ToolCallGroup
- Update subtitle text color from tertiary to secondary
* fix: consistent spacing and text styles across all tool components
- Standardize tool status row spacing to my-1/my-1.5 across all components
- Update ToolCallInfo text from tertiary to secondary, add vertical padding
- Animate ToolCallInfo parameters chevron rotation
- Update OutputRenderer link colors from tertiary to secondary
* feat: unify tool call grouping for all tool types
All consecutive tool calls (MCP, execute_code, web_search, image_gen,
file_search, code_interpreter) are now grouped under a single
collapsible "Used N tools" header instead of only grouping generic
tool calls.
- Remove SPECIAL_TOOL_NAMES blacklist from groupToolCalls
- Replace getToolCallData with getToolMeta to handle all tool types
- Use renderPart callback in ToolCallGroup for proper component routing
- Add file_search and code_interpreter mappings to getToolIconType
* feat: friendly tool group labels, more icons, and output copy button
- Show friendly names in group summary (Code, Web Search, Image
Generation) instead of raw tool names
- Display MCP server names instead of individual function names
- Deduplicate labels and show up to 3 with +N overflow
- Increase stacked icons from 3 to 4
- Add icon-only copy button to tool output (OutputRenderer)
* fix: execute_code spacing and syntax-highlighted code visibility
Match ToolCall spacing by using my-1.5 on status line and moving my-2
inside overflow-hidden. Replace broken hljs.highlight() with lowlight
(same engine used by rehype-highlight for markdown code blocks) to
render syntax-highlighted code as React elements. Handle object args
in useParseArgs to support both string and Record arg formats.
* feat: replace showCode with auto-expand tools setting
Replace the execute_code-only "Always show code when using code
interpreter" global toggle with a new "Auto-expand tool details"
setting that controls all tool types. Each tool instance now uses
independent local state initialized from the setting, so expanding
one tool no longer affects others. Applies to ToolCall, ExecuteCode,
ToolCallGroup, and CodeAnalyze components.
* fix: apply auto-expand tools setting to WebSearch and RetrievalCall
* fix: only auto-expand tools when content is available
Defer auto-expansion until tool output or content arrives, preventing
empty bordered containers from showing while tools are still running.
Uses useEffect to expand when output becomes available during streaming.
* feat: redesign file_search tool output, citations, and file preview
- Redesign RetrievalCall with per-file cards using OutputRenderer
(truncated content with show more/less, copy button) matching MCP
tool pattern
- Route file_search tool calls from Agents API to RetrievalCall
instead of generic ToolCall
- Add FilePreviewDialog for viewing files (PDF iframe, text content)
with download option, opened from clickable filenames
- Redesign file citations: FileText icon in badge, relevance and
page numbers in hovercard, click opens file preview instead of
downloading
- Add file preview to message file attachments (Files.tsx)
- Fix hovercard animation to slide top-to-bottom and dismiss
instantly on file click to prevent glitching over dialog
- Add localization keys for relevance, extracted content, preview
- Add top margin to ToolCallGroup
* chore: remove leftover .planning files
* fix: polish FilePreviewDialog, CodeBlock, LangIcon, and Sources
* fix: prevent keyboard focus on collapsed tool content
Add inert attribute to all expand/collapse wrapper divs so
collapsed content is removed from tab order and hidden from
assistive technology. Skip disabled ProgressText buttons from
tab order with tabIndex={-1}.
* feat: integrate file metadata into file_search UI
Pass fileType (MIME) and fileBytes from backend file records through
to the frontend. Add file-type-specific icons, file size display,
pages sorted by relevance, multi-snippet content per file, smart
preview detection by MIME type, and copy button in file preview dialog.
* fix: review fixes — inverted type check, wrong dimension, missing import, fail-open perms, timer leaks, dead code cleanup
* fix: update CodeBlock styling for improved visual consistency
* fix(chat): open composite file citations in preview
* fix(chat): restore file previews for parsed search results
* chore(git): ignore bg-shell artifacts
* fix(chat): restore readable code content in light theme
* style(chat): align code and output surfaces by theme
* chore(i18n): remove 6 unused translation keys
* fix(deps): replace private registry URL with public npm registry in lockfile
* fix: CI lint, build, and test failures
- Add missing scaleImage utility (fixes Vite build error)
- Export scaleImage from utils/index.ts
- Remove unused imports from Part.tsx (FunctionToolCall, CodeToolCall, Agents)
- Fix prettier formatting in Part.tsx (multi-line → single-line imports, conditions)
- Remove excess blank lines in Part.tsx
- Remove unused CodeEditorRef import from Artifacts.tsx
- Add useProgress mock to OpenAIImageGen.test.tsx
- Add scaleImage mock to OpenAIImageGen.test.tsx
- Update OpenAIImageGen tests to match redesigned component structure
- Remove dead collapsible output panel tests from ImageGen.test.tsx
- Add @icons-pack/react-simple-icons to Jest transformIgnorePatterns (ESM fix)
* refactor: reorganize imports order across multiple components for consistency
* fix: add scaleImage tests, delete dead ImageGen wrapper, wire up onUIAction in ToolCallInfo
- Add 7 unit tests for scaleImage utility covering null ref, scaling,
no-upscale, height clamping, landscape, and panoramic images
- Delete unused Content/ImageGen.tsx re-export wrapper (ImageGen is
imported from Parts/OpenAIImageGen via the Parts barrel)
- Wire up onUIAction in ToolCallInfo to use handleUIAction + ask from
useMessagesOperations, matching UIResourceCarousel's behavior
(was previously a silent no-op)
* refactor: optimize imports and enhance lazy loading for language icons
* fix: address review findings for tool call UI redesign
- Fix unstable array-index keys in ToolCallGroup (streaming state corruption)
- Add plain-text fallback in InputRenderer for non-JSON tool args
- Localize FRIENDLY_NAMES via translation keys instead of hardcoded English
- Guard autoCollapse against user-initiated manual expansion
- Fix CODE_INTERPRETER hasOutput to check actual outputs instead of hardcoding true
- Add logger.warn for Citations fail-closed behavior on permission errors
- Add Terminal icon to CodeAnalyze ProgressText for visual consistency
- Fix getMCPServerName to use indexOf instead of fragile split
- Use useLayoutEffect for inert attribute in useExpandCollapse (a11y)
- Memoize style object in useExpandCollapse to avoid defeating React.memo
- Memoize groupSequentialToolCalls in ContentParts to avoid recomputation
- Use source.link as stable key instead of array index in WebSearch
- Hoist rehypePlugins outside CodeMarkdown to prevent per-render recreation
* fix: revert useMemo after conditional returns in ContentParts
The useMemo placed after early returns violated React Rules of Hooks —
hook call count would change when transitioning between edit/view mode.
Reverted to the original plain forEach which is correct and equally
performant since content changes on every streaming token anyway.
* chore: remove unused com_ui_variables_info translation key
* fix: update tests and jest config for ESM compatibility after rebase
- Add ESM-only packages to transformIgnorePatterns (@dicebear, unified
ecosystem, react-dnd, lowlight, etc.) to fix Jest parse failures
introduced by dev rebase
- Update ToolCall.test.tsx to match new component API (CSS
expand/collapse instead of conditional rendering, simplified props)
- Update ToolCallInfo.test.tsx to mock OutputRenderer (avoids ESM
chain), align with current component interface (input/output/attachments)
* refactor: replace @icons-pack/react-simple-icons with inline SVGs
Inline the 51 Simple Icons SVG paths used by LangIcon directly into
langIconPaths.ts, eliminating the runtime dependency on
@icons-pack/react-simple-icons (which requires Node >= 24).
- LangIcon now renders a plain <svg> with the path data instead of
lazy-loading React components from the package
- Removes Suspense/React.lazy overhead for code block language icons
- SVG paths sourced from Simple Icons (CC0 1.0 license)
- Package kept in package.json for now (will be removed separately)
* fix: replace Plug icon with Wrench for MCP tools, remove unused i18n keys
- MCP tools without a custom iconPath now show Wrench instead of Plug,
matching the generic tool fallback and avoiding the "plugin" metaphor
- Remove unused translation keys: com_assistants_action_attempt,
com_assistants_attempt_info, com_assistants_domain_info,
com_ui_ui_resources
* fix: address second review findings
- Combine 3x getToolMeta loop into single toolMetadata pass (ToolCallGroup)
- Extract sortPagesByRelevance to shared util (was duplicated in
FilePreviewDialog and RetrievalCall)
- Deduplicate AGENT_STYLE_TOOLS Set (export from OpenAIImageGen/index.ts)
- Localize "source/sources" in WebSearch aria-label
- Add autoExpand useEffect to CodeAnalyze for live setting changes
- Log download errors in FilePreviewDialog instead of silently swallowing
- Replace @ts-ignore with @ts-expect-error + explanation in Code.tsx
- Remove dead currentContent alias in CodeMarkdown
* chore: remove @icons-pack/react-simple-icons dependency from package.json and package-lock.json
- Deleted the @icons-pack/react-simple-icons entry from both package.json and package-lock.json, following the previous refactor to use inline SVGs for icons.
* fix: address triage audit findings
- Remove unused gIdx variable (ESLint error)
- Fix singular/plural in web search sources aria-label
- Separate inline type import in ToolCallGroup per AGENTS.md
* fix: remove invalid placeholderDimensions prop from Image component
* chore: import order
* chore: import order
* fix: resolve TypeScript errors in PR-touched files
- Remove non-existent placeholderDimensions prop from Image in Files.tsx
- Fix localize count param type (number, not string) in WebSearch.tsx
- Pass full resource object instead of partial in UIResourceCarousel.tsx
- Add 'as const' to toggleSwitchConfigs localizationKey in General.tsx
- Fix SearchResultData type in Citation.test.tsx
- Fix TAttachment and UIResource test fixture types across test files
* docs: document formatBytes difference in FilePreviewDialog
The local formatBytes returns a human-readable string with units
("1.5 MB"), while ~/utils/formatBytes returns a raw number. They
serve different purposes, so the local copy is retained with a
JSDoc comment explaining the distinction.
* fix: address remaining review items
- Replace cancelled IIFE with documented ternary in OpenAIImageGen,
explaining the agent vs legacy path distinction
- Add .catch() fallback to loadLowlight() in useLazyHighlight — falls
back to plain text if the chunk fails to load
- Fix import ordering in ToolCallGroup.tsx (type imports grouped before
local value imports per AGENTS.md)
* fix: blob URL leak and useGetFiles over-fetch
- FilePreviewDialog: add cancelledRef guard to loadPreview so blob URLs
are never created after the dialog closes (prevents orphaned object
URLs on unmount during async PDF fetch)
- RetrievalCall: filter useGetFiles by fileIds from fileSources instead
of fetching the entire user file corpus for display-only name matching
* chore: fix com_nav_auto_expand_tools alphabetical order in translation.json
* fix: render non-object JSON params instead of returning null in InputRenderer
* refactor: render JSON tool output as syntax-highlighted code block
Replace the custom YAML-ish formatValue/formatObjectArray rendering
with JSON.stringify + hljs language-json styling. Structured API
responses (like GitHub search results) now display as proper
syntax-highlighted JSON with indentation instead of a flat key-value
text dump.
- Remove formatValue, formatObjectArray, isUniformObjectArray helpers
- Add isJson flag to extractText return type
- JSON output rendered in <code class="hljs language-json"> block
- Text content blocks (type: "text") still extracted and rendered
as plain text
- Error output unchanged
* fix: extract cancelled IIFE to named function in OpenAIImageGen
Replace nested ternary with a named computeCancelled() function that
documents the agent vs legacy path branching. Resolves eslint
no-nested-ternary warning.
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
|
||
|
Danny Avila
|
b5c097e5c7
|
⚗️ feat: Agent Context Compaction/Summarization (#12287)
* chore: imports/types
Add summarization config and package-level summarize handler contracts
Register summarize handlers across server controller paths
Port cursor dual-read/dual-write summary support and UI status handling
Selectively merge cursor branch files for BaseClient summary content
block detection (last-summary-wins), dual-write persistence, summary
block unit tests, and on_summarize_status SSE event handling with
started/completed/failed branches.
Co-authored-by: Cursor <cursoragent@cursor.com>
refactor: type safety
feat: add localization for summarization status messages
refactor: optimize summary block detection in BaseClient
Updated the logic for identifying existing summary content blocks to use a reverse loop for improved efficiency. Added a new test case to ensure the last summary content block is updated correctly when multiple summary blocks exist.
chore: add runName to chainOptions in AgentClient
refactor: streamline summarization configuration and handler integration
Removed the deprecated summarizeNotConfigured function and replaced it with a more flexible createSummarizeFn. Updated the summarization handler setup across various controllers to utilize the new function, enhancing error handling and configuration resolution. Improved overall code clarity and maintainability by consolidating summarization logic.
feat(summarization): add staged chunk-and-merge fallback
feat(usage): track summarization usage separately from messages
feat(summarization): resolve prompt from config in runtime
fix(endpoints): use @librechat/api provider config loader
refactor(agents): import getProviderConfig from @librechat/api
chore: code order
feat(app-config): auto-enable summarization when configured
feat: summarization config
refactor(summarization): streamline persist summary handling and enhance configuration validation
Removed the deprecated createDeferredPersistSummary function and integrated a new createPersistSummary function for MongoDB persistence. Updated summarization handlers across various controllers to utilize the new persistence method. Enhanced validation for summarization configuration to ensure provider, model, and prompt are properly set, improving error handling and overall robustness.
refactor(summarization): update event handling and remove legacy summarize handlers
Replaced the deprecated summarization handlers with new event-driven handlers for summarization start and completion across multiple controllers. This change enhances the clarity of the summarization process and improves the integration of summarization events in the application. Additionally, removed unused summarization functions and streamlined the configuration loading process.
refactor(summarization): standardize event names in handlers
Updated event names in the summarization handlers to use constants from GraphEvents for consistency and clarity. This change improves maintainability and reduces the risk of errors related to string literals in event handling.
feat(summarization): enhance usage tracking for summarization events
Added logic to track summarization usage in multiple controllers by checking the current node type. If the node indicates a summarization task, the usage type is set accordingly. This change improves the granularity of usage data collected during summarization processes.
feat(summarization): integrate SummarizationConfig into AppSummarizationConfig type
Enhanced the AppSummarizationConfig type by extending it with the SummarizationConfig type from librechat-data-provider. This change improves type safety and consistency in the summarization configuration structure.
test: add end-to-end tests for summarization functionality
Introduced a comprehensive suite of end-to-end tests for the summarization feature, covering the full LibreChat pipeline from message creation to summarization. This includes a new setup file for environment configuration and a Jest configuration specifically for E2E tests. The tests utilize real API keys and ensure proper integration with the summarization process, enhancing overall test coverage and reliability.
refactor(summarization): include initial summary in formatAgentMessages output
Updated the formatAgentMessages function to return an initial summary alongside messages and index token count map. This change is reflected in multiple controllers and the corresponding tests, enhancing the summarization process by providing additional context for each agent's response.
refactor: move hydrateMissingIndexTokenCounts to tokenMap utility
Extracted the hydrateMissingIndexTokenCounts function from the AgentClient and related tests into a new tokenMap utility file. This change improves code organization and reusability, allowing for better management of token counting logic across the application.
refactor(summarization): standardize step event handling and improve summary rendering
Refactored the step event handling in the useStepHandler and related components to utilize constants for event names, enhancing consistency and maintainability. Additionally, improved the rendering logic in the Summary component to conditionally display the summary text based on its availability, providing a better user experience during the summarization process.
feat(summarization): introduce baseContextTokens and reserveTokensRatio for improved context management
Added baseContextTokens to the InitializedAgent type to calculate the context budget based on agentMaxContextNum and maxOutputTokensNum. Implemented reserveTokensRatio in the createRun function to allow configurable context token management. Updated related tests to validate these changes and ensure proper functionality.
feat(summarization): add minReserveTokens, context pruning, and overflow recovery configurations
Introduced new configuration options for summarization, including minReserveTokens, context pruning settings, and overflow recovery parameters. Updated the createRun function to accommodate these new options and added a comprehensive test suite to validate their functionality and integration within the summarization process.
feat(summarization): add updatePrompt and reserveTokensRatio to summarization configuration
Introduced an updatePrompt field for updating existing summaries with new messages, enhancing the flexibility of the summarization process. Additionally, added reserveTokensRatio to the configuration schema, allowing for improved management of token allocation during summarization. Updated related tests to validate these new features.
feat(logging): add on_agent_log event handler for structured logging
Implemented an on_agent_log event handler in both the agents' callbacks and responses to facilitate structured logging of agent activities. This enhancement allows for better tracking and debugging of agent interactions by logging messages with associated metadata. Updated the summarization process to ensure proper handling of log events.
fix: remove duplicate IBalanceUpdate interface declaration
perf(usage): single-pass partition of collectedUsage
Replace two Array.filter() passes with a single for-of loop that
partitions message vs. summarization usages in one iteration.
fix(BaseClient): shallow-copy message content before mutating and preserve string content
Avoid mutating the original message.content array in-place when
appending a summary block. Also convert string content to a text
content part instead of silently discarding it.
fix(ui): fix Part.tsx indentation and useStepHandler summarize-complete handling
- Fix SUMMARY else-if branch indentation in Part.tsx to match chain level
- Guard ON_SUMMARIZE_COMPLETE with didFinalize flag to avoid unnecessary
re-renders when no summarizing parts exist
- Protect against undefined completeData.summary instead of unsafe spread
fix(agents): use strict enabled check for summarization handlers
Change summarizationConfig?.enabled !== false to === true so handlers
are not registered when summarizationConfig is undefined.
chore: fix initializeClient JSDoc and move DEFAULT_RESERVE_RATIO to module scope
refactor(Summary): align collapse/expand behavior with Reasoning component
- Single render path instead of separate streaming vs completed branches
- Use useMessageContext for isSubmitting/isLatestMessage awareness so
the "Summarizing..." label only shows during active streaming
- Default to collapsed (matching Reasoning), user toggles to expand
- Add proper aria attributes (aria-hidden, role, aria-controls, contentId)
- Hide copy button while actively streaming
feat(summarization): default to self-summarize using agent's own provider/model
When no summarization config is provided (neither in librechat.yaml nor
on the agent), automatically enable summarization using the agent's own
provider and model. The agents package already provides default prompts,
so no prompt configuration is needed.
Also removes the dead resolveSummarizationLLMConfig in summarize.ts
(and its spec) — run.ts buildAgentContext is the single source of truth
for summarization config resolution. Removes the duplicate
RuntimeSummarizationConfig local type in favor of the canonical
SummarizationConfig from data-provider.
chore: schema and type cleanup for summarization
- Add trigger field to summarizationAgentOverrideSchema so per-agent
trigger overrides in librechat.yaml are not silently stripped by Zod
- Remove unused SummarizationStatus type from runs.ts
- Make AppSummarizationConfig.enabled non-optional to reflect the
invariant that loadSummarizationConfig always sets it
refactor(responses): extract duplicated on_agent_log handler
refactor(run): use agents package types for summarization config
Import SummarizationConfig, ContextPruningConfig, and
OverflowRecoveryConfig from @librechat/agents and use them to
type-check the translation layer in buildAgentContext. This ensures
the config object passed to the agent graph matches what it expects.
- Use `satisfies AgentSummarizationConfig` on the config object
- Cast contextPruningConfig and overflowRecoveryConfig to agents types
- Properly narrow trigger fields from DeepPartial to required shape
feat(config): add maxToolResultChars to base endpoint schema
Add maxToolResultChars to baseEndpointSchema so it can be configured
on any endpoint in librechat.yaml. Resolved during agent initialization
using getProviderConfig's endpoint resolution: custom endpoint config
takes precedence, then the provider-specific endpoint config, then the
shared `all` config.
Passed through to the agents package ToolNode, which uses it to cap
tool result length before it enters the context window. When not
configured, the agents package computes a sensible default from
maxContextTokens.
fix(summarization): forward agent model_parameters in self-summarize default
When no explicit summarization config exists, the self-summarize
default now forwards the agent's model_parameters as the
summarization parameters. This ensures provider-specific settings
(e.g. Bedrock region, credentials, endpoint host) are available
when the agents package constructs the summarization LLM.
fix(agents): register summarization handlers by default
Change the enabled gate from === true to !== false so handlers
register when no explicit summarization config exists. This aligns
with the self-summarize default where summarization is always on
unless explicitly disabled via enabled: false.
refactor(summarization): let agents package inherit clientOptions for self-summarize
Remove model_parameters forwarding from the self-summarize default.
The agents package now reuses the agent's own clientOptions when the
summarization provider matches the agent's provider, inheriting all
provider-specific settings (region, credentials, proxy, etc.)
automatically.
refactor(summarization): use MessageContentComplex[] for summary content
Unify summary content to always use MessageContentComplex[] arrays,
matching the pattern used by on_message_delta. No more string | array
unions — content is always an array of typed blocks ({ type: 'text',
text: '...' } for text, { type: 'reasoning_content', ... } for
reasoning).
Agents package:
- SummaryContentBlock.content: MessageContentComplex[] (was string)
- tokenCount now optional (not sent on deltas)
- Removed reasoning field — reasoning is now a content block type
- streamAndCollect normalizes all chunks to content block arrays
- Delta events pass content blocks directly
LibreChat:
- SummaryContentPart.content: Agents.MessageContentComplex[]
- Updated Part.tsx, Summary.tsx, useStepHandler.ts, BaseClient.js
- Summary.tsx derives display text from content blocks via useMemo
- Aggregator uses simple array spread
refactor(summarization): enhance summary handling and text extraction
- Updated BaseClient.js to improve summary text extraction, accommodating both legacy and new content formats.
- Modified summarization logic to ensure consistent handling of summary content across different message formats.
- Adjusted test cases in summarization.e2e.spec.js to utilize the new summary text extraction method.
- Refined SSE useStepHandler to initialize summary content as an array.
- Updated configuration schema by removing unused minReserveTokens field.
- Cleaned up SummaryContentPart type by removing rangeHash property.
These changes streamline the summarization process and ensure compatibility with various content structures.
refactor(summarization): streamline usage tracking and logging
- Removed direct checks for summarization nodes in ModelEndHandler and replaced them with a dedicated markSummarizationUsage function for better readability and maintainability.
- Updated OpenAIChatCompletionController and responses handlers to utilize the new markSummarizationUsage function for setting usage types.
- Enhanced logging functionality by ensuring the logger correctly handles different log levels.
- Introduced a new useCopyToClipboard hook in the Summary component to encapsulate clipboard copy logic, improving code reusability and clarity.
These changes improve the overall structure and efficiency of the summarization handling and logging processes.
refactor(summarization): update summary content block documentation
- Removed outdated comment regarding the last summary content block in BaseClient.js.
- Added a new comment to clarify the purpose of the findSummaryContentBlock method, ensuring consistency in documentation.
These changes enhance code clarity and maintainability by providing accurate descriptions of the summarization logic.
refactor(summarization): update summary content structure in tests
- Modified the summarization content structure in e2e tests to use an array format for text, aligning with recent changes in summary handling.
- Updated test descriptions to clarify the behavior of context token calculations, ensuring consistency and clarity in the tests.
These changes enhance the accuracy and maintainability of the summarization tests by reflecting the updated content structure.
refactor(summarization): remove legacy E2E test setup and configuration
- Deleted the e2e-setup.js and jest.e2e.config.js files, which contained legacy configurations for E2E tests using real API keys.
- Introduced a new summarization.e2e.ts file that implements comprehensive E2E backend integration tests for the summarization process, utilizing real AI providers and tracking summaries throughout the run.
These changes streamline the testing framework by consolidating E2E tests into a single, more robust file while removing outdated configurations.
refactor(summarization): enhance E2E tests and error handling
- Added a cleanup step to force exit after all tests to manage Redis connections.
- Updated the summarization model to 'claude-haiku-4-5-20251001' for consistency across tests.
- Improved error handling in the processStream function to capture and return processing errors.
- Enhanced logging for cross-run tests and tight context scenarios to provide better insights into test execution.
These changes improve the reliability and clarity of the E2E tests for the summarization process.
refactor(summarization): enhance test coverage for maxContextTokens behavior
- Updated run-summarization.test.ts to include a new test case ensuring that maxContextTokens does not exceed user-defined limits, even when calculated ratios suggest otherwise.
- Modified summarization.e2e.ts to replace legacy UsageMetadata type with a more appropriate type for collectedUsage, improving type safety and clarity in the test setup.
These changes improve the robustness of the summarization tests by validating context token constraints and refining type definitions.
feat(summarization): add comprehensive E2E tests for summarization process
- Introduced a new summarization.e2e.test.ts file that implements extensive end-to-end integration tests for the summarization pipeline, covering the full flow from LibreChat to agents.
- The tests utilize real AI providers and include functionality to track summaries during and between runs.
- Added necessary cleanup steps to manage Redis connections post-tests and ensure proper exit.
These changes enhance the testing framework by providing robust coverage for the summarization process, ensuring reliability and performance under real-world conditions.
fix(service): import logger from winston configuration
- Removed the import statement for logger from '@librechat/data-schemas' and replaced it with an import from '~/config/winston'.
- This change ensures that the logger is correctly sourced from the updated configuration, improving consistency in logging practices across the application.
refactor(summary): simplify Summary component and enhance token display
- Removed the unused `meta` prop from the `SummaryButton` component to streamline its interface.
- Updated the token display logic to use a localized string for better internationalization support.
- Adjusted the rendering of the `meta` information to improve its visibility within the `Summary` component.
These changes enhance the clarity and usability of the Summary component while ensuring better localization practices.
feat(summarization): add maxInputTokens configuration for summarization
- Introduced a new `maxInputTokens` property in the summarization configuration schema to control the amount of conversation context sent to the summarizer, with a default value of 10000.
- Updated the `createRun` function to utilize the new `maxInputTokens` setting, allowing for more flexible summarization based on agent context.
These changes enhance the summarization capabilities by providing better control over input token limits, improving the overall summarization process.
refactor(summarization): simplify maxInputTokens logic in createRun function
- Updated the logic for the `maxInputTokens` property in the `createRun` function to directly use the agent's base context tokens when the resolved summarization configuration does not specify a value.
- This change streamlines the configuration process and enhances clarity in how input token limits are determined for summarization.
These modifications improve the maintainability of the summarization configuration by reducing complexity in the token calculation logic.
feat(summary): enhance Summary component to display meta information
- Updated the SummaryContent component to accept an optional `meta` prop, allowing for additional contextual information to be displayed above the main content.
- Adjusted the rendering logic in the Summary component to utilize the new `meta` prop, improving the visibility of supplementary details.
These changes enhance the user experience by providing more context within the Summary component, making it clearer and more informative.
refactor(summarization): standardize reserveRatio configuration in summarization logic
- Replaced instances of `reserveTokensRatio` with `reserveRatio` in the `createRun` function and related tests to unify the terminology across the codebase.
- Updated the summarization configuration schema to reflect this change, ensuring consistency in how the reserve ratio is defined and utilized.
- Removed the per-agent override logic for summarization configuration, simplifying the overall structure and enhancing clarity.
These modifications improve the maintainability and readability of the summarization logic by standardizing the configuration parameters.
* fix: circular dependency of `~/models`
* chore: update logging scope in agent log handlers
Changed log scope from `[agentus:${data.scope}]` to `[agents:${data.scope}]` in both the callbacks and responses controllers to ensure consistent logging format across the application.
* feat: calibration ratio
* refactor(tests): update summarizationConfig tests to reflect changes in enabled property
Modified tests to check for the new `summarizationEnabled` property instead of the deprecated `enabled` field in the summarization configuration. This change ensures that the tests accurately validate the current configuration structure and behavior of the agents.
* feat(tests): add markSummarizationUsage mock for improved test coverage
Introduced a mock for the markSummarizationUsage function in the responses unit tests to enhance the testing of summarization usage tracking. This addition supports better validation of summarization-related functionalities and ensures comprehensive test coverage for the agents' response handling.
* refactor(tests): simplify event handler setup in createResponse tests
Removed redundant mock implementations for event handlers in the createResponse unit tests, streamlining the setup process. This change enhances test clarity and maintainability while ensuring that the tests continue to validate the correct behavior of usage tracking during on_chat_model_end events.
* refactor(agents): move calibration ratio capture to finally block
Reorganized the logic for capturing the calibration ratio in the AgentClient class to ensure it is executed in the finally block. This change guarantees that the ratio is captured even if the run is aborted, enhancing the reliability of the response message persistence. Removed redundant code and improved clarity in the handling of context metadata.
* refactor(agents): streamline bulk write logic in recordCollectedUsage function
Removed redundant bulk write operations and consolidated document handling in the recordCollectedUsage function. The logic now combines all documents into a single bulk write operation, improving efficiency and reducing error handling complexity. Updated logging to provide consistent error messages for bulk write failures.
* refactor(agents): enhance summarization configuration resolution in createRun function
Streamlined the summarization configuration logic by introducing a base configuration and allowing for overrides from agent-specific settings. This change improves clarity and maintainability, ensuring that the summarization configuration is consistently applied while retaining flexibility for customization. Updated the handling of summarization parameters to ensure proper integration with the agent's model and provider settings.
* refactor(agents): remove unused tokenCountMap and streamline calibration ratio handling
Eliminated the unused tokenCountMap variable from the AgentClient class to enhance code clarity. Additionally, streamlined the logic for capturing the calibration ratio by using optional chaining and a fallback value, ensuring that context metadata is consistently defined. This change improves maintainability and reduces potential confusion in the codebase.
* refactor(agents): extract agent log handler for improved clarity and reusability
Refactored the agent log handling logic by extracting it into a dedicated function, `agentLogHandler`, enhancing code clarity and reusability across different modules. Updated the event handlers in both the OpenAI and responses controllers to utilize the new handler, ensuring consistent logging behavior throughout the application.
* test: add summarization event tests for useStepHandler
Implemented a series of tests for the summarization events in the useStepHandler hook. The tests cover scenarios for ON_SUMMARIZE_START, ON_SUMMARIZE_DELTA, and ON_SUMMARIZE_COMPLETE events, ensuring proper handling of summarization logic, including message accumulation and finalization. This addition enhances test coverage and validates the correct behavior of the summarization process within the application.
* refactor(config): update summarizationTriggerSchema to use enum for type validation
Changed the type of the `type` field in the summarizationTriggerSchema from a string to an enum with a single value 'token_count'. This modification enhances type safety and ensures that only valid types are accepted in the configuration, improving overall clarity and maintainability of the schema.
* test(usage): add bulk write tests for message and summarization usage
Implemented tests for the bulk write functionality in the recordCollectedUsage function, covering scenarios for combined message and summarization usage, summarization-only usage, and message-only usage. These tests ensure correct document handling and token rollup calculations, enhancing test coverage and validating the behavior of the usage tracking logic.
* refactor(Chat): enhance clipboard copy functionality and type definitions in Summary component
Updated the Summary component to improve the clipboard copy functionality by handling clipboard permission errors. Refactored type definitions for SummaryProps to use a more specific type, enhancing type safety. Adjusted the SummaryButton and FloatingSummaryBar components to accept isCopied and onCopy props, promoting better separation of concerns and reusability.
* chore(translations): remove unused "Expand Summary" key from English translations
Deleted the "Expand Summary" key from the English translation file to streamline the localization resources and improve clarity in the user interface. This change helps maintain an organized and efficient translation structure.
* refactor: adjust token counting for Claude model to account for API discrepancies
Implemented a correction factor for token counting when using the Claude model, addressing discrepancies between Anthropic's API and local tokenizer results. This change ensures accurate token counts by applying a scaling factor, improving the reliability of token-related functionalities.
* refactor(agents): implement token count adjustment for Claude model messages
Added a method to adjust token counts for messages processed by the Claude model, applying a correction factor to align with API expectations. This enhancement improves the accuracy of token counting, ensuring reliable functionality when interacting with the Claude model.
* refactor(agents): token counting for media content in messages
Introduced a new method to estimate token costs for image and document blocks in messages, improving the accuracy of token counting. This enhancement ensures that media content is properly accounted for, particularly for the Claude model, by integrating additional token estimation logic for various content types. Updated the token counting function to utilize this new method, enhancing overall reliability and functionality.
* chore: fix missing import
* fix(agents): clamp baseContextTokens and document reserve ratio change
Prevent negative baseContextTokens when maxOutputTokens exceeds the
context window (misconfigured models). Document the 10%→5% default
reserve ratio reduction introduced alongside summarization.
* fix(agents): include media tokens in hydrated token counts
Add estimateMediaTokensForMessage to createTokenCounter so the hydration
path (used by hydrateMissingIndexTokenCounts) matches the precomputed
path in AgentClient.getTokenCountForMessage. Without this, messages
containing images or documents were systematically undercounted during
hydration, risking context window overflow.
Add 34 unit tests covering all block-type branches of
estimateMediaTokensForMessage.
* fix(agents): include summarization output tokens in usage return value
The returned output_tokens from recordCollectedUsage now reflects all
billed LLM calls (message + summarization). Previously, summarization
completions were billed but excluded from the returned metadata, causing
a discrepancy between what users were charged and what the response
message reported.
* fix(tests): replace process.exit with proper Redis cleanup in e2e test
The summarization E2E test used process.exit(0) to work around a Redis
connection opened at import time, which killed the Jest runner and
bypassed teardown. Use ioredisClient.quit() and keyvRedisClient.disconnect()
for graceful cleanup instead.
* fix(tests): update getConvo imports in OpenAI and response tests
Refactor test files to import getConvo from the main models module instead of the Conversation submodule. This change ensures consistency across tests and simplifies the import structure, enhancing maintainability.
* fix(clients): improve summary text validation in BaseClient
Refactor the summary extraction logic to ensure that only non-empty summary texts are considered valid. This change enhances the robustness of the message processing by utilizing a dedicated method for summary text retrieval, improving overall reliability.
* fix(config): replace z.any() with explicit union in summarization schema
Model parameters (temperature, top_p, etc.) are constrained to
primitive types rather than the policy-violating z.any().
* refactor(agents): deduplicate CLAUDE_TOKEN_CORRECTION constant
Export from the TS source in packages/api and import in the JS client,
eliminating the static class property that could drift out of sync.
* refactor(agents): eliminate duplicate selfProvider in buildAgentContext
selfProvider and provider were derived from the same expression with
different type casts. Consolidated to a single provider variable.
* refactor(agents): extract shared SSE handlers and restrict log levels
- buildSummarizationHandlers() factory replaces triplicated handler
blocks across responses.js and openai.js
- agentLogHandlerObj exported from callbacks.js for consistent reuse
- agentLogHandler restricted to an allowlist of safe log levels
(debug, info, warn, error) instead of accepting arbitrary strings
* fix(SSE): batch summarize deltas, add exhaustiveness check, conditional error announcement
- ON_SUMMARIZE_DELTA coalesces rapid-fire renders via requestAnimationFrame
instead of calling setMessages per chunk
- Exhaustive never-check on TStepEvent catches unhandled variants at
compile time when new StepEvents are added
- ON_SUMMARIZE_COMPLETE error announcement only fires when a summary
part was actually present and removed
* feat(agents): persist instruction overhead in contextMeta and seed across runs
Extend contextMeta with instructionOverhead and toolCount so the
provider-observed instruction overhead is persisted on the response message
and seeded into the pruner on subsequent runs. This enables the pruner to
use a calibrated budget from the first call instead of waiting for a
provider observation, preventing the ratio collapse caused by local
tokenizer overestimating tool schema tokens.
The seeded overhead is only used when encoding and tool count match
between runs, ensuring stale values from different configurations
are discarded.
* test(agents): enhance OpenAI test mocks for summarization handlers
Updated the OpenAI test suite to include additional mock implementations for summarization handlers, including buildSummarizationHandlers, markSummarizationUsage, and agentLogHandlerObj. This improves test coverage and ensures consistent behavior during testing.
* fix(agents): address review findings for summarization v2
Cancel rAF on unmount to prevent stale Recoil writes from dead
component context. Clear orphaned summarizing:true parts when
ON_SUMMARIZE_COMPLETE arrives without a summary payload. Add null
guard and safe spread to agentLogHandler. Handle Anthropic-format
base64 image/* documents in estimateMediaTokensForMessage. Use
role="region" for expandable summary content. Add .describe() to
contextMeta Zod fields. Extract duplicate usage loop into helper.
* refactor: simplify contextMeta to calibrationRatio + encoding only
Remove instructionOverhead and toolCount from cross-run persistence —
instruction tokens change too frequently between runs (prompt edits,
tool changes) for a persisted seed to be reliable. The intra-run
calibration in the pruner still self-corrects via provider observations.
contextMeta now stores only the tokenizer-bias ratio and encoding,
which are stable across instruction changes.
* test(SSE): enhance useStepHandler tests for ON_SUMMARIZE_COMPLETE behavior
Updated the test for ON_SUMMARIZE_COMPLETE to clarify that it finalizes the existing part with summarizing set to false when the summary is undefined. Added assertions to verify the correct behavior of message updates and the state of summary parts.
* refactor(BaseClient): remove handleContextStrategy and truncateToolCallOutputs functions
Eliminated the handleContextStrategy method from BaseClient to streamline message handling. Also removed the truncateToolCallOutputs function from the prompts module, simplifying the codebase and improving maintainability.
* refactor: add AGENT_DEBUG_LOGGING option and refactor token count handling in BaseClient
Introduced AGENT_DEBUG_LOGGING to .env.example for enhanced debugging capabilities. Refactored token count handling in BaseClient by removing the handleTokenCountMap method and simplifying token count updates. Updated AgentClient to log detailed token count recalculations and adjustments, improving traceability during message processing.
* chore: update dependencies in package-lock.json and package.json files
Bumped versions of several dependencies, including @librechat/agents to ^3.1.62 and various AWS SDK packages to their latest versions. This ensures compatibility and incorporates the latest features and fixes.
* chore: imports order
* refactor: extract summarization config resolution from buildAgentContext
* refactor: rename and simplify summarization configuration shaping function
* refactor: replace AgentClient token counting methods with single-pass pure utility
Extract getTokenCount() and getTokenCountForMessage() from AgentClient
into countFormattedMessageTokens(), a pure function in packages/api that
handles text, tool_call, image, and document content types in one loop.
- Decompose estimateMediaTokensForMessage into block-level helpers
(estimateImageDataTokens, estimateImageBlockTokens, estimateDocumentBlockTokens)
shared by both estimateMediaTokensForMessage and the new single-pass function
- Remove redundant per-call getEncoding() resolution (closure captures once)
- Remove deprecated gpt-3.5-turbo-0301 model branching
- Drop this.getTokenCount guard from BaseClient.sendMessage
* refactor: streamline token counting in createTokenCounter function
Simplified the createTokenCounter function by removing the media token estimation and directly calculating the token count. This change enhances clarity and performance by consolidating the token counting logic into a single pass, while maintaining compatibility with Claude's token correction.
* refactor: simplify summarization configuration types
Removed the AppSummarizationConfig type and directly used SummarizationConfig in the AppConfig interface. This change streamlines the type definitions and enhances consistency across the codebase.
* chore: import order
* fix: summarization event handling in useStepHandler
- Cancel pending summarizeDeltaRaf in clearStepMaps to prevent stale
frames firing after map reset or component unmount
- Move announcePolite('summarize_completed') inside the didFinalize
guard so screen readers only announce when finalization actually occurs
- Remove dead cleanup closure returned from stepHandler useCallback body
that was never invoked by any caller
* fix: estimate tokens for non-PDF/non-image base64 document blocks
Previously estimateDocumentBlockTokens returned 0 for unrecognized MIME
types (e.g. text/plain, application/json), silently underestimating
context budget. Fall back to character-based heuristic or countTokens.
* refactor: return cloned usage from markSummarizationUsage
Avoid mutating LangChain's internal usage_metadata object by returning
a shallow clone with the usage_type tag. Update all call sites in
callbacks, openai, and responses controllers to use the returned value.
* refactor: consolidate debug logging loops in buildMessages
Merge the two sequential O(n) debug-logging passes over orderedMessages
into a single pass inside the map callback where all data is available.
* refactor: narrow SummaryContentPart.content type
Replace broad Agents.MessageContentComplex[] with the specific
Array<{ type: ContentTypes.TEXT; text: string }> that all producers
and consumers already use, improving compile-time safety.
* refactor: use single output array in recordCollectedUsage
Have processUsageGroup append to a shared array instead of returning
separate arrays that are spread into a third, reducing allocations.
* refactor: use for...in in hydrateMissingIndexTokenCounts
Replace Object.entries with for...in to avoid allocating an
intermediate tuple array during token map hydration.
|
||
|
Danny Avila
|
6976414464
|
🗣️ a11y: Distinguish Conversation Headings for Screen Readers (#12341)
* fix: distinguish message headings for screen readers Before, each message would have the heading of either the name of the user or the name of the agent (e.g. "Dan Lew" or "Claude Sonnet"). If you tried to navigate that with a screen reader, you'd just see a ton of headings switching back and forth between the two with no way to figure out where in the conversation each is. Now, we prefix each header with whether it's a "prompt" or "response", plus we number them so that you can distinguish how far in the conversation each part is. (This is a screen reader only change - there's no visual difference.) * fix: patch MessageParts heading, guard negative depth, add tests - Add sr-only heading prefix to MessageParts.tsx (Assistants endpoint path) - Extract shared getMessageNumber helper to avoid DRY violation between getMessageAriaLabel and getHeaderPrefixForScreenReader - Guard against depth < 0 producing "Prompt 0:" / "Response 0:" - Remove unused lodash import - Add unit tests covering all branches including depth edge cases --------- Co-authored-by: Dan Lew <daniel@mightyacorn.com> |
||
|
Danny Avila
|
b93d60c416
|
🎞️ refactor: Image Rendering with Preview Caching and Layout Reservation (#12114)
* refactor: Update Image Component to Remove Lazy Loading and Enhance Rendering - Removed the react-lazy-load-image-component dependency from the Image component, simplifying the image loading process. - Updated the Image component to use a standard <img> tag with async decoding for improved performance and user experience. - Adjusted related tests to reflect changes in image rendering behavior and ensure proper functionality without lazy loading. * refactor: Enhance Image Handling and Caching Across Components - Introduced a new previewCache utility for managing local blob preview URLs, improving image loading efficiency. - Updated the Image component and related parts (FileRow, Files, Part, ImageAttachment, LogContent) to utilize cached previews, enhancing rendering performance and user experience. - Added width and height properties to the Image component for better layout management and consistency across different usages. - Improved file handling logic in useFileHandling to cache previews during file uploads, ensuring quick access to image data. - Enhanced overall code clarity and maintainability by streamlining image rendering logic and reducing redundancy. * refactor: Enhance OpenAIImageGen Component with Image Dimensions - Added width and height properties to the OpenAIImageGen component for improved image rendering and layout management. - Updated the Image component usage within OpenAIImageGen to utilize the new dimensions, enhancing visual consistency and performance. - Improved code clarity by destructuring additional properties from the attachment object, streamlining the component's logic. * refactor: Implement Image Size Caching in DialogImage Component - Introduced an imageSizeCache to store and retrieve image sizes, enhancing performance by reducing redundant fetch requests. - Updated the getImageSize function to first check the cache before making network requests, improving efficiency in image handling. - Added decoding attribute to the image element for optimized rendering behavior. * refactor: Enhance UserAvatar Component with Avatar Caching and Error Handling - Introduced avatar caching logic to optimize avatar resolution based on user ID and avatar source, improving performance and reducing redundant image loads. - Implemented error handling for failed image loads, allowing for fallback to a default avatar when necessary. - Updated UserAvatar props to streamline the interface by removing the user object and directly accepting avatar-related properties. - Enhanced overall code clarity and maintainability by refactoring the component structure and logic. * fix: Layout Shift in Message and Placeholder Components for Consistent Height Management - Adjusted the height of the PlaceholderRow and related message components to ensure consistent rendering with a minimum height of 31px. - Updated the MessageParts and ContentRender components to utilize a minimum height for better layout stability. - Enhanced overall code clarity by standardizing the structure of message-related components. * tests: Update FileRow Component to Prefer Cached Previews for Image Rendering - Modified the image URL selection logic in the FileRow component to prioritize cached previews over file paths when uploads are complete, enhancing rendering performance and user experience. - Updated related tests to reflect changes in image URL handling, ensuring accurate assertions for both preview and file path scenarios. - Introduced a fallback mechanism to use file paths when no preview exists, improving robustness in file handling. * fix: Image cache lifecycle and dialog decoding - Add deletePreview/clearPreviewCache to previewCache.ts for blob URL cleanup - Wire deletePreview into useFileDeletion to revoke blobs on file delete - Move dimensionCache.set into useMemo to avoid side effects during render - Extract IMAGE_MAX_W_PX constant (512) to document coupling with max-w-lg - Export _resetImageCaches for test isolation - Change DialogImage decoding from "sync" to "async" to avoid blocking main thread * fix: Avatar cache invalidation and cleanup - Include avatarSrc in cache invalidation to prevent stale avatars - Remove unused username parameter from resolveAvatar - Skip caching when userId is empty to prevent cache key collisions * test: Fix test isolation and type safety - Reset module-level dimensionCache/paintedUrls in beforeEach via _resetImageCaches - Replace any[] with typed mock signature in cn mock for both test files * chore: Code quality improvements from review - Use barrel imports for previewCache in Files.tsx and Part.tsx - Single Map.get with truthy check instead of has+get in useEventHandlers - Add JSDoc comments explaining EmptyText margin removal and PlaceholderRow height - Fix FileRow toast showing "Deleting file" when file isn't actually deleted (progress < 1) * fix: Address remaining review findings (R1-R3) - Add deletePreview calls to deleteFiles batch path to prevent blob URL leaks - Change useFileDeletion import from deep path to barrel (~/utils) - Change useMemo to useEffect for dimensionCache.set (side effect, not derived value) * fix: Address audit comments 2, 5, and 7 - Fix files preservation to distinguish null (missing) from [] (empty) in finalHandler - Add auto-revoke on overwrite in cachePreview to prevent leaked blobs - Add removePreviewEntry for key transfer without revoke - Clean up stale temp_file_id cache entry after promotion to permanent file_id |
||
|
Carolina
|
3a73907daa
|
📐 fix: Replace JS Image Scaling with CSS Viewport Constraints (#12089)
* fix: remove scaleImage function that stretched vertical images * chore: lint * refactor: Simplify Image Component Usage Across Chat Parts - Removed height and width props from the Image component in various parts (Files, Part, ImageAttachment, LogContent) to streamline image rendering. - Introduced a constant for maximum image height in the Image component for consistent styling. - Updated related components to utilize the new simplified Image component structure, enhancing maintainability and reducing redundancy. * refactor: Simplify LogContent and Enhance Image Component Tests - Removed height and width properties from the ImageAttachment type in LogContent for cleaner code. - Updated the image rendering logic to rely solely on the filepath, improving clarity. - Enhanced the Image component tests with additional assertions for rendering behavior and accessibility. - Introduced new tests for OpenAIImageGen to validate image preloading and progress handling, ensuring robust functionality. --------- Co-authored-by: Danny Avila <danny@librechat.ai> |
||
|
Danny Avila
|
5209f1dc9e
|
⚡ refactor: Optimize Message Re-renders (#12097)
* 🔄 refactor: Update Artifacts and Messages Contexts to Use Latest Message ID and Depth
- Modified ArtifactsContext to retrieve latestMessage using Recoil state management.
- Updated MessagesViewContext to replace latestMessage with latestMessageId and latestMessageDepth for improved clarity and consistency.
- Adjusted various components (HoverButtons, MessageParts, MessageRender, ContentRender) to utilize latestMessageId instead of the entire message object, enhancing performance and reducing unnecessary re-renders.
- Refactored useChatHelpers to extract latestMessageId and latestMessageDepth, streamlining message handling across the application.
* refactor: Introduce PartWithContext Component for Optimized Message Rendering
- Added a new PartWithContext component to encapsulate message part rendering logic, improving context management and reducing redundancy in the ContentParts component.
- Updated MessageRender to utilize the new PartWithContext, streamlining the context provider setup and enhancing code clarity.
- Refactored related logic to ensure proper context values are passed, improving maintainability and performance in message rendering.
* refactor: Update Components to Use Function Declarations and Improve Readability
- Refactored several components (MessageContainer, Markdown, MarkdownCode, MarkdownCodeNoExecution, MarkdownAnchor, MarkdownParagraph, MarkdownImage, TextPart, PlaceholderRow) to use function declarations instead of arrow functions, enhancing readability and consistency across the codebase.
- Added display names to memoized components for better debugging and profiling in React DevTools.
- Improved overall code clarity and maintainability by standardizing component definitions.
* refactor: Standardize MessageRender and ContentRender Components for Improved Clarity
- Refactored MessageRender and ContentRender components to use function declarations, enhancing readability and consistency.
- Streamlined props handling by removing unnecessary parameters and improving the use of hooks for state management.
- Updated memoization and rendering logic to optimize performance and reduce unnecessary re-renders.
- Enhanced overall code clarity and maintainability by standardizing component definitions and structure.
* refactor: Enhance Header Component with Memoization for Performance
- Refactored the Header component to utilize React's memoization by wrapping it with the memo function, improving rendering performance by preventing unnecessary re-renders.
- Changed the export to a memoized version of the Header component, ensuring better debugging with a display name.
- Maintained overall code clarity and consistency in component structure.
* refactor: Transition Components to Use Recoil for State Management
- Updated multiple components (AddMultiConvo, TemporaryChat, HeaderNewChat, PresetsMenu, ModelSelectorChatContext) to utilize Recoil for state management, enhancing consistency and performance.
- Replaced useChatContext with Recoil selectors and atoms, improving data flow and reducing unnecessary re-renders.
- Introduced new selectors for conversation ID and endpoint retrieval, streamlining component logic and enhancing maintainability.
- Improved overall code clarity by standardizing state management practices across components.
* refactor: Integrate getConversation Callback for Enhanced State Management
- Updated multiple components (Mention, ModelSelectorChatContext, ModelSelectorContext, FavoritesList) to utilize a getConversation callback instead of directly accessing conversation state, improving encapsulation and maintainability.
- Refactored useSelectMention hook to accept getConversation, streamlining conversation retrieval and enhancing code clarity.
- Introduced new Recoil selectors for conversation properties, ensuring consistent state management across components.
- Enhanced overall code structure by standardizing the approach to conversation handling, reducing redundancy and improving performance.
* refactor: Optimize LiveAnnouncer Context Value with useMemo
- Updated the LiveAnnouncer component to utilize useMemo for context value creation, enhancing performance by preventing unnecessary recalculations of the context object.
- Improved overall code clarity and maintainability by ensuring that context values are only recomputed when their dependencies change.
* refactor: Update AgentPanelSwitch to Use Recoil for Agent ID Management
- Refactored AgentPanelSwitch component to utilize Recoil for retrieving the current agent ID, replacing the previous use of chat context.
- Improved state management by ensuring the agent ID is derived from Recoil, enhancing code clarity and maintainability.
- Adjusted useEffect dependencies to reflect the new state management approach, streamlining the component's logic.
* refactor: Enhance useLocalize Hook with useCallback for Improved Performance
- Updated the useLocalize hook to utilize useCallback for the translation function, optimizing performance by preventing unnecessary re-creations of the function on each render.
- Improved code clarity by ensuring that the translation function is memoized, enhancing maintainability and efficiency in localization handling.
* refactor: Rename useCreateConversationAtom to useSetConversationAtom for Clarity
- Updated the hook name from useCreateConversationAtom to useSetConversationAtom to better reflect its functionality in managing conversation state.
- Introduced a new implementation for setting conversation state, enhancing clarity and maintainability in the codebase.
- Adjusted related references in the useNewConvo hook to align with the new naming convention.
* refactor: Enhance useKeyDialog Hook with useMemo and useCallback for Improved Performance
- Updated the useKeyDialog hook to utilize useMemo for returning the dialog state and handlers, optimizing performance by preventing unnecessary recalculations.
- Refactored the onOpenChange function to use useCallback, ensuring it only changes when its dependencies do, enhancing maintainability and clarity in the code.
- Improved overall code structure and readability by streamlining the hook's logic and dependencies.
* feat: Add useRenderChangeLog Hook for Debugging Render Changes
- Introduced a new hook, useRenderChangeLog, that logs changes in tracked values between renders when a debug flag is enabled.
- Utilizes useEffect and useRef to track previous values and identify changes, enhancing debugging capabilities for component renders.
- Provides detailed console output for initial renders and value changes, improving developer insights during the rendering process.
* refactor: Update useSelectAgent Hook for Improved State Management and Performance
- Refactored the useSelectAgent hook to utilize useRecoilCallback for fetching conversation data, enhancing state management and performance.
- Replaced the use of useChatContext with a more efficient approach, streamlining the logic for selecting agents and updating conversations.
- Improved error handling and ensured asynchronous operations are properly awaited, enhancing reliability in agent selection and data fetching processes.
* refactor: Optimize useDefaultConvo Hook with useCallback for Improved Performance
- Refactored the getDefaultConversation function within the useDefaultConvo hook to utilize useCallback, enhancing performance by memoizing the function and preventing unnecessary re-creations on re-renders.
- Streamlined the logic for cleaning input and output in the conversation object, improving code clarity and maintainability.
- Ensured that dependencies for useCallback are correctly set, enhancing the reliability of the hook's behavior.
* refactor: Optimize Agent Components with Memoization for Improved Performance
- Refactored multiple agent-related components (AgentAvatar, AgentCategorySelector, AgentSelect, DeleteButton, FileContext, FileSearch, Files) to utilize React.memo for memoization, enhancing rendering performance by preventing unnecessary re-renders.
- Updated the FileRow component to make setFilesLoading optional, improving flexibility in file handling.
- Streamlined component logic and improved maintainability by ensuring that props are compared efficiently in memoized components.
* refactor: Enhance File Handling and Agent Components for Improved Performance
- Refactored multiple components (DeleteButton, FileContext, FileSearch, Files) to utilize new file handling hooks that separate chat context from file operations, improving performance and maintainability.
- Introduced useFileHandlingNoChatContext and useSharePointFileHandlingNoChatContext hooks to streamline file handling logic, enhancing flexibility in managing file states.
- Updated DeleteButton to improve conversation state management and ensure proper handling of agent deletions, enhancing user experience.
- Optimized imports and component structure for better clarity and organization across the affected files.
* refactor: Enhance useRenderChangeLog Hook with Improved Type Safety and Documentation
- Updated the useRenderChangeLog hook to improve type safety by specifying the value types as string, number, boolean, null, or undefined.
- Enhanced documentation to clarify usage and enablement of the debug feature, ensuring better developer insights during rendering.
- Added a production check to prevent logging in production builds, optimizing performance and maintaining clean console output.
* chore: imports
* refactor: Replace useRecoilCallback with useGetConversation Hook for Improved Clarity and Performance
- Refactored multiple components (AddMultiConvo, ModelSelectorChatContext, FavoritesList, useSelectAgent, usePresets) to utilize the new useGetConversation hook, enhancing clarity and reducing complexity by eliminating the use of useRecoilCallback.
- Streamlined conversation retrieval logic across components, improving maintainability and performance.
- Updated imports and component structure for better organization and readability.
* refactor: Enhance Memoization in DeleteButton Component for Improved Performance
- Updated the memoization logic in the DeleteButton component to include a comparison for the setCurrentAgentId prop, ensuring more efficient re-renders.
- This change improves performance by preventing unnecessary updates when the agent ID and current agent ID remain unchanged.
* chore: fix test
* refactor: Improve Memoization Logic in AgentSelect Component
- Updated the memoization comparison in the AgentSelect component to directly compare agentQuery.data objects, enhancing performance by ensuring accurate re-renders.
- Refactored the useCreateConversationAtom function to streamline the logic for updating conversation keys, improving clarity and maintainability.
* refactor: Simplify State Management in DeleteButton Component
- Removed unnecessary setConversationOption function, streamlining the logic for updating conversation state after agent deletion.
- Updated the conversation state directly within the deleteAgent mutation, improving clarity and maintainability of the component.
- Refactored conversationByKeySelector to directly reference conversationByIndex, enhancing performance and reducing complexity in state retrieval.
* refactor: Remove Unused Conversation Prop from Mention Component
- Eliminated the conversation prop from the Mention component, simplifying its interface and reducing unnecessary dependencies.
- Updated the ChatForm component to reflect this change, enhancing clarity and maintainability of the codebase.
- Introduced useGetConversation hook for improved conversation retrieval logic, streamlining the component's functionality.
* refactor: Simplify File Handling State Management Across Components
- Removed the unused setFilesLoading function from FileContext, FileSearch, and Files components, streamlining the file handling state management.
- Updated the FileHandlingState type to make setFilesLoading optional, enhancing flexibility in file operations.
- Improved memoization logic by directly referencing necessary state properties, ensuring better performance and maintainability.
* refactor: Update ArtifactsContext for Improved State Management
- Replaced the useChatContext hook with direct Recoil state retrieval for isSubmitting, latestMessage, and conversationId, simplifying the context provider's logic.
- Enhanced memoization by ensuring relevant state properties are directly referenced, improving performance and maintainability.
- Streamlined the context value creation to reflect the updated state management approach.
* refactor: Adjust Memoization Logic in ArtifactsContext for Consistency
- Updated the memoization logic in the ArtifactsProvider to ensure the messageId is consistently referenced, improving clarity and maintainability.
- This change enhances the performance of the context provider by ensuring all relevant properties are included in the memoization dependencies.
|
||
|
Danny Avila
|
44dbbd5328
|
♿ a11y: Hide Collapsed Thinking Content From Screen Readers (#11927)
* fix(a11y): hide collapsed thinking content from screen readers and link toggle to controlled region
The thinking/reasoning toggle button visually collapsed content using a CSS
grid animation (gridTemplateRows: 0fr + overflow-hidden), but the content
remained in the DOM and fully accessible to screen readers, cluttering the
reading flow for assistive technology users.
- Add aria-hidden={!isExpanded} to the collapsible content region in both
the legacy Thinking component and the modern Reasoning component, so
screen readers skip collapsed thoughts entirely
- Add role="region" and a unique id (via useId) to each collapsible content
div, giving it a semantic landmark for assistive technology
- Add contentId prop to the shared ThinkingButton and wire it to
aria-controls on the toggle button, establishing an explicit relationship
between the button and the region it expands/collapses
- aria-expanded was already present on the button; combined with
aria-controls, screen readers can now fully convey the toggle state and
its target
* fix(a11y): add aria-label to collapsible content regions in Thinking and Reasoning components
Enhanced accessibility by adding aria-label attributes to the collapsible content regions in both the Thinking and Reasoning components. This change ensures that screen readers can provide better context for users navigating through the content.
* fix(a11y): update roles and aria attributes in Thinking and Reasoning components
Changed role from "region" to "group" for collapsible content areas in both Thinking and Reasoning components to better align with ARIA practices. Updated aria-hidden to handle undefined values correctly and ensured contentId is passed to relevant components for improved accessibility and screen reader support.
|
||
|
Danny Avila
|
bf1f2f4313
|
🗨️ refactor: Better Whitespace handling in Chat Message rendering (#11791)
- Updated the rendering logic in the Part component to handle whitespace-only text more effectively. - Introduced a placeholder for whitespace-only last parts during streaming to enhance user experience. - Ensured non-last whitespace-only parts are skipped to avoid rendering empty containers, improving layout stability. |
||
|
Danny Avila
|
599f4a11f1
|
🛡️ fix: Secure MCP/Actions OAuth Flows, Resolve Race Condition & Tool Cache Cleanup (#11756)
* 🔧 fix: Update OAuth error message for clarity - Changed the default error message in the OAuth error route from 'Unknown error' to 'Unknown OAuth error' to provide clearer context during authentication failures. * 🔒 feat: Enhance OAuth flow with CSRF protection and session management - Implemented CSRF protection for OAuth flows by introducing `generateOAuthCsrfToken`, `setOAuthCsrfCookie`, and `validateOAuthCsrf` functions. - Added session management for OAuth with `setOAuthSession` and `validateOAuthSession` middleware. - Updated routes to bind CSRF tokens for MCP and action OAuth flows, ensuring secure authentication. - Enhanced tests to validate CSRF handling and session management in OAuth processes. * 🔧 refactor: Invalidate cached tools after user plugin disconnection - Added a call to `invalidateCachedTools` in the `updateUserPluginsController` to ensure that cached tools are refreshed when a user disconnects from an MCP server after a plugin authentication update. This change improves the accuracy of tool data for users. * chore: imports order * fix: domain separator regex usage in ToolService - Moved the declaration of `domainSeparatorRegex` to avoid redundancy in the `loadActionToolsForExecution` function, improving code clarity and performance. * chore: OAuth flow error handling and CSRF token generation - Enhanced the OAuth callback route to validate the flow ID format, ensuring proper error handling for invalid states. - Updated the CSRF token generation function to require a JWT secret, throwing an error if not provided, which improves security and clarity in token generation. - Adjusted tests to reflect changes in flow ID handling and ensure robust validation across various scenarios. |
||
|
Sean DMR
|
8da3c38780
|
🪟 fix: Update Link Target to Open in Separate Tabs (#11669)
`_new` is not a recognized keyword for the `target` attribute. While browsers treat it as a named window, `_blank` is the standard value for opening links in a new tab/window. |
||
|
Danny Avila
|
75c02a1a18
|
🗂️ feat: Better Persistence for Code Execution Files Between Sessions (#11362)
* refactor: process code output files for re-use (WIP) * feat: file attachment handling with additional metadata for downloads * refactor: Update directory path logic for local file saving based on basePath * refactor: file attachment handling to support TFile type and improve data merging logic * feat: thread filtering of code-generated files - Introduced parentMessageId parameter in addedConvo and initialize functions to enhance thread management. - Updated related methods to utilize parentMessageId for retrieving messages and filtering code-generated files by conversation threads. - Enhanced type definitions to include parentMessageId in relevant interfaces for better clarity and usage. * chore: imports/params ordering * feat: update file model to use messageId for filtering and processing - Changed references from 'message' to 'messageId' in file-related methods for consistency. - Added messageId field to the file schema and updated related types. - Enhanced file processing logic to accommodate the new messageId structure. * feat: enhance file retrieval methods to support user-uploaded execute_code files - Added a new method `getUserCodeFiles` to retrieve user-uploaded execute_code files, excluding code-generated files. - Updated existing file retrieval methods to improve filtering logic and handle edge cases. - Enhanced thread data extraction to collect both message IDs and file IDs efficiently. - Integrated `getUserCodeFiles` into relevant endpoints for better file management in conversations. * chore: update @librechat/agents package version to 3.0.78 in package-lock.json and related package.json files * refactor: file processing and retrieval logic - Added a fallback mechanism for download URLs when files exceed size limits or cannot be processed locally. - Implemented a deduplication strategy for code-generated files based on conversationId and filename to optimize storage. - Updated file retrieval methods to ensure proper filtering by messageIds, preventing orphaned files from being included. - Introduced comprehensive tests for new thread data extraction functionality, covering edge cases and performance considerations. * fix: improve file retrieval tests and handling of optional properties - Updated tests to safely access optional properties using non-null assertions. - Modified test descriptions for clarity regarding the exclusion of execute_code files. - Ensured that the retrieval logic correctly reflects the expected outcomes for file queries. * test: add comprehensive unit tests for processCodeOutput functionality - Introduced a new test suite for the processCodeOutput function, covering various scenarios including file retrieval, creation, and processing for both image and non-image files. - Implemented mocks for dependencies such as axios, logger, and file models to isolate tests and ensure reliable outcomes. - Validated behavior for existing files, new file creation, and error handling, including size limits and fallback mechanisms. - Enhanced test coverage for metadata handling and usage increment logic, ensuring robust verification of file processing outcomes. * test: enhance file size limit enforcement in processCodeOutput tests - Introduced a configurable file size limit for tests to improve flexibility and coverage. - Mocked the `librechat-data-provider` to allow dynamic adjustment of file size limits during tests. - Updated the file size limit enforcement test to validate behavior when files exceed specified limits, ensuring proper fallback to download URLs. - Reset file size limit after tests to maintain isolation for subsequent test cases. |
||
|
Danny Avila
|
7c9c7e530b
|
⏲️ feat: Defer Loading MCP Tools (#11270)
* WIP: code ptc
* refactor: tool classification and calling logic
* 🔧 fix: Update @librechat/agents dependency to version 3.0.68
* chore: import order and correct renamed tool name for tool search
* refactor: streamline tool classification logic for local and programmatic tools
* feat: add per-tool configuration options for agents, including deferred loading and allowed callers
- Introduced `tool_options` in agent forms to manage tool behavior.
- Updated tool classification logic to prioritize agent-level configurations.
- Enhanced UI components to support tool deferral functionality.
- Added localization strings for new tool options and actions.
* feat: enhance agent schema with per-tool options for configuration
- Added `tool_options` schema to support per-tool configurations, including `defer_loading` and `allowed_callers`.
- Updated agent data model to incorporate new tool options, ensuring flexibility in tool behavior management.
- Modified type definitions to reflect the new `tool_options` structure for agents.
* feat: add tool_options parameter to loadTools and initializeAgent for enhanced agent configuration
* chore: update @librechat/agents dependency to version 3.0.71 and enhance agent tool loading logic
- Updated the @librechat/agents package to version 3.0.71 across multiple files.
- Added support for handling deferred loading of tools in agent initialization and execution processes.
- Improved the extraction of discovered tools from message history to optimize tool loading behavior.
* chore: update @librechat/agents dependency to version 3.0.72
* chore: update @librechat/agents dependency to version 3.0.75
* refactor: simplify tool defer loading logic in MCPTool component
- Removed local state management for deferred tools, relying on form state instead.
- Updated related functions to directly use form values for checking and toggling defer loading.
- Cleaned up code by eliminating unnecessary optimistic updates and local state dependencies.
* chore: remove deprecated localization strings for tool deferral in translation.json
- Eliminated unused strings related to deferred loading descriptions in the English translation file.
- Streamlined localization to reflect recent changes in tool loading logic.
* refactor: improve tool defer loading handling in MCPTool component
- Enhanced the logic for managing deferred loading of tools by simplifying the update process for tool options.
- Ensured that the state reflects the correct loading behavior based on the new deferred loading conditions.
- Cleaned up the code to remove unnecessary complexity in handling tool options.
* refactor: update agent mocks in callbacks test to use actual implementations
- Modified the agent mocks in the callbacks test to include actual implementations from the @librechat/agents module.
- This change enhances the accuracy of the tests by ensuring they reflect the real behavior of the agent functions.
|
||
|
Danny Avila
|
d21dfba2ac
|
🏞️ fix: Image Preview Refactor with Accessibility Enhancements (#11217)
Some checks are pending
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Waiting to run
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
* 🔧 fix: Prevent race condition by saving user messages before final event in ResumableAgentController
- Updated the ResumableAgentController to save user messages prior to sending the final event. This change addresses a potential race condition where the client might refetch data before the database is updated.
- Removed redundant message saving logic that was previously located after the final event handling, ensuring a more reliable message processing flow.
* style: improve image preview dialogs with ChatGPT-like UX and accessibility
Refactored image preview dialogs (DialogImage and ImagePreview) to provide
a cleaner, more intuitive user experience similar to ChatGPT's implementation.
## DialogImage.tsx (generated images)
- Replaced OGDialog/OGDialogContent with direct Radix Dialog primitives
for finer control over behavior
- Full-screen dark overlay (bg-black/90) that closes on click outside image
- Restructured component so all interactive elements (close, download,
details panel buttons) are inside DialogPrimitive.Content for proper
focus trap and keyboard navigation
- Added onOpenAutoFocus to focus close button when dialog opens
- Added onCloseAutoFocus to return focus to trigger element on close
- Added triggerRef prop to enable focus restoration
- Removed animate-in/animate-out classes that caused stuttering on open
- Changed transition-all to transition-[margin] to prevent animation jank
- Added proper TypeScript types for component props
## ImagePreview.tsx (uploaded file thumbnails)
- Same Radix Dialog primitive refactor for consistent behavior
- Click-outside-to-close functionality
- Proper focus management with closeButtonRef and triggerRef
- Made button the container element to prevent focus ring clipping
- Added focus-visible ring styling for keyboard navigation visibility
## Image.tsx (image display component)
- Restructured so button is the outer container instead of being nested
inside a div with overflow-hidden (which was clipping focus ring)
- Added visible focus-visible:ring styling with ring-offset
- Added aria-haspopup="dialog" for screen reader context
- Added triggerRef and passed to DialogImage for focus restoration
## Accessibility improvements
- Keyboard navigation now works properly (Tab cycles through buttons)
- Escape key closes dialog (or resets zoom if zoomed in)
- Focus is trapped within dialog when open
- Focus returns to trigger element when dialog closes
- Visible focus indicators on image buttons when focused via keyboard
- Proper ARIA attributes (aria-label, aria-haspopup, aria-hidden)
## UX improvements
- Click anywhere outside the image to close (not just specific regions)
- No more weird scroll/navigation issues
- Instant dialog open without stuttering animations
- Clean, minimal overlay without container/header chrome
* refactor: Improve click handling in image preview dialogs
Updated the click handling logic in ImagePreview and DialogImage components to ensure that the dialog only closes when clicking directly on the overlay or content background, enhancing user experience by preventing unintended closures when interacting with child elements. Additionally, clarified comments to reflect the new behavior.
* chore: import order
|
||
|
Joseph Licata
|
200098d992
|
🍌 feat: Gemini Image Generation Tool (Nano Banana) (#10676)
* Added fully functioning Agent Tool supporting Google's Nano Banana * 🔧 refactor: Update Google credentials handling in GeminiImageGen.js * Refactored the credentials path to follow a consistent pattern with other Google service integrations, allowing for an environment variable override. * Updated documentation in README-GeminiNanoBanana.md to reflect the new credentials handling approach and removed references to hardcoded paths. * 🛠️ refactor: Remove unnecessary whitespace in handleTools.js * 🔧 feat: Update Gemini Image Generation Tool - Bump @google/genai package version to ^1.19.0 for improved functionality. - Refactor GeminiImageGen to createGeminiImageTool for better clarity and consistency. - Enhance manifest.json for Gemini Image Tools with updated descriptions and icon. - Add SVG icon for Gemini Image Tools. - Implement progress tracking for Gemini image generation in the UI. - Introduce new toolkit and context handling for image generation tools. This update improves the Gemini image generation capabilities and user experience. * 🗑️ chore: Remove outdated Gemini image generation PNG and update SVG icon - Deleted the obsolete PNG file for Gemini image generation. - Updated the SVG icon with a new design featuring a gradient and shadow effect, enhancing visual appeal and consistency. * fix: ESLint formatting and unused variable in GeminiImageGen * fix: Update default model to gemini-2.5-flash-image * ✨ feat: Enhance Gemini Image Generation Configuration - Updated .env.example to include new environment variables for Google Cloud region, service account configuration, and Gemini API key options. - Modified GeminiImageGen.js to support both user-provided API keys and Vertex AI service accounts, improving flexibility in client initialization. - Updated manifest.json to reflect changes in authentication methods for the Gemini Image Tools. - Bumped @google/genai package version to 1.19.0 in package-lock.json for compatibility with new features. * 🔧 fix: Format Default Service Key Path in GeminiImageGen.js - Adjusted the return statement in getDefaultServiceKeyPath function for improved readability by formatting it across multiple lines. This change enhances code clarity without altering functionality. * ✨ feat: Enhance Gemini Image Generation with Token Usage Tracking - Added `recordTokenUsage` function to track token usage for balance management. - Integrated token recording into the image generation process. - Updated Gemini image generation tool to accept optional `aspectRatio` and `imageSize` parameters for improved image customization. - Updated token values for new Gemini models in the transaction model. - Improved documentation for image generation tool descriptions and parameters. * ✨ feat: Add new Gemini models for image generation token limits - Introduced token limits for 'gemini-3-pro-image' and 'gemini-2.5-flash-image' models. - Updated token values to enhance the Gemini image generation capabilities. * 🔧 fix: Update Google Service Key Path for Consistency in Initialization (#11001) * 🔧 refactor: Update GeminiImageGen for improved file handling and path resolution - Changed the default service key path to use process.cwd() for better compatibility. - Replaced synchronous file system operations with asynchronous promises for mkdir and writeFile, enhancing performance and error handling. - Added error handling for credential file access to prevent crashes when the file does not exist. * 🔧 refactor: Update GeminiImageGen to streamline API key handling - Refactored API key checks to improve clarity and consistency. - Removed redundant checks for user-provided keys, enhancing code readability. - Ensured proper logging for API key usage across different configurations. * 🔧 fix: Update GeminiImageGen to handle imageSize support conditionally - Added a check to ensure imageSize is only applied if the gemini model does not include 'gemini-2.5-flash-image', improving compatibility. - Enhanced the logic for setting imageConfig to prevent potential issues with unsupported configurations. * 🔧 refactor: Simplify local storage condition in createGeminiImageTool function * 🔧 feat: Enhance image format handling in GeminiImageGen with conversion support * 🔧 refactor: Streamline API key initialization in GeminiImageGen - Simplified the handling of API keys by removing redundant checks for user-provided keys. - Updated logging to reflect the new priority order for API key usage, enhancing clarity and consistency. - Improved code readability by consolidating key retrieval logic. --------- Co-authored-by: Dev Bhanushali <dev.bhanushali@hingehealth.com> Co-authored-by: Danny Avila <danny@librechat.ai> |
||
|
Danny Avila
|
b1a2b96276
|
🪜 fix: Layering Conflicts and UX Polish (#11177)
* 🔧 refactor: Update z-index values for popover components
- Reduced z-index from 50 to 40 across various popover components including Artifacts, ArtifactsSubMenu, MCPSubMenu, CustomMenu, and others to ensure consistent layering and improve UI behavior.
- Adjusted related CSS styles in Dropdown.css and DropdownMenu.tsx to align with the new z-index values, enhancing overall component visibility and interaction.
* chore: remove string template for className concatenation in CustomMenu component
- Improved the readability of the className prop in the CustomMenu component by restructuring the concatenation of class names. This change enhances maintainability and clarity in the styling logic.
* refactor: Simplify button visibility logic in SiblingHeader component
- Updated the button rendering logic in the SiblingHeader component to improve clarity and maintainability. The button is now always rendered, with its visibility controlled by the disabled state based on messageId, agentId, and submission status, enhancing user experience during interactions.
* refactor: Update shift key handling in Conversation and ConvoOptions components
- Modified the handling of the `isShiftHeld` state in both the Conversation and ConvoOptions components to improve clarity and functionality. The logic now ensures that the shift key state is accurately reflected based on the active conversation status, enhancing user interaction during conversations.
- Cleaned up imports in ConvoOptions by removing the unused `useShiftKey` hook, streamlining the component's dependencies.
* refactor: Improve Escape key handling in OriginalDialog component
- Updated the Escape key handling logic to prevent closing the dialog when a tooltip or dropdown menu has focus. This change enhances accessibility by ensuring compliance with WCAG standards for dismissable tooltips.
- Simplified the focus checking mechanism by directly assessing the active element within dropdown menus and tooltips, improving code clarity and maintainability.
* chore: imports
* refactor: Enhance Escape key handling in OriginalDialog component
- Updated the Escape key handling logic to prevent closing the dialog when a trigger with an open popover is focused. This change improves accessibility and user experience by ensuring that the dialog remains open during interactions with popovers, dropdowns, and listboxes.
- Simplified the focus checking mechanism to include additional roles, enhancing the clarity and maintainability of the code.
* refactor: Add dropdownClassName prop to FilterPrompts component
- Enhanced the FilterPrompts component by introducing a new dropdownClassName prop, allowing for customizable styling of the dropdown element.
- Updated the PromptsView component to utilize the new prop, improving the flexibility of the FilterPrompts integration within the UI.
* refactor: Clean up imports and remove unused code in DashBreadcrumb component
- Streamlined the DashBreadcrumb component by removing commented-out imports and unused code, enhancing clarity and maintainability.
- Adjusted the import order for better organization and readability.
* refactor: Update z-index handling in Dropdown component
- Removed the z-index property from Dropdown.css to streamline styling.
- Adjusted the className in Dropdown.tsx to include a new z-40 class for consistent z-index management, enhancing UI layering and interaction.
* refactor: Enhance file type acceptance in AttachFileMenu and useDragHelpers
- Updated the AttachFileMenu component to accept additional image formats (.heif, .heic) alongside existing types, improving file upload flexibility.
- Modified the useDragHelpers hook to utilize inferMimeType for better file type detection, ensuring accurate handling of dragged files.
* refactor: Enhance FloatingThinkingBar with copy functionality
- Added a copy button to the FloatingThinkingBar component, allowing users to copy thoughts to the clipboard.
- Updated the tooltip descriptions for the expand/collapse and copy actions to improve user experience.
- Cleaned up imports and adjusted prop types for better clarity and maintainability.
* refactor: Enhance RunCode component with icon-only mode
- Updated the RunCode component to accept an `iconOnly` prop, allowing for a simplified button display that shows only the icon when desired.
- Adjusted the button rendering logic to improve user experience and maintainability.
- Cleaned up imports and ensured consistent styling in the FloatingCodeBar component.
|
||
|
Danny Avila
|
eb1a59d2fd
|
🧠 fix: Messages View Height Expansion from #11142 & improve Thinking/Code-Block UX (#11148)
This commit fixes the messages view height auto-expansion issue introduced in PR #11142 and improves the UX of both Thinking and CodeBlock components. ## Bug Fix The ThinkingFooter component added in PR #11142 was causing the messages view height to automatically expand. The footer was placed inside the CSS grid's overflow-hidden container, but its presence affected the grid height calculation, causing layout issues during streaming. ## Solution: FloatingThinkingBar Replaced ThinkingFooter with a new FloatingThinkingBar component that: - Uses absolute positioning (bottom-right) like CodeBlock's FloatingCodeBar - Only appears on hover/focus, not affecting layout - Shows expand/collapse button with dynamic icon based on state - Uses TooltipAnchor for accessible tooltips - Supports keyboard navigation by showing when any element in the container is focused (top bar buttons or content) ## Changes ### Thinking.tsx - Added FloatingThinkingBar component with hover/focus visibility - Updated ThinkingContent with additional bottom padding (pb-10) - Added containerRef and hover/focus event handlers on outer container - Removed ThinkingFooter component (replaced by FloatingThinkingBar) ### Reasoning.tsx - Integrated FloatingThinkingBar with same hover/focus pattern - Added containerRef and event handlers on outer container - Supports keyboard navigation through entire component ### CodeBlock.tsx - Updated FloatingCodeBar to show icons only (removed text labels) - Added TooltipAnchor wrapper for copy button with localized tooltips - Improved accessibility with proper aria-label and aria-hidden ### Localization - Added com_ui_expand_thoughts: "Expand Thoughts" ## Accessibility - Full keyboard navigation support: tabbing through ThinkingButton, copy button, and FloatingThinkingBar - TooltipAnchor provides hover tooltips for icon-only buttons - Proper aria-label attributes on all interactive elements - tabIndex management based on visibility state |
||
|
Marco Beretta
|
a59bab4dc7
|
🧠 style: Expanded Thinking footer, Banner links, and Copy Thoughts accessibility (#11142)
* feat(Thinking): Add ThinkingFooter component for improved UX * style(Banner): auto-style hyperlinks in banner messages * fix: Simplify ThinkingFooter component by removing unused content prop and update aria-label for accessibility * fix: Correct import order for consistency in Thinking component * fix(ThinkingFooter): Update documentation to clarify footer functionality |
||
|
Joel Hirzel
|
29275cdc2a
|
🪟 feat: Make Chat Header Transparent (#11122)
* feat: Make top bar transparent * style(AddMultiConvo): update PlusCircle icon size for consistency * refactor(Header): use semantic presentation token for header gradient * style: update buttons background color to use presentation --------- Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com> |
||
|
Marco Beretta
|
5181356bef
|
🪄 refactor: UI Polish and Admin Dialog Unification (#11108)
* refactor(OpenSidebar): removed useless classNames
* style(Header): update hover styles across various components for improved UI consistency
* style(Nav): update hover styles in AccountSettings and SearchBar for improved UI consistency
* style: update button classes for consistent hover effects and improved UI responsiveness
* style(Nav, OpenSidebar, Header, Convo): improve UI responsiveness and animation transitions
* style(PresetsMenu, NewChat): update icon sizes and improve component styling for better UI consistency
* style(Nav, Root): enhance sidebar mobile animations and responsiveness for better UI experience
* style(ExportAndShareMenu, BookmarkMenu): update icon sizes for improved UI consistency
* style: remove transition duration from button classes for improved UI responsiveness
* style(CustomMenu, ModelSelector): update background colors for improved UI consistency and responsiveness
* style(ExportAndShareMenu): update icon color for improved UI consistency
* style(TemporaryChat): refine button styles for improved UI consistency and responsiveness
* style(BookmarkNav): refactor to use DropdownPopup and remove BookmarkNavItems for improved UI consistency and functionality
* style(CustomMenu, EndpointItem): enhance UI elements for improved consistency and accessibility
* style(EndpointItem): adjust gap in icon container for improved layout consistency
* style(CustomMenu, EndpointItem): update focus ring color for improved UI consistency
* style(EndpointItem): update icon color for improved UI consistency in dark theme
* style: update focus styles for improved accessibility and consistency across components
* refactor(Nav): extract sidebar width to NAV_WIDTH constant
Centralize mobile (320px) and desktop (260px) sidebar widths in a single
exported constant to avoid magic numbers and ensure consistency.
* fix(BookmarkNav): memoize handlers used in useMemo
Wrap handleTagClick and handleClear in useCallback and add them to the
dropdownItems useMemo dependency array to prevent stale closures.
* feat: introduce FilterInput component and replace existing inputs with it across multiple components
* feat(DataTable): replace custom input with FilterInput component for improved filtering
* fix: Nested dialog overlay stacking issue
Fixes overlay appearing behind content when opening nested dialogs.
Introduced dynamic z-index calculation based on dialog depth using React context.
- First dialog: overlay z-50, content z-100
- Nested dialogs increment by 60: overlay z-110/content z-160, etc.
Preserves a11y escape key handling from #10975 and #10851.
Regression from #11008 (
|
||
|
Danny Avila
|
daed6d9c0e
|
📋 feat: Add Floating Copy Button to Code Blocks (#11113)
Some checks failed
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Has been cancelled
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Has been cancelled
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Has been cancelled
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Has been cancelled
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Has been cancelled
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Has been cancelled
* feat: Add MermaidErrorBoundary for handling rendering errors in Mermaid diagrams * feat: Implement FloatingCodeBar for enhanced code block interaction and copy functionality * feat: Add zoom-level bar copy functionality to Mermaid component * feat: Enhance button styles in FloatingCodeBar and RunCode components for improved user interaction * refactor: copy button rendering in CodeBar and FloatingCodeBar for improved accessibility and clarity * chore: linting * chore: import order |
||
|
Danny Avila
|
3503b7caeb
|
📊 feat: Render Inline Mermaid Diagrams (#11112)
* chore: add mermaid, swr, ts-md5 packages * WIP: first pass, inline mermaid * feat: Enhance Mermaid component with zoom, pan, and error handling features * feat: Update Mermaid component styles for improved UI consistency * feat: Improve Mermaid rendering with enhanced debouncing and error handling * refactor: Update Mermaid component styles and enhance error handling in useMermaid hook * feat: Enhance security settings in useMermaid configuration to prevent DoS attacks * feat: Add dialog for expanded Mermaid view with zoom and pan controls * feat: Implement auto-scroll for streaming code in Mermaid component * feat: Replace loading spinner with reusable Spinner component in Mermaid * feat: Sanitize SVG output in useMermaid to enhance security * feat: Enhance SVG sanitization in useMermaid to support additional elements for text rendering * refactor: Enhance initial content check in useDebouncedMermaid for improved rendering logic * feat: Refactor Mermaid component to use Button component and enhance focus management for code toggling and copying * chore: remove unused key * refactor: initial content check in useDebouncedMermaid to detect significant content changes |
||
|
Danny Avila
|
439bc98682
|
⏸ refactor: Improve UX for Parallel Streams (Multi-Convo) (#11096)
* 🌊 feat: Implement multi-conversation feature with added conversation context and payload adjustments
* refactor: Replace isSubmittingFamily with isSubmitting across message components for consistency
* feat: Add loadAddedAgent and processAddedConvo for multi-conversation agent execution
* refactor: Update ContentRender usage to conditionally render PlaceholderRow based on isLast and isSubmitting
* WIP: first pass, sibling index
* feat: Enhance multi-conversation support with agent tracking and display improvements
* refactor: Introduce isEphemeralAgentId utility and update related logic for agent handling
* refactor: Implement createDualMessageContent utility for sibling message display and enhance useStepHandler for added conversations
* refactor: duplicate tools for added agent if ephemeral and primary agent is also ephemeral
* chore: remove deprecated multimessage rendering
* refactor: enhance dual message content creation and agent handling for parallel rendering
* refactor: streamline message rendering and submission handling by removing unused state and optimizing conditional logic
* refactor: adjust content handling in parallel mode to utilize existing content for improved agent display
* refactor: update @librechat/agents dependency to version 3.0.53
* refactor: update @langchain/core and @librechat/agents dependencies to latest versions
* refactor: remove deprecated @langchain/core dependency from package.json
* chore: remove unused SearchToolConfig and GetSourcesParams types from web.ts
* refactor: remove unused message properties from Message component
* refactor: enhance parallel content handling with groupId support in ContentParts and useStepHandler
* refactor: implement parallel content styling in Message, MessageRender, and ContentRender components. use explicit model name
* refactor: improve agent ID handling in createDualMessageContent for dual message display
* refactor: simplify title generation in AddedConvo by removing unused sender and preset logic
* refactor: replace string interpolation with cn utility for className in HoverButtons component
* refactor: enhance agent ID handling by adding suffix management for parallel agents and updating related components
* refactor: enhance column ordering in ContentParts by sorting agents with suffix management
* refactor: update @librechat/agents dependency to version 3.0.55
* feat: implement parallel content rendering with metadata support
- Added `ParallelContentRenderer` and `ParallelColumns` components for rendering messages in parallel based on groupId and agentId.
- Introduced `contentMetadataMap` to store metadata for each content part, allowing efficient parallel content detection.
- Updated `Message` and `ContentRender` components to utilize the new metadata structure for rendering.
- Modified `useStepHandler` to manage content indices and metadata during message processing.
- Enhanced `IJobStore` interface and its implementations to support storing and retrieving content metadata.
- Updated data schemas to include `contentMetadataMap` for messages, enabling multi-agent and parallel execution scenarios.
* refactor: update @librechat/agents dependency to version 3.0.56
* refactor: remove unused EPHEMERAL_AGENT_ID constant and simplify agent ID check
* refactor: enhance multi-agent message processing and primary agent determination
* refactor: implement branch message functionality for parallel responses
* refactor: integrate added conversation retrieval into message editing and regeneration processes
* refactor: remove unused isCard and isMultiMessage props from MessageRender and ContentRender components
* refactor: update @librechat/agents dependency to version 3.0.60
* refactor: replace usage of EPHEMERAL_AGENT_ID constant with isEphemeralAgentId function for improved clarity and consistency
* refactor: standardize agent ID format in tests for consistency
* chore: move addedConvo property to the correct position in payload construction
* refactor: rename agent_id values in loadAgent tests for clarity
* chore: reorder props in ContentParts component for improved readability
* refactor: rename variable 'content' to 'result' for clarity in RedisJobStore tests
* refactor: streamline useMessageActions by removing duplicate handleFeedback assignment
* chore: revert placeholder rendering logic MessageRender and ContentRender components to original
* refactor: implement useContentMetadata hook for optimized content metadata handling
* refactor: remove contentMetadataMap and related logic from the codebase and revert back to agentId/groupId in content parts
- Eliminated contentMetadataMap from various components and services, simplifying the handling of message content.
- Updated functions to directly access agentId and groupId from content parts instead of relying on a separate metadata map.
- Adjusted related hooks and components to reflect the removal of contentMetadataMap, ensuring consistent handling of message content.
- Updated tests and documentation to align with the new structure of message content handling.
* refactor: remove logging from groupParallelContent function to clean up output
* refactor: remove model parameter from TBranchMessageRequest type for simplification
* refactor: enhance branch message creation by stripping metadata for standalone content
* chore: streamline branch message creation by simplifying content filtering and removing unnecessary metadata checks
* refactor: include attachments in branch message creation for improved content handling
* refactor: streamline agent content processing by consolidating primary agent identification and filtering logic
* refactor: simplify multi-agent message processing by creating a dedicated mapping method and enhancing content filtering
* refactor: remove unused parameter from loadEphemeralAgent function for cleaner code
* refactor: update groupId handling in metadata to only set when provided by the server
|
||
|
Danny Avila
|
0ae3b87b65
|
🌊 feat: Resumable LLM Streams with Horizontal Scaling (#10926)
* ✨ feat: Implement Resumable Generation Jobs with SSE Support
- Introduced GenerationJobManager to handle resumable LLM generation jobs independently of HTTP connections.
- Added support for subscribing to ongoing generation jobs via SSE, allowing clients to reconnect and receive updates without losing progress.
- Enhanced existing agent controllers and routes to integrate resumable functionality, including job creation, completion, and error handling.
- Updated client-side hooks to manage adaptive SSE streams, switching between standard and resumable modes based on user settings.
- Added UI components and settings for enabling/disabling resumable streams, improving user experience during unstable connections.
* WIP: resuming
* WIP: resumable stream
* feat: Enhance Stream Management with Abort Functionality
- Updated the abort endpoint to support aborting ongoing generation streams using either streamId or conversationId.
- Introduced a new mutation hook `useAbortStreamMutation` for client-side integration.
- Added `useStreamStatus` query to monitor stream status and facilitate resuming conversations.
- Enhanced `useChatHelpers` to incorporate abort functionality when stopping generation.
- Improved `useResumableSSE` to handle stream errors and token refresh seamlessly.
- Updated `useResumeOnLoad` to check for active streams and resume conversations appropriately.
* fix: Update query parameter handling in useChatHelpers
- Refactored the logic for determining the query parameter used in fetching messages to prioritize paramId from the URL, falling back to conversationId only if paramId is not available. This change ensures consistency with the ChatView component's expectations.
* fix: improve syncing when switching conversations
* fix: Prevent memory leaks in useResumableSSE by clearing handler maps on stream completion and cleanup
* fix: Improve content type mismatch handling in useStepHandler
- Enhanced the condition for detecting content type mismatches to include additional checks, ensuring more robust validation of content types before processing updates.
* fix: Allow dynamic content creation in useChatFunctions
- Updated the initial response handling to avoid pre-initializing content types, enabling dynamic creation of content parts based on incoming delta events. This change supports various content types such as think and text.
* fix: Refine response message handling in useStepHandler
- Updated logic to determine the appropriate response message based on the last message's origin, ensuring correct message replacement or appending based on user interaction. This change enhances the accuracy of message updates in the chat flow.
* refactor: Enhance GenerationJobManager with In-Memory Implementations
- Introduced InMemoryJobStore, InMemoryEventTransport, and InMemoryContentState for improved job management and event handling.
- Updated GenerationJobManager to utilize these new implementations, allowing for better separation of concerns and easier maintenance.
- Enhanced job metadata handling to support user messages and response IDs for resumable functionality.
- Improved cleanup and state management processes to prevent memory leaks and ensure efficient resource usage.
* refactor: Enhance GenerationJobManager with improved subscriber handling
- Updated RuntimeJobState to include allSubscribersLeftHandlers for managing client disconnections without affecting subscriber count.
- Refined createJob and subscribe methods to ensure generation starts only when the first real client connects.
- Added detailed documentation for methods and properties to clarify the synchronization of job generation with client readiness.
- Improved logging for subscriber checks and event handling to facilitate debugging and monitoring.
* chore: Adjust timeout for subscriber readiness in ResumableAgentController
- Reduced the timeout duration from 5000ms to 2500ms in the startGeneration function to improve responsiveness when waiting for subscriber readiness. This change aims to enhance the efficiency of the agent's background generation process.
* refactor: Update GenerationJobManager documentation and structure
- Enhanced the documentation for GenerationJobManager to clarify the architecture and pluggable service design.
- Updated comments to reflect the potential for Redis integration and the need for async refactoring.
- Improved the structure of the GenerationJob facade to emphasize the unified API while allowing for implementation swapping without affecting consumer code.
* refactor: Convert GenerationJobManager methods to async for improved performance
- Updated methods in GenerationJobManager and InMemoryJobStore to be asynchronous, enhancing the handling of job creation, retrieval, and management.
- Adjusted the ResumableAgentController and related routes to await job operations, ensuring proper flow and error handling.
- Increased timeout duration in ResumableAgentController's startGeneration function to 3500ms for better subscriber readiness management.
* refactor: Simplify initial response handling in useChatFunctions
- Removed unnecessary pre-initialization of content types in the initial response, allowing for dynamic content creation based on incoming delta events. This change enhances flexibility in handling various content types in the chat flow.
* refactor: Clarify content handling logic in useStepHandler
- Updated comments to better explain the handling of initialContent and existingContent in edit and resume scenarios.
- Simplified the logic for merging content, ensuring that initialContent is used directly when available, improving clarity and maintainability.
* refactor: Improve message handling logic in useStepHandler
- Enhanced the logic for managing messages in multi-tab scenarios, ensuring that the most up-to-date message history is utilized.
- Removed existing response placeholders and ensured user messages are included, improving the accuracy of message updates in the chat flow.
* fix: remove unnecessary content length logging in the chat stream response, simplifying the debug message while retaining essential information about run steps. This change enhances clarity in logging without losing critical context.
* refactor: Integrate streamId handling for improved resumable functionality for attachments
- Added streamId parameter to various functions to support resumable mode in tool loading and memory processing.
- Updated related methods to ensure proper handling of attachments and responses based on the presence of streamId, enhancing the overall streaming experience.
- Improved logging and attachment management to accommodate both standard and resumable modes.
* refactor: Streamline abort handling and integrate GenerationJobManager for improved job management
- Removed the abortControllers middleware and integrated abort handling directly into GenerationJobManager.
- Updated abortMessage function to utilize GenerationJobManager for aborting jobs by conversation ID, enhancing clarity and efficiency.
- Simplified cleanup processes and improved error handling during abort operations.
- Enhanced metadata management for jobs, including endpoint and model information, to facilitate better tracking and resource management.
* refactor: Unify streamId and conversationId handling for improved job management
- Updated ResumableAgentController and AgentController to generate conversationId upfront, ensuring it matches streamId for consistency.
- Simplified job creation and metadata management by removing redundant conversationId updates from callbacks.
- Refactored abortMiddleware and related methods to utilize the unified streamId/conversationId approach, enhancing clarity in job handling.
- Removed deprecated methods from GenerationJobManager and InMemoryJobStore, streamlining the codebase and improving maintainability.
* refactor: Enhance resumable SSE handling with improved UI state management and error recovery
- Added UI state restoration on successful SSE connection to indicate ongoing submission.
- Implemented detailed error handling for network failures, including retry logic with exponential backoff.
- Introduced abort event handling to reset UI state on intentional stream closure.
- Enhanced debugging capabilities for testing reconnection and clean close scenarios.
- Updated generation function to retry on network errors, improving resilience during submission processes.
* refactor: Consolidate content state management into IJobStore for improved job handling
- Removed InMemoryContentState and integrated its functionality into InMemoryJobStore, streamlining content state management.
- Updated GenerationJobManager to utilize jobStore for content state operations, enhancing clarity and reducing redundancy.
- Introduced RedisJobStore for horizontal scaling, allowing for efficient job management and content reconstruction from chunks.
- Updated IJobStore interface to reflect changes in content state handling, ensuring consistency across implementations.
* feat: Introduce Redis-backed stream services for enhanced job management
- Added createStreamServices function to configure job store and event transport, supporting both Redis and in-memory options.
- Updated GenerationJobManager to allow configuration with custom job stores and event transports, improving flexibility for different deployment scenarios.
- Refactored IJobStore interface to support asynchronous content retrieval, ensuring compatibility with Redis implementations.
- Implemented RedisEventTransport for real-time event delivery across instances, enhancing scalability and responsiveness.
- Updated InMemoryJobStore to align with new async patterns for content and run step retrieval, ensuring consistent behavior across storage options.
* refactor: Remove redundant debug logging in GenerationJobManager and RedisEventTransport
- Eliminated unnecessary debug statements in GenerationJobManager related to subscriber actions and job updates, enhancing log clarity.
- Removed debug logging in RedisEventTransport for subscription and subscriber disconnection events, streamlining the logging output.
- Cleaned up debug messages in RedisJobStore to focus on essential information, improving overall logging efficiency.
* refactor: Enhance job state management and TTL configuration in RedisJobStore
- Updated the RedisJobStore to allow customizable TTL values for job states, improving flexibility in job management.
- Refactored the handling of job expiration and cleanup processes to align with new TTL configurations.
- Simplified the response structure in the chat status endpoint by consolidating state retrieval, enhancing clarity and performance.
- Improved comments and documentation for better understanding of the changes made.
* refactor: cleanupOnComplete option to GenerationJobManager for flexible resource management
- Introduced a new configuration option, cleanupOnComplete, allowing immediate cleanup of event transport and job resources upon job completion.
- Updated completeJob and abortJob methods to respect the cleanupOnComplete setting, enhancing memory management.
- Improved cleanup logic in the cleanup method to handle orphaned resources effectively.
- Enhanced documentation and comments for better clarity on the new functionality.
* refactor: Update TTL configuration for completed jobs in InMemoryJobStore
- Changed the TTL for completed jobs from 5 minutes to 0, allowing for immediate cleanup.
- Enhanced cleanup logic to respect the new TTL setting, improving resource management.
- Updated comments for clarity on the behavior of the TTL configuration.
* refactor: Enhance RedisJobStore with local graph caching for improved performance
- Introduced a local cache for graph references using WeakRef to optimize reconnects for the same instance.
- Updated job deletion and cleanup methods to manage the local cache effectively, ensuring stale entries are removed.
- Enhanced content retrieval methods to prioritize local cache access, reducing Redis round-trips for same-instance reconnects.
- Improved documentation and comments for clarity on the caching mechanism and its benefits.
* feat: Add integration tests for GenerationJobManager, RedisEventTransport, and RedisJobStore, add Redis Cluster support
- Introduced comprehensive integration tests for GenerationJobManager, covering both in-memory and Redis modes to ensure consistent job management and event handling.
- Added tests for RedisEventTransport to validate pub/sub functionality, including cross-instance event delivery and error handling.
- Implemented integration tests for RedisJobStore, focusing on multi-instance job access, content reconstruction from chunks, and consumer group behavior.
- Enhanced test setup and teardown processes to ensure a clean environment for each test run, improving reliability and maintainability.
* fix: Improve error handling in GenerationJobManager for allSubscribersLeft handlers
- Enhanced the error handling logic when retrieving content parts for allSubscribersLeft handlers, ensuring that any failures are logged appropriately.
- Updated the promise chain to catch errors from getContentParts, improving robustness and clarity in error reporting.
* ci: Improve Redis client disconnection handling in integration tests
- Updated the afterAll cleanup logic in integration tests for GenerationJobManager, RedisEventTransport, and RedisJobStore to use `quit()` for graceful disconnection of the Redis client.
- Added fallback to `disconnect()` if `quit()` fails, enhancing robustness in resource management during test teardown.
- Improved comments for clarity on the disconnection process and error handling.
* refactor: Enhance GenerationJobManager and event transports for improved resource management
- Updated GenerationJobManager to prevent immediate cleanup of eventTransport upon job completion, allowing final events to transmit fully before cleanup.
- Added orphaned stream cleanup logic in GenerationJobManager to handle streams without corresponding jobs.
- Introduced getTrackedStreamIds method in both InMemoryEventTransport and RedisEventTransport for better management of orphaned streams.
- Improved comments for clarity on resource management and cleanup processes.
* refactor: Update GenerationJobManager and ResumableAgentController for improved event handling
- Modified GenerationJobManager to resolve readyPromise immediately, eliminating startup latency and allowing early event buffering for late subscribers.
- Enhanced event handling logic to replay buffered events when the first subscriber connects, ensuring no events are lost due to race conditions.
- Updated comments for clarity on the new event synchronization mechanism and its benefits in both Redis and in-memory modes.
* fix: Update cache integration test command for stream to ensure proper execution
- Modified the test command for cache integration related to streams by adding the --forceExit flag to prevent hanging tests.
- This change enhances the reliability of the test suite by ensuring all tests complete as expected.
* feat: Add active job management for user and show progress in conversation list
- Implemented a new endpoint to retrieve active generation job IDs for the current user, enhancing user experience by allowing visibility of ongoing tasks.
- Integrated active job tracking in the Conversations component, displaying generation indicators based on active jobs.
- Optimized job management in the GenerationJobManager and InMemoryJobStore to support user-specific job queries, ensuring efficient resource handling and cleanup.
- Updated relevant components and hooks to utilize the new active jobs feature, improving overall application responsiveness and user feedback.
* feat: Implement active job tracking by user in RedisJobStore
- Added functionality to retrieve active job IDs for a specific user, enhancing user experience by allowing visibility of ongoing tasks.
- Implemented self-healing cleanup for stale job entries, ensuring accurate tracking of active jobs.
- Updated job creation, update, and deletion methods to manage user-specific job sets effectively.
- Enhanced integration tests to validate the new user-specific job management features.
* refactor: Simplify job deletion logic by removing user job cleanup from InMemoryJobStore and RedisJobStore
* WIP: Add backend inspect script for easier debugging in production
* refactor: title generation logic
- Changed the title generation endpoint from POST to GET, allowing for more efficient retrieval of titles based on conversation ID.
- Implemented exponential backoff for title fetching retries, improving responsiveness and reducing server load.
- Introduced a queuing mechanism for title generation, ensuring titles are generated only after job completion.
- Updated relevant components and hooks to utilize the new title generation logic, enhancing user experience and application performance.
* feat: Enhance updateConvoInAllQueries to support moving conversations to the top
* chore: temp. remove added multi convo
* refactor: Update active jobs query integration for optimistic updates on abort
- Introduced a new interface for active jobs response to standardize data handling.
- Updated query keys for active jobs to ensure consistency across components.
- Enhanced job management logic in hooks to properly reflect active job states, improving overall application responsiveness.
* refactor: useResumableStreamToggle hook to manage resumable streams for legacy/assistants endpoints
- Introduced a new hook, useResumableStreamToggle, to automatically toggle resumable streams off for assistants endpoints and restore the previous value when switching away.
- Updated ChatView component to utilize the new hook, enhancing the handling of streaming behavior based on endpoint type.
- Refactored imports in ChatView for better organization.
* refactor: streamline conversation title generation handling
- Removed unused type definition for TGenTitleMutation in mutations.ts to clean up the codebase.
- Integrated queueTitleGeneration call in useEventHandlers to trigger title generation for new conversations, enhancing the responsiveness of the application.
* feat: Add USE_REDIS_STREAMS configuration for stream job storage
- Introduced USE_REDIS_STREAMS to control Redis usage for resumable stream job storage, defaulting to true if USE_REDIS is enabled but not explicitly set.
- Updated cacheConfig to include USE_REDIS_STREAMS and modified createStreamServices to utilize this new configuration.
- Enhanced unit tests to validate the behavior of USE_REDIS_STREAMS under various environment settings, ensuring correct defaults and overrides.
* fix: title generation queue management for assistants
- Introduced a queueListeners mechanism to notify changes in the title generation queue, improving responsiveness for non-resumable streams.
- Updated the useTitleGeneration hook to track queue changes with a queueVersion state, ensuring accurate updates when jobs complete.
- Refactored the queueTitleGeneration function to trigger listeners upon adding new conversation IDs, enhancing the overall title generation flow.
* refactor: streamline agent controller and remove legacy resumable handling
- Updated the AgentController to route all requests to ResumableAgentController, simplifying the logic.
- Deprecated the legacy non-resumable path, providing a clear migration path for future use.
- Adjusted setHeaders middleware to remove unnecessary checks for resumable mode.
- Cleaned up the useResumableSSE hook to eliminate redundant query parameters, enhancing clarity and performance.
* feat: Add USE_REDIS_STREAMS configuration to .env.example
- Updated .env.example to include USE_REDIS_STREAMS setting, allowing control over Redis usage for resumable LLM streams.
- Provided additional context on the behavior of USE_REDIS_STREAMS when not explicitly set, enhancing clarity for configuration management.
* refactor: remove unused setHeaders middleware from chat route
- Eliminated the setHeaders middleware from the chat route, streamlining the request handling process.
- This change contributes to cleaner code and improved performance by reducing unnecessary middleware checks.
* fix: Add streamId parameter for resumable stream handling across services (actions, mcp oauth)
* fix(flow): add immediate abort handling and fix intervalId initialization
- Add immediate abort handler that responds instantly to abort signal
- Declare intervalId before cleanup function to prevent 'Cannot access before initialization' error
- Consolidate cleanup logic into single function to avoid duplicate cleanup
- Properly remove abort event listener on cleanup
* fix(mcp): clean up OAuth flows on abort and simplify flow handling
- Add abort handler in reconnectServer to clean up mcp_oauth and mcp_get_tokens flows
- Update createAbortHandler to clean up both flow types on tool call abort
- Pass abort signal to createFlow in returnOnOAuth path
- Simplify handleOAuthRequired to always cancel existing flows and start fresh
- This ensures user always gets a new OAuth URL instead of waiting for stale flows
* fix(agents): handle 'new' conversationId and improve abort reliability
- Treat 'new' as placeholder that needs UUID in request controller
- Send JSON response immediately before tool loading for faster SSE connection
- Use job's abort controller instead of prelimAbortController
- Emit errors to stream if headers already sent
- Skip 'new' as valid ID in abort endpoint
- Add fallback to find active jobs by userId when conversationId is 'new'
* fix(stream): detect early abort and prevent navigation to non-existent conversation
- Abort controller on job completion to signal pending operations
- Detect early abort (no content, no responseMessageId) in abortJob
- Set conversation and responseMessage to null for early aborts
- Add earlyAbort flag to final event for frontend detection
- Remove unused text field from AbortResult interface
- Frontend handles earlyAbort by staying on/navigating to new chat
* test(mcp): update test to expect signal parameter in createFlow
fix(agents): include 'new' conversationId in newConvo check for title generation
When frontend sends 'new' as conversationId, it should still trigger
title generation since it's a new conversation. Rename boolean variable for clarity
fix(agents): check abort state before completeJob for title generation
completeJob now triggers abort signal for cleanup, so we need to
capture the abort state beforehand to correctly determine if title
generation should run.
|
||
|
Danny Avila
|
f9060fa25f
|
🔧 chore: Update ESLint Config & Run Linter (#10986) | ||
|
Danny Avila
|
06719794f6
|
🗝️ fix: React Key Props and Minor UI Fixes from a11y Updates (#10954)
* refactor: Update Frontend logger function to enhance logging conditions
- Modified the logger function to check for logger enablement and development environment more robustly.
- Adjusted the condition to ensure logging occurs only when the logger is enabled or when the environment variable for logger is not set in development mode.
* fix: Add key prop to MeasuredRow components in Conversations for improved rendering
- Updated MeasuredRow components to include a key prop for better performance and to prevent rendering issues during list updates.
- Ensured consistent handling of item types within the Conversations component.
* refactor: Enhance ScrollToBottom component with forwardRef for improved functionality
- Updated ScrollToBottom component to use forwardRef, allowing it to accept a ref for better integration with parent components.
- Modified MessagesView to utilize the new ref for the ScrollToBottom button, improving scrolling behavior and performance.
* refactor: Enhance EndpointItem and renderEndpoints for improved model render keys
- Updated EndpointItem to accept an endpointIndex prop for better indexing of endpoints.
- Modified renderEndpoints to pass the endpointIndex to EndpointItem, improving the rendering of endpoint models.
- Adjusted renderEndpointModels to utilize the endpointIndex for unique key generation, enhancing performance and preventing rendering issues.
* refactor: Update BaseClient to handle non-ephemeral agents in conversation logic
- Added a check for non-ephemeral agents in BaseClient, modifying the exceptions set to include 'model' when applicable.
- Enhanced conversation handling to improve flexibility based on agent type.
* refactor: Optimize FavoritesList component for agent handling and loading states
- Updated FavoritesList to improve agent ID management by introducing combinedAgentsMap for better handling of missing agents.
- Refactored loading state logic to ensure accurate representation of agent loading status.
- Enhanced the use of useQueries for fetching missing agent data, streamlining the overall data retrieval process.
- Improved memoization of agent IDs and loading conditions for better performance and reliability.
* Revert "refactor: Update BaseClient to handle non-ephemeral agents in conversation logic"
This reverts commit
|
||
|
Daniel Lew
|
ad733157d7
|
🦻 fix: Standardize Message Aria Labels for Assistive Technology (#10796)
Message aria labels were using the format `message-{depth}-{id}`, which
reads out in a very unfriendly way to a screen reader (e.g.
"message-3-65a38d32-4d6e-42c7-b67b-9ee62f8cedb1").
I standardized all the labeling in one place (`getMessageAriaLabel()`).
They now read out "Message 1", "Message 2", and so on.
|
||
|
Samuel Path
|
304bba853c
|
💻 feat: Deeper MCP UI integration in the Chat UI (#9669)
* 💻 feat: deeper MCP UI integration in the chat UI using plugins --------- Co-authored-by: Samuel Path <samuel.path@shopify.com> Co-authored-by: Pierre-Luc Godin <pierreluc.godin@shopify.com> * 💻 refactor: Migrate MCP UI resources from index-based to ID-based referencing - Replace index-based resource markers with stable resource IDs - Update plugin to parse \ui{resourceId} format instead of \ui0 - Refactor components to use useMessagesOperations instead of useSubmitMessage - Add ShareMessagesProvider for UI resources in share view - Add useConversationUIResources hook for cross-turn resource lookups - Update parsers to generate resource IDs from content hashes - Update all tests to use resource IDs instead of indices - Add sandbox permissions for iframe popups - Remove deprecated MCP tool context instructions --------- Co-authored-by: Pierre-Luc Godin <pierreluc.godin@shopify.com> |
||
|
Dustin Healy
|
4a0fbb07bc
|
🚹 feat: Miscellaneous Accessibility Improvements (#10913)
* 🔱 fix: Fork Menu Accessibility Improvements (#10910) * feat: more accessible aria-label for fork button * fix: alignment between text and checkbox * feat: add text change on focus for parity with on hover for keyboard accessibility * 🤔 fix: Programmatic Expansion State for Thinking Button (#10912) * 🙋♂️ feat: Accessible Default User Icon Colors (#10909) * fix: downshift values for all non-compliant default bg-colors for user icons to achieve 4.5:1 contrast threshold minimums with text * 🚪 feat: Open Sidebar Label Accessibility (#10893) * feat: more accessible labelling on open / close sidebar |
||
|
Dustin Healy
|
2989ebd649
|
🛗 fix: Address Accessibility Issues - Axe Rating: Serious (#10521)
* feat: add light/dark differentiation on text color for login footer links for more accessible contrast in light mode * feat: add darker color focus ring on ThemeSelector in light mode for more accessible contrast * feat: increase contrast on text color for rendered error messages in light and dark mode so that they pass the 4.5:1 accessibility contrast threshold against their backgrounds * feat: add more accessible color vars to style.css for better contrast against light/dark backgrounds * feat: un-nest DropdownMenu from ListCard and make them siblings instead for better accessibility * feat: tweak --border-heavy in light mode so that it uses --gray-410 rather than --gray-400 so that the contrast ratio threshold is hit for accessibility * feat: switch email and password input border to border-heavy for more accessible contrast on Login page * fix: add proper focus ring for Action menu button in Prompts Sidenav * fix: align light and dark focus rings with surrounding elements on preview/edit menu dropdown button in Prompt Card * fix: remove aria-hidden on parent div with focusable child element according to accessibility guidelines * fix: add missing aria-readonly false property that should have been in previous accessibility PR * feat: add horizontal padding on rowRenderer's CellMeasurer div so that focus ring on rows doesnt clip behind virtualized table borders side-to-side (still need to figure out vertical clipping on final row / a better solution to be able to get overflows to work properly within the virtualized table) * feat: remove render prop override so that Share and Delete Buttons in Conversation dropdown can be pressed with Enter keystroke * fix: undo additional colors and changes to --surface-hover the initial changes came from a misunderstanding of contrast threshold requirements for hover effect accessibility * feat: better layout for non-nested prompt card / action menu combination * fix: add proper focus restoration behavior for Preview modal on close * fix: undo change to --border-heavy in light mode * fix: set borders for login input boxes back to light * feat: add announcement for state change when link copied to clipboard in conversation share modal * feat: add announcement to Refresh Link button * feat: add announcement for archiving chats * feat: make date sections in conversation history list <h2> rather than generic <div> for improved screen reader support * feat: ensure Share Link modal is accessible at high zoom percentage and low viewport width / height requirements by adding max height and overflow attributes to allow scrolling * feat: bold toast text so that it hits font size accessibility threshold (above 14 px when bolded - change makes text 16 px bold) so that the more disruptive contrast change of the toast background color is no longer necessary. The background color would need to achieve a 4.5:1 contrast ratio, which would significantly affect the established aesthetic of the current toast system if achieved. * fix: do not render side nav when it is hidden to avoid keyboard navigation with screen reader * fix: add side nav button state change announcements and don't render components that were previosuly reachable via keyboard navigation while in the side nav * feat: add tooltip anchor for Model Select * fix: only hide the model selector, export, and temp chat buttons when in mobile view and the sidenav is expanded * feat: add aria-haspopup support for MenuItems and add aria-haspopup: 'dialog' for Share and Delete buttons in ConvoOptions * feat: add label for DataTable search so that it does not rely on placeholder attribute for function identification * feat: make X buttons on dialogs 24x24px to achieve AA compliance * feat: add announcements for the search bar for model selector * feat: persistent label for DataTable * feat: make filter files text contrast compliant * feat: add non-color visual indicator to AudioRecorder listening state * feat: add aria-expanded attribute to tool call dropdown for screen reader * feat: add high contrast and rounded outlines for focus indicators on Run Code and Copy Code buttons for code blocks * fix: change Button to anchor tag in Shared Links component when linking to original conversation * fix: allow overflow in datatable cells so that focus indicators dont get cut off * feat: round out focus outline for link name in SharedLinks modal * feat: add aria-controls and aria-haspopup: "dialog" to SharedLinks delete button and modal * feat: add aria-controls for dropdown menu items on ConvoOptions for share and delete modals * feat: add trigger ref to 2FA button and modal in settings menu so focus returns to button on modal close * feat: add refs so that open sidebar and close sidebar buttons transfer focus to one another * chore: formatting * feat: make sure settings modal is accessible at 200% zoom for screen size 1366x768 viewport * feat: round out focus outline for link names in archived chats modal * feat: add result announcements for screen reader in DataTable search * feat: simplify layout for checkbox / api key components for better accessibility * feat: return focus to chat input on prompt variables modal close * feat: add persistent labels to TextareaAutosize Inputs in Variable form * feat: tighten max width so side scrolling not necessary at 400% zoom for VariableForm modal * feat: add persistent labels to prompt management page * feat: announce results found for search bars in prompts page and improve them in datatable * feat: de-nest DashGroupItem buttons in Prompts page to allow better navigation and comply with accessibility standard * feat: add heading for new prompt creation page for screen readers * feat: remove non-compliant description truncation for small screen sizes by making labels static on small enough viewport width * feat: add mobile view sidebar for prompts page * feat: add bolded text on select for AdvancedSwitch so that there is a visual indicator of selection and it does not rely solely on color as an indication of state * feat: add persistent labels to ModelSelector search inputs * feat: align aria-label with visual label for speech recognition users * feat: make MemoryCreateDialog accessible at 400% zoom (introduce max viewport height attr and make scrollable) * feat: add persistent label to Filter input for DataTable in file attach sidebar menu * feat: add persistent label for bookmark filter input in bookmarks sidebar menu * feat: add alert for screen readers for invalid inputs when editting bookmarks * feat: bold font in BookmarkForm error readout to pass contrast compliance thresholds for 14pt text * feat: align aria-label with visual label for BookmarkForm Ttile input * feat: add 400% zoom support for ALL modals utilizing OriginalDialog to prevent clipping * feat: remove state change on aria label and give consistent labelling for button, offload state change notification to the announcement div and make more assertive * feat: add aria-labels which convey that the buttons are sortable (divergence from visual text because iconography is used to signify sort functionality) * feat: add supplemental visuals to indicate link is clickable other than color in SharedLinks * feat: increase saturation to hit contrast threshold minimums on Link color in SharedLinks * feat: stop DataTable from disappearing at 400% zoom in SharedLinks * feat: increase contrast to hit contrast threshold minimums on Animated Search Input visual indicators * feat: add aria-label for AnimatedSearchInput (doesn't require explicit labelling because of Search icon) * fix: stop long example variable declaration from clipping at high zoom in variables info * feat: add aria-label to bettter describe sort button functionality for vision impaired users * chore: remove unused translation key * chore: address ESLint comments * fix: modify test to account for new alert on theme toggle switch for login page * chore: interpolate translation key |
||
|
Danny Avila
|
656e1abaea
|
🪦 refactor: Remove Legacy Code (#10533)
* 🗑️ chore: Remove unused Legacy Provider clients and related helpers * Deleted OpenAIClient and GoogleClient files along with their associated tests. * Removed references to these clients in the clients index file. * Cleaned up typedefs by removing the OpenAISpecClient export. * Updated chat controllers to use the OpenAI SDK directly instead of the removed client classes. * chore/remove-openapi-specs * 🗑️ chore: Remove unused mergeSort and misc utility functions * Deleted mergeSort.js and misc.js files as they are no longer needed. * Removed references to cleanUpPrimaryKeyValue in messages.js and adjusted related logic. * Updated mongoMeili.ts to eliminate local implementations of removed functions. * chore: remove legacy endpoints * chore: remove all plugins endpoint related code * chore: remove unused prompt handling code and clean up imports * Deleted handleInputs.js and instructions.js files as they are no longer needed. * Removed references to these files in the prompts index.js. * Updated docker-compose.yml to simplify reverse proxy configuration. * chore: remove unused LightningIcon import from Icons.tsx * chore: clean up translation.json by removing deprecated and unused keys * chore: update Jest configuration and remove unused mock file * Simplified the setupFiles array in jest.config.js by removing the fetchEventSource mock. * Deleted the fetchEventSource.js mock file as it is no longer needed. * fix: simplify endpoint type check in Landing and ConversationStarters components * Updated the endpoint type check to use strict equality for better clarity and performance. * Ensured consistency in the handling of the azureOpenAI endpoint across both components. * chore: remove unused dependencies from package.json and package-lock.json * chore: remove legacy EditController, associated routes and imports * chore: update banResponse logic to refine request handling for banned users * chore: remove unused validateEndpoint middleware and its references * chore: remove unused 'res' parameter from initializeClient in multiple endpoint files * chore: remove unused 'isSmallScreen' prop from BookmarkNav and NewChat components; clean up imports in ArchivedChatsTable and useSetIndexOptions hooks; enhance localization in PromptVersions * chore: remove unused import of Constants and TMessage from MobileNav; retain only necessary QueryKeys import * chore: remove unused TResPlugin type and related references; clean up imports in types and schemas |
||
|
Daniel Lew
|
1143f73f59
|
🔇 fix: Hide Button Icons from Screen Readers (#10776)
If you've got a screen reader that is reading out the whole page, each icon button (i.e., `<button><SVG></button>`) will have both the button's aria-label read out as well as the title from the SVG (which is usually just "image"). Since we are pretty good about setting aria-labels, we should instead use `aria-hidden="true"` on these images, since they are not useful to be read out. I don't consider this a comprehensive review of all icons in the app, but I knocked out all the low hanging fruit in this commit. |
||
|
Danny Avila
|
026890cd27
|
🛡️ fix: Improve Error Handling and Null Safety in SSE Event Processing (#10751)
* 🔧 fix: Handle null content parts in message processing - Added checks to filter out null content parts in various message handling functions, ensuring robustness against undefined values. - Updated the `extractMessageContent`, `useContentHandler`, `useEventHandlers`, and `useStepHandler` hooks to prevent errors caused by null parts. - Enhanced the `getAllContentText` utility to only include valid content types, improving overall message integrity. * 🔧 fix: Enhance error handling in event and SSE handlers - Wrapped critical sections in try-catch blocks within `useEventHandlers` and `useSSE` hooks to improve error management and prevent application crashes. - Added console error logging for better debugging and tracking of issues during message processing and conversation aborting. - Ensured that UI states like `setIsSubmitting` and `setShowStopButton` are correctly updated in case of errors, maintaining a consistent user experience. * 🔧 fix: Filter out null and empty content in message export - Enhanced the `useExportConversation` hook to filter out null content parts and empty strings during message processing, ensuring only valid content is included in the export. - This change improves the integrity of exported conversations by preventing unnecessary empty entries in the output. |
||
|
Joel Hirzel
|
90f0bcde44
|
🖼️ fix: resolve stuck pixel animation for image generation (#10716) | ||
|
catmeme
|
7aa8d49f3a
|
🧭 fix: Add Base Path Support for Login/Register and Image Paths (#10116)
* fix: add basePath pattern to support login/register and image paths * Fix linter errors * refactor: Update import statements for getBasePath and isEnabled, and add path utility functions with tests - Refactored imports in addImages.js and StableDiffusion.js to use getBasePath from '@librechat/api'. - Consolidated isEnabled and getBasePath imports in validateImageRequest.js. - Introduced new path utility functions in path.ts and corresponding unit tests in path.spec.ts to validate base path extraction logic. * fix: Update domain server base URL in MarkdownComponents and refactor authentication redirection logic - Changed the domain server base URL in MarkdownComponents.tsx to use the API base URL. - Refactored the useAuthRedirect hook to utilize React Router's navigate for redirection instead of window.location, ensuring a smoother SPA experience. - Added unit tests for the useAuthRedirect hook to verify authentication redirection behavior. * test: Mock isEnabled in validateImages.spec.js for improved test isolation - Updated validateImages.spec.js to mock the isEnabled function from @librechat/api, ensuring that tests can run independently of the actual implementation. - Cleared the DOMAIN_CLIENT environment variable before tests to avoid interference with basePath resolution. --------- Co-authored-by: Danny Avila <danny@librechat.ai> |
||
|
Daniel Lew
|
014eb10662
|
📢 fix: Resolved Screen Reader Issues with TooltipAnchor (#10580)
TooltipAnchor was automatically adding an `aria-describedby` tag which often duplicated the labeling already present inside of the anchor. E.g., the screen reader might say "New Chat, New Chat, button" instead of just "New Chat, button." I've removed the TooltipAnchor's automatic `aria-describedby` and worked to make sure that anyone using TooltipAnchor properly defines its labeling. |
||
|
Marco Beretta
|
e71c48ec3d
|
🎨 fix: Correct Read-Only State Logic in Code Editor (#10508)
* ✨ style: Update ThinkingButton container background color for improved visibility * ✨ style: Refactor Clipboard icon rendering for improved readability * ✨ style: Simplify readOnly state initialization and update logic in ArtifactCodeEditor * ✨ style: Update Thinking component background color for improved aesthetics * Update client/src/components/Chat/Messages/MinimalHoverButtons.tsx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Marco Beretta
|
c2505d2bc9
|
🤝 feat: View Artifacts in Shared Conversations (#10477)
* feat: Integrate logger for MessageIcon component * feat: Enhance artifact sharing functionality with updated path checks and read-only state management * feat: Refactor Thinking and Reasoning components for improved structure and styling * feat: Enhance artifact sharing with context value management and responsive layout * feat: Enhance ShareView with theme and language management features * feat: Improve ThinkingButton accessibility and styling for better user interaction * feat: Introduce isArtifactRoute utility for route validation in Artifact components * feat: Add latest message text extraction in SharedView for improved message display * feat: Update locale handling in SharedView for dynamic date formatting * feat: Refactor ArtifactsContext and SharedView for improved context handling and styling adjustments * feat: Enhance artifact panel size management with local storage integration * chore: imports * refactor: move ShareArtifactsContainer out of ShareView --------- Co-authored-by: Danny Avila <danny@librechat.ai> |
||
|
Danny Avila
|
cabc8afeac
|
🔧 fix: Await MCP Instructions and Filter Malformed Tool Calls (#10485)
* fix: Await MCP instructions formatting in AgentClient * fix: don't render or aggregate malformed tool calls * fix: implement filter for malformed tool call content parts and add tests |
||
|
Danny Avila
|
2524d33362
|
📂 refactor: Cleanup File Filtering Logic, Improve Validation (#10414)
* feat: add filterFilesByEndpointConfig to filter disabled file processing by provider * chore: explicit define of endpointFileConfig for better debugging * refactor: move `normalizeEndpointName` to data-provider as used app-wide * chore: remove overrideEndpoint from useFileHandling * refactor: improve endpoint file config selection * refactor: update filterFilesByEndpointConfig to accept structured parameters and improve endpoint file config handling * refactor: replace defaultFileConfig with getEndpointFileConfig for improved file configuration handling across components * test: add comprehensive unit tests for getEndpointFileConfig to validate endpoint configuration handling * refactor: streamline agent endpoint assignment and improve file filtering logic * feat: add error handling for disabled file uploads in endpoint configuration * refactor: update encodeAndFormat functions to accept structured parameters for provider and endpoint * refactor: streamline requestFiles handling in initializeAgent function * fix: getEndpointFileConfig partial config merging scenarios * refactor: enhance mergeWithDefault function to support document-supported providers with comprehensive MIME types * refactor: user-configured default file config in getEndpointFileConfig * fix: prevent file handling when endpoint is disabled and file is dragged to chat * refactor: move `getEndpointField` to `data-provider` and update usage across components and hooks * fix: prioritize endpointType based on agent.endpoint in file filtering logic * fix: prioritize agent.endpoint in file filtering logic and remove unnecessary endpointType defaulting |
||
|
Danny Avila
|
8a4a5a4790
|
🤖 feat: Agent Handoffs (Routing) (#10176)
* feat: Add support for agent handoffs with edges in agent forms and schemas chore: Mark `agent_ids` field as deprecated in favor of edges across various schemas and types chore: Update dependencies for @langchain/core and @librechat/agents to latest versions chore: Update peer dependency for @librechat/agents to version 3.0.0-rc2 in package.json chore: Update @librechat/agents dependency to version 3.0.0-rc3 in package.json and package-lock.json feat: first pass, multi-agent handoffs fix: update output type to ToolMessage in memory handling functions fix: improve type checking for graphConfig in createRun function refactor: remove unused content filtering logic in AgentClient chore: update @librechat/agents dependency to version 3.0.0-rc4 in package.json and package-lock.json fix: update @langchain/core peer dependency version to ^0.3.72 in package.json and package-lock.json fix: update @librechat/agents dependency to version 3.0.0-rc6 in package.json and package-lock.json; refactor stream rate handling in various endpoints feat: Agent handoff UI chore: update @librechat/agents dependency to version 3.0.0-rc8 in package.json and package-lock.json fix: improve hasInfo condition and adjust UI element classes in AgentHandoff component refactor: remove current fixed agent display from AgentHandoffs component due to redundancy feat: enhance AgentHandoffs UI with localized beta label and improved layout chore: update @librechat/agents dependency to version 3.0.0-rc10 in package.json and package-lock.json feat: add `createSequentialChainEdges` function to add back agent chaining via multi-agents feat: update `createSequentialChainEdges` call to only provide conversation context between agents feat: deprecate Agent Chain functionality and update related methods for improved clarity * chore: update @librechat/agents dependency to version 3.0.0-rc11 in package.json and package-lock.json * refactor: remove unused addCacheControl function and related imports and import from @librechat/agents * chore: remove unused i18n keys * refactor: remove unused format export from index.ts * chore: update @librechat/agents to v3.0.0-rc13 * chore: remove BEDROCK_LEGACY provider from Providers enum * chore: update @librechat/agents to version 3.0.2 in package.json |
||
|
Danny Avila
|
9b4c4cafb6
|
🧠 refactor: Improve Reasoning Component Structure and UX (#10320)
* refactor: Reasoning components with independent toggle buttons - Refactored ThinkingButton to remove unnecessary state and props. - Updated ContentParts to simplify content rendering and remove hover handling. - Improved Reasoning component to include independent toggle functionality for each THINK part. - Adjusted styles for better layout consistency and user experience. * refactor: isolate hover effects for Reasoning - Updated ThinkingButton to improve hover effects and layout consistency. - Refactored Reasoning component to include a new wrapper class for better styling. - Adjusted icon visibility and transitions for a smoother user experience. * fix: Prevent rendering of empty messages in Chat component - Added a check to skip rendering if the message text is only whitespace, improving the user interface by avoiding empty containers. * chore: Replace div with fragment in Thinking component for cleaner markup * chore: move Thinking component to Content Parts directory * refactor: prevent rendering of whitespace-only text in Part component only for edge cases |
||
|
Marco Beretta
|
c0f1cfcaba
|
💡 feat: Improve Reasoning Content UI, copy-to-clipboard, and error handling (#10278)
* ✨ feat: Refactor error handling and improve loading states in MessageContent component * ✨ feat: Enhance Thinking and ContentParts components with improved hover functionality and clipboard support * fix: Adjust padding in Thinking and ContentParts components for consistent layout * ✨ feat: Add response label and improve message editing UI with contextual indicators * ✨ feat: Add isEditing prop to Feedback and Fork components for improved editing state handling * refactor: Remove isEditing prop from Feedback and Fork components for cleaner state management * refactor: Migrate state management from Recoil to Jotai for font size and show thinking features * refactor: Separate ToggleSwitch into RecoilToggle and JotaiToggle components for improved clarity and state management * refactor: Remove unnecessary comments in ToggleSwitch and MessageContent components for cleaner code * chore: reorder import statements in Thinking.tsx * chore: reorder import statement in EditTextPart.tsx * chore: reorder import statement * chore: Reorganize imports in ToggleSwitch.tsx --------- Co-authored-by: Danny Avila <danny@librechat.ai> |
||
|
Dustin Healy
|
0446d0e190
|
♿ fix: Address Accessibility Issues (#10260)
* chore: add i18n localization comment for AlwaysMakeProd component * feat: enhance accessibility by adding aria-label and aria-labelledby to Switch component * feat: add aria-labels for accessibility in Agent and Assistant avatar buttons * fix: add switch aria-labels for accessibility in various components * feat: add aria-labels and localization keys for accessibility in DataTable, DataTableColumnHeader, and OGDialogTemplate components * chore: refactor out nested ternary * feat: add aria-label to DataTable filter button for My Files modal * feat: add aria-labels for Buttons and localization strings * feat: add aria-labels to Checkboxes in Agent Builder * feat: enhance accessibility by adding aria-label and aria-labelledby to Checkbox component * feat: add aria-label to FileSearchCheckbox in Agent Builder * feat: add aria-label to Prompts text input area * feat: enhance accessibility by adding aria-label and aria-labelledby to TextAreaAutosize component * feat: remove improper role: "list" prop from List in Conversations.tsx to enhance accessibility and stop aria rules conflicting within react-virtualized component * feat: enhance accessibility by allowing tab navigation and adding ring highlights for conversation title editing accept/reject buttons * feat: add aria-label to Copy Link button in the conversation share modal * feat: add title to QR code svg in conversation share modal to describe the image content * feat: enhance accessibility by making Agent Avatar upload keyboard navigable and round out highlight border on focus * feat: enhance accessibility by adding aria attributes around alerting users with screen readers to invalid email address inputs in the Agent Builder * feat: add aria-labels to buttons in Advanced panel of Agent Builder * feat: enhance accessibility by making FileUpload and Clear All buttons in PresetItems keyboard navigable * feat: enchance accessiblity by indexing view and delete button aria-labels in shared links management modal to their specific chat titles * feat: add border highlighting on focus for AnimatedSearchInput * feat: add category description to aria-labels for prompts in ListCard * feat: add proper scoping to rows and columns in table headers * feat: add localized aria-labelling to EditTextPart's TextAreaAutosize component and base dynamic paramters panel components and their supporting translation keys * feat: add localized aria-labels and aria-labelledBy to Checkbox components without them * feat: add localized aria-labeledBy for endpoint settings Sliders * feat: add localized aria-labels for TextareaAutosize components * chore: remove unused i18n string * feat: add localized aria-label for BookmarkForm Checkbox * fix: add stopPropagation onKeyDown for Preview and Edit menu items in prompts that was causing the prompts to inadvertently be sent when triggered with keyboard navigation when Auto-send Prompts was toggled on * fix: switch TableCell to TableHead for title cells according to harvard issue #789 * fix: add more descriptive localization key for file filter button in DataTable * chore: remove self-explanatory code comment from RenameForm * fix: remove stray bg-yellow highlight that was left in during debugging * fix: add aria-label to model configurator panel back button * fix: undo incorrect hoist of tool name split for aria-label and span in MCPInput --------- Co-authored-by: Danny Avila <danny@librechat.ai> |
||
|
Marco Beretta
|
d41b07c0af
|
♻️ refactor: Replace fontSize Recoil atom with Jotai (#10171)
* fix: reapply chat font size on load * refactor: streamline font size handling in localStorage * fix: update matchMedia mock to accurately reflect desktop and touchscreen capabilities * refactor: implement Jotai for font size management and initialize on app load - Replaced Recoil with Jotai for font size state management across components. - Added a new `fontSize` atom to handle font size changes and persist them in localStorage. - Implemented `initializeFontSize` function to apply saved font size on app load. - Updated relevant components to utilize the new font size atom. --------- Co-authored-by: ddooochii <ddooochii@gmail.com> Co-authored-by: Danny Avila <danny@librechat.ai> |
||
|
Danny Avila
|
fbe341a171
|
⚡ refactor: Latest Message Tracking with Robust Text Key Generation (#10059)
* chore: enhance logging for latest message actions in message components
* fix: Extract previous convoId from latest text in message helpers and process hooks
- Updated `useMessageHelpers` and `useMessageProcess` to extract `convoId` from the previous text key for improved message handling.
- Refactored `getLengthAndLastTenChars` to `getLengthAndLastNChars` for better flexibility in character length retrieval.
- Introduced `getLatestContentForKey` function to streamline content extraction from messages.
* chore: Enhance logging for clearing latest messages in conversation hooks
* refactor: Update message key formatting for improved URL parameter handling
- Modified `getLatestContentForKey` to change the format from `${text}-${i}` to `${text}&i=${i}` for better URL parameter structure.
- Adjusted `getTextKey` to increase character length retrieval from 12 to 16 in `getLengthAndLastNChars` for enhanced text processing.
* refactor: Simplify convoId extraction and enhance message formatting
- Updated `useMessageHelpers` and `useMessageProcess` to extract `convoId` using a new format for improved clarity.
- Refactored `getLatestContentForKey` to streamline content formatting and ensure consistent use of `Constants.COMMON_DIVIDER` for better message structure.
- Removed redundant length and last character extraction logic from `getLengthAndLastNChars` for cleaner code.
* chore: linting
* chore: Simplify pre-commit hook by removing unnecessary lines
|
||
|
Danny Avila
|
bcd97aad2f
|
📎 feat: Direct Provider Attachment Support for Multimodal Content (#9994)
Some checks failed
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Has been cancelled
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Has been cancelled
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Has been cancelled
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Has been cancelled
* 📎 feat: Direct Provider Attachment Support for Multimodal Content * 📑 feat: Anthropic Direct Provider Upload (#9072) * feat: implement Anthropic native PDF support with document preservation - Add comprehensive debug logging throughout PDF processing pipeline - Refactor attachment processing to separate image and document handling - Create distinct addImageURLs(), addDocuments(), and processAttachments() methods - Fix critical bugs in stream handling and parameter passing - Add streamToBuffer utility for proper stream-to-buffer conversion - Remove api/agents submodule from repository 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * chore: remove out of scope formatting changes * fix: stop duplication of file in chat on end of response stream * chore: bring back file search and ocr options * chore: localize upload to provider string in file menu * refactor: change createMenuItems args to fit new pattern introduced by anthropic-native-pdf-support * feat: add cache point for pdfs processed by anthropic endpoint since they are unlikely to change and should benefit from caching * feat: combine Upload Image into Upload to Provider since they both perform direct upload and change provider upload icon to reflect multimodal upload * feat: add citations support according to docs * refactor: remove redundant 'document' check since documents are handled properly by formatMessage in the agents repo now * refactor: change upload logic so anthropic endpoint isn't exempted from normal upload path using Agents for consistency with the rest of the upload logic * fix: include width and height in return from uploadLocalFile so images are correctly identified when going through an AgentUpload in addImageURLs * chore: remove client specific handling since the direct provider stuff is handled by the agent client * feat: handle documents in AgentClient so no need for change to agents repo * chore: removed unused changes * chore: remove auto generated comments from OG commit * feat: add logic for agents to use direct to provider uploads if supported (currently just anthropic) * fix: reintroduce role check to fix render error because of undefined value for Content Part * fix: actually fix render bug by using proper isCreatedByUser check and making sure our mutation of formattedMessage.content is consistent --------- Co-authored-by: Andres Restrepo <andres@thelinuxkid.com> Co-authored-by: Claude <noreply@anthropic.com> 📁 feat: Send Attachments Directly to Provider (OpenAI) (#9098) * refactor: change references from direct upload to direct attach to better reflect functionality since we are just using base64 encoding strategy now rather than Files/File API for sending our attachments directly to the provider, the upload nomenclature no longer makes sense. direct_attach better describes the different methods of sending attachments to providers anyways even if we later introduce direct upload support * feat: add upload to provider option for openai (and agent) ui * chore: move anthropic pdf validator over to packages/api * feat: simple pdf validation according to openai docs * feat: add provider agnostic validatePdf logic to start handling multiple endpoints * feat: add handling for openai specific documentPart formatting * refactor: move require statement to proper place at top of file * chore: add in openAI endpoint for the rest of the document handling logic * feat: add direct attach support for azureOpenAI endpoint and agents * feat: add pdf validation for azureOpenAI endpoint * refactor: unify all the endpoint checks with isDocumentSupportedEndpoint * refactor: consolidate Upload to Provider vs Upload image logic for clarity * refactor: remove anthropic from anthropic_multimodal fileType since we support multiple providers now 🗂️ feat: Send Attachments Directly to Provider (Google) (#9100) * feat: add validation for google PDFs and add google endpoint as a document supporting endpoint * feat: add proper pdf formatting for google endpoints (requires PR #14 in agents) * feat: add multimodal support for google endpoint attachments * feat: add audio file svg * fix: refactor attachments logic so multi-attachment messages work properly * feat: add video file svg * fix: allows for followup questions of uploaded multimodal attachments * fix: remove incorrect final message filtering that was breaking Attachment component rendering fix: manualy rename 'documents' to 'Documents' in git since it wasn't picked up due to case insensitivity in dir name fix: add logic so filepicker for a google agent has proper filetype filtering 🛫 refactor: Move Encoding Logic to packages/api (#9182) * refactor: move audio encode over to TS * refactor: audio encoding now functional in LC again * refactor: move video encode over to TS * refactor: move document encode over to TS * refactor: video encoding now functional in LC again * refactor: document encoding now functional in LC again * fix: extend file type options in AttachFileMenu to include 'google_multimodal' and update dependency array to include agent?.provider * feat: only accept pdfs if responses api is enabled for openai convos chore: address ESLint comments chore: add missing audio mimetype * fix: type safety for message content parts and improve null handling * chore: reorder AttachFileMenuProps for consistency and clarity * chore: import order in AttachFileMenu * fix: improve null handling for text parts in parseTextParts function * fix: remove no longer used unsupported capability error message for file uploads * fix: OpenAI Direct File Attachment Format * fix: update encodeAndFormatDocuments to support OpenAI responses API and enhance document result types * refactor: broaden providers supported for documents * feat: enhance DragDrop context and modal to support document uploads based on provider capabilities * fix: reorder import statements for consistency in video encoding module --------- Co-authored-by: Dustin Healy <54083382+dustinhealy@users.noreply.github.com> |
||
|
Danny Avila
|
aae3694b11
|
🛠️ chore: Typing and Remove Comments (#9732)
* chore: Update documentation for formatToolContent function, remove JSDoc types and duplicate comments * chore: fix type errors due to attachment.filename in Attachment component |
||
|
Danny Avila
|
45ab4d4503
|
🎋 refactor: Improve Message UI State Handling (#9678)
* refactor: `ExecuteCode` component with submission state handling and cancellation message * fix: Remove unnecessary argument check for execute_code tool call * refactor: streamlined messages context * chore: remove unused Convo prop * chore: remove unnecessary whitespace in Message component * refactor: enhance message context with submission state and latest message tracking * chore: import order |