LibreChat/client/src/utils/messages.ts

274 lines
8.8 KiB
TypeScript
Raw Normal View History

⏸ refactor: Improve UX for Parallel Streams (Multi-Convo) (#11096) * 🌊 feat: Implement multi-conversation feature with added conversation context and payload adjustments * refactor: Replace isSubmittingFamily with isSubmitting across message components for consistency * feat: Add loadAddedAgent and processAddedConvo for multi-conversation agent execution * refactor: Update ContentRender usage to conditionally render PlaceholderRow based on isLast and isSubmitting * WIP: first pass, sibling index * feat: Enhance multi-conversation support with agent tracking and display improvements * refactor: Introduce isEphemeralAgentId utility and update related logic for agent handling * refactor: Implement createDualMessageContent utility for sibling message display and enhance useStepHandler for added conversations * refactor: duplicate tools for added agent if ephemeral and primary agent is also ephemeral * chore: remove deprecated multimessage rendering * refactor: enhance dual message content creation and agent handling for parallel rendering * refactor: streamline message rendering and submission handling by removing unused state and optimizing conditional logic * refactor: adjust content handling in parallel mode to utilize existing content for improved agent display * refactor: update @librechat/agents dependency to version 3.0.53 * refactor: update @langchain/core and @librechat/agents dependencies to latest versions * refactor: remove deprecated @langchain/core dependency from package.json * chore: remove unused SearchToolConfig and GetSourcesParams types from web.ts * refactor: remove unused message properties from Message component * refactor: enhance parallel content handling with groupId support in ContentParts and useStepHandler * refactor: implement parallel content styling in Message, MessageRender, and ContentRender components. use explicit model name * refactor: improve agent ID handling in createDualMessageContent for dual message display * refactor: simplify title generation in AddedConvo by removing unused sender and preset logic * refactor: replace string interpolation with cn utility for className in HoverButtons component * refactor: enhance agent ID handling by adding suffix management for parallel agents and updating related components * refactor: enhance column ordering in ContentParts by sorting agents with suffix management * refactor: update @librechat/agents dependency to version 3.0.55 * feat: implement parallel content rendering with metadata support - Added `ParallelContentRenderer` and `ParallelColumns` components for rendering messages in parallel based on groupId and agentId. - Introduced `contentMetadataMap` to store metadata for each content part, allowing efficient parallel content detection. - Updated `Message` and `ContentRender` components to utilize the new metadata structure for rendering. - Modified `useStepHandler` to manage content indices and metadata during message processing. - Enhanced `IJobStore` interface and its implementations to support storing and retrieving content metadata. - Updated data schemas to include `contentMetadataMap` for messages, enabling multi-agent and parallel execution scenarios. * refactor: update @librechat/agents dependency to version 3.0.56 * refactor: remove unused EPHEMERAL_AGENT_ID constant and simplify agent ID check * refactor: enhance multi-agent message processing and primary agent determination * refactor: implement branch message functionality for parallel responses * refactor: integrate added conversation retrieval into message editing and regeneration processes * refactor: remove unused isCard and isMultiMessage props from MessageRender and ContentRender components * refactor: update @librechat/agents dependency to version 3.0.60 * refactor: replace usage of EPHEMERAL_AGENT_ID constant with isEphemeralAgentId function for improved clarity and consistency * refactor: standardize agent ID format in tests for consistency * chore: move addedConvo property to the correct position in payload construction * refactor: rename agent_id values in loadAgent tests for clarity * chore: reorder props in ContentParts component for improved readability * refactor: rename variable 'content' to 'result' for clarity in RedisJobStore tests * refactor: streamline useMessageActions by removing duplicate handleFeedback assignment * chore: revert placeholder rendering logic MessageRender and ContentRender components to original * refactor: implement useContentMetadata hook for optimized content metadata handling * refactor: remove contentMetadataMap and related logic from the codebase and revert back to agentId/groupId in content parts - Eliminated contentMetadataMap from various components and services, simplifying the handling of message content. - Updated functions to directly access agentId and groupId from content parts instead of relying on a separate metadata map. - Adjusted related hooks and components to reflect the removal of contentMetadataMap, ensuring consistent handling of message content. - Updated tests and documentation to align with the new structure of message content handling. * refactor: remove logging from groupParallelContent function to clean up output * refactor: remove model parameter from TBranchMessageRequest type for simplification * refactor: enhance branch message creation by stripping metadata for standalone content * chore: streamline branch message creation by simplifying content filtering and removing unnecessary metadata checks * refactor: include attachments in branch message creation for improved content handling * refactor: streamline agent content processing by consolidating primary agent identification and filtering logic * refactor: simplify multi-agent message processing by creating a dedicated mapping method and enhancing content filtering * refactor: remove unused parameter from loadEphemeralAgent function for cleaner code * refactor: update groupId handling in metadata to only set when provided by the server
2025-12-25 01:43:54 -05:00
import {
QueryKeys,
Constants,
ContentTypes,
getResponseSender,
isEphemeralAgentId,
appendAgentIdSuffix,
encodeEphemeralAgentId,
} from 'librechat-data-provider';
import type {
TMessage,
TConversation,
TEndpointsConfig,
TMessageContentParts,
} from 'librechat-data-provider';
import type { QueryClient } from '@tanstack/react-query';
import type { LocalizeFunction } from '~/common';
import _ from 'lodash';
⚡ refactor: Latest Message Tracking with Robust Text Key Generation (#10059) * chore: enhance logging for latest message actions in message components * fix: Extract previous convoId from latest text in message helpers and process hooks - Updated `useMessageHelpers` and `useMessageProcess` to extract `convoId` from the previous text key for improved message handling. - Refactored `getLengthAndLastTenChars` to `getLengthAndLastNChars` for better flexibility in character length retrieval. - Introduced `getLatestContentForKey` function to streamline content extraction from messages. * chore: Enhance logging for clearing latest messages in conversation hooks * refactor: Update message key formatting for improved URL parameter handling - Modified `getLatestContentForKey` to change the format from `${text}-${i}` to `${text}&i=${i}` for better URL parameter structure. - Adjusted `getTextKey` to increase character length retrieval from 12 to 16 in `getLengthAndLastNChars` for enhanced text processing. * refactor: Simplify convoId extraction and enhance message formatting - Updated `useMessageHelpers` and `useMessageProcess` to extract `convoId` using a new format for improved clarity. - Refactored `getLatestContentForKey` to streamline content formatting and ensure consistent use of `Constants.COMMON_DIVIDER` for better message structure. - Removed redundant length and last character extraction logic from `getLengthAndLastNChars` for cleaner code. * chore: linting * chore: Simplify pre-commit hook by removing unnecessary lines
2025-10-10 11:22:16 +03:00
export const TEXT_KEY_DIVIDER = '|||';
export const getLatestText = (message?: TMessage | null, includeIndex?: boolean): string => {
if (!message) {
return '';
}
if (message.text) {
return message.text;
}
🔖 feat: Enhance Bookmarks UX, add RBAC, toggle via `librechat.yaml` (#3747) * chore: update package version to 0.7.416 * chore: Update Role.js imports order * refactor: move updateTagsInConvo to tags route, add RBAC for tags * refactor: add updateTagsInConvoOptions * fix: loading state for bookmark form * refactor: update primaryText class in TitleButton component * refactor: remove duplicate bookmarks and theming * refactor: update EditIcon component to use React.forwardRef * refactor: add _id field to tConversationTagSchema * refactor: remove promises * refactor: move mutation logic from BookmarkForm -> BookmarkEditDialog * refactor: update button class in BookmarkForm component * fix: conversation mutations and add better logging to useConversationTagMutation * refactor: update logger message in BookmarkEditDialog component * refactor: improve UI consistency in BookmarkNav and NewChat components * refactor: update logger message in BookmarkEditDialog component * refactor: Add tags prop to BookmarkForm component * refactor: Update BookmarkForm to avoid tag mutation if the tag already exists; also close dialog on submission programmatically * refactor: general role helper function to support updating access permissions for different permission types * refactor: Update getLatestText function to handle undefined values in message.content * refactor: Update useHasAccess hook to handle null role values for authenticated users * feat: toggle bookmarks access * refactor: Update PromptsCommand to handle access permissions for prompts * feat: updateConversationSelector * refactor: rename `vars` to `tagToDelete` for clarity * fix: prevent recreation of deleted tags in BookmarkMenu on Item Click * ci: mock updateBookmarksAccess function * ci: mock updateBookmarksAccess function
2024-08-22 17:09:05 -04:00
if (message.content && message.content.length > 0) {
for (let i = message.content.length - 1; i >= 0; i--) {
🎉 feat: Code Interpreter API and Agents Release (#4860) * feat: Code Interpreter API & File Search Agent Uploads chore: add back code files wip: first pass, abstract key dialog refactor: influence checkbox on key changes refactor: update localization keys for 'execute code' to 'run code' wip: run code button refactor: add throwError parameter to loadAuthValues and getUserPluginAuthValue functions feat: first pass, API tool calling fix: handle missing toolId in callTool function and return 404 for non-existent tools feat: show code outputs fix: improve error handling in callTool function and log errors fix: handle potential null value for filepath in attachment destructuring fix: normalize language before rendering and prevent null return fix: add loading indicator in RunCode component while executing code feat: add support for conditional code execution in Markdown components feat: attachments refactor: remove bash fix: pass abort signal to graph/run refactor: debounce and rate limit tool call refactor: increase debounce delay for execute function feat: set code output attachments feat: image attachments refactor: apply message context refactor: pass `partIndex` feat: toolCall schema/model/methods feat: block indexing feat: get tool calls chore: imports chore: typing chore: condense type imports feat: get tool calls fix: block indexing chore: typing refactor: update tool calls mapping to support multiple results fix: add unique key to nav link for rendering wip: first pass, tool call results refactor: update query cache from successful tool call mutation style: improve result switcher styling chore: note on using \`.toObject()\` feat: add agent_id field to conversation schema chore: typing refactor: rename agentMap to agentsMap for consistency feat: Agent Name as chat input placeholder chore: bump agents 📦 chore: update @langchain dependencies to latest versions to match agents package 📦 chore: update @librechat/agents dependency to version 1.8.0 fix: Aborting agent stream removes sender; fix(bedrock): completion removes preset name label refactor: remove direct file parameter to use req.file, add `processAgentFileUpload` for image uploads feat: upload menu feat: prime message_file resources feat: implement conversation access validation in chat route refactor: remove file parameter from processFileUpload and use req.file instead feat: add savedMessageIds set to track saved message IDs in BaseClient, to prevent unnecessary double-write to db feat: prevent duplicate message saves by checking savedMessageIds in AgentController refactor: skip legacy RAG API handling for agents feat: add files field to convoSchema refactor: update request type annotations from Express.Request to ServerRequest in file processing functions feat: track conversation files fix: resendFiles, addPreviousAttachments handling feat: add ID validation for session_id and file_id in download route feat: entity_id for code file uploads/downloads fix: code file edge cases feat: delete related tool calls feat: add stream rate handling for LLM configuration feat: enhance system content with attached file information fix: improve error logging in resource priming function * WIP: PoC, sequential agents WIP: PoC Sequential Agents, first pass content data + bump agents package fix: package-lock WIP: PoC, o1 support, refactor bufferString feat: convertJsonSchemaToZod fix: form issues and schema defining erroneous model fix: max length issue on agent form instructions, limit conversation messages to sequential agents feat: add abort signal support to createRun function and AgentClient feat: PoC, hide prior sequential agent steps fix: update parameter naming from config to metadata in event handlers for clarity, add model to usage data refactor: use only last contentData, track model for usage data chore: bump agents package fix: content parts issue refactor: filter contentParts to include tool calls and relevant indices feat: show function calls refactor: filter context messages to exclude tool calls when no tools are available to the agent fix: ensure tool call content is not undefined in formatMessages feat: add agent_id field to conversationPreset schema feat: hide sequential agents feat: increase upload toast duration to 10 seconds * refactor: tool context handling & update Code API Key Dialog feat: toolContextMap chore: skipSpecs -> useSpecs ci: fix handleTools tests feat: API Key Dialog * feat: Agent Permissions Admin Controls feat: replace label with button for prompt permission toggle feat: update agent permissions feat: enable experimental agents and streamline capability configuration feat: implement access control for agents and enhance endpoint menu items feat: add welcome message for agent selection in localization feat: add agents permission to access control and update version to 0.7.57 * fix: update types in useAssistantListMap and useMentions hooks for better null handling * feat: mention agents * fix: agent tool resource race conditions when deleting agent tool resource files * feat: add error handling for code execution with user feedback * refactor: rename AdminControls to AdminSettings for clarity * style: add gap to button in AdminSettings for improved layout * refactor: separate agent query hooks and check access to enable fetching * fix: remove unused provider from agent initialization options, creates issue with custom endpoints * refactor: remove redundant/deprecated modelOptions from AgentClient processes * chore: update @librechat/agents to version 1.8.5 in package.json and package-lock.json * fix: minor styling issues + agent panel uniformity * fix: agent edge cases when set endpoint is no longer defined * refactor: remove unused cleanup function call from AppService * fix: update link in ApiKeyDialog to point to pricing page * fix: improve type handling and layout calculations in SidePanel component * fix: add missing localization string for agent selection in SidePanel * chore: form styling and localizations for upload filesearch/code interpreter * fix: model selection placeholder logic in AgentConfig component * style: agent capabilities * fix: add localization for provider selection and improve dropdown styling in ModelPanel * refactor: use gpt-4o-mini > gpt-3.5-turbo * fix: agents configuration for loadDefaultInterface and update related tests * feat: DALLE Agents support
2024-12-04 15:48:13 -05:00
const part = message.content[i] as TMessageContentParts | undefined;
if (part && part.type !== ContentTypes.TEXT) {
continue;
}
📎 feat: Direct Provider Attachment Support for Multimodal Content (#9994) * 📎 feat: Direct Provider Attachment Support for Multimodal Content * 📑 feat: Anthropic Direct Provider Upload (#9072) * feat: implement Anthropic native PDF support with document preservation - Add comprehensive debug logging throughout PDF processing pipeline - Refactor attachment processing to separate image and document handling - Create distinct addImageURLs(), addDocuments(), and processAttachments() methods - Fix critical bugs in stream handling and parameter passing - Add streamToBuffer utility for proper stream-to-buffer conversion - Remove api/agents submodule from repository 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * chore: remove out of scope formatting changes * fix: stop duplication of file in chat on end of response stream * chore: bring back file search and ocr options * chore: localize upload to provider string in file menu * refactor: change createMenuItems args to fit new pattern introduced by anthropic-native-pdf-support * feat: add cache point for pdfs processed by anthropic endpoint since they are unlikely to change and should benefit from caching * feat: combine Upload Image into Upload to Provider since they both perform direct upload and change provider upload icon to reflect multimodal upload * feat: add citations support according to docs * refactor: remove redundant 'document' check since documents are handled properly by formatMessage in the agents repo now * refactor: change upload logic so anthropic endpoint isn't exempted from normal upload path using Agents for consistency with the rest of the upload logic * fix: include width and height in return from uploadLocalFile so images are correctly identified when going through an AgentUpload in addImageURLs * chore: remove client specific handling since the direct provider stuff is handled by the agent client * feat: handle documents in AgentClient so no need for change to agents repo * chore: removed unused changes * chore: remove auto generated comments from OG commit * feat: add logic for agents to use direct to provider uploads if supported (currently just anthropic) * fix: reintroduce role check to fix render error because of undefined value for Content Part * fix: actually fix render bug by using proper isCreatedByUser check and making sure our mutation of formattedMessage.content is consistent --------- Co-authored-by: Andres Restrepo <andres@thelinuxkid.com> Co-authored-by: Claude <noreply@anthropic.com> 📁 feat: Send Attachments Directly to Provider (OpenAI) (#9098) * refactor: change references from direct upload to direct attach to better reflect functionality since we are just using base64 encoding strategy now rather than Files/File API for sending our attachments directly to the provider, the upload nomenclature no longer makes sense. direct_attach better describes the different methods of sending attachments to providers anyways even if we later introduce direct upload support * feat: add upload to provider option for openai (and agent) ui * chore: move anthropic pdf validator over to packages/api * feat: simple pdf validation according to openai docs * feat: add provider agnostic validatePdf logic to start handling multiple endpoints * feat: add handling for openai specific documentPart formatting * refactor: move require statement to proper place at top of file * chore: add in openAI endpoint for the rest of the document handling logic * feat: add direct attach support for azureOpenAI endpoint and agents * feat: add pdf validation for azureOpenAI endpoint * refactor: unify all the endpoint checks with isDocumentSupportedEndpoint * refactor: consolidate Upload to Provider vs Upload image logic for clarity * refactor: remove anthropic from anthropic_multimodal fileType since we support multiple providers now 🗂️ feat: Send Attachments Directly to Provider (Google) (#9100) * feat: add validation for google PDFs and add google endpoint as a document supporting endpoint * feat: add proper pdf formatting for google endpoints (requires PR #14 in agents) * feat: add multimodal support for google endpoint attachments * feat: add audio file svg * fix: refactor attachments logic so multi-attachment messages work properly * feat: add video file svg * fix: allows for followup questions of uploaded multimodal attachments * fix: remove incorrect final message filtering that was breaking Attachment component rendering fix: manualy rename 'documents' to 'Documents' in git since it wasn't picked up due to case insensitivity in dir name fix: add logic so filepicker for a google agent has proper filetype filtering 🛫 refactor: Move Encoding Logic to packages/api (#9182) * refactor: move audio encode over to TS * refactor: audio encoding now functional in LC again * refactor: move video encode over to TS * refactor: move document encode over to TS * refactor: video encoding now functional in LC again * refactor: document encoding now functional in LC again * fix: extend file type options in AttachFileMenu to include 'google_multimodal' and update dependency array to include agent?.provider * feat: only accept pdfs if responses api is enabled for openai convos chore: address ESLint comments chore: add missing audio mimetype * fix: type safety for message content parts and improve null handling * chore: reorder AttachFileMenuProps for consistency and clarity * chore: import order in AttachFileMenu * fix: improve null handling for text parts in parseTextParts function * fix: remove no longer used unsupported capability error message for file uploads * fix: OpenAI Direct File Attachment Format * fix: update encodeAndFormatDocuments to support OpenAI responses API and enhance document result types * refactor: broaden providers supported for documents * feat: enhance DragDrop context and modal to support document uploads based on provider capabilities * fix: reorder import statements for consistency in video encoding module --------- Co-authored-by: Dustin Healy <54083382+dustinhealy@users.noreply.github.com>
2025-10-06 17:30:16 -04:00
const text = (typeof part?.text === 'string' ? part.text : part?.text?.value) ?? '';
if (text.length > 0) {
🔖 feat: Enhance Bookmarks UX, add RBAC, toggle via `librechat.yaml` (#3747) * chore: update package version to 0.7.416 * chore: Update Role.js imports order * refactor: move updateTagsInConvo to tags route, add RBAC for tags * refactor: add updateTagsInConvoOptions * fix: loading state for bookmark form * refactor: update primaryText class in TitleButton component * refactor: remove duplicate bookmarks and theming * refactor: update EditIcon component to use React.forwardRef * refactor: add _id field to tConversationTagSchema * refactor: remove promises * refactor: move mutation logic from BookmarkForm -> BookmarkEditDialog * refactor: update button class in BookmarkForm component * fix: conversation mutations and add better logging to useConversationTagMutation * refactor: update logger message in BookmarkEditDialog component * refactor: improve UI consistency in BookmarkNav and NewChat components * refactor: update logger message in BookmarkEditDialog component * refactor: Add tags prop to BookmarkForm component * refactor: Update BookmarkForm to avoid tag mutation if the tag already exists; also close dialog on submission programmatically * refactor: general role helper function to support updating access permissions for different permission types * refactor: Update getLatestText function to handle undefined values in message.content * refactor: Update useHasAccess hook to handle null role values for authenticated users * feat: toggle bookmarks access * refactor: Update PromptsCommand to handle access permissions for prompts * feat: updateConversationSelector * refactor: rename `vars` to `tagToDelete` for clarity * fix: prevent recreation of deleted tags in BookmarkMenu on Item Click * ci: mock updateBookmarksAccess function * ci: mock updateBookmarksAccess function
2024-08-22 17:09:05 -04:00
if (includeIndex === true) {
return `${text}-${i}`;
} else {
return text;
}
🚧 chore: merge latest dev build to main repo (#3844) * agents - phase 1 (#30) * chore: copy assistant files * feat: frontend and data-provider * feat: backend get endpoint test * fix(MessageEndpointIcon): switched to AgentName and AgentAvatar * fix: small fixes * fix: agent endpoint config * fix: show Agent Builder * chore: install agentus * chore: initial scaffolding for agents * fix: updated Assistant logic to Agent Logic for some Agent components * WIP first pass, demo of agent package * WIP: initial backend infra for agents * fix: agent list error * wip: agents routing * chore: Refactor useSSE hook to handle different data events * wip: correctly emit events * chore: Update @librechat/agentus npm dependency to version 1.0.9 * remove comment * first pass: streaming agent text * chore: Remove @librechat/agentus root-level workspace npm dependency * feat: Agent Schema and Model * fix: content handling fixes * fix: content message save * WIP: new content data * fix: run step issue with tool calls * chore: Update @librechat/agentus npm dependency to version 1.1.5 * feat: update controller and agent routes * wip: initial backend tool and tool error handling support * wip: tool chunks * chore: Update @librechat/agentus npm dependency to version 1.1.7 * chore: update tool_call typing, add test conditions and logs * fix: create agent * fix: create agent * first pass: render completed content parts * fix: remove logging, fix step handler typing * chore: Update @librechat/agentus npm dependency to version 1.1.9 * refactor: cleanup maps on unmount * chore: Update BaseClient.js to safely count tokens for string, number, and boolean values * fix: support subsequent messages with tool_calls * chore: export order * fix: select agent * fix: tool call types and handling * chore: switch to anthropic for testing * fix: AgentSelect * refactor: experimental: OpenAIClient to use array for intermediateReply * fix(useSSE): revert old condition for streaming legacy client tokens * fix: lint * revert `agent_id` to `id` * chore: update localization keys for agent-related components * feat: zod schema handling for actions * refactor(actions): if no params, no zodSchema * chore: Update @librechat/agentus npm dependency to version 1.2.1 * feat: first pass, actions * refactor: empty schema for actions without params * feat: Update createRun function to accept additional options * fix: message payload formatting; feat: add more client options * fix: ToolCall component rendering when action has no args but has output * refactor(ToolCall): allow non-stringy args * WIP: first pass, correctly formatted tool_calls between providers * refactor: Remove duplicate import of 'roles' module * refactor: Exclude 'vite.config.ts' from TypeScript compilation * refactor: fix agent related types > - no need to use endpoint/model fields for identifying agent metadata > - add `provider` distinction for agent-configured 'endpoint' - no need for agent-endpoint map - reduce complexity of tools as functions into tools as string[] - fix types related to above changes - reduce unnecessary variables for queries/mutations and corresponding react-query keys * refactor: Add tools and tool_kwargs fields to agent schema * refactor: Remove unused code and update dependencies * refactor: Update updateAgentHandler to use req.body directly * refactor: Update AgentSelect component to use localized hooks * refactor: Update agent schema to include tools and provider fields * refactor(AgentPanel): add scrollbar gutter, add provider field to form, fix agent schema required values * refactor: Update AgentSwitcher component to use selectedAgentId instead of selectedAgent * refactor: Update AgentPanel component to include alternateName import and defaultAgentFormValues * refactor(SelectDropDown): allow setting value as option while still supporting legacy usage (string values only) * refactor: SelectDropdown changes - Only necessary when the available values are objects with label/value fields and the selected value is expected to be a string. * refactor: TypeError issues and handle provider as option * feat: Add placeholder for provider selection in AgentPanel component * refactor: Update agent schema to include author and provider fields * fix: show expected 'create agent' placeholder when creating agent * chore: fix localization strings, hide capabilities form for now * chore: typing * refactor: import order and use compact agents schema for now * chore: typing * refactor: Update AgentForm type to use AgentCapabilities * fix agent form agent selection issues * feat: responsive agent selection * fix: Handle cancelled fetch in useSelectAgent hook * fix: reset agent form on accordion close/open * feat: Add agent_id to default conversation for agents endpoint * feat: agents endpoint request handling * refactor: reset conversation model on agent select * refactor: add `additional_instructions` to conversation schema, organize other fields * chore: casing * chore: types * refactor(loadAgentTools): explicitly pass agent_id, do not pass `model` to loadAgentTools for now, load action sets by agent_id * WIP: initial draft of real agent client initialization * WIP: first pass, anthropic agent requests * feat: remember last selected agent * feat: openai and azure connected * fix: prioritize agent model for runs unless an explicit override model is passed from client * feat: Agent Actions * fix: save agent id to convo * feat: model panel (#29) * feat: model panel * bring back comments * fix: method still null * fix: AgentPanel FormContext * feat: add more parameters * fix: style issues; refactor: Agent Controller * fix: cherry-pick * fix: Update AgentAvatar component to use AssistantIcon instead of BrainCircuit * feat: OGDialog for delete agent; feat(assistant): update Agent types, introduced `model_parameters` * feat: icon and general `model_parameters` update * feat: use react-hook-form better * fix: agent builder form reset issue when switching panels * refactor: modularize agent builder form --------- Co-authored-by: Danny Avila <danny@librechat.ai> * fix: AgentPanel and ModelPanel type issues and use `useFormContext` and `watch` instead of `methods` directly and `useWatch`. * fix: tool call issues due to invalid input (anthropic) of empty string * fix: handle empty text in Part component --------- Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com> * refactor: remove form ModelPanel and fixed nested ternary expressions in AgentConfig * fix: Model Parameters not saved correctly * refactor: remove console log * feat: avatar upload and get for Agents (#36) Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com> * chore: update to public package * fix: typing, optional chaining * fix: cursor not showing for content parts * chore: conditionally enable agents * ci: fix azure test * ci: fix frontend tests, fix eslint api * refactor: Remove unused errorContentPart variable * continue of the agent message PR (#40) * last fixes * fix: agentMap * pr merge test (#41) * fix: model icon not fetching correctly * remove console logs * feat: agent name * refactor: pass documentsMap as a prop to allow re-render of assistant form * refactor: pass documentsMap as a prop to allow re-render of assistant form * chore: Bump version to 0.7.419 * fix: TypeError: Cannot read properties of undefined (reading 'id') * refactor: update AgentSwitcher component to use ControlCombobox instead of Combobox --------- Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
2024-08-31 16:33:51 -04:00
} else {
continue;
}
}
}
return '';
};
export const getAllContentText = (message?: TMessage | null): string => {
if (!message) {
return '';
}
if (message.text) {
return message.text;
}
if (message.content && message.content.length > 0) {
return message.content
.filter((part) => part != null && part.type === ContentTypes.TEXT)
📎 feat: Direct Provider Attachment Support for Multimodal Content (#9994) * 📎 feat: Direct Provider Attachment Support for Multimodal Content * 📑 feat: Anthropic Direct Provider Upload (#9072) * feat: implement Anthropic native PDF support with document preservation - Add comprehensive debug logging throughout PDF processing pipeline - Refactor attachment processing to separate image and document handling - Create distinct addImageURLs(), addDocuments(), and processAttachments() methods - Fix critical bugs in stream handling and parameter passing - Add streamToBuffer utility for proper stream-to-buffer conversion - Remove api/agents submodule from repository 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * chore: remove out of scope formatting changes * fix: stop duplication of file in chat on end of response stream * chore: bring back file search and ocr options * chore: localize upload to provider string in file menu * refactor: change createMenuItems args to fit new pattern introduced by anthropic-native-pdf-support * feat: add cache point for pdfs processed by anthropic endpoint since they are unlikely to change and should benefit from caching * feat: combine Upload Image into Upload to Provider since they both perform direct upload and change provider upload icon to reflect multimodal upload * feat: add citations support according to docs * refactor: remove redundant 'document' check since documents are handled properly by formatMessage in the agents repo now * refactor: change upload logic so anthropic endpoint isn't exempted from normal upload path using Agents for consistency with the rest of the upload logic * fix: include width and height in return from uploadLocalFile so images are correctly identified when going through an AgentUpload in addImageURLs * chore: remove client specific handling since the direct provider stuff is handled by the agent client * feat: handle documents in AgentClient so no need for change to agents repo * chore: removed unused changes * chore: remove auto generated comments from OG commit * feat: add logic for agents to use direct to provider uploads if supported (currently just anthropic) * fix: reintroduce role check to fix render error because of undefined value for Content Part * fix: actually fix render bug by using proper isCreatedByUser check and making sure our mutation of formattedMessage.content is consistent --------- Co-authored-by: Andres Restrepo <andres@thelinuxkid.com> Co-authored-by: Claude <noreply@anthropic.com> 📁 feat: Send Attachments Directly to Provider (OpenAI) (#9098) * refactor: change references from direct upload to direct attach to better reflect functionality since we are just using base64 encoding strategy now rather than Files/File API for sending our attachments directly to the provider, the upload nomenclature no longer makes sense. direct_attach better describes the different methods of sending attachments to providers anyways even if we later introduce direct upload support * feat: add upload to provider option for openai (and agent) ui * chore: move anthropic pdf validator over to packages/api * feat: simple pdf validation according to openai docs * feat: add provider agnostic validatePdf logic to start handling multiple endpoints * feat: add handling for openai specific documentPart formatting * refactor: move require statement to proper place at top of file * chore: add in openAI endpoint for the rest of the document handling logic * feat: add direct attach support for azureOpenAI endpoint and agents * feat: add pdf validation for azureOpenAI endpoint * refactor: unify all the endpoint checks with isDocumentSupportedEndpoint * refactor: consolidate Upload to Provider vs Upload image logic for clarity * refactor: remove anthropic from anthropic_multimodal fileType since we support multiple providers now 🗂️ feat: Send Attachments Directly to Provider (Google) (#9100) * feat: add validation for google PDFs and add google endpoint as a document supporting endpoint * feat: add proper pdf formatting for google endpoints (requires PR #14 in agents) * feat: add multimodal support for google endpoint attachments * feat: add audio file svg * fix: refactor attachments logic so multi-attachment messages work properly * feat: add video file svg * fix: allows for followup questions of uploaded multimodal attachments * fix: remove incorrect final message filtering that was breaking Attachment component rendering fix: manualy rename 'documents' to 'Documents' in git since it wasn't picked up due to case insensitivity in dir name fix: add logic so filepicker for a google agent has proper filetype filtering 🛫 refactor: Move Encoding Logic to packages/api (#9182) * refactor: move audio encode over to TS * refactor: audio encoding now functional in LC again * refactor: move video encode over to TS * refactor: move document encode over to TS * refactor: video encoding now functional in LC again * refactor: document encoding now functional in LC again * fix: extend file type options in AttachFileMenu to include 'google_multimodal' and update dependency array to include agent?.provider * feat: only accept pdfs if responses api is enabled for openai convos chore: address ESLint comments chore: add missing audio mimetype * fix: type safety for message content parts and improve null handling * chore: reorder AttachFileMenuProps for consistency and clarity * chore: import order in AttachFileMenu * fix: improve null handling for text parts in parseTextParts function * fix: remove no longer used unsupported capability error message for file uploads * fix: OpenAI Direct File Attachment Format * fix: update encodeAndFormatDocuments to support OpenAI responses API and enhance document result types * refactor: broaden providers supported for documents * feat: enhance DragDrop context and modal to support document uploads based on provider capabilities * fix: reorder import statements for consistency in video encoding module --------- Co-authored-by: Dustin Healy <54083382+dustinhealy@users.noreply.github.com>
2025-10-06 17:30:16 -04:00
.map((part) => {
if (!('text' in part)) return '';
const text = part.text;
if (typeof text === 'string') return text;
return text?.value || '';
})
.filter((text) => text.length > 0)
.join('\n');
}
return '';
};
⚡ refactor: Latest Message Tracking with Robust Text Key Generation (#10059) * chore: enhance logging for latest message actions in message components * fix: Extract previous convoId from latest text in message helpers and process hooks - Updated `useMessageHelpers` and `useMessageProcess` to extract `convoId` from the previous text key for improved message handling. - Refactored `getLengthAndLastTenChars` to `getLengthAndLastNChars` for better flexibility in character length retrieval. - Introduced `getLatestContentForKey` function to streamline content extraction from messages. * chore: Enhance logging for clearing latest messages in conversation hooks * refactor: Update message key formatting for improved URL parameter handling - Modified `getLatestContentForKey` to change the format from `${text}-${i}` to `${text}&i=${i}` for better URL parameter structure. - Adjusted `getTextKey` to increase character length retrieval from 12 to 16 in `getLengthAndLastNChars` for enhanced text processing. * refactor: Simplify convoId extraction and enhance message formatting - Updated `useMessageHelpers` and `useMessageProcess` to extract `convoId` using a new format for improved clarity. - Refactored `getLatestContentForKey` to streamline content formatting and ensure consistent use of `Constants.COMMON_DIVIDER` for better message structure. - Removed redundant length and last character extraction logic from `getLengthAndLastNChars` for cleaner code. * chore: linting * chore: Simplify pre-commit hook by removing unnecessary lines
2025-10-10 11:22:16 +03:00
const getLatestContentForKey = (message: TMessage): string => {
const formatText = (str: string, index: number): string => {
if (str.length === 0) {
return '0';
}
const length = str.length;
const lastChars = str.slice(-16);
return `${length}${TEXT_KEY_DIVIDER}${lastChars}${TEXT_KEY_DIVIDER}${index}`;
};
if (message.text) {
return formatText(message.text, -1);
}
if (!message.content || message.content.length === 0) {
return '';
}
for (let i = message.content.length - 1; i >= 0; i--) {
const part = message.content[i] as TMessageContentParts | undefined;
if (!part?.type) {
continue;
}
const type = part.type;
let text = '';
// Handle THINK type - extract think content
if (type === ContentTypes.THINK && 'think' in part) {
text = typeof part.think === 'string' ? part.think : (part.think?.value ?? '');
}
// Handle TEXT type
else if (type === ContentTypes.TEXT && 'text' in part) {
text = typeof part.text === 'string' ? part.text : (part.text?.value ?? '');
}
// Handle ERROR type
else if (type === ContentTypes.ERROR && 'error' in part) {
text = String(part.error || 'err').slice(0, 30);
}
// Handle TOOL_CALL - use simple marker with type
else if (type === ContentTypes.TOOL_CALL && 'tool_call' in part) {
const tcType = part.tool_call?.type || 'x';
const tcName = String(part.tool_call?.['name'] || 'unknown').slice(0, 20);
const tcArgs = String(part.tool_call?.['args'] || 'none').slice(0, 20);
const tcOutput = String(part.tool_call?.['output'] || 'none').slice(0, 20);
text = `tc_${tcType}_${tcName}_${tcArgs}_${tcOutput}`;
}
// Handle IMAGE_FILE - use simple marker with file_id suffix
else if (type === ContentTypes.IMAGE_FILE && 'image_file' in part) {
const fileId = part.image_file?.file_id || 'x';
text = `if_${fileId.slice(-8)}`;
}
// Handle IMAGE_URL - use simple marker
else if (type === ContentTypes.IMAGE_URL) {
text = 'iu';
}
// Handle AGENT_UPDATE - use simple marker with agentId suffix
else if (type === ContentTypes.AGENT_UPDATE && 'agent_update' in part) {
const agentId = String(part.agent_update?.agentId || 'x').slice(0, 30);
text = `au_${agentId}`;
} else {
text = type;
}
if (text.length > 0) {
return formatText(text, i);
}
}
return '';
};
export const getTextKey = (message?: TMessage | null, convoId?: string | null) => {
if (!message) {
return '';
}
⚡ refactor: Latest Message Tracking with Robust Text Key Generation (#10059) * chore: enhance logging for latest message actions in message components * fix: Extract previous convoId from latest text in message helpers and process hooks - Updated `useMessageHelpers` and `useMessageProcess` to extract `convoId` from the previous text key for improved message handling. - Refactored `getLengthAndLastTenChars` to `getLengthAndLastNChars` for better flexibility in character length retrieval. - Introduced `getLatestContentForKey` function to streamline content extraction from messages. * chore: Enhance logging for clearing latest messages in conversation hooks * refactor: Update message key formatting for improved URL parameter handling - Modified `getLatestContentForKey` to change the format from `${text}-${i}` to `${text}&i=${i}` for better URL parameter structure. - Adjusted `getTextKey` to increase character length retrieval from 12 to 16 in `getLengthAndLastNChars` for enhanced text processing. * refactor: Simplify convoId extraction and enhance message formatting - Updated `useMessageHelpers` and `useMessageProcess` to extract `convoId` using a new format for improved clarity. - Refactored `getLatestContentForKey` to streamline content formatting and ensure consistent use of `Constants.COMMON_DIVIDER` for better message structure. - Removed redundant length and last character extraction logic from `getLengthAndLastNChars` for cleaner code. * chore: linting * chore: Simplify pre-commit hook by removing unnecessary lines
2025-10-10 11:22:16 +03:00
const contentKey = getLatestContentForKey(message);
return `${(message.messageId as string | null) ?? ''}${TEXT_KEY_DIVIDER}${contentKey}${TEXT_KEY_DIVIDER}${message.conversationId ?? convoId}`;
};
export const scrollToEnd = (callback?: () => void) => {
const messagesEndElement = document.getElementById('messages-end');
if (messagesEndElement) {
messagesEndElement.scrollIntoView({ behavior: 'instant' });
if (callback) {
callback();
}
}
};
/**
* Clears messages for both the specified conversation ID and the NEW_CONVO query key.
* This ensures that messages are properly cleared in all contexts, preventing stale data
* from persisting in the NEW_CONVO cache.
*
* @param queryClient - The React Query client instance
* @param conversationId - The conversation ID to clear messages for
*/
export const clearMessagesCache = (
queryClient: QueryClient,
conversationId: string | undefined | null,
): void => {
const convoId = conversationId ?? Constants.NEW_CONVO;
// Clear messages for the current conversation
queryClient.setQueryData<TMessage[]>([QueryKeys.messages, convoId], []);
// Also clear NEW_CONVO messages if we're not already on NEW_CONVO
if (convoId !== Constants.NEW_CONVO) {
queryClient.setQueryData<TMessage[]>([QueryKeys.messages, Constants.NEW_CONVO], []);
}
};
export const getMessageAriaLabel = (message: TMessage, localize: LocalizeFunction): string => {
return !_.isNil(message.depth)
? localize('com_endpoint_message_new', { 0: message.depth + 1 })
: localize('com_endpoint_message');
};
⏸ refactor: Improve UX for Parallel Streams (Multi-Convo) (#11096) * 🌊 feat: Implement multi-conversation feature with added conversation context and payload adjustments * refactor: Replace isSubmittingFamily with isSubmitting across message components for consistency * feat: Add loadAddedAgent and processAddedConvo for multi-conversation agent execution * refactor: Update ContentRender usage to conditionally render PlaceholderRow based on isLast and isSubmitting * WIP: first pass, sibling index * feat: Enhance multi-conversation support with agent tracking and display improvements * refactor: Introduce isEphemeralAgentId utility and update related logic for agent handling * refactor: Implement createDualMessageContent utility for sibling message display and enhance useStepHandler for added conversations * refactor: duplicate tools for added agent if ephemeral and primary agent is also ephemeral * chore: remove deprecated multimessage rendering * refactor: enhance dual message content creation and agent handling for parallel rendering * refactor: streamline message rendering and submission handling by removing unused state and optimizing conditional logic * refactor: adjust content handling in parallel mode to utilize existing content for improved agent display * refactor: update @librechat/agents dependency to version 3.0.53 * refactor: update @langchain/core and @librechat/agents dependencies to latest versions * refactor: remove deprecated @langchain/core dependency from package.json * chore: remove unused SearchToolConfig and GetSourcesParams types from web.ts * refactor: remove unused message properties from Message component * refactor: enhance parallel content handling with groupId support in ContentParts and useStepHandler * refactor: implement parallel content styling in Message, MessageRender, and ContentRender components. use explicit model name * refactor: improve agent ID handling in createDualMessageContent for dual message display * refactor: simplify title generation in AddedConvo by removing unused sender and preset logic * refactor: replace string interpolation with cn utility for className in HoverButtons component * refactor: enhance agent ID handling by adding suffix management for parallel agents and updating related components * refactor: enhance column ordering in ContentParts by sorting agents with suffix management * refactor: update @librechat/agents dependency to version 3.0.55 * feat: implement parallel content rendering with metadata support - Added `ParallelContentRenderer` and `ParallelColumns` components for rendering messages in parallel based on groupId and agentId. - Introduced `contentMetadataMap` to store metadata for each content part, allowing efficient parallel content detection. - Updated `Message` and `ContentRender` components to utilize the new metadata structure for rendering. - Modified `useStepHandler` to manage content indices and metadata during message processing. - Enhanced `IJobStore` interface and its implementations to support storing and retrieving content metadata. - Updated data schemas to include `contentMetadataMap` for messages, enabling multi-agent and parallel execution scenarios. * refactor: update @librechat/agents dependency to version 3.0.56 * refactor: remove unused EPHEMERAL_AGENT_ID constant and simplify agent ID check * refactor: enhance multi-agent message processing and primary agent determination * refactor: implement branch message functionality for parallel responses * refactor: integrate added conversation retrieval into message editing and regeneration processes * refactor: remove unused isCard and isMultiMessage props from MessageRender and ContentRender components * refactor: update @librechat/agents dependency to version 3.0.60 * refactor: replace usage of EPHEMERAL_AGENT_ID constant with isEphemeralAgentId function for improved clarity and consistency * refactor: standardize agent ID format in tests for consistency * chore: move addedConvo property to the correct position in payload construction * refactor: rename agent_id values in loadAgent tests for clarity * chore: reorder props in ContentParts component for improved readability * refactor: rename variable 'content' to 'result' for clarity in RedisJobStore tests * refactor: streamline useMessageActions by removing duplicate handleFeedback assignment * chore: revert placeholder rendering logic MessageRender and ContentRender components to original * refactor: implement useContentMetadata hook for optimized content metadata handling * refactor: remove contentMetadataMap and related logic from the codebase and revert back to agentId/groupId in content parts - Eliminated contentMetadataMap from various components and services, simplifying the handling of message content. - Updated functions to directly access agentId and groupId from content parts instead of relying on a separate metadata map. - Adjusted related hooks and components to reflect the removal of contentMetadataMap, ensuring consistent handling of message content. - Updated tests and documentation to align with the new structure of message content handling. * refactor: remove logging from groupParallelContent function to clean up output * refactor: remove model parameter from TBranchMessageRequest type for simplification * refactor: enhance branch message creation by stripping metadata for standalone content * chore: streamline branch message creation by simplifying content filtering and removing unnecessary metadata checks * refactor: include attachments in branch message creation for improved content handling * refactor: streamline agent content processing by consolidating primary agent identification and filtering logic * refactor: simplify multi-agent message processing by creating a dedicated mapping method and enhancing content filtering * refactor: remove unused parameter from loadEphemeralAgent function for cleaner code * refactor: update groupId handling in metadata to only set when provided by the server
2025-12-25 01:43:54 -05:00
/**
* Creates initial content parts for dual message display with agent-based grouping.
* Sets up primary and added agent content parts with agentId for column rendering.
*
* @param primaryConvo - The primary conversation configuration
* @param addedConvo - The added conversation configuration
* @param endpointsConfig - Endpoints configuration for getting model display labels
* @returns Array of content parts with agentId for side-by-side rendering
*/
export const createDualMessageContent = (
primaryConvo: TConversation,
addedConvo: TConversation,
endpointsConfig?: TEndpointsConfig,
): TMessageContentParts[] => {
// For real agents (agent_id starts with "agent_"), use agent_id directly
// Otherwise create ephemeral ID from endpoint/model
let primaryAgentId: string;
if (primaryConvo.agent_id && !isEphemeralAgentId(primaryConvo.agent_id)) {
primaryAgentId = primaryConvo.agent_id;
} else {
const primaryEndpoint = primaryConvo.endpoint;
const primaryModel = primaryConvo.model ?? '';
const primarySender = getResponseSender({
modelDisplayLabel: primaryEndpoint
? endpointsConfig?.[primaryEndpoint]?.modelDisplayLabel
: undefined,
});
primaryAgentId = encodeEphemeralAgentId({
endpoint: primaryEndpoint ?? '',
model: primaryModel,
sender: primarySender,
});
}
// Both agents run in parallel, so they share the same groupId
const parallelGroupId = 1;
// Use empty type - these are just placeholders to establish agentId/groupId
// The actual type will be set when real content arrives from the server
const primaryContent = {
type: '' as const,
agentId: primaryAgentId,
groupId: parallelGroupId,
};
// For added agent, use agent_id if it's a real agent (starts with "agent_")
// Otherwise create ephemeral ID with index suffix
// Always append index suffix for added agent to distinguish from primary (even if same agent_id)
let addedAgentId: string;
if (addedConvo.agent_id && !isEphemeralAgentId(addedConvo.agent_id)) {
// Append suffix to distinguish from primary agent (matches ephemeral format)
addedAgentId = appendAgentIdSuffix(addedConvo.agent_id, 1);
} else {
const addedEndpoint = addedConvo.endpoint;
const addedModel = addedConvo.model ?? '';
const addedSender = addedEndpoint
? getResponseSender({
modelDisplayLabel: endpointsConfig?.[addedEndpoint]?.modelDisplayLabel,
})
: '';
addedAgentId = encodeEphemeralAgentId({
endpoint: addedEndpoint ?? '',
model: addedModel,
sender: addedSender,
index: 1,
});
}
// Use empty type - placeholder to establish agentId/groupId
const addedContent = {
type: '' as const,
agentId: addedAgentId,
groupId: parallelGroupId,
};
// Cast through unknown since these are placeholder objects with empty type
// that will be replaced by real content with proper types from the server
return [primaryContent, addedContent] as unknown as TMessageContentParts[];
};