⏸ refactor: Improve UX for Parallel Streams (Multi-Convo) (#11096)

* 🌊 feat: Implement multi-conversation feature with added conversation context and payload adjustments

* refactor: Replace isSubmittingFamily with isSubmitting across message components for consistency

* feat: Add loadAddedAgent and processAddedConvo for multi-conversation agent execution

* refactor: Update ContentRender usage to conditionally render PlaceholderRow based on isLast and isSubmitting

* WIP: first pass, sibling index

* feat: Enhance multi-conversation support with agent tracking and display improvements

* refactor: Introduce isEphemeralAgentId utility and update related logic for agent handling

* refactor: Implement createDualMessageContent utility for sibling message display and enhance useStepHandler for added conversations

* refactor: duplicate tools for added agent if ephemeral and primary agent is also ephemeral

* chore: remove deprecated multimessage rendering

* refactor: enhance dual message content creation and agent handling for parallel rendering

* refactor: streamline message rendering and submission handling by removing unused state and optimizing conditional logic

* refactor: adjust content handling in parallel mode to utilize existing content for improved agent display

* refactor: update @librechat/agents dependency to version 3.0.53

* refactor: update @langchain/core and @librechat/agents dependencies to latest versions

* refactor: remove deprecated @langchain/core dependency from package.json

* chore: remove unused SearchToolConfig and GetSourcesParams types from web.ts

* refactor: remove unused message properties from Message component

* refactor: enhance parallel content handling with groupId support in ContentParts and useStepHandler

* refactor: implement parallel content styling in Message, MessageRender, and ContentRender components. use explicit model name

* refactor: improve agent ID handling in createDualMessageContent for dual message display

* refactor: simplify title generation in AddedConvo by removing unused sender and preset logic

* refactor: replace string interpolation with cn utility for className in HoverButtons component

* refactor: enhance agent ID handling by adding suffix management for parallel agents and updating related components

* refactor: enhance column ordering in ContentParts by sorting agents with suffix management

* refactor: update @librechat/agents dependency to version 3.0.55

* feat: implement parallel content rendering with metadata support

- Added `ParallelContentRenderer` and `ParallelColumns` components for rendering messages in parallel based on groupId and agentId.
- Introduced `contentMetadataMap` to store metadata for each content part, allowing efficient parallel content detection.
- Updated `Message` and `ContentRender` components to utilize the new metadata structure for rendering.
- Modified `useStepHandler` to manage content indices and metadata during message processing.
- Enhanced `IJobStore` interface and its implementations to support storing and retrieving content metadata.
- Updated data schemas to include `contentMetadataMap` for messages, enabling multi-agent and parallel execution scenarios.

* refactor: update @librechat/agents dependency to version 3.0.56

* refactor: remove unused EPHEMERAL_AGENT_ID constant and simplify agent ID check

* refactor: enhance multi-agent message processing and primary agent determination

* refactor: implement branch message functionality for parallel responses

* refactor: integrate added conversation retrieval into message editing and regeneration processes

* refactor: remove unused isCard and isMultiMessage props from MessageRender and ContentRender components

* refactor: update @librechat/agents dependency to version 3.0.60

* refactor: replace usage of EPHEMERAL_AGENT_ID constant with isEphemeralAgentId function for improved clarity and consistency

* refactor: standardize agent ID format in tests for consistency

* chore: move addedConvo property to the correct position in payload construction

* refactor: rename agent_id values in loadAgent tests for clarity

* chore: reorder props in ContentParts component for improved readability

* refactor: rename variable 'content' to 'result' for clarity in RedisJobStore tests

* refactor: streamline useMessageActions by removing duplicate handleFeedback assignment

* chore: revert placeholder rendering logic MessageRender and ContentRender components to original

* refactor: implement useContentMetadata hook for optimized content metadata handling

* refactor: remove contentMetadataMap and related logic from the codebase and revert back to agentId/groupId in content parts

- Eliminated contentMetadataMap from various components and services, simplifying the handling of message content.
- Updated functions to directly access agentId and groupId from content parts instead of relying on a separate metadata map.
- Adjusted related hooks and components to reflect the removal of contentMetadataMap, ensuring consistent handling of message content.
- Updated tests and documentation to align with the new structure of message content handling.

* refactor: remove logging from groupParallelContent function to clean up output

* refactor: remove model parameter from TBranchMessageRequest type for simplification

* refactor: enhance branch message creation by stripping metadata for standalone content

* chore: streamline branch message creation by simplifying content filtering and removing unnecessary metadata checks

* refactor: include attachments in branch message creation for improved content handling

* refactor: streamline agent content processing by consolidating primary agent identification and filtering logic

* refactor: simplify multi-agent message processing by creating a dedicated mapping method and enhancing content filtering

* refactor: remove unused parameter from loadEphemeralAgent function for cleaner code

* refactor: update groupId handling in metadata to only set when provided by the server

api/server/services/Endpoints/agents/addedConvo.js

@@ -0,0 +1,136 @@
+const { logger } = require('@librechat/data-schemas');
+const { initializeAgent, validateAgentModel } = require('@librechat/api');
+const { loadAddedAgent, setGetAgent, ADDED_AGENT_ID } = require('~/models/loadAddedAgent');
+const { getConvoFiles } = require('~/models/Conversation');
+const { getAgent } = require('~/models/Agent');
+const db = require('~/models');
+
+// Initialize the getAgent dependency
+setGetAgent(getAgent);
+
+/**
+ * Process addedConvo for parallel agent execution.
+ * Creates a parallel agent config from an added conversation.
+ *
+ * When an added agent has no incoming edges, it becomes a start node
+ * and runs in parallel with the primary agent automatically.
+ *
+ * Edge cases handled:
+ * - Primary agent has edges (handoffs): Added agent runs in parallel with primary,
+ *   but doesn't participate in the primary's handoff graph
+ * - Primary agent has agent_ids (legacy chain): Added agent runs in parallel with primary,
+ *   but doesn't participate in the chain
+ * - Primary agent has both: Added agent is independent, runs parallel from start
+ *
+ * @param {Object} params
+ * @param {import('express').Request} params.req
+ * @param {import('express').Response} params.res
+ * @param {Object} params.endpointOption - The endpoint option containing addedConvo
+ * @param {Object} params.modelsConfig - The models configuration
+ * @param {Function} params.logViolation - Function to log violations
+ * @param {Function} params.loadTools - Function to load agent tools
+ * @param {Array} params.requestFiles - Request files
+ * @param {string} params.conversationId - The conversation ID
+ * @param {Set} params.allowedProviders - Set of allowed providers
+ * @param {Map} params.agentConfigs - Map of agent configs to add to
+ * @param {string} params.primaryAgentId - The primary agent ID
+ * @param {Object|undefined} params.userMCPAuthMap - User MCP auth map to merge into
+ * @returns {Promise<{userMCPAuthMap: Object|undefined}>} The updated userMCPAuthMap
+ */
+const processAddedConvo = async ({
+  req,
+  res,
+  endpointOption,
+  modelsConfig,
+  logViolation,
+  loadTools,
+  requestFiles,
+  conversationId,
+  allowedProviders,
+  agentConfigs,
+  primaryAgentId,
+  primaryAgent,
+  userMCPAuthMap,
+}) => {
+  const addedConvo = endpointOption.addedConvo;
+  logger.debug('[processAddedConvo] Called with addedConvo:', {
+    hasAddedConvo: addedConvo != null,
+    addedConvoEndpoint: addedConvo?.endpoint,
+    addedConvoModel: addedConvo?.model,
+    addedConvoAgentId: addedConvo?.agent_id,
+  });
+  if (addedConvo == null) {
+    return { userMCPAuthMap };
+  }
+
+  try {
+    const addedAgent = await loadAddedAgent({ req, conversation: addedConvo, primaryAgent });
+    if (!addedAgent) {
+      return { userMCPAuthMap };
+    }
+
+    const addedValidation = await validateAgentModel({
+      req,
+      res,
+      modelsConfig,
+      logViolation,
+      agent: addedAgent,
+    });
+
+    if (!addedValidation.isValid) {
+      logger.warn(
+        `[processAddedConvo] Added agent validation failed: ${addedValidation.error?.message}`,
+      );
+      return { userMCPAuthMap };
+    }
+
+    const addedConfig = await initializeAgent(
+      {
+        req,
+        res,
+        loadTools,
+        requestFiles,
+        conversationId,
+        agent: addedAgent,
+        endpointOption,
+        allowedProviders,
+      },
+      {
+        getConvoFiles,
+        getFiles: db.getFiles,
+        getUserKey: db.getUserKey,
+        updateFilesUsage: db.updateFilesUsage,
+        getUserKeyValues: db.getUserKeyValues,
+        getToolFilesByIds: db.getToolFilesByIds,
+      },
+    );
+
+    if (userMCPAuthMap != null) {
+      Object.assign(userMCPAuthMap, addedConfig.userMCPAuthMap ?? {});
+    } else {
+      userMCPAuthMap = addedConfig.userMCPAuthMap;
+    }
+
+    const addedAgentId = addedConfig.id || ADDED_AGENT_ID;
+    agentConfigs.set(addedAgentId, addedConfig);
+
+    // No edges needed - agent without incoming edges becomes a start node
+    // and runs in parallel with the primary agent automatically.
+    // This is independent of any edges/agent_ids the primary agent has.
+
+    logger.debug(
+      `[processAddedConvo] Added parallel agent: ${addedAgentId} (primary: ${primaryAgentId}, ` +
+        `primary has edges: ${!!endpointOption.edges}, primary has agent_ids: ${!!endpointOption.agent_ids})`,
+    );
+
+    return { userMCPAuthMap };
+  } catch (err) {
+    logger.error('[processAddedConvo] Error processing addedConvo for parallel agent', err);
+    return { userMCPAuthMap };
+  }
+};
+
+module.exports = {
+  processAddedConvo,
+  ADDED_AGENT_ID,
+};

api/server/services/Endpoints/agents/build.js

@@ -15,6 +15,9 @@ const buildOptions = (req, endpoint, parsedBody, endpointType) => {
     return undefined;
   });
 
+  /** @type {import('librechat-data-provider').TConversation | undefined} */
+  const addedConvo = req.body?.addedConvo;
+
   return removeNullishValues({
     spec,
     iconURL,
@@ -23,6 +26,7 @@ const buildOptions = (req, endpoint, parsedBody, endpointType) => {
     endpointType,
     model_parameters,
     agent: agentPromise,
+    addedConvo,
   });
 };
 

api/server/services/Endpoints/agents/initialize.js

@@ -7,10 +7,10 @@ const {
   createSequentialChainEdges,
 } = require('@librechat/api');
 const {
-  Constants,
   EModelEndpoint,
   isAgentsEndpoint,
   getResponseSender,
+  isEphemeralAgentId,
 } = require('librechat-data-provider');
 const {
   createToolEndCallback,
@@ -20,6 +20,7 @@ const { getModelsConfig } = require('~/server/controllers/ModelController');
 const { loadAgentTools } = require('~/server/services/ToolService');
 const AgentClient = require('~/server/controllers/agents/client');
 const { getConvoFiles } = require('~/models/Conversation');
+const { processAddedConvo } = require('./addedConvo');
 const { getAgent } = require('~/models/Agent');
 const { logViolation } = require('~/cache');
 const db = require('~/models');
@@ -233,6 +234,33 @@ const initializeClient = async ({ req, res, signal, endpointOption }) => {
     edges = edges ? edges.concat(chain) : chain;
   }
 
+  /** Multi-Convo: Process addedConvo for parallel agent execution */
+  const { userMCPAuthMap: updatedMCPAuthMap } = await processAddedConvo({
+    req,
+    res,
+    endpointOption,
+    modelsConfig,
+    logViolation,
+    loadTools,
+    requestFiles,
+    conversationId,
+    allowedProviders,
+    agentConfigs,
+    primaryAgentId: primaryConfig.id,
+    primaryAgent,
+    userMCPAuthMap,
+  });
+
+  if (updatedMCPAuthMap) {
+    userMCPAuthMap = updatedMCPAuthMap;
+  }
+
+  // Ensure edges is an array when we have multiple agents (multi-agent mode)
+  // MultiAgentGraph.categorizeEdges requires edges to be iterable
+  if (agentConfigs.size > 0 && !edges) {
+    edges = [];
+  }
+
   primaryConfig.edges = edges;
 
   let endpointConfig = appConfig.endpoints?.[primaryConfig.endpoint];
@@ -276,10 +304,7 @@ const initializeClient = async ({ req, res, signal, endpointOption }) => {
     endpointType: endpointOption.endpointType,
     resendFiles: primaryConfig.resendFiles ?? true,
     maxContextTokens: primaryConfig.maxContextTokens,
-    endpoint:
-      primaryConfig.id === Constants.EPHEMERAL_AGENT_ID
-        ? primaryConfig.endpoint
-        : EModelEndpoint.agents,
+    endpoint: isEphemeralAgentId(primaryConfig.id) ? primaryConfig.endpoint : EModelEndpoint.agents,
   });
 
   return { client, userMCPAuthMap };

api/server/services/ToolService.js

@@ -9,7 +9,6 @@ const {
 } = require('@librechat/api');
 const {
   Tools,
-  Constants,
   ErrorTypes,
   ContentTypes,
   imageGenTools,
@@ -18,6 +17,7 @@ const {
   ImageVisionTool,
   openapiToFunction,
   AgentCapabilities,
+  isEphemeralAgentId,
   validateActionDomain,
   defaultAgentCapabilities,
   validateAndParseOpenAPISpec,
@@ -393,7 +393,7 @@ async function loadAgentTools({
   const endpointsConfig = await getEndpointsConfig(req);
   let enabledCapabilities = new Set(endpointsConfig?.[EModelEndpoint.agents]?.capabilities ?? []);
   /** Edge case: use defined/fallback capabilities when the "agents" endpoint is not enabled */
-  if (enabledCapabilities.size === 0 && agent.id === Constants.EPHEMERAL_AGENT_ID) {
+  if (enabledCapabilities.size === 0 && isEphemeralAgentId(agent.id)) {
     enabledCapabilities = new Set(
       appConfig.endpoints?.[EModelEndpoint.agents]?.capabilities ?? defaultAgentCapabilities,
     );