🎉 feat: Code Interpreter API and Agents Release (#4860)

* feat: Code Interpreter API & File Search Agent Uploads

chore: add back code files

wip: first pass, abstract key dialog

refactor: influence checkbox on key changes

refactor: update localization keys for 'execute code' to 'run code'

wip: run code button

refactor: add throwError parameter to loadAuthValues and getUserPluginAuthValue functions

feat: first pass, API tool calling

fix: handle missing toolId in callTool function and return 404 for non-existent tools

feat: show code outputs

fix: improve error handling in callTool function and log errors

fix: handle potential null value for filepath in attachment destructuring

fix: normalize language before rendering and prevent null return

fix: add loading indicator in RunCode component while executing code

feat: add support for conditional code execution in Markdown components

feat: attachments

refactor: remove bash

fix: pass abort signal to graph/run

refactor: debounce and rate limit tool call

refactor: increase debounce delay for execute function

feat: set code output attachments

feat: image attachments

refactor: apply message context

refactor: pass `partIndex`

feat: toolCall schema/model/methods

feat: block indexing

feat: get tool calls

chore: imports

chore: typing

chore: condense type imports

feat: get tool calls

fix: block indexing

chore: typing

refactor: update tool calls mapping to support multiple results

fix: add unique key to nav link for rendering

wip: first pass, tool call results

refactor: update query cache from successful tool call mutation

style: improve result switcher styling

chore: note on using \`.toObject()\`

feat: add agent_id field to conversation schema

chore: typing

refactor: rename agentMap to agentsMap for consistency

feat: Agent Name as chat input placeholder

chore: bump agents

📦 chore: update @langchain dependencies to latest versions to match agents package

📦 chore: update @librechat/agents dependency to version 1.8.0

fix: Aborting agent stream removes sender; fix(bedrock): completion removes preset name label

refactor: remove direct file parameter to use req.file, add `processAgentFileUpload` for image uploads

feat: upload menu

feat: prime message_file resources

feat: implement conversation access validation in chat route

refactor: remove file parameter from processFileUpload and use req.file instead

feat: add savedMessageIds set to track saved message IDs in BaseClient, to prevent unnecessary double-write to db

feat: prevent duplicate message saves by checking savedMessageIds in AgentController

refactor: skip legacy RAG API handling for agents

feat: add files field to convoSchema

refactor: update request type annotations from Express.Request to ServerRequest in file processing functions

feat: track conversation files

fix: resendFiles, addPreviousAttachments handling

feat: add ID validation for session_id and file_id in download route

feat: entity_id for code file uploads/downloads

fix: code file edge cases

feat: delete related tool calls

feat: add stream rate handling for LLM configuration

feat: enhance system content with attached file information

fix: improve error logging in resource priming function

* WIP: PoC, sequential agents

WIP: PoC Sequential Agents, first pass content data + bump agents package

fix: package-lock

WIP: PoC, o1 support, refactor bufferString

feat: convertJsonSchemaToZod

fix: form issues and schema defining erroneous model

fix: max length issue on agent form instructions, limit conversation messages to sequential agents

feat: add abort signal support to createRun function and AgentClient

feat: PoC, hide prior sequential agent steps

fix: update parameter naming from config to metadata in event handlers for clarity, add model to usage data

refactor: use only last contentData, track model for usage data

chore: bump agents package

fix: content parts issue

refactor: filter contentParts to include tool calls and relevant indices

feat: show function calls

refactor: filter context messages to exclude tool calls when no tools are available to the agent

fix: ensure tool call content is not undefined in formatMessages

feat: add agent_id field to conversationPreset schema

feat: hide sequential agents

feat: increase upload toast duration to 10 seconds

* refactor: tool context handling & update Code API Key Dialog

feat: toolContextMap

chore: skipSpecs -> useSpecs

ci: fix handleTools tests

feat: API Key Dialog

* feat: Agent Permissions Admin Controls

feat: replace label with button for prompt permission toggle

feat: update agent permissions

feat: enable experimental agents and streamline capability configuration

feat: implement access control for agents and enhance endpoint menu items

feat: add welcome message for agent selection in localization

feat: add agents permission to access control and update version to 0.7.57

* fix: update types in useAssistantListMap and useMentions hooks for better null handling

* feat: mention agents

* fix: agent tool resource race conditions when deleting agent tool resource files

* feat: add error handling for code execution with user feedback

* refactor: rename AdminControls to AdminSettings for clarity

* style: add gap to button in AdminSettings for improved layout

* refactor: separate agent query hooks and check access to enable fetching

* fix: remove unused provider from agent initialization options, creates issue with custom endpoints

* refactor: remove redundant/deprecated modelOptions from AgentClient processes

* chore: update @librechat/agents to version 1.8.5 in package.json and package-lock.json

* fix: minor styling issues + agent panel uniformity

* fix: agent edge cases when set endpoint is no longer defined

* refactor: remove unused cleanup function call from AppService

* fix: update link in ApiKeyDialog to point to pricing page

* fix: improve type handling and layout calculations in SidePanel component

* fix: add missing localization string for agent selection in SidePanel

* chore: form styling and localizations for upload filesearch/code interpreter

* fix: model selection placeholder logic in AgentConfig component

* style: agent capabilities

* fix: add localization for provider selection and improve dropdown styling in ModelPanel

* refactor: use gpt-4o-mini > gpt-3.5-turbo

* fix: agents configuration for loadDefaultInterface and update related tests

* feat: DALLE Agents support

client/src/components/Chat/Messages/Content/Part.tsx

@@ -21,143 +21,130 @@ type PartProps = {
   part?: TMessageContentParts;
   isSubmitting: boolean;
   showCursor: boolean;
-  messageId: string;
   isCreatedByUser: boolean;
   attachments?: TAttachment[];
 };
 
-const Part = memo(
-  ({ part, isSubmitting, attachments, showCursor, messageId, isCreatedByUser }: PartProps) => {
-    attachments && console.log(attachments);
-    if (!part) {
+const Part = memo(({ part, isSubmitting, attachments, showCursor, isCreatedByUser }: PartProps) => {
+  if (!part) {
+    return null;
+  }
+
+  if (part.type === ContentTypes.ERROR) {
+    return <ErrorMessage text={part[ContentTypes.TEXT].value} className="my-2" />;
+  } else if (part.type === ContentTypes.TEXT) {
+    const text = typeof part.text === 'string' ? part.text : part.text.value;
+
+    if (typeof text !== 'string') {
+      return null;
+    }
+    if (part.tool_call_ids != null && !text) {
+      return null;
+    }
+    return (
+      <Container>
+        <Text text={text} isCreatedByUser={isCreatedByUser} showCursor={showCursor} />
+      </Container>
+    );
+  } else if (part.type === ContentTypes.TOOL_CALL) {
+    const toolCall = part[ContentTypes.TOOL_CALL];
+
+    if (!toolCall) {
       return null;
     }
 
-    if (part.type === ContentTypes.ERROR) {
-      return <ErrorMessage text={part[ContentTypes.TEXT].value} className="my-2" />;
-    } else if (part.type === ContentTypes.TEXT) {
-      const text = typeof part.text === 'string' ? part.text : part.text.value;
-
-      if (typeof text !== 'string') {
-        return null;
-      }
-      if (part.tool_call_ids != null && !text) {
-        return null;
-      }
+    const isToolCall =
+      'args' in toolCall && (!toolCall.type || toolCall.type === ToolCallTypes.TOOL_CALL);
+    if (isToolCall && toolCall.name === Tools.execute_code) {
       return (
-        <Container>
-          <Text
-            text={text}
-            isCreatedByUser={isCreatedByUser}
-            messageId={messageId}
-            showCursor={showCursor}
-          />
-        </Container>
+        <ExecuteCode
+          args={typeof toolCall.args === 'string' ? toolCall.args : ''}
+          output={toolCall.output ?? ''}
+          initialProgress={toolCall.progress ?? 0.1}
+          isSubmitting={isSubmitting}
+          attachments={attachments}
+        />
       );
-    } else if (part.type === ContentTypes.TOOL_CALL) {
-      const toolCall = part[ContentTypes.TOOL_CALL];
-
-      if (!toolCall) {
+    } else if (isToolCall) {
+      return (
+        <ToolCall
+          args={toolCall.args ?? ''}
+          name={toolCall.name || ''}
+          output={toolCall.output ?? ''}
+          initialProgress={toolCall.progress ?? 0.1}
+          isSubmitting={isSubmitting}
+          attachments={attachments}
+        />
+      );
+    } else if (toolCall.type === ToolCallTypes.CODE_INTERPRETER) {
+      const code_interpreter = toolCall[ToolCallTypes.CODE_INTERPRETER];
+      return (
+        <CodeAnalyze
+          initialProgress={toolCall.progress ?? 0.1}
+          code={code_interpreter.input}
+          outputs={code_interpreter.outputs ?? []}
+          isSubmitting={isSubmitting}
+        />
+      );
+    } else if (
+      toolCall.type === ToolCallTypes.RETRIEVAL ||
+      toolCall.type === ToolCallTypes.FILE_SEARCH
+    ) {
+      return (
+        <RetrievalCall initialProgress={toolCall.progress ?? 0.1} isSubmitting={isSubmitting} />
+      );
+    } else if (
+      toolCall.type === ToolCallTypes.FUNCTION &&
+      ToolCallTypes.FUNCTION in toolCall &&
+      imageGenTools.has(toolCall.function.name)
+    ) {
+      return (
+        <ImageGen
+          initialProgress={toolCall.progress ?? 0.1}
+          args={toolCall.function.arguments as string}
+        />
+      );
+    } else if (toolCall.type === ToolCallTypes.FUNCTION && ToolCallTypes.FUNCTION in toolCall) {
+      if (isImageVisionTool(toolCall)) {
+        if (isSubmitting && showCursor) {
+          return (
+            <Container>
+              <Text text={''} isCreatedByUser={isCreatedByUser} showCursor={showCursor} />
+            </Container>
+          );
+        }
         return null;
       }
 
-      const isToolCall =
-        'args' in toolCall && (!toolCall.type || toolCall.type === ToolCallTypes.TOOL_CALL);
-      if (isToolCall && toolCall.name === Tools.execute_code) {
-        return (
-          <ExecuteCode
-            args={typeof toolCall.args === 'string' ? toolCall.args : ''}
-            output={toolCall.output ?? ''}
-            initialProgress={toolCall.progress ?? 0.1}
-            isSubmitting={isSubmitting}
-            attachments={attachments}
-          />
-        );
-      } else if (isToolCall) {
-        return (
-          <ToolCall
-            args={toolCall.args ?? ''}
-            name={toolCall.name ?? ''}
-            output={toolCall.output ?? ''}
-            initialProgress={toolCall.progress ?? 0.1}
-            isSubmitting={isSubmitting}
-          />
-        );
-      } else if (toolCall.type === ToolCallTypes.CODE_INTERPRETER) {
-        const code_interpreter = toolCall[ToolCallTypes.CODE_INTERPRETER];
-        return (
-          <CodeAnalyze
-            initialProgress={toolCall.progress ?? 0.1}
-            code={code_interpreter.input}
-            outputs={code_interpreter.outputs ?? []}
-            isSubmitting={isSubmitting}
-          />
-        );
-      } else if (
-        toolCall.type === ToolCallTypes.RETRIEVAL ||
-        toolCall.type === ToolCallTypes.FILE_SEARCH
-      ) {
-        return (
-          <RetrievalCall initialProgress={toolCall.progress ?? 0.1} isSubmitting={isSubmitting} />
-        );
-      } else if (
-        toolCall.type === ToolCallTypes.FUNCTION &&
-        ToolCallTypes.FUNCTION in toolCall &&
-        imageGenTools.has(toolCall.function.name)
-      ) {
-        return (
-          <ImageGen
-            initialProgress={toolCall.progress ?? 0.1}
-            args={toolCall.function.arguments as string}
-          />
-        );
-      } else if (toolCall.type === ToolCallTypes.FUNCTION && ToolCallTypes.FUNCTION in toolCall) {
-        if (isImageVisionTool(toolCall)) {
-          if (isSubmitting && showCursor) {
-            return (
-              <Container>
-                <Text
-                  text={''}
-                  isCreatedByUser={isCreatedByUser}
-                  messageId={messageId}
-                  showCursor={showCursor}
-                />
-              </Container>
-            );
-          }
-          return null;
-        }
-
-        return (
-          <ToolCall
-            initialProgress={toolCall.progress ?? 0.1}
-            isSubmitting={isSubmitting}
-            args={toolCall.function.arguments as string}
-            name={toolCall.function.name}
-            output={toolCall.function.output}
-          />
-        );
-      }
-    } else if (part.type === ContentTypes.IMAGE_FILE) {
-      const imageFile = part[ContentTypes.IMAGE_FILE];
-      const height = imageFile.height ?? 1920;
-      const width = imageFile.width ?? 1080;
       return (
-        <Image
-          imagePath={imageFile.filepath}
-          height={height}
-          width={width}
-          altText={imageFile.filename ?? 'Uploaded Image'}
-          placeholderDimensions={{
-            height: height + 'px',
-            width: width + 'px',
-          }}
+        <ToolCall
+          initialProgress={toolCall.progress ?? 0.1}
+          isSubmitting={isSubmitting}
+          args={toolCall.function.arguments as string}
+          name={toolCall.function.name}
+          output={toolCall.function.output}
         />
       );
     }
+  } else if (part.type === ContentTypes.IMAGE_FILE) {
+    const imageFile = part[ContentTypes.IMAGE_FILE];
+    const height = imageFile.height ?? 1920;
+    const width = imageFile.width ?? 1080;
+    return (
+      <Image
+        imagePath={imageFile.filepath}
+        height={height}
+        width={width}
+        altText={imageFile.filename ?? 'Uploaded Image'}
+        placeholderDimensions={{
+          height: height + 'px',
+          width: width + 'px',
+        }}
+      />
+    );
+  }
 
-    return null;
-  },
-);
+  return null;
+});
 
 export default Part;