🎉 feat: Code Interpreter API and Agents Release (#4860)

* feat: Code Interpreter API & File Search Agent Uploads

chore: add back code files

wip: first pass, abstract key dialog

refactor: influence checkbox on key changes

refactor: update localization keys for 'execute code' to 'run code'

wip: run code button

refactor: add throwError parameter to loadAuthValues and getUserPluginAuthValue functions

feat: first pass, API tool calling

fix: handle missing toolId in callTool function and return 404 for non-existent tools

feat: show code outputs

fix: improve error handling in callTool function and log errors

fix: handle potential null value for filepath in attachment destructuring

fix: normalize language before rendering and prevent null return

fix: add loading indicator in RunCode component while executing code

feat: add support for conditional code execution in Markdown components

feat: attachments

refactor: remove bash

fix: pass abort signal to graph/run

refactor: debounce and rate limit tool call

refactor: increase debounce delay for execute function

feat: set code output attachments

feat: image attachments

refactor: apply message context

refactor: pass `partIndex`

feat: toolCall schema/model/methods

feat: block indexing

feat: get tool calls

chore: imports

chore: typing

chore: condense type imports

feat: get tool calls

fix: block indexing

chore: typing

refactor: update tool calls mapping to support multiple results

fix: add unique key to nav link for rendering

wip: first pass, tool call results

refactor: update query cache from successful tool call mutation

style: improve result switcher styling

chore: note on using \`.toObject()\`

feat: add agent_id field to conversation schema

chore: typing

refactor: rename agentMap to agentsMap for consistency

feat: Agent Name as chat input placeholder

chore: bump agents

📦 chore: update @langchain dependencies to latest versions to match agents package

📦 chore: update @librechat/agents dependency to version 1.8.0

fix: Aborting agent stream removes sender; fix(bedrock): completion removes preset name label

refactor: remove direct file parameter to use req.file, add `processAgentFileUpload` for image uploads

feat: upload menu

feat: prime message_file resources

feat: implement conversation access validation in chat route

refactor: remove file parameter from processFileUpload and use req.file instead

feat: add savedMessageIds set to track saved message IDs in BaseClient, to prevent unnecessary double-write to db

feat: prevent duplicate message saves by checking savedMessageIds in AgentController

refactor: skip legacy RAG API handling for agents

feat: add files field to convoSchema

refactor: update request type annotations from Express.Request to ServerRequest in file processing functions

feat: track conversation files

fix: resendFiles, addPreviousAttachments handling

feat: add ID validation for session_id and file_id in download route

feat: entity_id for code file uploads/downloads

fix: code file edge cases

feat: delete related tool calls

feat: add stream rate handling for LLM configuration

feat: enhance system content with attached file information

fix: improve error logging in resource priming function

* WIP: PoC, sequential agents

WIP: PoC Sequential Agents, first pass content data + bump agents package

fix: package-lock

WIP: PoC, o1 support, refactor bufferString

feat: convertJsonSchemaToZod

fix: form issues and schema defining erroneous model

fix: max length issue on agent form instructions, limit conversation messages to sequential agents

feat: add abort signal support to createRun function and AgentClient

feat: PoC, hide prior sequential agent steps

fix: update parameter naming from config to metadata in event handlers for clarity, add model to usage data

refactor: use only last contentData, track model for usage data

chore: bump agents package

fix: content parts issue

refactor: filter contentParts to include tool calls and relevant indices

feat: show function calls

refactor: filter context messages to exclude tool calls when no tools are available to the agent

fix: ensure tool call content is not undefined in formatMessages

feat: add agent_id field to conversationPreset schema

feat: hide sequential agents

feat: increase upload toast duration to 10 seconds

* refactor: tool context handling & update Code API Key Dialog

feat: toolContextMap

chore: skipSpecs -> useSpecs

ci: fix handleTools tests

feat: API Key Dialog

* feat: Agent Permissions Admin Controls

feat: replace label with button for prompt permission toggle

feat: update agent permissions

feat: enable experimental agents and streamline capability configuration

feat: implement access control for agents and enhance endpoint menu items

feat: add welcome message for agent selection in localization

feat: add agents permission to access control and update version to 0.7.57

* fix: update types in useAssistantListMap and useMentions hooks for better null handling

* feat: mention agents

* fix: agent tool resource race conditions when deleting agent tool resource files

* feat: add error handling for code execution with user feedback

* refactor: rename AdminControls to AdminSettings for clarity

* style: add gap to button in AdminSettings for improved layout

* refactor: separate agent query hooks and check access to enable fetching

* fix: remove unused provider from agent initialization options, creates issue with custom endpoints

* refactor: remove redundant/deprecated modelOptions from AgentClient processes

* chore: update @librechat/agents to version 1.8.5 in package.json and package-lock.json

* fix: minor styling issues + agent panel uniformity

* fix: agent edge cases when set endpoint is no longer defined

* refactor: remove unused cleanup function call from AppService

* fix: update link in ApiKeyDialog to point to pricing page

* fix: improve type handling and layout calculations in SidePanel component

* fix: add missing localization string for agent selection in SidePanel

* chore: form styling and localizations for upload filesearch/code interpreter

* fix: model selection placeholder logic in AgentConfig component

* style: agent capabilities

* fix: add localization for provider selection and improve dropdown styling in ModelPanel

* refactor: use gpt-4o-mini > gpt-3.5-turbo

* fix: agents configuration for loadDefaultInterface and update related tests

* feat: DALLE Agents support

client/src/components/Messages/Content/CodeBlock.tsx

@@ -1,81 +1,133 @@
 import copy from 'copy-to-clipboard';
 import { InfoIcon } from 'lucide-react';
-import React, { useRef, useState, RefObject } from 'react';
+import { Tools } from 'librechat-data-provider';
+import React, { useRef, useState, useMemo, useEffect } from 'react';
+import type { CodeBarProps } from '~/common';
+import LogContent from '~/components/Chat/Messages/Content/Parts/LogContent';
+import ResultSwitcher from '~/components/Messages/Content/ResultSwitcher';
+import { useToolCallsMapContext, useMessageContext } from '~/Providers';
+import RunCode from '~/components/Messages/Content/RunCode';
 import Clipboard from '~/components/svg/Clipboard';
 import CheckMark from '~/components/svg/CheckMark';
 import useLocalize from '~/hooks/useLocalize';
 import cn from '~/utils/cn';
 
-type CodeBarProps = {
-  lang: string;
-  codeRef: RefObject<HTMLElement>;
-  plugin?: boolean;
-  error?: boolean;
-};
-
-type CodeBlockProps = Pick<CodeBarProps, 'lang' | 'plugin' | 'error'> & {
+type CodeBlockProps = Pick<
+  CodeBarProps,
+  'lang' | 'plugin' | 'error' | 'allowExecution' | 'blockIndex'
+> & {
   codeChildren: React.ReactNode;
   classProp?: string;
 };
 
-const CodeBar: React.FC<CodeBarProps> = React.memo(({ lang, codeRef, error, plugin = null }) => {
-  const localize = useLocalize();
-  const [isCopied, setIsCopied] = useState(false);
-  return (
-    <div className="relative flex items-center rounded-tl-md rounded-tr-md bg-gray-700 px-4 py-2 font-sans text-xs text-gray-200 dark:bg-gray-700">
-      <span className="">{lang}</span>
-      {plugin === true ? (
-        <InfoIcon className="ml-auto flex h-4 w-4 gap-2 text-white/50" />
-      ) : (
-        <button
-          type="button"
-          className={cn(
-            'ml-auto flex gap-2',
-            error === true ? 'h-4 w-4 items-start text-white/50' : '',
-          )}
-          onClick={async () => {
-            const codeString = codeRef.current?.textContent;
-            if (codeString != null) {
-              setIsCopied(true);
-              copy(codeString.trim(), { format: 'text/plain' });
+const CodeBar: React.FC<CodeBarProps> = React.memo(
+  ({ lang, error, codeRef, blockIndex, plugin = null, allowExecution = true }) => {
+    const localize = useLocalize();
+    const [isCopied, setIsCopied] = useState(false);
+    return (
+      <div className="relative flex items-center justify-between rounded-tl-md rounded-tr-md bg-gray-700 px-4 py-2 font-sans text-xs text-gray-200 dark:bg-gray-700">
+        <span className="">{lang}</span>
+        {plugin === true ? (
+          <InfoIcon className="ml-auto flex h-4 w-4 gap-2 text-white/50" />
+        ) : (
+          <div className="flex items-center justify-center gap-4">
+            {allowExecution === true && (
+              <RunCode lang={lang} codeRef={codeRef} blockIndex={blockIndex} />
+            )}
+            <button
+              type="button"
+              className={cn(
+                'ml-auto flex gap-2',
+                error === true ? 'h-4 w-4 items-start text-white/50' : '',
+              )}
+              onClick={async () => {
+                const codeString = codeRef.current?.textContent;
+                if (codeString != null) {
+                  setIsCopied(true);
+                  copy(codeString.trim(), { format: 'text/plain' });
 
-              setTimeout(() => {
-                setIsCopied(false);
-              }, 3000);
-            }
-          }}
-        >
-          {isCopied ? (
-            <>
-              <CheckMark className="h-[18px] w-[18px]" />
-              {error === true ? '' : localize('com_ui_copied')}
-            </>
-          ) : (
-            <>
-              <Clipboard />
-              {error === true ? '' : localize('com_ui_copy_code')}
-            </>
-          )}
-        </button>
-      )}
-    </div>
-  );
-});
+                  setTimeout(() => {
+                    setIsCopied(false);
+                  }, 3000);
+                }
+              }}
+            >
+              {isCopied ? (
+                <>
+                  <CheckMark className="h-[18px] w-[18px]" />
+                  {error === true ? '' : localize('com_ui_copied')}
+                </>
+              ) : (
+                <>
+                  <Clipboard />
+                  {error === true ? '' : localize('com_ui_copy_code')}
+                </>
+              )}
+            </button>
+          </div>
+        )}
+      </div>
+    );
+  },
+);
 
 const CodeBlock: React.FC<CodeBlockProps> = ({
   lang,
+  blockIndex,
   codeChildren,
   classProp = '',
+  allowExecution = true,
   plugin = null,
   error,
 }) => {
   const codeRef = useRef<HTMLElement>(null);
+  const toolCallsMap = useToolCallsMapContext();
+  const { messageId, partIndex } = useMessageContext();
+  const key = allowExecution
+    ? `${messageId}_${partIndex ?? 0}_${blockIndex ?? 0}_${Tools.execute_code}`
+    : '';
+  const [currentIndex, setCurrentIndex] = useState(0);
+
+  const fetchedToolCalls = toolCallsMap?.[key];
+  const [toolCalls, setToolCalls] = useState(toolCallsMap?.[key] ?? null);
+
+  useEffect(() => {
+    if (fetchedToolCalls) {
+      setToolCalls(fetchedToolCalls);
+      setCurrentIndex(fetchedToolCalls.length - 1);
+    }
+  }, [fetchedToolCalls]);
+
+  const currentToolCall = useMemo(() => toolCalls?.[currentIndex], [toolCalls, currentIndex]);
+
+  const next = () => {
+    if (!toolCalls) {
+      return;
+    }
+    if (currentIndex < toolCalls.length - 1) {
+      setCurrentIndex(currentIndex + 1);
+    }
+  };
+
+  const previous = () => {
+    if (currentIndex > 0) {
+      setCurrentIndex(currentIndex - 1);
+    }
+  };
+
   const isNonCode = !!(plugin === true || error === true);
   const language = isNonCode ? 'json' : lang;
 
   return (
     <div className="w-full rounded-md bg-gray-900 text-xs text-white/80">
-      <CodeBar lang={lang} codeRef={codeRef} plugin={plugin === true} error={error} />
+      <CodeBar
+        lang={lang}
+        error={error}
+        codeRef={codeRef}
+        blockIndex={blockIndex}
+        plugin={plugin === true}
+        allowExecution={allowExecution}
+      />
       <div className={cn(classProp, 'overflow-y-auto p-4')}>
         <code
           ref={codeRef}
@@ -86,6 +138,34 @@ const CodeBlock: React.FC<CodeBlockProps> = ({
           {codeChildren}
         </code>
       </div>
+      {allowExecution === true && toolCalls && toolCalls.length > 0 && (
+        <>
+          <div className="bg-gray-700 p-4 text-xs">
+            <div
+              className="prose flex flex-col-reverse text-white"
+              style={{
+                color: 'white',
+              }}
+            >
+              <pre className="shrink-0">
+                <LogContent
+                  output={(currentToolCall?.result as string | undefined) ?? ''}
+                  attachments={currentToolCall?.attachments ?? []}
+                  renderImages={true}
+                />
+              </pre>
+            </div>
+          </div>
+          {toolCalls.length > 1 && (
+            <ResultSwitcher
+              currentIndex={currentIndex}
+              totalCount={toolCalls.length}
+              onPrevious={previous}
+              onNext={next}
+            />
+          )}
+        </>
+      )}
     </div>
   );
 };

client/src/components/Messages/Content/ResultSwitcher.tsx

@@ -0,0 +1,69 @@
+interface ResultSwitcherProps {
+  currentIndex: number;
+  totalCount: number;
+  onPrevious: () => void;
+  onNext: () => void;
+}
+
+const ResultSwitcher: React.FC<ResultSwitcherProps> = ({
+  currentIndex,
+  totalCount,
+  onPrevious,
+  onNext,
+}) => {
+  if (totalCount <= 1) {
+    return null;
+  }
+
+  return (
+    <div className="flex items-center justify-start gap-1 self-center bg-gray-700 pb-2 text-xs">
+      <button
+        className="hover-button rounded-md p-1 text-gray-400 hover:bg-gray-700 hover:text-gray-200 disabled:hover:text-gray-400"
+        type="button"
+        onClick={onPrevious}
+        disabled={currentIndex === 0}
+      >
+        <svg
+          stroke="currentColor"
+          fill="none"
+          strokeWidth="1.5"
+          viewBox="0 0 24 24"
+          strokeLinecap="round"
+          strokeLinejoin="round"
+          className="h-4 w-4"
+          height="1em"
+          width="1em"
+          xmlns="http://www.w3.org/2000/svg"
+        >
+          <polyline points="15 18 9 12 15 6" />
+        </svg>
+      </button>
+      <span className="flex-shrink-0 tabular-nums">
+        {currentIndex + 1} / {totalCount}
+      </span>
+      <button
+        className="hover-button rounded-md p-1 text-gray-400 hover:bg-gray-700 hover:text-gray-200 disabled:hover:text-gray-400"
+        type="button"
+        onClick={onNext}
+        disabled={currentIndex === totalCount - 1}
+      >
+        <svg
+          stroke="currentColor"
+          fill="none"
+          strokeWidth="1.5"
+          viewBox="0 0 24 24"
+          strokeLinecap="round"
+          strokeLinejoin="round"
+          className="h-4 w-4"
+          height="1em"
+          width="1em"
+          xmlns="http://www.w3.org/2000/svg"
+        >
+          <polyline points="9 18 15 12 9 6" />
+        </svg>
+      </button>
+    </div>
+  );
+};
+
+export default ResultSwitcher;

client/src/components/Messages/Content/RunCode.tsx

@@ -0,0 +1,109 @@
+import debounce from 'lodash/debounce';
+import { Tools, AuthType } from 'librechat-data-provider';
+import { TerminalSquareIcon, Loader } from 'lucide-react';
+import React, { useMemo, useCallback, useEffect } from 'react';
+import type { CodeBarProps } from '~/common';
+import { useVerifyAgentToolAuth, useToolCallMutation } from '~/data-provider';
+import ApiKeyDialog from '~/components/SidePanel/Agents/Code/ApiKeyDialog';
+import { useLocalize, useCodeApiKeyForm } from '~/hooks';
+import { useMessageContext } from '~/Providers';
+import { cn, normalizeLanguage } from '~/utils';
+import { useToastContext } from '~/Providers';
+
+const RunCode: React.FC<CodeBarProps> = React.memo(({ lang, codeRef, blockIndex }) => {
+  const localize = useLocalize();
+  const { showToast } = useToastContext();
+  const execute = useToolCallMutation(Tools.execute_code, {
+    onError: () => {
+      showToast({ message: localize('com_ui_run_code_error'), status: 'error' });
+    },
+  });
+
+  const { messageId, conversationId, partIndex } = useMessageContext();
+  const normalizedLang = useMemo(() => normalizeLanguage(lang), [lang]);
+  const { data } = useVerifyAgentToolAuth({ toolId: Tools.execute_code });
+  const authType = useMemo(() => data?.message ?? false, [data?.message]);
+  const isAuthenticated = useMemo(() => data?.authenticated ?? false, [data?.authenticated]);
+  const { methods, onSubmit, isDialogOpen, setIsDialogOpen, handleRevokeApiKey } =
+    useCodeApiKeyForm({});
+
+  const handleExecute = useCallback(async () => {
+    if (!isAuthenticated) {
+      setIsDialogOpen(true);
+      return;
+    }
+    const codeString: string = codeRef.current?.textContent ?? '';
+    if (
+      typeof codeString !== 'string' ||
+      codeString.length === 0 ||
+      typeof normalizedLang !== 'string' ||
+      normalizedLang.length === 0
+    ) {
+      return;
+    }
+
+    execute.mutate({
+      partIndex,
+      messageId,
+      blockIndex,
+      conversationId: conversationId ?? '',
+      lang: normalizedLang,
+      code: codeString,
+    });
+  }, [
+    codeRef,
+    execute,
+    partIndex,
+    messageId,
+    blockIndex,
+    conversationId,
+    normalizedLang,
+    setIsDialogOpen,
+    isAuthenticated,
+  ]);
+
+  const debouncedExecute = useMemo(
+    () => debounce(handleExecute, 1000, { leading: true }),
+    [handleExecute],
+  );
+
+  useEffect(() => {
+    return () => {
+      debouncedExecute.cancel();
+    };
+  }, [debouncedExecute]);
+
+  if (typeof normalizedLang !== 'string' || normalizedLang.length === 0) {
+    return null;
+  }
+
+  return (
+    <>
+      <button
+        type="button"
+        className={cn('ml-auto flex gap-2')}
+        onClick={debouncedExecute}
+        disabled={execute.isLoading}
+      >
+        {execute.isLoading ? (
+          <Loader className="animate-spin" size={18} />
+        ) : (
+          <TerminalSquareIcon size={18} />
+        )}
+        {localize('com_ui_run_code')}
+      </button>
+      <ApiKeyDialog
+        onSubmit={onSubmit}
+        isOpen={isDialogOpen}
+        register={methods.register}
+        onRevoke={handleRevokeApiKey}
+        onOpenChange={setIsDialogOpen}
+        handleSubmit={methods.handleSubmit}
+        isToolAuthenticated={isAuthenticated}
+        isUserProvided={authType === AuthType.USER_PROVIDED}
+      />
+    </>
+  );
+});
+
+export default RunCode;

client/src/components/Messages/ContentRender.tsx

@@ -129,16 +129,17 @@ const ContentRender = memo(
           <div className="flex-col gap-1 md:gap-3">
             <div className="flex max-w-full flex-grow flex-col gap-0">
               <ContentParts
-                content={msg.content as Array<TMessageContentParts | undefined>}
-                messageId={msg.messageId}
-                isCreatedByUser={msg.isCreatedByUser}
-                isLast={isLast}
-                isSubmitting={isSubmitting}
                 edit={edit}
+                isLast={isLast}
                 enterEdit={enterEdit}
                 siblingIdx={siblingIdx}
+                messageId={msg.messageId}
+                isSubmitting={isSubmitting}
                 setSiblingIdx={setSiblingIdx}
                 attachments={msg.attachments}
+                isCreatedByUser={msg.isCreatedByUser}
+                conversationId={conversation?.conversationId}
+                content={msg.content as Array<TMessageContentParts | undefined>}
               />
             </div>
           </div>