mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-02-02 07:41:49 +01:00
🦥 refactor: Event-Driven Lazy Tool Loading (#11588)
* refactor: json schema tools with lazy loading - Added LocalToolExecutor class for lazy loading and caching of tools during execution. - Introduced ToolExecutionContext and ToolExecutor interfaces for better type management. - Created utility functions to generate tool proxies with JSON schema support. - Added ExtendedJsonSchema type for enhanced schema definitions. - Updated existing toolkits to utilize the new schema and executor functionalities. - Introduced a comprehensive tool definitions registry for managing various tool schemas. chore: update @librechat/agents to version 3.1.2 refactor: enhance tool loading optimization and classification - Improved the loadAgentToolsOptimized function to utilize a proxy pattern for all tools, enabling deferred execution and reducing overhead. - Introduced caching for tool instances and refined tool classification logic to streamline tool management. - Updated the handling of MCP tools to improve logging and error reporting for missing tools in the cache. - Enhanced the structure of tool definitions to support better classification and integration with existing tools. refactor: modularize tool loading and enhance optimization - Moved the loadAgentToolsOptimized function to a new service file for better organization and maintainability. - Updated the ToolService to utilize the new service for optimized tool loading, improving code clarity. - Removed legacy tool loading methods and streamlined the tool loading process to enhance performance and reduce complexity. - Introduced feature flag handling for optimized tool loading, allowing for easier toggling of this functionality. refactor: replace loadAgentToolsWithFlag with loadAgentTools in tool loader refactor: enhance MCP tool loading with proxy creation and classification refactor: optimize MCP tool loading by grouping tools by server - Introduced a Map to group cached tools by server name, improving the organization of tool data. - Updated the createMCPProxyTool function to accept server name directly, enhancing clarity. - Refactored the logic for handling MCP tools, streamlining the process of creating proxy tools for classification. refactor: enhance MCP tool loading and proxy creation - Added functionality to retrieve MCP server tools and reinitialize servers if necessary, improving tool availability. - Updated the tool loading logic to utilize a Map for organizing tools by server, enhancing clarity and performance. - Refactored the createToolProxy function to ensure a default response format, streamlining tool creation. refactor: update createToolProxy to ensure consistent response format - Modified the createToolProxy function to await the executor's execution and validate the result format. - Ensured that the function returns a default response structure when the result is not an array of two elements, enhancing reliability in tool proxy creation. refactor: ToolExecutionContext with toolCall property - Added toolCall property to ToolExecutionContext interface for improved context handling during tool execution. - Updated LocalToolExecutor to include toolCall in the runnable configuration, allowing for more flexible tool invocation. - Modified createToolProxy to pass toolCall from the configuration, ensuring consistent context across tool executions. refactor: enhance event-driven tool execution and logging - Introduced ToolExecuteOptions for improved handling of event-driven tool execution, allowing for parallel execution of tool calls. - Updated getDefaultHandlers to include support for ON_TOOL_EXECUTE events, enhancing the flexibility of tool invocation. - Added detailed logging in LocalToolExecutor to track tool loading and execution metrics, improving observability and debugging capabilities. - Refactored initializeClient to integrate event-driven tool loading, ensuring compatibility with the new execution model. chore: update @librechat/agents to version 3.1.21 refactor: remove legacy tool loading and executor components - Eliminated the loadAgentToolsWithFlag function, simplifying the tool loading process by directly using loadAgentTools. - Removed the LocalToolExecutor and related executor components to streamline the tool execution architecture. - Updated ToolService and related files to reflect the removal of deprecated features, enhancing code clarity and maintainability. refactor: enhance tool classification and definitions handling - Updated the loadAgentTools function to return toolDefinitions alongside toolRegistry, improving the structure of tool data returned to clients. - Removed the convertRegistryToDefinitions function from the initialize.js file, simplifying the initialization process. - Adjusted the buildToolClassification function to ensure toolDefinitions are built and returned simultaneously with the toolRegistry, enhancing efficiency in tool management. - Updated type definitions in initialize.ts to include toolDefinitions, ensuring consistency across the codebase. refactor: implement event-driven tool execution handler - Introduced createToolExecuteHandler function to streamline the handling of ON_TOOL_EXECUTE events, allowing for parallel execution of tool calls. - Updated getDefaultHandlers to utilize the new handler, simplifying the event-driven architecture. - Added handlers.ts file to encapsulate tool execution logic, improving code organization and maintainability. - Enhanced OpenAI handlers to integrate the new tool execution capabilities, ensuring consistent event handling across the application. refactor: integrate event-driven tool execution options - Added toolExecuteOptions to support event-driven tool execution in OpenAI and responses controllers, enhancing flexibility in tool handling. - Updated handlers to utilize createToolExecuteHandler, allowing for streamlined execution of tools during agent interactions. - Refactored service dependencies to include toolExecuteOptions, ensuring consistent integration across the application. refactor: enhance tool loading with definitionsOnly parameter - Updated createToolLoader and loadAgentTools functions to include a definitionsOnly parameter, allowing for the retrieval of only serializable tool definitions in event-driven mode. - Adjusted related interfaces and documentation to reflect the new parameter, improving clarity and flexibility in tool management. - Ensured compatibility across various components by integrating the definitionsOnly option in the initialization process. refactor: improve agent tool presence check in initialization - Added a check for tool presence using a new hasAgentTools variable, which evaluates both structuredTools and toolDefinitions. - Updated the conditional logic in the agent initialization process to utilize the hasAgentTools variable, enhancing clarity and maintainability in tool management. refactor: enhance agent tool extraction to support tool definitions - Updated the extractMCPServers function to handle both tool instances and serializable tool definitions, improving flexibility in agent tool management. - Added a new property toolDefinitions to the AgentWithTools type for better integration of event-driven mode. - Enhanced documentation to clarify the function's capabilities in extracting unique MCP server names from both tools and tool definitions. refactor: enhance tool classification and registry building - Added serverName property to ToolDefinition for improved tool identification. - Introduced buildToolRegistry function to streamline the creation of tool registries based on MCP tool definitions and agent options. - Updated buildToolClassification to utilize the new registry building logic, ensuring basic definitions are returned even when advanced classification features are not allowed. - Enhanced documentation and logging for clarity in tool classification processes. refactor: update @librechat/agents dependency to version 3.1.22 fix: expose loadTools function in ToolService - Added loadTools function to the exported module in ToolService.js, enhancing the accessibility of tool loading functionality. chore: remove configurable options from tool execute options in OpenAI controller refactor: enhance tool loading mechanism to utilize agent-specific context chore: update @librechat/agents dependency to version 3.1.23 fix: simplify result handling in createToolExecuteHandler * refactor: loadToolDefinitions for efficient tool loading in event-driven mode * refactor: replace legacy tool loading with loadToolsForExecution in OpenAI and responses controllers - Updated OpenAIChatCompletionController and createResponse functions to utilize loadToolsForExecution for improved tool loading. - Removed deprecated loadToolsLegacy references, streamlining the tool execution process. - Enhanced tool loading options to include agent-specific context and configurations. * refactor: enhance tool loading and execution handling - Introduced loadActionToolsForExecution function to streamline loading of action tools, improving organization and maintainability. - Updated loadToolsForExecution to handle both regular and action tools, optimizing the tool loading process. - Added detailed logging for missing tools in createToolExecuteHandler, enhancing error visibility. - Refactored tool definitions to normalize action tool names, improving consistency in tool management. * refactor: enhance built-in tool definitions loading - Updated loadToolDefinitions to include descriptions and parameters from the tool registry for built-in tools, improving the clarity and usability of tool definitions. - Integrated getToolDefinition to streamline the retrieval of tool metadata, enhancing the overall tool management process. * feat: add action tool definitions loading to tool service - Introduced getActionToolDefinitions function to load action tool definitions based on agent ID and tool names, enhancing the tool loading process. - Updated loadToolDefinitions to integrate action tool definitions, allowing for better management and retrieval of action-specific tools. - Added comprehensive tests for action tool definitions to ensure correct loading and parameter handling, improving overall reliability and functionality. * chore: update @librechat/agents dependency to version 3.1.26 * refactor: add toolEndCallback to handle tool execution results * fix: tool definitions and execution handling - Introduced native tools (execute_code, file_search, web_search) to the tool service, allowing for better integration and management of these tools. - Updated isBuiltInTool function to include native tools in the built-in check, improving tool recognition. - Added comprehensive tests for loading parameters of native tools, ensuring correct functionality and parameter handling. - Enhanced tool definitions registry to include new agent tool definitions, streamlining tool retrieval and management. * refactor: enhance tool loading and execution context - Added toolRegistry to the context for OpenAIChatCompletionController and createResponse functions, improving tool management. - Updated loadToolsForExecution to utilize toolRegistry for better integration of programmatic tools and tool search functionalities. - Enhanced the initialization process to include toolRegistry in agent context, streamlining tool access and configuration. - Refactored tool classification logic to support event-driven execution, ensuring compatibility with new tool definitions. * chore: add request duration logging to OpenAI and Responses controllers - Introduced logging for request start and completion times in OpenAIChatCompletionController and createResponse functions. - Calculated and logged the duration of each request, enhancing observability and performance tracking. - Improved debugging capabilities by providing detailed logs for both streaming and non-streaming responses. * chore: update @librechat/agents dependency to version 3.1.27 * refactor: implement buildToolSet function for tool management - Introduced buildToolSet function to streamline the creation of tool sets from agent configurations, enhancing tool management across various controllers. - Updated AgentClient, OpenAIChatCompletionController, and createResponse functions to utilize buildToolSet, improving consistency in tool handling. - Added comprehensive tests for buildToolSet to ensure correct functionality and edge case handling, enhancing overall reliability. * refactor: update import paths for ToolExecuteOptions and createToolExecuteHandler * fix: update GoogleSearch.js description for maximum search results - Changed the default maximum number of search results from 10 to 5 in the Google Search JSON schema description, ensuring accurate documentation of the expected behavior. * chore: remove deprecated Browser tool and associated assets - Deleted the Browser tool definition from manifest.json, which included its name, plugin key, description, and authentication configuration. - Removed the web-browser.svg asset as it is no longer needed following the removal of the Browser tool. * fix: ensure tool definitions are valid before processing - Added a check to verify the existence of tool definitions in the registry before accessing their properties, preventing potential runtime errors. - Updated the loading logic for built-in tool definitions to ensure that only valid definitions are pushed to the built-in tool definitions array. * fix: extend ExtendedJsonSchema to support 'null' type and nullable enums - Updated the ExtendedJsonSchema type to include 'null' as a valid type option. - Modified the enum property to accept an array of values that can include strings, numbers, booleans, and null, enhancing schema flexibility. * test: add comprehensive tests for tool definitions loading and registry behavior - Implemented tests to verify the handling of built-in tools without registry definitions, ensuring they are skipped correctly. - Added tests to confirm that built-in tools include descriptions and parameters in the registry. - Enhanced tests for action tools, checking for proper inclusion of metadata and handling of tools without parameters in the registry. * test: add tests for mixed-type and number enum schema handling - Introduced tests to validate the parsing of mixed-type enum values, including strings, numbers, booleans, and null. - Added tests for number enum schema values to ensure correct parsing of numeric inputs, enhancing schema validation coverage. * fix: update mock implementation for @librechat/agents - Changed the mock for @librechat/agents to spread the actual module's properties, ensuring that all necessary functionalities are preserved in tests. - This adjustment enhances the accuracy of the tests by reflecting the real structure of the module. * fix: change max_results type in GoogleSearch schema from number to integer - Updated the type of max_results in the Google Search JSON schema to 'integer' for better type accuracy and validation consistency. * fix: update max_results description and type in GoogleSearch schema - Changed the type of max_results from 'number' to 'integer' for improved type accuracy. - Updated the description to reflect the new default maximum number of search results, changing it from 10 to 5. * refactor: remove unused code and improve tool registry handling - Eliminated outdated comments and conditional logic related to event-driven mode in the ToolService. - Enhanced the handling of the tool registry by ensuring it is configurable for better integration during tool execution. * feat: add definitionsOnly option to buildToolClassification for event-driven mode - Introduced a new parameter, definitionsOnly, to the BuildToolClassificationParams interface to enable a mode that skips tool instance creation. - Updated the buildToolClassification function to conditionally add tool definitions without instantiating tools when definitionsOnly is true. - Modified the loadToolDefinitions function to pass definitionsOnly as true, ensuring compatibility with the new feature. * test: add unit tests for buildToolClassification with definitionsOnly option - Implemented tests to verify the behavior of buildToolClassification when definitionsOnly is set to true or false. - Ensured that tool instances are not created when definitionsOnly is true, while still adding necessary tool definitions. - Confirmed that loadAuthValues is called appropriately based on the definitionsOnly parameter, enhancing test coverage for this new feature.
This commit is contained in:
Danny Avila
GitHub
parent
6279ea8dd7
commit
5af1342dbb
46 changed files with 3297 additions and 565 deletions
|
|
@ -1,7 +1,12 @@
|
|||
const { nanoid } = require('nanoid');
|
||||
const { Constants } = require('@librechat/agents');
|
||||
const { logger } = require('@librechat/data-schemas');
|
||||
const { sendEvent, GenerationJobManager, writeAttachmentEvent } = require('@librechat/api');
|
||||
const {
|
||||
sendEvent,
|
||||
GenerationJobManager,
|
||||
writeAttachmentEvent,
|
||||
createToolExecuteHandler,
|
||||
} = require('@librechat/api');
|
||||
const { Tools, StepTypes, FileContext, ErrorTypes } = require('librechat-data-provider');
|
||||
const {
|
||||
EnvVar,
|
||||
|
|
@ -159,6 +164,12 @@ function emitEvent(res, streamId, eventData) {
|
|||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* @typedef {Object} ToolExecuteOptions
|
||||
* @property {(toolNames: string[]) => Promise<{loadedTools: StructuredTool[]}>} loadTools - Function to load tools by name
|
||||
* @property {Object} configurable - Configurable context for tool invocation
|
||||
*/
|
||||
|
||||
/**
|
||||
* Get default handlers for stream events.
|
||||
* @param {Object} options - The options object.
|
||||
|
|
@ -167,6 +178,7 @@ function emitEvent(res, streamId, eventData) {
|
|||
* @param {ToolEndCallback} options.toolEndCallback - Callback to use when tool ends.
|
||||
* @param {Array<UsageMetadata>} options.collectedUsage - The list of collected usage metadata.
|
||||
* @param {string | null} [options.streamId] - The stream ID for resumable mode, or null for standard mode.
|
||||
* @param {ToolExecuteOptions} [options.toolExecuteOptions] - Options for event-driven tool execution.
|
||||
* @returns {Record<string, t.EventHandler>} The default handlers.
|
||||
* @throws {Error} If the request is not found.
|
||||
*/
|
||||
|
|
@ -176,6 +188,7 @@ function getDefaultHandlers({
|
|||
toolEndCallback,
|
||||
collectedUsage,
|
||||
streamId = null,
|
||||
toolExecuteOptions = null,
|
||||
}) {
|
||||
if (!res || !aggregateContent) {
|
||||
throw new Error(
|
||||
|
|
@ -285,6 +298,10 @@ function getDefaultHandlers({
|
|||
},
|
||||
};
|
||||
|
||||
if (toolExecuteOptions) {
|
||||
handlers[GraphEvents.ON_TOOL_EXECUTE] = createToolExecuteHandler(toolExecuteOptions);
|
||||
}
|
||||
|
||||
return handlers;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -5,6 +5,7 @@ const {
|
|||
createRun,
|
||||
Tokenizer,
|
||||
checkAccess,
|
||||
buildToolSet,
|
||||
logAxiosError,
|
||||
sanitizeTitle,
|
||||
resolveHeaders,
|
||||
|
|
@ -974,7 +975,7 @@ class AgentClient extends BaseClient {
|
|||
version: 'v2',
|
||||
};
|
||||
|
||||
const toolSet = new Set((this.options.agent.tools ?? []).map((tool) => tool && tool.name));
|
||||
const toolSet = buildToolSet(this.options.agent);
|
||||
let { messages: initialMessages, indexTokenCountMap } = formatAgentMessages(
|
||||
payload,
|
||||
this.indexTokenCountMap,
|
||||
|
|
|
|||
|
|
@ -11,6 +11,7 @@ const {
|
|||
writeSSE,
|
||||
createRun,
|
||||
createChunk,
|
||||
buildToolSet,
|
||||
sendFinalChunk,
|
||||
createSafeUser,
|
||||
validateRequest,
|
||||
|
|
@ -19,11 +20,12 @@ const {
|
|||
buildNonStreamingResponse,
|
||||
createOpenAIStreamTracker,
|
||||
createOpenAIContentAggregator,
|
||||
createToolExecuteHandler,
|
||||
isChatCompletionValidationFailure,
|
||||
} = require('@librechat/api');
|
||||
const { loadAgentTools, loadToolsForExecution } = require('~/server/services/ToolService');
|
||||
const { createToolEndCallback } = require('~/server/controllers/agents/callbacks');
|
||||
const { findAccessibleResources } = require('~/server/services/PermissionService');
|
||||
const { loadAgentTools } = require('~/server/services/ToolService');
|
||||
const { getConvoFiles } = require('~/models/Conversation');
|
||||
const { getAgent, getAgents } = require('~/models/Agent');
|
||||
const db = require('~/models');
|
||||
|
|
@ -31,8 +33,10 @@ const db = require('~/models');
|
|||
/**
|
||||
* Creates a tool loader function for the agent.
|
||||
* @param {AbortSignal} signal - The abort signal
|
||||
* @param {boolean} [definitionsOnly=true] - When true, returns only serializable
|
||||
* tool definitions without creating full tool instances (for event-driven mode)
|
||||
*/
|
||||
function createToolLoader(signal) {
|
||||
function createToolLoader(signal, definitionsOnly = true) {
|
||||
return async function loadTools({
|
||||
req,
|
||||
res,
|
||||
|
|
@ -51,6 +55,7 @@ function createToolLoader(signal) {
|
|||
agent,
|
||||
signal,
|
||||
tool_resources,
|
||||
definitionsOnly,
|
||||
streamId: null, // No resumable stream for OpenAI compat
|
||||
});
|
||||
} catch (error) {
|
||||
|
|
@ -123,6 +128,7 @@ function sendErrorResponse(res, statusCode, message, type = 'invalid_request_err
|
|||
*/
|
||||
const OpenAIChatCompletionController = async (req, res) => {
|
||||
const appConfig = req.config;
|
||||
const requestStartTime = Date.now();
|
||||
|
||||
// Validate request
|
||||
const validation = validateRequest(req.body);
|
||||
|
|
@ -157,6 +163,10 @@ const OpenAIChatCompletionController = async (req, res) => {
|
|||
model: agentId,
|
||||
};
|
||||
|
||||
logger.debug(
|
||||
`[OpenAI API] Request ${requestId} started for agent ${agentId}, stream: ${request.stream}`,
|
||||
);
|
||||
|
||||
// Set up abort controller
|
||||
const abortController = new AbortController();
|
||||
|
||||
|
|
@ -239,19 +249,31 @@ const OpenAIChatCompletionController = async (req, res) => {
|
|||
}
|
||||
: null;
|
||||
|
||||
// We need custom handlers that stream in OpenAI format
|
||||
const collectedUsage = [];
|
||||
/** @type {Promise<import('librechat-data-provider').TAttachment | null>[]} */
|
||||
const artifactPromises = [];
|
||||
|
||||
// Create tool end callback for processing artifacts (images, file citations, code output)
|
||||
const toolEndCallback = createToolEndCallback({ req, res, artifactPromises, streamId: null });
|
||||
|
||||
// Convert messages to internal format
|
||||
const toolExecuteOptions = {
|
||||
loadTools: async (toolNames) => {
|
||||
return loadToolsForExecution({
|
||||
req,
|
||||
res,
|
||||
agent,
|
||||
toolNames,
|
||||
signal: abortController.signal,
|
||||
toolRegistry: primaryConfig.toolRegistry,
|
||||
userMCPAuthMap: primaryConfig.userMCPAuthMap,
|
||||
tool_resources: primaryConfig.tool_resources,
|
||||
});
|
||||
},
|
||||
toolEndCallback,
|
||||
};
|
||||
|
||||
const openaiMessages = convertMessages(request.messages);
|
||||
|
||||
// Format for agent
|
||||
const toolSet = new Set((primaryConfig.tools ?? []).map((tool) => tool && tool.name));
|
||||
const toolSet = buildToolSet(primaryConfig);
|
||||
const { messages: formattedMessages, indexTokenCountMap } = formatAgentMessages(
|
||||
openaiMessages,
|
||||
{},
|
||||
|
|
@ -425,6 +447,8 @@ const OpenAIChatCompletionController = async (req, res) => {
|
|||
on_chain_end: createHandler(),
|
||||
on_agent_update: createHandler(),
|
||||
on_custom_event: createHandler(),
|
||||
// Event-driven tool execution handler
|
||||
on_tool_execute: createToolExecuteHandler(toolExecuteOptions),
|
||||
};
|
||||
|
||||
// Create and run the agent
|
||||
|
|
@ -474,9 +498,11 @@ const OpenAIChatCompletionController = async (req, res) => {
|
|||
});
|
||||
|
||||
// Finalize response
|
||||
const duration = Date.now() - requestStartTime;
|
||||
if (isStreaming) {
|
||||
sendFinalChunk(handlerConfig);
|
||||
res.end();
|
||||
logger.debug(`[OpenAI API] Request ${requestId} completed in ${duration}ms (streaming)`);
|
||||
|
||||
// Wait for artifact processing after response ends (non-blocking)
|
||||
if (artifactPromises.length > 0) {
|
||||
|
|
@ -515,6 +541,7 @@ const OpenAIChatCompletionController = async (req, res) => {
|
|||
usage,
|
||||
);
|
||||
res.json(response);
|
||||
logger.debug(`[OpenAI API] Request ${requestId} completed in ${duration}ms (non-streaming)`);
|
||||
}
|
||||
} catch (error) {
|
||||
const errorMessage = error instanceof Error ? error.message : 'An error occurred';
|
||||
|
|
|
|||
|
|
@ -10,8 +10,10 @@ const {
|
|||
} = require('@librechat/agents');
|
||||
const {
|
||||
createRun,
|
||||
buildToolSet,
|
||||
createSafeUser,
|
||||
initializeAgent,
|
||||
createToolExecuteHandler,
|
||||
// Responses API
|
||||
writeDone,
|
||||
buildResponse,
|
||||
|
|
@ -34,9 +36,9 @@ const {
|
|||
createResponsesToolEndCallback,
|
||||
createToolEndCallback,
|
||||
} = require('~/server/controllers/agents/callbacks');
|
||||
const { loadAgentTools, loadToolsForExecution } = require('~/server/services/ToolService');
|
||||
const { findAccessibleResources } = require('~/server/services/PermissionService');
|
||||
const { getConvoFiles, saveConvo, getConvo } = require('~/models/Conversation');
|
||||
const { loadAgentTools } = require('~/server/services/ToolService');
|
||||
const { getAgent, getAgents } = require('~/models/Agent');
|
||||
const db = require('~/models');
|
||||
|
||||
|
|
@ -54,8 +56,10 @@ function setAppConfig(config) {
|
|||
/**
|
||||
* Creates a tool loader function for the agent.
|
||||
* @param {AbortSignal} signal - The abort signal
|
||||
* @param {boolean} [definitionsOnly=true] - When true, returns only serializable
|
||||
* tool definitions without creating full tool instances (for event-driven mode)
|
||||
*/
|
||||
function createToolLoader(signal) {
|
||||
function createToolLoader(signal, definitionsOnly = true) {
|
||||
return async function loadTools({
|
||||
req,
|
||||
res,
|
||||
|
|
@ -74,6 +78,7 @@ function createToolLoader(signal) {
|
|||
agent,
|
||||
signal,
|
||||
tool_resources,
|
||||
definitionsOnly,
|
||||
streamId: null,
|
||||
});
|
||||
} catch (error) {
|
||||
|
|
@ -261,6 +266,8 @@ function convertMessagesToOutputItems(messages) {
|
|||
* @param {import('express').Response} res
|
||||
*/
|
||||
const createResponse = async (req, res) => {
|
||||
const requestStartTime = Date.now();
|
||||
|
||||
// Validate request
|
||||
const validation = validateResponseRequest(req.body);
|
||||
if (isValidationFailure(validation)) {
|
||||
|
|
@ -291,6 +298,10 @@ const createResponse = async (req, res) => {
|
|||
// Create response context
|
||||
const context = createResponseContext(request, responseId);
|
||||
|
||||
logger.debug(
|
||||
`[Responses API] Request ${responseId} started for agent ${agentId}, stream: ${isStreaming}`,
|
||||
);
|
||||
|
||||
// Set up abort controller
|
||||
const abortController = new AbortController();
|
||||
|
||||
|
|
@ -362,8 +373,7 @@ const createResponse = async (req, res) => {
|
|||
// Merge previous messages with new input
|
||||
const allMessages = [...previousMessages, ...inputMessages];
|
||||
|
||||
// Format for agent
|
||||
const toolSet = new Set((primaryConfig.tools ?? []).map((tool) => tool && tool.name));
|
||||
const toolSet = buildToolSet(primaryConfig);
|
||||
const { messages: formattedMessages, indexTokenCountMap } = formatAgentMessages(
|
||||
allMessages,
|
||||
{},
|
||||
|
|
@ -407,6 +417,23 @@ const createResponse = async (req, res) => {
|
|||
artifactPromises,
|
||||
});
|
||||
|
||||
// Create tool execute options for event-driven tool execution
|
||||
const toolExecuteOptions = {
|
||||
loadTools: async (toolNames) => {
|
||||
return loadToolsForExecution({
|
||||
req,
|
||||
res,
|
||||
agent,
|
||||
toolNames,
|
||||
signal: abortController.signal,
|
||||
toolRegistry: primaryConfig.toolRegistry,
|
||||
userMCPAuthMap: primaryConfig.userMCPAuthMap,
|
||||
tool_resources: primaryConfig.tool_resources,
|
||||
});
|
||||
},
|
||||
toolEndCallback,
|
||||
};
|
||||
|
||||
// Combine handlers
|
||||
const handlers = {
|
||||
on_chat_model_stream: {
|
||||
|
|
@ -425,6 +452,7 @@ const createResponse = async (req, res) => {
|
|||
on_chain_end: { handle: () => {} },
|
||||
on_agent_update: { handle: () => {} },
|
||||
on_custom_event: { handle: () => {} },
|
||||
on_tool_execute: createToolExecuteHandler(toolExecuteOptions),
|
||||
};
|
||||
|
||||
// Create and run the agent
|
||||
|
|
@ -475,6 +503,9 @@ const createResponse = async (req, res) => {
|
|||
finalizeStream();
|
||||
res.end();
|
||||
|
||||
const duration = Date.now() - requestStartTime;
|
||||
logger.debug(`[Responses API] Request ${responseId} completed in ${duration}ms (streaming)`);
|
||||
|
||||
// Save to database if store: true
|
||||
if (request.store === true) {
|
||||
try {
|
||||
|
|
@ -504,18 +535,30 @@ const createResponse = async (req, res) => {
|
|||
});
|
||||
}
|
||||
} else {
|
||||
// Non-streaming response
|
||||
const aggregatorHandlers = createAggregatorEventHandlers(aggregator);
|
||||
|
||||
// Built-in handler for processing raw model stream chunks
|
||||
const chatModelStreamHandler = new ChatModelStreamHandler();
|
||||
|
||||
// Artifact promises for processing tool outputs
|
||||
/** @type {Promise<import('librechat-data-provider').TAttachment | null>[]} */
|
||||
const artifactPromises = [];
|
||||
const toolEndCallback = createToolEndCallback({ req, res, artifactPromises, streamId: null });
|
||||
|
||||
// Combine handlers
|
||||
const toolExecuteOptions = {
|
||||
loadTools: async (toolNames) => {
|
||||
return loadToolsForExecution({
|
||||
req,
|
||||
res,
|
||||
agent,
|
||||
toolNames,
|
||||
signal: abortController.signal,
|
||||
toolRegistry: primaryConfig.toolRegistry,
|
||||
userMCPAuthMap: primaryConfig.userMCPAuthMap,
|
||||
tool_resources: primaryConfig.tool_resources,
|
||||
});
|
||||
},
|
||||
toolEndCallback,
|
||||
};
|
||||
|
||||
const handlers = {
|
||||
on_chat_model_stream: {
|
||||
handle: async (event, data, metadata, graph) => {
|
||||
|
|
@ -533,9 +576,9 @@ const createResponse = async (req, res) => {
|
|||
on_chain_end: { handle: () => {} },
|
||||
on_agent_update: { handle: () => {} },
|
||||
on_custom_event: { handle: () => {} },
|
||||
on_tool_execute: createToolExecuteHandler(toolExecuteOptions),
|
||||
};
|
||||
|
||||
// Create and run the agent
|
||||
const userId = req.user?.id ?? 'api-user';
|
||||
const userMCPAuthMap = primaryConfig.userMCPAuthMap;
|
||||
|
||||
|
|
@ -557,7 +600,6 @@ const createResponse = async (req, res) => {
|
|||
throw new Error('Failed to create agent run');
|
||||
}
|
||||
|
||||
// Process the stream
|
||||
const config = {
|
||||
runName: 'AgentRun',
|
||||
configurable: {
|
||||
|
|
@ -579,7 +621,6 @@ const createResponse = async (req, res) => {
|
|||
},
|
||||
});
|
||||
|
||||
// Wait for artifacts before sending response
|
||||
if (artifactPromises.length > 0) {
|
||||
try {
|
||||
await Promise.all(artifactPromises);
|
||||
|
|
@ -588,19 +629,14 @@ const createResponse = async (req, res) => {
|
|||
}
|
||||
}
|
||||
|
||||
// Build and send the response
|
||||
const response = buildAggregatedResponse(context, aggregator);
|
||||
|
||||
// Save to database if store: true
|
||||
if (request.store === true) {
|
||||
try {
|
||||
// Save conversation
|
||||
await saveConversation(req, conversationId, agentId, agent);
|
||||
|
||||
// Save input messages
|
||||
await saveInputMessages(req, conversationId, inputMessages, agentId);
|
||||
|
||||
// Save response output
|
||||
await saveResponseOutput(req, conversationId, responseId, response, agentId);
|
||||
|
||||
logger.debug(
|
||||
|
|
@ -613,6 +649,11 @@ const createResponse = async (req, res) => {
|
|||
}
|
||||
|
||||
res.json(response);
|
||||
|
||||
const duration = Date.now() - requestStartTime;
|
||||
logger.debug(
|
||||
`[Responses API] Request ${responseId} completed in ${duration}ms (non-streaming)`,
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
const errorMessage = error instanceof Error ? error.message : 'An error occurred';
|
||||
|
|
|
|||
|
|
@ -19,8 +19,8 @@ const {
|
|||
createToolEndCallback,
|
||||
getDefaultHandlers,
|
||||
} = require('~/server/controllers/agents/callbacks');
|
||||
const { loadAgentTools, loadToolsForExecution } = require('~/server/services/ToolService');
|
||||
const { getModelsConfig } = require('~/server/controllers/ModelController');
|
||||
const { loadAgentTools } = require('~/server/services/ToolService');
|
||||
const AgentClient = require('~/server/controllers/agents/client');
|
||||
const { getConvoFiles } = require('~/models/Conversation');
|
||||
const { processAddedConvo } = require('./addedConvo');
|
||||
|
|
@ -32,8 +32,10 @@ const db = require('~/models');
|
|||
* Creates a tool loader function for the agent.
|
||||
* @param {AbortSignal} signal - The abort signal
|
||||
* @param {string | null} [streamId] - The stream ID for resumable mode
|
||||
* @param {boolean} [definitionsOnly=false] - When true, returns only serializable
|
||||
* tool definitions without creating full tool instances (for event-driven mode)
|
||||
*/
|
||||
function createToolLoader(signal, streamId = null) {
|
||||
function createToolLoader(signal, streamId = null, definitionsOnly = false) {
|
||||
/**
|
||||
* @param {object} params
|
||||
* @param {ServerRequest} params.req
|
||||
|
|
@ -44,8 +46,9 @@ function createToolLoader(signal, streamId = null) {
|
|||
* @param {string} params.model
|
||||
* @param {AgentToolResources} params.tool_resources
|
||||
* @returns {Promise<{
|
||||
* tools: StructuredTool[],
|
||||
* tools?: StructuredTool[],
|
||||
* toolContextMap: Record<string, unknown>,
|
||||
* toolDefinitions?: import('@librechat/agents').LCTool[],
|
||||
* userMCPAuthMap?: Record<string, Record<string, string>>,
|
||||
* toolRegistry?: import('@librechat/agents').LCToolRegistry
|
||||
* } | undefined>}
|
||||
|
|
@ -67,8 +70,9 @@ function createToolLoader(signal, streamId = null) {
|
|||
res,
|
||||
agent,
|
||||
signal,
|
||||
tool_resources,
|
||||
streamId,
|
||||
tool_resources,
|
||||
definitionsOnly,
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Error loading tools for agent ' + agentId, error);
|
||||
|
|
@ -91,8 +95,46 @@ const initializeClient = async ({ req, res, signal, endpointOption }) => {
|
|||
const artifactPromises = [];
|
||||
const { contentParts, aggregateContent } = createContentAggregator();
|
||||
const toolEndCallback = createToolEndCallback({ req, res, artifactPromises, streamId });
|
||||
|
||||
/**
|
||||
* Agent context store - populated after initialization, accessed by callback via closure.
|
||||
* Maps agentId -> { userMCPAuthMap, agent, tool_resources, toolRegistry, openAIApiKey }
|
||||
* @type {Map<string, {
|
||||
* userMCPAuthMap?: Record<string, Record<string, string>>,
|
||||
* agent?: object,
|
||||
* tool_resources?: object,
|
||||
* toolRegistry?: import('@librechat/agents').LCToolRegistry,
|
||||
* openAIApiKey?: string
|
||||
* }>}
|
||||
*/
|
||||
const agentToolContexts = new Map();
|
||||
|
||||
const toolExecuteOptions = {
|
||||
loadTools: async (toolNames, agentId) => {
|
||||
const ctx = agentToolContexts.get(agentId) ?? {};
|
||||
logger.debug(`[ON_TOOL_EXECUTE] ctx found: ${!!ctx.userMCPAuthMap}, agent: ${ctx.agent?.id}`);
|
||||
|
||||
const result = await loadToolsForExecution({
|
||||
req,
|
||||
res,
|
||||
signal,
|
||||
streamId,
|
||||
toolNames,
|
||||
agent: ctx.agent,
|
||||
toolRegistry: ctx.toolRegistry,
|
||||
userMCPAuthMap: ctx.userMCPAuthMap,
|
||||
tool_resources: ctx.tool_resources,
|
||||
});
|
||||
|
||||
logger.debug(`[ON_TOOL_EXECUTE] loaded ${result.loadedTools?.length ?? 0} tools`);
|
||||
return result;
|
||||
},
|
||||
toolEndCallback,
|
||||
};
|
||||
|
||||
const eventHandlers = getDefaultHandlers({
|
||||
res,
|
||||
toolExecuteOptions,
|
||||
aggregateContent,
|
||||
toolEndCallback,
|
||||
collectedUsage,
|
||||
|
|
@ -125,7 +167,8 @@ const initializeClient = async ({ req, res, signal, endpointOption }) => {
|
|||
const agentConfigs = new Map();
|
||||
const allowedProviders = new Set(appConfig?.endpoints?.[EModelEndpoint.agents]?.allowedProviders);
|
||||
|
||||
const loadTools = createToolLoader(signal, streamId);
|
||||
/** Event-driven mode: only load tool definitions, not full instances */
|
||||
const loadTools = createToolLoader(signal, streamId, true);
|
||||
/** @type {Array<MongoFile>} */
|
||||
const requestFiles = req.body.files ?? [];
|
||||
/** @type {string} */
|
||||
|
|
@ -159,6 +202,19 @@ const initializeClient = async ({ req, res, signal, endpointOption }) => {
|
|||
},
|
||||
);
|
||||
|
||||
logger.debug(
|
||||
`[initializeClient] Tool definitions for primary agent: ${primaryConfig.toolDefinitions?.length ?? 0}`,
|
||||
);
|
||||
|
||||
/** Store primary agent's tool context for ON_TOOL_EXECUTE callback */
|
||||
logger.debug(`[initializeClient] Storing tool context for agentId: ${primaryConfig.id}`);
|
||||
agentToolContexts.set(primaryConfig.id, {
|
||||
agent: primaryAgent,
|
||||
toolRegistry: primaryConfig.toolRegistry,
|
||||
userMCPAuthMap: primaryConfig.userMCPAuthMap,
|
||||
tool_resources: primaryConfig.tool_resources,
|
||||
});
|
||||
|
||||
const agent_ids = primaryConfig.agent_ids;
|
||||
let userMCPAuthMap = primaryConfig.userMCPAuthMap;
|
||||
|
||||
|
|
@ -211,11 +267,21 @@ const initializeClient = async ({ req, res, signal, endpointOption }) => {
|
|||
getCodeGeneratedFiles: db.getCodeGeneratedFiles,
|
||||
},
|
||||
);
|
||||
|
||||
if (userMCPAuthMap != null) {
|
||||
Object.assign(userMCPAuthMap, config.userMCPAuthMap ?? {});
|
||||
} else {
|
||||
userMCPAuthMap = config.userMCPAuthMap;
|
||||
}
|
||||
|
||||
/** Store handoff agent's tool context for ON_TOOL_EXECUTE callback */
|
||||
agentToolContexts.set(agentId, {
|
||||
agent,
|
||||
toolRegistry: config.toolRegistry,
|
||||
userMCPAuthMap: config.userMCPAuthMap,
|
||||
tool_resources: config.tool_resources,
|
||||
});
|
||||
|
||||
agentConfigs.set(agentId, config);
|
||||
return agent;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,4 +1,3 @@
|
|||
const { z } = require('zod');
|
||||
const { tool } = require('@langchain/core/tools');
|
||||
const { logger } = require('@librechat/data-schemas');
|
||||
const {
|
||||
|
|
@ -12,7 +11,7 @@ const {
|
|||
MCPOAuthHandler,
|
||||
isMCPDomainAllowed,
|
||||
normalizeServerName,
|
||||
convertWithResolvedRefs,
|
||||
resolveJsonSchemaRefs,
|
||||
GenerationJobManager,
|
||||
} = require('@librechat/api');
|
||||
const {
|
||||
|
|
@ -34,6 +33,16 @@ const { reinitMCPServer } = require('./Tools/mcp');
|
|||
const { getAppConfig } = require('./Config');
|
||||
const { getLogStores } = require('~/cache');
|
||||
|
||||
function isEmptyObjectSchema(jsonSchema) {
|
||||
return (
|
||||
jsonSchema != null &&
|
||||
typeof jsonSchema === 'object' &&
|
||||
jsonSchema.type === 'object' &&
|
||||
(jsonSchema.properties == null || Object.keys(jsonSchema.properties).length === 0) &&
|
||||
!jsonSchema.additionalProperties
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {object} params
|
||||
* @param {ServerResponse} params.res - The Express response object for sending events.
|
||||
|
|
@ -197,6 +206,9 @@ async function reconnectServer({
|
|||
userMCPAuthMap,
|
||||
streamId = null,
|
||||
}) {
|
||||
logger.debug(
|
||||
`[MCP][reconnectServer] serverName: ${serverName}, user: ${user?.id}, hasUserMCPAuthMap: ${!!userMCPAuthMap}`,
|
||||
);
|
||||
const runId = Constants.USE_PRELIM_RESPONSE_MESSAGE_ID;
|
||||
const flowId = `${user.id}:${serverName}:${Date.now()}`;
|
||||
const flowManager = getFlowStateManager(getLogStores(CacheKeys.FLOWS));
|
||||
|
|
@ -429,13 +441,17 @@ function createToolInstance({
|
|||
/** @type {LCTool} */
|
||||
const { description, parameters } = toolDefinition;
|
||||
const isGoogle = _provider === Providers.VERTEXAI || _provider === Providers.GOOGLE;
|
||||
let schema = convertWithResolvedRefs(parameters, {
|
||||
allowEmptyObject: !isGoogle,
|
||||
transformOneOfAnyOf: true,
|
||||
});
|
||||
|
||||
if (!schema) {
|
||||
schema = z.object({ input: z.string().optional() });
|
||||
let schema = parameters ? resolveJsonSchemaRefs(parameters) : null;
|
||||
|
||||
if (!schema || (isGoogle && isEmptyObjectSchema(schema))) {
|
||||
schema = {
|
||||
type: 'object',
|
||||
properties: {
|
||||
input: { type: 'string', description: 'Input for the tool' },
|
||||
},
|
||||
required: [],
|
||||
};
|
||||
}
|
||||
|
||||
const normalizedToolKey = `${toolName}${Constants.mcp_delimiter}${normalizeServerName(serverName)}`;
|
||||
|
|
|
|||
|
|
@ -53,7 +53,7 @@ jest.mock('@librechat/api', () => {
|
|||
},
|
||||
sendEvent: jest.fn(),
|
||||
normalizeServerName: jest.fn((name) => name),
|
||||
convertWithResolvedRefs: jest.fn((params) => params),
|
||||
resolveJsonSchemaRefs: jest.fn((params) => params),
|
||||
get isMCPDomainAllowed() {
|
||||
return mockIsMCPDomainAllowed;
|
||||
},
|
||||
|
|
|
|||
|
|
@ -1,10 +1,17 @@
|
|||
const { sleep } = require('@librechat/agents');
|
||||
const {
|
||||
sleep,
|
||||
EnvVar,
|
||||
Constants,
|
||||
createToolSearch,
|
||||
createProgrammaticToolCallingTool,
|
||||
} = require('@librechat/agents');
|
||||
const { logger } = require('@librechat/data-schemas');
|
||||
const { tool: toolFn, DynamicStructuredTool } = require('@langchain/core/tools');
|
||||
const {
|
||||
getToolkitKey,
|
||||
hasCustomUserVars,
|
||||
getUserMCPAuthMap,
|
||||
loadToolDefinitions,
|
||||
isActionDomainAllowed,
|
||||
buildToolClassification,
|
||||
} = require('@librechat/api');
|
||||
|
|
@ -20,9 +27,12 @@ const {
|
|||
AgentCapabilities,
|
||||
isEphemeralAgentId,
|
||||
validateActionDomain,
|
||||
actionDomainSeparator,
|
||||
defaultAgentCapabilities,
|
||||
validateAndParseOpenAPISpec,
|
||||
} = require('librechat-data-provider');
|
||||
|
||||
const domainSeparatorRegex = new RegExp(actionDomainSeparator, 'g');
|
||||
const {
|
||||
createActionTool,
|
||||
decryptMetadata,
|
||||
|
|
@ -30,14 +40,19 @@ const {
|
|||
domainParser,
|
||||
} = require('./ActionService');
|
||||
const { processFileURL, uploadImageBuffer } = require('~/server/services/Files/process');
|
||||
const { getEndpointsConfig, getCachedTools } = require('~/server/services/Config');
|
||||
const {
|
||||
getEndpointsConfig,
|
||||
getCachedTools,
|
||||
getMCPServerTools,
|
||||
} = require('~/server/services/Config');
|
||||
const { manifestToolMap, toolkits } = require('~/app/clients/tools/manifest');
|
||||
const { createOnSearchResults } = require('~/server/services/Tools/search');
|
||||
const { loadAuthValues } = require('~/server/services/Tools/credentials');
|
||||
const { reinitMCPServer } = require('~/server/services/Tools/mcp');
|
||||
const { recordUsage } = require('~/server/services/Threads');
|
||||
const { loadTools } = require('~/app/clients/tools/util');
|
||||
const { redactMessage } = require('~/config/parsers');
|
||||
const { findPluginAuthsByKeys } = require('~/models');
|
||||
const { loadAuthValues } = require('~/server/services/Tools/credentials');
|
||||
/**
|
||||
* Processes the required actions by calling the appropriate tools and returning the outputs.
|
||||
* @param {OpenAIClient} client - OpenAI or StreamRunManager Client.
|
||||
|
|
@ -377,6 +392,187 @@ async function processRequiredActions(client, requiredActions) {
|
|||
* hasDeferredTools?: boolean;
|
||||
* }>} The agent tools and registry.
|
||||
*/
|
||||
/** Native LibreChat tools that are not in the manifest */
|
||||
const nativeTools = new Set([Tools.execute_code, Tools.file_search, Tools.web_search]);
|
||||
|
||||
/** Checks if a tool name is a known built-in tool */
|
||||
const isBuiltInTool = (toolName) =>
|
||||
Boolean(
|
||||
manifestToolMap[toolName] ||
|
||||
toolkits.some((t) => t.pluginKey === toolName) ||
|
||||
nativeTools.has(toolName),
|
||||
);
|
||||
|
||||
/**
|
||||
* Loads only tool definitions without creating tool instances.
|
||||
* This is the efficient path for event-driven mode where tools are loaded on-demand.
|
||||
*
|
||||
* @param {Object} params
|
||||
* @param {ServerRequest} params.req - The request object
|
||||
* @param {Object} params.agent - The agent configuration
|
||||
* @returns {Promise<{
|
||||
* toolDefinitions?: import('@librechat/api').LCTool[];
|
||||
* toolRegistry?: Map<string, import('@librechat/api').LCTool>;
|
||||
* userMCPAuthMap?: Record<string, Record<string, string>>;
|
||||
* hasDeferredTools?: boolean;
|
||||
* }>}
|
||||
*/
|
||||
async function loadToolDefinitionsWrapper({ req, agent }) {
|
||||
if (!agent.tools || agent.tools.length === 0) {
|
||||
return { toolDefinitions: [] };
|
||||
}
|
||||
|
||||
if (
|
||||
agent.tools.length === 1 &&
|
||||
(agent.tools[0] === AgentCapabilities.context || agent.tools[0] === AgentCapabilities.ocr)
|
||||
) {
|
||||
return { toolDefinitions: [] };
|
||||
}
|
||||
|
||||
const appConfig = req.config;
|
||||
const endpointsConfig = await getEndpointsConfig(req);
|
||||
let enabledCapabilities = new Set(endpointsConfig?.[EModelEndpoint.agents]?.capabilities ?? []);
|
||||
|
||||
if (enabledCapabilities.size === 0 && isEphemeralAgentId(agent.id)) {
|
||||
enabledCapabilities = new Set(
|
||||
appConfig.endpoints?.[EModelEndpoint.agents]?.capabilities ?? defaultAgentCapabilities,
|
||||
);
|
||||
}
|
||||
|
||||
const checkCapability = (capability) => enabledCapabilities.has(capability);
|
||||
const areToolsEnabled = checkCapability(AgentCapabilities.tools);
|
||||
const deferredToolsEnabled = checkCapability(AgentCapabilities.deferred_tools);
|
||||
|
||||
const filteredTools = agent.tools?.filter((tool) => {
|
||||
if (tool === Tools.file_search) {
|
||||
return checkCapability(AgentCapabilities.file_search);
|
||||
}
|
||||
if (tool === Tools.execute_code) {
|
||||
return checkCapability(AgentCapabilities.execute_code);
|
||||
}
|
||||
if (tool === Tools.web_search) {
|
||||
return checkCapability(AgentCapabilities.web_search);
|
||||
}
|
||||
if (!areToolsEnabled && !tool.includes(actionDelimiter)) {
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
});
|
||||
|
||||
if (!filteredTools || filteredTools.length === 0) {
|
||||
return { toolDefinitions: [] };
|
||||
}
|
||||
|
||||
/** @type {Record<string, Record<string, string>>} */
|
||||
let userMCPAuthMap;
|
||||
if (hasCustomUserVars(req.config)) {
|
||||
userMCPAuthMap = await getUserMCPAuthMap({
|
||||
tools: agent.tools,
|
||||
userId: req.user.id,
|
||||
findPluginAuthsByKeys,
|
||||
});
|
||||
}
|
||||
|
||||
const getOrFetchMCPServerTools = async (userId, serverName) => {
|
||||
const cached = await getMCPServerTools(userId, serverName);
|
||||
if (cached) {
|
||||
return cached;
|
||||
}
|
||||
|
||||
const result = await reinitMCPServer({
|
||||
user: req.user,
|
||||
serverName,
|
||||
userMCPAuthMap,
|
||||
});
|
||||
|
||||
return result?.availableTools || null;
|
||||
};
|
||||
|
||||
const getActionToolDefinitions = async (agentId, actionToolNames) => {
|
||||
const actionSets = (await loadActionSets({ agent_id: agentId })) ?? [];
|
||||
if (actionSets.length === 0) {
|
||||
return [];
|
||||
}
|
||||
|
||||
const definitions = [];
|
||||
const allowedDomains = appConfig?.actions?.allowedDomains;
|
||||
|
||||
for (const action of actionSets) {
|
||||
const domain = await domainParser(action.metadata.domain, true);
|
||||
const normalizedDomain = domain.replace(domainSeparatorRegex, '_');
|
||||
|
||||
const isDomainAllowed = await isActionDomainAllowed(action.metadata.domain, allowedDomains);
|
||||
if (!isDomainAllowed) {
|
||||
logger.warn(
|
||||
`[Actions] Domain "${action.metadata.domain}" not in allowedDomains. ` +
|
||||
`Add it to librechat.yaml actions.allowedDomains to enable this action.`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
const validationResult = validateAndParseOpenAPISpec(action.metadata.raw_spec);
|
||||
if (!validationResult.spec || !validationResult.serverUrl) {
|
||||
logger.warn(`[Actions] Invalid OpenAPI spec for domain: ${domain}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
const { functionSignatures } = openapiToFunction(validationResult.spec, true);
|
||||
|
||||
for (const sig of functionSignatures) {
|
||||
const toolName = `${sig.name}${actionDelimiter}${normalizedDomain}`;
|
||||
if (!actionToolNames.some((name) => name.replace(domainSeparatorRegex, '_') === toolName)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
definitions.push({
|
||||
name: toolName,
|
||||
description: sig.description,
|
||||
parameters: sig.parameters,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return definitions;
|
||||
};
|
||||
|
||||
const { toolDefinitions, toolRegistry, hasDeferredTools } = await loadToolDefinitions(
|
||||
{
|
||||
userId: req.user.id,
|
||||
agentId: agent.id,
|
||||
tools: filteredTools,
|
||||
toolOptions: agent.tool_options,
|
||||
deferredToolsEnabled,
|
||||
},
|
||||
{
|
||||
isBuiltInTool,
|
||||
loadAuthValues,
|
||||
getOrFetchMCPServerTools,
|
||||
getActionToolDefinitions,
|
||||
},
|
||||
);
|
||||
|
||||
return {
|
||||
toolRegistry,
|
||||
userMCPAuthMap,
|
||||
toolDefinitions,
|
||||
hasDeferredTools,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Loads agent tools for initialization or execution.
|
||||
* @param {Object} params
|
||||
* @param {ServerRequest} params.req - The request object
|
||||
* @param {ServerResponse} params.res - The response object
|
||||
* @param {Object} params.agent - The agent configuration
|
||||
* @param {AbortSignal} [params.signal] - Abort signal
|
||||
* @param {Object} [params.tool_resources] - Tool resources
|
||||
* @param {string} [params.openAIApiKey] - OpenAI API key
|
||||
* @param {string|null} [params.streamId] - Stream ID for resumable mode
|
||||
* @param {boolean} [params.definitionsOnly=true] - When true, returns only serializable
|
||||
* tool definitions without creating full tool instances. Use for event-driven mode
|
||||
* where tools are loaded on-demand during execution.
|
||||
*/
|
||||
async function loadAgentTools({
|
||||
req,
|
||||
res,
|
||||
|
|
@ -385,16 +581,21 @@ async function loadAgentTools({
|
|||
tool_resources,
|
||||
openAIApiKey,
|
||||
streamId = null,
|
||||
definitionsOnly = true,
|
||||
}) {
|
||||
if (definitionsOnly) {
|
||||
return loadToolDefinitionsWrapper({ req, agent });
|
||||
}
|
||||
|
||||
if (!agent.tools || agent.tools.length === 0) {
|
||||
return {};
|
||||
return { toolDefinitions: [] };
|
||||
} else if (
|
||||
agent.tools &&
|
||||
agent.tools.length === 1 &&
|
||||
/** Legacy handling for `ocr` as may still exist in existing Agents */
|
||||
(agent.tools[0] === AgentCapabilities.context || agent.tools[0] === AgentCapabilities.ocr)
|
||||
) {
|
||||
return {};
|
||||
return { toolDefinitions: [] };
|
||||
}
|
||||
|
||||
const appConfig = req.config;
|
||||
|
|
@ -480,6 +681,18 @@ async function loadAgentTools({
|
|||
imageOutputType: appConfig.imageOutputType,
|
||||
});
|
||||
|
||||
/** Build tool registry from MCP tools and create PTC/tool search tools if configured */
|
||||
const deferredToolsEnabled = checkCapability(AgentCapabilities.deferred_tools);
|
||||
const { toolRegistry, toolDefinitions, additionalTools, hasDeferredTools } =
|
||||
await buildToolClassification({
|
||||
loadedTools,
|
||||
userId: req.user.id,
|
||||
agentId: agent.id,
|
||||
agentToolOptions: agent.tool_options,
|
||||
deferredToolsEnabled,
|
||||
loadAuthValues,
|
||||
});
|
||||
|
||||
const agentTools = [];
|
||||
for (let i = 0; i < loadedTools.length; i++) {
|
||||
const tool = loadedTools[i];
|
||||
|
|
@ -524,25 +737,16 @@ async function loadAgentTools({
|
|||
return map;
|
||||
}, {});
|
||||
|
||||
/** Build tool registry from MCP tools and create PTC/tool search tools if configured */
|
||||
const deferredToolsEnabled = checkCapability(AgentCapabilities.deferred_tools);
|
||||
const { toolRegistry, additionalTools, hasDeferredTools } = await buildToolClassification({
|
||||
loadedTools,
|
||||
userId: req.user.id,
|
||||
agentId: agent.id,
|
||||
agentToolOptions: agent.tool_options,
|
||||
deferredToolsEnabled,
|
||||
loadAuthValues,
|
||||
});
|
||||
agentTools.push(...additionalTools);
|
||||
|
||||
if (!checkCapability(AgentCapabilities.actions)) {
|
||||
return {
|
||||
tools: agentTools,
|
||||
toolRegistry,
|
||||
userMCPAuthMap,
|
||||
toolContextMap,
|
||||
toolRegistry,
|
||||
toolDefinitions,
|
||||
hasDeferredTools,
|
||||
tools: agentTools,
|
||||
};
|
||||
}
|
||||
|
||||
|
|
@ -552,11 +756,12 @@ async function loadAgentTools({
|
|||
logger.warn(`No tools found for the specified tool calls: ${_agentTools.join(', ')}`);
|
||||
}
|
||||
return {
|
||||
tools: agentTools,
|
||||
toolRegistry,
|
||||
userMCPAuthMap,
|
||||
toolContextMap,
|
||||
toolRegistry,
|
||||
toolDefinitions,
|
||||
hasDeferredTools,
|
||||
tools: agentTools,
|
||||
};
|
||||
}
|
||||
|
||||
|
|
@ -681,16 +886,293 @@ async function loadAgentTools({
|
|||
}
|
||||
|
||||
return {
|
||||
tools: agentTools,
|
||||
toolRegistry,
|
||||
toolContextMap,
|
||||
userMCPAuthMap,
|
||||
toolRegistry,
|
||||
toolDefinitions,
|
||||
hasDeferredTools,
|
||||
tools: agentTools,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Loads tools for event-driven execution (ON_TOOL_EXECUTE handler).
|
||||
* This function encapsulates all dependencies needed for tool loading,
|
||||
* so callers don't need to import processFileURL, uploadImageBuffer, etc.
|
||||
*
|
||||
* Handles both regular tools (MCP, built-in) and action tools.
|
||||
*
|
||||
* @param {Object} params
|
||||
* @param {ServerRequest} params.req - The request object
|
||||
* @param {ServerResponse} params.res - The response object
|
||||
* @param {AbortSignal} [params.signal] - Abort signal
|
||||
* @param {Object} params.agent - The agent object
|
||||
* @param {string[]} params.toolNames - Names of tools to load
|
||||
* @param {Record<string, Record<string, string>>} [params.userMCPAuthMap] - User MCP auth map
|
||||
* @param {Object} [params.tool_resources] - Tool resources
|
||||
* @param {string|null} [params.streamId] - Stream ID for web search callbacks
|
||||
* @returns {Promise<{ loadedTools: Array, configurable: Object }>}
|
||||
*/
|
||||
async function loadToolsForExecution({
|
||||
req,
|
||||
res,
|
||||
signal,
|
||||
agent,
|
||||
toolNames,
|
||||
toolRegistry,
|
||||
userMCPAuthMap,
|
||||
tool_resources,
|
||||
streamId = null,
|
||||
}) {
|
||||
const appConfig = req.config;
|
||||
const allLoadedTools = [];
|
||||
const configurable = { userMCPAuthMap };
|
||||
|
||||
const isToolSearch = toolNames.includes(Constants.TOOL_SEARCH);
|
||||
const isPTC = toolNames.includes(Constants.PROGRAMMATIC_TOOL_CALLING);
|
||||
|
||||
if (isToolSearch && toolRegistry) {
|
||||
const toolSearchTool = createToolSearch({
|
||||
mode: 'local',
|
||||
toolRegistry,
|
||||
});
|
||||
allLoadedTools.push(toolSearchTool);
|
||||
configurable.toolRegistry = toolRegistry;
|
||||
}
|
||||
|
||||
if (isPTC && toolRegistry) {
|
||||
configurable.toolRegistry = toolRegistry;
|
||||
try {
|
||||
const authValues = await loadAuthValues({
|
||||
userId: req.user.id,
|
||||
authFields: [EnvVar.CODE_API_KEY],
|
||||
});
|
||||
const codeApiKey = authValues[EnvVar.CODE_API_KEY];
|
||||
|
||||
if (codeApiKey) {
|
||||
const ptcTool = createProgrammaticToolCallingTool({ apiKey: codeApiKey });
|
||||
allLoadedTools.push(ptcTool);
|
||||
} else {
|
||||
logger.warn('[loadToolsForExecution] PTC requested but CODE_API_KEY not available');
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error('[loadToolsForExecution] Error creating PTC tool:', error);
|
||||
}
|
||||
}
|
||||
|
||||
const specialToolNames = new Set([Constants.TOOL_SEARCH, Constants.PROGRAMMATIC_TOOL_CALLING]);
|
||||
|
||||
let ptcOrchestratedToolNames = [];
|
||||
if (isPTC && toolRegistry) {
|
||||
ptcOrchestratedToolNames = Array.from(toolRegistry.keys()).filter(
|
||||
(name) => !specialToolNames.has(name),
|
||||
);
|
||||
}
|
||||
|
||||
const requestedNonSpecialToolNames = toolNames.filter((name) => !specialToolNames.has(name));
|
||||
const allToolNamesToLoad = isPTC
|
||||
? [...new Set([...requestedNonSpecialToolNames, ...ptcOrchestratedToolNames])]
|
||||
: requestedNonSpecialToolNames;
|
||||
|
||||
const actionToolNames = allToolNamesToLoad.filter((name) => name.includes(actionDelimiter));
|
||||
const regularToolNames = allToolNamesToLoad.filter((name) => !name.includes(actionDelimiter));
|
||||
|
||||
if (regularToolNames.length > 0) {
|
||||
const includesWebSearch = regularToolNames.includes(Tools.web_search);
|
||||
const webSearchCallbacks = includesWebSearch ? createOnSearchResults(res, streamId) : undefined;
|
||||
|
||||
const { loadedTools } = await loadTools({
|
||||
agent,
|
||||
signal,
|
||||
userMCPAuthMap,
|
||||
functions: true,
|
||||
tools: regularToolNames,
|
||||
user: req.user.id,
|
||||
options: {
|
||||
req,
|
||||
res,
|
||||
processFileURL,
|
||||
uploadImageBuffer,
|
||||
returnMetadata: true,
|
||||
tool_resources,
|
||||
[Tools.web_search]: webSearchCallbacks,
|
||||
},
|
||||
webSearch: appConfig?.webSearch,
|
||||
fileStrategy: appConfig?.fileStrategy,
|
||||
imageOutputType: appConfig?.imageOutputType,
|
||||
});
|
||||
|
||||
if (loadedTools) {
|
||||
allLoadedTools.push(...loadedTools);
|
||||
}
|
||||
}
|
||||
|
||||
if (actionToolNames.length > 0 && agent) {
|
||||
const actionTools = await loadActionToolsForExecution({
|
||||
req,
|
||||
res,
|
||||
agent,
|
||||
appConfig,
|
||||
streamId,
|
||||
actionToolNames,
|
||||
});
|
||||
allLoadedTools.push(...actionTools);
|
||||
}
|
||||
|
||||
if (isPTC && allLoadedTools.length > 0) {
|
||||
const ptcToolMap = new Map();
|
||||
for (const tool of allLoadedTools) {
|
||||
if (tool.name && tool.name !== Constants.PROGRAMMATIC_TOOL_CALLING) {
|
||||
ptcToolMap.set(tool.name, tool);
|
||||
}
|
||||
}
|
||||
configurable.ptcToolMap = ptcToolMap;
|
||||
}
|
||||
|
||||
return {
|
||||
configurable,
|
||||
loadedTools: allLoadedTools,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Loads action tools for event-driven execution.
|
||||
* @param {Object} params
|
||||
* @param {ServerRequest} params.req - The request object
|
||||
* @param {ServerResponse} params.res - The response object
|
||||
* @param {Object} params.agent - The agent object
|
||||
* @param {Object} params.appConfig - App configuration
|
||||
* @param {string|null} params.streamId - Stream ID
|
||||
* @param {string[]} params.actionToolNames - Action tool names to load
|
||||
* @returns {Promise<Array>} Loaded action tools
|
||||
*/
|
||||
async function loadActionToolsForExecution({
|
||||
req,
|
||||
res,
|
||||
agent,
|
||||
appConfig,
|
||||
streamId,
|
||||
actionToolNames,
|
||||
}) {
|
||||
const loadedActionTools = [];
|
||||
|
||||
const actionSets = (await loadActionSets({ agent_id: agent.id })) ?? [];
|
||||
if (actionSets.length === 0) {
|
||||
return loadedActionTools;
|
||||
}
|
||||
|
||||
const processedActionSets = new Map();
|
||||
const domainMap = new Map();
|
||||
const allowedDomains = appConfig?.actions?.allowedDomains;
|
||||
|
||||
for (const action of actionSets) {
|
||||
const domain = await domainParser(action.metadata.domain, true);
|
||||
domainMap.set(domain, action);
|
||||
|
||||
const isDomainAllowed = await isActionDomainAllowed(action.metadata.domain, allowedDomains);
|
||||
if (!isDomainAllowed) {
|
||||
logger.warn(
|
||||
`[Actions] Domain "${action.metadata.domain}" not in allowedDomains. ` +
|
||||
`Add it to librechat.yaml actions.allowedDomains to enable this action.`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
const validationResult = validateAndParseOpenAPISpec(action.metadata.raw_spec);
|
||||
if (!validationResult.spec || !validationResult.serverUrl) {
|
||||
logger.warn(`[Actions] Invalid OpenAPI spec for domain: ${domain}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
const domainValidation = validateActionDomain(
|
||||
action.metadata.domain,
|
||||
validationResult.serverUrl,
|
||||
);
|
||||
if (!domainValidation.isValid) {
|
||||
logger.error(`Domain mismatch in stored action: ${domainValidation.message}`, {
|
||||
userId: req.user.id,
|
||||
agent_id: agent.id,
|
||||
action_id: action.action_id,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
const encrypted = {
|
||||
oauth_client_id: action.metadata.oauth_client_id,
|
||||
oauth_client_secret: action.metadata.oauth_client_secret,
|
||||
};
|
||||
|
||||
const decryptedAction = { ...action };
|
||||
decryptedAction.metadata = await decryptMetadata(action.metadata);
|
||||
|
||||
const { requestBuilders, functionSignatures, zodSchemas } = openapiToFunction(
|
||||
validationResult.spec,
|
||||
true,
|
||||
);
|
||||
|
||||
processedActionSets.set(domain, {
|
||||
action: decryptedAction,
|
||||
requestBuilders,
|
||||
functionSignatures,
|
||||
zodSchemas,
|
||||
encrypted,
|
||||
});
|
||||
}
|
||||
|
||||
for (const toolName of actionToolNames) {
|
||||
let currentDomain = '';
|
||||
for (const domain of domainMap.keys()) {
|
||||
const normalizedDomain = domain.replace(domainSeparatorRegex, '_');
|
||||
if (toolName.includes(normalizedDomain)) {
|
||||
currentDomain = domain;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (!currentDomain || !processedActionSets.has(currentDomain)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const { action, encrypted, zodSchemas, requestBuilders, functionSignatures } =
|
||||
processedActionSets.get(currentDomain);
|
||||
const normalizedDomain = currentDomain.replace(domainSeparatorRegex, '_');
|
||||
const functionName = toolName.replace(`${actionDelimiter}${normalizedDomain}`, '');
|
||||
const functionSig = functionSignatures.find((sig) => sig.name === functionName);
|
||||
const requestBuilder = requestBuilders[functionName];
|
||||
const zodSchema = zodSchemas[functionName];
|
||||
|
||||
if (!requestBuilder) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const tool = await createActionTool({
|
||||
userId: req.user.id,
|
||||
res,
|
||||
action,
|
||||
streamId,
|
||||
zodSchema,
|
||||
encrypted,
|
||||
requestBuilder,
|
||||
name: toolName,
|
||||
description: functionSig?.description ?? '',
|
||||
});
|
||||
|
||||
if (!tool) {
|
||||
logger.warn(`[Actions] Failed to create action tool: ${toolName}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
loadedActionTools.push(tool);
|
||||
}
|
||||
|
||||
return loadedActionTools;
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
loadTools,
|
||||
isBuiltInTool,
|
||||
getToolkitKey,
|
||||
loadAgentTools,
|
||||
loadToolsForExecution,
|
||||
processRequiredActions,
|
||||
};
|
||||
|
|
|
|||
|
|
@ -107,22 +107,33 @@ function loadAndFormatTools({ directory, adminFilter = [], adminIncluded = [] })
|
|||
}, {});
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks if a schema is a Zod schema by looking for the _def property
|
||||
* @param {unknown} schema - The schema to check
|
||||
* @returns {boolean} True if it's a Zod schema
|
||||
*/
|
||||
function isZodSchema(schema) {
|
||||
return schema && typeof schema === 'object' && '_def' in schema;
|
||||
}
|
||||
|
||||
/**
|
||||
* Formats a `StructuredTool` instance into a format that is compatible
|
||||
* with OpenAI's ChatCompletionFunctions. It uses the `zodToJsonSchema`
|
||||
* function to convert the schema of the `StructuredTool` into a JSON
|
||||
* schema, which is then used as the parameters for the OpenAI function.
|
||||
* If the schema is already a JSON schema, it is used directly.
|
||||
*
|
||||
* @param {StructuredTool} tool - The StructuredTool to format.
|
||||
* @returns {FunctionTool} The OpenAI Assistant Tool.
|
||||
*/
|
||||
function formatToOpenAIAssistantTool(tool) {
|
||||
const parameters = isZodSchema(tool.schema) ? zodToJsonSchema(tool.schema) : tool.schema;
|
||||
return {
|
||||
type: Tools.function,
|
||||
[Tools.function]: {
|
||||
name: tool.name,
|
||||
description: tool.description,
|
||||
parameters: zodToJsonSchema(tool.schema),
|
||||
parameters,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue