🦥 refactor: Event-Driven Lazy Tool Loading (#11588)

* refactor: json schema tools with lazy loading

- Added LocalToolExecutor class for lazy loading and caching of tools during execution.
- Introduced ToolExecutionContext and ToolExecutor interfaces for better type management.
- Created utility functions to generate tool proxies with JSON schema support.
- Added ExtendedJsonSchema type for enhanced schema definitions.
- Updated existing toolkits to utilize the new schema and executor functionalities.
- Introduced a comprehensive tool definitions registry for managing various tool schemas.

chore: update @librechat/agents to version 3.1.2

refactor: enhance tool loading optimization and classification

- Improved the loadAgentToolsOptimized function to utilize a proxy pattern for all tools, enabling deferred execution and reducing overhead.
- Introduced caching for tool instances and refined tool classification logic to streamline tool management.
- Updated the handling of MCP tools to improve logging and error reporting for missing tools in the cache.
- Enhanced the structure of tool definitions to support better classification and integration with existing tools.

refactor: modularize tool loading and enhance optimization

- Moved the loadAgentToolsOptimized function to a new service file for better organization and maintainability.
- Updated the ToolService to utilize the new service for optimized tool loading, improving code clarity.
- Removed legacy tool loading methods and streamlined the tool loading process to enhance performance and reduce complexity.
- Introduced feature flag handling for optimized tool loading, allowing for easier toggling of this functionality.

refactor: replace loadAgentToolsWithFlag with loadAgentTools in tool loader

refactor: enhance MCP tool loading with proxy creation and classification

refactor: optimize MCP tool loading by grouping tools by server

- Introduced a Map to group cached tools by server name, improving the organization of tool data.
- Updated the createMCPProxyTool function to accept server name directly, enhancing clarity.
- Refactored the logic for handling MCP tools, streamlining the process of creating proxy tools for classification.

refactor: enhance MCP tool loading and proxy creation

- Added functionality to retrieve MCP server tools and reinitialize servers if necessary, improving tool availability.
- Updated the tool loading logic to utilize a Map for organizing tools by server, enhancing clarity and performance.
- Refactored the createToolProxy function to ensure a default response format, streamlining tool creation.

refactor: update createToolProxy to ensure consistent response format

- Modified the createToolProxy function to await the executor's execution and validate the result format.
- Ensured that the function returns a default response structure when the result is not an array of two elements, enhancing reliability in tool proxy creation.

refactor: ToolExecutionContext with toolCall property

- Added toolCall property to ToolExecutionContext interface for improved context handling during tool execution.
- Updated LocalToolExecutor to include toolCall in the runnable configuration, allowing for more flexible tool invocation.
- Modified createToolProxy to pass toolCall from the configuration, ensuring consistent context across tool executions.

refactor: enhance event-driven tool execution and logging

- Introduced ToolExecuteOptions for improved handling of event-driven tool execution, allowing for parallel execution of tool calls.
- Updated getDefaultHandlers to include support for ON_TOOL_EXECUTE events, enhancing the flexibility of tool invocation.
- Added detailed logging in LocalToolExecutor to track tool loading and execution metrics, improving observability and debugging capabilities.
- Refactored initializeClient to integrate event-driven tool loading, ensuring compatibility with the new execution model.

chore: update @librechat/agents to version 3.1.21

refactor: remove legacy tool loading and executor components

- Eliminated the loadAgentToolsWithFlag function, simplifying the tool loading process by directly using loadAgentTools.
- Removed the LocalToolExecutor and related executor components to streamline the tool execution architecture.
- Updated ToolService and related files to reflect the removal of deprecated features, enhancing code clarity and maintainability.

refactor: enhance tool classification and definitions handling

- Updated the loadAgentTools function to return toolDefinitions alongside toolRegistry, improving the structure of tool data returned to clients.
- Removed the convertRegistryToDefinitions function from the initialize.js file, simplifying the initialization process.
- Adjusted the buildToolClassification function to ensure toolDefinitions are built and returned simultaneously with the toolRegistry, enhancing efficiency in tool management.
- Updated type definitions in initialize.ts to include toolDefinitions, ensuring consistency across the codebase.

refactor: implement event-driven tool execution handler

- Introduced createToolExecuteHandler function to streamline the handling of ON_TOOL_EXECUTE events, allowing for parallel execution of tool calls.
- Updated getDefaultHandlers to utilize the new handler, simplifying the event-driven architecture.
- Added handlers.ts file to encapsulate tool execution logic, improving code organization and maintainability.
- Enhanced OpenAI handlers to integrate the new tool execution capabilities, ensuring consistent event handling across the application.

refactor: integrate event-driven tool execution options

- Added toolExecuteOptions to support event-driven tool execution in OpenAI and responses controllers, enhancing flexibility in tool handling.
- Updated handlers to utilize createToolExecuteHandler, allowing for streamlined execution of tools during agent interactions.
- Refactored service dependencies to include toolExecuteOptions, ensuring consistent integration across the application.

refactor: enhance tool loading with definitionsOnly parameter

- Updated createToolLoader and loadAgentTools functions to include a definitionsOnly parameter, allowing for the retrieval of only serializable tool definitions in event-driven mode.
- Adjusted related interfaces and documentation to reflect the new parameter, improving clarity and flexibility in tool management.
- Ensured compatibility across various components by integrating the definitionsOnly option in the initialization process.

refactor: improve agent tool presence check in initialization

- Added a check for tool presence using a new hasAgentTools variable, which evaluates both structuredTools and toolDefinitions.
- Updated the conditional logic in the agent initialization process to utilize the hasAgentTools variable, enhancing clarity and maintainability in tool management.

refactor: enhance agent tool extraction to support tool definitions

- Updated the extractMCPServers function to handle both tool instances and serializable tool definitions, improving flexibility in agent tool management.
- Added a new property toolDefinitions to the AgentWithTools type for better integration of event-driven mode.
- Enhanced documentation to clarify the function's capabilities in extracting unique MCP server names from both tools and tool definitions.

refactor: enhance tool classification and registry building

- Added serverName property to ToolDefinition for improved tool identification.
- Introduced buildToolRegistry function to streamline the creation of tool registries based on MCP tool definitions and agent options.
- Updated buildToolClassification to utilize the new registry building logic, ensuring basic definitions are returned even when advanced classification features are not allowed.
- Enhanced documentation and logging for clarity in tool classification processes.

refactor: update @librechat/agents dependency to version 3.1.22

fix: expose loadTools function in ToolService

- Added loadTools function to the exported module in ToolService.js, enhancing the accessibility of tool loading functionality.

chore: remove configurable options from tool execute options in OpenAI controller

refactor: enhance tool loading mechanism to utilize agent-specific context

chore: update @librechat/agents dependency to version 3.1.23

fix: simplify result handling in createToolExecuteHandler

* refactor: loadToolDefinitions for efficient tool loading in event-driven mode

* refactor: replace legacy tool loading with loadToolsForExecution in OpenAI and responses controllers

- Updated OpenAIChatCompletionController and createResponse functions to utilize loadToolsForExecution for improved tool loading.
- Removed deprecated loadToolsLegacy references, streamlining the tool execution process.
- Enhanced tool loading options to include agent-specific context and configurations.

* refactor: enhance tool loading and execution handling

- Introduced loadActionToolsForExecution function to streamline loading of action tools, improving organization and maintainability.
- Updated loadToolsForExecution to handle both regular and action tools, optimizing the tool loading process.
- Added detailed logging for missing tools in createToolExecuteHandler, enhancing error visibility.
- Refactored tool definitions to normalize action tool names, improving consistency in tool management.

* refactor: enhance built-in tool definitions loading

- Updated loadToolDefinitions to include descriptions and parameters from the tool registry for built-in tools, improving the clarity and usability of tool definitions.
- Integrated getToolDefinition to streamline the retrieval of tool metadata, enhancing the overall tool management process.

* feat: add action tool definitions loading to tool service

- Introduced getActionToolDefinitions function to load action tool definitions based on agent ID and tool names, enhancing the tool loading process.
- Updated loadToolDefinitions to integrate action tool definitions, allowing for better management and retrieval of action-specific tools.
- Added comprehensive tests for action tool definitions to ensure correct loading and parameter handling, improving overall reliability and functionality.

* chore: update @librechat/agents dependency to version 3.1.26

* refactor: add toolEndCallback to handle tool execution results

* fix: tool definitions and execution handling

- Introduced native tools (execute_code, file_search, web_search) to the tool service, allowing for better integration and management of these tools.
- Updated isBuiltInTool function to include native tools in the built-in check, improving tool recognition.
- Added comprehensive tests for loading parameters of native tools, ensuring correct functionality and parameter handling.
- Enhanced tool definitions registry to include new agent tool definitions, streamlining tool retrieval and management.

* refactor: enhance tool loading and execution context

- Added toolRegistry to the context for OpenAIChatCompletionController and createResponse functions, improving tool management.
- Updated loadToolsForExecution to utilize toolRegistry for better integration of programmatic tools and tool search functionalities.
- Enhanced the initialization process to include toolRegistry in agent context, streamlining tool access and configuration.
- Refactored tool classification logic to support event-driven execution, ensuring compatibility with new tool definitions.

* chore: add request duration logging to OpenAI and Responses controllers

- Introduced logging for request start and completion times in OpenAIChatCompletionController and createResponse functions.
- Calculated and logged the duration of each request, enhancing observability and performance tracking.
- Improved debugging capabilities by providing detailed logs for both streaming and non-streaming responses.

* chore: update @librechat/agents dependency to version 3.1.27

* refactor: implement buildToolSet function for tool management

- Introduced buildToolSet function to streamline the creation of tool sets from agent configurations, enhancing tool management across various controllers.
- Updated AgentClient, OpenAIChatCompletionController, and createResponse functions to utilize buildToolSet, improving consistency in tool handling.
- Added comprehensive tests for buildToolSet to ensure correct functionality and edge case handling, enhancing overall reliability.

* refactor: update import paths for ToolExecuteOptions and createToolExecuteHandler

* fix: update GoogleSearch.js description for maximum search results

- Changed the default maximum number of search results from 10 to 5 in the Google Search JSON schema description, ensuring accurate documentation of the expected behavior.

* chore: remove deprecated Browser tool and associated assets

- Deleted the Browser tool definition from manifest.json, which included its name, plugin key, description, and authentication configuration.
- Removed the web-browser.svg asset as it is no longer needed following the removal of the Browser tool.

* fix: ensure tool definitions are valid before processing

- Added a check to verify the existence of tool definitions in the registry before accessing their properties, preventing potential runtime errors.
- Updated the loading logic for built-in tool definitions to ensure that only valid definitions are pushed to the built-in tool definitions array.

* fix: extend ExtendedJsonSchema to support 'null' type and nullable enums

- Updated the ExtendedJsonSchema type to include 'null' as a valid type option.
- Modified the enum property to accept an array of values that can include strings, numbers, booleans, and null, enhancing schema flexibility.

* test: add comprehensive tests for tool definitions loading and registry behavior

- Implemented tests to verify the handling of built-in tools without registry definitions, ensuring they are skipped correctly.
- Added tests to confirm that built-in tools include descriptions and parameters in the registry.
- Enhanced tests for action tools, checking for proper inclusion of metadata and handling of tools without parameters in the registry.

* test: add tests for mixed-type and number enum schema handling

- Introduced tests to validate the parsing of mixed-type enum values, including strings, numbers, booleans, and null.
- Added tests for number enum schema values to ensure correct parsing of numeric inputs, enhancing schema validation coverage.

* fix: update mock implementation for @librechat/agents

- Changed the mock for @librechat/agents to spread the actual module's properties, ensuring that all necessary functionalities are preserved in tests.
- This adjustment enhances the accuracy of the tests by reflecting the real structure of the module.

* fix: change max_results type in GoogleSearch schema from number to integer

- Updated the type of max_results in the Google Search JSON schema to 'integer' for better type accuracy and validation consistency.

* fix: update max_results description and type in GoogleSearch schema

- Changed the type of max_results from 'number' to 'integer' for improved type accuracy.
- Updated the description to reflect the new default maximum number of search results, changing it from 10 to 5.

* refactor: remove unused code and improve tool registry handling

- Eliminated outdated comments and conditional logic related to event-driven mode in the ToolService.
- Enhanced the handling of the tool registry by ensuring it is configurable for better integration during tool execution.

* feat: add definitionsOnly option to buildToolClassification for event-driven mode

- Introduced a new parameter, definitionsOnly, to the BuildToolClassificationParams interface to enable a mode that skips tool instance creation.
- Updated the buildToolClassification function to conditionally add tool definitions without instantiating tools when definitionsOnly is true.
- Modified the loadToolDefinitions function to pass definitionsOnly as true, ensuring compatibility with the new feature.

* test: add unit tests for buildToolClassification with definitionsOnly option

- Implemented tests to verify the behavior of buildToolClassification when definitionsOnly is set to true or false.
- Ensured that tool instances are not created when definitionsOnly is true, while still adding necessary tool definitions.
- Confirmed that loadAuthValues is called appropriately based on the definitionsOnly parameter, enhancing test coverage for this new feature.
This commit is contained in:
Danny Avila 2026-02-01 08:50:57 -05:00 committed by GitHub
parent 6279ea8dd7
commit 5af1342dbb
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
46 changed files with 3297 additions and 565 deletions

View file

@ -26,7 +26,13 @@
import { logger } from '@librechat/data-schemas';
import { Constants } from 'librechat-data-provider';
import { EnvVar, createProgrammaticToolCallingTool, createToolSearch } from '@librechat/agents';
import {
EnvVar,
createToolSearch,
ToolSearchToolDefinition,
createProgrammaticToolCallingTool,
ProgrammaticToolCallingDefinition,
} from '@librechat/agents';
import type { AgentToolOptions } from 'librechat-data-provider';
import type {
LCToolRegistry,
@ -45,6 +51,8 @@ export interface ToolDefinition {
name: string;
description?: string;
parameters?: JsonSchemaType;
/** MCP server name extracted from tool name */
serverName?: string;
}
/**
@ -286,6 +294,12 @@ export function extractMCPToolDefinition(tool: MCPToolInstance): ToolDefinition
def.parameters = tool.mcpJsonSchema;
}
/** Extract server name from tool name (format: toolName_mcp_ServerName) */
const serverName = getServerNameFromTool(tool.name);
if (serverName) {
def.serverName = serverName;
}
return def;
}
@ -312,6 +326,36 @@ export function cleanupMCPToolSchemas(tools: MCPToolInstance[]): void {
}
}
/**
* Builds tool registry from MCP tool definitions using the appropriate strategy.
* Uses early returns to avoid nesting (Torvalds principle).
*/
function buildToolRegistry(
mcpToolDefs: ToolDefinition[],
agentToolOptions?: AgentToolOptions,
): LCToolRegistry {
if (agentToolOptions && Object.keys(agentToolOptions).length > 0) {
return buildToolRegistryFromAgentOptions(mcpToolDefs, agentToolOptions);
}
if (process.env.TOOL_CLASSIFICATION_FROM_ENV === 'true') {
return buildToolRegistryFromEnv(mcpToolDefs);
}
/** No classification config - build basic definitions for event-driven mode */
const registry: LCToolRegistry = new Map<string, LCTool>();
for (const toolDef of mcpToolDefs) {
registry.set(toolDef.name, {
name: toolDef.name,
description: toolDef.description,
parameters: toolDef.parameters,
serverName: toolDef.serverName,
toolType: 'mcp',
});
}
return registry;
}
/** Parameters for building tool classification and creating PTC/tool search tools */
export interface BuildToolClassificationParams {
/** All loaded tools (will be filtered for MCP tools) */
@ -324,6 +368,8 @@ export interface BuildToolClassificationParams {
agentToolOptions?: AgentToolOptions;
/** Whether the deferred_tools capability is enabled (from agent config) */
deferredToolsEnabled?: boolean;
/** When true, skip creating tool instances (for event-driven mode) */
definitionsOnly?: boolean;
/** Function to load auth values (dependency injection) */
loadAuthValues: (params: {
userId: string;
@ -335,6 +381,8 @@ export interface BuildToolClassificationParams {
export interface BuildToolClassificationResult {
/** Tool registry built from MCP tools (undefined if no MCP tools) */
toolRegistry?: LCToolRegistry;
/** Tool definitions array for event-driven execution (built simultaneously with registry) */
toolDefinitions: LCTool[];
/** Additional tools created (PTC and/or tool search) */
additionalTools: GenericTool[];
/** Whether any tools have defer_loading enabled (precomputed for efficiency) */
@ -407,26 +455,35 @@ export async function buildToolClassification(
params: BuildToolClassificationParams,
): Promise<BuildToolClassificationResult> {
const {
loadedTools,
userId,
agentId,
loadedTools,
agentToolOptions,
definitionsOnly = false,
deferredToolsEnabled = true,
loadAuthValues,
} = params;
const additionalTools: GenericTool[] = [];
/** Check if this agent is allowed to have classification features (requires agentId) */
if (!isAgentAllowedForClassification(agentId)) {
logger.debug(
`[buildToolClassification] Agent ${agentId ?? 'undefined'} not allowed for classification, skipping`,
);
return { toolRegistry: undefined, additionalTools, hasDeferredTools: false };
}
const mcpTools = loadedTools.filter(isMCPTool);
if (mcpTools.length === 0) {
return { toolRegistry: undefined, additionalTools, hasDeferredTools: false };
return {
additionalTools,
toolDefinitions: [],
toolRegistry: undefined,
hasDeferredTools: false,
};
}
/**
* Check if this agent is allowed to have advanced classification features (PTC, deferred tools).
* Even if not allowed, we still build basic tool definitions for event-driven execution.
*/
const isAllowedForClassification = isAgentAllowedForClassification(agentId);
if (!isAllowedForClassification) {
logger.debug(
`[buildToolClassification] Agent ${agentId ?? 'undefined'} not allowed for classification, building basic definitions only`,
);
}
const mcpToolDefs = mcpTools.map(extractMCPToolDefinition);
@ -435,17 +492,11 @@ export async function buildToolClassification(
* Build registry from agent's tool_options if provided (UI config).
* Environment variable-based classification is only used as fallback
* when TOOL_CLASSIFICATION_FROM_ENV=true is explicitly set.
*
* Even without classification config, we still build basic tool definitions
* for event-driven execution.
*/
let toolRegistry: LCToolRegistry | undefined;
if (agentToolOptions && Object.keys(agentToolOptions).length > 0) {
toolRegistry = buildToolRegistryFromAgentOptions(mcpToolDefs, agentToolOptions);
} else if (process.env.TOOL_CLASSIFICATION_FROM_ENV === 'true') {
toolRegistry = buildToolRegistryFromEnv(mcpToolDefs);
} else {
/** No agent-level config and env-based classification not enabled */
return { toolRegistry: undefined, additionalTools, hasDeferredTools: false };
}
const toolRegistry: LCToolRegistry = buildToolRegistry(mcpToolDefs, agentToolOptions);
/** Clean up temporary mcpJsonSchema property from tools now that registry is populated */
cleanupMCPToolSchemas(mcpTools);
@ -458,55 +509,111 @@ export async function buildToolClassification(
const hasProgrammaticTools = agentHasProgrammaticTools(toolRegistry);
const hasDeferredTools = deferredToolsEnabled && agentHasDeferredTools(toolRegistry);
/**
* If deferred tools capability is disabled, clear defer_loading from all tools
* to ensure no tools are treated as deferred at runtime.
*/
/** Clear defer_loading if capability disabled */
if (!deferredToolsEnabled) {
for (const toolDef of toolRegistry.values()) {
if (toolDef.defer_loading === true) {
toolDef.defer_loading = false;
if (toolDef.defer_loading !== true) {
continue;
}
toolDef.defer_loading = false;
}
}
/** Build toolDefinitions array from registry (single pass, reused) */
const toolDefinitions: LCTool[] = Array.from(toolRegistry.values());
/** Agent not allowed for classification - return basic definitions */
if (!isAllowedForClassification) {
logger.debug(
`[buildToolClassification] Agent ${agentId} not allowed for classification, returning basic definitions`,
);
return { toolRegistry, toolDefinitions, additionalTools, hasDeferredTools: false };
}
/** No programmatic or deferred tools - skip PTC/ToolSearch */
if (!hasProgrammaticTools && !hasDeferredTools) {
logger.debug(
`[buildToolClassification] Agent ${agentId} has no programmatic or deferred tools, skipping PTC/ToolSearch`,
);
return { toolRegistry, additionalTools, hasDeferredTools: false };
return { toolRegistry, toolDefinitions, additionalTools, hasDeferredTools: false };
}
/** Tool search uses local mode (no API key needed) */
if (hasDeferredTools) {
const toolSearchTool = createToolSearch({
mode: 'local',
toolRegistry,
if (!definitionsOnly) {
const toolSearchTool = createToolSearch({
mode: 'local',
toolRegistry,
});
additionalTools.push(toolSearchTool);
}
/** Add ToolSearch definition for event-driven mode */
toolDefinitions.push({
name: ToolSearchToolDefinition.name,
description: ToolSearchToolDefinition.description,
parameters: ToolSearchToolDefinition.schema as unknown as LCTool['parameters'],
});
additionalTools.push(toolSearchTool);
toolRegistry.set(ToolSearchToolDefinition.name, {
name: ToolSearchToolDefinition.name,
allowed_callers: ['direct'],
});
logger.debug(`[buildToolClassification] Tool Search enabled for agent ${agentId}`);
}
/** PTC requires CODE_API_KEY for sandbox execution */
if (hasProgrammaticTools) {
try {
const authValues = await loadAuthValues({
userId,
authFields: [EnvVar.CODE_API_KEY],
});
const codeApiKey = authValues[EnvVar.CODE_API_KEY];
if (!codeApiKey) {
logger.warn('[buildToolClassification] PTC configured but CODE_API_KEY not available');
} else {
const ptcTool = createProgrammaticToolCallingTool({ apiKey: codeApiKey });
additionalTools.push(ptcTool);
logger.debug(`[buildToolClassification] PTC tool enabled for agent ${agentId}`);
}
} catch (error) {
logger.error('[buildToolClassification] Error creating PTC tool:', error);
}
if (!hasProgrammaticTools) {
return { toolRegistry, toolDefinitions, additionalTools, hasDeferredTools };
}
return { toolRegistry, additionalTools, hasDeferredTools };
/** In definitions-only mode, add PTC definition without creating the tool instance */
if (definitionsOnly) {
toolDefinitions.push({
name: ProgrammaticToolCallingDefinition.name,
description: ProgrammaticToolCallingDefinition.description,
parameters: ProgrammaticToolCallingDefinition.schema as unknown as LCTool['parameters'],
});
toolRegistry.set(ProgrammaticToolCallingDefinition.name, {
name: ProgrammaticToolCallingDefinition.name,
allowed_callers: ['direct'],
});
logger.debug(
`[buildToolClassification] PTC definition added for agent ${agentId} (definitions only)`,
);
return { toolRegistry, toolDefinitions, additionalTools, hasDeferredTools };
}
try {
const authValues = await loadAuthValues({
userId,
authFields: [EnvVar.CODE_API_KEY],
});
const codeApiKey = authValues[EnvVar.CODE_API_KEY];
if (!codeApiKey) {
logger.warn('[buildToolClassification] PTC configured but CODE_API_KEY not available');
return { toolRegistry, toolDefinitions, additionalTools, hasDeferredTools };
}
const ptcTool = createProgrammaticToolCallingTool({ apiKey: codeApiKey });
additionalTools.push(ptcTool);
/** Add PTC definition for event-driven mode */
toolDefinitions.push({
name: ProgrammaticToolCallingDefinition.name,
description: ProgrammaticToolCallingDefinition.description,
parameters: ProgrammaticToolCallingDefinition.schema as unknown as LCTool['parameters'],
});
toolRegistry.set(ProgrammaticToolCallingDefinition.name, {
name: ProgrammaticToolCallingDefinition.name,
allowed_callers: ['direct'],
});
logger.debug(`[buildToolClassification] PTC tool enabled for agent ${agentId}`);
} catch (error) {
logger.error('[buildToolClassification] Error creating PTC tool:', error);
}
return { toolRegistry, toolDefinitions, additionalTools, hasDeferredTools };
}