🗝️ feat: Credential Variables for DB-Sourced MCP Servers (#12044)

* feat: Allow Credential Variables in Headers for DB-sourced MCP Servers

- Removed the hasCustomUserVars check from ToolService.js, directly retrieving userMCPAuthMap.
- Updated MCPConnectionFactory and related classes to include a dbSourced flag for better handling of database-sourced configurations.
- Added integration tests to ensure proper behavior of dbSourced servers, verifying that sensitive placeholders are not resolved while allowing customUserVars.
- Adjusted various MCP-related files to accommodate the new dbSourced logic, ensuring consistent handling across the codebase.

* chore: MCPConnectionFactory Tests with Additional Flow Metadata for typing

- Updated MCPConnectionFactory tests to include new fields in flowMetadata: serverUrl and state.
- Enhanced mockFlowData in multiple test cases to reflect the updated structure, ensuring comprehensive coverage of the OAuth flow scenarios.
- Added authorization_endpoint to metadata in the test setup for improved validation of the OAuth process.

* refactor: Simplify MCPManager Configuration Handling

- Removed unnecessary type assertions and streamlined the retrieval of server configuration in MCPManager.
- Enhanced the handling of OAuth and database-sourced flags for improved clarity and efficiency.
- Updated tests to reflect changes in user object structure and ensure proper processing of MCP environment variables.

* refactor: Optimize User MCP Auth Map Retrieval in ToolService

- Introduced conditional loading of userMCPAuthMap based on the presence of MCP-delimited tools, improving efficiency by avoiding unnecessary calls.
- Updated the loadToolDefinitionsWrapper and loadAgentTools functions to reflect this change, enhancing overall performance and clarity.

* test: Add userMCPAuthMap gating tests in ToolService

- Introduced new tests to validate the logic for determining if MCP tools are present in the agent's tool list.
- Implemented various scenarios to ensure accurate detection of MCP tools, including edge cases for empty, undefined, and null tool lists.
- Enhanced clarity and coverage of the ToolService capability checking logic.

* refactor: Enhance MCP Environment Variable Processing

- Simplified the handling of the dbSourced parameter in the processMCPEnv function.
- Introduced a failsafe mechanism to derive dbSourced from options if not explicitly provided, improving robustness and clarity in MCP environment variable processing.

* refactor: Update Regex Patterns for Credential Placeholders in ServerConfigsDB

- Modified regex patterns to include additional credential/env placeholders that should not be allowed in user-provided configurations.
- Clarified comments to emphasize the security risks associated with credential exfiltration when MCP servers are shared between users.

* chore: field order

* refactor: Clean Up dbSourced Parameter Handling in processMCPEnv

- Reintroduced the failsafe mechanism for deriving the dbSourced parameter from options, ensuring clarity and robustness in MCP environment variable processing.
- Enhanced code readability by maintaining consistent comment structure.

* refactor: Update MCPOptions Type to Include Optional dbId

- Modified the processMCPEnv function to extend the MCPOptions type, allowing for an optional dbId property.
- Simplified the logic for deriving the dbSourced parameter by directly checking the dbId property, enhancing code clarity and maintainability.

packages/api/src/mcp/registry/MCPServerInspector.ts

@@ -59,8 +59,9 @@ export class MCPServerInspector {
       if (!this.connection) {
         tempConnection = true;
         this.connection = await MCPConnectionFactory.create({
-          serverName: this.serverName,
           serverConfig: this.config,
+          serverName: this.serverName,
+          dbSourced: !!this.config.dbId,
           useSSRFProtection: this.useSSRFProtection,
         });
       }