LibreChat/packages/api/src/mcp/oauth/tokens.ts
Danny Avila ec7370dfe9
🪐 feat: MCP OAuth 2.0 Discovery Support (#7924)
* chore: Update @modelcontextprotocol/sdk to version 1.12.3 in package.json and package-lock.json

- Bump version of @modelcontextprotocol/sdk to 1.12.3 to incorporate recent updates.
- Update dependencies for ajv and cross-spawn to their latest versions.
- Add ajv as a new dependency in the sdk module.
- Include json-schema-traverse as a new dependency in the sdk module.

* feat: @librechat/auth

* feat: Add crypto module exports to auth package

- Introduced a new crypto module by creating index.ts in the crypto directory.
- Updated the main index.ts of the auth package to export from the new crypto module.

* feat: Update package dependencies and build scripts for auth package

- Added @librechat/auth as a dependency in package.json and package-lock.json.
- Updated build scripts to include the auth package in both frontend and bun build processes.
- Removed unused mongoose and openid-client dependencies from package-lock.json for cleaner dependency management.

* refactor: Migrate crypto utility functions to @librechat/auth

- Replaced local crypto utility imports with the new @librechat/auth package across multiple files.
- Removed the obsolete crypto.js file and its exports.
- Updated relevant services and models to utilize the new encryption and decryption methods from @librechat/auth.

* feat: Enhance OAuth token handling and update dependencies in auth package

* chore: Remove Token model and TokenService due to restructuring of OAuth handling

- Deleted the Token.js model and TokenService.js, which were responsible for managing OAuth tokens.
- This change is part of a broader refactor to streamline OAuth token management and improve code organization.

* refactor: imports from '@librechat/auth' to '@librechat/api' and add OAuth token handling functionality

* refactor: Simplify logger usage in MCP and FlowStateManager classes

* chore: fix imports

* feat: Add OAuth configuration schema to MCP with token exchange method support

* feat: FIRST PASS Implement MCP OAuth flow with token management and error handling

- Added a new route for handling OAuth callbacks and token retrieval.
- Integrated OAuth token storage and retrieval mechanisms.
- Enhanced MCP connection to support automatic OAuth flow initiation on 401 errors.
- Implemented dynamic client registration and metadata discovery for OAuth.
- Updated MCPManager to manage OAuth tokens and handle authentication requirements.
- Introduced comprehensive logging for OAuth processes and error handling.

* refactor: Update MCPConnection and MCPManager to utilize new URL handling

- Added a `url` property to MCPConnection for better URL management.
- Refactored MCPManager to use the new `url` property instead of a deprecated method for OAuth handling.
- Changed logging from info to debug level for flow manager and token methods initialization.
- Improved comments for clarity on existing tokens and OAuth event listener setup.

* refactor: Improve connection timeout error messages in MCPConnection and MCPManager and use initTimeout for connection

- Updated the connection timeout error messages to include the duration of the timeout.
- Introduced a configurable `connectTimeout` variable in both MCPConnection and MCPManager for better flexibility.

* chore: cleanup MCP OAuth Token exchange handling; fix: erroneous use of flowsCache and remove verbose logs

* refactor: Update MCPManager and MCPTokenStorage to use TokenMethods for token management

- Removed direct token storage handling in MCPManager and replaced it with TokenMethods for better abstraction.
- Refactored MCPTokenStorage methods to accept parameters for token operations, enhancing flexibility and readability.
- Improved logging messages related to token persistence and retrieval processes.

* refactor: Update MCP OAuth handling to use static methods and improve flow management

- Refactored MCPOAuthHandler to utilize static methods for initiating and completing OAuth flows, enhancing clarity and reducing instance dependencies.
- Updated MCPManager to pass flowManager explicitly to OAuth handling methods, improving flexibility in flow state management.
- Enhanced comments and logging for better understanding of OAuth processes and flow state retrieval.

* refactor: Integrate token methods into createMCPTool for enhanced token management

* refactor: Change logging from info to debug level in MCPOAuthHandler for improved log management

* chore: clean up logging

* feat: first pass, auth URL from MCP OAuth flow

* chore: Improve logging format for OAuth authentication URL display

* chore: cleanup mcp manager comments

* feat: add connection reconnection logic in MCPManager

* refactor: reorganize token storage handling in MCP

- Moved token storage logic from MCPManager to a new MCPTokenStorage class for better separation of concerns.
- Updated imports to reflect the new token storage structure.
- Enhanced methods for storing, retrieving, updating, and deleting OAuth tokens, improving overall token management.

* chore: update comment for SYSTEM_USER_ID in MCPManager for clarity

* feat: implement refresh token functionality in MCP

- Added refresh token handling in MCPManager to support token renewal for both app-level and user-specific connections.
- Introduced a refreshTokens function to facilitate token refresh logic.
- Enhanced MCPTokenStorage to manage client information and refresh token processes.
- Updated logging for better traceability during token operations.

* chore: cleanup @librechat/auth

* feat: implement MCP server initialization in a separate service

- Added a new service to handle the initialization of MCP servers, improving code organization and readability.
- Refactored the server startup logic to utilize the new initializeMCP function.
- Removed redundant MCP initialization code from the main server file.

* fix: don't log auth url for user connections

* feat: enhance OAuth flow with success and error handling components

- Updated OAuth callback routes to redirect to new success and error pages instead of sending status messages.
- Introduced `OAuthSuccess` and `OAuthError` components to provide user feedback during authentication.
- Added localization support for success and error messages in the translation files.
- Implemented countdown functionality in the success component for a better user experience.

* fix: refresh token handling for user connections, add missing URL and methods

- add standard enum for system user id and helper for determining app-lvel vs. user-level connections

* refactor: update token handling in MCPManager and MCPTokenStorage

* fix: improve error logging in OAuth authentication handler

* fix: concurrency issues for both login url emission and concurrency of oauth flows for shared flows (same user, same server, multiple calls for same server)

* fix: properly fail shared flows for concurrent server calls and prevent duplication of tokens

* chore: remove unused auth package directory from update configuration

* ci: fix mocks in samlStrategy tests

* ci: add mcpConfig to AppService test setup

* chore: remove obsolete MCP OAuth implementation documentation

* fix: update build script for API to use correct command

* chore: bump version of @librechat/api to 1.2.4

* fix: update abort signal handling in createMCPTool function

* fix: add optional clientInfo parameter to refreshTokensFunction metadata

* refactor: replace app.locals.availableTools with getCachedTools in multiple services and controllers for improved tool management

* fix: concurrent refresh token handling issue

* refactor: add signal parameter to getUserConnection method for improved abort handling

* chore: JSDoc typing for `loadEphemeralAgent`

* refactor: update isConnectionActive method to use destructured parameters for improved readability

* feat: implement caching for MCP tools to handle app-level disconnects for loading list of tools

* ci: fix agent test
2025-06-17 13:50:33 -04:00

382 lines
14 KiB
TypeScript

import { logger } from '@librechat/data-schemas';
import type { OAuthTokens, OAuthClientInformation } from '@modelcontextprotocol/sdk/shared/auth.js';
import type { TokenMethods, IToken } from '@librechat/data-schemas';
import type { MCPOAuthTokens, ExtendedOAuthTokens } from './types';
import { encryptV2, decryptV2 } from '~/crypto';
import { isSystemUserId } from '~/mcp/enum';
interface StoreTokensParams {
userId: string;
serverName: string;
tokens: OAuthTokens | ExtendedOAuthTokens | MCPOAuthTokens;
createToken: TokenMethods['createToken'];
updateToken?: TokenMethods['updateToken'];
findToken?: TokenMethods['findToken'];
clientInfo?: OAuthClientInformation;
/** Optional: Pass existing token state to avoid duplicate DB calls */
existingTokens?: {
accessToken?: IToken | null;
refreshToken?: IToken | null;
clientInfoToken?: IToken | null;
};
}
interface GetTokensParams {
userId: string;
serverName: string;
findToken: TokenMethods['findToken'];
refreshTokens?: (
refreshToken: string,
metadata: { userId: string; serverName: string; identifier: string },
) => Promise<MCPOAuthTokens>;
createToken?: TokenMethods['createToken'];
updateToken?: TokenMethods['updateToken'];
}
export class MCPTokenStorage {
static getLogPrefix(userId: string, serverName: string): string {
return isSystemUserId(userId)
? `[MCP][${serverName}]`
: `[MCP][User: ${userId}][${serverName}]`;
}
/**
* Stores OAuth tokens for an MCP server
*
* @param params.existingTokens - Optional: Pass existing token state to avoid duplicate DB calls.
* This is useful when refreshing tokens, as getTokens() already has the token state.
*/
static async storeTokens({
userId,
serverName,
tokens,
createToken,
updateToken,
findToken,
clientInfo,
existingTokens,
}: StoreTokensParams): Promise<void> {
const logPrefix = this.getLogPrefix(userId, serverName);
try {
const identifier = `mcp:${serverName}`;
// Encrypt and store access token
const encryptedAccessToken = await encryptV2(tokens.access_token);
logger.debug(
`${logPrefix} Token expires_in: ${'expires_in' in tokens ? tokens.expires_in : 'N/A'}, expires_at: ${'expires_at' in tokens ? tokens.expires_at : 'N/A'}`,
);
// Handle both expires_in and expires_at formats
let accessTokenExpiry: Date;
if ('expires_at' in tokens && tokens.expires_at) {
/** MCPOAuthTokens format - already has calculated expiry */
logger.debug(`${logPrefix} Using expires_at: ${tokens.expires_at}`);
accessTokenExpiry = new Date(tokens.expires_at);
} else if (tokens.expires_in) {
/** Standard OAuthTokens format - calculate expiry */
logger.debug(`${logPrefix} Using expires_in: ${tokens.expires_in}`);
accessTokenExpiry = new Date(Date.now() + tokens.expires_in * 1000);
} else {
/** No expiry provided - default to 1 year */
logger.debug(`${logPrefix} No expiry provided, using default`);
accessTokenExpiry = new Date(Date.now() + 365 * 24 * 60 * 60 * 1000);
}
logger.debug(`${logPrefix} Calculated expiry date: ${accessTokenExpiry.toISOString()}`);
logger.debug(
`${logPrefix} Date object: ${JSON.stringify({
time: accessTokenExpiry.getTime(),
valid: !isNaN(accessTokenExpiry.getTime()),
iso: accessTokenExpiry.toISOString(),
})}`,
);
// Ensure the date is valid before passing to createToken
if (isNaN(accessTokenExpiry.getTime())) {
logger.error(`${logPrefix} Invalid expiry date calculated, using default`);
accessTokenExpiry = new Date(Date.now() + 365 * 24 * 60 * 60 * 1000);
}
// Calculate expiresIn (seconds from now)
const expiresIn = Math.floor((accessTokenExpiry.getTime() - Date.now()) / 1000);
const accessTokenData = {
userId,
type: 'mcp_oauth',
identifier,
token: encryptedAccessToken,
expiresIn: expiresIn > 0 ? expiresIn : 365 * 24 * 60 * 60, // Default to 1 year if negative
};
// Check if token already exists and update if it does
if (findToken && updateToken) {
// Use provided existing token state if available, otherwise look it up
const existingToken =
existingTokens?.accessToken !== undefined
? existingTokens.accessToken
: await findToken({ userId, identifier });
if (existingToken) {
await updateToken({ userId, identifier }, accessTokenData);
logger.debug(`${logPrefix} Updated existing access token`);
} else {
await createToken(accessTokenData);
logger.debug(`${logPrefix} Created new access token`);
}
} else {
// Create new token if it's initial store or update methods not provided
await createToken(accessTokenData);
logger.debug(`${logPrefix} Created access token (no update methods available)`);
}
// Store refresh token if available
if (tokens.refresh_token) {
const encryptedRefreshToken = await encryptV2(tokens.refresh_token);
const extendedTokens = tokens as ExtendedOAuthTokens;
const refreshTokenExpiry = extendedTokens.refresh_token_expires_in
? new Date(Date.now() + extendedTokens.refresh_token_expires_in * 1000)
: new Date(Date.now() + 365 * 24 * 60 * 60 * 1000); // Default to 1 year
/** Calculated expiresIn for refresh token */
const refreshExpiresIn = Math.floor((refreshTokenExpiry.getTime() - Date.now()) / 1000);
const refreshTokenData = {
userId,
type: 'mcp_oauth_refresh',
identifier: `${identifier}:refresh`,
token: encryptedRefreshToken,
expiresIn: refreshExpiresIn > 0 ? refreshExpiresIn : 365 * 24 * 60 * 60,
};
// Check if refresh token already exists and update if it does
if (findToken && updateToken) {
// Use provided existing token state if available, otherwise look it up
const existingRefreshToken =
existingTokens?.refreshToken !== undefined
? existingTokens.refreshToken
: await findToken({
userId,
identifier: `${identifier}:refresh`,
});
if (existingRefreshToken) {
await updateToken({ userId, identifier: `${identifier}:refresh` }, refreshTokenData);
logger.debug(`${logPrefix} Updated existing refresh token`);
} else {
await createToken(refreshTokenData);
logger.debug(`${logPrefix} Created new refresh token`);
}
} else {
await createToken(refreshTokenData);
logger.debug(`${logPrefix} Created refresh token (no update methods available)`);
}
}
/** Store client information if provided */
if (clientInfo) {
logger.debug(`${logPrefix} Storing client info:`, {
client_id: clientInfo.client_id,
has_client_secret: !!clientInfo.client_secret,
});
const encryptedClientInfo = await encryptV2(JSON.stringify(clientInfo));
const clientInfoData = {
userId,
type: 'mcp_oauth_client',
identifier: `${identifier}:client`,
token: encryptedClientInfo,
expiresIn: 365 * 24 * 60 * 60,
};
// Check if client info already exists and update if it does
if (findToken && updateToken) {
// Use provided existing token state if available, otherwise look it up
const existingClientInfo =
existingTokens?.clientInfoToken !== undefined
? existingTokens.clientInfoToken
: await findToken({
userId,
identifier: `${identifier}:client`,
});
if (existingClientInfo) {
await updateToken({ userId, identifier: `${identifier}:client` }, clientInfoData);
logger.debug(`${logPrefix} Updated existing client info`);
} else {
await createToken(clientInfoData);
logger.debug(`${logPrefix} Created new client info`);
}
} else {
await createToken(clientInfoData);
logger.debug(`${logPrefix} Created client info (no update methods available)`);
}
}
logger.debug(`${logPrefix} Stored OAuth tokens`);
} catch (error) {
const logPrefix = this.getLogPrefix(userId, serverName);
logger.error(`${logPrefix} Failed to store tokens`, error);
throw error;
}
}
/**
* Retrieves OAuth tokens for an MCP server
*/
static async getTokens({
userId,
serverName,
findToken,
createToken,
updateToken,
refreshTokens,
}: GetTokensParams): Promise<MCPOAuthTokens | null> {
const logPrefix = this.getLogPrefix(userId, serverName);
try {
const identifier = `mcp:${serverName}`;
// Get access token
const accessTokenData = await findToken({
userId,
type: 'mcp_oauth',
identifier,
});
/** Check if access token is missing or expired */
const isMissing = !accessTokenData;
const isExpired = accessTokenData?.expiresAt && new Date() >= accessTokenData.expiresAt;
if (isMissing || isExpired) {
logger.info(`${logPrefix} Access token ${isMissing ? 'missing' : 'expired'}`);
/** Refresh data if we have a refresh token and refresh function */
const refreshTokenData = await findToken({
userId,
type: 'mcp_oauth_refresh',
identifier: `${identifier}:refresh`,
});
if (!refreshTokenData) {
logger.info(
`${logPrefix} Access token ${isMissing ? 'missing' : 'expired'} and no refresh token available`,
);
return null;
}
if (!refreshTokens) {
logger.warn(
`${logPrefix} Access token ${isMissing ? 'missing' : 'expired'}, refresh token available but no \`refreshTokens\` provided`,
);
return null;
}
if (!createToken) {
logger.warn(
`${logPrefix} Access token ${isMissing ? 'missing' : 'expired'}, refresh token available but no \`createToken\` function provided`,
);
return null;
}
try {
logger.info(`${logPrefix} Attempting to refresh token`);
const decryptedRefreshToken = await decryptV2(refreshTokenData.token);
/** Client information if available */
let clientInfo;
let clientInfoData;
try {
clientInfoData = await findToken({
userId,
type: 'mcp_oauth_client',
identifier: `${identifier}:client`,
});
if (clientInfoData) {
const decryptedClientInfo = await decryptV2(clientInfoData.token);
clientInfo = JSON.parse(decryptedClientInfo);
logger.debug(`${logPrefix} Retrieved client info:`, {
client_id: clientInfo.client_id,
has_client_secret: !!clientInfo.client_secret,
});
}
} catch {
logger.debug(`${logPrefix} No client info found`);
}
const metadata = {
userId,
serverName,
identifier,
clientInfo,
};
const newTokens = await refreshTokens(decryptedRefreshToken, metadata);
// Store the refreshed tokens (handles both create and update)
// Pass existing token state to avoid duplicate DB calls
await this.storeTokens({
userId,
serverName,
tokens: newTokens,
createToken,
updateToken,
findToken,
clientInfo,
existingTokens: {
accessToken: accessTokenData, // We know this is expired/missing
refreshToken: refreshTokenData, // We already have this
clientInfoToken: clientInfoData, // We already looked this up
},
});
logger.info(`${logPrefix} Successfully refreshed and stored OAuth tokens`);
return newTokens;
} catch (refreshError) {
logger.error(`${logPrefix} Failed to refresh tokens`, refreshError);
// Check if it's an unauthorized_client error (refresh not supported)
const errorMessage =
refreshError instanceof Error ? refreshError.message : String(refreshError);
if (errorMessage.includes('unauthorized_client')) {
logger.info(
`${logPrefix} Server does not support refresh tokens for this client. New authentication required.`,
);
}
return null;
}
}
// If we reach here, access token should exist and be valid
if (!accessTokenData) {
return null;
}
const decryptedAccessToken = await decryptV2(accessTokenData.token);
/** Get refresh token if available */
const refreshTokenData = await findToken({
userId,
type: 'mcp_oauth_refresh',
identifier: `${identifier}:refresh`,
});
const tokens: MCPOAuthTokens = {
access_token: decryptedAccessToken,
token_type: 'Bearer',
obtained_at: accessTokenData.createdAt.getTime(),
expires_at: accessTokenData.expiresAt?.getTime(),
};
if (refreshTokenData) {
tokens.refresh_token = await decryptV2(refreshTokenData.token);
}
logger.debug(`${logPrefix} Loaded existing OAuth tokens from storage`);
return tokens;
} catch (error) {
logger.error(`${logPrefix} Failed to retrieve tokens`, error);
return null;
}
}
}