mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-02-21 09:54:08 +01:00
* chore: Update @modelcontextprotocol/sdk to version 1.12.3 in package.json and package-lock.json - Bump version of @modelcontextprotocol/sdk to 1.12.3 to incorporate recent updates. - Update dependencies for ajv and cross-spawn to their latest versions. - Add ajv as a new dependency in the sdk module. - Include json-schema-traverse as a new dependency in the sdk module. * feat: @librechat/auth * feat: Add crypto module exports to auth package - Introduced a new crypto module by creating index.ts in the crypto directory. - Updated the main index.ts of the auth package to export from the new crypto module. * feat: Update package dependencies and build scripts for auth package - Added @librechat/auth as a dependency in package.json and package-lock.json. - Updated build scripts to include the auth package in both frontend and bun build processes. - Removed unused mongoose and openid-client dependencies from package-lock.json for cleaner dependency management. * refactor: Migrate crypto utility functions to @librechat/auth - Replaced local crypto utility imports with the new @librechat/auth package across multiple files. - Removed the obsolete crypto.js file and its exports. - Updated relevant services and models to utilize the new encryption and decryption methods from @librechat/auth. * feat: Enhance OAuth token handling and update dependencies in auth package * chore: Remove Token model and TokenService due to restructuring of OAuth handling - Deleted the Token.js model and TokenService.js, which were responsible for managing OAuth tokens. - This change is part of a broader refactor to streamline OAuth token management and improve code organization. * refactor: imports from '@librechat/auth' to '@librechat/api' and add OAuth token handling functionality * refactor: Simplify logger usage in MCP and FlowStateManager classes * chore: fix imports * feat: Add OAuth configuration schema to MCP with token exchange method support * feat: FIRST PASS Implement MCP OAuth flow with token management and error handling - Added a new route for handling OAuth callbacks and token retrieval. - Integrated OAuth token storage and retrieval mechanisms. - Enhanced MCP connection to support automatic OAuth flow initiation on 401 errors. - Implemented dynamic client registration and metadata discovery for OAuth. - Updated MCPManager to manage OAuth tokens and handle authentication requirements. - Introduced comprehensive logging for OAuth processes and error handling. * refactor: Update MCPConnection and MCPManager to utilize new URL handling - Added a `url` property to MCPConnection for better URL management. - Refactored MCPManager to use the new `url` property instead of a deprecated method for OAuth handling. - Changed logging from info to debug level for flow manager and token methods initialization. - Improved comments for clarity on existing tokens and OAuth event listener setup. * refactor: Improve connection timeout error messages in MCPConnection and MCPManager and use initTimeout for connection - Updated the connection timeout error messages to include the duration of the timeout. - Introduced a configurable `connectTimeout` variable in both MCPConnection and MCPManager for better flexibility. * chore: cleanup MCP OAuth Token exchange handling; fix: erroneous use of flowsCache and remove verbose logs * refactor: Update MCPManager and MCPTokenStorage to use TokenMethods for token management - Removed direct token storage handling in MCPManager and replaced it with TokenMethods for better abstraction. - Refactored MCPTokenStorage methods to accept parameters for token operations, enhancing flexibility and readability. - Improved logging messages related to token persistence and retrieval processes. * refactor: Update MCP OAuth handling to use static methods and improve flow management - Refactored MCPOAuthHandler to utilize static methods for initiating and completing OAuth flows, enhancing clarity and reducing instance dependencies. - Updated MCPManager to pass flowManager explicitly to OAuth handling methods, improving flexibility in flow state management. - Enhanced comments and logging for better understanding of OAuth processes and flow state retrieval. * refactor: Integrate token methods into createMCPTool for enhanced token management * refactor: Change logging from info to debug level in MCPOAuthHandler for improved log management * chore: clean up logging * feat: first pass, auth URL from MCP OAuth flow * chore: Improve logging format for OAuth authentication URL display * chore: cleanup mcp manager comments * feat: add connection reconnection logic in MCPManager * refactor: reorganize token storage handling in MCP - Moved token storage logic from MCPManager to a new MCPTokenStorage class for better separation of concerns. - Updated imports to reflect the new token storage structure. - Enhanced methods for storing, retrieving, updating, and deleting OAuth tokens, improving overall token management. * chore: update comment for SYSTEM_USER_ID in MCPManager for clarity * feat: implement refresh token functionality in MCP - Added refresh token handling in MCPManager to support token renewal for both app-level and user-specific connections. - Introduced a refreshTokens function to facilitate token refresh logic. - Enhanced MCPTokenStorage to manage client information and refresh token processes. - Updated logging for better traceability during token operations. * chore: cleanup @librechat/auth * feat: implement MCP server initialization in a separate service - Added a new service to handle the initialization of MCP servers, improving code organization and readability. - Refactored the server startup logic to utilize the new initializeMCP function. - Removed redundant MCP initialization code from the main server file. * fix: don't log auth url for user connections * feat: enhance OAuth flow with success and error handling components - Updated OAuth callback routes to redirect to new success and error pages instead of sending status messages. - Introduced `OAuthSuccess` and `OAuthError` components to provide user feedback during authentication. - Added localization support for success and error messages in the translation files. - Implemented countdown functionality in the success component for a better user experience. * fix: refresh token handling for user connections, add missing URL and methods - add standard enum for system user id and helper for determining app-lvel vs. user-level connections * refactor: update token handling in MCPManager and MCPTokenStorage * fix: improve error logging in OAuth authentication handler * fix: concurrency issues for both login url emission and concurrency of oauth flows for shared flows (same user, same server, multiple calls for same server) * fix: properly fail shared flows for concurrent server calls and prevent duplication of tokens * chore: remove unused auth package directory from update configuration * ci: fix mocks in samlStrategy tests * ci: add mcpConfig to AppService test setup * chore: remove obsolete MCP OAuth implementation documentation * fix: update build script for API to use correct command * chore: bump version of @librechat/api to 1.2.4 * fix: update abort signal handling in createMCPTool function * fix: add optional clientInfo parameter to refreshTokensFunction metadata * refactor: replace app.locals.availableTools with getCachedTools in multiple services and controllers for improved tool management * fix: concurrent refresh token handling issue * refactor: add signal parameter to getUserConnection method for improved abort handling * chore: JSDoc typing for `loadEphemeralAgent` * refactor: update isConnectionActive method to use destructured parameters for improved readability * feat: implement caching for MCP tools to handle app-level disconnects for loading list of tools * ci: fix agent test
382 lines
14 KiB
TypeScript
382 lines
14 KiB
TypeScript
import { logger } from '@librechat/data-schemas';
|
|
import type { OAuthTokens, OAuthClientInformation } from '@modelcontextprotocol/sdk/shared/auth.js';
|
|
import type { TokenMethods, IToken } from '@librechat/data-schemas';
|
|
import type { MCPOAuthTokens, ExtendedOAuthTokens } from './types';
|
|
import { encryptV2, decryptV2 } from '~/crypto';
|
|
import { isSystemUserId } from '~/mcp/enum';
|
|
|
|
interface StoreTokensParams {
|
|
userId: string;
|
|
serverName: string;
|
|
tokens: OAuthTokens | ExtendedOAuthTokens | MCPOAuthTokens;
|
|
createToken: TokenMethods['createToken'];
|
|
updateToken?: TokenMethods['updateToken'];
|
|
findToken?: TokenMethods['findToken'];
|
|
clientInfo?: OAuthClientInformation;
|
|
/** Optional: Pass existing token state to avoid duplicate DB calls */
|
|
existingTokens?: {
|
|
accessToken?: IToken | null;
|
|
refreshToken?: IToken | null;
|
|
clientInfoToken?: IToken | null;
|
|
};
|
|
}
|
|
|
|
interface GetTokensParams {
|
|
userId: string;
|
|
serverName: string;
|
|
findToken: TokenMethods['findToken'];
|
|
refreshTokens?: (
|
|
refreshToken: string,
|
|
metadata: { userId: string; serverName: string; identifier: string },
|
|
) => Promise<MCPOAuthTokens>;
|
|
createToken?: TokenMethods['createToken'];
|
|
updateToken?: TokenMethods['updateToken'];
|
|
}
|
|
|
|
export class MCPTokenStorage {
|
|
static getLogPrefix(userId: string, serverName: string): string {
|
|
return isSystemUserId(userId)
|
|
? `[MCP][${serverName}]`
|
|
: `[MCP][User: ${userId}][${serverName}]`;
|
|
}
|
|
|
|
/**
|
|
* Stores OAuth tokens for an MCP server
|
|
*
|
|
* @param params.existingTokens - Optional: Pass existing token state to avoid duplicate DB calls.
|
|
* This is useful when refreshing tokens, as getTokens() already has the token state.
|
|
*/
|
|
static async storeTokens({
|
|
userId,
|
|
serverName,
|
|
tokens,
|
|
createToken,
|
|
updateToken,
|
|
findToken,
|
|
clientInfo,
|
|
existingTokens,
|
|
}: StoreTokensParams): Promise<void> {
|
|
const logPrefix = this.getLogPrefix(userId, serverName);
|
|
|
|
try {
|
|
const identifier = `mcp:${serverName}`;
|
|
|
|
// Encrypt and store access token
|
|
const encryptedAccessToken = await encryptV2(tokens.access_token);
|
|
|
|
logger.debug(
|
|
`${logPrefix} Token expires_in: ${'expires_in' in tokens ? tokens.expires_in : 'N/A'}, expires_at: ${'expires_at' in tokens ? tokens.expires_at : 'N/A'}`,
|
|
);
|
|
|
|
// Handle both expires_in and expires_at formats
|
|
let accessTokenExpiry: Date;
|
|
if ('expires_at' in tokens && tokens.expires_at) {
|
|
/** MCPOAuthTokens format - already has calculated expiry */
|
|
logger.debug(`${logPrefix} Using expires_at: ${tokens.expires_at}`);
|
|
accessTokenExpiry = new Date(tokens.expires_at);
|
|
} else if (tokens.expires_in) {
|
|
/** Standard OAuthTokens format - calculate expiry */
|
|
logger.debug(`${logPrefix} Using expires_in: ${tokens.expires_in}`);
|
|
accessTokenExpiry = new Date(Date.now() + tokens.expires_in * 1000);
|
|
} else {
|
|
/** No expiry provided - default to 1 year */
|
|
logger.debug(`${logPrefix} No expiry provided, using default`);
|
|
accessTokenExpiry = new Date(Date.now() + 365 * 24 * 60 * 60 * 1000);
|
|
}
|
|
|
|
logger.debug(`${logPrefix} Calculated expiry date: ${accessTokenExpiry.toISOString()}`);
|
|
logger.debug(
|
|
`${logPrefix} Date object: ${JSON.stringify({
|
|
time: accessTokenExpiry.getTime(),
|
|
valid: !isNaN(accessTokenExpiry.getTime()),
|
|
iso: accessTokenExpiry.toISOString(),
|
|
})}`,
|
|
);
|
|
|
|
// Ensure the date is valid before passing to createToken
|
|
if (isNaN(accessTokenExpiry.getTime())) {
|
|
logger.error(`${logPrefix} Invalid expiry date calculated, using default`);
|
|
accessTokenExpiry = new Date(Date.now() + 365 * 24 * 60 * 60 * 1000);
|
|
}
|
|
|
|
// Calculate expiresIn (seconds from now)
|
|
const expiresIn = Math.floor((accessTokenExpiry.getTime() - Date.now()) / 1000);
|
|
|
|
const accessTokenData = {
|
|
userId,
|
|
type: 'mcp_oauth',
|
|
identifier,
|
|
token: encryptedAccessToken,
|
|
expiresIn: expiresIn > 0 ? expiresIn : 365 * 24 * 60 * 60, // Default to 1 year if negative
|
|
};
|
|
|
|
// Check if token already exists and update if it does
|
|
if (findToken && updateToken) {
|
|
// Use provided existing token state if available, otherwise look it up
|
|
const existingToken =
|
|
existingTokens?.accessToken !== undefined
|
|
? existingTokens.accessToken
|
|
: await findToken({ userId, identifier });
|
|
|
|
if (existingToken) {
|
|
await updateToken({ userId, identifier }, accessTokenData);
|
|
logger.debug(`${logPrefix} Updated existing access token`);
|
|
} else {
|
|
await createToken(accessTokenData);
|
|
logger.debug(`${logPrefix} Created new access token`);
|
|
}
|
|
} else {
|
|
// Create new token if it's initial store or update methods not provided
|
|
await createToken(accessTokenData);
|
|
logger.debug(`${logPrefix} Created access token (no update methods available)`);
|
|
}
|
|
|
|
// Store refresh token if available
|
|
if (tokens.refresh_token) {
|
|
const encryptedRefreshToken = await encryptV2(tokens.refresh_token);
|
|
const extendedTokens = tokens as ExtendedOAuthTokens;
|
|
const refreshTokenExpiry = extendedTokens.refresh_token_expires_in
|
|
? new Date(Date.now() + extendedTokens.refresh_token_expires_in * 1000)
|
|
: new Date(Date.now() + 365 * 24 * 60 * 60 * 1000); // Default to 1 year
|
|
|
|
/** Calculated expiresIn for refresh token */
|
|
const refreshExpiresIn = Math.floor((refreshTokenExpiry.getTime() - Date.now()) / 1000);
|
|
|
|
const refreshTokenData = {
|
|
userId,
|
|
type: 'mcp_oauth_refresh',
|
|
identifier: `${identifier}:refresh`,
|
|
token: encryptedRefreshToken,
|
|
expiresIn: refreshExpiresIn > 0 ? refreshExpiresIn : 365 * 24 * 60 * 60,
|
|
};
|
|
|
|
// Check if refresh token already exists and update if it does
|
|
if (findToken && updateToken) {
|
|
// Use provided existing token state if available, otherwise look it up
|
|
const existingRefreshToken =
|
|
existingTokens?.refreshToken !== undefined
|
|
? existingTokens.refreshToken
|
|
: await findToken({
|
|
userId,
|
|
identifier: `${identifier}:refresh`,
|
|
});
|
|
|
|
if (existingRefreshToken) {
|
|
await updateToken({ userId, identifier: `${identifier}:refresh` }, refreshTokenData);
|
|
logger.debug(`${logPrefix} Updated existing refresh token`);
|
|
} else {
|
|
await createToken(refreshTokenData);
|
|
logger.debug(`${logPrefix} Created new refresh token`);
|
|
}
|
|
} else {
|
|
await createToken(refreshTokenData);
|
|
logger.debug(`${logPrefix} Created refresh token (no update methods available)`);
|
|
}
|
|
}
|
|
|
|
/** Store client information if provided */
|
|
if (clientInfo) {
|
|
logger.debug(`${logPrefix} Storing client info:`, {
|
|
client_id: clientInfo.client_id,
|
|
has_client_secret: !!clientInfo.client_secret,
|
|
});
|
|
const encryptedClientInfo = await encryptV2(JSON.stringify(clientInfo));
|
|
|
|
const clientInfoData = {
|
|
userId,
|
|
type: 'mcp_oauth_client',
|
|
identifier: `${identifier}:client`,
|
|
token: encryptedClientInfo,
|
|
expiresIn: 365 * 24 * 60 * 60,
|
|
};
|
|
|
|
// Check if client info already exists and update if it does
|
|
if (findToken && updateToken) {
|
|
// Use provided existing token state if available, otherwise look it up
|
|
const existingClientInfo =
|
|
existingTokens?.clientInfoToken !== undefined
|
|
? existingTokens.clientInfoToken
|
|
: await findToken({
|
|
userId,
|
|
identifier: `${identifier}:client`,
|
|
});
|
|
|
|
if (existingClientInfo) {
|
|
await updateToken({ userId, identifier: `${identifier}:client` }, clientInfoData);
|
|
logger.debug(`${logPrefix} Updated existing client info`);
|
|
} else {
|
|
await createToken(clientInfoData);
|
|
logger.debug(`${logPrefix} Created new client info`);
|
|
}
|
|
} else {
|
|
await createToken(clientInfoData);
|
|
logger.debug(`${logPrefix} Created client info (no update methods available)`);
|
|
}
|
|
}
|
|
|
|
logger.debug(`${logPrefix} Stored OAuth tokens`);
|
|
} catch (error) {
|
|
const logPrefix = this.getLogPrefix(userId, serverName);
|
|
logger.error(`${logPrefix} Failed to store tokens`, error);
|
|
throw error;
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Retrieves OAuth tokens for an MCP server
|
|
*/
|
|
static async getTokens({
|
|
userId,
|
|
serverName,
|
|
findToken,
|
|
createToken,
|
|
updateToken,
|
|
refreshTokens,
|
|
}: GetTokensParams): Promise<MCPOAuthTokens | null> {
|
|
const logPrefix = this.getLogPrefix(userId, serverName);
|
|
|
|
try {
|
|
const identifier = `mcp:${serverName}`;
|
|
|
|
// Get access token
|
|
const accessTokenData = await findToken({
|
|
userId,
|
|
type: 'mcp_oauth',
|
|
identifier,
|
|
});
|
|
|
|
/** Check if access token is missing or expired */
|
|
const isMissing = !accessTokenData;
|
|
const isExpired = accessTokenData?.expiresAt && new Date() >= accessTokenData.expiresAt;
|
|
|
|
if (isMissing || isExpired) {
|
|
logger.info(`${logPrefix} Access token ${isMissing ? 'missing' : 'expired'}`);
|
|
|
|
/** Refresh data if we have a refresh token and refresh function */
|
|
const refreshTokenData = await findToken({
|
|
userId,
|
|
type: 'mcp_oauth_refresh',
|
|
identifier: `${identifier}:refresh`,
|
|
});
|
|
|
|
if (!refreshTokenData) {
|
|
logger.info(
|
|
`${logPrefix} Access token ${isMissing ? 'missing' : 'expired'} and no refresh token available`,
|
|
);
|
|
return null;
|
|
}
|
|
|
|
if (!refreshTokens) {
|
|
logger.warn(
|
|
`${logPrefix} Access token ${isMissing ? 'missing' : 'expired'}, refresh token available but no \`refreshTokens\` provided`,
|
|
);
|
|
return null;
|
|
}
|
|
|
|
if (!createToken) {
|
|
logger.warn(
|
|
`${logPrefix} Access token ${isMissing ? 'missing' : 'expired'}, refresh token available but no \`createToken\` function provided`,
|
|
);
|
|
return null;
|
|
}
|
|
|
|
try {
|
|
logger.info(`${logPrefix} Attempting to refresh token`);
|
|
const decryptedRefreshToken = await decryptV2(refreshTokenData.token);
|
|
|
|
/** Client information if available */
|
|
let clientInfo;
|
|
let clientInfoData;
|
|
try {
|
|
clientInfoData = await findToken({
|
|
userId,
|
|
type: 'mcp_oauth_client',
|
|
identifier: `${identifier}:client`,
|
|
});
|
|
if (clientInfoData) {
|
|
const decryptedClientInfo = await decryptV2(clientInfoData.token);
|
|
clientInfo = JSON.parse(decryptedClientInfo);
|
|
logger.debug(`${logPrefix} Retrieved client info:`, {
|
|
client_id: clientInfo.client_id,
|
|
has_client_secret: !!clientInfo.client_secret,
|
|
});
|
|
}
|
|
} catch {
|
|
logger.debug(`${logPrefix} No client info found`);
|
|
}
|
|
|
|
const metadata = {
|
|
userId,
|
|
serverName,
|
|
identifier,
|
|
clientInfo,
|
|
};
|
|
|
|
const newTokens = await refreshTokens(decryptedRefreshToken, metadata);
|
|
|
|
// Store the refreshed tokens (handles both create and update)
|
|
// Pass existing token state to avoid duplicate DB calls
|
|
await this.storeTokens({
|
|
userId,
|
|
serverName,
|
|
tokens: newTokens,
|
|
createToken,
|
|
updateToken,
|
|
findToken,
|
|
clientInfo,
|
|
existingTokens: {
|
|
accessToken: accessTokenData, // We know this is expired/missing
|
|
refreshToken: refreshTokenData, // We already have this
|
|
clientInfoToken: clientInfoData, // We already looked this up
|
|
},
|
|
});
|
|
|
|
logger.info(`${logPrefix} Successfully refreshed and stored OAuth tokens`);
|
|
return newTokens;
|
|
} catch (refreshError) {
|
|
logger.error(`${logPrefix} Failed to refresh tokens`, refreshError);
|
|
// Check if it's an unauthorized_client error (refresh not supported)
|
|
const errorMessage =
|
|
refreshError instanceof Error ? refreshError.message : String(refreshError);
|
|
if (errorMessage.includes('unauthorized_client')) {
|
|
logger.info(
|
|
`${logPrefix} Server does not support refresh tokens for this client. New authentication required.`,
|
|
);
|
|
}
|
|
return null;
|
|
}
|
|
}
|
|
|
|
// If we reach here, access token should exist and be valid
|
|
if (!accessTokenData) {
|
|
return null;
|
|
}
|
|
|
|
const decryptedAccessToken = await decryptV2(accessTokenData.token);
|
|
|
|
/** Get refresh token if available */
|
|
const refreshTokenData = await findToken({
|
|
userId,
|
|
type: 'mcp_oauth_refresh',
|
|
identifier: `${identifier}:refresh`,
|
|
});
|
|
|
|
const tokens: MCPOAuthTokens = {
|
|
access_token: decryptedAccessToken,
|
|
token_type: 'Bearer',
|
|
obtained_at: accessTokenData.createdAt.getTime(),
|
|
expires_at: accessTokenData.expiresAt?.getTime(),
|
|
};
|
|
|
|
if (refreshTokenData) {
|
|
tokens.refresh_token = await decryptV2(refreshTokenData.token);
|
|
}
|
|
|
|
logger.debug(`${logPrefix} Loaded existing OAuth tokens from storage`);
|
|
return tokens;
|
|
} catch (error) {
|
|
logger.error(`${logPrefix} Failed to retrieve tokens`, error);
|
|
return null;
|
|
}
|
|
}
|
|
}
|