🧠 feat: User Memories for Conversational Context (#7760)

* 🧠 feat: User Memories for Conversational Context

chore: mcp typing, use `t`

WIP: first pass, Memories UI

- Added MemoryViewer component for displaying, editing, and deleting user memories.
- Integrated data provider hooks for fetching, updating, and deleting memories.
- Implemented pagination and loading states for better user experience.
- Created unit tests for MemoryViewer to ensure functionality and interaction with data provider.
- Updated translation files to include new UI strings related to memories.

chore: move mcp-related files to own directory

chore: rename librechat-mcp to librechat-api

WIP: first pass, memory processing and data schemas

chore: linting in fileSearch.js query description

chore: rename librechat-api to @librechat/api across the project

WIP: first pass, functional memory agent

feat: add MemoryEditDialog and MemoryViewer components for managing user memories

- Introduced MemoryEditDialog for editing memory entries with validation and toast notifications.
- Updated MemoryViewer to support editing and deleting memories, including pagination and loading states.
- Enhanced data provider to handle memory updates with optional original key for better management.
- Added new localization strings for memory-related UI elements.

feat: add memory permissions management

- Implemented memory permissions in the backend, allowing roles to have specific permissions for using, creating, updating, and reading memories.
- Added new API endpoints for updating memory permissions associated with roles.
- Created a new AdminSettings component for managing memory permissions in the frontend.
- Integrated memory permissions into the existing roles and permissions schemas.
- Updated the interface to include memory settings and permissions.
- Enhanced the MemoryViewer component to conditionally render admin settings based on user roles.
- Added localization support for memory permissions in the translation files.

feat: move AdminSettings component to a new position in MemoryViewer for better visibility

refactor: clean up commented code in MemoryViewer component

feat: enhance MemoryViewer with search functionality and improve MemoryEditDialog integration

- Added a search input to filter memories in the MemoryViewer component.
- Refactored MemoryEditDialog to accept children for better customization.
- Updated MemoryViewer to utilize the new EditMemoryButton and DeleteMemoryButton components for editing and deleting memories.
- Improved localization support by adding new strings for memory filtering and deletion confirmation.

refactor: optimize memory filtering in MemoryViewer using match-sorter

- Replaced manual filtering logic with match-sorter for improved search functionality.
- Enhanced performance and readability of the filteredMemories computation.

feat: enhance MemoryEditDialog with triggerRef and improve updateMemory mutation handling

feat: implement access control for MemoryEditDialog and MemoryViewer components

refactor: remove commented out code and create runMemory method

refactor: rename role based files

feat: implement access control for memory usage in AgentClient

refactor: simplify checkVisionRequest method in AgentClient by removing commented-out code

refactor: make `agents` dir in api package

refactor: migrate Azure utilities to TypeScript and consolidate imports

refactor: move sanitizeFilename function to a new file and update imports, add related tests

refactor: update LLM configuration types and consolidate Azure options in the API package

chore: linting

chore: import order

refactor: replace getLLMConfig with getOpenAIConfig and remove unused LLM configuration file

chore: update winston-daily-rotate-file to version 5.0.0 and add object-hash dependency in package-lock.json

refactor: move primeResources and optionalChainWithEmptyCheck functions to resources.ts and update imports

refactor: move createRun function to a new run.ts file and update related imports

fix: ensure safeAttachments is correctly typed as an array of TFile

chore: add node-fetch dependency and refactor fetch-related functions into packages/api/utils, removing the old generators file

refactor: enhance TEndpointOption type by using Pick to streamline endpoint fields and add new properties for model parameters and client options

feat: implement initializeOpenAIOptions function and update OpenAI types for enhanced configuration handling

fix: update types due to new TEndpointOption typing

fix: ensure safe access to group parameters in initializeOpenAIOptions function

fix: remove redundant API key validation comment in initializeOpenAIOptions function

refactor: rename initializeOpenAIOptions to initializeOpenAI for consistency and update related documentation

refactor: decouple req.body fields and tool loading from initializeAgentOptions

chore: linting

refactor: adjust column widths in MemoryViewer for improved layout

refactor: simplify agent initialization by creating loadAgent function and removing unused code

feat: add memory configuration loading and validation functions

WIP: first pass, memory processing with config

feat: implement memory callback and artifact handling

feat: implement memory artifacts display and processing updates

feat: add memory configuration options and schema validation for validKeys

fix: update MemoryEditDialog and MemoryViewer to handle memory state and display improvements

refactor: remove padding from BookmarkTable and MemoryViewer headers for consistent styling

WIP: initial tokenLimit config and move Tokenizer to @librechat/api

refactor: update mongoMeili plugin methods to use callback for better error handling

feat: enhance memory management with token tracking and usage metrics

- Added token counting for memory entries to enforce limits and provide usage statistics.
- Updated memory retrieval and update routes to include total token usage and limit.
- Enhanced MemoryEditDialog and MemoryViewer components to display memory usage and token information.
- Refactored memory processing functions to handle token limits and provide feedback on memory capacity.

feat: implement memory artifact handling in attachment handler

- Enhanced useAttachmentHandler to process memory artifacts when receiving updates.
- Introduced handleMemoryArtifact utility to manage memory updates and deletions.
- Updated query client to reflect changes in memory state based on incoming data.

refactor: restructure web search key extraction logic

- Moved the logic for extracting API keys from the webSearchAuth configuration into a dedicated function, getWebSearchKeys.
- Updated webSearchKeys to utilize the new function for improved clarity and maintainability.
- Prevents build time errors

feat: add personalization settings and memory preferences management

- Introduced a new Personalization tab in settings to manage user memory preferences.
- Implemented API endpoints and client-side logic for updating memory preferences.
- Enhanced user interface components to reflect personalization options and memory usage.
- Updated permissions to allow users to opt out of memory features.
- Added localization support for new settings and messages related to personalization.

style: personalization switch class

feat: add PersonalizationIcon and align Side Panel UI

feat: implement memory creation functionality

- Added a new API endpoint for creating memory entries, including validation for key and value.
- Introduced MemoryCreateDialog component for user interface to facilitate memory creation.
- Integrated token limit checks to prevent exceeding user memory capacity.
- Updated MemoryViewer to include a button for opening the memory creation dialog.
- Enhanced localization support for new messages related to memory creation.

feat: enhance message processing with configurable window size

- Updated AgentClient to use a configurable message window size for processing messages.
- Introduced messageWindowSize option in memory configuration schema with a default value of 5.
- Improved logic for selecting messages to process based on the configured window size.

chore: update librechat-data-provider version to 0.7.87 in package.json and package-lock.json

chore: remove OpenAPIPlugin and its associated tests

chore: remove MIGRATION_README.md as migration tasks are completed

ci: fix backend tests

chore: remove unused translation keys from localization file

chore: remove problematic test file and unused var in AgentClient

chore: remove unused import and import directly for JSDoc

* feat: add api package build stage in Dockerfile for improved modularity

* docs: reorder build steps in contributing guide for clarity
This commit is contained in:
Danny Avila 2025-06-07 18:52:22 -04:00 committed by GitHub
parent cd7dd576c1
commit 29ef91b4dd
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
170 changed files with 5700 additions and 3632 deletions

View file

@ -0,0 +1,583 @@
import { EventEmitter } from 'events';
import { Client } from '@modelcontextprotocol/sdk/client/index.js';
import { SSEClientTransport } from '@modelcontextprotocol/sdk/client/sse.js';
import {
StdioClientTransport,
getDefaultEnvironment,
} from '@modelcontextprotocol/sdk/client/stdio.js';
import { WebSocketClientTransport } from '@modelcontextprotocol/sdk/client/websocket.js';
import { ResourceListChangedNotificationSchema } from '@modelcontextprotocol/sdk/types.js';
import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js';
import type { Transport } from '@modelcontextprotocol/sdk/shared/transport.js';
import type { JSONRPCMessage } from '@modelcontextprotocol/sdk/types.js';
import type { Logger } from 'winston';
import type * as t from './types';
function isStdioOptions(options: t.MCPOptions): options is t.StdioOptions {
return 'command' in options;
}
function isWebSocketOptions(options: t.MCPOptions): options is t.WebSocketOptions {
if ('url' in options) {
const protocol = new URL(options.url).protocol;
return protocol === 'ws:' || protocol === 'wss:';
}
return false;
}
function isSSEOptions(options: t.MCPOptions): options is t.SSEOptions {
if ('url' in options) {
const protocol = new URL(options.url).protocol;
return protocol !== 'ws:' && protocol !== 'wss:';
}
return false;
}
/**
* Checks if the provided options are for a Streamable HTTP transport.
*
* Streamable HTTP is an MCP transport that uses HTTP POST for sending messages
* and supports streaming responses. It provides better performance than
* SSE transport while maintaining compatibility with most network environments.
*
* @param options MCP connection options to check
* @returns True if options are for a streamable HTTP transport
*/
function isStreamableHTTPOptions(options: t.MCPOptions): options is t.StreamableHTTPOptions {
if ('url' in options && options.type === 'streamable-http') {
const protocol = new URL(options.url).protocol;
return protocol !== 'ws:' && protocol !== 'wss:';
}
return false;
}
const FIVE_MINUTES = 5 * 60 * 1000;
export class MCPConnection extends EventEmitter {
private static instance: MCPConnection | null = null;
public client: Client;
private transport: Transport | null = null; // Make this nullable
private connectionState: t.ConnectionState = 'disconnected';
private connectPromise: Promise<void> | null = null;
private lastError: Error | null = null;
private lastConfigUpdate = 0;
private readonly CONFIG_TTL = 5 * 60 * 1000; // 5 minutes
private readonly MAX_RECONNECT_ATTEMPTS = 3;
public readonly serverName: string;
private shouldStopReconnecting = false;
private isReconnecting = false;
private isInitializing = false;
private reconnectAttempts = 0;
iconPath?: string;
timeout?: number;
private readonly userId?: string;
private lastPingTime: number;
constructor(
serverName: string,
private readonly options: t.MCPOptions,
private logger?: Logger,
userId?: string,
) {
super();
this.serverName = serverName;
this.logger = logger;
this.userId = userId;
this.iconPath = options.iconPath;
this.timeout = options.timeout;
this.lastPingTime = Date.now();
this.client = new Client(
{
name: '@librechat/api-client',
version: '1.2.2',
},
{
capabilities: {},
},
);
this.setupEventListeners();
}
/** Helper to generate consistent log prefixes */
private getLogPrefix(): string {
const userPart = this.userId ? `[User: ${this.userId}]` : '';
return `[MCP]${userPart}[${this.serverName}]`;
}
public static getInstance(
serverName: string,
options: t.MCPOptions,
logger?: Logger,
userId?: string,
): MCPConnection {
if (!MCPConnection.instance) {
MCPConnection.instance = new MCPConnection(serverName, options, logger, userId);
}
return MCPConnection.instance;
}
public static getExistingInstance(): MCPConnection | null {
return MCPConnection.instance;
}
public static async destroyInstance(): Promise<void> {
if (MCPConnection.instance) {
await MCPConnection.instance.disconnect();
MCPConnection.instance = null;
}
}
private emitError(error: unknown, errorContext: string): void {
const errorMessage = error instanceof Error ? error.message : String(error);
this.logger?.error(`${this.getLogPrefix()} ${errorContext}: ${errorMessage}`);
this.emit('error', new Error(`${errorContext}: ${errorMessage}`));
}
private constructTransport(options: t.MCPOptions): Transport {
try {
let type: t.MCPOptions['type'];
if (isStdioOptions(options)) {
type = 'stdio';
} else if (isWebSocketOptions(options)) {
type = 'websocket';
} else if (isStreamableHTTPOptions(options)) {
type = 'streamable-http';
} else if (isSSEOptions(options)) {
type = 'sse';
} else {
throw new Error(
'Cannot infer transport type: options.type is not provided and cannot be inferred from other properties.',
);
}
switch (type) {
case 'stdio':
if (!isStdioOptions(options)) {
throw new Error('Invalid options for stdio transport.');
}
return new StdioClientTransport({
command: options.command,
args: options.args,
// workaround bug of mcp sdk that can't pass env:
// https://github.com/modelcontextprotocol/typescript-sdk/issues/216
env: { ...getDefaultEnvironment(), ...(options.env ?? {}) },
});
case 'websocket':
if (!isWebSocketOptions(options)) {
throw new Error('Invalid options for websocket transport.');
}
return new WebSocketClientTransport(new URL(options.url));
case 'sse': {
if (!isSSEOptions(options)) {
throw new Error('Invalid options for sse transport.');
}
const url = new URL(options.url);
this.logger?.info(`${this.getLogPrefix()} Creating SSE transport: ${url.toString()}`);
const abortController = new AbortController();
const transport = new SSEClientTransport(url, {
requestInit: {
headers: options.headers,
signal: abortController.signal,
},
eventSourceInit: {
fetch: (url, init) => {
const headers = new Headers(Object.assign({}, init?.headers, options.headers));
return fetch(url, {
...init,
headers,
});
},
},
});
transport.onclose = () => {
this.logger?.info(`${this.getLogPrefix()} SSE transport closed`);
this.emit('connectionChange', 'disconnected');
};
transport.onerror = (error) => {
this.logger?.error(`${this.getLogPrefix()} SSE transport error:`, error);
this.emitError(error, 'SSE transport error:');
};
transport.onmessage = (message) => {
this.logger?.info(
`${this.getLogPrefix()} Message received: ${JSON.stringify(message)}`,
);
};
this.setupTransportErrorHandlers(transport);
return transport;
}
case 'streamable-http': {
if (!isStreamableHTTPOptions(options)) {
throw new Error('Invalid options for streamable-http transport.');
}
const url = new URL(options.url);
this.logger?.info(
`${this.getLogPrefix()} Creating streamable-http transport: ${url.toString()}`,
);
const abortController = new AbortController();
const transport = new StreamableHTTPClientTransport(url, {
requestInit: {
headers: options.headers,
signal: abortController.signal,
},
});
transport.onclose = () => {
this.logger?.info(`${this.getLogPrefix()} Streamable-http transport closed`);
this.emit('connectionChange', 'disconnected');
};
transport.onerror = (error: Error | unknown) => {
this.logger?.error(`${this.getLogPrefix()} Streamable-http transport error:`, error);
this.emitError(error, 'Streamable-http transport error:');
};
transport.onmessage = (message: JSONRPCMessage) => {
this.logger?.info(
`${this.getLogPrefix()} Message received: ${JSON.stringify(message)}`,
);
};
this.setupTransportErrorHandlers(transport);
return transport;
}
default: {
throw new Error(`Unsupported transport type: ${type}`);
}
}
} catch (error) {
this.emitError(error, 'Failed to construct transport:');
throw error;
}
}
private setupEventListeners(): void {
this.isInitializing = true;
this.on('connectionChange', (state: t.ConnectionState) => {
this.connectionState = state;
if (state === 'connected') {
this.isReconnecting = false;
this.isInitializing = false;
this.shouldStopReconnecting = false;
this.reconnectAttempts = 0;
/**
* // FOR DEBUGGING
* // this.client.setRequestHandler(PingRequestSchema, async (request, extra) => {
* // this.logger?.info(`[MCP][${this.serverName}] PingRequest: ${JSON.stringify(request)}`);
* // if (getEventListeners && extra.signal) {
* // const listenerCount = getEventListeners(extra.signal, 'abort').length;
* // this.logger?.debug(`Signal has ${listenerCount} abort listeners`);
* // }
* // return {};
* // });
*/
} else if (state === 'error' && !this.isReconnecting && !this.isInitializing) {
this.handleReconnection().catch((error) => {
this.logger?.error(`${this.getLogPrefix()} Reconnection handler failed:`, error);
});
}
});
this.subscribeToResources();
}
private async handleReconnection(): Promise<void> {
if (this.isReconnecting || this.shouldStopReconnecting || this.isInitializing) {
return;
}
this.isReconnecting = true;
const backoffDelay = (attempt: number) => Math.min(1000 * Math.pow(2, attempt), 30000);
try {
while (
this.reconnectAttempts < this.MAX_RECONNECT_ATTEMPTS &&
!(this.shouldStopReconnecting as boolean)
) {
this.reconnectAttempts++;
const delay = backoffDelay(this.reconnectAttempts);
this.logger?.info(
`${this.getLogPrefix()} Reconnecting ${this.reconnectAttempts}/${this.MAX_RECONNECT_ATTEMPTS} (delay: ${delay}ms)`,
);
await new Promise((resolve) => setTimeout(resolve, delay));
try {
await this.connect();
this.reconnectAttempts = 0;
return;
} catch (error) {
this.logger?.error(`${this.getLogPrefix()} Reconnection attempt failed:`, error);
if (
this.reconnectAttempts === this.MAX_RECONNECT_ATTEMPTS ||
(this.shouldStopReconnecting as boolean)
) {
this.logger?.error(`${this.getLogPrefix()} Stopping reconnection attempts`);
return;
}
}
}
} finally {
this.isReconnecting = false;
}
}
private subscribeToResources(): void {
this.client.setNotificationHandler(ResourceListChangedNotificationSchema, async () => {
this.invalidateCache();
this.emit('resourcesChanged');
});
}
private invalidateCache(): void {
// this.cachedConfig = null;
this.lastConfigUpdate = 0;
}
async connectClient(): Promise<void> {
if (this.connectionState === 'connected') {
return;
}
if (this.connectPromise) {
return this.connectPromise;
}
if (this.shouldStopReconnecting) {
return;
}
this.emit('connectionChange', 'connecting');
this.connectPromise = (async () => {
try {
if (this.transport) {
try {
await this.client.close();
this.transport = null;
} catch (error) {
this.logger?.warn(`${this.getLogPrefix()} Error closing connection:`, error);
}
}
this.transport = this.constructTransport(this.options);
this.setupTransportDebugHandlers();
const connectTimeout = this.options.initTimeout ?? 10000;
await Promise.race([
this.client.connect(this.transport),
new Promise((_resolve, reject) =>
setTimeout(() => reject(new Error('Connection timeout')), connectTimeout),
),
]);
this.connectionState = 'connected';
this.emit('connectionChange', 'connected');
this.reconnectAttempts = 0;
} catch (error) {
this.connectionState = 'error';
this.emit('connectionChange', 'error');
this.lastError = error instanceof Error ? error : new Error(String(error));
throw error;
} finally {
this.connectPromise = null;
}
})();
return this.connectPromise;
}
private setupTransportDebugHandlers(): void {
if (!this.transport) {
return;
}
this.transport.onmessage = (msg) => {
this.logger?.debug(`${this.getLogPrefix()} Transport received: ${JSON.stringify(msg)}`);
};
const originalSend = this.transport.send.bind(this.transport);
this.transport.send = async (msg) => {
if ('result' in msg && !('method' in msg) && Object.keys(msg.result ?? {}).length === 0) {
if (Date.now() - this.lastPingTime < FIVE_MINUTES) {
throw new Error('Empty result');
}
this.lastPingTime = Date.now();
}
this.logger?.debug(`${this.getLogPrefix()} Transport sending: ${JSON.stringify(msg)}`);
return originalSend(msg);
};
}
async connect(): Promise<void> {
try {
await this.disconnect();
await this.connectClient();
if (!this.isConnected()) {
throw new Error('Connection not established');
}
} catch (error) {
this.logger?.error(`${this.getLogPrefix()} Connection failed:`, error);
throw error;
}
}
private setupTransportErrorHandlers(transport: Transport): void {
transport.onerror = (error) => {
this.logger?.error(`${this.getLogPrefix()} Transport error:`, error);
this.emit('connectionChange', 'error');
};
}
public async disconnect(): Promise<void> {
try {
if (this.transport) {
await this.client.close();
this.transport = null;
}
if (this.connectionState === 'disconnected') {
return;
}
this.connectionState = 'disconnected';
this.emit('connectionChange', 'disconnected');
} catch (error) {
this.emit('error', error);
throw error;
} finally {
this.invalidateCache();
this.connectPromise = null;
}
}
async fetchResources(): Promise<t.MCPResource[]> {
try {
const { resources } = await this.client.listResources();
return resources;
} catch (error) {
this.emitError(error, 'Failed to fetch resources:');
return [];
}
}
async fetchTools() {
try {
const { tools } = await this.client.listTools();
return tools;
} catch (error) {
this.emitError(error, 'Failed to fetch tools:');
return [];
}
}
async fetchPrompts(): Promise<t.MCPPrompt[]> {
try {
const { prompts } = await this.client.listPrompts();
return prompts;
} catch (error) {
this.emitError(error, 'Failed to fetch prompts:');
return [];
}
}
// public async modifyConfig(config: ContinueConfig): Promise<ContinueConfig> {
// try {
// // Check cache
// if (this.cachedConfig && Date.now() - this.lastConfigUpdate < this.CONFIG_TTL) {
// return this.cachedConfig;
// }
// await this.connectClient();
// // Fetch and process resources
// const resources = await this.fetchResources();
// const submenuItems = resources.map(resource => ({
// title: resource.name,
// description: resource.description,
// id: resource.uri,
// }));
// if (!config.contextProviders) {
// config.contextProviders = [];
// }
// config.contextProviders.push(
// new MCPContextProvider({
// submenuItems,
// client: this.client,
// }),
// );
// // Fetch and process tools
// const tools = await this.fetchTools();
// const continueTools: Tool[] = tools.map(tool => ({
// displayTitle: tool.name,
// function: {
// description: tool.description,
// name: tool.name,
// parameters: tool.inputSchema,
// },
// readonly: false,
// type: 'function',
// wouldLikeTo: `use the ${tool.name} tool`,
// uri: `mcp://${tool.name}`,
// }));
// config.tools = [...(config.tools || []), ...continueTools];
// // Fetch and process prompts
// const prompts = await this.fetchPrompts();
// if (!config.slashCommands) {
// config.slashCommands = [];
// }
// const slashCommands: SlashCommand[] = prompts.map(prompt =>
// constructMcpSlashCommand(
// this.client,
// prompt.name,
// prompt.description,
// prompt.arguments?.map(a => a.name),
// ),
// );
// config.slashCommands.push(...slashCommands);
// // Update cache
// this.cachedConfig = config;
// this.lastConfigUpdate = Date.now();
// return config;
// } catch (error) {
// this.emit('error', error);
// // Return original config if modification fails
// return config;
// }
// }
// Public getters for state information
public getConnectionState(): t.ConnectionState {
return this.connectionState;
}
public async isConnected(): Promise<boolean> {
try {
await this.client.ping();
return this.connectionState === 'connected';
} catch (error) {
this.logger?.error(`${this.getLogPrefix()} Ping failed:`, error);
return false;
}
}
public getLastError(): Error | null {
return this.lastError;
}
}