🧠 feat: User Memories for Conversational Context (#7760)

* 🧠 feat: User Memories for Conversational Context

chore: mcp typing, use `t`

WIP: first pass, Memories UI

- Added MemoryViewer component for displaying, editing, and deleting user memories.
- Integrated data provider hooks for fetching, updating, and deleting memories.
- Implemented pagination and loading states for better user experience.
- Created unit tests for MemoryViewer to ensure functionality and interaction with data provider.
- Updated translation files to include new UI strings related to memories.

chore: move mcp-related files to own directory

chore: rename librechat-mcp to librechat-api

WIP: first pass, memory processing and data schemas

chore: linting in fileSearch.js query description

chore: rename librechat-api to @librechat/api across the project

WIP: first pass, functional memory agent

feat: add MemoryEditDialog and MemoryViewer components for managing user memories

- Introduced MemoryEditDialog for editing memory entries with validation and toast notifications.
- Updated MemoryViewer to support editing and deleting memories, including pagination and loading states.
- Enhanced data provider to handle memory updates with optional original key for better management.
- Added new localization strings for memory-related UI elements.

feat: add memory permissions management

- Implemented memory permissions in the backend, allowing roles to have specific permissions for using, creating, updating, and reading memories.
- Added new API endpoints for updating memory permissions associated with roles.
- Created a new AdminSettings component for managing memory permissions in the frontend.
- Integrated memory permissions into the existing roles and permissions schemas.
- Updated the interface to include memory settings and permissions.
- Enhanced the MemoryViewer component to conditionally render admin settings based on user roles.
- Added localization support for memory permissions in the translation files.

feat: move AdminSettings component to a new position in MemoryViewer for better visibility

refactor: clean up commented code in MemoryViewer component

feat: enhance MemoryViewer with search functionality and improve MemoryEditDialog integration

- Added a search input to filter memories in the MemoryViewer component.
- Refactored MemoryEditDialog to accept children for better customization.
- Updated MemoryViewer to utilize the new EditMemoryButton and DeleteMemoryButton components for editing and deleting memories.
- Improved localization support by adding new strings for memory filtering and deletion confirmation.

refactor: optimize memory filtering in MemoryViewer using match-sorter

- Replaced manual filtering logic with match-sorter for improved search functionality.
- Enhanced performance and readability of the filteredMemories computation.

feat: enhance MemoryEditDialog with triggerRef and improve updateMemory mutation handling

feat: implement access control for MemoryEditDialog and MemoryViewer components

refactor: remove commented out code and create runMemory method

refactor: rename role based files

feat: implement access control for memory usage in AgentClient

refactor: simplify checkVisionRequest method in AgentClient by removing commented-out code

refactor: make `agents` dir in api package

refactor: migrate Azure utilities to TypeScript and consolidate imports

refactor: move sanitizeFilename function to a new file and update imports, add related tests

refactor: update LLM configuration types and consolidate Azure options in the API package

chore: linting

chore: import order

refactor: replace getLLMConfig with getOpenAIConfig and remove unused LLM configuration file

chore: update winston-daily-rotate-file to version 5.0.0 and add object-hash dependency in package-lock.json

refactor: move primeResources and optionalChainWithEmptyCheck functions to resources.ts and update imports

refactor: move createRun function to a new run.ts file and update related imports

fix: ensure safeAttachments is correctly typed as an array of TFile

chore: add node-fetch dependency and refactor fetch-related functions into packages/api/utils, removing the old generators file

refactor: enhance TEndpointOption type by using Pick to streamline endpoint fields and add new properties for model parameters and client options

feat: implement initializeOpenAIOptions function and update OpenAI types for enhanced configuration handling

fix: update types due to new TEndpointOption typing

fix: ensure safe access to group parameters in initializeOpenAIOptions function

fix: remove redundant API key validation comment in initializeOpenAIOptions function

refactor: rename initializeOpenAIOptions to initializeOpenAI for consistency and update related documentation

refactor: decouple req.body fields and tool loading from initializeAgentOptions

chore: linting

refactor: adjust column widths in MemoryViewer for improved layout

refactor: simplify agent initialization by creating loadAgent function and removing unused code

feat: add memory configuration loading and validation functions

WIP: first pass, memory processing with config

feat: implement memory callback and artifact handling

feat: implement memory artifacts display and processing updates

feat: add memory configuration options and schema validation for validKeys

fix: update MemoryEditDialog and MemoryViewer to handle memory state and display improvements

refactor: remove padding from BookmarkTable and MemoryViewer headers for consistent styling

WIP: initial tokenLimit config and move Tokenizer to @librechat/api

refactor: update mongoMeili plugin methods to use callback for better error handling

feat: enhance memory management with token tracking and usage metrics

- Added token counting for memory entries to enforce limits and provide usage statistics.
- Updated memory retrieval and update routes to include total token usage and limit.
- Enhanced MemoryEditDialog and MemoryViewer components to display memory usage and token information.
- Refactored memory processing functions to handle token limits and provide feedback on memory capacity.

feat: implement memory artifact handling in attachment handler

- Enhanced useAttachmentHandler to process memory artifacts when receiving updates.
- Introduced handleMemoryArtifact utility to manage memory updates and deletions.
- Updated query client to reflect changes in memory state based on incoming data.

refactor: restructure web search key extraction logic

- Moved the logic for extracting API keys from the webSearchAuth configuration into a dedicated function, getWebSearchKeys.
- Updated webSearchKeys to utilize the new function for improved clarity and maintainability.
- Prevents build time errors

feat: add personalization settings and memory preferences management

- Introduced a new Personalization tab in settings to manage user memory preferences.
- Implemented API endpoints and client-side logic for updating memory preferences.
- Enhanced user interface components to reflect personalization options and memory usage.
- Updated permissions to allow users to opt out of memory features.
- Added localization support for new settings and messages related to personalization.

style: personalization switch class

feat: add PersonalizationIcon and align Side Panel UI

feat: implement memory creation functionality

- Added a new API endpoint for creating memory entries, including validation for key and value.
- Introduced MemoryCreateDialog component for user interface to facilitate memory creation.
- Integrated token limit checks to prevent exceeding user memory capacity.
- Updated MemoryViewer to include a button for opening the memory creation dialog.
- Enhanced localization support for new messages related to memory creation.

feat: enhance message processing with configurable window size

- Updated AgentClient to use a configurable message window size for processing messages.
- Introduced messageWindowSize option in memory configuration schema with a default value of 5.
- Improved logic for selecting messages to process based on the configured window size.

chore: update librechat-data-provider version to 0.7.87 in package.json and package-lock.json

chore: remove OpenAPIPlugin and its associated tests

chore: remove MIGRATION_README.md as migration tasks are completed

ci: fix backend tests

chore: remove unused translation keys from localization file

chore: remove problematic test file and unused var in AgentClient

chore: remove unused import and import directly for JSDoc

* feat: add api package build stage in Dockerfile for improved modularity

* docs: reorder build steps in contributing guide for clarity

api/utils/azureUtils.js

@@ -1,105 +0,0 @@
-const { isEnabled } = require('~/server/utils/handleText');
-
-/**
- * Sanitizes the model name to be used in the URL by removing or replacing disallowed characters.
- * @param {string} modelName - The model name to be sanitized.
- * @returns {string} The sanitized model name.
- */
-const sanitizeModelName = (modelName) => {
-  // Replace periods with empty strings and other disallowed characters as needed.
-  return modelName.replace(/\./g, '');
-};
-
-/**
- * Generates the Azure OpenAI API endpoint URL.
- * @param {Object} params - The parameters object.
- * @param {string} params.azureOpenAIApiInstanceName - The Azure OpenAI API instance name.
- * @param {string} params.azureOpenAIApiDeploymentName - The Azure OpenAI API deployment name.
- * @returns {string} The complete endpoint URL for the Azure OpenAI API.
- */
-const genAzureEndpoint = ({ azureOpenAIApiInstanceName, azureOpenAIApiDeploymentName }) => {
-  return `https://${azureOpenAIApiInstanceName}.openai.azure.com/openai/deployments/${azureOpenAIApiDeploymentName}`;
-};
-
-/**
- * Generates the Azure OpenAI API chat completion endpoint URL with the API version.
- * If both deploymentName and modelName are provided, modelName takes precedence.
- * @param {Object} AzureConfig - The Azure configuration object.
- * @param {string} AzureConfig.azureOpenAIApiInstanceName - The Azure OpenAI API instance name.
- * @param {string} [AzureConfig.azureOpenAIApiDeploymentName] - The Azure OpenAI API deployment name (optional).
- * @param {string} AzureConfig.azureOpenAIApiVersion - The Azure OpenAI API version.
- * @param {string} [modelName] - The model name to be included in the deployment name (optional).
- * @param {Object} [client] - The API Client class for optionally setting properties (optional).
- * @returns {string} The complete chat completion endpoint URL for the Azure OpenAI API.
- * @throws {Error} If neither azureOpenAIApiDeploymentName nor modelName is provided.
- */
-const genAzureChatCompletion = (
-  { azureOpenAIApiInstanceName, azureOpenAIApiDeploymentName, azureOpenAIApiVersion },
-  modelName,
-  client,
-) => {
-  // Determine the deployment segment of the URL based on provided modelName or azureOpenAIApiDeploymentName
-  let deploymentSegment;
-  if (isEnabled(process.env.AZURE_USE_MODEL_AS_DEPLOYMENT_NAME) && modelName) {
-    const sanitizedModelName = sanitizeModelName(modelName);
-    deploymentSegment = `${sanitizedModelName}`;
-    client &&
-      typeof client === 'object' &&
-      (client.azure.azureOpenAIApiDeploymentName = sanitizedModelName);
-  } else if (azureOpenAIApiDeploymentName) {
-    deploymentSegment = azureOpenAIApiDeploymentName;
-  } else if (!process.env.AZURE_OPENAI_BASEURL) {
-    throw new Error(
-      'Either a model name with the `AZURE_USE_MODEL_AS_DEPLOYMENT_NAME` setting or a deployment name must be provided if `AZURE_OPENAI_BASEURL` is omitted.',
-    );
-  }
-
-  return `https://${azureOpenAIApiInstanceName}.openai.azure.com/openai/deployments/${deploymentSegment}/chat/completions?api-version=${azureOpenAIApiVersion}`;
-};
-
-/**
- * Retrieves the Azure OpenAI API credentials from environment variables.
- * @returns {AzureOptions} An object containing the Azure OpenAI API credentials.
- */
-const getAzureCredentials = () => {
-  return {
-    azureOpenAIApiKey: process.env.AZURE_API_KEY ?? process.env.AZURE_OPENAI_API_KEY,
-    azureOpenAIApiInstanceName: process.env.AZURE_OPENAI_API_INSTANCE_NAME,
-    azureOpenAIApiDeploymentName: process.env.AZURE_OPENAI_API_DEPLOYMENT_NAME,
-    azureOpenAIApiVersion: process.env.AZURE_OPENAI_API_VERSION,
-  };
-};
-
-/**
- * Constructs a URL by replacing placeholders in the baseURL with values from the azure object.
- * It specifically looks for '${INSTANCE_NAME}' and '${DEPLOYMENT_NAME}' within the baseURL and replaces
- * them with 'azureOpenAIApiInstanceName' and 'azureOpenAIApiDeploymentName' from the azure object.
- * If the respective azure property is not provided, the placeholder is replaced with an empty string.
- *
- * @param {Object} params - The parameters object.
- * @param {string} params.baseURL - The baseURL to inspect for replacement placeholders.
- * @param {AzureOptions} params.azureOptions - The azure options object containing the instance and deployment names.
- * @returns {string} The complete baseURL with credentials injected for the Azure OpenAI API.
- */
-function constructAzureURL({ baseURL, azureOptions }) {
-  let finalURL = baseURL;
-
-  // Replace INSTANCE_NAME and DEPLOYMENT_NAME placeholders with actual values if available
-  if (azureOptions) {
-    finalURL = finalURL.replace('${INSTANCE_NAME}', azureOptions.azureOpenAIApiInstanceName ?? '');
-    finalURL = finalURL.replace(
-      '${DEPLOYMENT_NAME}',
-      azureOptions.azureOpenAIApiDeploymentName ?? '',
-    );
-  }
-
-  return finalURL;
-}
-
-module.exports = {
-  sanitizeModelName,
-  genAzureEndpoint,
-  genAzureChatCompletion,
-  getAzureCredentials,
-  constructAzureURL,
-};

api/utils/azureUtils.spec.js

@@ -1,268 +0,0 @@
-const {
-  sanitizeModelName,
-  genAzureEndpoint,
-  genAzureChatCompletion,
-  getAzureCredentials,
-  constructAzureURL,
-} = require('./azureUtils');
-
-describe('sanitizeModelName', () => {
-  test('removes periods from the model name', () => {
-    const sanitized = sanitizeModelName('model.name');
-    expect(sanitized).toBe('modelname');
-  });
-
-  test('leaves model name unchanged if no periods are present', () => {
-    const sanitized = sanitizeModelName('modelname');
-    expect(sanitized).toBe('modelname');
-  });
-});
-
-describe('genAzureEndpoint', () => {
-  test('generates correct endpoint URL', () => {
-    const url = genAzureEndpoint({
-      azureOpenAIApiInstanceName: 'instanceName',
-      azureOpenAIApiDeploymentName: 'deploymentName',
-    });
-    expect(url).toBe('https://instanceName.openai.azure.com/openai/deployments/deploymentName');
-  });
-});
-
-describe('genAzureChatCompletion', () => {
-  // Test with both deployment name and model name provided
-  test('prefers model name over deployment name when both are provided and feature enabled', () => {
-    process.env.AZURE_USE_MODEL_AS_DEPLOYMENT_NAME = 'true';
-    const url = genAzureChatCompletion(
-      {
-        azureOpenAIApiInstanceName: 'instanceName',
-        azureOpenAIApiDeploymentName: 'deploymentName',
-        azureOpenAIApiVersion: 'v1',
-      },
-      'modelName',
-    );
-    expect(url).toBe(
-      'https://instanceName.openai.azure.com/openai/deployments/modelName/chat/completions?api-version=v1',
-    );
-  });
-
-  // Test with only deployment name provided
-  test('uses deployment name when model name is not provided', () => {
-    const url = genAzureChatCompletion({
-      azureOpenAIApiInstanceName: 'instanceName',
-      azureOpenAIApiDeploymentName: 'deploymentName',
-      azureOpenAIApiVersion: 'v1',
-    });
-    expect(url).toBe(
-      'https://instanceName.openai.azure.com/openai/deployments/deploymentName/chat/completions?api-version=v1',
-    );
-  });
-
-  // Test with only model name provided
-  test('uses model name when deployment name is not provided and feature enabled', () => {
-    process.env.AZURE_USE_MODEL_AS_DEPLOYMENT_NAME = 'true';
-    const url = genAzureChatCompletion(
-      {
-        azureOpenAIApiInstanceName: 'instanceName',
-        azureOpenAIApiVersion: 'v1',
-      },
-      'modelName',
-    );
-    expect(url).toBe(
-      'https://instanceName.openai.azure.com/openai/deployments/modelName/chat/completions?api-version=v1',
-    );
-  });
-
-  // Test with neither deployment name nor model name provided
-  test('throws error if neither deployment name nor model name is provided', () => {
-    expect(() => {
-      genAzureChatCompletion({
-        azureOpenAIApiInstanceName: 'instanceName',
-        azureOpenAIApiVersion: 'v1',
-      });
-    }).toThrow(
-      'Either a model name with the `AZURE_USE_MODEL_AS_DEPLOYMENT_NAME` setting or a deployment name must be provided if `AZURE_OPENAI_BASEURL` is omitted.',
-    );
-  });
-
-  // Test with feature disabled but model name provided
-  test('ignores model name and uses deployment name when feature is disabled', () => {
-    process.env.AZURE_USE_MODEL_AS_DEPLOYMENT_NAME = 'false';
-    const url = genAzureChatCompletion(
-      {
-        azureOpenAIApiInstanceName: 'instanceName',
-        azureOpenAIApiDeploymentName: 'deploymentName',
-        azureOpenAIApiVersion: 'v1',
-      },
-      'modelName',
-    );
-    expect(url).toBe(
-      'https://instanceName.openai.azure.com/openai/deployments/deploymentName/chat/completions?api-version=v1',
-    );
-  });
-
-  // Test with sanitized model name
-  test('sanitizes model name when used in URL', () => {
-    process.env.AZURE_USE_MODEL_AS_DEPLOYMENT_NAME = 'true';
-    const url = genAzureChatCompletion(
-      {
-        azureOpenAIApiInstanceName: 'instanceName',
-        azureOpenAIApiVersion: 'v1',
-      },
-      'model.name',
-    );
-    expect(url).toBe(
-      'https://instanceName.openai.azure.com/openai/deployments/modelname/chat/completions?api-version=v1',
-    );
-  });
-
-  // Test with client parameter and model name
-  test('updates client with sanitized model name when provided and feature enabled', () => {
-    process.env.AZURE_USE_MODEL_AS_DEPLOYMENT_NAME = 'true';
-    const clientMock = { azure: {} };
-    const url = genAzureChatCompletion(
-      {
-        azureOpenAIApiInstanceName: 'instanceName',
-        azureOpenAIApiVersion: 'v1',
-      },
-      'model.name',
-      clientMock,
-    );
-    expect(url).toBe(
-      'https://instanceName.openai.azure.com/openai/deployments/modelname/chat/completions?api-version=v1',
-    );
-    expect(clientMock.azure.azureOpenAIApiDeploymentName).toBe('modelname');
-  });
-
-  // Test with client parameter but without model name
-  test('does not update client when model name is not provided', () => {
-    const clientMock = { azure: {} };
-    const url = genAzureChatCompletion(
-      {
-        azureOpenAIApiInstanceName: 'instanceName',
-        azureOpenAIApiDeploymentName: 'deploymentName',
-        azureOpenAIApiVersion: 'v1',
-      },
-      undefined,
-      clientMock,
-    );
-    expect(url).toBe(
-      'https://instanceName.openai.azure.com/openai/deployments/deploymentName/chat/completions?api-version=v1',
-    );
-    expect(clientMock.azure.azureOpenAIApiDeploymentName).toBeUndefined();
-  });
-
-  // Test with client parameter and deployment name when feature is disabled
-  test('does not update client when feature is disabled', () => {
-    process.env.AZURE_USE_MODEL_AS_DEPLOYMENT_NAME = 'false';
-    const clientMock = { azure: {} };
-    const url = genAzureChatCompletion(
-      {
-        azureOpenAIApiInstanceName: 'instanceName',
-        azureOpenAIApiDeploymentName: 'deploymentName',
-        azureOpenAIApiVersion: 'v1',
-      },
-      'modelName',
-      clientMock,
-    );
-    expect(url).toBe(
-      'https://instanceName.openai.azure.com/openai/deployments/deploymentName/chat/completions?api-version=v1',
-    );
-    expect(clientMock.azure.azureOpenAIApiDeploymentName).toBeUndefined();
-  });
-
-  // Reset environment variable after tests
-  afterEach(() => {
-    delete process.env.AZURE_USE_MODEL_AS_DEPLOYMENT_NAME;
-  });
-});
-
-describe('getAzureCredentials', () => {
-  beforeEach(() => {
-    process.env.AZURE_API_KEY = 'testApiKey';
-    process.env.AZURE_OPENAI_API_INSTANCE_NAME = 'instanceName';
-    process.env.AZURE_OPENAI_API_DEPLOYMENT_NAME = 'deploymentName';
-    process.env.AZURE_OPENAI_API_VERSION = 'v1';
-  });
-
-  test('retrieves Azure OpenAI API credentials from environment variables', () => {
-    const credentials = getAzureCredentials();
-    expect(credentials).toEqual({
-      azureOpenAIApiKey: 'testApiKey',
-      azureOpenAIApiInstanceName: 'instanceName',
-      azureOpenAIApiDeploymentName: 'deploymentName',
-      azureOpenAIApiVersion: 'v1',
-    });
-  });
-});
-
-describe('constructAzureURL', () => {
-  test('replaces both placeholders when both properties are provided', () => {
-    const url = constructAzureURL({
-      baseURL: 'https://example.com/${INSTANCE_NAME}/${DEPLOYMENT_NAME}',
-      azureOptions: {
-        azureOpenAIApiInstanceName: 'instance1',
-        azureOpenAIApiDeploymentName: 'deployment1',
-      },
-    });
-    expect(url).toBe('https://example.com/instance1/deployment1');
-  });
-
-  test('replaces only INSTANCE_NAME when only azureOpenAIApiInstanceName is provided', () => {
-    const url = constructAzureURL({
-      baseURL: 'https://example.com/${INSTANCE_NAME}/${DEPLOYMENT_NAME}',
-      azureOptions: {
-        azureOpenAIApiInstanceName: 'instance2',
-      },
-    });
-    expect(url).toBe('https://example.com/instance2/');
-  });
-
-  test('replaces only DEPLOYMENT_NAME when only azureOpenAIApiDeploymentName is provided', () => {
-    const url = constructAzureURL({
-      baseURL: 'https://example.com/${INSTANCE_NAME}/${DEPLOYMENT_NAME}',
-      azureOptions: {
-        azureOpenAIApiDeploymentName: 'deployment2',
-      },
-    });
-    expect(url).toBe('https://example.com//deployment2');
-  });
-
-  test('does not replace any placeholders when azure object is empty', () => {
-    const url = constructAzureURL({
-      baseURL: 'https://example.com/${INSTANCE_NAME}/${DEPLOYMENT_NAME}',
-      azureOptions: {},
-    });
-    expect(url).toBe('https://example.com//');
-  });
-
-  test('returns baseURL as is when `azureOptions` object is not provided', () => {
-    const url = constructAzureURL({
-      baseURL: 'https://example.com/${INSTANCE_NAME}/${DEPLOYMENT_NAME}',
-    });
-    expect(url).toBe('https://example.com/${INSTANCE_NAME}/${DEPLOYMENT_NAME}');
-  });
-
-  test('returns baseURL as is when no placeholders are set', () => {
-    const url = constructAzureURL({
-      baseURL: 'https://example.com/my_custom_instance/my_deployment',
-      azureOptions: {
-        azureOpenAIApiInstanceName: 'instance1',
-        azureOpenAIApiDeploymentName: 'deployment1',
-      },
-    });
-    expect(url).toBe('https://example.com/my_custom_instance/my_deployment');
-  });
-
-  test('returns regular Azure OpenAI baseURL with placeholders set', () => {
-    const baseURL =
-      'https://${INSTANCE_NAME}.openai.azure.com/openai/deployments/${DEPLOYMENT_NAME}';
-    const url = constructAzureURL({
-      baseURL,
-      azureOptions: {
-        azureOpenAIApiInstanceName: 'instance1',
-        azureOpenAIApiDeploymentName: 'deployment1',
-      },
-    });
-    expect(url).toBe('https://instance1.openai.azure.com/openai/deployments/deployment1');
-  });
-});

api/utils/index.js

@@ -1,7 +1,6 @@
 const loadYaml = require('./loadYaml');
 const axiosHelpers = require('./axios');
 const tokenHelpers = require('./tokens');
-const azureUtils = require('./azureUtils');
 const deriveBaseURL = require('./deriveBaseURL');
 const extractBaseURL = require('./extractBaseURL');
 const findMessageContent = require('./findMessageContent');
@@ -10,7 +9,6 @@ module.exports = {
   loadYaml,
   deriveBaseURL,
   extractBaseURL,
-  ...azureUtils,
   ...axiosHelpers,
   ...tokenHelpers,
   findMessageContent,