🧠 feat: User Memories for Conversational Context (#7760)

* 🧠 feat: User Memories for Conversational Context

chore: mcp typing, use `t`

WIP: first pass, Memories UI

- Added MemoryViewer component for displaying, editing, and deleting user memories.
- Integrated data provider hooks for fetching, updating, and deleting memories.
- Implemented pagination and loading states for better user experience.
- Created unit tests for MemoryViewer to ensure functionality and interaction with data provider.
- Updated translation files to include new UI strings related to memories.

chore: move mcp-related files to own directory

chore: rename librechat-mcp to librechat-api

WIP: first pass, memory processing and data schemas

chore: linting in fileSearch.js query description

chore: rename librechat-api to @librechat/api across the project

WIP: first pass, functional memory agent

feat: add MemoryEditDialog and MemoryViewer components for managing user memories

- Introduced MemoryEditDialog for editing memory entries with validation and toast notifications.
- Updated MemoryViewer to support editing and deleting memories, including pagination and loading states.
- Enhanced data provider to handle memory updates with optional original key for better management.
- Added new localization strings for memory-related UI elements.

feat: add memory permissions management

- Implemented memory permissions in the backend, allowing roles to have specific permissions for using, creating, updating, and reading memories.
- Added new API endpoints for updating memory permissions associated with roles.
- Created a new AdminSettings component for managing memory permissions in the frontend.
- Integrated memory permissions into the existing roles and permissions schemas.
- Updated the interface to include memory settings and permissions.
- Enhanced the MemoryViewer component to conditionally render admin settings based on user roles.
- Added localization support for memory permissions in the translation files.

feat: move AdminSettings component to a new position in MemoryViewer for better visibility

refactor: clean up commented code in MemoryViewer component

feat: enhance MemoryViewer with search functionality and improve MemoryEditDialog integration

- Added a search input to filter memories in the MemoryViewer component.
- Refactored MemoryEditDialog to accept children for better customization.
- Updated MemoryViewer to utilize the new EditMemoryButton and DeleteMemoryButton components for editing and deleting memories.
- Improved localization support by adding new strings for memory filtering and deletion confirmation.

refactor: optimize memory filtering in MemoryViewer using match-sorter

- Replaced manual filtering logic with match-sorter for improved search functionality.
- Enhanced performance and readability of the filteredMemories computation.

feat: enhance MemoryEditDialog with triggerRef and improve updateMemory mutation handling

feat: implement access control for MemoryEditDialog and MemoryViewer components

refactor: remove commented out code and create runMemory method

refactor: rename role based files

feat: implement access control for memory usage in AgentClient

refactor: simplify checkVisionRequest method in AgentClient by removing commented-out code

refactor: make `agents` dir in api package

refactor: migrate Azure utilities to TypeScript and consolidate imports

refactor: move sanitizeFilename function to a new file and update imports, add related tests

refactor: update LLM configuration types and consolidate Azure options in the API package

chore: linting

chore: import order

refactor: replace getLLMConfig with getOpenAIConfig and remove unused LLM configuration file

chore: update winston-daily-rotate-file to version 5.0.0 and add object-hash dependency in package-lock.json

refactor: move primeResources and optionalChainWithEmptyCheck functions to resources.ts and update imports

refactor: move createRun function to a new run.ts file and update related imports

fix: ensure safeAttachments is correctly typed as an array of TFile

chore: add node-fetch dependency and refactor fetch-related functions into packages/api/utils, removing the old generators file

refactor: enhance TEndpointOption type by using Pick to streamline endpoint fields and add new properties for model parameters and client options

feat: implement initializeOpenAIOptions function and update OpenAI types for enhanced configuration handling

fix: update types due to new TEndpointOption typing

fix: ensure safe access to group parameters in initializeOpenAIOptions function

fix: remove redundant API key validation comment in initializeOpenAIOptions function

refactor: rename initializeOpenAIOptions to initializeOpenAI for consistency and update related documentation

refactor: decouple req.body fields and tool loading from initializeAgentOptions

chore: linting

refactor: adjust column widths in MemoryViewer for improved layout

refactor: simplify agent initialization by creating loadAgent function and removing unused code

feat: add memory configuration loading and validation functions

WIP: first pass, memory processing with config

feat: implement memory callback and artifact handling

feat: implement memory artifacts display and processing updates

feat: add memory configuration options and schema validation for validKeys

fix: update MemoryEditDialog and MemoryViewer to handle memory state and display improvements

refactor: remove padding from BookmarkTable and MemoryViewer headers for consistent styling

WIP: initial tokenLimit config and move Tokenizer to @librechat/api

refactor: update mongoMeili plugin methods to use callback for better error handling

feat: enhance memory management with token tracking and usage metrics

- Added token counting for memory entries to enforce limits and provide usage statistics.
- Updated memory retrieval and update routes to include total token usage and limit.
- Enhanced MemoryEditDialog and MemoryViewer components to display memory usage and token information.
- Refactored memory processing functions to handle token limits and provide feedback on memory capacity.

feat: implement memory artifact handling in attachment handler

- Enhanced useAttachmentHandler to process memory artifacts when receiving updates.
- Introduced handleMemoryArtifact utility to manage memory updates and deletions.
- Updated query client to reflect changes in memory state based on incoming data.

refactor: restructure web search key extraction logic

- Moved the logic for extracting API keys from the webSearchAuth configuration into a dedicated function, getWebSearchKeys.
- Updated webSearchKeys to utilize the new function for improved clarity and maintainability.
- Prevents build time errors

feat: add personalization settings and memory preferences management

- Introduced a new Personalization tab in settings to manage user memory preferences.
- Implemented API endpoints and client-side logic for updating memory preferences.
- Enhanced user interface components to reflect personalization options and memory usage.
- Updated permissions to allow users to opt out of memory features.
- Added localization support for new settings and messages related to personalization.

style: personalization switch class

feat: add PersonalizationIcon and align Side Panel UI

feat: implement memory creation functionality

- Added a new API endpoint for creating memory entries, including validation for key and value.
- Introduced MemoryCreateDialog component for user interface to facilitate memory creation.
- Integrated token limit checks to prevent exceeding user memory capacity.
- Updated MemoryViewer to include a button for opening the memory creation dialog.
- Enhanced localization support for new messages related to memory creation.

feat: enhance message processing with configurable window size

- Updated AgentClient to use a configurable message window size for processing messages.
- Introduced messageWindowSize option in memory configuration schema with a default value of 5.
- Improved logic for selecting messages to process based on the configured window size.

chore: update librechat-data-provider version to 0.7.87 in package.json and package-lock.json

chore: remove OpenAPIPlugin and its associated tests

chore: remove MIGRATION_README.md as migration tasks are completed

ci: fix backend tests

chore: remove unused translation keys from localization file

chore: remove problematic test file and unused var in AgentClient

chore: remove unused import and import directly for JSDoc

* feat: add api package build stage in Dockerfile for improved modularity

* docs: reorder build steps in contributing guide for clarity

packages/data-schemas/src/methods/index.ts

@@ -2,6 +2,8 @@ import { createUserMethods, type UserMethods } from './user';
 import { createSessionMethods, type SessionMethods } from './session';
 import { createTokenMethods, type TokenMethods } from './token';
 import { createRoleMethods, type RoleMethods } from './role';
+/* Memories */
+import { createMemoryMethods, type MemoryMethods } from './memory';
 
 /**
  * Creates all database methods for all collections
@@ -12,7 +14,9 @@ export function createMethods(mongoose: typeof import('mongoose')) {
     ...createSessionMethods(mongoose),
     ...createTokenMethods(mongoose),
     ...createRoleMethods(mongoose),
+    ...createMemoryMethods(mongoose),
   };
 }
 
-export type AllMethods = UserMethods & SessionMethods & TokenMethods & RoleMethods;
+export type { MemoryMethods };
+export type AllMethods = UserMethods & SessionMethods & TokenMethods & RoleMethods & MemoryMethods;

packages/data-schemas/src/methods/memory.ts

@@ -0,0 +1,168 @@
+import { Types } from 'mongoose';
+import logger from '~/config/winston';
+import type * as t from '~/types';
+
+/**
+ * Formats a date in YYYY-MM-DD format
+ */
+const formatDate = (date: Date): string => {
+  return date.toISOString().split('T')[0];
+};
+
+// Factory function that takes mongoose instance and returns the methods
+export function createMemoryMethods(mongoose: typeof import('mongoose')) {
+  const MemoryEntry = mongoose.models.MemoryEntry;
+
+  /**
+   * Creates a new memory entry for a user
+   * Throws an error if a memory with the same key already exists
+   */
+  async function createMemory({
+    userId,
+    key,
+    value,
+    tokenCount = 0,
+  }: t.SetMemoryParams): Promise<t.MemoryResult> {
+    try {
+      if (key?.toLowerCase() === 'nothing') {
+        return { ok: false };
+      }
+
+      const existingMemory = await MemoryEntry.findOne({ userId, key });
+      if (existingMemory) {
+        throw new Error('Memory with this key already exists');
+      }
+
+      await MemoryEntry.create({
+        userId,
+        key,
+        value,
+        tokenCount,
+        updated_at: new Date(),
+      });
+
+      return { ok: true };
+    } catch (error) {
+      throw new Error(
+        `Failed to create memory: ${error instanceof Error ? error.message : 'Unknown error'}`,
+      );
+    }
+  }
+
+  /**
+   * Sets or updates a memory entry for a user
+   */
+  async function setMemory({
+    userId,
+    key,
+    value,
+    tokenCount = 0,
+  }: t.SetMemoryParams): Promise<t.MemoryResult> {
+    try {
+      if (key?.toLowerCase() === 'nothing') {
+        return { ok: false };
+      }
+
+      await MemoryEntry.findOneAndUpdate(
+        { userId, key },
+        {
+          value,
+          tokenCount,
+          updated_at: new Date(),
+        },
+        {
+          upsert: true,
+          new: true,
+        },
+      );
+
+      return { ok: true };
+    } catch (error) {
+      throw new Error(
+        `Failed to set memory: ${error instanceof Error ? error.message : 'Unknown error'}`,
+      );
+    }
+  }
+
+  /**
+   * Deletes a specific memory entry for a user
+   */
+  async function deleteMemory({ userId, key }: t.DeleteMemoryParams): Promise<t.MemoryResult> {
+    try {
+      const result = await MemoryEntry.findOneAndDelete({ userId, key });
+      return { ok: !!result };
+    } catch (error) {
+      throw new Error(
+        `Failed to delete memory: ${error instanceof Error ? error.message : 'Unknown error'}`,
+      );
+    }
+  }
+
+  /**
+   * Gets all memory entries for a user
+   */
+  async function getAllUserMemories(
+    userId: string | Types.ObjectId,
+  ): Promise<t.IMemoryEntryLean[]> {
+    try {
+      return (await MemoryEntry.find({ userId }).lean()) as t.IMemoryEntryLean[];
+    } catch (error) {
+      throw new Error(
+        `Failed to get all memories: ${error instanceof Error ? error.message : 'Unknown error'}`,
+      );
+    }
+  }
+
+  /**
+   * Gets and formats all memories for a user in two different formats
+   */
+  async function getFormattedMemories({
+    userId,
+  }: t.GetFormattedMemoriesParams): Promise<t.FormattedMemoriesResult> {
+    try {
+      const memories = await getAllUserMemories(userId);
+
+      if (!memories || memories.length === 0) {
+        return { withKeys: '', withoutKeys: '', totalTokens: 0 };
+      }
+
+      const sortedMemories = memories.sort(
+        (a, b) => new Date(a.updated_at!).getTime() - new Date(b.updated_at!).getTime(),
+      );
+
+      const totalTokens = sortedMemories.reduce((sum, memory) => {
+        return sum + (memory.tokenCount || 0);
+      }, 0);
+
+      const withKeys = sortedMemories
+        .map((memory, index) => {
+          const date = formatDate(new Date(memory.updated_at!));
+          const tokenInfo = memory.tokenCount ? ` [${memory.tokenCount} tokens]` : '';
+          return `${index + 1}. [${date}]. ["key": "${memory.key}"]${tokenInfo}. ["value": "${memory.value}"]`;
+        })
+        .join('\n\n');
+
+      const withoutKeys = sortedMemories
+        .map((memory, index) => {
+          const date = formatDate(new Date(memory.updated_at!));
+          return `${index + 1}. [${date}]. ${memory.value}`;
+        })
+        .join('\n\n');
+
+      return { withKeys, withoutKeys, totalTokens };
+    } catch (error) {
+      logger.error('Failed to get formatted memories:', error);
+      return { withKeys: '', withoutKeys: '', totalTokens: 0 };
+    }
+  }
+
+  return {
+    setMemory,
+    createMemory,
+    deleteMemory,
+    getAllUserMemories,
+    getFormattedMemories,
+  };
+}
+
+export type MemoryMethods = ReturnType<typeof createMemoryMethods>;

packages/data-schemas/src/methods/user.ts

@@ -170,6 +170,35 @@ export function createUserMethods(mongoose: typeof import('mongoose')) {
     });
   }
 
+  /**
+   * Update a user's personalization memories setting.
+   * Handles the edge case where the personalization object doesn't exist.
+   */
+  async function toggleUserMemories(
+    userId: string,
+    memoriesEnabled: boolean,
+  ): Promise<IUser | null> {
+    const User = mongoose.models.User;
+
+    // First, ensure the personalization object exists
+    const user = await User.findById(userId);
+    if (!user) {
+      return null;
+    }
+
+    // Use $set to update the nested field, which will create the personalization object if it doesn't exist
+    const updateOperation = {
+      $set: {
+        'personalization.memories': memoriesEnabled,
+      },
+    };
+
+    return (await User.findByIdAndUpdate(userId, updateOperation, {
+      new: true,
+      runValidators: true,
+    }).lean()) as IUser | null;
+  }
+
   // Return all methods
   return {
     findUser,
@@ -179,6 +208,7 @@ export function createUserMethods(mongoose: typeof import('mongoose')) {
     getUserById,
     deleteUserById,
     generateToken,
+    toggleUserMemories,
   };
 }