🧠 feat: User Memories for Conversational Context (#7760)

* 🧠 feat: User Memories for Conversational Context

chore: mcp typing, use `t`

WIP: first pass, Memories UI

- Added MemoryViewer component for displaying, editing, and deleting user memories.
- Integrated data provider hooks for fetching, updating, and deleting memories.
- Implemented pagination and loading states for better user experience.
- Created unit tests for MemoryViewer to ensure functionality and interaction with data provider.
- Updated translation files to include new UI strings related to memories.

chore: move mcp-related files to own directory

chore: rename librechat-mcp to librechat-api

WIP: first pass, memory processing and data schemas

chore: linting in fileSearch.js query description

chore: rename librechat-api to @librechat/api across the project

WIP: first pass, functional memory agent

feat: add MemoryEditDialog and MemoryViewer components for managing user memories

- Introduced MemoryEditDialog for editing memory entries with validation and toast notifications.
- Updated MemoryViewer to support editing and deleting memories, including pagination and loading states.
- Enhanced data provider to handle memory updates with optional original key for better management.
- Added new localization strings for memory-related UI elements.

feat: add memory permissions management

- Implemented memory permissions in the backend, allowing roles to have specific permissions for using, creating, updating, and reading memories.
- Added new API endpoints for updating memory permissions associated with roles.
- Created a new AdminSettings component for managing memory permissions in the frontend.
- Integrated memory permissions into the existing roles and permissions schemas.
- Updated the interface to include memory settings and permissions.
- Enhanced the MemoryViewer component to conditionally render admin settings based on user roles.
- Added localization support for memory permissions in the translation files.

feat: move AdminSettings component to a new position in MemoryViewer for better visibility

refactor: clean up commented code in MemoryViewer component

feat: enhance MemoryViewer with search functionality and improve MemoryEditDialog integration

- Added a search input to filter memories in the MemoryViewer component.
- Refactored MemoryEditDialog to accept children for better customization.
- Updated MemoryViewer to utilize the new EditMemoryButton and DeleteMemoryButton components for editing and deleting memories.
- Improved localization support by adding new strings for memory filtering and deletion confirmation.

refactor: optimize memory filtering in MemoryViewer using match-sorter

- Replaced manual filtering logic with match-sorter for improved search functionality.
- Enhanced performance and readability of the filteredMemories computation.

feat: enhance MemoryEditDialog with triggerRef and improve updateMemory mutation handling

feat: implement access control for MemoryEditDialog and MemoryViewer components

refactor: remove commented out code and create runMemory method

refactor: rename role based files

feat: implement access control for memory usage in AgentClient

refactor: simplify checkVisionRequest method in AgentClient by removing commented-out code

refactor: make `agents` dir in api package

refactor: migrate Azure utilities to TypeScript and consolidate imports

refactor: move sanitizeFilename function to a new file and update imports, add related tests

refactor: update LLM configuration types and consolidate Azure options in the API package

chore: linting

chore: import order

refactor: replace getLLMConfig with getOpenAIConfig and remove unused LLM configuration file

chore: update winston-daily-rotate-file to version 5.0.0 and add object-hash dependency in package-lock.json

refactor: move primeResources and optionalChainWithEmptyCheck functions to resources.ts and update imports

refactor: move createRun function to a new run.ts file and update related imports

fix: ensure safeAttachments is correctly typed as an array of TFile

chore: add node-fetch dependency and refactor fetch-related functions into packages/api/utils, removing the old generators file

refactor: enhance TEndpointOption type by using Pick to streamline endpoint fields and add new properties for model parameters and client options

feat: implement initializeOpenAIOptions function and update OpenAI types for enhanced configuration handling

fix: update types due to new TEndpointOption typing

fix: ensure safe access to group parameters in initializeOpenAIOptions function

fix: remove redundant API key validation comment in initializeOpenAIOptions function

refactor: rename initializeOpenAIOptions to initializeOpenAI for consistency and update related documentation

refactor: decouple req.body fields and tool loading from initializeAgentOptions

chore: linting

refactor: adjust column widths in MemoryViewer for improved layout

refactor: simplify agent initialization by creating loadAgent function and removing unused code

feat: add memory configuration loading and validation functions

WIP: first pass, memory processing with config

feat: implement memory callback and artifact handling

feat: implement memory artifacts display and processing updates

feat: add memory configuration options and schema validation for validKeys

fix: update MemoryEditDialog and MemoryViewer to handle memory state and display improvements

refactor: remove padding from BookmarkTable and MemoryViewer headers for consistent styling

WIP: initial tokenLimit config and move Tokenizer to @librechat/api

refactor: update mongoMeili plugin methods to use callback for better error handling

feat: enhance memory management with token tracking and usage metrics

- Added token counting for memory entries to enforce limits and provide usage statistics.
- Updated memory retrieval and update routes to include total token usage and limit.
- Enhanced MemoryEditDialog and MemoryViewer components to display memory usage and token information.
- Refactored memory processing functions to handle token limits and provide feedback on memory capacity.

feat: implement memory artifact handling in attachment handler

- Enhanced useAttachmentHandler to process memory artifacts when receiving updates.
- Introduced handleMemoryArtifact utility to manage memory updates and deletions.
- Updated query client to reflect changes in memory state based on incoming data.

refactor: restructure web search key extraction logic

- Moved the logic for extracting API keys from the webSearchAuth configuration into a dedicated function, getWebSearchKeys.
- Updated webSearchKeys to utilize the new function for improved clarity and maintainability.
- Prevents build time errors

feat: add personalization settings and memory preferences management

- Introduced a new Personalization tab in settings to manage user memory preferences.
- Implemented API endpoints and client-side logic for updating memory preferences.
- Enhanced user interface components to reflect personalization options and memory usage.
- Updated permissions to allow users to opt out of memory features.
- Added localization support for new settings and messages related to personalization.

style: personalization switch class

feat: add PersonalizationIcon and align Side Panel UI

feat: implement memory creation functionality

- Added a new API endpoint for creating memory entries, including validation for key and value.
- Introduced MemoryCreateDialog component for user interface to facilitate memory creation.
- Integrated token limit checks to prevent exceeding user memory capacity.
- Updated MemoryViewer to include a button for opening the memory creation dialog.
- Enhanced localization support for new messages related to memory creation.

feat: enhance message processing with configurable window size

- Updated AgentClient to use a configurable message window size for processing messages.
- Introduced messageWindowSize option in memory configuration schema with a default value of 5.
- Improved logic for selecting messages to process based on the configured window size.

chore: update librechat-data-provider version to 0.7.87 in package.json and package-lock.json

chore: remove OpenAPIPlugin and its associated tests

chore: remove MIGRATION_README.md as migration tasks are completed

ci: fix backend tests

chore: remove unused translation keys from localization file

chore: remove problematic test file and unused var in AgentClient

chore: remove unused import and import directly for JSDoc

* feat: add api package build stage in Dockerfile for improved modularity

* docs: reorder build steps in contributing guide for clarity

client/src/data-provider/Memories/index.ts

@@ -0,0 +1,2 @@
+/* Memories */
+export * from './queries';

client/src/data-provider/Memories/queries.ts

@@ -0,0 +1,116 @@
+/* Memories */
+import { QueryKeys, MutationKeys, dataService } from 'librechat-data-provider';
+import { useQuery, useQueryClient, useMutation } from '@tanstack/react-query';
+import type {
+  UseQueryOptions,
+  UseMutationOptions,
+  QueryObserverResult,
+} from '@tanstack/react-query';
+import type { TUserMemory, MemoriesResponse } from 'librechat-data-provider';
+
+export const useMemoriesQuery = (
+  config?: UseQueryOptions<MemoriesResponse>,
+): QueryObserverResult<MemoriesResponse> => {
+  return useQuery<MemoriesResponse>([QueryKeys.memories], () => dataService.getMemories(), {
+    refetchOnWindowFocus: false,
+    refetchOnReconnect: false,
+    refetchOnMount: false,
+    ...config,
+  });
+};
+
+export const useDeleteMemoryMutation = () => {
+  const queryClient = useQueryClient();
+  return useMutation((key: string) => dataService.deleteMemory(key), {
+    onSuccess: () => {
+      queryClient.invalidateQueries([QueryKeys.memories]);
+    },
+  });
+};
+
+export type UpdateMemoryParams = { key: string; value: string; originalKey?: string };
+export const useUpdateMemoryMutation = (
+  options?: UseMutationOptions<TUserMemory, Error, UpdateMemoryParams>,
+) => {
+  const queryClient = useQueryClient();
+  return useMutation(
+    ({ key, value, originalKey }: UpdateMemoryParams) =>
+      dataService.updateMemory(key, value, originalKey),
+    {
+      ...options,
+      onSuccess: (...params) => {
+        queryClient.invalidateQueries([QueryKeys.memories]);
+        options?.onSuccess?.(...params);
+      },
+    },
+  );
+};
+
+export type UpdateMemoryPreferencesParams = { memories: boolean };
+export type UpdateMemoryPreferencesResponse = {
+  updated: boolean;
+  preferences: { memories: boolean };
+};
+
+export const useUpdateMemoryPreferencesMutation = (
+  options?: UseMutationOptions<
+    UpdateMemoryPreferencesResponse,
+    Error,
+    UpdateMemoryPreferencesParams
+  >,
+) => {
+  const queryClient = useQueryClient();
+  return useMutation<UpdateMemoryPreferencesResponse, Error, UpdateMemoryPreferencesParams>(
+    [MutationKeys.updateMemoryPreferences],
+    (preferences: UpdateMemoryPreferencesParams) =>
+      dataService.updateMemoryPreferences(preferences),
+    {
+      ...options,
+      onSuccess: (...params) => {
+        queryClient.invalidateQueries([QueryKeys.user]);
+        options?.onSuccess?.(...params);
+      },
+    },
+  );
+};
+
+export type CreateMemoryParams = { key: string; value: string };
+export type CreateMemoryResponse = { created: boolean; memory: TUserMemory };
+
+export const useCreateMemoryMutation = (
+  options?: UseMutationOptions<CreateMemoryResponse, Error, CreateMemoryParams>,
+) => {
+  const queryClient = useQueryClient();
+  return useMutation<CreateMemoryResponse, Error, CreateMemoryParams>(
+    ({ key, value }: CreateMemoryParams) => dataService.createMemory({ key, value }),
+    {
+      ...options,
+      onSuccess: (data, variables, context) => {
+        queryClient.setQueryData<MemoriesResponse>([QueryKeys.memories], (oldData) => {
+          if (!oldData) return oldData;
+
+          const newMemories = [...oldData.memories, data.memory];
+          const totalTokens = newMemories.reduce(
+            (sum, memory) => sum + (memory.tokenCount || 0),
+            0,
+          );
+          const tokenLimit = oldData.tokenLimit;
+          let usagePercentage = oldData.usagePercentage;
+
+          if (tokenLimit && tokenLimit > 0) {
+            usagePercentage = Math.min(100, Math.round((totalTokens / tokenLimit) * 100));
+          }
+
+          return {
+            ...oldData,
+            memories: newMemories,
+            totalTokens,
+            usagePercentage,
+          };
+        });
+
+        options?.onSuccess?.(data, variables, context);
+      },
+    },
+  );
+};

client/src/data-provider/__tests__/memories.test.ts

@@ -0,0 +1,19 @@
+import { dataService as _dataService } from 'librechat-data-provider';
+import axios from 'axios';
+
+jest.mock('axios');
+
+const mockedAxios = axios as jest.Mocked<typeof axios>;
+
+describe('getMemories', () => {
+  it('should fetch memories from /api/memories', async () => {
+    const mockData = [{ key: 'foo', value: 'bar', updated_at: '2024-05-01T00:00:00Z' }];
+
+    mockedAxios.get.mockResolvedValueOnce({ data: mockData } as any);
+
+    const result = await (_dataService as any).getMemories();
+
+    expect(mockedAxios.get).toHaveBeenCalledWith('/api/memories', expect.any(Object));
+    expect(result).toEqual(mockData);
+  });
+});

client/src/data-provider/index.ts

@@ -2,6 +2,8 @@ export * from './Auth';
 export * from './Agents';
 export * from './Endpoints';
 export * from './Files';
+/* Memories */
+export * from './Memories';
 export * from './Messages';
 export * from './Misc';
 export * from './Tools';

client/src/data-provider/roles.ts

@@ -1,10 +1,15 @@
 import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
+import {
+  QueryKeys,
+  dataService,
+  promptPermissionsSchema,
+  memoryPermissionsSchema,
+} from 'librechat-data-provider';
 import type {
+  UseQueryOptions,
   UseMutationResult,
   QueryObserverResult,
-  UseQueryOptions,
 } from '@tanstack/react-query';
-import { QueryKeys, dataService, promptPermissionsSchema } from 'librechat-data-provider';
 import type * as t from 'librechat-data-provider';
 
 export const useGetRole = (
@@ -91,3 +96,39 @@ export const useUpdateAgentPermissionsMutation = (
     },
   );
 };
+
+export const useUpdateMemoryPermissionsMutation = (
+  options?: t.UpdateMemoryPermOptions,
+): UseMutationResult<
+  t.UpdatePermResponse,
+  t.TError | undefined,
+  t.UpdateMemoryPermVars,
+  unknown
+> => {
+  const queryClient = useQueryClient();
+  const { onMutate, onSuccess, onError } = options ?? {};
+  return useMutation(
+    (variables) => {
+      memoryPermissionsSchema.partial().parse(variables.updates);
+      return dataService.updateMemoryPermissions(variables);
+    },
+    {
+      onSuccess: (data, variables, context) => {
+        queryClient.invalidateQueries([QueryKeys.roles, variables.roleName]);
+        if (onSuccess) {
+          onSuccess(data, variables, context);
+        }
+      },
+      onError: (...args) => {
+        const error = args[0];
+        if (error != null) {
+          console.error('Failed to update memory permissions:', error);
+        }
+        if (onError) {
+          onError(...args);
+        }
+      },
+      onMutate,
+    },
+  );
+};