Andreas/LibreChat
1
0
Fork 0
mirror of https://github.com/danny-avila/LibreChat.git synced 2025-12-22 11:20:15 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

🧠 feat: User Memories for Conversational Context (#7760)

Browse source 
* 🧠 feat: User Memories for Conversational Context

chore: mcp typing, use `t`

WIP: first pass, Memories UI

- Added MemoryViewer component for displaying, editing, and deleting user memories.
- Integrated data provider hooks for fetching, updating, and deleting memories.
- Implemented pagination and loading states for better user experience.
- Created unit tests for MemoryViewer to ensure functionality and interaction with data provider.
- Updated translation files to include new UI strings related to memories.

chore: move mcp-related files to own directory

chore: rename librechat-mcp to librechat-api

WIP: first pass, memory processing and data schemas

chore: linting in fileSearch.js query description

chore: rename librechat-api to @librechat/api across the project

WIP: first pass, functional memory agent

feat: add MemoryEditDialog and MemoryViewer components for managing user memories

- Introduced MemoryEditDialog for editing memory entries with validation and toast notifications.
- Updated MemoryViewer to support editing and deleting memories, including pagination and loading states.
- Enhanced data provider to handle memory updates with optional original key for better management.
- Added new localization strings for memory-related UI elements.

feat: add memory permissions management

- Implemented memory permissions in the backend, allowing roles to have specific permissions for using, creating, updating, and reading memories.
- Added new API endpoints for updating memory permissions associated with roles.
- Created a new AdminSettings component for managing memory permissions in the frontend.
- Integrated memory permissions into the existing roles and permissions schemas.
- Updated the interface to include memory settings and permissions.
- Enhanced the MemoryViewer component to conditionally render admin settings based on user roles.
- Added localization support for memory permissions in the translation files.

feat: move AdminSettings component to a new position in MemoryViewer for better visibility

refactor: clean up commented code in MemoryViewer component

feat: enhance MemoryViewer with search functionality and improve MemoryEditDialog integration

- Added a search input to filter memories in the MemoryViewer component.
- Refactored MemoryEditDialog to accept children for better customization.
- Updated MemoryViewer to utilize the new EditMemoryButton and DeleteMemoryButton components for editing and deleting memories.
- Improved localization support by adding new strings for memory filtering and deletion confirmation.

refactor: optimize memory filtering in MemoryViewer using match-sorter

- Replaced manual filtering logic with match-sorter for improved search functionality.
- Enhanced performance and readability of the filteredMemories computation.

feat: enhance MemoryEditDialog with triggerRef and improve updateMemory mutation handling

feat: implement access control for MemoryEditDialog and MemoryViewer components

refactor: remove commented out code and create runMemory method

refactor: rename role based files

feat: implement access control for memory usage in AgentClient

refactor: simplify checkVisionRequest method in AgentClient by removing commented-out code

refactor: make `agents` dir in api package

refactor: migrate Azure utilities to TypeScript and consolidate imports

refactor: move sanitizeFilename function to a new file and update imports, add related tests

refactor: update LLM configuration types and consolidate Azure options in the API package

chore: linting

chore: import order

refactor: replace getLLMConfig with getOpenAIConfig and remove unused LLM configuration file

chore: update winston-daily-rotate-file to version 5.0.0 and add object-hash dependency in package-lock.json

refactor: move primeResources and optionalChainWithEmptyCheck functions to resources.ts and update imports

refactor: move createRun function to a new run.ts file and update related imports

fix: ensure safeAttachments is correctly typed as an array of TFile

chore: add node-fetch dependency and refactor fetch-related functions into packages/api/utils, removing the old generators file

refactor: enhance TEndpointOption type by using Pick to streamline endpoint fields and add new properties for model parameters and client options

feat: implement initializeOpenAIOptions function and update OpenAI types for enhanced configuration handling

fix: update types due to new TEndpointOption typing

fix: ensure safe access to group parameters in initializeOpenAIOptions function

fix: remove redundant API key validation comment in initializeOpenAIOptions function

refactor: rename initializeOpenAIOptions to initializeOpenAI for consistency and update related documentation

refactor: decouple req.body fields and tool loading from initializeAgentOptions

chore: linting

refactor: adjust column widths in MemoryViewer for improved layout

refactor: simplify agent initialization by creating loadAgent function and removing unused code

feat: add memory configuration loading and validation functions

WIP: first pass, memory processing with config

feat: implement memory callback and artifact handling

feat: implement memory artifacts display and processing updates

feat: add memory configuration options and schema validation for validKeys

fix: update MemoryEditDialog and MemoryViewer to handle memory state and display improvements

refactor: remove padding from BookmarkTable and MemoryViewer headers for consistent styling

WIP: initial tokenLimit config and move Tokenizer to @librechat/api

refactor: update mongoMeili plugin methods to use callback for better error handling

feat: enhance memory management with token tracking and usage metrics

- Added token counting for memory entries to enforce limits and provide usage statistics.
- Updated memory retrieval and update routes to include total token usage and limit.
- Enhanced MemoryEditDialog and MemoryViewer components to display memory usage and token information.
- Refactored memory processing functions to handle token limits and provide feedback on memory capacity.

feat: implement memory artifact handling in attachment handler

- Enhanced useAttachmentHandler to process memory artifacts when receiving updates.
- Introduced handleMemoryArtifact utility to manage memory updates and deletions.
- Updated query client to reflect changes in memory state based on incoming data.

refactor: restructure web search key extraction logic

- Moved the logic for extracting API keys from the webSearchAuth configuration into a dedicated function, getWebSearchKeys.
- Updated webSearchKeys to utilize the new function for improved clarity and maintainability.
- Prevents build time errors

feat: add personalization settings and memory preferences management

- Introduced a new Personalization tab in settings to manage user memory preferences.
- Implemented API endpoints and client-side logic for updating memory preferences.
- Enhanced user interface components to reflect personalization options and memory usage.
- Updated permissions to allow users to opt out of memory features.
- Added localization support for new settings and messages related to personalization.

style: personalization switch class

feat: add PersonalizationIcon and align Side Panel UI

feat: implement memory creation functionality

- Added a new API endpoint for creating memory entries, including validation for key and value.
- Introduced MemoryCreateDialog component for user interface to facilitate memory creation.
- Integrated token limit checks to prevent exceeding user memory capacity.
- Updated MemoryViewer to include a button for opening the memory creation dialog.
- Enhanced localization support for new messages related to memory creation.

feat: enhance message processing with configurable window size

- Updated AgentClient to use a configurable message window size for processing messages.
- Introduced messageWindowSize option in memory configuration schema with a default value of 5.
- Improved logic for selecting messages to process based on the configured window size.

chore: update librechat-data-provider version to 0.7.87 in package.json and package-lock.json

chore: remove OpenAPIPlugin and its associated tests

chore: remove MIGRATION_README.md as migration tasks are completed

ci: fix backend tests

chore: remove unused translation keys from localization file

chore: remove problematic test file and unused var in AgentClient

chore: remove unused import and import directly for JSDoc

* feat: add api package build stage in Dockerfile for improved modularity

* docs: reorder build steps in contributing guide for clarity
This commit is contained in:
Danny Avila 2025-06-07 18:52:22 -04:00 • committed by GitHub
parent cd7dd576c1
commit 29ef91b4dd
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
170 changed files with 5700 additions and 3632 deletions
Download patch file Download diff file Expand all files Collapse all files

269
packages/api/src/utils/azure.spec.ts Normal file
View file

@ -0,0 +1,269 @@
import {
genAzureChatCompletion,
getAzureCredentials,
constructAzureURL,
sanitizeModelName,
genAzureEndpoint,
} from './azure';
import type { GenericClient } from '~/types';
describe('sanitizeModelName', () => {
test('removes periods from the model name', () => {
const sanitized = sanitizeModelName('model.name');
expect(sanitized).toBe('modelname');
});
test('leaves model name unchanged if no periods are present', () => {
const sanitized = sanitizeModelName('modelname');
expect(sanitized).toBe('modelname');
});
});
describe('genAzureEndpoint', () => {
test('generates correct endpoint URL', () => {
const url = genAzureEndpoint({
azureOpenAIApiInstanceName: 'instanceName',
azureOpenAIApiDeploymentName: 'deploymentName',
});
expect(url).toBe('https://instanceName.openai.azure.com/openai/deployments/deploymentName');
});
});
describe('genAzureChatCompletion', () => {
// Test with both deployment name and model name provided
test('prefers model name over deployment name when both are provided and feature enabled', () => {
process.env.AZURE_USE_MODEL_AS_DEPLOYMENT_NAME = 'true';
const url = genAzureChatCompletion(
{
azureOpenAIApiInstanceName: 'instanceName',
azureOpenAIApiDeploymentName: 'deploymentName',
azureOpenAIApiVersion: 'v1',
},
'modelName',
);
expect(url).toBe(
'https://instanceName.openai.azure.com/openai/deployments/modelName/chat/completions?api-version=v1',
);
});
// Test with only deployment name provided
test('uses deployment name when model name is not provided', () => {
const url = genAzureChatCompletion({
azureOpenAIApiInstanceName: 'instanceName',
azureOpenAIApiDeploymentName: 'deploymentName',
azureOpenAIApiVersion: 'v1',
});
expect(url).toBe(
'https://instanceName.openai.azure.com/openai/deployments/deploymentName/chat/completions?api-version=v1',
);
});
// Test with only model name provided
test('uses model name when deployment name is not provided and feature enabled', () => {
process.env.AZURE_USE_MODEL_AS_DEPLOYMENT_NAME = 'true';
const url = genAzureChatCompletion(
{
azureOpenAIApiInstanceName: 'instanceName',
azureOpenAIApiVersion: 'v1',
},
'modelName',
);
expect(url).toBe(
'https://instanceName.openai.azure.com/openai/deployments/modelName/chat/completions?api-version=v1',
);
});
// Test with neither deployment name nor model name provided
test('throws error if neither deployment name nor model name is provided', () => {
expect(() => {
genAzureChatCompletion({
azureOpenAIApiInstanceName: 'instanceName',
azureOpenAIApiVersion: 'v1',
});
}).toThrow(
'Either a model name with the `AZURE_USE_MODEL_AS_DEPLOYMENT_NAME` setting or a deployment name must be provided if `AZURE_OPENAI_BASEURL` is omitted.',
);
});
// Test with feature disabled but model name provided
test('ignores model name and uses deployment name when feature is disabled', () => {
process.env.AZURE_USE_MODEL_AS_DEPLOYMENT_NAME = 'false';
const url = genAzureChatCompletion(
{
azureOpenAIApiInstanceName: 'instanceName',
azureOpenAIApiDeploymentName: 'deploymentName',
azureOpenAIApiVersion: 'v1',
},
'modelName',
);
expect(url).toBe(
'https://instanceName.openai.azure.com/openai/deployments/deploymentName/chat/completions?api-version=v1',
);
});
// Test with sanitized model name
test('sanitizes model name when used in URL', () => {
process.env.AZURE_USE_MODEL_AS_DEPLOYMENT_NAME = 'true';
const url = genAzureChatCompletion(
{
azureOpenAIApiInstanceName: 'instanceName',
azureOpenAIApiVersion: 'v1',
},
'model.name',
);
expect(url).toBe(
'https://instanceName.openai.azure.com/openai/deployments/modelname/chat/completions?api-version=v1',
);
});
// Test with client parameter and model name
test('updates client with sanitized model name when provided and feature enabled', () => {
process.env.AZURE_USE_MODEL_AS_DEPLOYMENT_NAME = 'true';
const clientMock = { azure: {} } as GenericClient;
const url = genAzureChatCompletion(
{
azureOpenAIApiInstanceName: 'instanceName',
azureOpenAIApiVersion: 'v1',
},
'model.name',
clientMock,
);
expect(url).toBe(
'https://instanceName.openai.azure.com/openai/deployments/modelname/chat/completions?api-version=v1',
);
expect(clientMock.azure.azureOpenAIApiDeploymentName).toBe('modelname');
});
// Test with client parameter but without model name
test('does not update client when model name is not provided', () => {
const clientMock = { azure: {} } as GenericClient;
const url = genAzureChatCompletion(
{
azureOpenAIApiInstanceName: 'instanceName',
azureOpenAIApiDeploymentName: 'deploymentName',
azureOpenAIApiVersion: 'v1',
},
undefined,
clientMock,
);
expect(url).toBe(
'https://instanceName.openai.azure.com/openai/deployments/deploymentName/chat/completions?api-version=v1',
);
expect(clientMock.azure.azureOpenAIApiDeploymentName).toBeUndefined();
});
// Test with client parameter and deployment name when feature is disabled
test('does not update client when feature is disabled', () => {
process.env.AZURE_USE_MODEL_AS_DEPLOYMENT_NAME = 'false';
const clientMock = { azure: {} } as GenericClient;
const url = genAzureChatCompletion(
{
azureOpenAIApiInstanceName: 'instanceName',
azureOpenAIApiDeploymentName: 'deploymentName',
azureOpenAIApiVersion: 'v1',
},
'modelName',
clientMock,
);
expect(url).toBe(
'https://instanceName.openai.azure.com/openai/deployments/deploymentName/chat/completions?api-version=v1',
);
expect(clientMock.azure.azureOpenAIApiDeploymentName).toBeUndefined();
});
// Reset environment variable after tests
afterEach(() => {
delete process.env.AZURE_USE_MODEL_AS_DEPLOYMENT_NAME;
});
});
describe('getAzureCredentials', () => {
beforeEach(() => {
process.env.AZURE_API_KEY = 'testApiKey';
process.env.AZURE_OPENAI_API_INSTANCE_NAME = 'instanceName';
process.env.AZURE_OPENAI_API_DEPLOYMENT_NAME = 'deploymentName';
process.env.AZURE_OPENAI_API_VERSION = 'v1';
});
test('retrieves Azure OpenAI API credentials from environment variables', () => {
const credentials = getAzureCredentials();
expect(credentials).toEqual({
azureOpenAIApiKey: 'testApiKey',
azureOpenAIApiInstanceName: 'instanceName',
azureOpenAIApiDeploymentName: 'deploymentName',
azureOpenAIApiVersion: 'v1',
});
});
});
describe('constructAzureURL', () => {
test('replaces both placeholders when both properties are provided', () => {
const url = constructAzureURL({
baseURL: 'https://example.com/${INSTANCE_NAME}/${DEPLOYMENT_NAME}',
azureOptions: {
azureOpenAIApiInstanceName: 'instance1',
azureOpenAIApiDeploymentName: 'deployment1',
},
});
expect(url).toBe('https://example.com/instance1/deployment1');
});
test('replaces only INSTANCE_NAME when only azureOpenAIApiInstanceName is provided', () => {
const url = constructAzureURL({
baseURL: 'https://example.com/${INSTANCE_NAME}/${DEPLOYMENT_NAME}',
azureOptions: {
azureOpenAIApiInstanceName: 'instance2',
},
});
expect(url).toBe('https://example.com/instance2/');
});
test('replaces only DEPLOYMENT_NAME when only azureOpenAIApiDeploymentName is provided', () => {
const url = constructAzureURL({
baseURL: 'https://example.com/${INSTANCE_NAME}/${DEPLOYMENT_NAME}',
azureOptions: {
azureOpenAIApiDeploymentName: 'deployment2',
},
});
expect(url).toBe('https://example.com//deployment2');
});
test('does not replace any placeholders when azure object is empty', () => {
const url = constructAzureURL({
baseURL: 'https://example.com/${INSTANCE_NAME}/${DEPLOYMENT_NAME}',
azureOptions: {},
});
expect(url).toBe('https://example.com//');
});
test('returns baseURL as is when `azureOptions` object is not provided', () => {
const url = constructAzureURL({
baseURL: 'https://example.com/${INSTANCE_NAME}/${DEPLOYMENT_NAME}',
});
expect(url).toBe('https://example.com/${INSTANCE_NAME}/${DEPLOYMENT_NAME}');
});
test('returns baseURL as is when no placeholders are set', () => {
const url = constructAzureURL({
baseURL: 'https://example.com/my_custom_instance/my_deployment',
azureOptions: {
azureOpenAIApiInstanceName: 'instance1',
azureOpenAIApiDeploymentName: 'deployment1',
},
});
expect(url).toBe('https://example.com/my_custom_instance/my_deployment');
});
test('returns regular Azure OpenAI baseURL with placeholders set', () => {
const baseURL =
'https://${INSTANCE_NAME}.openai.azure.com/openai/deployments/${DEPLOYMENT_NAME}';
const url = constructAzureURL({
baseURL,
azureOptions: {
azureOpenAIApiInstanceName: 'instance1',
azureOpenAIApiDeploymentName: 'deployment1',
},
});
expect(url).toBe('https://instance1.openai.azure.com/openai/deployments/deployment1');
});
});

120
packages/api/src/utils/azure.ts Normal file
View file

@ -0,0 +1,120 @@
import { isEnabled } from './common';
import type { AzureOptions, GenericClient } from '~/types';
/**
* Sanitizes the model name to be used in the URL by removing or replacing disallowed characters.
* @param modelName - The model name to be sanitized.
* @returns The sanitized model name.
*/
export const sanitizeModelName = (modelName: string): string => {
// Replace periods with empty strings and other disallowed characters as needed.
return modelName.replace(/\./g, '');
};
/**
* Generates the Azure OpenAI API endpoint URL.
* @param params - The parameters object.
* @param params.azureOpenAIApiInstanceName - The Azure OpenAI API instance name.
* @param params.azureOpenAIApiDeploymentName - The Azure OpenAI API deployment name.
* @returns The complete endpoint URL for the Azure OpenAI API.
*/
export const genAzureEndpoint = ({
azureOpenAIApiInstanceName,
azureOpenAIApiDeploymentName,
}: {
azureOpenAIApiInstanceName: string;
azureOpenAIApiDeploymentName: string;
}): string => {
return `https://${azureOpenAIApiInstanceName}.openai.azure.com/openai/deployments/${azureOpenAIApiDeploymentName}`;
};
/**
* Generates the Azure OpenAI API chat completion endpoint URL with the API version.
* If both deploymentName and modelName are provided, modelName takes precedence.
* @param azureConfig - The Azure configuration object.
* @param azureConfig.azureOpenAIApiInstanceName - The Azure OpenAI API instance name.
* @param azureConfig.azureOpenAIApiDeploymentName - The Azure OpenAI API deployment name (optional).
* @param azureConfig.azureOpenAIApiVersion - The Azure OpenAI API version.
* @param modelName - The model name to be included in the deployment name (optional).
* @param client - The API Client class for optionally setting properties (optional).
* @returns The complete chat completion endpoint URL for the Azure OpenAI API.
* @throws Error if neither azureOpenAIApiDeploymentName nor modelName is provided.
*/
export const genAzureChatCompletion = (
{
azureOpenAIApiInstanceName,
azureOpenAIApiDeploymentName,
azureOpenAIApiVersion,
}: {
azureOpenAIApiInstanceName: string;
azureOpenAIApiDeploymentName?: string;
azureOpenAIApiVersion: string;
},
modelName?: string,
client?: GenericClient,
): string => {
// Determine the deployment segment of the URL based on provided modelName or azureOpenAIApiDeploymentName
let deploymentSegment: string;
if (isEnabled(process.env.AZURE_USE_MODEL_AS_DEPLOYMENT_NAME) && modelName) {
const sanitizedModelName = sanitizeModelName(modelName);
deploymentSegment = sanitizedModelName;
if (client && typeof client === 'object') {
client.azure.azureOpenAIApiDeploymentName = sanitizedModelName;
}
} else if (azureOpenAIApiDeploymentName) {
deploymentSegment = azureOpenAIApiDeploymentName;
} else if (!process.env.AZURE_OPENAI_BASEURL) {
throw new Error(
'Either a model name with the `AZURE_USE_MODEL_AS_DEPLOYMENT_NAME` setting or a deployment name must be provided if `AZURE_OPENAI_BASEURL` is omitted.',
);
} else {
deploymentSegment = '';
}
return `https://${azureOpenAIApiInstanceName}.openai.azure.com/openai/deployments/${deploymentSegment}/chat/completions?api-version=${azureOpenAIApiVersion}`;
};
/**
* Retrieves the Azure OpenAI API credentials from environment variables.
* @returns An object containing the Azure OpenAI API credentials.
*/
export const getAzureCredentials = (): AzureOptions => {
return {
azureOpenAIApiKey: process.env.AZURE_API_KEY ?? process.env.AZURE_OPENAI_API_KEY,
azureOpenAIApiInstanceName: process.env.AZURE_OPENAI_API_INSTANCE_NAME,
azureOpenAIApiDeploymentName: process.env.AZURE_OPENAI_API_DEPLOYMENT_NAME,
azureOpenAIApiVersion: process.env.AZURE_OPENAI_API_VERSION,
};
};
/**
* Constructs a URL by replacing placeholders in the baseURL with values from the azure object.
* It specifically looks for '${INSTANCE_NAME}' and '${DEPLOYMENT_NAME}' within the baseURL and replaces
* them with 'azureOpenAIApiInstanceName' and 'azureOpenAIApiDeploymentName' from the azure object.
* If the respective azure property is not provided, the placeholder is replaced with an empty string.
*
* @param params - The parameters object.
* @param params.baseURL - The baseURL to inspect for replacement placeholders.
* @param params.azureOptions - The azure options object containing the instance and deployment names.
* @returns The complete baseURL with credentials injected for the Azure OpenAI API.
*/
export function constructAzureURL({
baseURL,
azureOptions,
}: {
baseURL: string;
azureOptions?: AzureOptions;
}): string {
let finalURL = baseURL;
// Replace INSTANCE_NAME and DEPLOYMENT_NAME placeholders with actual values if available
if (azureOptions) {
finalURL = finalURL.replace('${INSTANCE_NAME}', azureOptions.azureOpenAIApiInstanceName ?? '');
finalURL = finalURL.replace(
'${DEPLOYMENT_NAME}',
azureOptions.azureOpenAIApiDeploymentName ?? '',
);
}
return finalURL;
}

55
packages/api/src/utils/common.spec.ts Normal file
View file

@ -0,0 +1,55 @@
/* eslint-disable @typescript-eslint/ban-ts-comment */
import { isEnabled } from './common';
describe('isEnabled', () => {
test('should return true when input is "true"', () => {
expect(isEnabled('true')).toBe(true);
});
test('should return true when input is "TRUE"', () => {
expect(isEnabled('TRUE')).toBe(true);
});
test('should return true when input is true', () => {
expect(isEnabled(true)).toBe(true);
});
test('should return false when input is "false"', () => {
expect(isEnabled('false')).toBe(false);
});
test('should return false when input is false', () => {
expect(isEnabled(false)).toBe(false);
});
test('should return false when input is null', () => {
expect(isEnabled(null)).toBe(false);
});
test('should return false when input is undefined', () => {
expect(isEnabled()).toBe(false);
});
test('should return false when input is an empty string', () => {
expect(isEnabled('')).toBe(false);
});
test('should return false when input is a whitespace string', () => {
expect(isEnabled(' ')).toBe(false);
});
test('should return false when input is a number', () => {
// @ts-expect-error
expect(isEnabled(123)).toBe(false);
});
test('should return false when input is an object', () => {
// @ts-expect-error
expect(isEnabled({})).toBe(false);
});
test('should return false when input is an array', () => {
// @ts-expect-error
expect(isEnabled([])).toBe(false);
});
});

48
packages/api/src/utils/common.ts Normal file
View file

@ -0,0 +1,48 @@
/**
* Checks if the given value is truthy by being either the boolean `true` or a string
* that case-insensitively matches 'true'.
*
* @param value - The value to check.
* @returns Returns `true` if the value is the boolean `true` or a case-insensitive
* match for the string 'true', otherwise returns `false`.
* @example
*
* isEnabled("True"); // returns true
* isEnabled("TRUE"); // returns true
* isEnabled(true); // returns true
* isEnabled("false"); // returns false
* isEnabled(false); // returns false
* isEnabled(null); // returns false
* isEnabled(); // returns false
*/
export function isEnabled(value?: string | boolean | null | undefined): boolean {
if (typeof value === 'boolean') {
return value;
}
if (typeof value === 'string') {
return value.toLowerCase().trim() === 'true';
}
return false;
}
/**
* Checks if the provided value is 'user_provided'.
*
* @param value - The value to check.
* @returns - Returns true if the value is 'user_provided', otherwise false.
*/
export const isUserProvided = (value?: string): boolean => value === 'user_provided';
/**
* @param values
*/
export function optionalChainWithEmptyCheck(
...values: (string | number | undefined)[]
): string | number | undefined {
for (const value of values) {
if (value !== undefined && value !== null && value !== '') {
return value;
}
}
return values[values.length - 1];
}

16
packages/api/src/utils/events.ts Normal file
View file

@ -0,0 +1,16 @@
import type { Response as ServerResponse } from 'express';
import type { ServerSentEvent } from '~/types';
/**
* Sends message data in Server Sent Events format.
* @param res - The server response.
* @param event - The message event.
* @param event.event - The type of event.
* @param event.data - The message to be sent.
*/
export function sendEvent(res: ServerResponse, event: ServerSentEvent): void {
if (typeof event.data === 'string' && event.data.length === 0) {
return;
}
res.write(`event: message\ndata: ${JSON.stringify(event)}\n\n`);
}

115
packages/api/src/utils/files.spec.ts Normal file
View file

@ -0,0 +1,115 @@
import { sanitizeFilename } from './files';
jest.mock('node:crypto', () => {
const actualModule = jest.requireActual('node:crypto');
return {
...actualModule,
randomBytes: jest.fn().mockReturnValue(Buffer.from('abc123', 'hex')),
};
});
describe('sanitizeFilename', () => {
test('removes directory components (1/2)', () => {
expect(sanitizeFilename('/path/to/file.txt')).toBe('file.txt');
});
test('removes directory components (2/2)', () => {
expect(sanitizeFilename('../../../../file.txt')).toBe('file.txt');
});
test('replaces non-alphanumeric characters', () => {
expect(sanitizeFilename('file name@#$.txt')).toBe('file_name___.txt');
});
test('preserves dots and hyphens', () => {
expect(sanitizeFilename('file-name.with.dots.txt')).toBe('file-name.with.dots.txt');
});
test('prepends underscore to filenames starting with a dot', () => {
expect(sanitizeFilename('.hiddenfile')).toBe('_.hiddenfile');
});
test('truncates long filenames', () => {
const longName = 'a'.repeat(300) + '.txt';
const result = sanitizeFilename(longName);
expect(result.length).toBe(255);
expect(result).toMatch(/^a+-abc123\.txt$/);
});
test('handles filenames with no extension', () => {
const longName = 'a'.repeat(300);
const result = sanitizeFilename(longName);
expect(result.length).toBe(255);
expect(result).toMatch(/^a+-abc123$/);
});
test('handles empty input', () => {
expect(sanitizeFilename('')).toBe('_');
});
test('handles input with only special characters', () => {
expect(sanitizeFilename('@#$%^&*')).toBe('_______');
});
});
describe('sanitizeFilename with real crypto', () => {
// Temporarily unmock crypto for these tests
beforeAll(() => {
jest.resetModules();
jest.unmock('node:crypto');
});
afterAll(() => {
jest.resetModules();
jest.mock('node:crypto', () => {
const actualModule = jest.requireActual('node:crypto');
return {
...actualModule,
randomBytes: jest.fn().mockReturnValue(Buffer.from('abc123', 'hex')),
};
});
});
test('truncates long filenames with real crypto', async () => {
const { sanitizeFilename: realSanitizeFilename } = await import('./files');
const longName = 'b'.repeat(300) + '.pdf';
const result = realSanitizeFilename(longName);
expect(result.length).toBe(255);
expect(result).toMatch(/^b+-[a-f0-9]{6}\.pdf$/);
expect(result.endsWith('.pdf')).toBe(true);
});
test('handles filenames with no extension with real crypto', async () => {
const { sanitizeFilename: realSanitizeFilename } = await import('./files');
const longName = 'c'.repeat(300);
const result = realSanitizeFilename(longName);
expect(result.length).toBe(255);
expect(result).toMatch(/^c+-[a-f0-9]{6}$/);
expect(result).not.toContain('.');
});
test('generates unique suffixes for identical long filenames', async () => {
const { sanitizeFilename: realSanitizeFilename } = await import('./files');
const longName = 'd'.repeat(300) + '.doc';
const result1 = realSanitizeFilename(longName);
const result2 = realSanitizeFilename(longName);
expect(result1.length).toBe(255);
expect(result2.length).toBe(255);
expect(result1).not.toBe(result2); // Should be different due to random suffix
expect(result1.endsWith('.doc')).toBe(true);
expect(result2.endsWith('.doc')).toBe(true);
});
test('real crypto produces valid hex strings', async () => {
const { sanitizeFilename: realSanitizeFilename } = await import('./files');
const longName = 'test'.repeat(100) + '.txt';
const result = realSanitizeFilename(longName);
const hexMatch = result.match(/-([a-f0-9]{6})\.txt$/);
expect(hexMatch).toBeTruthy();
expect(hexMatch![1]).toMatch(/^[a-f0-9]{6}$/);
});
});

33
packages/api/src/utils/files.ts Normal file
View file

@ -0,0 +1,33 @@
import path from 'path';
import crypto from 'node:crypto';
/**
* Sanitize a filename by removing any directory components, replacing non-alphanumeric characters
* @param inputName
*/
export function sanitizeFilename(inputName: string): string {
// Remove any directory components
let name = path.basename(inputName);
// Replace any non-alphanumeric characters except for '.' and '-'
name = name.replace(/[^a-zA-Z0-9.-]/g, '_');
// Ensure the name doesn't start with a dot (hidden file in Unix-like systems)
if (name.startsWith('.') || name === '') {
name = '_' + name;
}
// Limit the length of the filename
const MAX_LENGTH = 255;
if (name.length > MAX_LENGTH) {
const ext = path.extname(name);
const nameWithoutExt = path.basename(name, ext);
name =
nameWithoutExt.slice(0, MAX_LENGTH - ext.length - 7) +
'-' +
crypto.randomBytes(3).toString('hex') +
ext;
}
return name;
}

75
packages/api/src/utils/generators.ts Normal file
View file

@ -0,0 +1,75 @@
import fetch from 'node-fetch';
import { logger } from '@librechat/data-schemas';
import { GraphEvents, sleep } from '@librechat/agents';
import type { Response as ServerResponse } from 'express';
import type { ServerSentEvent } from '~/types';
import { sendEvent } from './events';
/**
* Makes a function to make HTTP request and logs the process.
* @param params
* @param params.directEndpoint - Whether to use a direct endpoint.
* @param params.reverseProxyUrl - The reverse proxy URL to use for the request.
* @returns A promise that resolves to the response of the fetch request.
*/
export function createFetch({
directEndpoint = false,
reverseProxyUrl = '',
}: {
directEndpoint?: boolean;
reverseProxyUrl?: string;
}) {
/**
* Makes an HTTP request and logs the process.
* @param url - The URL to make the request to. Can be a string or a Request object.
* @param init - Optional init options for the request.
* @returns A promise that resolves to the response of the fetch request.
*/
return async function (
_url: fetch.RequestInfo,
init: fetch.RequestInit,
): Promise<fetch.Response> {
let url = _url;
if (directEndpoint) {
url = reverseProxyUrl;
}
logger.debug(`Making request to ${url}`);
if (typeof Bun !== 'undefined') {
return await fetch(url, init);
}
return await fetch(url, init);
};
}
/**
* Creates event handlers for stream events that don't capture client references
* @param res - The response object to send events to
* @returns Object containing handler functions
*/
export function createStreamEventHandlers(res: ServerResponse) {
return {
[GraphEvents.ON_RUN_STEP]: function (event: ServerSentEvent) {
if (res) {
sendEvent(res, event);
}
},
[GraphEvents.ON_MESSAGE_DELTA]: function (event: ServerSentEvent) {
if (res) {
sendEvent(res, event);
}
},
[GraphEvents.ON_REASONING_DELTA]: function (event: ServerSentEvent) {
if (res) {
sendEvent(res, event);
}
},
};
}
export function createHandleLLMNewToken(streamRate: number) {
return async function () {
if (streamRate) {
await sleep(streamRate);
}
};
}

5
packages/api/src/utils/index.ts Normal file
View file

@ -0,0 +1,5 @@
export * from './azure';
export * from './common';
export * from './events';
export * from './generators';
export { default as Tokenizer } from './tokenizer';

143
packages/api/src/utils/tokenizer.spec.ts Normal file
View file

@ -0,0 +1,143 @@
/**
* @file Tokenizer.spec.cjs
*
* Tests the real TokenizerSingleton (no mocking of `tiktoken`).
* Make sure to install `tiktoken` and have it configured properly.
*/
import { logger } from '@librechat/data-schemas';
import type { Tiktoken } from 'tiktoken';
import Tokenizer from './tokenizer';
jest.mock('@librechat/data-schemas', () => ({
logger: {
error: jest.fn(),
},
}));
describe('Tokenizer', () => {
it('should be a singleton (same instance)', async () => {
const AnotherTokenizer = await import('./tokenizer'); // same path
expect(Tokenizer).toBe(AnotherTokenizer.default);
});
describe('getTokenizer', () => {
it('should create an encoder for an explicit model name (e.g., "gpt-4")', () => {
// The real `encoding_for_model` will be called internally
// as soon as we pass isModelName = true.
const tokenizer = Tokenizer.getTokenizer('gpt-4', true);
// Basic sanity checks
expect(tokenizer).toBeDefined();
// You can optionally check certain properties from `tiktoken` if they exist
// e.g., expect(typeof tokenizer.encode).toBe('function');
});
it('should create an encoder for a known encoding (e.g., "cl100k_base")', () => {
// The real `get_encoding` will be called internally
// as soon as we pass isModelName = false.
const tokenizer = Tokenizer.getTokenizer('cl100k_base', false);
expect(tokenizer).toBeDefined();
// e.g., expect(typeof tokenizer.encode).toBe('function');
});
it('should return cached tokenizer if previously fetched', () => {
const tokenizer1 = Tokenizer.getTokenizer('cl100k_base', false);
const tokenizer2 = Tokenizer.getTokenizer('cl100k_base', false);
// Should be the exact same instance from the cache
expect(tokenizer1).toBe(tokenizer2);
});
});
describe('freeAndResetAllEncoders', () => {
beforeEach(() => {
jest.clearAllMocks();
});
it('should free all encoders and reset tokenizerCallsCount to 1', () => {
// By creating two different encodings, we populate the cache
Tokenizer.getTokenizer('cl100k_base', false);
Tokenizer.getTokenizer('r50k_base', false);
// Now free them
Tokenizer.freeAndResetAllEncoders();
// The internal cache is cleared
expect(Tokenizer.tokenizersCache['cl100k_base']).toBeUndefined();
expect(Tokenizer.tokenizersCache['r50k_base']).toBeUndefined();
// tokenizerCallsCount is reset to 1
expect(Tokenizer.tokenizerCallsCount).toBe(1);
});
it('should catch and log errors if freeing fails', () => {
// Mock logger.error before the test
const mockLoggerError = jest.spyOn(logger, 'error');
// Set up a problematic tokenizer in the cache
Tokenizer.tokenizersCache['cl100k_base'] = {
free() {
throw new Error('Intentional free error');
},
} as unknown as Tiktoken;
// Should not throw uncaught errors
Tokenizer.freeAndResetAllEncoders();
// Verify logger.error was called with correct arguments
expect(mockLoggerError).toHaveBeenCalledWith(
'[Tokenizer] Free and reset encoders error',
expect.any(Error),
);
// Clean up
mockLoggerError.mockRestore();
Tokenizer.tokenizersCache = {};
});
});
describe('getTokenCount', () => {
beforeEach(() => {
jest.clearAllMocks();
Tokenizer.freeAndResetAllEncoders();
});
it('should return the number of tokens in the given text', () => {
const text = 'Hello, world!';
const count = Tokenizer.getTokenCount(text, 'cl100k_base');
expect(count).toBeGreaterThan(0);
});
it('should reset encoders if an error is thrown', () => {
// We can simulate an error by temporarily overriding the selected tokenizer's `encode` method.
const tokenizer = Tokenizer.getTokenizer('cl100k_base', false);
const originalEncode = tokenizer.encode;
tokenizer.encode = () => {
throw new Error('Forced error');
};
// Despite the forced error, the code should catch and reset, then re-encode
const count = Tokenizer.getTokenCount('Hello again', 'cl100k_base');
expect(count).toBeGreaterThan(0);
// Restore the original encode
tokenizer.encode = originalEncode;
});
it('should reset tokenizers after 25 calls', () => {
// Spy on freeAndResetAllEncoders
const resetSpy = jest.spyOn(Tokenizer, 'freeAndResetAllEncoders');
// Make 24 calls; should NOT reset yet
for (let i = 0; i < 24; i++) {
Tokenizer.getTokenCount('test text', 'cl100k_base');
}
expect(resetSpy).not.toHaveBeenCalled();
// 25th call triggers the reset
Tokenizer.getTokenCount('the 25th call!', 'cl100k_base');
expect(resetSpy).toHaveBeenCalledTimes(1);
});
});
});

78
packages/api/src/utils/tokenizer.ts Normal file
View file

@ -0,0 +1,78 @@
import { logger } from '@librechat/data-schemas';
import { encoding_for_model as encodingForModel, get_encoding as getEncoding } from 'tiktoken';
import type { Tiktoken, TiktokenModel, TiktokenEncoding } from 'tiktoken';
interface TokenizerOptions {
debug?: boolean;
}
class Tokenizer {
tokenizersCache: Record<string, Tiktoken>;
tokenizerCallsCount: number;
private options?: TokenizerOptions;
constructor() {
this.tokenizersCache = {};
this.tokenizerCallsCount = 0;
}
getTokenizer(
encoding: TiktokenModel | TiktokenEncoding,
isModelName = false,
extendSpecialTokens: Record<string, number> = {},
): Tiktoken {
let tokenizer: Tiktoken;
if (this.tokenizersCache[encoding]) {
tokenizer = this.tokenizersCache[encoding];
} else {
if (isModelName) {
tokenizer = encodingForModel(encoding as TiktokenModel, extendSpecialTokens);
} else {
tokenizer = getEncoding(encoding as TiktokenEncoding, extendSpecialTokens);
}
this.tokenizersCache[encoding] = tokenizer;
}
return tokenizer;
}
freeAndResetAllEncoders(): void {
try {
Object.keys(this.tokenizersCache).forEach((key) => {
if (this.tokenizersCache[key]) {
this.tokenizersCache[key].free();
delete this.tokenizersCache[key];
}
});
this.tokenizerCallsCount = 1;
} catch (error) {
logger.error('[Tokenizer] Free and reset encoders error', error);
}
}
resetTokenizersIfNecessary(): void {
if (this.tokenizerCallsCount >= 25) {
if (this.options?.debug) {
logger.debug('[Tokenizer] freeAndResetAllEncoders: reached 25 encodings, resetting...');
}
this.freeAndResetAllEncoders();
}
this.tokenizerCallsCount++;
}
getTokenCount(text: string, encoding: TiktokenModel | TiktokenEncoding = 'cl100k_base'): number {
this.resetTokenizersIfNecessary();
try {
const tokenizer = this.getTokenizer(encoding);
return tokenizer.encode(text, 'all').length;
} catch (error) {
logger.error('[Tokenizer] Error getting token count:', error);
this.freeAndResetAllEncoders();
const tokenizer = this.getTokenizer(encoding);
return tokenizer.encode(text, 'all').length;
}
}
}
const TokenizerSingleton = new Tokenizer();
export default TokenizerSingleton;