LibreChat/api/app/clients/tools/structured/specs/DALLE3.spec.js
Danny Avila 29ef91b4dd
🧠 feat: User Memories for Conversational Context (#7760)
* 🧠 feat: User Memories for Conversational Context

chore: mcp typing, use `t`

WIP: first pass, Memories UI

- Added MemoryViewer component for displaying, editing, and deleting user memories.
- Integrated data provider hooks for fetching, updating, and deleting memories.
- Implemented pagination and loading states for better user experience.
- Created unit tests for MemoryViewer to ensure functionality and interaction with data provider.
- Updated translation files to include new UI strings related to memories.

chore: move mcp-related files to own directory

chore: rename librechat-mcp to librechat-api

WIP: first pass, memory processing and data schemas

chore: linting in fileSearch.js query description

chore: rename librechat-api to @librechat/api across the project

WIP: first pass, functional memory agent

feat: add MemoryEditDialog and MemoryViewer components for managing user memories

- Introduced MemoryEditDialog for editing memory entries with validation and toast notifications.
- Updated MemoryViewer to support editing and deleting memories, including pagination and loading states.
- Enhanced data provider to handle memory updates with optional original key for better management.
- Added new localization strings for memory-related UI elements.

feat: add memory permissions management

- Implemented memory permissions in the backend, allowing roles to have specific permissions for using, creating, updating, and reading memories.
- Added new API endpoints for updating memory permissions associated with roles.
- Created a new AdminSettings component for managing memory permissions in the frontend.
- Integrated memory permissions into the existing roles and permissions schemas.
- Updated the interface to include memory settings and permissions.
- Enhanced the MemoryViewer component to conditionally render admin settings based on user roles.
- Added localization support for memory permissions in the translation files.

feat: move AdminSettings component to a new position in MemoryViewer for better visibility

refactor: clean up commented code in MemoryViewer component

feat: enhance MemoryViewer with search functionality and improve MemoryEditDialog integration

- Added a search input to filter memories in the MemoryViewer component.
- Refactored MemoryEditDialog to accept children for better customization.
- Updated MemoryViewer to utilize the new EditMemoryButton and DeleteMemoryButton components for editing and deleting memories.
- Improved localization support by adding new strings for memory filtering and deletion confirmation.

refactor: optimize memory filtering in MemoryViewer using match-sorter

- Replaced manual filtering logic with match-sorter for improved search functionality.
- Enhanced performance and readability of the filteredMemories computation.

feat: enhance MemoryEditDialog with triggerRef and improve updateMemory mutation handling

feat: implement access control for MemoryEditDialog and MemoryViewer components

refactor: remove commented out code and create runMemory method

refactor: rename role based files

feat: implement access control for memory usage in AgentClient

refactor: simplify checkVisionRequest method in AgentClient by removing commented-out code

refactor: make `agents` dir in api package

refactor: migrate Azure utilities to TypeScript and consolidate imports

refactor: move sanitizeFilename function to a new file and update imports, add related tests

refactor: update LLM configuration types and consolidate Azure options in the API package

chore: linting

chore: import order

refactor: replace getLLMConfig with getOpenAIConfig and remove unused LLM configuration file

chore: update winston-daily-rotate-file to version 5.0.0 and add object-hash dependency in package-lock.json

refactor: move primeResources and optionalChainWithEmptyCheck functions to resources.ts and update imports

refactor: move createRun function to a new run.ts file and update related imports

fix: ensure safeAttachments is correctly typed as an array of TFile

chore: add node-fetch dependency and refactor fetch-related functions into packages/api/utils, removing the old generators file

refactor: enhance TEndpointOption type by using Pick to streamline endpoint fields and add new properties for model parameters and client options

feat: implement initializeOpenAIOptions function and update OpenAI types for enhanced configuration handling

fix: update types due to new TEndpointOption typing

fix: ensure safe access to group parameters in initializeOpenAIOptions function

fix: remove redundant API key validation comment in initializeOpenAIOptions function

refactor: rename initializeOpenAIOptions to initializeOpenAI for consistency and update related documentation

refactor: decouple req.body fields and tool loading from initializeAgentOptions

chore: linting

refactor: adjust column widths in MemoryViewer for improved layout

refactor: simplify agent initialization by creating loadAgent function and removing unused code

feat: add memory configuration loading and validation functions

WIP: first pass, memory processing with config

feat: implement memory callback and artifact handling

feat: implement memory artifacts display and processing updates

feat: add memory configuration options and schema validation for validKeys

fix: update MemoryEditDialog and MemoryViewer to handle memory state and display improvements

refactor: remove padding from BookmarkTable and MemoryViewer headers for consistent styling

WIP: initial tokenLimit config and move Tokenizer to @librechat/api

refactor: update mongoMeili plugin methods to use callback for better error handling

feat: enhance memory management with token tracking and usage metrics

- Added token counting for memory entries to enforce limits and provide usage statistics.
- Updated memory retrieval and update routes to include total token usage and limit.
- Enhanced MemoryEditDialog and MemoryViewer components to display memory usage and token information.
- Refactored memory processing functions to handle token limits and provide feedback on memory capacity.

feat: implement memory artifact handling in attachment handler

- Enhanced useAttachmentHandler to process memory artifacts when receiving updates.
- Introduced handleMemoryArtifact utility to manage memory updates and deletions.
- Updated query client to reflect changes in memory state based on incoming data.

refactor: restructure web search key extraction logic

- Moved the logic for extracting API keys from the webSearchAuth configuration into a dedicated function, getWebSearchKeys.
- Updated webSearchKeys to utilize the new function for improved clarity and maintainability.
- Prevents build time errors

feat: add personalization settings and memory preferences management

- Introduced a new Personalization tab in settings to manage user memory preferences.
- Implemented API endpoints and client-side logic for updating memory preferences.
- Enhanced user interface components to reflect personalization options and memory usage.
- Updated permissions to allow users to opt out of memory features.
- Added localization support for new settings and messages related to personalization.

style: personalization switch class

feat: add PersonalizationIcon and align Side Panel UI

feat: implement memory creation functionality

- Added a new API endpoint for creating memory entries, including validation for key and value.
- Introduced MemoryCreateDialog component for user interface to facilitate memory creation.
- Integrated token limit checks to prevent exceeding user memory capacity.
- Updated MemoryViewer to include a button for opening the memory creation dialog.
- Enhanced localization support for new messages related to memory creation.

feat: enhance message processing with configurable window size

- Updated AgentClient to use a configurable message window size for processing messages.
- Introduced messageWindowSize option in memory configuration schema with a default value of 5.
- Improved logic for selecting messages to process based on the configured window size.

chore: update librechat-data-provider version to 0.7.87 in package.json and package-lock.json

chore: remove OpenAPIPlugin and its associated tests

chore: remove MIGRATION_README.md as migration tasks are completed

ci: fix backend tests

chore: remove unused translation keys from localization file

chore: remove problematic test file and unused var in AgentClient

chore: remove unused import and import directly for JSDoc

* feat: add api package build stage in Dockerfile for improved modularity

* docs: reorder build steps in contributing guide for clarity
2025-06-07 18:52:22 -04:00

236 lines
6.7 KiB
JavaScript

const OpenAI = require('openai');
const DALLE3 = require('../DALLE3');
const logger = require('~/config/winston');
jest.mock('openai');
jest.mock('@librechat/data-schemas', () => {
return {
logger: {
info: jest.fn(),
warn: jest.fn(),
debug: jest.fn(),
error: jest.fn(),
},
};
});
jest.mock('tiktoken', () => {
return {
encoding_for_model: jest.fn().mockReturnValue({
encode: jest.fn(),
decode: jest.fn(),
}),
};
});
const processFileURL = jest.fn();
jest.mock('~/server/services/Files/images', () => ({
getImageBasename: jest.fn().mockImplementation((url) => {
// Split the URL by '/'
const parts = url.split('/');
// Get the last part of the URL
const lastPart = parts.pop();
// Check if the last part of the URL matches the image extension regex
const imageExtensionRegex = /\.(jpg|jpeg|png|gif|bmp|tiff|svg)$/i;
if (imageExtensionRegex.test(lastPart)) {
return lastPart;
}
// If the regex test fails, return an empty string
return '';
}),
}));
const generate = jest.fn();
OpenAI.mockImplementation(() => ({
images: {
generate,
},
}));
jest.mock('fs', () => {
return {
existsSync: jest.fn(),
mkdirSync: jest.fn(),
promises: {
writeFile: jest.fn(),
readFile: jest.fn(),
unlink: jest.fn(),
},
};
});
jest.mock('path', () => {
return {
resolve: jest.fn(),
join: jest.fn(),
relative: jest.fn(),
extname: jest.fn().mockImplementation((filename) => {
return filename.slice(filename.lastIndexOf('.'));
}),
};
});
describe('DALLE3', () => {
let originalEnv;
let dalle; // Keep this declaration if you need to use dalle in other tests
const mockApiKey = 'mock_api_key';
beforeAll(() => {
// Save the original process.env
originalEnv = { ...process.env };
});
beforeEach(() => {
// Reset the process.env before each test
jest.resetModules();
process.env = { ...originalEnv, DALLE_API_KEY: mockApiKey };
// Instantiate DALLE3 for tests that do not depend on DALLE3_SYSTEM_PROMPT
dalle = new DALLE3({ processFileURL });
});
afterEach(() => {
jest.clearAllMocks();
// Restore the original process.env after each test
process.env = originalEnv;
});
it('should throw an error if all potential API keys are missing', () => {
delete process.env.DALLE3_API_KEY;
delete process.env.DALLE_API_KEY;
expect(() => new DALLE3()).toThrow('Missing DALLE_API_KEY environment variable.');
});
it('should replace unwanted characters in input string', () => {
const input = 'This is a test\nstring with "quotes" and new lines.';
const expectedOutput = 'This is a test string with quotes and new lines.';
expect(dalle.replaceUnwantedChars(input)).toBe(expectedOutput);
});
it('should generate markdown image URL correctly', () => {
const imageName = 'test.png';
const markdownImage = dalle.wrapInMarkdown(imageName);
expect(markdownImage).toBe('![generated image](test.png)');
});
it('should call OpenAI API with correct parameters', async () => {
const mockData = {
prompt: 'A test prompt',
quality: 'standard',
size: '1024x1024',
style: 'vivid',
};
const mockResponse = {
data: [
{
url: 'http://example.com/img-test.png',
},
],
};
generate.mockResolvedValue(mockResponse);
processFileURL.mockResolvedValue({
filepath: 'http://example.com/img-test.png',
});
const result = await dalle._call(mockData);
expect(generate).toHaveBeenCalledWith({
model: 'dall-e-3',
quality: mockData.quality,
style: mockData.style,
size: mockData.size,
prompt: mockData.prompt,
n: 1,
});
expect(result).toContain('![generated image]');
});
it('should use the system prompt if provided', () => {
process.env.DALLE3_SYSTEM_PROMPT = 'System prompt for testing';
jest.resetModules(); // This will ensure the module is fresh and will read the new env var
const DALLE3 = require('../DALLE3'); // Re-require after setting the env var
const dalleWithSystemPrompt = new DALLE3();
expect(dalleWithSystemPrompt.description_for_model).toBe('System prompt for testing');
});
it('should not use the system prompt if not provided', async () => {
delete process.env.DALLE3_SYSTEM_PROMPT;
const dalleWithoutSystemPrompt = new DALLE3();
expect(dalleWithoutSystemPrompt.description_for_model).not.toBe('System prompt for testing');
});
it('should throw an error if prompt is missing', async () => {
const mockData = {
quality: 'standard',
size: '1024x1024',
style: 'vivid',
};
await expect(dalle._call(mockData)).rejects.toThrow('Missing required field: prompt');
});
it('should log appropriate debug values', async () => {
const mockData = {
prompt: 'A test prompt',
};
const mockResponse = {
data: [
{
url: 'http://example.com/invalid-url',
},
],
};
generate.mockResolvedValue(mockResponse);
await dalle._call(mockData);
expect(logger.debug).toHaveBeenCalledWith('[DALL-E-3]', {
data: { url: 'http://example.com/invalid-url' },
theImageUrl: 'http://example.com/invalid-url',
extension: expect.any(String),
imageBasename: expect.any(String),
imageExt: expect.any(String),
imageName: expect.any(String),
});
});
it('should log an error and return the image URL if there is an error saving the image', async () => {
const mockData = {
prompt: 'A test prompt',
};
const mockResponse = {
data: [
{
url: 'http://example.com/img-test.png',
},
],
};
const error = new Error('Error while saving the image');
generate.mockResolvedValue(mockResponse);
processFileURL.mockRejectedValue(error);
const result = await dalle._call(mockData);
expect(logger.error).toHaveBeenCalledWith('Error while saving the image:', error);
expect(result).toBe('Failed to save the image locally. Error while saving the image');
});
it('should handle error when saving image to Firebase Storage fails', async () => {
const mockData = {
prompt: 'A test prompt',
};
const mockImageUrl = 'http://example.com/img-test.png';
const mockResponse = { data: [{ url: mockImageUrl }] };
const error = new Error('Error while saving to Firebase');
generate.mockResolvedValue(mockResponse);
processFileURL.mockRejectedValue(error);
const result = await dalle._call(mockData);
expect(logger.error).toHaveBeenCalledWith('Error while saving the image:', error);
expect(result).toContain('Failed to save the image');
});
});