Andreas/LibreChat
1
0
Fork 0
mirror of https://github.com/danny-avila/LibreChat.git synced 2025-12-17 00:40:14 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 6

🧵 refactor: Migrate Endpoint Initialization to TypeScript (#10794)

Browse source 
* refactor: move endpoint initialization methods to typescript

* refactor: move agent init to packages/api

- Introduced `initialize.ts` for agent initialization, including file processing and tool loading.
- Updated `resources.ts` to allow optional appConfig parameter.
- Enhanced endpoint configuration handling in various initialization files to support model parameters.
- Added new artifacts and prompts for React component generation.
- Refactored existing code to improve type safety and maintainability.

* refactor: streamline endpoint initialization and enhance type safety

- Updated initialization functions across various endpoints to use a consistent request structure, replacing `unknown` types with `ServerResponse`.
- Simplified request handling by directly extracting keys from the request body.
- Improved type safety by ensuring user IDs are safely accessed with optional chaining.
- Removed unnecessary parameters and streamlined model options handling for better clarity and maintainability.

* refactor: moved ModelService and extractBaseURL to packages/api

- Added comprehensive tests for the models fetching functionality, covering scenarios for OpenAI, Anthropic, Google, and Ollama models.
- Updated existing endpoint index to include the new models module.
- Enhanced utility functions for URL extraction and model data processing.
- Improved type safety and error handling across the models fetching logic.

* refactor: consolidate utility functions and remove unused files

- Merged `deriveBaseURL` and `extractBaseURL` into the `@librechat/api` module for better organization.
- Removed redundant utility files and their associated tests to streamline the codebase.
- Updated imports across various client files to utilize the new consolidated functions.
- Enhanced overall maintainability by reducing the number of utility modules.

* refactor: replace ModelService references with direct imports from @librechat/api and remove ModelService file

* refactor: move encrypt/decrypt methods and key db methods to data-schemas, use `getProviderConfig` from `@librechat/api`

* chore: remove unused 'res' from options in AgentClient

* refactor: file model imports and methods

- Updated imports in various controllers and services to use the unified file model from '~/models' instead of '~/models/File'.
- Consolidated file-related methods into a new file methods module in the data-schemas package.
- Added comprehensive tests for file methods including creation, retrieval, updating, and deletion.
- Enhanced the initializeAgent function to accept dependency injection for file-related methods.
- Improved error handling and logging in file methods.

* refactor: streamline database method references in agent initialization

* refactor: enhance file method tests and update type references to IMongoFile

* refactor: consolidate database method imports in agent client and initialization

* chore: remove redundant import of initializeAgent from @librechat/api

* refactor: move checkUserKeyExpiry utility to @librechat/api and update references across endpoints

* refactor: move updateUserPlugins logic to user.ts and simplify UserController

* refactor: update imports for user key management and remove UserService

* refactor: remove unused Anthropics and Bedrock endpoint files and clean up imports

* refactor: consolidate and update encryption imports across various files to use @librechat/data-schemas

* chore: update file model mock to use unified import from '~/models'

* chore: import order

* refactor: remove migrated to TS agent.js file and its associated logic from the endpoints

* chore: add reusable function to extract imports from source code in unused-packages workflow

* chore: enhance unused-packages workflow to include @librechat/api dependencies and improve dependency extraction

* chore: improve dependency extraction in unused-packages workflow with enhanced error handling and debugging output

* chore: add detailed debugging output to unused-packages workflow for better visibility into unused dependencies and exclusion lists

* chore: refine subpath handling in unused-packages workflow to correctly process scoped and non-scoped package imports

* chore: clean up unused debug output in unused-packages workflow and reorganize type imports in initialize.ts
This commit is contained in:
Danny Avila 2025-12-03 17:21:41 -05:00
parent 1a11b64266
commit 04a4a2aa44
No known key found for this signature in database
GPG key ID: BF31EEB2C5CA0956
103 changed files with 4135 additions and 2647 deletions
Download patch file Download diff file Expand all files Collapse all files

11
api/server/controllers/TwoFactorController.js
View file

@ -1,11 +1,10 @@
const { encryptV3 } = require('@librechat/api');
const { logger } = require('@librechat/data-schemas');
const { encryptV3, logger } = require('@librechat/data-schemas');
const {
verifyTOTP,
getTOTPSecret,
verifyBackupCode,
generateTOTPSecret,
generateBackupCodes,
generateTOTPSecret,
verifyBackupCode,
getTOTPSecret,
verifyTOTP,
} = require('~/server/services/twoFactorService');
const { getUserById, updateUser } = require('~/models');

11
api/server/controllers/UserController.js
View file

@ -9,9 +9,11 @@ const {
const {
deleteAllUserSessions,
deleteAllSharedLinks,
updateUserPlugins,
deleteUserById,
deleteMessages,
deletePresets,
deleteUserKey,
deleteConvos,
deleteFiles,
updateUser,
@ -31,7 +33,6 @@ const {
User,
} = require('~/db/models');
const { updateUserPluginAuth, deleteUserPluginAuth } = require('~/server/services/PluginService');
const { updateUserPluginsService, deleteUserKey } = require('~/server/services/UserService');
const { verifyEmail, resendVerificationEmail } = require('~/server/services/AuthService');
const { getMCPManager, getFlowStateManager, getMCPServersRegistry } = require('~/config');
const { needsRefresh, getNewS3URL } = require('~/server/services/Files/S3/crud');
@ -114,13 +115,7 @@ const updateUserPluginsController = async (req, res) => {
const { pluginKey, action, auth, isEntityTool } = req.body;
try {
if (!isEntityTool) {
const userPluginsService = await updateUserPluginsService(user, pluginKey, action);
if (userPluginsService instanceof Error) {
logger.error('[userPluginsService]', userPluginsService);
const { status, message } = normalizeHttpError(userPluginsService);
return res.status(status).send({ message });
}
await updateUserPlugins(user._id, user.plugins, pluginKey, action);
}
if (auth == null) {

58
api/server/controllers/agents/client.js
View file

@ -10,7 +10,9 @@ const {
sanitizeTitle,
resolveHeaders,
createSafeUser,
initializeAgent,
getBalanceConfig,
getProviderConfig,
memoryInstructions,
getTransactionsConfig,
createMemoryProcessor,
@ -38,17 +40,16 @@ const {
bedrockInputSchema,
removeNullishValues,
} = require('librechat-data-provider');
const { initializeAgent } = require('~/server/services/Endpoints/agents/agent');
const { spendTokens, spendStructuredTokens } = require('~/models/spendTokens');
const { getFormattedMemories, deleteMemory, setMemory } = require('~/models');
const { encodeAndFormat } = require('~/server/services/Files/images/encode');
const { getProviderConfig } = require('~/server/services/Endpoints');
const { createContextHandlers } = require('~/app/clients/prompts');
const { checkCapability } = require('~/server/services/Config');
const { getConvoFiles } = require('~/models/Conversation');
const BaseClient = require('~/app/clients/BaseClient');
const { getRoleByName } = require('~/models/Role');
const { loadAgent } = require('~/models/Agent');
const { getMCPManager } = require('~/config');
const db = require('~/models');
const omitTitleOptions = new Set([
'stream',
@ -542,18 +543,28 @@ class AgentClient extends BaseClient {
);
}
const agent = await initializeAgent({
req: this.options.req,
res: this.options.res,
agent: prelimAgent,
allowedProviders,
endpointOption: {
endpoint:
prelimAgent.id !== Constants.EPHEMERAL_AGENT_ID
? EModelEndpoint.agents
: memoryConfig.agent?.provider,
const agent = await initializeAgent(
{
req: this.options.req,
res: this.options.res,
agent: prelimAgent,
allowedProviders,
endpointOption: {
endpoint:
prelimAgent.id !== Constants.EPHEMERAL_AGENT_ID
? EModelEndpoint.agents
: memoryConfig.agent?.provider,
},
},
});
{
getConvoFiles,
getFiles: db.getFiles,
getUserKey: db.getUserKey,
updateFilesUsage: db.updateFilesUsage,
getUserKeyValues: db.getUserKeyValues,
getToolFilesByIds: db.getToolFilesByIds,
},
);
if (!agent) {
logger.warn(
@ -588,9 +599,9 @@ class AgentClient extends BaseClient {
messageId,
conversationId,
memoryMethods: {
setMemory,
deleteMemory,
getFormattedMemories,
setMemory: db.setMemory,
deleteMemory: db.deleteMemory,
getFormattedMemories: db.getFormattedMemories,
},
res: this.options.res,
});
@ -1040,7 +1051,7 @@ class AgentClient extends BaseClient {
throw new Error('Run not initialized');
}
const { handleLLMEnd, collected: collectedMetadata } = createMetadataAggregator();
const { req, res, agent } = this.options;
const { req, agent } = this.options;
const appConfig = req.config;
let endpoint = agent.endpoint;
@ -1097,11 +1108,12 @@ class AgentClient extends BaseClient {
const options = await titleProviderConfig.getOptions({
req,
res,
optionsOnly: true,
overrideEndpoint: endpoint,
overrideModel: clientOptions.model,
endpointOption: { model_parameters: clientOptions },
endpoint,
model_parameters: clientOptions,
db: {
getUserKey: db.getUserKey,
getUserKeyValues: db.getUserKeyValues,
},
});
let provider = options.provider ?? titleProviderConfig.overrideProvider ?? agent.provider;

3
api/server/controllers/agents/v1.js
View file

@ -38,14 +38,13 @@ const {
grantPermission,
} = require('~/server/services/PermissionService');
const { getStrategyFunctions } = require('~/server/services/Files/strategies');
const { getCategoriesWithCounts, deleteFileByFilter } = require('~/models');
const { resizeAvatar } = require('~/server/services/Files/images/avatar');
const { getFileStrategy } = require('~/server/utils/getFileStrategy');
const { refreshS3Url } = require('~/server/services/Files/S3/crud');
const { filterFile } = require('~/server/services/Files/process');
const { updateAction, getActions } = require('~/models/Action');
const { getCachedTools } = require('~/server/services/Config');
const { deleteFileByFilter } = require('~/models/File');
const { getCategoriesWithCounts } = require('~/models');
const { getLogStores } = require('~/cache');
const systemTools = {

2
api/server/controllers/assistants/v1.js
View file

@ -9,7 +9,7 @@ const { updateAssistantDoc, getAssistants } = require('~/models/Assistant');
const { getOpenAIClient, fetchAssistants } = require('./helpers');
const { getCachedTools } = require('~/server/services/Config');
const { manifestToolMap } = require('~/app/clients/tools');
const { deleteFileByFilter } = require('~/models/File');
const { deleteFileByFilter } = require('~/models');
/**
* Create an assistant.

2
api/server/middleware/accessResources/fileAccess.js
View file

@ -2,7 +2,7 @@ const { logger } = require('@librechat/data-schemas');
const { PermissionBits, hasPermissions, ResourceType } = require('librechat-data-provider');
const { getEffectivePermissions } = require('~/server/services/PermissionService');
const { getAgents } = require('~/models/Agent');
const { getFiles } = require('~/models/File');
const { getFiles } = require('~/models');
/**
* Checks if user has access to a file through agent permissions

2
api/server/middleware/accessResources/fileAccess.spec.js
View file

@ -4,7 +4,7 @@ const { MongoMemoryServer } = require('mongodb-memory-server');
const { fileAccess } = require('./fileAccess');
const { User, Role, AclEntry } = require('~/db/models');
const { createAgent } = require('~/models/Agent');
const { createFile } = require('~/models/File');
const { createFile } = require('~/models');
describe('fileAccess middleware', () => {
let mongoServer;

15
api/server/middleware/buildEndpointOption.js
View file

@ -8,22 +8,11 @@ const {
} = require('librechat-data-provider');
const azureAssistants = require('~/server/services/Endpoints/azureAssistants');
const assistants = require('~/server/services/Endpoints/assistants');
const { processFiles } = require('~/server/services/Files/process');
const anthropic = require('~/server/services/Endpoints/anthropic');
const bedrock = require('~/server/services/Endpoints/bedrock');
const openAI = require('~/server/services/Endpoints/openAI');
const agents = require('~/server/services/Endpoints/agents');
const custom = require('~/server/services/Endpoints/custom');
const google = require('~/server/services/Endpoints/google');
const { updateFilesUsage } = require('~/models');
const buildFunction = {
[EModelEndpoint.openAI]: openAI.buildOptions,
[EModelEndpoint.google]: google.buildOptions,
[EModelEndpoint.custom]: custom.buildOptions,
[EModelEndpoint.agents]: agents.buildOptions,
[EModelEndpoint.bedrock]: bedrock.buildOptions,
[EModelEndpoint.azureOpenAI]: openAI.buildOptions,
[EModelEndpoint.anthropic]: anthropic.buildOptions,
[EModelEndpoint.assistants]: assistants.buildOptions,
[EModelEndpoint.azureAssistants]: azureAssistants.buildOptions,
};
@ -93,7 +82,7 @@ async function buildEndpointOption(req, res, next) {
req.body.endpointOption = await builder(endpoint, parsedBody, endpointType);
if (req.body.files && !isAgents) {
req.body.endpointOption.attachments = processFiles(req.body.files);
req.body.endpointOption.attachments = updateFilesUsage(req.body.files);
}
next();

2
api/server/routes/files/files.agents.test.js
View file

@ -6,7 +6,7 @@ const { createMethods } = require('@librechat/data-schemas');
const { MongoMemoryServer } = require('mongodb-memory-server');
const { AccessRoleIds, ResourceType, PrincipalType } = require('librechat-data-provider');
const { createAgent } = require('~/models/Agent');
const { createFile } = require('~/models/File');
const { createFile } = require('~/models');
// Only mock the external dependencies that we don't want to test
jest.mock('~/server/services/Files/process', () => ({

2
api/server/routes/files/files.js
View file

@ -26,7 +26,7 @@ const { checkPermission } = require('~/server/services/PermissionService');
const { loadAuthValues } = require('~/server/services/Tools/credentials');
const { refreshS3FileUrls } = require('~/server/services/Files/S3/crud');
const { hasAccessToFilesViaAgent } = require('~/server/services/Files');
const { getFiles, batchUpdateFiles } = require('~/models/File');
const { getFiles, batchUpdateFiles } = require('~/models');
const { cleanFileName } = require('~/server/utils/files');
const { getAssistant } = require('~/models/Assistant');
const { getAgent } = require('~/models/Agent');

2
api/server/routes/files/files.test.js
View file

@ -11,7 +11,7 @@ const {
PrincipalType,
} = require('librechat-data-provider');
const { createAgent } = require('~/models/Agent');
const { createFile } = require('~/models/File');
const { createFile } = require('~/models');
// Only mock the external dependencies that we don't want to test
jest.mock('~/server/services/Files/process', () => ({

5
api/server/routes/keys.js
View file

@ -1,7 +1,8 @@
const express = require('express');
const { updateUserKey, deleteUserKey, getUserKeyExpiry } = require('~/models');
const { requireJwtAuth } = require('~/server/middleware');
const router = express.Router();
const { updateUserKey, deleteUserKey, getUserKeyExpiry } = require('../services/UserService');
const { requireJwtAuth } = require('../middleware/');
router.put('/', requireJwtAuth, async (req, res) => {
await updateUserKey({ userId: req.user.id, ...req.body });

10
api/server/services/ActionService.js
View file

@ -1,15 +1,9 @@
const jwt = require('jsonwebtoken');
const { nanoid } = require('nanoid');
const { tool } = require('@langchain/core/tools');
const { logger } = require('@librechat/data-schemas');
const { GraphEvents, sleep } = require('@librechat/agents');
const {
sendEvent,
encryptV2,
decryptV2,
logAxiosError,
refreshAccessToken,
} = require('@librechat/api');
const { logger, encryptV2, decryptV2 } = require('@librechat/data-schemas');
const { sendEvent, logAxiosError, refreshAccessToken } = require('@librechat/api');
const {
Time,
CacheKeys,

3
api/server/services/Config/loadConfigModels.js
View file

@ -1,10 +1,9 @@
const { isUserProvided } = require('@librechat/api');
const { isUserProvided, fetchModels } = require('@librechat/api');
const {
EModelEndpoint,
extractEnvVariable,
normalizeEndpointName,
} = require('librechat-data-provider');
const { fetchModels } = require('~/server/services/ModelService');
const { getAppConfig } = require('./app');
/**

7
api/server/services/Config/loadConfigModels.spec.js
View file

@ -1,8 +1,11 @@
const { fetchModels } = require('~/server/services/ModelService');
const { fetchModels } = require('@librechat/api');
const loadConfigModels = require('./loadConfigModels');
const { getAppConfig } = require('./app');
jest.mock('~/server/services/ModelService');
jest.mock('@librechat/api', () => ({
...jest.requireActual('@librechat/api'),
fetchModels: jest.fn(),
}));
jest.mock('./app');
const exampleConfig = {

2
api/server/services/Config/loadDefaultModels.js
View file

@ -5,7 +5,7 @@ const {
getBedrockModels,
getOpenAIModels,
getGoogleModels,
} = require('~/server/services/ModelService');
} = require('@librechat/api');
/**
* Loads the default models for the application.

226
api/server/services/Endpoints/agents/agent.js
View file

@ -1,226 +0,0 @@
const { Providers } = require('@librechat/agents');
const {
primeResources,
getModelMaxTokens,
extractLibreChatParams,
filterFilesByEndpointConfig,
optionalChainWithEmptyCheck,
} = require('@librechat/api');
const {
ErrorTypes,
EModelEndpoint,
EToolResources,
paramEndpoints,
isAgentsEndpoint,
replaceSpecialVars,
providerEndpointMap,
} = require('librechat-data-provider');
const generateArtifactsPrompt = require('~/app/clients/prompts/artifacts');
const { getProviderConfig } = require('~/server/services/Endpoints');
const { processFiles } = require('~/server/services/Files/process');
const { getFiles, getToolFilesByIds } = require('~/models/File');
const { getConvoFiles } = require('~/models/Conversation');
/**
* @param {object} params
* @param {ServerRequest} params.req
* @param {ServerResponse} params.res
* @param {Agent} params.agent
* @param {string | null} [params.conversationId]
* @param {Array<IMongoFile>} [params.requestFiles]
* @param {typeof import('~/server/services/ToolService').loadAgentTools | undefined} [params.loadTools]
* @param {TEndpointOption} [params.endpointOption]
* @param {Set<string>} [params.allowedProviders]
* @param {boolean} [params.isInitialAgent]
* @returns {Promise<Agent & {
* tools: StructuredTool[],
* attachments: Array<MongoFile>,
* toolContextMap: Record<string, unknown>,
* maxContextTokens: number,
* userMCPAuthMap?: Record<string, Record<string, string>>
* }>}
*/
const initializeAgent = async ({
req,
res,
agent,
loadTools,
requestFiles,
conversationId,
endpointOption,
allowedProviders,
isInitialAgent = false,
}) => {
const appConfig = req.config;
if (
isAgentsEndpoint(endpointOption?.endpoint) &&
allowedProviders.size > 0 &&
!allowedProviders.has(agent.provider)
) {
throw new Error(
`{ "type": "${ErrorTypes.INVALID_AGENT_PROVIDER}", "info": "${agent.provider}" }`,
);
}
let currentFiles;
const _modelOptions = structuredClone(
Object.assign(
{ model: agent.model },
agent.model_parameters ?? { model: agent.model },
isInitialAgent === true ? endpointOption?.model_parameters : {},
),
);
const { resendFiles, maxContextTokens, modelOptions } = extractLibreChatParams(_modelOptions);
const provider = agent.provider;
agent.endpoint = provider;
if (isInitialAgent && conversationId != null && resendFiles) {
const fileIds = (await getConvoFiles(conversationId)) ?? [];
/** @type {Set<EToolResources>} */
const toolResourceSet = new Set();
for (const tool of agent.tools) {
if (EToolResources[tool]) {
toolResourceSet.add(EToolResources[tool]);
}
}
const toolFiles = await getToolFilesByIds(fileIds, toolResourceSet);
if (requestFiles.length || toolFiles.length) {
currentFiles = await processFiles(requestFiles.concat(toolFiles));
}
} else if (isInitialAgent && requestFiles.length) {
currentFiles = await processFiles(requestFiles);
}
if (currentFiles && currentFiles.length) {
let endpointType;
if (!paramEndpoints.has(agent.endpoint)) {
endpointType = EModelEndpoint.custom;
}
currentFiles = filterFilesByEndpointConfig(req, {
files: currentFiles,
endpoint: agent.endpoint,
endpointType,
});
}
const { attachments, tool_resources } = await primeResources({
req,
getFiles,
appConfig,
agentId: agent.id,
attachments: currentFiles,
tool_resources: agent.tool_resources,
requestFileSet: new Set(requestFiles?.map((file) => file.file_id)),
});
const {
tools: structuredTools,
toolContextMap,
userMCPAuthMap,
} = (await loadTools?.({
req,
res,
provider,
agentId: agent.id,
tools: agent.tools,
model: agent.model,
tool_resources,
})) ?? {};
const { getOptions, overrideProvider } = getProviderConfig({ provider, appConfig });
if (overrideProvider !== agent.provider) {
agent.provider = overrideProvider;
}
const _endpointOption =
isInitialAgent === true
? Object.assign({}, endpointOption, { model_parameters: modelOptions })
: { model_parameters: modelOptions };
const options = await getOptions({
req,
res,
optionsOnly: true,
overrideEndpoint: provider,
overrideModel: agent.model,
endpointOption: _endpointOption,
});
const tokensModel =
agent.provider === EModelEndpoint.azureOpenAI ? agent.model : options.llmConfig?.model;
const maxOutputTokens = optionalChainWithEmptyCheck(
options.llmConfig?.maxOutputTokens,
options.llmConfig?.maxTokens,
0,
);
const agentMaxContextTokens = optionalChainWithEmptyCheck(
maxContextTokens,
getModelMaxTokens(tokensModel, providerEndpointMap[provider], options.endpointTokenConfig),
18000,
);
if (
agent.endpoint === EModelEndpoint.azureOpenAI &&
options.llmConfig?.azureOpenAIApiInstanceName == null
) {
agent.provider = Providers.OPENAI;
}
if (options.provider != null) {
agent.provider = options.provider;
}
/** @type {import('@librechat/agents').GenericTool[]} */
let tools = options.tools?.length ? options.tools : structuredTools;
if (
(agent.provider === Providers.GOOGLE || agent.provider === Providers.VERTEXAI) &&
options.tools?.length &&
structuredTools?.length
) {
throw new Error(`{ "type": "${ErrorTypes.GOOGLE_TOOL_CONFLICT}"}`);
} else if (
(agent.provider === Providers.OPENAI ||
agent.provider === Providers.AZURE ||
agent.provider === Providers.ANTHROPIC) &&
options.tools?.length &&
structuredTools?.length
) {
tools = structuredTools.concat(options.tools);
}
/** @type {import('@librechat/agents').ClientOptions} */
agent.model_parameters = { ...options.llmConfig };
if (options.configOptions) {
agent.model_parameters.configuration = options.configOptions;
}
if (agent.instructions && agent.instructions !== '') {
agent.instructions = replaceSpecialVars({
text: agent.instructions,
user: req.user,
});
}
if (typeof agent.artifacts === 'string' && agent.artifacts !== '') {
agent.additional_instructions = generateArtifactsPrompt({
endpoint: agent.provider,
artifacts: agent.artifacts,
});
}
return {
...agent,
tools,
attachments,
resendFiles,
userMCPAuthMap,
toolContextMap,
useLegacyContent: !!options.useLegacyContent,
maxContextTokens: Math.round((agentMaxContextTokens - maxOutputTokens) * 0.9),
};
};
module.exports = { initializeAgent };

66
api/server/services/Endpoints/agents/initialize.js
View file

@ -1,6 +1,7 @@
const { logger } = require('@librechat/data-schemas');
const { createContentAggregator } = require('@librechat/agents');
const {
initializeAgent,
validateAgentModel,
getCustomEndpointConfig,
createSequentialChainEdges,
@ -15,12 +16,13 @@ const {
createToolEndCallback,
getDefaultHandlers,
} = require('~/server/controllers/agents/callbacks');
const { initializeAgent } = require('~/server/services/Endpoints/agents/agent');
const { getModelsConfig } = require('~/server/controllers/ModelController');
const { loadAgentTools } = require('~/server/services/ToolService');
const AgentClient = require('~/server/controllers/agents/client');
const { getConvoFiles } = require('~/models/Conversation');
const { getAgent } = require('~/models/Agent');
const { logViolation } = require('~/cache');
const db = require('~/models');
/**
* @param {AbortSignal} signal
@ -109,17 +111,27 @@ const initializeClient = async ({ req, res, signal, endpointOption }) => {
/** @type {string} */
const conversationId = req.body.conversationId;
const primaryConfig = await initializeAgent({
req,
res,
loadTools,
requestFiles,
conversationId,
agent: primaryAgent,
endpointOption,
allowedProviders,
isInitialAgent: true,
});
const primaryConfig = await initializeAgent(
{
req,
res,
loadTools,
requestFiles,
conversationId,
agent: primaryAgent,
endpointOption,
allowedProviders,
isInitialAgent: true,
},
{
getConvoFiles,
getFiles: db.getFiles,
getUserKey: db.getUserKey,
updateFilesUsage: db.updateFilesUsage,
getUserKeyValues: db.getUserKeyValues,
getToolFilesByIds: db.getToolFilesByIds,
},
);
const agent_ids = primaryConfig.agent_ids;
let userMCPAuthMap = primaryConfig.userMCPAuthMap;
@ -142,16 +154,26 @@ const initializeClient = async ({ req, res, signal, endpointOption }) => {
throw new Error(validationResult.error?.message);
}
const config = await initializeAgent({
req,
res,
agent,
loadTools,
requestFiles,
conversationId,
endpointOption,
allowedProviders,
});
const config = await initializeAgent(
{
req,
res,
agent,
loadTools,
requestFiles,
conversationId,
endpointOption,
allowedProviders,
},
{
getConvoFiles,
getFiles: db.getFiles,
getUserKey: db.getUserKey,
updateFilesUsage: db.updateFilesUsage,
getUserKeyValues: db.getUserKeyValues,
getToolFilesByIds: db.getToolFilesByIds,
},
);
if (userMCPAuthMap != null) {
Object.assign(userMCPAuthMap, config.userMCPAuthMap ?? {});
} else {

44
api/server/services/Endpoints/anthropic/build.js
View file

@ -1,44 +0,0 @@
const { removeNullishValues, anthropicSettings } = require('librechat-data-provider');
const generateArtifactsPrompt = require('~/app/clients/prompts/artifacts');
const buildOptions = (endpoint, parsedBody) => {
const {
modelLabel,
promptPrefix,
maxContextTokens,
fileTokenLimit,
resendFiles = anthropicSettings.resendFiles.default,
promptCache = anthropicSettings.promptCache.default,
thinking = anthropicSettings.thinking.default,
thinkingBudget = anthropicSettings.thinkingBudget.default,
iconURL,
greeting,
spec,
artifacts,
...modelOptions
} = parsedBody;
const endpointOption = removeNullishValues({
endpoint,
modelLabel,
promptPrefix,
resendFiles,
promptCache,
thinking,
thinkingBudget,
iconURL,
greeting,
spec,
maxContextTokens,
fileTokenLimit,
modelOptions,
});
if (typeof artifacts === 'string') {
endpointOption.artifactsPrompt = generateArtifactsPrompt({ endpoint, artifacts });
}
return endpointOption;
};
module.exports = buildOptions;

9
api/server/services/Endpoints/anthropic/index.js
View file

@ -1,9 +0,0 @@
const addTitle = require('./title');
const buildOptions = require('./build');
const initializeClient = require('./initialize');
module.exports = {
addTitle,
buildOptions,
initializeClient,
};

53
api/server/services/Endpoints/anthropic/initialize.js
View file

@ -1,53 +0,0 @@
const { getLLMConfig } = require('@librechat/api');
const { EModelEndpoint } = require('librechat-data-provider');
const { getUserKey, checkUserKeyExpiry } = require('~/server/services/UserService');
const initializeClient = async ({ req, endpointOption, overrideModel }) => {
const appConfig = req.config;
const { ANTHROPIC_API_KEY, ANTHROPIC_REVERSE_PROXY, PROXY } = process.env;
const expiresAt = req.body.key;
const isUserProvided = ANTHROPIC_API_KEY === 'user_provided';
const anthropicApiKey = isUserProvided
? await getUserKey({ userId: req.user.id, name: EModelEndpoint.anthropic })
: ANTHROPIC_API_KEY;
if (!anthropicApiKey) {
throw new Error('Anthropic API key not provided. Please provide it again.');
}
if (expiresAt && isUserProvided) {
checkUserKeyExpiry(expiresAt, EModelEndpoint.anthropic);
}
let clientOptions = {};
/** @type {undefined | TBaseEndpoint} */
const anthropicConfig = appConfig.endpoints?.[EModelEndpoint.anthropic];
if (anthropicConfig) {
clientOptions._lc_stream_delay = anthropicConfig.streamRate;
clientOptions.titleModel = anthropicConfig.titleModel;
}
const allConfig = appConfig.endpoints?.all;
if (allConfig) {
clientOptions._lc_stream_delay = allConfig.streamRate;
}
clientOptions = Object.assign(
{
proxy: PROXY ?? null,
reverseProxyUrl: ANTHROPIC_REVERSE_PROXY ?? null,
modelOptions: endpointOption?.model_parameters ?? {},
},
clientOptions,
);
if (overrideModel) {
clientOptions.modelOptions.model = overrideModel;
}
clientOptions.modelOptions.user = req.user.id;
return getLLMConfig(anthropicApiKey, clientOptions);
};
module.exports = initializeClient;

35
api/server/services/Endpoints/anthropic/title.js
View file

@ -1,35 +0,0 @@
const { isEnabled } = require('@librechat/api');
const { CacheKeys } = require('librechat-data-provider');
const getLogStores = require('~/cache/getLogStores');
const { saveConvo } = require('~/models');
const addTitle = async (req, { text, response, client }) => {
const { TITLE_CONVO = 'true' } = process.env ?? {};
if (!isEnabled(TITLE_CONVO)) {
return;
}
if (client.options.titleConvo === false) {
return;
}
const titleCache = getLogStores(CacheKeys.GEN_TITLE);
const key = `${req.user.id}-${response.conversationId}`;
const title = await client.titleConvo({
text,
responseText: response?.text ?? '',
conversationId: response.conversationId,
});
await titleCache.set(key, title, 120000);
await saveConvo(
req,
{
conversationId: response.conversationId,
title,
},
{ context: 'api/server/services/Endpoints/anthropic/addTitle.js' },
);
};
module.exports = addTitle;

8
api/server/services/Endpoints/assistants/initalize.js
View file

@ -1,12 +1,8 @@
const OpenAI = require('openai');
const { ProxyAgent } = require('undici');
const { isUserProvided } = require('@librechat/api');
const { isUserProvided, checkUserKeyExpiry } = require('@librechat/api');
const { ErrorTypes, EModelEndpoint } = require('librechat-data-provider');
const {
getUserKeyValues,
getUserKeyExpiry,
checkUserKeyExpiry,
} = require('~/server/services/UserService');
const { getUserKeyValues, getUserKeyExpiry } = require('~/models');
const initializeClient = async ({ req, res, version }) => {
const { PROXY, OPENAI_ORGANIZATION, ASSISTANTS_API_KEY, ASSISTANTS_BASE_URL } = process.env;

11
api/server/services/Endpoints/azureAssistants/initialize.js
View file

@ -1,12 +1,13 @@
const OpenAI = require('openai');
const { ProxyAgent } = require('undici');
const { constructAzureURL, isUserProvided, resolveHeaders } = require('@librechat/api');
const { ErrorTypes, EModelEndpoint, mapModelToAzureConfig } = require('librechat-data-provider');
const {
isUserProvided,
resolveHeaders,
constructAzureURL,
checkUserKeyExpiry,
getUserKeyValues,
getUserKeyExpiry,
} = require('~/server/services/UserService');
} = require('@librechat/api');
const { ErrorTypes, EModelEndpoint, mapModelToAzureConfig } = require('librechat-data-provider');
const { getUserKeyValues, getUserKeyExpiry } = require('~/models');
class Files {
constructor(client) {

39
api/server/services/Endpoints/bedrock/build.js
View file

@ -1,39 +0,0 @@
const { removeNullishValues } = require('librechat-data-provider');
const generateArtifactsPrompt = require('~/app/clients/prompts/artifacts');
const buildOptions = (endpoint, parsedBody) => {
const {
modelLabel: name,
promptPrefix,
maxContextTokens,
fileTokenLimit,
resendFiles = true,
imageDetail,
iconURL,
greeting,
spec,
artifacts,
...model_parameters
} = parsedBody;
const endpointOption = removeNullishValues({
endpoint,
name,
resendFiles,
imageDetail,
iconURL,
greeting,
spec,
promptPrefix,
maxContextTokens,
fileTokenLimit,
model_parameters,
});
if (typeof artifacts === 'string') {
endpointOption.artifactsPrompt = generateArtifactsPrompt({ endpoint, artifacts });
}
return endpointOption;
};
module.exports = { buildOptions };

7
api/server/services/Endpoints/bedrock/index.js
View file

@ -1,7 +0,0 @@
const build = require('./build');
const initialize = require('./initialize');
module.exports = {
...build,
...initialize,
};

79
api/server/services/Endpoints/bedrock/initialize.js
View file

@ -1,79 +0,0 @@
const { getModelMaxTokens } = require('@librechat/api');
const { createContentAggregator } = require('@librechat/agents');
const {
EModelEndpoint,
providerEndpointMap,
getResponseSender,
} = require('librechat-data-provider');
const { getDefaultHandlers } = require('~/server/controllers/agents/callbacks');
const getOptions = require('~/server/services/Endpoints/bedrock/options');
const AgentClient = require('~/server/controllers/agents/client');
const initializeClient = async ({ req, res, endpointOption }) => {
if (!endpointOption) {
throw new Error('Endpoint option not provided');
}
/** @type {Array<UsageMetadata>} */
const collectedUsage = [];
const { contentParts, aggregateContent } = createContentAggregator();
const eventHandlers = getDefaultHandlers({ res, aggregateContent, collectedUsage });
/** @type {Agent} */
const agent = {
id: EModelEndpoint.bedrock,
name: endpointOption.name,
provider: EModelEndpoint.bedrock,
endpoint: EModelEndpoint.bedrock,
instructions: endpointOption.promptPrefix,
model: endpointOption.model_parameters.model,
model_parameters: endpointOption.model_parameters,
};
if (typeof endpointOption.artifactsPrompt === 'string' && endpointOption.artifactsPrompt) {
agent.instructions = `${agent.instructions ?? ''}\n${endpointOption.artifactsPrompt}`.trim();
}
// TODO: pass-in override settings that are specific to current run
const options = await getOptions({
req,
res,
endpointOption,
});
agent.model_parameters = Object.assign(agent.model_parameters, options.llmConfig);
if (options.configOptions) {
agent.model_parameters.configuration = options.configOptions;
}
const sender =
agent.name ??
getResponseSender({
...endpointOption,
model: endpointOption.model_parameters.model,
});
const client = new AgentClient({
req,
res,
agent,
sender,
// tools,
contentParts,
eventHandlers,
collectedUsage,
spec: endpointOption.spec,
iconURL: endpointOption.iconURL,
endpoint: EModelEndpoint.bedrock,
resendFiles: endpointOption.resendFiles,
maxContextTokens:
endpointOption.maxContextTokens ??
agent.max_context_tokens ??
getModelMaxTokens(agent.model_parameters.model, providerEndpointMap[agent.provider]) ??
4000,
attachments: endpointOption.attachments,
});
return { client };
};
module.exports = { initializeClient };

150
api/server/services/Endpoints/bedrock/options.js
View file

@ -1,150 +0,0 @@
/**
* Bedrock endpoint options configuration
*
* This module handles configuration for AWS Bedrock endpoints, including support for
* HTTP/HTTPS proxies and reverse proxies.
*
* Proxy Support:
* - When the PROXY environment variable is set, creates a custom BedrockRuntimeClient
* with an HttpsProxyAgent to route all Bedrock API calls through the specified proxy
* - The custom client is fully configured with credentials, region, and endpoint,
* and is passed directly to ChatBedrockConverse via the 'client' parameter
*
* Reverse Proxy Support:
* - When BEDROCK_REVERSE_PROXY is set, routes Bedrock API calls through a custom endpoint
* - Works with or without the PROXY setting
*
* Without Proxy:
* - Credentials and endpoint configuration are passed separately to ChatBedrockConverse,
* which creates its own BedrockRuntimeClient internally
*
* Environment Variables:
* - PROXY: HTTP/HTTPS proxy URL (e.g., http://proxy.example.com:8080)
* - BEDROCK_REVERSE_PROXY: Custom Bedrock API endpoint host
* - BEDROCK_AWS_DEFAULT_REGION: AWS region for Bedrock service
* - BEDROCK_AWS_ACCESS_KEY_ID: AWS access key (or set to 'user_provided')
* - BEDROCK_AWS_SECRET_ACCESS_KEY: AWS secret key (or set to 'user_provided')
* - BEDROCK_AWS_SESSION_TOKEN: Optional AWS session token
*/
const { HttpsProxyAgent } = require('https-proxy-agent');
const { NodeHttpHandler } = require('@smithy/node-http-handler');
const { BedrockRuntimeClient } = require('@aws-sdk/client-bedrock-runtime');
const {
AuthType,
EModelEndpoint,
bedrockInputParser,
bedrockOutputParser,
removeNullishValues,
} = require('librechat-data-provider');
const { getUserKey, checkUserKeyExpiry } = require('~/server/services/UserService');
const getOptions = async ({ req, overrideModel, endpointOption }) => {
const {
BEDROCK_AWS_SECRET_ACCESS_KEY,
BEDROCK_AWS_ACCESS_KEY_ID,
BEDROCK_AWS_SESSION_TOKEN,
BEDROCK_REVERSE_PROXY,
BEDROCK_AWS_DEFAULT_REGION,
PROXY,
} = process.env;
const expiresAt = req.body.key;
const isUserProvided = BEDROCK_AWS_SECRET_ACCESS_KEY === AuthType.USER_PROVIDED;
let credentials = isUserProvided
? await getUserKey({ userId: req.user.id, name: EModelEndpoint.bedrock })
: {
accessKeyId: BEDROCK_AWS_ACCESS_KEY_ID,
secretAccessKey: BEDROCK_AWS_SECRET_ACCESS_KEY,
...(BEDROCK_AWS_SESSION_TOKEN && { sessionToken: BEDROCK_AWS_SESSION_TOKEN }),
};
if (!credentials) {
throw new Error('Bedrock credentials not provided. Please provide them again.');
}
if (
!isUserProvided &&
(credentials.accessKeyId === undefined || credentials.accessKeyId === '') &&
(credentials.secretAccessKey === undefined || credentials.secretAccessKey === '')
) {
credentials = undefined;
}
if (expiresAt && isUserProvided) {
checkUserKeyExpiry(expiresAt, EModelEndpoint.bedrock);
}
/*
Callback for stream rate no longer awaits and may end the stream prematurely
/** @type {number}
let streamRate = Constants.DEFAULT_STREAM_RATE;
/** @type {undefined | TBaseEndpoint}
const bedrockConfig = appConfig.endpoints?.[EModelEndpoint.bedrock];
if (bedrockConfig && bedrockConfig.streamRate) {
streamRate = bedrockConfig.streamRate;
}
const allConfig = appConfig.endpoints?.all;
if (allConfig && allConfig.streamRate) {
streamRate = allConfig.streamRate;
}
*/
/** @type {BedrockClientOptions} */
const requestOptions = {
model: overrideModel ?? endpointOption?.model,
region: BEDROCK_AWS_DEFAULT_REGION,
};
const configOptions = {};
const llmConfig = bedrockOutputParser(
bedrockInputParser.parse(
removeNullishValues(Object.assign(requestOptions, endpointOption?.model_parameters ?? {})),
),
);
if (PROXY) {
const proxyAgent = new HttpsProxyAgent(PROXY);
// Create a custom BedrockRuntimeClient with proxy-enabled request handler.
// ChatBedrockConverse will use this pre-configured client directly instead of
// creating its own. Credentials are only set if explicitly provided; otherwise
// the AWS SDK's default credential provider chain is used (instance profiles,
// AWS profiles, environment variables, etc.)
const customClient = new BedrockRuntimeClient({
region: llmConfig.region ?? BEDROCK_AWS_DEFAULT_REGION,
...(credentials && { credentials }),
requestHandler: new NodeHttpHandler({
httpAgent: proxyAgent,
httpsAgent: proxyAgent,
}),
...(BEDROCK_REVERSE_PROXY && {
endpoint: `https://${BEDROCK_REVERSE_PROXY}`,
}),
});
llmConfig.client = customClient;
} else {
// When not using a proxy, let ChatBedrockConverse create its own client
// by providing credentials and endpoint separately
if (credentials) {
llmConfig.credentials = credentials;
}
if (BEDROCK_REVERSE_PROXY) {
llmConfig.endpointHost = BEDROCK_REVERSE_PROXY;
}
}
return {
/** @type {BedrockClientOptions} */
llmConfig,
configOptions,
};
};
module.exports = getOptions;

42
api/server/services/Endpoints/custom/build.js
View file

@ -1,42 +0,0 @@
const { removeNullishValues } = require('librechat-data-provider');
const generateArtifactsPrompt = require('~/app/clients/prompts/artifacts');
const buildOptions = (endpoint, parsedBody, endpointType) => {
const {
modelLabel,
chatGptLabel,
promptPrefix,
maxContextTokens,
fileTokenLimit,
resendFiles = true,
imageDetail,
iconURL,
greeting,
spec,
artifacts,
...modelOptions
} = parsedBody;
const endpointOption = removeNullishValues({
endpoint,
endpointType,
modelLabel,
chatGptLabel,
promptPrefix,
resendFiles,
imageDetail,
iconURL,
greeting,
spec,
maxContextTokens,
fileTokenLimit,
modelOptions,
});
if (typeof artifacts === 'string') {
endpointOption.artifactsPrompt = generateArtifactsPrompt({ endpoint, artifacts });
}
return endpointOption;
};
module.exports = buildOptions;

7
api/server/services/Endpoints/custom/index.js
View file

@ -1,7 +0,0 @@
const initializeClient = require('./initialize');
const buildOptions = require('./build');
module.exports = {
initializeClient,
buildOptions,
};

145
api/server/services/Endpoints/custom/initialize.js
View file

@ -1,145 +0,0 @@
const { isUserProvided, getOpenAIConfig, getCustomEndpointConfig } = require('@librechat/api');
const {
CacheKeys,
ErrorTypes,
envVarRegex,
FetchTokenConfig,
extractEnvVariable,
} = require('librechat-data-provider');
const { getUserKeyValues, checkUserKeyExpiry } = require('~/server/services/UserService');
const { fetchModels } = require('~/server/services/ModelService');
const getLogStores = require('~/cache/getLogStores');
const { PROXY } = process.env;
const initializeClient = async ({ req, endpointOption, overrideEndpoint }) => {
const appConfig = req.config;
const { key: expiresAt } = req.body;
const endpoint = overrideEndpoint ?? req.body.endpoint;
const endpointConfig = getCustomEndpointConfig({
endpoint,
appConfig,
});
if (!endpointConfig) {
throw new Error(`Config not found for the ${endpoint} custom endpoint.`);
}
const CUSTOM_API_KEY = extractEnvVariable(endpointConfig.apiKey);
const CUSTOM_BASE_URL = extractEnvVariable(endpointConfig.baseURL);
if (CUSTOM_API_KEY.match(envVarRegex)) {
throw new Error(`Missing API Key for ${endpoint}.`);
}
if (CUSTOM_BASE_URL.match(envVarRegex)) {
throw new Error(`Missing Base URL for ${endpoint}.`);
}
const userProvidesKey = isUserProvided(CUSTOM_API_KEY);
const userProvidesURL = isUserProvided(CUSTOM_BASE_URL);
let userValues = null;
if (expiresAt && (userProvidesKey || userProvidesURL)) {
checkUserKeyExpiry(expiresAt, endpoint);
userValues = await getUserKeyValues({ userId: req.user.id, name: endpoint });
}
let apiKey = userProvidesKey ? userValues?.apiKey : CUSTOM_API_KEY;
let baseURL = userProvidesURL ? userValues?.baseURL : CUSTOM_BASE_URL;
if (userProvidesKey & !apiKey) {
throw new Error(
JSON.stringify({
type: ErrorTypes.NO_USER_KEY,
}),
);
}
if (userProvidesURL && !baseURL) {
throw new Error(
JSON.stringify({
type: ErrorTypes.NO_BASE_URL,
}),
);
}
if (!apiKey) {
throw new Error(`${endpoint} API key not provided.`);
}
if (!baseURL) {
throw new Error(`${endpoint} Base URL not provided.`);
}
const cache = getLogStores(CacheKeys.TOKEN_CONFIG);
const tokenKey =
!endpointConfig.tokenConfig && (userProvidesKey || userProvidesURL)
? `${endpoint}:${req.user.id}`
: endpoint;
let endpointTokenConfig =
!endpointConfig.tokenConfig &&
FetchTokenConfig[endpoint.toLowerCase()] &&
(await cache.get(tokenKey));
if (
FetchTokenConfig[endpoint.toLowerCase()] &&
endpointConfig &&
endpointConfig.models.fetch &&
!endpointTokenConfig
) {
await fetchModels({ apiKey, baseURL, name: endpoint, user: req.user.id, tokenKey });
endpointTokenConfig = await cache.get(tokenKey);
}
const customOptions = {
headers: endpointConfig.headers,
addParams: endpointConfig.addParams,
dropParams: endpointConfig.dropParams,
customParams: endpointConfig.customParams,
titleConvo: endpointConfig.titleConvo,
titleModel: endpointConfig.titleModel,
forcePrompt: endpointConfig.forcePrompt,
summaryModel: endpointConfig.summaryModel,
modelDisplayLabel: endpointConfig.modelDisplayLabel,
titleMethod: endpointConfig.titleMethod ?? 'completion',
contextStrategy: endpointConfig.summarize ? 'summarize' : null,
directEndpoint: endpointConfig.directEndpoint,
titleMessageRole: endpointConfig.titleMessageRole,
streamRate: endpointConfig.streamRate,
endpointTokenConfig,
};
const allConfig = appConfig.endpoints?.all;
if (allConfig) {
customOptions.streamRate = allConfig.streamRate;
}
let clientOptions = {
reverseProxyUrl: baseURL ?? null,
proxy: PROXY ?? null,
...customOptions,
...endpointOption,
};
const modelOptions = endpointOption?.model_parameters ?? {};
clientOptions = Object.assign(
{
modelOptions,
},
clientOptions,
);
clientOptions.modelOptions.user = req.user.id;
const options = getOpenAIConfig(apiKey, clientOptions, endpoint);
if (options != null) {
options.useLegacyContent = true;
options.endpointTokenConfig = endpointTokenConfig;
}
if (clientOptions.streamRate) {
options.llmConfig._lc_stream_delay = clientOptions.streamRate;
}
return options;
};
module.exports = initializeClient;

39
api/server/services/Endpoints/google/build.js
View file

@ -1,39 +0,0 @@
const { removeNullishValues } = require('librechat-data-provider');
const generateArtifactsPrompt = require('~/app/clients/prompts/artifacts');
const buildOptions = (endpoint, parsedBody) => {
const {
examples,
modelLabel,
resendFiles = true,
promptPrefix,
iconURL,
greeting,
spec,
artifacts,
maxContextTokens,
fileTokenLimit,
...modelOptions
} = parsedBody;
const endpointOption = removeNullishValues({
examples,
endpoint,
modelLabel,
resendFiles,
promptPrefix,
iconURL,
greeting,
spec,
maxContextTokens,
fileTokenLimit,
modelOptions,
});
if (typeof artifacts === 'string') {
endpointOption.artifactsPrompt = generateArtifactsPrompt({ endpoint, artifacts });
}
return endpointOption;
};
module.exports = buildOptions;

9
api/server/services/Endpoints/google/index.js
View file

@ -1,9 +0,0 @@
const addTitle = require('./title');
const buildOptions = require('./build');
const initializeClient = require('./initialize');
module.exports = {
addTitle,
buildOptions,
initializeClient,
};

83
api/server/services/Endpoints/google/initialize.js
View file

@ -1,83 +0,0 @@
const path = require('path');
const { EModelEndpoint, AuthKeys } = require('librechat-data-provider');
const { getGoogleConfig, isEnabled, loadServiceKey } = require('@librechat/api');
const { getUserKey, checkUserKeyExpiry } = require('~/server/services/UserService');
const initializeClient = async ({ req, endpointOption, overrideModel }) => {
const { GOOGLE_KEY, GOOGLE_REVERSE_PROXY, GOOGLE_AUTH_HEADER, PROXY } = process.env;
const isUserProvided = GOOGLE_KEY === 'user_provided';
const { key: expiresAt } = req.body;
let userKey = null;
if (expiresAt && isUserProvided) {
checkUserKeyExpiry(expiresAt, EModelEndpoint.google);
userKey = await getUserKey({ userId: req.user.id, name: EModelEndpoint.google });
}
let serviceKey = {};
/** Check if GOOGLE_KEY is provided at all (including 'user_provided') */
const isGoogleKeyProvided =
(GOOGLE_KEY && GOOGLE_KEY.trim() !== '') || (isUserProvided && userKey != null);
if (!isGoogleKeyProvided) {
/** Only attempt to load service key if GOOGLE_KEY is not provided */
try {
const serviceKeyPath =
process.env.GOOGLE_SERVICE_KEY_FILE ||
path.join(__dirname, '../../../..', 'data', 'auth.json');
serviceKey = await loadServiceKey(serviceKeyPath);
if (!serviceKey) {
serviceKey = {};
}
} catch (_e) {
// Service key loading failed, but that's okay if not required
serviceKey = {};
}
}
const credentials = isUserProvided
? userKey
: {
[AuthKeys.GOOGLE_SERVICE_KEY]: serviceKey,
[AuthKeys.GOOGLE_API_KEY]: GOOGLE_KEY,
};
let clientOptions = {};
const appConfig = req.config;
/** @type {undefined | TBaseEndpoint} */
const allConfig = appConfig.endpoints?.all;
/** @type {undefined | TBaseEndpoint} */
const googleConfig = appConfig.endpoints?.[EModelEndpoint.google];
if (googleConfig) {
clientOptions.streamRate = googleConfig.streamRate;
clientOptions.titleModel = googleConfig.titleModel;
}
if (allConfig) {
clientOptions.streamRate = allConfig.streamRate;
}
clientOptions = {
reverseProxyUrl: GOOGLE_REVERSE_PROXY ?? null,
authHeader: isEnabled(GOOGLE_AUTH_HEADER) ?? null,
proxy: PROXY ?? null,
...clientOptions,
...endpointOption,
};
clientOptions = Object.assign(
{
modelOptions: endpointOption?.model_parameters ?? {},
},
clientOptions,
);
if (overrideModel) {
clientOptions.modelOptions.model = overrideModel;
}
return getGoogleConfig(credentials, clientOptions);
};
module.exports = initializeClient;

60
api/server/services/Endpoints/google/title.js
View file

@ -1,60 +0,0 @@
const { isEnabled } = require('@librechat/api');
const { EModelEndpoint, CacheKeys, Constants, googleSettings } = require('librechat-data-provider');
const getLogStores = require('~/cache/getLogStores');
const initializeClient = require('./initialize');
const { saveConvo } = require('~/models');
const addTitle = async (req, { text, response, client }) => {
const { TITLE_CONVO = 'true' } = process.env ?? {};
if (!isEnabled(TITLE_CONVO)) {
return;
}
if (client.options.titleConvo === false) {
return;
}
const { GOOGLE_TITLE_MODEL } = process.env ?? {};
const appConfig = req.config;
const providerConfig = appConfig.endpoints?.[EModelEndpoint.google];
let model =
providerConfig?.titleModel ??
GOOGLE_TITLE_MODEL ??
client.options?.modelOptions.model ??
googleSettings.model.default;
if (GOOGLE_TITLE_MODEL === Constants.CURRENT_MODEL) {
model = client.options?.modelOptions.model;
}
const titleEndpointOptions = {
...client.options,
modelOptions: { ...client.options?.modelOptions, model: model },
attachments: undefined, // After a response, this is set to an empty array which results in an error during setOptions
};
const { client: titleClient } = await initializeClient({
req,
res: response,
endpointOption: titleEndpointOptions,
});
const titleCache = getLogStores(CacheKeys.GEN_TITLE);
const key = `${req.user.id}-${response.conversationId}`;
const title = await titleClient.titleConvo({
text,
responseText: response?.text ?? '',
conversationId: response.conversationId,
});
await titleCache.set(key, title, 120000);
await saveConvo(
req,
{
conversationId: response.conversationId,
title,
},
{ context: 'api/server/services/Endpoints/google/addTitle.js' },
);
};
module.exports = addTitle;

42
api/server/services/Endpoints/openAI/build.js
View file

@ -1,42 +0,0 @@
const { removeNullishValues } = require('librechat-data-provider');
const generateArtifactsPrompt = require('~/app/clients/prompts/artifacts');
const buildOptions = (endpoint, parsedBody) => {
const {
modelLabel,
chatGptLabel,
promptPrefix,
maxContextTokens,
fileTokenLimit,
resendFiles = true,
imageDetail,
iconURL,
greeting,
spec,
artifacts,
...modelOptions
} = parsedBody;
const endpointOption = removeNullishValues({
endpoint,
modelLabel,
chatGptLabel,
promptPrefix,
resendFiles,
imageDetail,
iconURL,
greeting,
spec,
maxContextTokens,
fileTokenLimit,
modelOptions,
});
if (typeof artifacts === 'string') {
endpointOption.artifactsPrompt = generateArtifactsPrompt({ endpoint, artifacts });
}
return endpointOption;
};
module.exports = buildOptions;

9
api/server/services/Endpoints/openAI/index.js
View file

@ -1,9 +0,0 @@
const addTitle = require('./title');
const buildOptions = require('./build');
const initializeClient = require('./initialize');
module.exports = {
addTitle,
buildOptions,
initializeClient,
};

147
api/server/services/Endpoints/openAI/initialize.js
View file

@ -1,147 +0,0 @@
const { ErrorTypes, EModelEndpoint, mapModelToAzureConfig } = require('librechat-data-provider');
const {
isEnabled,
resolveHeaders,
isUserProvided,
getOpenAIConfig,
getAzureCredentials,
} = require('@librechat/api');
const { getUserKeyValues, checkUserKeyExpiry } = require('~/server/services/UserService');
const initializeClient = async ({ req, endpointOption, overrideEndpoint, overrideModel }) => {
const appConfig = req.config;
const {
PROXY,
OPENAI_API_KEY,
AZURE_API_KEY,
OPENAI_REVERSE_PROXY,
AZURE_OPENAI_BASEURL,
OPENAI_SUMMARIZE,
DEBUG_OPENAI,
} = process.env;
const { key: expiresAt } = req.body;
const modelName = overrideModel ?? req.body.model;
const endpoint = overrideEndpoint ?? req.body.endpoint;
const contextStrategy = isEnabled(OPENAI_SUMMARIZE) ? 'summarize' : null;
const credentials = {
[EModelEndpoint.openAI]: OPENAI_API_KEY,
[EModelEndpoint.azureOpenAI]: AZURE_API_KEY,
};
const baseURLOptions = {
[EModelEndpoint.openAI]: OPENAI_REVERSE_PROXY,
[EModelEndpoint.azureOpenAI]: AZURE_OPENAI_BASEURL,
};
const userProvidesKey = isUserProvided(credentials[endpoint]);
const userProvidesURL = isUserProvided(baseURLOptions[endpoint]);
let userValues = null;
if (expiresAt && (userProvidesKey || userProvidesURL)) {
checkUserKeyExpiry(expiresAt, endpoint);
userValues = await getUserKeyValues({ userId: req.user.id, name: endpoint });
}
let apiKey = userProvidesKey ? userValues?.apiKey : credentials[endpoint];
let baseURL = userProvidesURL ? userValues?.baseURL : baseURLOptions[endpoint];
let clientOptions = {
contextStrategy,
proxy: PROXY ?? null,
debug: isEnabled(DEBUG_OPENAI),
reverseProxyUrl: baseURL ? baseURL : null,
...endpointOption,
};
const isAzureOpenAI = endpoint === EModelEndpoint.azureOpenAI;
/** @type {false | TAzureConfig} */
const azureConfig = isAzureOpenAI && appConfig.endpoints?.[EModelEndpoint.azureOpenAI];
let serverless = false;
if (isAzureOpenAI && azureConfig) {
const { modelGroupMap, groupMap } = azureConfig;
const {
azureOptions,
baseURL,
headers = {},
serverless: _serverless,
} = mapModelToAzureConfig({
modelName,
modelGroupMap,
groupMap,
});
serverless = _serverless;
clientOptions.reverseProxyUrl = baseURL ?? clientOptions.reverseProxyUrl;
clientOptions.headers = resolveHeaders({
headers: { ...headers, ...(clientOptions.headers ?? {}) },
user: req.user,
});
clientOptions.titleConvo = azureConfig.titleConvo;
clientOptions.titleModel = azureConfig.titleModel;
const azureRate = modelName.includes('gpt-4') ? 30 : 17;
clientOptions.streamRate = azureConfig.streamRate ?? azureRate;
clientOptions.titleMethod = azureConfig.titleMethod ?? 'completion';
const groupName = modelGroupMap[modelName].group;
clientOptions.addParams = azureConfig.groupMap[groupName].addParams;
clientOptions.dropParams = azureConfig.groupMap[groupName].dropParams;
clientOptions.forcePrompt = azureConfig.groupMap[groupName].forcePrompt;
apiKey = azureOptions.azureOpenAIApiKey;
clientOptions.azure = !serverless && azureOptions;
if (serverless === true) {
clientOptions.defaultQuery = azureOptions.azureOpenAIApiVersion
? { 'api-version': azureOptions.azureOpenAIApiVersion }
: undefined;
clientOptions.headers['api-key'] = apiKey;
}
} else if (isAzureOpenAI) {
clientOptions.azure = userProvidesKey ? JSON.parse(userValues.apiKey) : getAzureCredentials();
apiKey = clientOptions.azure.azureOpenAIApiKey;
}
/** @type {undefined | TBaseEndpoint} */
const openAIConfig = appConfig.endpoints?.[EModelEndpoint.openAI];
if (!isAzureOpenAI && openAIConfig) {
clientOptions.streamRate = openAIConfig.streamRate;
clientOptions.titleModel = openAIConfig.titleModel;
}
const allConfig = appConfig.endpoints?.all;
if (allConfig) {
clientOptions.streamRate = allConfig.streamRate;
}
if (userProvidesKey & !apiKey) {
throw new Error(
JSON.stringify({
type: ErrorTypes.NO_USER_KEY,
}),
);
}
if (!apiKey) {
throw new Error(`${endpoint} API Key not provided.`);
}
const modelOptions = endpointOption?.model_parameters ?? {};
modelOptions.model = modelName;
clientOptions = Object.assign({ modelOptions }, clientOptions);
clientOptions.modelOptions.user = req.user.id;
const options = getOpenAIConfig(apiKey, clientOptions, endpoint);
if (options != null && serverless === true) {
options.useLegacyContent = true;
}
const streamRate = clientOptions.streamRate;
if (streamRate) {
options.llmConfig._lc_stream_delay = streamRate;
}
return options;
};
module.exports = initializeClient;

35
api/server/services/Endpoints/openAI/title.js
View file

@ -1,35 +0,0 @@
const { isEnabled } = require('@librechat/api');
const { CacheKeys } = require('librechat-data-provider');
const getLogStores = require('~/cache/getLogStores');
const { saveConvo } = require('~/models');
const addTitle = async (req, { text, response, client }) => {
const { TITLE_CONVO = 'true' } = process.env ?? {};
if (!isEnabled(TITLE_CONVO)) {
return;
}
if (client.options.titleConvo === false) {
return;
}
const titleCache = getLogStores(CacheKeys.GEN_TITLE);
const key = `${req.user.id}-${response.conversationId}`;
const title = await client.titleConvo({
text,
responseText: response?.text ?? '',
conversationId: response.conversationId,
});
await titleCache.set(key, title, 120000);
await saveConvo(
req,
{
conversationId: response.conversationId,
title,
},
{ context: 'api/server/services/Endpoints/openAI/addTitle.js' },
);
};
module.exports = addTitle;

2
api/server/services/Files/Code/process.js
View file

@ -14,7 +14,7 @@ const {
const { filterFilesByAgentAccess } = require('~/server/services/Files/permissions');
const { getStrategyFunctions } = require('~/server/services/Files/strategies');
const { convertImage } = require('~/server/services/Files/images/convert');
const { createFile, getFiles, updateFile } = require('~/models/File');
const { createFile, getFiles, updateFile } = require('~/models');
/**
* Process OpenAI image files, convert to target format, save and return file metadata.

42
api/server/services/Files/process.js
View file

@ -28,8 +28,8 @@ const {
const { addResourceFileId, deleteResourceFileId } = require('~/server/controllers/assistants/v2');
const { addAgentResourceFile, removeAgentResourceFiles } = require('~/models/Agent');
const { getOpenAIClient } = require('~/server/controllers/assistants/helpers');
const { createFile, updateFileUsage, deleteFiles } = require('~/models/File');
const { loadAuthValues } = require('~/server/services/Tools/credentials');
const { createFile, updateFileUsage, deleteFiles } = require('~/models');
const { getFileStrategy } = require('~/server/utils/getFileStrategy');
const { checkCapability } = require('~/server/services/Config');
const { LB_QueueAsyncCall } = require('~/server/utils/queue');
@ -60,45 +60,6 @@ const createSanitizedUploadWrapper = (uploadFunction) => {
};
};
/**
*
* @param {Array<MongoFile>} files
* @param {Array<string>} [fileIds]
* @returns
*/
const processFiles = async (files, fileIds) => {
const promises = [];
const seen = new Set();
for (let file of files) {
const { file_id } = file;
if (seen.has(file_id)) {
continue;
}
seen.add(file_id);
promises.push(updateFileUsage({ file_id }));
}
if (!fileIds) {
const results = await Promise.all(promises);
// Filter out null results from failed updateFileUsage calls
return results.filter((result) => result != null);
}
for (let file_id of fileIds) {
if (seen.has(file_id)) {
continue;
}
seen.add(file_id);
promises.push(updateFileUsage({ file_id }));
}
// TODO: calculate token cost when image is first uploaded
const results = await Promise.all(promises);
// Filter out null results from failed updateFileUsage calls
return results.filter((result) => result != null);
};
/**
* Enqueues the delete operation to the leaky bucket queue if necessary, or adds it directly to promises.
*
@ -1057,7 +1018,6 @@ function filterFile({ req, image, isAvatar }) {
module.exports = {
filterFile,
processFiles,
processFileURL,
saveBase64Image,
processImageFile,

248
api/server/services/Files/processFiles.test.js
View file

@ -1,248 +0,0 @@
// Mock the updateFileUsage function before importing the actual processFiles
jest.mock('~/models/File', () => ({
updateFileUsage: jest.fn(),
}));
// Mock winston and logger configuration to avoid dependency issues
jest.mock('~/config', () => ({
logger: {
info: jest.fn(),
warn: jest.fn(),
debug: jest.fn(),
error: jest.fn(),
},
}));
// Mock all other dependencies that might cause issues
jest.mock('librechat-data-provider', () => ({
isUUID: { parse: jest.fn() },
megabyte: 1024 * 1024,
PrincipalType: {
USER: 'user',
GROUP: 'group',
PUBLIC: 'public',
},
PrincipalModel: {
USER: 'User',
GROUP: 'Group',
},
ResourceType: {
AGENT: 'agent',
PROJECT: 'project',
FILE: 'file',
PROMPTGROUP: 'promptGroup',
},
FileContext: { message_attachment: 'message_attachment' },
FileSources: { local: 'local' },
EModelEndpoint: { assistants: 'assistants' },
EToolResources: { file_search: 'file_search' },
mergeFileConfig: jest.fn(),
removeNullishValues: jest.fn((obj) => obj),
isAssistantsEndpoint: jest.fn(),
Constants: { COMMANDS_MAX_LENGTH: 56 },
PermissionTypes: {
BOOKMARKS: 'BOOKMARKS',
PROMPTS: 'PROMPTS',
MEMORIES: 'MEMORIES',
MULTI_CONVO: 'MULTI_CONVO',
AGENTS: 'AGENTS',
TEMPORARY_CHAT: 'TEMPORARY_CHAT',
RUN_CODE: 'RUN_CODE',
WEB_SEARCH: 'WEB_SEARCH',
FILE_CITATIONS: 'FILE_CITATIONS',
},
Permissions: {
USE: 'USE',
OPT_OUT: 'OPT_OUT',
},
SystemRoles: {
USER: 'USER',
ADMIN: 'ADMIN',
},
}));
jest.mock('~/server/services/Files/images', () => ({
convertImage: jest.fn(),
resizeAndConvert: jest.fn(),
resizeImageBuffer: jest.fn(),
}));
jest.mock('~/server/controllers/assistants/v2', () => ({
addResourceFileId: jest.fn(),
deleteResourceFileId: jest.fn(),
}));
jest.mock('~/models/Agent', () => ({
addAgentResourceFile: jest.fn(),
removeAgentResourceFiles: jest.fn(),
}));
jest.mock('~/server/controllers/assistants/helpers', () => ({
getOpenAIClient: jest.fn(),
}));
jest.mock('~/server/services/Tools/credentials', () => ({
loadAuthValues: jest.fn(),
}));
jest.mock('~/server/services/Config', () => ({
checkCapability: jest.fn(),
}));
jest.mock('~/server/utils/queue', () => ({
LB_QueueAsyncCall: jest.fn(),
}));
jest.mock('./strategies', () => ({
getStrategyFunctions: jest.fn(),
}));
jest.mock('~/server/utils', () => ({
determineFileType: jest.fn(),
}));
jest.mock('@librechat/api', () => ({
parseText: jest.fn(),
parseTextNative: jest.fn(),
}));
// Import the actual processFiles function after all mocks are set up
const { processFiles } = require('./process');
const { updateFileUsage } = require('~/models/File');
describe('processFiles', () => {
beforeEach(() => {
jest.clearAllMocks();
});
describe('null filtering functionality', () => {
it('should filter out null results from updateFileUsage when files do not exist', async () => {
const mockFiles = [
{ file_id: 'existing-file-1' },
{ file_id: 'non-existent-file' },
{ file_id: 'existing-file-2' },
];
// Mock updateFileUsage to return null for non-existent files
updateFileUsage.mockImplementation(({ file_id }) => {
if (file_id === 'non-existent-file') {
return Promise.resolve(null); // Simulate file not found in the database
}
return Promise.resolve({ file_id, usage: 1 });
});
const result = await processFiles(mockFiles);
expect(updateFileUsage).toHaveBeenCalledTimes(3);
expect(result).toEqual([
{ file_id: 'existing-file-1', usage: 1 },
{ file_id: 'existing-file-2', usage: 1 },
]);
// Critical test - ensure no null values in result
expect(result).not.toContain(null);
expect(result).not.toContain(undefined);
expect(result.length).toBe(2); // Only valid files should be returned
});
it('should return empty array when all updateFileUsage calls return null', async () => {
const mockFiles = [{ file_id: 'non-existent-1' }, { file_id: 'non-existent-2' }];
// All updateFileUsage calls return null
updateFileUsage.mockResolvedValue(null);
const result = await processFiles(mockFiles);
expect(updateFileUsage).toHaveBeenCalledTimes(2);
expect(result).toEqual([]);
expect(result).not.toContain(null);
expect(result.length).toBe(0);
});
it('should work correctly when all files exist', async () => {
const mockFiles = [{ file_id: 'file-1' }, { file_id: 'file-2' }];
updateFileUsage.mockImplementation(({ file_id }) => {
return Promise.resolve({ file_id, usage: 1 });
});
const result = await processFiles(mockFiles);
expect(result).toEqual([
{ file_id: 'file-1', usage: 1 },
{ file_id: 'file-2', usage: 1 },
]);
expect(result).not.toContain(null);
expect(result.length).toBe(2);
});
it('should handle fileIds parameter and filter nulls correctly', async () => {
const mockFiles = [{ file_id: 'file-1' }];
const mockFileIds = ['file-2', 'non-existent-file'];
updateFileUsage.mockImplementation(({ file_id }) => {
if (file_id === 'non-existent-file') {
return Promise.resolve(null);
}
return Promise.resolve({ file_id, usage: 1 });
});
const result = await processFiles(mockFiles, mockFileIds);
expect(result).toEqual([
{ file_id: 'file-1', usage: 1 },
{ file_id: 'file-2', usage: 1 },
]);
expect(result).not.toContain(null);
expect(result).not.toContain(undefined);
expect(result.length).toBe(2);
});
it('should handle duplicate file_ids correctly', async () => {
const mockFiles = [
{ file_id: 'duplicate-file' },
{ file_id: 'duplicate-file' }, // Duplicate should be ignored
{ file_id: 'unique-file' },
];
updateFileUsage.mockImplementation(({ file_id }) => {
return Promise.resolve({ file_id, usage: 1 });
});
const result = await processFiles(mockFiles);
// Should only call updateFileUsage twice (duplicate ignored)
expect(updateFileUsage).toHaveBeenCalledTimes(2);
expect(result).toEqual([
{ file_id: 'duplicate-file', usage: 1 },
{ file_id: 'unique-file', usage: 1 },
]);
expect(result.length).toBe(2);
});
});
describe('edge cases', () => {
it('should handle empty files array', async () => {
const result = await processFiles([]);
expect(result).toEqual([]);
expect(updateFileUsage).not.toHaveBeenCalled();
});
it('should handle mixed null and undefined returns from updateFileUsage', async () => {
const mockFiles = [{ file_id: 'file-1' }, { file_id: 'file-2' }, { file_id: 'file-3' }];
updateFileUsage.mockImplementation(({ file_id }) => {
if (file_id === 'file-1') return Promise.resolve(null);
if (file_id === 'file-2') return Promise.resolve(undefined);
return Promise.resolve({ file_id, usage: 1 });
});
const result = await processFiles(mockFiles);
expect(result).toEqual([{ file_id: 'file-3', usage: 1 }]);
expect(result).not.toContain(null);
expect(result).not.toContain(undefined);
expect(result.length).toBe(1);
});
});
});

3
api/server/services/GraphApiService.spec.js
View file

@ -18,9 +18,6 @@ jest.mock('~/config', () => ({
defaults: {},
})),
}));
jest.mock('~/utils', () => ({
logAxiosError: jest.fn(),
}));
jest.mock('~/server/services/Config', () => ({}));
jest.mock('~/server/services/Files/strategies', () => ({

330
api/server/services/ModelService.js
View file

@ -1,330 +0,0 @@
const axios = require('axios');
const { logger } = require('@librechat/data-schemas');
const { HttpsProxyAgent } = require('https-proxy-agent');
const { logAxiosError, inputSchema, processModelData, isUserProvided } = require('@librechat/api');
const {
CacheKeys,
defaultModels,
KnownEndpoints,
EModelEndpoint,
} = require('librechat-data-provider');
const { OllamaClient } = require('~/app/clients/OllamaClient');
const { config } = require('./Config/EndpointService');
const getLogStores = require('~/cache/getLogStores');
const { extractBaseURL } = require('~/utils');
/**
* Splits a string by commas and trims each resulting value.
* @param {string} input - The input string to split.
* @returns {string[]} An array of trimmed values.
*/
const splitAndTrim = (input) => {
if (!input || typeof input !== 'string') {
return [];
}
return input
.split(',')
.map((item) => item.trim())
.filter(Boolean);
};
/**
* Fetches OpenAI models from the specified base API path or Azure, based on the provided configuration.
*
* @param {Object} params - The parameters for fetching the models.
* @param {Object} params.user - The user ID to send to the API.
* @param {string} params.apiKey - The API key for authentication with the API.
* @param {string} params.baseURL - The base path URL for the API.
* @param {string} [params.name='OpenAI'] - The name of the API; defaults to 'OpenAI'.
* @param {boolean} [params.direct=false] - Whether `directEndpoint` was configured
* @param {boolean} [params.azure=false] - Whether to fetch models from Azure.
* @param {boolean} [params.userIdQuery=false] - Whether to send the user ID as a query parameter.
* @param {boolean} [params.createTokenConfig=true] - Whether to create a token configuration from the API response.
* @param {string} [params.tokenKey] - The cache key to save the token configuration. Uses `name` if omitted.
* @param {Record<string, string>} [params.headers] - Optional headers for the request.
* @param {Partial<IUser>} [params.userObject] - Optional user object for header resolution.
* @returns {Promise<string[]>} A promise that resolves to an array of model identifiers.
* @async
*/
const fetchModels = async ({
user,
apiKey,
baseURL: _baseURL,
name = EModelEndpoint.openAI,
direct,
azure = false,
userIdQuery = false,
createTokenConfig = true,
tokenKey,
headers,
userObject,
}) => {
let models = [];
const baseURL = direct ? extractBaseURL(_baseURL) : _baseURL;
if (!baseURL && !azure) {
return models;
}
if (!apiKey) {
return models;
}
if (name && name.toLowerCase().startsWith(KnownEndpoints.ollama)) {
try {
return await OllamaClient.fetchModels(baseURL, { headers, user: userObject });
} catch (ollamaError) {
const logMessage =
'Failed to fetch models from Ollama API. Attempting to fetch via OpenAI-compatible endpoint.';
logAxiosError({ message: logMessage, error: ollamaError });
}
}
try {
const options = {
headers: {
...(headers ?? {}),
},
timeout: 5000,
};
if (name === EModelEndpoint.anthropic) {
options.headers = {
'x-api-key': apiKey,
'anthropic-version': process.env.ANTHROPIC_VERSION || '2023-06-01',
};
} else {
options.headers.Authorization = `Bearer ${apiKey}`;
}
if (process.env.PROXY) {
options.httpsAgent = new HttpsProxyAgent(process.env.PROXY);
}
if (process.env.OPENAI_ORGANIZATION && baseURL.includes('openai')) {
options.headers['OpenAI-Organization'] = process.env.OPENAI_ORGANIZATION;
}
const url = new URL(`${baseURL.replace(/\/+$/, '')}${azure ? '' : '/models'}`);
if (user && userIdQuery) {
url.searchParams.append('user', user);
}
const res = await axios.get(url.toString(), options);
/** @type {z.infer<typeof inputSchema>} */
const input = res.data;
const validationResult = inputSchema.safeParse(input);
if (validationResult.success && createTokenConfig) {
const endpointTokenConfig = processModelData(input);
const cache = getLogStores(CacheKeys.TOKEN_CONFIG);
await cache.set(tokenKey ?? name, endpointTokenConfig);
}
models = input.data.map((item) => item.id);
} catch (error) {
const logMessage = `Failed to fetch models from ${azure ? 'Azure ' : ''}${name} API`;
logAxiosError({ message: logMessage, error });
}
return models;
};
/**
* Fetches models from the specified API path or Azure, based on the provided options.
* @async
* @function
* @param {object} opts - The options for fetching the models.
* @param {string} opts.user - The user ID to send to the API.
* @param {boolean} [opts.azure=false] - Whether to fetch models from Azure.
* @param {boolean} [opts.assistants=false] - Whether to fetch models from Azure.
* @param {string[]} [_models=[]] - The models to use as a fallback.
*/
const fetchOpenAIModels = async (opts, _models = []) => {
let models = _models.slice() ?? [];
const { openAIApiKey } = config;
let apiKey = openAIApiKey;
const openaiBaseURL = 'https://api.openai.com/v1';
let baseURL = openaiBaseURL;
let reverseProxyUrl = process.env.OPENAI_REVERSE_PROXY;
if (opts.assistants && process.env.ASSISTANTS_BASE_URL) {
reverseProxyUrl = process.env.ASSISTANTS_BASE_URL;
} else if (opts.azure) {
return models;
// const azure = getAzureCredentials();
// baseURL = (genAzureChatCompletion(azure))
// .split('/deployments')[0]
// .concat(`/models?api-version=${azure.azureOpenAIApiVersion}`);
// apiKey = azureOpenAIApiKey;
}
if (reverseProxyUrl) {
baseURL = extractBaseURL(reverseProxyUrl);
}
const modelsCache = getLogStores(CacheKeys.MODEL_QUERIES);
const cachedModels = await modelsCache.get(baseURL);
if (cachedModels) {
return cachedModels;
}
if (baseURL || opts.azure) {
models = await fetchModels({
apiKey,
baseURL,
azure: opts.azure,
user: opts.user,
name: EModelEndpoint.openAI,
});
}
if (models.length === 0) {
return _models;
}
if (baseURL === openaiBaseURL) {
const regex = /(text-davinci-003|gpt-|o\d+)/;
const excludeRegex = /audio|realtime/;
models = models.filter((model) => regex.test(model) && !excludeRegex.test(model));
const instructModels = models.filter((model) => model.includes('instruct'));
const otherModels = models.filter((model) => !model.includes('instruct'));
models = otherModels.concat(instructModels);
}
await modelsCache.set(baseURL, models);
return models;
};
/**
* Loads the default models for the application.
* @async
* @function
* @param {object} opts - The options for fetching the models.
* @param {string} opts.user - The user ID to send to the API.
* @param {boolean} [opts.azure=false] - Whether to fetch models from Azure.
* @param {boolean} [opts.assistants=false] - Whether to fetch models for the Assistants endpoint.
*/
const getOpenAIModels = async (opts) => {
let models = defaultModels[EModelEndpoint.openAI];
if (opts.assistants) {
models = defaultModels[EModelEndpoint.assistants];
} else if (opts.azure) {
models = defaultModels[EModelEndpoint.azureAssistants];
}
let key;
if (opts.assistants) {
key = 'ASSISTANTS_MODELS';
} else if (opts.azure) {
key = 'AZURE_OPENAI_MODELS';
} else {
key = 'OPENAI_MODELS';
}
if (process.env[key]) {
models = splitAndTrim(process.env[key]);
return models;
}
if (config.userProvidedOpenAI) {
return models;
}
return await fetchOpenAIModels(opts, models);
};
/**
* Fetches models from the Anthropic API.
* @async
* @function
* @param {object} opts - The options for fetching the models.
* @param {string} opts.user - The user ID to send to the API.
* @param {string[]} [_models=[]] - The models to use as a fallback.
*/
const fetchAnthropicModels = async (opts, _models = []) => {
let models = _models.slice() ?? [];
let apiKey = process.env.ANTHROPIC_API_KEY;
const anthropicBaseURL = 'https://api.anthropic.com/v1';
let baseURL = anthropicBaseURL;
let reverseProxyUrl = process.env.ANTHROPIC_REVERSE_PROXY;
if (reverseProxyUrl) {
baseURL = extractBaseURL(reverseProxyUrl);
}
if (!apiKey) {
return models;
}
const modelsCache = getLogStores(CacheKeys.MODEL_QUERIES);
const cachedModels = await modelsCache.get(baseURL);
if (cachedModels) {
return cachedModels;
}
if (baseURL) {
models = await fetchModels({
apiKey,
baseURL,
user: opts.user,
name: EModelEndpoint.anthropic,
tokenKey: EModelEndpoint.anthropic,
});
}
if (models.length === 0) {
return _models;
}
await modelsCache.set(baseURL, models);
return models;
};
const getAnthropicModels = async (opts = {}) => {
let models = defaultModels[EModelEndpoint.anthropic];
if (process.env.ANTHROPIC_MODELS) {
models = splitAndTrim(process.env.ANTHROPIC_MODELS);
return models;
}
if (isUserProvided(process.env.ANTHROPIC_API_KEY)) {
return models;
}
try {
return await fetchAnthropicModels(opts, models);
} catch (error) {
logger.error('Error fetching Anthropic models:', error);
return models;
}
};
const getGoogleModels = () => {
let models = defaultModels[EModelEndpoint.google];
if (process.env.GOOGLE_MODELS) {
models = splitAndTrim(process.env.GOOGLE_MODELS);
}
return models;
};
const getBedrockModels = () => {
let models = defaultModels[EModelEndpoint.bedrock];
if (process.env.BEDROCK_AWS_MODELS) {
models = splitAndTrim(process.env.BEDROCK_AWS_MODELS);
}
return models;
};
module.exports = {
fetchModels,
splitAndTrim,
getOpenAIModels,
getGoogleModels,
getBedrockModels,
getAnthropicModels,
};

619
api/server/services/ModelService.spec.js
View file

@ -1,619 +0,0 @@
const axios = require('axios');
const { logAxiosError, resolveHeaders } = require('@librechat/api');
const { EModelEndpoint, defaultModels } = require('librechat-data-provider');
const {
fetchModels,
splitAndTrim,
getOpenAIModels,
getGoogleModels,
getBedrockModels,
getAnthropicModels,
} = require('./ModelService');
jest.mock('@librechat/api', () => {
const originalUtils = jest.requireActual('@librechat/api');
return {
...originalUtils,
processModelData: jest.fn((...args) => {
return originalUtils.processModelData(...args);
}),
logAxiosError: jest.fn(),
resolveHeaders: jest.fn((options) => options?.headers || {}),
};
});
jest.mock('axios');
jest.mock('~/cache/getLogStores', () =>
jest.fn().mockImplementation(() => ({
get: jest.fn().mockResolvedValue(undefined),
set: jest.fn().mockResolvedValue(true),
})),
);
jest.mock('@librechat/data-schemas', () => ({
...jest.requireActual('@librechat/data-schemas'),
logger: {
error: jest.fn(),
},
}));
jest.mock('./Config/EndpointService', () => ({
config: {
openAIApiKey: 'mockedApiKey',
userProvidedOpenAI: false,
},
}));
axios.get.mockResolvedValue({
data: {
data: [{ id: 'model-1' }, { id: 'model-2' }],
},
});
describe('fetchModels', () => {
it('fetches models successfully from the API', async () => {
const models = await fetchModels({
user: 'user123',
apiKey: 'testApiKey',
baseURL: 'https://api.test.com',
name: 'TestAPI',
});
expect(models).toEqual(['model-1', 'model-2']);
expect(axios.get).toHaveBeenCalledWith(
expect.stringContaining('https://api.test.com/models'),
expect.any(Object),
);
});
it('adds the user ID to the models query when option and ID are passed', async () => {
const models = await fetchModels({
user: 'user123',
apiKey: 'testApiKey',
baseURL: 'https://api.test.com',
userIdQuery: true,
name: 'TestAPI',
});
expect(models).toEqual(['model-1', 'model-2']);
expect(axios.get).toHaveBeenCalledWith(
expect.stringContaining('https://api.test.com/models?user=user123'),
expect.any(Object),
);
});
it('should pass custom headers to the API request', async () => {
const customHeaders = {
'X-Custom-Header': 'custom-value',
'X-API-Version': 'v2',
};
await fetchModels({
user: 'user123',
apiKey: 'testApiKey',
baseURL: 'https://api.test.com',
name: 'TestAPI',
headers: customHeaders,
});
expect(axios.get).toHaveBeenCalledWith(
expect.stringContaining('https://api.test.com/models'),
expect.objectContaining({
headers: expect.objectContaining({
'X-Custom-Header': 'custom-value',
'X-API-Version': 'v2',
Authorization: 'Bearer testApiKey',
}),
}),
);
});
it('should handle null headers gracefully', async () => {
await fetchModels({
user: 'user123',
apiKey: 'testApiKey',
baseURL: 'https://api.test.com',
name: 'TestAPI',
headers: null,
});
expect(axios.get).toHaveBeenCalledWith(
expect.stringContaining('https://api.test.com/models'),
expect.objectContaining({
headers: expect.objectContaining({
Authorization: 'Bearer testApiKey',
}),
}),
);
});
it('should handle undefined headers gracefully', async () => {
await fetchModels({
user: 'user123',
apiKey: 'testApiKey',
baseURL: 'https://api.test.com',
name: 'TestAPI',
headers: undefined,
});
expect(axios.get).toHaveBeenCalledWith(
expect.stringContaining('https://api.test.com/models'),
expect.objectContaining({
headers: expect.objectContaining({
Authorization: 'Bearer testApiKey',
}),
}),
);
});
afterEach(() => {
jest.clearAllMocks();
});
});
describe('fetchModels with createTokenConfig true', () => {
const data = {
data: [
{
id: 'model-1',
pricing: {
prompt: '0.002',
completion: '0.001',
},
context_length: 1024,
},
{
id: 'model-2',
pricing: {
prompt: '0.003',
completion: '0.0015',
},
context_length: 2048,
},
],
};
beforeEach(() => {
// Clears the mock's history before each test
const _utils = require('@librechat/api');
axios.get.mockResolvedValue({ data });
});
it('creates and stores token configuration if createTokenConfig is true', async () => {
await fetchModels({
user: 'user123',
apiKey: 'testApiKey',
baseURL: 'https://api.test.com',
createTokenConfig: true,
});
const { processModelData } = require('@librechat/api');
expect(processModelData).toHaveBeenCalled();
expect(processModelData).toHaveBeenCalledWith(data);
});
});
describe('getOpenAIModels', () => {
let originalEnv;
beforeEach(() => {
originalEnv = { ...process.env };
axios.get.mockRejectedValue(new Error('Network error'));
});
afterEach(() => {
process.env = originalEnv;
axios.get.mockReset();
});
it('returns default models when no environment configurations are provided (and fetch fails)', async () => {
const models = await getOpenAIModels({ user: 'user456' });
expect(models).toContain('gpt-4');
});
it('returns `AZURE_OPENAI_MODELS` with `azure` flag (and fetch fails)', async () => {
process.env.AZURE_OPENAI_MODELS = 'azure-model,azure-model-2';
const models = await getOpenAIModels({ azure: true });
expect(models).toEqual(expect.arrayContaining(['azure-model', 'azure-model-2']));
});
it('returns `OPENAI_MODELS` with no flags (and fetch fails)', async () => {
process.env.OPENAI_MODELS = 'openai-model,openai-model-2';
const models = await getOpenAIModels({});
expect(models).toEqual(expect.arrayContaining(['openai-model', 'openai-model-2']));
});
it('utilizes proxy configuration when PROXY is set', async () => {
axios.get.mockResolvedValue({
data: {
data: [],
},
});
process.env.PROXY = 'http://localhost:8888';
await getOpenAIModels({ user: 'user456' });
expect(axios.get).toHaveBeenCalledWith(
expect.any(String),
expect.objectContaining({
httpsAgent: expect.anything(),
}),
);
});
});
describe('getOpenAIModels with mocked config', () => {
it('uses alternative behavior when userProvidedOpenAI is true', async () => {
jest.mock('./Config/EndpointService', () => ({
config: {
openAIApiKey: 'mockedApiKey',
userProvidedOpenAI: true,
},
}));
jest.mock('librechat-data-provider', () => {
const original = jest.requireActual('librechat-data-provider');
return {
...original,
defaultModels: {
[original.EModelEndpoint.openAI]: ['some-default-model'],
},
};
});
jest.resetModules();
const { getOpenAIModels } = require('./ModelService');
const models = await getOpenAIModels({ user: 'user456' });
expect(models).toContain('some-default-model');
});
});
describe('getOpenAIModels sorting behavior', () => {
beforeEach(() => {
axios.get.mockResolvedValue({
data: {
data: [
{ id: 'gpt-3.5-turbo-instruct-0914' },
{ id: 'gpt-3.5-turbo-instruct' },
{ id: 'gpt-3.5-turbo' },
{ id: 'gpt-4-0314' },
{ id: 'gpt-4-turbo-preview' },
],
},
});
});
it('ensures instruct models are listed last', async () => {
const models = await getOpenAIModels({ user: 'user456' });
// Check if the last model is an "instruct" model
expect(models[models.length - 1]).toMatch(/instruct/);
// Check if the "instruct" models are placed at the end
const instructIndexes = models
.map((model, index) => (model.includes('instruct') ? index : -1))
.filter((index) => index !== -1);
const nonInstructIndexes = models
.map((model, index) => (!model.includes('instruct') ? index : -1))
.filter((index) => index !== -1);
expect(Math.max(...nonInstructIndexes)).toBeLessThan(Math.min(...instructIndexes));
const expectedOrder = [
'gpt-3.5-turbo',
'gpt-4-0314',
'gpt-4-turbo-preview',
'gpt-3.5-turbo-instruct-0914',
'gpt-3.5-turbo-instruct',
];
expect(models).toEqual(expectedOrder);
});
afterEach(() => {
jest.clearAllMocks();
});
});
describe('fetchModels with Ollama specific logic', () => {
const mockOllamaData = {
data: {
models: [{ name: 'Ollama-Base' }, { name: 'Ollama-Advanced' }],
},
};
beforeEach(() => {
axios.get.mockResolvedValue(mockOllamaData);
});
afterEach(() => {
jest.clearAllMocks();
});
it('should fetch Ollama models when name starts with "ollama"', async () => {
const models = await fetchModels({
user: 'user789',
apiKey: 'testApiKey',
baseURL: 'https://api.ollama.test.com',
name: 'OllamaAPI',
});
expect(models).toEqual(['Ollama-Base', 'Ollama-Advanced']);
expect(axios.get).toHaveBeenCalledWith('https://api.ollama.test.com/api/tags', {
headers: {},
timeout: 5000,
});
});
it('should pass headers and user object to Ollama fetchModels', async () => {
const customHeaders = {
'Content-Type': 'application/json',
Authorization: 'Bearer custom-token',
};
const userObject = {
id: 'user789',
email: 'test@example.com',
};
resolveHeaders.mockReturnValueOnce(customHeaders);
const models = await fetchModels({
user: 'user789',
apiKey: 'testApiKey',
baseURL: 'https://api.ollama.test.com',
name: 'ollama',
headers: customHeaders,
userObject,
});
expect(models).toEqual(['Ollama-Base', 'Ollama-Advanced']);
expect(resolveHeaders).toHaveBeenCalledWith({
headers: customHeaders,
user: userObject,
});
expect(axios.get).toHaveBeenCalledWith('https://api.ollama.test.com/api/tags', {
headers: customHeaders,
timeout: 5000,
});
});
it('should handle errors gracefully when fetching Ollama models fails and fallback to OpenAI-compatible fetch', async () => {
axios.get.mockRejectedValueOnce(new Error('Ollama API error'));
axios.get.mockResolvedValueOnce({
data: {
data: [{ id: 'fallback-model-1' }, { id: 'fallback-model-2' }],
},
});
const models = await fetchModels({
user: 'user789',
apiKey: 'testApiKey',
baseURL: 'https://api.ollama.test.com',
name: 'OllamaAPI',
});
expect(models).toEqual(['fallback-model-1', 'fallback-model-2']);
expect(logAxiosError).toHaveBeenCalledWith({
message:
'Failed to fetch models from Ollama API. Attempting to fetch via OpenAI-compatible endpoint.',
error: expect.any(Error),
});
expect(axios.get).toHaveBeenCalledTimes(2);
});
it('should return an empty array if no baseURL is provided', async () => {
const models = await fetchModels({
user: 'user789',
apiKey: 'testApiKey',
name: 'OllamaAPI',
});
expect(models).toEqual([]);
});
it('should not fetch Ollama models if the name does not start with "ollama"', async () => {
// Mock axios to return a different set of models for non-Ollama API calls
axios.get.mockResolvedValue({
data: {
data: [{ id: 'model-1' }, { id: 'model-2' }],
},
});
const models = await fetchModels({
user: 'user789',
apiKey: 'testApiKey',
baseURL: 'https://api.test.com',
name: 'TestAPI',
});
expect(models).toEqual(['model-1', 'model-2']);
expect(axios.get).toHaveBeenCalledWith(
'https://api.test.com/models', // Ensure the correct API endpoint is called
expect.any(Object), // Ensuring some object (headers, etc.) is passed
);
});
});
describe('fetchModels URL construction with trailing slashes', () => {
beforeEach(() => {
axios.get.mockResolvedValue({
data: {
data: [{ id: 'model-1' }, { id: 'model-2' }],
},
});
});
afterEach(() => {
jest.clearAllMocks();
});
it('should not create double slashes when baseURL has a trailing slash', async () => {
await fetchModels({
user: 'user123',
apiKey: 'testApiKey',
baseURL: 'https://api.test.com/v1/',
name: 'TestAPI',
});
expect(axios.get).toHaveBeenCalledWith('https://api.test.com/v1/models', expect.any(Object));
});
it('should handle baseURL without trailing slash normally', async () => {
await fetchModels({
user: 'user123',
apiKey: 'testApiKey',
baseURL: 'https://api.test.com/v1',
name: 'TestAPI',
});
expect(axios.get).toHaveBeenCalledWith('https://api.test.com/v1/models', expect.any(Object));
});
it('should handle baseURL with multiple trailing slashes', async () => {
await fetchModels({
user: 'user123',
apiKey: 'testApiKey',
baseURL: 'https://api.test.com/v1///',
name: 'TestAPI',
});
expect(axios.get).toHaveBeenCalledWith('https://api.test.com/v1/models', expect.any(Object));
});
it('should correctly append query params after stripping trailing slashes', async () => {
await fetchModels({
user: 'user123',
apiKey: 'testApiKey',
baseURL: 'https://api.test.com/v1/',
name: 'TestAPI',
userIdQuery: true,
});
expect(axios.get).toHaveBeenCalledWith(
'https://api.test.com/v1/models?user=user123',
expect.any(Object),
);
});
});
describe('splitAndTrim', () => {
it('should split a string by commas and trim each value', () => {
const input = ' model1, model2 , model3,model4 ';
const expected = ['model1', 'model2', 'model3', 'model4'];
expect(splitAndTrim(input)).toEqual(expected);
});
it('should return an empty array for empty input', () => {
expect(splitAndTrim('')).toEqual([]);
});
it('should return an empty array for null input', () => {
expect(splitAndTrim(null)).toEqual([]);
});
it('should return an empty array for undefined input', () => {
expect(splitAndTrim(undefined)).toEqual([]);
});
it('should filter out empty values after trimming', () => {
const input = 'model1,, ,model2,';
const expected = ['model1', 'model2'];
expect(splitAndTrim(input)).toEqual(expected);
});
});
describe('getAnthropicModels', () => {
it('returns default models when ANTHROPIC_MODELS is not set', async () => {
delete process.env.ANTHROPIC_MODELS;
const models = await getAnthropicModels();
expect(models).toEqual(defaultModels[EModelEndpoint.anthropic]);
});
it('returns models from ANTHROPIC_MODELS when set', async () => {
process.env.ANTHROPIC_MODELS = 'claude-1, claude-2 ';
const models = await getAnthropicModels();
expect(models).toEqual(['claude-1', 'claude-2']);
});
it('should use Anthropic-specific headers when fetching models', async () => {
delete process.env.ANTHROPIC_MODELS;
process.env.ANTHROPIC_API_KEY = 'test-anthropic-key';
axios.get.mockResolvedValue({
data: {
data: [{ id: 'claude-3' }, { id: 'claude-4' }],
},
});
await fetchModels({
user: 'user123',
apiKey: 'test-anthropic-key',
baseURL: 'https://api.anthropic.com/v1',
name: EModelEndpoint.anthropic,
});
expect(axios.get).toHaveBeenCalledWith(
expect.any(String),
expect.objectContaining({
headers: {
'x-api-key': 'test-anthropic-key',
'anthropic-version': expect.any(String),
},
}),
);
});
it('should pass custom headers for Anthropic endpoint', async () => {
const customHeaders = {
'X-Custom-Header': 'custom-value',
};
axios.get.mockResolvedValue({
data: {
data: [{ id: 'claude-3' }],
},
});
await fetchModels({
user: 'user123',
apiKey: 'test-anthropic-key',
baseURL: 'https://api.anthropic.com/v1',
name: EModelEndpoint.anthropic,
headers: customHeaders,
});
expect(axios.get).toHaveBeenCalledWith(
expect.any(String),
expect.objectContaining({
headers: {
'x-api-key': 'test-anthropic-key',
'anthropic-version': expect.any(String),
},
}),
);
});
});
describe('getGoogleModels', () => {
it('returns default models when GOOGLE_MODELS is not set', () => {
delete process.env.GOOGLE_MODELS;
const models = getGoogleModels();
expect(models).toEqual(defaultModels[EModelEndpoint.google]);
});
it('returns models from GOOGLE_MODELS when set', () => {
process.env.GOOGLE_MODELS = 'gemini-pro, bard ';
const models = getGoogleModels();
expect(models).toEqual(['gemini-pro', 'bard']);
});
});
describe('getBedrockModels', () => {
it('returns default models when BEDROCK_AWS_MODELS is not set', () => {
delete process.env.BEDROCK_AWS_MODELS;
const models = getBedrockModels();
expect(models).toEqual(defaultModels[EModelEndpoint.bedrock]);
});
it('returns models from BEDROCK_AWS_MODELS when set', () => {
process.env.BEDROCK_AWS_MODELS = 'anthropic.claude-v2, ai21.j2-ultra ';
const models = getBedrockModels();
expect(models).toEqual(['anthropic.claude-v2', 'ai21.j2-ultra']);
});
});

183
api/server/services/UserService.js
View file

@ -1,183 +0,0 @@
const { logger } = require('@librechat/data-schemas');
const { encrypt, decrypt } = require('@librechat/api');
const { ErrorTypes } = require('librechat-data-provider');
const { updateUser } = require('~/models');
const { Key } = require('~/db/models');
/**
* Updates the plugins for a user based on the action specified (install/uninstall).
* @async
* @param {Object} user - The user whose plugins are to be updated.
* @param {string} pluginKey - The key of the plugin to install or uninstall.
* @param {'install' | 'uninstall'} action - The action to perform, 'install' or 'uninstall'.
* @returns {Promise<Object>} The result of the update operation.
* @throws Logs the error internally if the update operation fails.
* @description This function updates the plugin array of a user document based on the specified action.
* It adds a plugin key to the plugins array for an 'install' action, and removes it for an 'uninstall' action.
*/
const updateUserPluginsService = async (user, pluginKey, action) => {
try {
const userPlugins = user.plugins || [];
if (action === 'install') {
return await updateUser(user._id, { plugins: [...userPlugins, pluginKey] });
} else if (action === 'uninstall') {
return await updateUser(user._id, {
plugins: userPlugins.filter((plugin) => plugin !== pluginKey),
});
}
} catch (err) {
logger.error('[updateUserPluginsService]', err);
return err;
}
};
/**
* Retrieves and decrypts the key value for a given user identified by userId and identifier name.
* @param {Object} params - The parameters object.
* @param {string} params.userId - The unique identifier for the user.
* @param {string} params.name - The name associated with the key.
* @returns {Promise<string>} The decrypted key value.
* @throws {Error} Throws an error if the key is not found or if there is a problem during key retrieval.
* @description This function searches for a user's key in the database using their userId and name.
* If found, it decrypts the value of the key and returns it. If no key is found, it throws
* an error indicating that there is no user key available.
*/
const getUserKey = async ({ userId, name }) => {
const keyValue = await Key.findOne({ userId, name }).lean();
if (!keyValue) {
throw new Error(
JSON.stringify({
type: ErrorTypes.NO_USER_KEY,
}),
);
}
return await decrypt(keyValue.value);
};
/**
* Retrieves, decrypts, and parses the key values for a given user identified by userId and name.
* @param {Object} params - The parameters object.
* @param {string} params.userId - The unique identifier for the user.
* @param {string} params.name - The name associated with the key.
* @returns {Promise<Record<string,string>>} The decrypted and parsed key values.
* @throws {Error} Throws an error if the key is invalid or if there is a problem during key value parsing.
* @description This function retrieves a user's encrypted key using their userId and name, decrypts it,
* and then attempts to parse the decrypted string into a JSON object. If the parsing fails,
* it throws an error indicating that the user key is invalid.
*/
const getUserKeyValues = async ({ userId, name }) => {
let userValues = await getUserKey({ userId, name });
try {
userValues = JSON.parse(userValues);
} catch (e) {
logger.error('[getUserKeyValues]', e);
throw new Error(
JSON.stringify({
type: ErrorTypes.INVALID_USER_KEY,
}),
);
}
return userValues;
};
/**
* Retrieves the expiry information of a user's key identified by userId and name.
* @async
* @param {Object} params - The parameters object.
* @param {string} params.userId - The unique identifier for the user.
* @param {string} params.name - The name associated with the key.
* @returns {Promise<{expiresAt: Date | null}>} The expiry date of the key or null if the key doesn't exist.
* @description This function fetches a user's key from the database using their userId and name and
* returns its expiry date. If the key is not found, it returns null for the expiry date.
*/
const getUserKeyExpiry = async ({ userId, name }) => {
const keyValue = await Key.findOne({ userId, name }).lean();
if (!keyValue) {
return { expiresAt: null };
}
return { expiresAt: keyValue.expiresAt || 'never' };
};
/**
* Updates or inserts a new key for a given user identified by userId and name, with a specified value and expiry date.
* @async
* @param {Object} params - The parameters object.
* @param {string} params.userId - The unique identifier for the user.
* @param {string} params.name - The name associated with the key.
* @param {string} params.value - The value to be encrypted and stored as the key's value.
* @param {Date} params.expiresAt - The expiry date for the key [optional]
* @returns {Promise<Object>} The updated or newly inserted key document.
* @description This function either updates an existing user key or inserts a new one into the database,
* after encrypting the provided value. It sets the provided expiry date for the key (or unsets for no expiry).
*/
const updateUserKey = async ({ userId, name, value, expiresAt = null }) => {
const encryptedValue = await encrypt(value);
let updateObject = {
userId,
name,
value: encryptedValue,
};
const updateQuery = { $set: updateObject };
// add expiresAt to the update object if it's not null
if (expiresAt) {
updateObject.expiresAt = new Date(expiresAt);
} else {
// make sure to remove if already present
updateQuery.$unset = { expiresAt };
}
return await Key.findOneAndUpdate({ userId, name }, updateQuery, {
upsert: true,
new: true,
}).lean();
};
/**
* Deletes a key or all keys for a given user identified by userId, optionally based on a specified name.
* @async
* @param {Object} params - The parameters object.
* @param {string} params.userId - The unique identifier for the user.
* @param {string} [params.name] - The name associated with the key to delete. If not provided and all is true, deletes all keys.
* @param {boolean} [params.all=false] - Whether to delete all keys for the user.
* @returns {Promise<Object>} The result of the deletion operation.
* @description This function deletes a specific key or all keys for a user from the database.
* If a name is provided and all is false, it deletes only the key with that name.
* If all is true, it ignores the name and deletes all keys for the user.
*/
const deleteUserKey = async ({ userId, name, all = false }) => {
if (all) {
return await Key.deleteMany({ userId });
}
await Key.findOneAndDelete({ userId, name }).lean();
};
/**
* Checks if a user key has expired based on the provided expiration date and endpoint.
* If the key has expired, it throws an Error with details including the type of error, the expiration date, and the endpoint.
*
* @param {string} expiresAt - The expiration date of the user key in a format that can be parsed by the Date constructor.
* @param {string} endpoint - The endpoint associated with the user key to be checked.
* @throws {Error} Throws an error if the user key has expired. The error message is a stringified JSON object
* containing the type of error (`ErrorTypes.EXPIRED_USER_KEY`), the expiration date in the local string format, and the endpoint.
*/
const checkUserKeyExpiry = (expiresAt, endpoint) => {
const expiresAtDate = new Date(expiresAt);
if (expiresAtDate < new Date()) {
const errorMessage = JSON.stringify({
type: ErrorTypes.EXPIRED_USER_KEY,
expiredAt: expiresAtDate.toLocaleString(),
endpoint,
});
throw new Error(errorMessage);
}
};
module.exports = {
getUserKey,
updateUserKey,
deleteUserKey,
getUserKeyValues,
getUserKeyExpiry,
checkUserKeyExpiry,
updateUserPluginsService,
};

2
api/server/services/twoFactorService.js
View file

@ -1,5 +1,5 @@
const { webcrypto } = require('node:crypto');
const { hashBackupCode, decryptV3, decryptV2 } = require('@librechat/api');
const { hashBackupCode, decryptV3, decryptV2 } = require('@librechat/data-schemas');
const { updateUser } = require('~/models');
// Base32 alphabet for TOTP secret encoding.