mirror of
https://github.com/danny-avila/LibreChat.git
synced 2025-09-22 06:00:56 +02:00

* WIP: app.locals refactoring
WIP: appConfig
fix: update memory configuration retrieval to use getAppConfig based on user role
fix: update comment for AppConfig interface to clarify purpose
🏷️ refactor: Update tests to use getAppConfig for endpoint configurations
ci: Update AppService tests to initialize app config instead of app.locals
ci: Integrate getAppConfig into remaining tests
refactor: Update multer storage destination to use promise-based getAppConfig and improve error handling in tests
refactor: Rename initializeAppConfig to setAppConfig and update related tests
ci: Mock getAppConfig in various tests to provide default configurations
refactor: Update convertMCPToolsToPlugins to use mcpManager for server configuration and adjust related tests
chore: rename `Config/getAppConfig` -> `Config/app`
fix: streamline OpenAI image tools configuration by removing direct appConfig dependency and using function parameters
chore: correct parameter documentation for imageOutputType in ToolService.js
refactor: remove `getCustomConfig` dependency in config route
refactor: update domain validation to use appConfig for allowed domains
refactor: use appConfig registration property
chore: remove app parameter from AppService invocation
refactor: update AppConfig interface to correct registration and turnstile configurations
refactor: remove getCustomConfig dependency and use getAppConfig in PluginController, multer, and MCP services
refactor: replace getCustomConfig with getAppConfig in STTService, TTSService, and related files
refactor: replace getCustomConfig with getAppConfig in Conversation and Message models, update tempChatRetention functions to use AppConfig type
refactor: update getAppConfig calls in Conversation and Message models to include user role for temporary chat expiration
ci: update related tests
refactor: update getAppConfig call in getCustomConfigSpeech to include user role
fix: update appConfig usage to access allowedDomains from actions instead of registration
refactor: enhance AppConfig to include fileStrategies and update related file strategy logic
refactor: update imports to use normalizeEndpointName from @librechat/api and remove redundant definitions
chore: remove deprecated unused RunManager
refactor: get balance config primarily from appConfig
refactor: remove customConfig dependency for appConfig and streamline loadConfigModels logic
refactor: remove getCustomConfig usage and use app config in file citations
refactor: consolidate endpoint loading logic into loadEndpoints function
refactor: update appConfig access to use endpoints structure across various services
refactor: implement custom endpoints configuration and streamline endpoint loading logic
refactor: update getAppConfig call to include user role parameter
refactor: streamline endpoint configuration and enhance appConfig usage across services
refactor: replace getMCPAuthMap with getUserMCPAuthMap and remove unused getCustomConfig file
refactor: add type annotation for loadedEndpoints in loadEndpoints function
refactor: move /services/Files/images/parse to TS API
chore: add missing FILE_CITATIONS permission to IRole interface
refactor: restructure toolkits to TS API
refactor: separate manifest logic into its own module
refactor: consolidate tool loading logic into a new tools module for startup logic
refactor: move interface config logic to TS API
refactor: migrate checkEmailConfig to TypeScript and update imports
refactor: add FunctionTool interface and availableTools to AppConfig
refactor: decouple caching and DB operations from AppService, make part of consolidated `getAppConfig`
WIP: fix tests
* fix: rebase conflicts
* refactor: remove app.locals references
* refactor: replace getBalanceConfig with getAppConfig in various strategies and middleware
* refactor: replace appConfig?.balance with getBalanceConfig in various controllers and clients
* test: add balance configuration to titleConvo method in AgentClient tests
* chore: remove unused `openai-chat-tokens` package
* chore: remove unused imports in initializeMCPs.js
* refactor: update balance configuration to use getAppConfig instead of getBalanceConfig
* refactor: integrate configMiddleware for centralized configuration handling
* refactor: optimize email domain validation by removing unnecessary async calls
* refactor: simplify multer storage configuration by removing async calls
* refactor: reorder imports for better readability in user.js
* refactor: replace getAppConfig calls with req.config for improved performance
* chore: replace getAppConfig calls with req.config in tests for centralized configuration handling
* chore: remove unused override config
* refactor: add configMiddleware to endpoint route and replace getAppConfig with req.config
* chore: remove customConfig parameter from TTSService constructor
* refactor: pass appConfig from request to processFileCitations for improved configuration handling
* refactor: remove configMiddleware from endpoint route and retrieve appConfig directly in getEndpointsConfig if not in `req.config`
* test: add mockAppConfig to processFileCitations tests for improved configuration handling
* fix: pass req.config to hasCustomUserVars and call without await after synchronous refactor
* fix: type safety in useExportConversation
* refactor: retrieve appConfig using getAppConfig in PluginController and remove configMiddleware from plugins route, to avoid always retrieving when plugins are cached
* chore: change `MongoUser` typedef to `IUser`
* fix: Add `user` and `config` fields to ServerRequest and update JSDoc type annotations from Express.Request to ServerRequest
* fix: remove unused setAppConfig mock from Server configuration tests
314 lines
11 KiB
JavaScript
314 lines
11 KiB
JavaScript
const { logger } = require('@librechat/data-schemas');
|
|
const { createTempChatExpirationDate } = require('@librechat/api');
|
|
const { getMessages, deleteMessages } = require('./Message');
|
|
const { Conversation } = require('~/db/models');
|
|
|
|
/**
|
|
* Searches for a conversation by conversationId and returns a lean document with only conversationId and user.
|
|
* @param {string} conversationId - The conversation's ID.
|
|
* @returns {Promise<{conversationId: string, user: string} | null>} The conversation object with selected fields or null if not found.
|
|
*/
|
|
const searchConversation = async (conversationId) => {
|
|
try {
|
|
return await Conversation.findOne({ conversationId }, 'conversationId user').lean();
|
|
} catch (error) {
|
|
logger.error('[searchConversation] Error searching conversation', error);
|
|
throw new Error('Error searching conversation');
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Retrieves a single conversation for a given user and conversation ID.
|
|
* @param {string} user - The user's ID.
|
|
* @param {string} conversationId - The conversation's ID.
|
|
* @returns {Promise<TConversation>} The conversation object.
|
|
*/
|
|
const getConvo = async (user, conversationId) => {
|
|
try {
|
|
return await Conversation.findOne({ user, conversationId }).lean();
|
|
} catch (error) {
|
|
logger.error('[getConvo] Error getting single conversation', error);
|
|
return { message: 'Error getting single conversation' };
|
|
}
|
|
};
|
|
|
|
const deleteNullOrEmptyConversations = async () => {
|
|
try {
|
|
const filter = {
|
|
$or: [
|
|
{ conversationId: null },
|
|
{ conversationId: '' },
|
|
{ conversationId: { $exists: false } },
|
|
],
|
|
};
|
|
|
|
const result = await Conversation.deleteMany(filter);
|
|
|
|
// Delete associated messages
|
|
const messageDeleteResult = await deleteMessages(filter);
|
|
|
|
logger.info(
|
|
`[deleteNullOrEmptyConversations] Deleted ${result.deletedCount} conversations and ${messageDeleteResult.deletedCount} messages`,
|
|
);
|
|
|
|
return {
|
|
conversations: result,
|
|
messages: messageDeleteResult,
|
|
};
|
|
} catch (error) {
|
|
logger.error('[deleteNullOrEmptyConversations] Error deleting conversations', error);
|
|
throw new Error('Error deleting conversations with null or empty conversationId');
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Searches for a conversation by conversationId and returns associated file ids.
|
|
* @param {string} conversationId - The conversation's ID.
|
|
* @returns {Promise<string[] | null>}
|
|
*/
|
|
const getConvoFiles = async (conversationId) => {
|
|
try {
|
|
return (await Conversation.findOne({ conversationId }, 'files').lean())?.files ?? [];
|
|
} catch (error) {
|
|
logger.error('[getConvoFiles] Error getting conversation files', error);
|
|
throw new Error('Error getting conversation files');
|
|
}
|
|
};
|
|
|
|
module.exports = {
|
|
getConvoFiles,
|
|
searchConversation,
|
|
deleteNullOrEmptyConversations,
|
|
/**
|
|
* Saves a conversation to the database.
|
|
* @param {Object} req - The request object.
|
|
* @param {string} conversationId - The conversation's ID.
|
|
* @param {Object} metadata - Additional metadata to log for operation.
|
|
* @returns {Promise<TConversation>} The conversation object.
|
|
*/
|
|
saveConvo: async (req, { conversationId, newConversationId, ...convo }, metadata) => {
|
|
try {
|
|
if (metadata?.context) {
|
|
logger.debug(`[saveConvo] ${metadata.context}`);
|
|
}
|
|
|
|
const messages = await getMessages({ conversationId }, '_id');
|
|
const update = { ...convo, messages, user: req.user.id };
|
|
|
|
if (newConversationId) {
|
|
update.conversationId = newConversationId;
|
|
}
|
|
|
|
if (req?.body?.isTemporary) {
|
|
try {
|
|
const appConfig = req.config;
|
|
update.expiredAt = createTempChatExpirationDate(appConfig?.interfaceConfig);
|
|
} catch (err) {
|
|
logger.error('Error creating temporary chat expiration date:', err);
|
|
logger.info(`---\`saveConvo\` context: ${metadata?.context}`);
|
|
update.expiredAt = null;
|
|
}
|
|
} else {
|
|
update.expiredAt = null;
|
|
}
|
|
|
|
/** @type {{ $set: Partial<TConversation>; $unset?: Record<keyof TConversation, number> }} */
|
|
const updateOperation = { $set: update };
|
|
if (metadata && metadata.unsetFields && Object.keys(metadata.unsetFields).length > 0) {
|
|
updateOperation.$unset = metadata.unsetFields;
|
|
}
|
|
|
|
/** Note: the resulting Model object is necessary for Meilisearch operations */
|
|
const conversation = await Conversation.findOneAndUpdate(
|
|
{ conversationId, user: req.user.id },
|
|
updateOperation,
|
|
{
|
|
new: true,
|
|
upsert: true,
|
|
},
|
|
);
|
|
|
|
return conversation.toObject();
|
|
} catch (error) {
|
|
logger.error('[saveConvo] Error saving conversation', error);
|
|
if (metadata && metadata?.context) {
|
|
logger.info(`[saveConvo] ${metadata.context}`);
|
|
}
|
|
return { message: 'Error saving conversation' };
|
|
}
|
|
},
|
|
bulkSaveConvos: async (conversations) => {
|
|
try {
|
|
const bulkOps = conversations.map((convo) => ({
|
|
updateOne: {
|
|
filter: { conversationId: convo.conversationId, user: convo.user },
|
|
update: convo,
|
|
upsert: true,
|
|
timestamps: false,
|
|
},
|
|
}));
|
|
|
|
const result = await Conversation.bulkWrite(bulkOps);
|
|
return result;
|
|
} catch (error) {
|
|
logger.error('[saveBulkConversations] Error saving conversations in bulk', error);
|
|
throw new Error('Failed to save conversations in bulk.');
|
|
}
|
|
},
|
|
getConvosByCursor: async (
|
|
user,
|
|
{ cursor, limit = 25, isArchived = false, tags, search, order = 'desc' } = {},
|
|
) => {
|
|
const filters = [{ user }];
|
|
if (isArchived) {
|
|
filters.push({ isArchived: true });
|
|
} else {
|
|
filters.push({ $or: [{ isArchived: false }, { isArchived: { $exists: false } }] });
|
|
}
|
|
|
|
if (Array.isArray(tags) && tags.length > 0) {
|
|
filters.push({ tags: { $in: tags } });
|
|
}
|
|
|
|
filters.push({ $or: [{ expiredAt: null }, { expiredAt: { $exists: false } }] });
|
|
|
|
if (search) {
|
|
try {
|
|
const meiliResults = await Conversation.meiliSearch(search);
|
|
const matchingIds = Array.isArray(meiliResults.hits)
|
|
? meiliResults.hits.map((result) => result.conversationId)
|
|
: [];
|
|
if (!matchingIds.length) {
|
|
return { conversations: [], nextCursor: null };
|
|
}
|
|
filters.push({ conversationId: { $in: matchingIds } });
|
|
} catch (error) {
|
|
logger.error('[getConvosByCursor] Error during meiliSearch', error);
|
|
return { message: 'Error during meiliSearch' };
|
|
}
|
|
}
|
|
|
|
if (cursor) {
|
|
filters.push({ updatedAt: { $lt: new Date(cursor) } });
|
|
}
|
|
|
|
const query = filters.length === 1 ? filters[0] : { $and: filters };
|
|
|
|
try {
|
|
const convos = await Conversation.find(query)
|
|
.select(
|
|
'conversationId endpoint title createdAt updatedAt user model agent_id assistant_id spec iconURL',
|
|
)
|
|
.sort({ updatedAt: order === 'asc' ? 1 : -1 })
|
|
.limit(limit + 1)
|
|
.lean();
|
|
|
|
let nextCursor = null;
|
|
if (convos.length > limit) {
|
|
const lastConvo = convos.pop();
|
|
nextCursor = lastConvo.updatedAt.toISOString();
|
|
}
|
|
|
|
return { conversations: convos, nextCursor };
|
|
} catch (error) {
|
|
logger.error('[getConvosByCursor] Error getting conversations', error);
|
|
return { message: 'Error getting conversations' };
|
|
}
|
|
},
|
|
getConvosQueried: async (user, convoIds, cursor = null, limit = 25) => {
|
|
try {
|
|
if (!convoIds?.length) {
|
|
return { conversations: [], nextCursor: null, convoMap: {} };
|
|
}
|
|
|
|
const conversationIds = convoIds.map((convo) => convo.conversationId);
|
|
|
|
const results = await Conversation.find({
|
|
user,
|
|
conversationId: { $in: conversationIds },
|
|
$or: [{ expiredAt: { $exists: false } }, { expiredAt: null }],
|
|
}).lean();
|
|
|
|
results.sort((a, b) => new Date(b.updatedAt) - new Date(a.updatedAt));
|
|
|
|
let filtered = results;
|
|
if (cursor && cursor !== 'start') {
|
|
const cursorDate = new Date(cursor);
|
|
filtered = results.filter((convo) => new Date(convo.updatedAt) < cursorDate);
|
|
}
|
|
|
|
const limited = filtered.slice(0, limit + 1);
|
|
let nextCursor = null;
|
|
if (limited.length > limit) {
|
|
const lastConvo = limited.pop();
|
|
nextCursor = lastConvo.updatedAt.toISOString();
|
|
}
|
|
|
|
const convoMap = {};
|
|
limited.forEach((convo) => {
|
|
convoMap[convo.conversationId] = convo;
|
|
});
|
|
|
|
return { conversations: limited, nextCursor, convoMap };
|
|
} catch (error) {
|
|
logger.error('[getConvosQueried] Error getting conversations', error);
|
|
return { message: 'Error fetching conversations' };
|
|
}
|
|
},
|
|
getConvo,
|
|
/* chore: this method is not properly error handled */
|
|
getConvoTitle: async (user, conversationId) => {
|
|
try {
|
|
const convo = await getConvo(user, conversationId);
|
|
/* ChatGPT Browser was triggering error here due to convo being saved later */
|
|
if (convo && !convo.title) {
|
|
return null;
|
|
} else {
|
|
// TypeError: Cannot read properties of null (reading 'title')
|
|
return convo?.title || 'New Chat';
|
|
}
|
|
} catch (error) {
|
|
logger.error('[getConvoTitle] Error getting conversation title', error);
|
|
return { message: 'Error getting conversation title' };
|
|
}
|
|
},
|
|
/**
|
|
* Asynchronously deletes conversations and associated messages for a given user and filter.
|
|
*
|
|
* @async
|
|
* @function
|
|
* @param {string|ObjectId} user - The user's ID.
|
|
* @param {Object} filter - Additional filter criteria for the conversations to be deleted.
|
|
* @returns {Promise<{ n: number, ok: number, deletedCount: number, messages: { n: number, ok: number, deletedCount: number } }>}
|
|
* An object containing the count of deleted conversations and associated messages.
|
|
* @throws {Error} Throws an error if there's an issue with the database operations.
|
|
*
|
|
* @example
|
|
* const user = 'someUserId';
|
|
* const filter = { someField: 'someValue' };
|
|
* const result = await deleteConvos(user, filter);
|
|
* logger.error(result); // { n: 5, ok: 1, deletedCount: 5, messages: { n: 10, ok: 1, deletedCount: 10 } }
|
|
*/
|
|
deleteConvos: async (user, filter) => {
|
|
try {
|
|
const userFilter = { ...filter, user };
|
|
const conversations = await Conversation.find(userFilter).select('conversationId');
|
|
const conversationIds = conversations.map((c) => c.conversationId);
|
|
|
|
if (!conversationIds.length) {
|
|
throw new Error('Conversation not found or already deleted.');
|
|
}
|
|
|
|
const deleteConvoResult = await Conversation.deleteMany(userFilter);
|
|
|
|
const deleteMessagesResult = await deleteMessages({
|
|
conversationId: { $in: conversationIds },
|
|
});
|
|
|
|
return { ...deleteConvoResult, messages: deleteMessagesResult };
|
|
} catch (error) {
|
|
logger.error('[deleteConvos] Error deleting conversations and messages', error);
|
|
throw error;
|
|
}
|
|
},
|
|
};
|