mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-04-02 22:07:19 +02:00
🧵 feat: ALS Context Middleware, Tenant Threading, and Config Cache Invalidation (#12407)
* feat: add tenant context middleware for ALS-based isolation Introduces tenantContextMiddleware that propagates req.user.tenantId into AsyncLocalStorage, activating the Mongoose applyTenantIsolation plugin for all downstream DB queries within a request. - Strict mode (TENANT_ISOLATION_STRICT=true) returns 403 if no tenantId - Non-strict mode passes through for backward compatibility - No-op for unauthenticated requests - Includes 6 unit tests covering all paths * feat: register tenant middleware and wrap startup/auth in runAsSystem() - Register tenantContextMiddleware in Express app after capability middleware - Wrap server startup initialization in runAsSystem() for strict mode compat - Wrap auth strategy getAppConfig() calls in runAsSystem() since they run before user context is established (LDAP, SAML, OpenID, social login, AuthService) * feat: thread tenantId through all getAppConfig callers Pass tenantId from req.user to getAppConfig() across all callers that have request context, ensuring correct per-tenant cache key resolution. Also fixes getBaseConfig admin endpoint to scope to requesting admin's tenant instead of returning the unscoped base config. Files updated: - Controllers: UserController, PluginController - Middleware: checkDomainAllowed, balance - Routes: config - Services: loadConfigModels, loadDefaultModels, getEndpointsConfig, MCP - Audio services: TTSService, STTService, getVoices, getCustomConfigSpeech - Admin: getBaseConfig endpoint * feat: add config cache invalidation on admin mutations - Add clearOverrideCache(tenantId?) to flush per-principal override caches by enumerating Keyv store keys matching _OVERRIDE_: prefix - Add invalidateConfigCaches() helper that clears base config, override caches, tool caches, and endpoint config cache in one call - Wire invalidation into all 5 admin config mutation handlers (upsert, patch, delete field, delete overrides, toggle active) - Add strict mode warning when __default__ tenant fallback is used - Add 3 new tests for clearOverrideCache (all/scoped/base-preserving) * chore: update getUserPrincipals comment to reflect ALS-based tenant filtering The TODO(#12091) about missing tenantId filtering is resolved by the tenant context middleware + applyTenantIsolation Mongoose plugin. Group queries are now automatically scoped by tenantId via ALS. * fix: replace runAsSystem with baseOnly for pre-tenant code paths App configs are tenant-owned — runAsSystem() would bypass tenant isolation and return cross-tenant DB overrides. Instead, add baseOnly option to getAppConfig() that returns YAML-derived config only, with zero DB queries. All startup code, auth strategies, and MCP initialization now use getAppConfig({ baseOnly: true }) to get the YAML config without touching the Config collection. * fix: address PR review findings — middleware ordering, types, cache safety - Chain tenantContextMiddleware inside requireJwtAuth after passport auth instead of global app.use() where req.user is always undefined (Finding 1) - Remove global tenantContextMiddleware registration from index.js - Update BalanceMiddlewareOptions to include tenantId, remove redundant cast (Finding 4) - Add warning log when clearOverrideCache cannot enumerate keys on Redis (Finding 3) - Use startsWith instead of includes for cache key filtering (Finding 12) - Use generator loop instead of Array.from for key enumeration (Finding 3) - Selective barrel export — exclude _resetTenantMiddlewareStrictCache (Finding 5) - Move isMainThread check to module level, remove per-request check (Finding 9) - Move mid-file require to top of app.js (Finding 8) - Parallelize invalidateConfigCaches with Promise.all (Finding 10) - Remove clearOverrideCache from public app.js exports (internal only) - Strengthen getUserPrincipals comment re: ALS dependency (Finding 2) * fix: restore runAsSystem for startup DB ops, consolidate require, clarify baseOnly - Restore runAsSystem() around performStartupChecks, updateInterfacePermissions, initializeMCPs, and initializeOAuthReconnectManager — these make Mongoose queries that need system context in strict tenant mode (NEW-3) - Consolidate duplicate require('@librechat/api') in requireJwtAuth.js (NEW-1) - Document that baseOnly ignores role/userId/tenantId in JSDoc (NEW-2) * test: add requireJwtAuth tenant chaining + invalidateConfigCaches tests - requireJwtAuth: 5 tests verifying ALS tenant context is set after passport auth, isolated between concurrent requests, and not set when user has no tenantId (Finding 6) - invalidateConfigCaches: 4 tests verifying all four caches are cleared, tenantId is threaded through, partial failure is handled gracefully, and operations run in parallel via Promise.all (Finding 11) * fix: address Copilot review — passport errors, namespaced cache keys, /base scoping - Forward passport errors in requireJwtAuth before entering tenant middleware — prevents silent auth failures from reaching handlers (P1) - Account for Keyv namespace prefix in clearOverrideCache — stored keys are namespaced as "APP_CONFIG:_OVERRIDE_:..." not "_OVERRIDE_:...", so override caches were never actually matched/cleared (P2) - Remove role from getBaseConfig — /base should return tenant-scoped base config, not role-merged config that drifts per admin role (P2) - Return tenantStorage.run() for cleaner async semantics - Update mock cache in service.spec.ts to simulate Keyv namespacing * fix: address second review — cache safety, code quality, test reliability - Decouple cache invalidation from mutation response: fire-and-forget with logging so DB mutation success is not masked by cache failures - Extract clearEndpointConfigCache helper from inline IIFE - Move isMainThread check to lazy once-per-process guard (no import side effect) - Memoize process.env read in overrideCacheKey to avoid per-request env lookups and log flooding in strict mode - Remove flaky timer-based parallelism assertion, use structural check - Merge orphaned double JSDoc block on getUserPrincipals - Fix stale [getAppConfig] log prefix → [ensureBaseConfig] - Fix import order in tenant.spec.ts (package types before local values) - Replace "Finding 1" reference with self-contained description - Use real tenantStorage primitives in requireJwtAuth spec mock * fix: move JSDoc to correct function after clearEndpointConfigCache extraction * refactor: remove Redis SCAN from clearOverrideCache, rely on TTL expiry Redis SCAN causes 60s+ stalls under concurrent load (see #12410). APP_CONFIG defaults to FORCED_IN_MEMORY_CACHE_NAMESPACES, so the in-memory store.keys() path handles the standard case. When APP_CONFIG is Redis-backed, overrides expire naturally via overrideCacheTtl (60s default) — an acceptable window for admin config mutations. * fix: remove return from tenantStorage.run to satisfy void middleware signature * fix: address second review — cache safety, code quality, test reliability - Switch invalidateConfigCaches from Promise.all to Promise.allSettled so partial failures are logged individually instead of producing one undifferentiated error (Finding 3) - Gate overrideCacheKey strict-mode warning behind a once-per-process flag to prevent log flooding under load (Finding 4) - Add test for passport error forwarding in requireJwtAuth — the if (err) { return next(err) } branch now has coverage (Finding 5) - Add test for real partial failure in invalidateConfigCaches where clearAppConfigCache rejects (not just the swallowed endpoint error) * chore: reorder imports in index.js and app.js for consistency - Moved logger and runAsSystem imports to maintain a consistent import order across files. - Improved code readability by ensuring related imports are grouped together.
This commit is contained in:
parent
083042e56c
commit
9f6d8c6e93
32 changed files with 768 additions and 63 deletions
|
|
@ -15,7 +15,7 @@ const getAvailablePluginsController = async (req, res) => {
|
|||
return;
|
||||
}
|
||||
|
||||
const appConfig = await getAppConfig({ role: req.user?.role });
|
||||
const appConfig = await getAppConfig({ role: req.user?.role, tenantId: req.user?.tenantId });
|
||||
/** @type {{ filteredTools: string[], includedTools: string[] }} */
|
||||
const { filteredTools = [], includedTools = [] } = appConfig;
|
||||
/** @type {import('@librechat/api').LCManifestTool[]} */
|
||||
|
|
@ -66,7 +66,8 @@ const getAvailableTools = async (req, res) => {
|
|||
const cache = getLogStores(CacheKeys.TOOL_CACHE);
|
||||
const cachedToolsArray = await cache.get(CacheKeys.TOOLS);
|
||||
|
||||
const appConfig = req.config ?? (await getAppConfig({ role: req.user?.role }));
|
||||
const appConfig =
|
||||
req.config ?? (await getAppConfig({ role: req.user?.role, tenantId: req.user?.tenantId }));
|
||||
|
||||
// Return early if we have cached tools
|
||||
if (cachedToolsArray != null) {
|
||||
|
|
|
|||
|
|
@ -26,7 +26,7 @@ const { getLogStores } = require('~/cache');
|
|||
const db = require('~/models');
|
||||
|
||||
const getUserController = async (req, res) => {
|
||||
const appConfig = await getAppConfig({ role: req.user?.role });
|
||||
const appConfig = await getAppConfig({ role: req.user?.role, tenantId: req.user?.tenantId });
|
||||
/** @type {IUser} */
|
||||
const userData = req.user.toObject != null ? req.user.toObject() : { ...req.user };
|
||||
/**
|
||||
|
|
@ -165,7 +165,7 @@ const deleteUserMcpServers = async (userId) => {
|
|||
};
|
||||
|
||||
const updateUserPluginsController = async (req, res) => {
|
||||
const appConfig = await getAppConfig({ role: req.user?.role });
|
||||
const appConfig = await getAppConfig({ role: req.user?.role, tenantId: req.user?.tenantId });
|
||||
const { user } = req;
|
||||
const { pluginKey, action, auth, isEntityTool } = req.body;
|
||||
try {
|
||||
|
|
|
|||
|
|
@ -8,8 +8,8 @@ const express = require('express');
|
|||
const passport = require('passport');
|
||||
const compression = require('compression');
|
||||
const cookieParser = require('cookie-parser');
|
||||
const { logger } = require('@librechat/data-schemas');
|
||||
const mongoSanitize = require('express-mongo-sanitize');
|
||||
const { logger, runAsSystem } = require('@librechat/data-schemas');
|
||||
const {
|
||||
isEnabled,
|
||||
apiNotFound,
|
||||
|
|
@ -60,10 +60,12 @@ const startServer = async () => {
|
|||
app.set('trust proxy', trusted_proxy);
|
||||
|
||||
await seedDatabase();
|
||||
const appConfig = await getAppConfig();
|
||||
const appConfig = await getAppConfig({ baseOnly: true });
|
||||
initializeFileStorage(appConfig);
|
||||
await performStartupChecks(appConfig);
|
||||
await updateInterfacePermissions({ appConfig, getRoleByName, updateAccessPermissions });
|
||||
await runAsSystem(async () => {
|
||||
await performStartupChecks(appConfig);
|
||||
await updateInterfacePermissions({ appConfig, getRoleByName, updateAccessPermissions });
|
||||
});
|
||||
|
||||
const indexPath = path.join(appConfig.paths.dist, 'index.html');
|
||||
let indexHTML = fs.readFileSync(indexPath, 'utf8');
|
||||
|
|
@ -205,8 +207,10 @@ const startServer = async () => {
|
|||
logger.info(`Server listening at http://${host == '0.0.0.0' ? 'localhost' : host}:${port}`);
|
||||
}
|
||||
|
||||
await initializeMCPs();
|
||||
await initializeOAuthReconnectManager();
|
||||
await runAsSystem(async () => {
|
||||
await initializeMCPs();
|
||||
await initializeOAuthReconnectManager();
|
||||
});
|
||||
await checkMigrations();
|
||||
|
||||
// Configure stream services (auto-detects Redis from USE_REDIS env var)
|
||||
|
|
|
|||
116
api/server/middleware/__tests__/requireJwtAuth.spec.js
Normal file
116
api/server/middleware/__tests__/requireJwtAuth.spec.js
Normal file
|
|
@ -0,0 +1,116 @@
|
|||
/**
|
||||
* Integration test: verifies that requireJwtAuth chains tenantContextMiddleware
|
||||
* after successful passport authentication, so ALS tenant context is set for
|
||||
* all downstream middleware and route handlers.
|
||||
*
|
||||
* requireJwtAuth must chain tenantContextMiddleware after passport populates
|
||||
* req.user (not at global app.use() scope where req.user is undefined).
|
||||
* If the chaining is removed, these tests fail.
|
||||
*/
|
||||
|
||||
const { getTenantId } = require('@librechat/data-schemas');
|
||||
|
||||
// ── Mocks ──────────────────────────────────────────────────────────────
|
||||
|
||||
let mockPassportError = null;
|
||||
|
||||
jest.mock('passport', () => ({
|
||||
authenticate: jest.fn(() => {
|
||||
return (req, _res, done) => {
|
||||
if (mockPassportError) {
|
||||
return done(mockPassportError);
|
||||
}
|
||||
if (req._mockUser) {
|
||||
req.user = req._mockUser;
|
||||
}
|
||||
done();
|
||||
};
|
||||
}),
|
||||
}));
|
||||
|
||||
// Mock @librechat/api — the real tenantContextMiddleware is TS and cannot be
|
||||
// required directly from CJS tests. This thin wrapper mirrors the real logic
|
||||
// (read req.user.tenantId, call tenantStorage.run) using the same data-schemas
|
||||
// primitives. The real implementation is covered by packages/api tenant.spec.ts.
|
||||
jest.mock('@librechat/api', () => {
|
||||
const { tenantStorage } = require('@librechat/data-schemas');
|
||||
return {
|
||||
isEnabled: jest.fn(() => false),
|
||||
tenantContextMiddleware: (req, res, next) => {
|
||||
const tenantId = req.user?.tenantId;
|
||||
if (!tenantId) {
|
||||
return next();
|
||||
}
|
||||
return tenantStorage.run({ tenantId }, async () => next());
|
||||
},
|
||||
};
|
||||
});
|
||||
|
||||
// ── Helpers ─────────────────────────────────────────────────────────────
|
||||
|
||||
const requireJwtAuth = require('../requireJwtAuth');
|
||||
|
||||
function mockReq(user) {
|
||||
return { headers: {}, _mockUser: user };
|
||||
}
|
||||
|
||||
function mockRes() {
|
||||
return { status: jest.fn().mockReturnThis(), json: jest.fn().mockReturnThis() };
|
||||
}
|
||||
|
||||
/** Runs requireJwtAuth and returns the tenantId observed inside next(). */
|
||||
function runAuth(user) {
|
||||
return new Promise((resolve) => {
|
||||
const req = mockReq(user);
|
||||
const res = mockRes();
|
||||
requireJwtAuth(req, res, () => {
|
||||
resolve(getTenantId());
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// ── Tests ──────────────────────────────────────────────────────────────
|
||||
|
||||
describe('requireJwtAuth tenant context chaining', () => {
|
||||
afterEach(() => {
|
||||
mockPassportError = null;
|
||||
});
|
||||
|
||||
it('forwards passport errors to next() without entering tenant middleware', async () => {
|
||||
mockPassportError = new Error('JWT signature invalid');
|
||||
const req = mockReq(undefined);
|
||||
const res = mockRes();
|
||||
const err = await new Promise((resolve) => {
|
||||
requireJwtAuth(req, res, (e) => resolve(e));
|
||||
});
|
||||
expect(err).toBeInstanceOf(Error);
|
||||
expect(err.message).toBe('JWT signature invalid');
|
||||
expect(getTenantId()).toBeUndefined();
|
||||
});
|
||||
|
||||
it('sets ALS tenant context after passport auth succeeds', async () => {
|
||||
const tenantId = await runAuth({ tenantId: 'tenant-abc', role: 'user' });
|
||||
expect(tenantId).toBe('tenant-abc');
|
||||
});
|
||||
|
||||
it('ALS tenant context is NOT set when user has no tenantId', async () => {
|
||||
const tenantId = await runAuth({ role: 'user' });
|
||||
expect(tenantId).toBeUndefined();
|
||||
});
|
||||
|
||||
it('ALS tenant context is NOT set when user is undefined', async () => {
|
||||
const tenantId = await runAuth(undefined);
|
||||
expect(tenantId).toBeUndefined();
|
||||
});
|
||||
|
||||
it('concurrent requests get isolated tenant contexts', async () => {
|
||||
const results = await Promise.all(
|
||||
['tenant-1', 'tenant-2', 'tenant-3'].map((tid) => runAuth({ tenantId: tid, role: 'user' })),
|
||||
);
|
||||
expect(results).toEqual(['tenant-1', 'tenant-2', 'tenant-3']);
|
||||
});
|
||||
|
||||
it('ALS context is not set at top-level scope (outside any request)', () => {
|
||||
expect(getTenantId()).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
|
@ -18,6 +18,7 @@ const checkDomainAllowed = async (req, res, next) => {
|
|||
const email = req?.user?.email;
|
||||
const appConfig = await getAppConfig({
|
||||
role: req?.user?.role,
|
||||
tenantId: req?.user?.tenantId,
|
||||
});
|
||||
|
||||
if (email && !isEmailDomainAllowed(email, appConfig?.registration?.allowedDomains)) {
|
||||
|
|
|
|||
|
|
@ -1,20 +1,29 @@
|
|||
const cookies = require('cookie');
|
||||
const passport = require('passport');
|
||||
const { isEnabled } = require('@librechat/api');
|
||||
const { isEnabled, tenantContextMiddleware } = require('@librechat/api');
|
||||
|
||||
/**
|
||||
* Custom Middleware to handle JWT authentication, with support for OpenID token reuse
|
||||
* Switches between JWT and OpenID authentication based on cookies and environment settings
|
||||
* Custom Middleware to handle JWT authentication, with support for OpenID token reuse.
|
||||
* Switches between JWT and OpenID authentication based on cookies and environment settings.
|
||||
*
|
||||
* After successful authentication (req.user populated), automatically chains into
|
||||
* `tenantContextMiddleware` to propagate `req.user.tenantId` into AsyncLocalStorage
|
||||
* for downstream Mongoose tenant isolation.
|
||||
*/
|
||||
const requireJwtAuth = (req, res, next) => {
|
||||
const cookieHeader = req.headers.cookie;
|
||||
const tokenProvider = cookieHeader ? cookies.parse(cookieHeader).token_provider : null;
|
||||
|
||||
if (tokenProvider === 'openid' && isEnabled(process.env.OPENID_REUSE_TOKENS)) {
|
||||
return passport.authenticate('openidJwt', { session: false })(req, res, next);
|
||||
}
|
||||
const strategy =
|
||||
tokenProvider === 'openid' && isEnabled(process.env.OPENID_REUSE_TOKENS) ? 'openidJwt' : 'jwt';
|
||||
|
||||
return passport.authenticate('jwt', { session: false })(req, res, next);
|
||||
passport.authenticate(strategy, { session: false })(req, res, (err) => {
|
||||
if (err) {
|
||||
return next(err);
|
||||
}
|
||||
// req.user is now populated by passport — set up tenant ALS context
|
||||
tenantContextMiddleware(req, res, next);
|
||||
});
|
||||
};
|
||||
|
||||
module.exports = requireJwtAuth;
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ const {
|
|||
hasConfigCapability,
|
||||
requireCapability,
|
||||
} = require('~/server/middleware/roles/capabilities');
|
||||
const { getAppConfig } = require('~/server/services/Config');
|
||||
const { getAppConfig, invalidateConfigCaches } = require('~/server/services/Config');
|
||||
const { requireJwtAuth } = require('~/server/middleware');
|
||||
const db = require('~/models');
|
||||
|
||||
|
|
@ -23,6 +23,7 @@ const handlers = createAdminConfigHandlers({
|
|||
toggleConfigActive: db.toggleConfigActive,
|
||||
hasConfigCapability,
|
||||
getAppConfig,
|
||||
invalidateConfigCaches,
|
||||
});
|
||||
|
||||
router.use(requireJwtAuth, requireAdminAccess);
|
||||
|
|
|
|||
|
|
@ -37,7 +37,7 @@ router.get('/', async function (req, res) {
|
|||
const ldap = getLdapConfig();
|
||||
|
||||
try {
|
||||
const appConfig = await getAppConfig({ role: req.user?.role });
|
||||
const appConfig = await getAppConfig({ role: req.user?.role, tenantId: req.user?.tenantId });
|
||||
|
||||
const isOpenIdEnabled =
|
||||
!!process.env.OPENID_CLIENT_ID &&
|
||||
|
|
|
|||
|
|
@ -189,7 +189,7 @@ const registerUser = async (user, additionalData = {}) => {
|
|||
|
||||
let newUserId;
|
||||
try {
|
||||
const appConfig = await getAppConfig();
|
||||
const appConfig = await getAppConfig({ baseOnly: true });
|
||||
if (!isEmailDomainAllowed(email, appConfig?.registration?.allowedDomains)) {
|
||||
const errorMessage =
|
||||
'The email address provided cannot be used. Please use a different email address.';
|
||||
|
|
@ -260,7 +260,7 @@ const registerUser = async (user, additionalData = {}) => {
|
|||
*/
|
||||
const requestPasswordReset = async (req) => {
|
||||
const { email } = req.body;
|
||||
const appConfig = await getAppConfig();
|
||||
const appConfig = await getAppConfig({ baseOnly: true });
|
||||
if (!isEmailDomainAllowed(email, appConfig?.registration?.allowedDomains)) {
|
||||
const error = new Error(ErrorTypes.AUTH_FAILED);
|
||||
error.code = ErrorTypes.AUTH_FAILED;
|
||||
|
|
|
|||
|
|
@ -0,0 +1,137 @@
|
|||
// ── Mocks ──────────────────────────────────────────────────────────────
|
||||
|
||||
const mockConfigStoreDelete = jest.fn().mockResolvedValue(true);
|
||||
const mockClearAppConfigCache = jest.fn().mockResolvedValue(undefined);
|
||||
const mockClearOverrideCache = jest.fn().mockResolvedValue(undefined);
|
||||
|
||||
jest.mock('~/cache/getLogStores', () => {
|
||||
return jest.fn(() => ({
|
||||
delete: mockConfigStoreDelete,
|
||||
}));
|
||||
});
|
||||
|
||||
jest.mock('~/server/services/start/tools', () => ({
|
||||
loadAndFormatTools: jest.fn(() => ({})),
|
||||
}));
|
||||
|
||||
jest.mock('../loadCustomConfig', () => jest.fn().mockResolvedValue({}));
|
||||
|
||||
jest.mock('@librechat/data-schemas', () => {
|
||||
const actual = jest.requireActual('@librechat/data-schemas');
|
||||
return { ...actual, AppService: jest.fn(() => ({ availableTools: {} })) };
|
||||
});
|
||||
|
||||
jest.mock('~/models', () => ({
|
||||
getApplicableConfigs: jest.fn().mockResolvedValue([]),
|
||||
getUserPrincipals: jest.fn().mockResolvedValue([]),
|
||||
}));
|
||||
|
||||
const mockInvalidateCachedTools = jest.fn().mockResolvedValue(undefined);
|
||||
jest.mock('../getCachedTools', () => ({
|
||||
setCachedTools: jest.fn().mockResolvedValue(undefined),
|
||||
invalidateCachedTools: mockInvalidateCachedTools,
|
||||
}));
|
||||
|
||||
jest.mock('@librechat/api', () => ({
|
||||
createAppConfigService: jest.fn(() => ({
|
||||
getAppConfig: jest.fn().mockResolvedValue({ availableTools: {} }),
|
||||
clearAppConfigCache: mockClearAppConfigCache,
|
||||
clearOverrideCache: mockClearOverrideCache,
|
||||
})),
|
||||
}));
|
||||
|
||||
// ── Tests ──────────────────────────────────────────────────────────────
|
||||
|
||||
const { CacheKeys } = require('librechat-data-provider');
|
||||
const { invalidateConfigCaches } = require('../app');
|
||||
|
||||
describe('invalidateConfigCaches', () => {
|
||||
beforeEach(() => {
|
||||
jest.clearAllMocks();
|
||||
});
|
||||
|
||||
it('clears all four caches', async () => {
|
||||
await invalidateConfigCaches();
|
||||
|
||||
expect(mockClearAppConfigCache).toHaveBeenCalledTimes(1);
|
||||
expect(mockClearOverrideCache).toHaveBeenCalledTimes(1);
|
||||
expect(mockInvalidateCachedTools).toHaveBeenCalledWith({ invalidateGlobal: true });
|
||||
expect(mockConfigStoreDelete).toHaveBeenCalledWith(CacheKeys.ENDPOINT_CONFIG);
|
||||
});
|
||||
|
||||
it('passes tenantId through to clearOverrideCache', async () => {
|
||||
await invalidateConfigCaches('tenant-a');
|
||||
|
||||
expect(mockClearOverrideCache).toHaveBeenCalledWith('tenant-a');
|
||||
expect(mockClearAppConfigCache).toHaveBeenCalledTimes(1);
|
||||
expect(mockInvalidateCachedTools).toHaveBeenCalledWith({ invalidateGlobal: true });
|
||||
});
|
||||
|
||||
it('does not throw when CONFIG_STORE.delete fails', async () => {
|
||||
mockConfigStoreDelete.mockRejectedValueOnce(new Error('store not found'));
|
||||
|
||||
await expect(invalidateConfigCaches()).resolves.not.toThrow();
|
||||
|
||||
// Other caches should still have been invalidated
|
||||
expect(mockClearAppConfigCache).toHaveBeenCalledTimes(1);
|
||||
expect(mockClearOverrideCache).toHaveBeenCalledTimes(1);
|
||||
expect(mockInvalidateCachedTools).toHaveBeenCalledWith({ invalidateGlobal: true });
|
||||
});
|
||||
|
||||
it('all operations run in parallel (not sequentially)', async () => {
|
||||
const order = [];
|
||||
|
||||
mockClearAppConfigCache.mockImplementation(
|
||||
() =>
|
||||
new Promise((r) =>
|
||||
setTimeout(() => {
|
||||
order.push('base');
|
||||
r();
|
||||
}, 10),
|
||||
),
|
||||
);
|
||||
mockClearOverrideCache.mockImplementation(
|
||||
() =>
|
||||
new Promise((r) =>
|
||||
setTimeout(() => {
|
||||
order.push('override');
|
||||
r();
|
||||
}, 10),
|
||||
),
|
||||
);
|
||||
mockInvalidateCachedTools.mockImplementation(
|
||||
() =>
|
||||
new Promise((r) =>
|
||||
setTimeout(() => {
|
||||
order.push('tools');
|
||||
r();
|
||||
}, 10),
|
||||
),
|
||||
);
|
||||
mockConfigStoreDelete.mockImplementation(
|
||||
() =>
|
||||
new Promise((r) =>
|
||||
setTimeout(() => {
|
||||
order.push('endpoint');
|
||||
r();
|
||||
}, 10),
|
||||
),
|
||||
);
|
||||
|
||||
await invalidateConfigCaches();
|
||||
|
||||
// All four should have been called (parallel execution via Promise.allSettled)
|
||||
expect(order).toHaveLength(4);
|
||||
expect(new Set(order)).toEqual(new Set(['base', 'override', 'tools', 'endpoint']));
|
||||
});
|
||||
|
||||
it('resolves even when clearAppConfigCache throws (partial failure)', async () => {
|
||||
mockClearAppConfigCache.mockRejectedValueOnce(new Error('cache connection lost'));
|
||||
|
||||
await expect(invalidateConfigCaches()).resolves.not.toThrow();
|
||||
|
||||
// Other caches should still have been invalidated despite the failure
|
||||
expect(mockClearOverrideCache).toHaveBeenCalledTimes(1);
|
||||
expect(mockInvalidateCachedTools).toHaveBeenCalledWith({ invalidateGlobal: true });
|
||||
});
|
||||
});
|
||||
|
|
@ -1,9 +1,9 @@
|
|||
const { CacheKeys } = require('librechat-data-provider');
|
||||
const { AppService } = require('@librechat/data-schemas');
|
||||
const { createAppConfigService } = require('@librechat/api');
|
||||
const { AppService, logger } = require('@librechat/data-schemas');
|
||||
const { setCachedTools, invalidateCachedTools } = require('./getCachedTools');
|
||||
const { loadAndFormatTools } = require('~/server/services/start/tools');
|
||||
const loadCustomConfig = require('./loadCustomConfig');
|
||||
const { setCachedTools } = require('./getCachedTools');
|
||||
const getLogStores = require('~/cache/getLogStores');
|
||||
const paths = require('~/config/paths');
|
||||
const db = require('~/models');
|
||||
|
|
@ -20,7 +20,7 @@ const loadBaseConfig = async () => {
|
|||
return AppService({ config, paths, systemTools });
|
||||
};
|
||||
|
||||
const { getAppConfig, clearAppConfigCache } = createAppConfigService({
|
||||
const { getAppConfig, clearAppConfigCache, clearOverrideCache } = createAppConfigService({
|
||||
loadBaseConfig,
|
||||
setCachedTools,
|
||||
getCache: getLogStores,
|
||||
|
|
@ -29,7 +29,44 @@ const { getAppConfig, clearAppConfigCache } = createAppConfigService({
|
|||
getUserPrincipals: db.getUserPrincipals,
|
||||
});
|
||||
|
||||
/** Deletes the ENDPOINT_CONFIG entry from CONFIG_STORE. Failures are non-critical and swallowed. */
|
||||
async function clearEndpointConfigCache() {
|
||||
try {
|
||||
const configStore = getLogStores(CacheKeys.CONFIG_STORE);
|
||||
await configStore.delete(CacheKeys.ENDPOINT_CONFIG);
|
||||
} catch {
|
||||
// CONFIG_STORE or ENDPOINT_CONFIG may not exist — not critical
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Invalidate all config-related caches after an admin config mutation.
|
||||
* Clears the base config, per-principal override caches, tool caches,
|
||||
* and the endpoints config cache.
|
||||
* @param {string} [tenantId] - Optional tenant ID to scope override cache clearing.
|
||||
*/
|
||||
async function invalidateConfigCaches(tenantId) {
|
||||
const results = await Promise.allSettled([
|
||||
clearAppConfigCache(),
|
||||
clearOverrideCache(tenantId),
|
||||
invalidateCachedTools({ invalidateGlobal: true }),
|
||||
clearEndpointConfigCache(),
|
||||
]);
|
||||
const labels = [
|
||||
'clearAppConfigCache',
|
||||
'clearOverrideCache',
|
||||
'invalidateCachedTools',
|
||||
'clearEndpointConfigCache',
|
||||
];
|
||||
for (let i = 0; i < results.length; i++) {
|
||||
if (results[i].status === 'rejected') {
|
||||
logger.error(`[invalidateConfigCaches] ${labels[i]} failed:`, results[i].reason);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
getAppConfig,
|
||||
clearAppConfigCache,
|
||||
invalidateConfigCaches,
|
||||
};
|
||||
|
|
|
|||
|
|
@ -26,7 +26,8 @@ async function getEndpointsConfig(req) {
|
|||
}
|
||||
}
|
||||
|
||||
const appConfig = req.config ?? (await getAppConfig({ role: req.user?.role }));
|
||||
const appConfig =
|
||||
req.config ?? (await getAppConfig({ role: req.user?.role, tenantId: req.user?.tenantId }));
|
||||
const defaultEndpointsConfig = await loadDefaultEndpointsConfig(appConfig);
|
||||
const customEndpointsConfig = loadCustomEndpointsConfig(appConfig?.endpoints?.custom);
|
||||
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ const { getAppConfig } = require('./app');
|
|||
* @param {ServerRequest} req - The Express request object.
|
||||
*/
|
||||
async function loadConfigModels(req) {
|
||||
const appConfig = await getAppConfig({ role: req.user?.role });
|
||||
const appConfig = await getAppConfig({ role: req.user?.role, tenantId: req.user?.tenantId });
|
||||
if (!appConfig) {
|
||||
return {};
|
||||
}
|
||||
|
|
|
|||
|
|
@ -16,7 +16,8 @@ const { getAppConfig } = require('./app');
|
|||
*/
|
||||
async function loadDefaultModels(req) {
|
||||
try {
|
||||
const appConfig = req.config ?? (await getAppConfig({ role: req.user?.role }));
|
||||
const appConfig =
|
||||
req.config ?? (await getAppConfig({ role: req.user?.role, tenantId: req.user?.tenantId }));
|
||||
const vertexConfig = appConfig?.endpoints?.[EModelEndpoint.anthropic]?.vertexConfig;
|
||||
|
||||
const [openAI, anthropic, azureOpenAI, assistants, azureAssistants, google, bedrock] =
|
||||
|
|
|
|||
|
|
@ -142,6 +142,7 @@ class STTService {
|
|||
req.config ??
|
||||
(await getAppConfig({
|
||||
role: req?.user?.role,
|
||||
tenantId: req?.user?.tenantId,
|
||||
}));
|
||||
const sttSchema = appConfig?.speech?.stt;
|
||||
if (!sttSchema) {
|
||||
|
|
|
|||
|
|
@ -297,6 +297,7 @@ class TTSService {
|
|||
req.config ??
|
||||
(await getAppConfig({
|
||||
role: req.user?.role,
|
||||
tenantId: req.user?.tenantId,
|
||||
}));
|
||||
try {
|
||||
res.setHeader('Content-Type', 'audio/mpeg');
|
||||
|
|
@ -365,6 +366,7 @@ class TTSService {
|
|||
req.config ??
|
||||
(await getAppConfig({
|
||||
role: req.user?.role,
|
||||
tenantId: req.user?.tenantId,
|
||||
}));
|
||||
const provider = this.getProvider(appConfig);
|
||||
const ttsSchema = appConfig?.speech?.tts?.[provider];
|
||||
|
|
|
|||
|
|
@ -17,6 +17,7 @@ async function getCustomConfigSpeech(req, res) {
|
|||
try {
|
||||
const appConfig = await getAppConfig({
|
||||
role: req.user?.role,
|
||||
tenantId: req.user?.tenantId,
|
||||
});
|
||||
|
||||
if (!appConfig) {
|
||||
|
|
|
|||
|
|
@ -18,6 +18,7 @@ async function getVoices(req, res) {
|
|||
req.config ??
|
||||
(await getAppConfig({
|
||||
role: req.user?.role,
|
||||
tenantId: req.user?.tenantId,
|
||||
}));
|
||||
|
||||
const ttsSchema = appConfig?.speech?.tts;
|
||||
|
|
|
|||
|
|
@ -367,7 +367,7 @@ async function createMCPTools({
|
|||
const serverConfig =
|
||||
config ?? (await getMCPServersRegistry().getServerConfig(serverName, user?.id));
|
||||
if (serverConfig?.url) {
|
||||
const appConfig = await getAppConfig({ role: user?.role });
|
||||
const appConfig = await getAppConfig({ role: user?.role, tenantId: user?.tenantId });
|
||||
const allowedDomains = appConfig?.mcpSettings?.allowedDomains;
|
||||
const isDomainAllowed = await isMCPDomainAllowed(serverConfig, allowedDomains);
|
||||
if (!isDomainAllowed) {
|
||||
|
|
@ -449,7 +449,7 @@ async function createMCPTool({
|
|||
const serverConfig =
|
||||
config ?? (await getMCPServersRegistry().getServerConfig(serverName, user?.id));
|
||||
if (serverConfig?.url) {
|
||||
const appConfig = await getAppConfig({ role: user?.role });
|
||||
const appConfig = await getAppConfig({ role: user?.role, tenantId: user?.tenantId });
|
||||
const allowedDomains = appConfig?.mcpSettings?.allowedDomains;
|
||||
const isDomainAllowed = await isMCPDomainAllowed(serverConfig, allowedDomains);
|
||||
if (!isDomainAllowed) {
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ const { createMCPServersRegistry, createMCPManager } = require('~/config');
|
|||
* Initialize MCP servers
|
||||
*/
|
||||
async function initializeMCPs() {
|
||||
const appConfig = await getAppConfig();
|
||||
const appConfig = await getAppConfig({ baseOnly: true });
|
||||
const mcpServers = appConfig.mcpConfig;
|
||||
|
||||
try {
|
||||
|
|
|
|||
|
|
@ -122,7 +122,7 @@ const ldapLogin = new LdapStrategy(ldapOptions, async (userinfo, done) => {
|
|||
);
|
||||
}
|
||||
|
||||
const appConfig = await getAppConfig();
|
||||
const appConfig = await getAppConfig({ baseOnly: true });
|
||||
if (!isEmailDomainAllowed(mail, appConfig?.registration?.allowedDomains)) {
|
||||
logger.error(
|
||||
`[LDAP Strategy] Authentication blocked - email domain not allowed [Email: ${mail}]`,
|
||||
|
|
|
|||
|
|
@ -468,7 +468,7 @@ async function processOpenIDAuth(tokenset, existingUsersOnly = false) {
|
|||
Object.assign(userinfo, providerUserinfo);
|
||||
}
|
||||
|
||||
const appConfig = await getAppConfig();
|
||||
const appConfig = await getAppConfig({ baseOnly: true });
|
||||
const email = getOpenIdEmail(userinfo);
|
||||
if (!isEmailDomainAllowed(email, appConfig?.registration?.allowedDomains)) {
|
||||
logger.error(
|
||||
|
|
|
|||
|
|
@ -193,7 +193,7 @@ async function setupSaml() {
|
|||
logger.debug('[samlStrategy] SAML profile:', profile);
|
||||
|
||||
const userEmail = getEmail(profile) || '';
|
||||
const appConfig = await getAppConfig();
|
||||
const appConfig = await getAppConfig({ baseOnly: true });
|
||||
|
||||
if (!isEmailDomainAllowed(userEmail, appConfig?.registration?.allowedDomains)) {
|
||||
logger.error(
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@ const socialLogin =
|
|||
profile,
|
||||
});
|
||||
|
||||
const appConfig = await getAppConfig();
|
||||
const appConfig = await getAppConfig({ baseOnly: true });
|
||||
|
||||
if (!isEmailDomainAllowed(email, appConfig?.registration?.allowedDomains)) {
|
||||
logger.error(
|
||||
|
|
|
|||
|
|
@ -77,6 +77,8 @@ export interface AdminConfigDeps {
|
|||
userId?: string;
|
||||
tenantId?: string;
|
||||
}) => Promise<AppConfig>;
|
||||
/** Invalidate all config-related caches after a mutation. */
|
||||
invalidateConfigCaches?: (tenantId?: string) => Promise<void>;
|
||||
}
|
||||
|
||||
// ── Validation helpers ───────────────────────────────────────────────
|
||||
|
|
@ -133,6 +135,7 @@ export function createAdminConfigHandlers(deps: AdminConfigDeps) {
|
|||
toggleConfigActive,
|
||||
hasConfigCapability,
|
||||
getAppConfig,
|
||||
invalidateConfigCaches,
|
||||
} = deps;
|
||||
|
||||
/**
|
||||
|
|
@ -176,7 +179,9 @@ export function createAdminConfigHandlers(deps: AdminConfigDeps) {
|
|||
return res.status(501).json({ error: 'Base config endpoint not configured' });
|
||||
}
|
||||
|
||||
const appConfig = await getAppConfig();
|
||||
const appConfig = await getAppConfig({
|
||||
tenantId: user.tenantId,
|
||||
});
|
||||
return res.status(200).json({ config: appConfig });
|
||||
} catch (error) {
|
||||
logger.error('[adminConfig] getBaseConfig error:', error);
|
||||
|
|
@ -278,6 +283,9 @@ export function createAdminConfigHandlers(deps: AdminConfigDeps) {
|
|||
priority ?? DEFAULT_PRIORITY,
|
||||
);
|
||||
|
||||
invalidateConfigCaches?.(user.tenantId)?.catch((err) =>
|
||||
logger.error('[adminConfig] Cache invalidation failed after upsert:', err),
|
||||
);
|
||||
return res.status(config?.configVersion === 1 ? 201 : 200).json({ config });
|
||||
} catch (error) {
|
||||
logger.error('[adminConfig] upsertConfigOverrides error:', error);
|
||||
|
|
@ -367,6 +375,9 @@ export function createAdminConfigHandlers(deps: AdminConfigDeps) {
|
|||
priority ?? existing?.priority ?? DEFAULT_PRIORITY,
|
||||
);
|
||||
|
||||
invalidateConfigCaches?.(user.tenantId)?.catch((err) =>
|
||||
logger.error('[adminConfig] Cache invalidation failed after patch:', err),
|
||||
);
|
||||
return res.status(200).json({ config });
|
||||
} catch (error) {
|
||||
logger.error('[adminConfig] patchConfigField error:', error);
|
||||
|
|
@ -414,6 +425,9 @@ export function createAdminConfigHandlers(deps: AdminConfigDeps) {
|
|||
return res.status(404).json({ error: 'Config not found' });
|
||||
}
|
||||
|
||||
invalidateConfigCaches?.(user.tenantId)?.catch((err) =>
|
||||
logger.error('[adminConfig] Cache invalidation failed after field delete:', err),
|
||||
);
|
||||
return res.status(200).json({ config });
|
||||
} catch (error) {
|
||||
logger.error('[adminConfig] deleteConfigField error:', error);
|
||||
|
|
@ -449,6 +463,9 @@ export function createAdminConfigHandlers(deps: AdminConfigDeps) {
|
|||
return res.status(404).json({ error: 'Config not found' });
|
||||
}
|
||||
|
||||
invalidateConfigCaches?.(user.tenantId)?.catch((err) =>
|
||||
logger.error('[adminConfig] Cache invalidation failed after config delete:', err),
|
||||
);
|
||||
return res.status(200).json({ success: true });
|
||||
} catch (error) {
|
||||
logger.error('[adminConfig] deleteConfigOverrides error:', error);
|
||||
|
|
@ -489,6 +506,9 @@ export function createAdminConfigHandlers(deps: AdminConfigDeps) {
|
|||
return res.status(404).json({ error: 'Config not found' });
|
||||
}
|
||||
|
||||
invalidateConfigCaches?.(user.tenantId)?.catch((err) =>
|
||||
logger.error('[adminConfig] Cache invalidation failed after toggle:', err),
|
||||
);
|
||||
return res.status(200).json({ config });
|
||||
} catch (error) {
|
||||
logger.error('[adminConfig] toggleConfig error:', error);
|
||||
|
|
|
|||
|
|
@ -1,17 +1,24 @@
|
|||
import { createAppConfigService } from './service';
|
||||
|
||||
function createMockCache() {
|
||||
/**
|
||||
* Creates a mock cache that simulates Keyv's namespace behavior.
|
||||
* Keyv stores keys internally as `namespace:key` but its API (get/set/delete)
|
||||
* accepts un-namespaced keys and auto-prepends the namespace.
|
||||
*/
|
||||
function createMockCache(namespace = 'app_config') {
|
||||
const store = new Map();
|
||||
return {
|
||||
get: jest.fn((key) => Promise.resolve(store.get(key))),
|
||||
get: jest.fn((key) => Promise.resolve(store.get(`${namespace}:${key}`))),
|
||||
set: jest.fn((key, value) => {
|
||||
store.set(key, value);
|
||||
store.set(`${namespace}:${key}`, value);
|
||||
return Promise.resolve(undefined);
|
||||
}),
|
||||
delete: jest.fn((key) => {
|
||||
store.delete(key);
|
||||
store.delete(`${namespace}:${key}`);
|
||||
return Promise.resolve(true);
|
||||
}),
|
||||
/** Mimic Keyv's opts.store structure for key enumeration in clearOverrideCache */
|
||||
opts: { store: { keys: () => store.keys() } },
|
||||
_store: store,
|
||||
};
|
||||
}
|
||||
|
|
@ -58,6 +65,23 @@ describe('createAppConfigService', () => {
|
|||
expect(deps.loadBaseConfig).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('baseOnly returns YAML config without DB queries', async () => {
|
||||
const deps = createDeps({
|
||||
getApplicableConfigs: jest
|
||||
.fn()
|
||||
.mockResolvedValue([
|
||||
{ priority: 10, overrides: { interface: { endpointsMenu: false } }, isActive: true },
|
||||
]),
|
||||
});
|
||||
const { getAppConfig } = createAppConfigService(deps);
|
||||
|
||||
const config = await getAppConfig({ baseOnly: true });
|
||||
|
||||
expect(deps.loadBaseConfig).toHaveBeenCalledTimes(1);
|
||||
expect(deps.getApplicableConfigs).not.toHaveBeenCalled();
|
||||
expect(config).toEqual(deps._baseConfig);
|
||||
});
|
||||
|
||||
it('reloads base config when refresh is true', async () => {
|
||||
const deps = createDeps();
|
||||
const { getAppConfig } = createAppConfigService(deps);
|
||||
|
|
@ -144,8 +168,8 @@ describe('createAppConfigService', () => {
|
|||
await getAppConfig({ userId: 'uid1' });
|
||||
|
||||
const cachedKeys = [...deps._cache._store.keys()];
|
||||
const overrideKey = cachedKeys.find((k) => k.startsWith('_OVERRIDE_:'));
|
||||
expect(overrideKey).toBe('_OVERRIDE_:__default__:uid1');
|
||||
const overrideKey = cachedKeys.find((k) => k.includes('_OVERRIDE_:'));
|
||||
expect(overrideKey).toBe('app_config:_OVERRIDE_:__default__:uid1');
|
||||
});
|
||||
|
||||
it('tenantId is included in cache key to prevent cross-tenant contamination', async () => {
|
||||
|
|
@ -241,4 +265,70 @@ describe('createAppConfigService', () => {
|
|||
expect(deps.loadBaseConfig).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('clearOverrideCache', () => {
|
||||
it('clears all override caches when no tenantId is provided', async () => {
|
||||
const deps = createDeps({
|
||||
getApplicableConfigs: jest
|
||||
.fn()
|
||||
.mockResolvedValue([{ priority: 10, overrides: { x: 1 }, isActive: true }]),
|
||||
});
|
||||
const { getAppConfig, clearOverrideCache } = createAppConfigService(deps);
|
||||
|
||||
await getAppConfig({ role: 'ADMIN', tenantId: 'tenant-a' });
|
||||
await getAppConfig({ role: 'ADMIN', tenantId: 'tenant-b' });
|
||||
expect(deps.getApplicableConfigs).toHaveBeenCalledTimes(2);
|
||||
|
||||
await clearOverrideCache();
|
||||
|
||||
// After clearing, both tenants should re-query DB
|
||||
await getAppConfig({ role: 'ADMIN', tenantId: 'tenant-a' });
|
||||
await getAppConfig({ role: 'ADMIN', tenantId: 'tenant-b' });
|
||||
expect(deps.getApplicableConfigs).toHaveBeenCalledTimes(4);
|
||||
});
|
||||
|
||||
it('clears only specified tenant override caches', async () => {
|
||||
const deps = createDeps({
|
||||
getApplicableConfigs: jest
|
||||
.fn()
|
||||
.mockResolvedValue([{ priority: 10, overrides: { x: 1 }, isActive: true }]),
|
||||
});
|
||||
const { getAppConfig, clearOverrideCache } = createAppConfigService(deps);
|
||||
|
||||
await getAppConfig({ role: 'ADMIN', tenantId: 'tenant-a' });
|
||||
await getAppConfig({ role: 'ADMIN', tenantId: 'tenant-b' });
|
||||
expect(deps.getApplicableConfigs).toHaveBeenCalledTimes(2);
|
||||
|
||||
await clearOverrideCache('tenant-a');
|
||||
|
||||
// tenant-a should re-query, tenant-b should be cached
|
||||
await getAppConfig({ role: 'ADMIN', tenantId: 'tenant-a' });
|
||||
await getAppConfig({ role: 'ADMIN', tenantId: 'tenant-b' });
|
||||
expect(deps.getApplicableConfigs).toHaveBeenCalledTimes(3);
|
||||
});
|
||||
|
||||
it('does not clear base config', async () => {
|
||||
const deps = createDeps();
|
||||
const { getAppConfig, clearOverrideCache } = createAppConfigService(deps);
|
||||
|
||||
await getAppConfig();
|
||||
expect(deps.loadBaseConfig).toHaveBeenCalledTimes(1);
|
||||
|
||||
await clearOverrideCache();
|
||||
|
||||
await getAppConfig();
|
||||
// Base config should still be cached
|
||||
expect(deps.loadBaseConfig).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('does not throw when store.keys is unavailable (Redis fallback to TTL expiry)', async () => {
|
||||
const deps = createDeps();
|
||||
// Remove store.keys to simulate Redis-backed cache
|
||||
deps._cache.opts = {};
|
||||
const { clearOverrideCache } = createAppConfigService(deps);
|
||||
|
||||
// Should not throw — logs warning and relies on TTL expiry
|
||||
await expect(clearOverrideCache()).resolves.toBeUndefined();
|
||||
});
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -13,6 +13,12 @@ interface CacheStore {
|
|||
get: (key: string) => Promise<unknown>;
|
||||
set: (key: string, value: unknown, ttl?: number) => Promise<unknown>;
|
||||
delete: (key: string) => Promise<boolean>;
|
||||
/** Keyv options — used for key enumeration when clearing override caches. */
|
||||
opts?: {
|
||||
store?: {
|
||||
keys?: () => IterableIterator<string>;
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
export interface AppConfigServiceDeps {
|
||||
|
|
@ -39,8 +45,28 @@ export interface AppConfigServiceDeps {
|
|||
|
||||
// ── Helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
let _strictOverride: boolean | undefined;
|
||||
function isStrictOverrideMode(): boolean {
|
||||
return (_strictOverride ??= process.env.TENANT_ISOLATION_STRICT === 'true');
|
||||
}
|
||||
|
||||
/** @internal Resets the cached strict-override flag. Exposed for test teardown only. */
|
||||
let _warnedNoTenantInStrictMode = false;
|
||||
|
||||
export function _resetOverrideStrictCache(): void {
|
||||
_strictOverride = undefined;
|
||||
_warnedNoTenantInStrictMode = false;
|
||||
}
|
||||
|
||||
function overrideCacheKey(role?: string, userId?: string, tenantId?: string): string {
|
||||
const tenant = tenantId || '__default__';
|
||||
if (!tenantId && isStrictOverrideMode() && !_warnedNoTenantInStrictMode) {
|
||||
_warnedNoTenantInStrictMode = true;
|
||||
logger.warn(
|
||||
'[overrideCacheKey] No tenantId in strict mode — falling back to __default__. ' +
|
||||
'This likely indicates a code path that bypasses the tenant context middleware.',
|
||||
);
|
||||
}
|
||||
if (userId && role) {
|
||||
return `_OVERRIDE_:${tenant}:${role}:${userId}`;
|
||||
}
|
||||
|
|
@ -83,20 +109,13 @@ export function createAppConfigService(deps: AppConfigServiceDeps) {
|
|||
}
|
||||
|
||||
/**
|
||||
* Get the app configuration, optionally merged with DB overrides for the given principal.
|
||||
*
|
||||
* The base config (from YAML + AppService) is cached indefinitely. Per-principal merged
|
||||
* configs are cached with a short TTL (`overrideCacheTtl`, default 60s). On cache miss,
|
||||
* `getApplicableConfigs` queries the DB for matching overrides and merges them by priority.
|
||||
* Ensure the YAML-derived base config is loaded and cached.
|
||||
* Returns the `_BASE_` config (YAML + AppService). No DB queries.
|
||||
*/
|
||||
async function getAppConfig(
|
||||
options: { role?: string; userId?: string; tenantId?: string; refresh?: boolean } = {},
|
||||
): Promise<AppConfig> {
|
||||
const { role, userId, tenantId, refresh } = options;
|
||||
|
||||
async function ensureBaseConfig(refresh?: boolean): Promise<AppConfig> {
|
||||
let baseConfig = (await cache.get(BASE_CONFIG_KEY)) as AppConfig | undefined;
|
||||
if (!baseConfig || refresh) {
|
||||
logger.info('[getAppConfig] Loading base configuration...');
|
||||
logger.info('[ensureBaseConfig] Loading base configuration...');
|
||||
baseConfig = await loadBaseConfig();
|
||||
|
||||
if (!baseConfig) {
|
||||
|
|
@ -109,6 +128,37 @@ export function createAppConfigService(deps: AppConfigServiceDeps) {
|
|||
|
||||
await cache.set(BASE_CONFIG_KEY, baseConfig);
|
||||
}
|
||||
return baseConfig;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the app configuration, optionally merged with DB overrides for the given principal.
|
||||
*
|
||||
* The base config (from YAML + AppService) is cached indefinitely. Per-principal merged
|
||||
* configs are cached with a short TTL (`overrideCacheTtl`, default 60s). On cache miss,
|
||||
* `getApplicableConfigs` queries the DB for matching overrides and merges them by priority.
|
||||
*
|
||||
* When `baseOnly` is true, returns the YAML-derived config without any DB queries.
|
||||
* `role`, `userId`, and `tenantId` are ignored in this mode.
|
||||
* Use this for startup, auth strategies, and other pre-tenant code paths.
|
||||
*/
|
||||
async function getAppConfig(
|
||||
options: {
|
||||
role?: string;
|
||||
userId?: string;
|
||||
tenantId?: string;
|
||||
refresh?: boolean;
|
||||
/** When true, return only the YAML-derived base config — no DB override queries. */
|
||||
baseOnly?: boolean;
|
||||
} = {},
|
||||
): Promise<AppConfig> {
|
||||
const { role, userId, tenantId, refresh, baseOnly } = options;
|
||||
|
||||
const baseConfig = await ensureBaseConfig(refresh);
|
||||
|
||||
if (baseOnly) {
|
||||
return baseConfig;
|
||||
}
|
||||
|
||||
const cacheKey = overrideCacheKey(role, userId, tenantId);
|
||||
if (!refresh) {
|
||||
|
|
@ -146,9 +196,55 @@ export function createAppConfigService(deps: AppConfigServiceDeps) {
|
|||
await cache.delete(BASE_CONFIG_KEY);
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear per-principal override caches. When `tenantId` is provided, only caches
|
||||
* matching `_OVERRIDE_:${tenantId}:*` are deleted. When omitted, ALL override
|
||||
* caches are cleared.
|
||||
*/
|
||||
async function clearOverrideCache(tenantId?: string): Promise<void> {
|
||||
const namespace = cacheKeys.APP_CONFIG;
|
||||
const overrideSegment = tenantId ? `_OVERRIDE_:${tenantId}:` : '_OVERRIDE_:';
|
||||
|
||||
// In-memory store — enumerate keys directly.
|
||||
// APP_CONFIG defaults to FORCED_IN_MEMORY_CACHE_NAMESPACES, so this is the
|
||||
// standard path. Redis SCAN is intentionally avoided here — it can cause 60s+
|
||||
// stalls under concurrent load (see #12410). When APP_CONFIG is Redis-backed
|
||||
// and store.keys() is unavailable, overrides expire naturally via TTL.
|
||||
const store = (cache as CacheStore).opts?.store;
|
||||
if (store && typeof store.keys === 'function') {
|
||||
// Keyv stores keys with a namespace prefix (e.g. "APP_CONFIG:_OVERRIDE_:...").
|
||||
// We match on the namespaced key but delete using the un-namespaced key
|
||||
// because Keyv.delete() auto-prepends the namespace.
|
||||
const namespacedPrefix = `${namespace}:${overrideSegment}`;
|
||||
const toDelete: string[] = [];
|
||||
for (const key of store.keys()) {
|
||||
if (key.startsWith(namespacedPrefix)) {
|
||||
toDelete.push(key.slice(namespace.length + 1));
|
||||
}
|
||||
}
|
||||
if (toDelete.length > 0) {
|
||||
await Promise.all(toDelete.map((key) => cache.delete(key)));
|
||||
logger.info(
|
||||
`[clearOverrideCache] Cleared ${toDelete.length} override cache entries` +
|
||||
(tenantId ? ` for tenant ${tenantId}` : ''),
|
||||
);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
logger.warn(
|
||||
'[clearOverrideCache] Cache store does not support key enumeration. ' +
|
||||
'Override caches will expire naturally via TTL (%dms). ' +
|
||||
'This is expected when APP_CONFIG is Redis-backed — Redis SCAN is avoided ' +
|
||||
'for performance reasons (see #12410).',
|
||||
overrideCacheTtl,
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
getAppConfig,
|
||||
clearAppConfigCache,
|
||||
clearOverrideCache,
|
||||
};
|
||||
}
|
||||
|
||||
|
|
|
|||
101
packages/api/src/middleware/__tests__/tenant.spec.ts
Normal file
101
packages/api/src/middleware/__tests__/tenant.spec.ts
Normal file
|
|
@ -0,0 +1,101 @@
|
|||
import { getTenantId } from '@librechat/data-schemas';
|
||||
import type { Response, NextFunction } from 'express';
|
||||
import type { ServerRequest } from '~/types/http';
|
||||
// Import directly from source file — _resetTenantMiddlewareStrictCache is intentionally
|
||||
// excluded from the public barrel export (index.ts).
|
||||
import { tenantContextMiddleware, _resetTenantMiddlewareStrictCache } from '../tenant';
|
||||
|
||||
function mockReq(user?: Record<string, unknown>): ServerRequest {
|
||||
return { user } as unknown as ServerRequest;
|
||||
}
|
||||
|
||||
function mockRes(): Response {
|
||||
const res = {
|
||||
status: jest.fn().mockReturnThis(),
|
||||
json: jest.fn().mockReturnThis(),
|
||||
};
|
||||
return res as unknown as Response;
|
||||
}
|
||||
|
||||
/** Runs the middleware and returns a Promise that resolves when next() is called. */
|
||||
function runMiddleware(req: ServerRequest, res: Response): Promise<string | undefined> {
|
||||
return new Promise((resolve) => {
|
||||
const next: NextFunction = () => {
|
||||
resolve(getTenantId());
|
||||
};
|
||||
tenantContextMiddleware(req, res, next);
|
||||
});
|
||||
}
|
||||
|
||||
describe('tenantContextMiddleware', () => {
|
||||
afterEach(() => {
|
||||
_resetTenantMiddlewareStrictCache();
|
||||
delete process.env.TENANT_ISOLATION_STRICT;
|
||||
});
|
||||
|
||||
it('sets ALS tenant context for authenticated requests with tenantId', async () => {
|
||||
const req = mockReq({ tenantId: 'tenant-x', role: 'user' });
|
||||
const res = mockRes();
|
||||
|
||||
const tenantId = await runMiddleware(req, res);
|
||||
expect(tenantId).toBe('tenant-x');
|
||||
});
|
||||
|
||||
it('is a no-op for unauthenticated requests (no user)', async () => {
|
||||
const req = mockReq();
|
||||
const res = mockRes();
|
||||
|
||||
const tenantId = await runMiddleware(req, res);
|
||||
expect(tenantId).toBeUndefined();
|
||||
});
|
||||
|
||||
it('passes through without ALS when user has no tenantId in non-strict mode', async () => {
|
||||
const req = mockReq({ role: 'user' });
|
||||
const res = mockRes();
|
||||
|
||||
const tenantId = await runMiddleware(req, res);
|
||||
expect(tenantId).toBeUndefined();
|
||||
});
|
||||
|
||||
it('returns 403 when user has no tenantId in strict mode', () => {
|
||||
process.env.TENANT_ISOLATION_STRICT = 'true';
|
||||
_resetTenantMiddlewareStrictCache();
|
||||
|
||||
const req = mockReq({ role: 'user' });
|
||||
const res = mockRes();
|
||||
const next: NextFunction = jest.fn();
|
||||
|
||||
tenantContextMiddleware(req, res, next);
|
||||
|
||||
expect(res.status).toHaveBeenCalledWith(403);
|
||||
expect(res.json).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ error: expect.stringContaining('Tenant context required') }),
|
||||
);
|
||||
expect(next).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('allows authenticated requests with tenantId in strict mode', async () => {
|
||||
process.env.TENANT_ISOLATION_STRICT = 'true';
|
||||
_resetTenantMiddlewareStrictCache();
|
||||
|
||||
const req = mockReq({ tenantId: 'tenant-y', role: 'admin' });
|
||||
const res = mockRes();
|
||||
|
||||
const tenantId = await runMiddleware(req, res);
|
||||
expect(tenantId).toBe('tenant-y');
|
||||
});
|
||||
|
||||
it('different requests get independent tenant contexts', async () => {
|
||||
const runRequest = (tid: string) => {
|
||||
const req = mockReq({ tenantId: tid, role: 'user' });
|
||||
const res = mockRes();
|
||||
return runMiddleware(req, res);
|
||||
};
|
||||
|
||||
const results = await Promise.all([runRequest('tenant-1'), runRequest('tenant-2')]);
|
||||
|
||||
expect(results).toHaveLength(2);
|
||||
expect(results).toContain('tenant-1');
|
||||
expect(results).toContain('tenant-2');
|
||||
});
|
||||
});
|
||||
|
|
@ -12,7 +12,11 @@ import type { BalanceUpdateFields } from '~/types';
|
|||
import { getBalanceConfig } from '~/app/config';
|
||||
|
||||
export interface BalanceMiddlewareOptions {
|
||||
getAppConfig: (options?: { role?: string; refresh?: boolean }) => Promise<AppConfig>;
|
||||
getAppConfig: (options?: {
|
||||
role?: string;
|
||||
tenantId?: string;
|
||||
refresh?: boolean;
|
||||
}) => Promise<AppConfig>;
|
||||
findBalanceByUser: (userId: string) => Promise<IBalance | null>;
|
||||
upsertBalanceFields: (userId: string, fields: IBalanceUpdate) => Promise<IBalance | null>;
|
||||
}
|
||||
|
|
@ -92,7 +96,10 @@ export function createSetBalanceConfig({
|
|||
return async (req: ServerRequest, res: ServerResponse, next: NextFunction): Promise<void> => {
|
||||
try {
|
||||
const user = req.user as IUser & { _id: string | ObjectId };
|
||||
const appConfig = await getAppConfig({ role: user?.role });
|
||||
const appConfig = await getAppConfig({
|
||||
role: user?.role,
|
||||
tenantId: user?.tenantId,
|
||||
});
|
||||
const balanceConfig = getBalanceConfig(appConfig);
|
||||
if (!balanceConfig?.enabled) {
|
||||
return next();
|
||||
|
|
|
|||
|
|
@ -5,5 +5,6 @@ export * from './notFound';
|
|||
export * from './balance';
|
||||
export * from './json';
|
||||
export * from './capabilities';
|
||||
export { tenantContextMiddleware } from './tenant';
|
||||
export * from './concurrency';
|
||||
export * from './checkBalance';
|
||||
|
|
|
|||
70
packages/api/src/middleware/tenant.ts
Normal file
70
packages/api/src/middleware/tenant.ts
Normal file
|
|
@ -0,0 +1,70 @@
|
|||
import { isMainThread } from 'worker_threads';
|
||||
import { tenantStorage, logger } from '@librechat/data-schemas';
|
||||
import type { Response, NextFunction } from 'express';
|
||||
import type { ServerRequest } from '~/types/http';
|
||||
|
||||
let _checkedThread = false;
|
||||
|
||||
let _strictMode: boolean | undefined;
|
||||
|
||||
function isStrict(): boolean {
|
||||
return (_strictMode ??= process.env.TENANT_ISOLATION_STRICT === 'true');
|
||||
}
|
||||
|
||||
/** Resets the cached strict-mode flag. Exposed for test teardown only. */
|
||||
export function _resetTenantMiddlewareStrictCache(): void {
|
||||
_strictMode = undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Express middleware that propagates the authenticated user's `tenantId` into
|
||||
* the AsyncLocalStorage context used by the Mongoose tenant-isolation plugin.
|
||||
*
|
||||
* **Placement**: Chained automatically by `requireJwtAuth` after successful
|
||||
* passport authentication (req.user is populated). Must NOT be registered at
|
||||
* global `app.use()` scope — `req.user` is undefined at that stage.
|
||||
*
|
||||
* Behaviour:
|
||||
* - Authenticated request with `tenantId` → wraps downstream in `tenantStorage.run({ tenantId })`
|
||||
* - Authenticated request **without** `tenantId`:
|
||||
* - Strict mode (`TENANT_ISOLATION_STRICT=true`) → responds 403
|
||||
* - Non-strict (default) → passes through without ALS context (backward compat)
|
||||
* - Unauthenticated request → no-op (calls `next()` directly)
|
||||
*/
|
||||
export function tenantContextMiddleware(
|
||||
req: ServerRequest,
|
||||
res: Response,
|
||||
next: NextFunction,
|
||||
): void {
|
||||
if (!_checkedThread) {
|
||||
_checkedThread = true;
|
||||
if (!isMainThread) {
|
||||
logger.error(
|
||||
'[tenantContextMiddleware] Running in a worker thread — ' +
|
||||
'ALS context will not propagate. This middleware must only run in the main Express process.',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const user = req.user as { tenantId?: string } | undefined;
|
||||
|
||||
if (!user) {
|
||||
next();
|
||||
return;
|
||||
}
|
||||
|
||||
const tenantId = user.tenantId;
|
||||
|
||||
if (!tenantId) {
|
||||
if (isStrict()) {
|
||||
res.status(403).json({ error: 'Tenant context required in strict isolation mode' });
|
||||
return;
|
||||
}
|
||||
next();
|
||||
return;
|
||||
}
|
||||
|
||||
return void tenantStorage.run({ tenantId }, async () => {
|
||||
next();
|
||||
});
|
||||
}
|
||||
|
|
@ -236,21 +236,28 @@ export function createUserGroupMethods(mongoose: typeof import('mongoose')) {
|
|||
}
|
||||
|
||||
/**
|
||||
* Get a list of all principal identifiers for a user (user ID + group IDs + public)
|
||||
* For use in permission checks
|
||||
* Get a list of all principal identifiers for a user (user ID + group IDs + public).
|
||||
* For use in permission checks.
|
||||
*
|
||||
* Tenant filtering for group memberships is handled automatically by the
|
||||
* `applyTenantIsolation` Mongoose plugin on the Group schema. The
|
||||
* `tenantContextMiddleware` (chained by `requireJwtAuth` after passport auth)
|
||||
* sets the ALS context, so `getUserGroups()` → `findGroupsByMemberId()` queries
|
||||
* are scoped to the requesting tenant. No explicit tenantId parameter is needed.
|
||||
*
|
||||
* IMPORTANT: This relies on the ALS tenant context being active. If this
|
||||
* function is called outside a request context (e.g. startup, background jobs),
|
||||
* group queries will be unscoped. In strict mode, the Mongoose plugin will
|
||||
* reject such queries.
|
||||
*
|
||||
* Ref: #12091 (resolved by tenant context middleware in requireJwtAuth)
|
||||
*
|
||||
* @param params - Parameters object
|
||||
* @param params.userId - The user ID
|
||||
* @param params.role - Optional user role (if not provided, will query from DB)
|
||||
* @param session - Optional MongoDB session for transactions
|
||||
* @returns Array of principal objects with type and id
|
||||
*/
|
||||
/**
|
||||
* TODO(#12091): This method has no tenantId parameter — it returns ALL group
|
||||
* memberships for a user regardless of tenant. In multi-tenant mode, group
|
||||
* principals from other tenants will be included in capability checks, which
|
||||
* could grant cross-tenant capabilities. Add tenantId filtering here when
|
||||
* tenant isolation is activated.
|
||||
*/
|
||||
async function getUserPrincipals(
|
||||
params: {
|
||||
userId: string | Types.ObjectId;
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue