1045 commits

Author	SHA1	Message	Date
Danny Avila	fda72ac621	🏗️ refactor: Remove Redundant Caching, Migrate Config Services to TypeScript (#12466) ... * ♻️ refactor: Remove redundant scopedCacheKey caching, support user-provided key model fetching Remove redundant cache layers that used `scopedCacheKey()` (tenant-only scoping) on top of `getAppConfig()` which already caches per-principal (role+user+tenant). This caused config overrides for different principals within the same tenant to be invisible due to stale cached data. Changes: - Add `requireJwtAuth` to `/api/endpoints` route for proper user context - Remove ENDPOINT_CONFIG, STARTUP_CONFIG, PLUGINS, TOOLS, and MODELS_CONFIG cache layers — all derive from `getAppConfig()` with cheap computation - Enhance MODEL_QUERIES cache: hash(baseURL+apiKey) keys, 2-minute TTL, caching centralized in `fetchModels()` base function - Support fetching models with user-provided API keys in `loadConfigModels` via `getUserKeyValues` lookup (no caching for user keys) - Update all affected tests Closes #1028 * ♻️ refactor: Migrate config services to TypeScript in packages/api Move core config logic from CJS /api wrappers to typed TypeScript in packages/api using dependency injection factories: - `createEndpointsConfigService` — endpoint config merging + checkCapability - `createLoadConfigModels` — custom endpoint model loading with user key support - `createMCPToolCacheService` — MCP tool cache operations (update, merge, cache) /api files become thin wrappers that wire dependencies (getAppConfig, loadDefaultEndpointsConfig, getUserKeyValues, getCachedTools, etc.) into the typed factories. Also moves existing `endpoints/config.ts` → `endpoints/config/providers.ts` to accommodate the new `config/` directory structure. * 🔄 fix: Invalidate models query when user API key is set or revoked Without this, users had to refresh the page after entering their API key to see the updated model list fetched with their credentials. - Invalidate QueryKeys.models in useUpdateUserKeysMutation onSuccess - Invalidate QueryKeys.models in useRevokeUserKeyMutation onSuccess - Invalidate QueryKeys.models in useRevokeAllUserKeysMutation onSuccess * 🗺️ fix: Remap YAML-level override keys to AppConfig equivalents in mergeConfigOverrides Config overrides stored in the DB use YAML-level keys (TCustomConfig), but they're merged into the already-processed AppConfig where some fields have been renamed by AppService. This caused mcpServers overrides to land on a nonexistent key instead of mcpConfig, so config-override MCP servers never appeared in the UI. - Add OVERRIDE_KEY_MAP to remap mcpServers→mcpConfig, interface→interfaceConfig - Apply remapping before deep merge in mergeConfigOverrides - Add test for YAML-level key remapping behavior - Update existing tests to use AppConfig field names in assertions * 🧪 test: Update service.spec to use AppConfig field names after override key remapping * 🛡️ fix: Address code review findings — reliability, types, tests, and performance - Pass tenant context (getTenantId) in importers.js getEndpointsConfig call - Add 5 tests for user-provided API key model fetching (key found, no key, DB error, missing userId, apiKey-only with fixed baseURL) - Distinguish NO_USER_KEY (debug) from infrastructure errors (warn) in catch - Switch fetchPromisesMap from Promise.all to Promise.allSettled so one failing provider doesn't kill the entire model config - Parallelize getUserKeyValues DB lookups via batched Promise.allSettled instead of sequential awaits in the loop - Hoist standardCache instance in fetchModels to avoid double instantiation - Replace Record<string, unknown> types with Partial<TConfig>-based types; remove as unknown as T double-cast in endpoints config - Narrow Bedrock availableRegions to typed destructure - Narrow version field from string|number|undefined to string|undefined - Fix import ordering in mcp/tools.ts and config/models.ts per AGENTS.md - Add JSDoc to getModelsConfig alias clarifying caching semantics * fix: Guard against null getCachedTools in mergeAppTools * 🔍 fix: Address follow-up review — deduplicate extractEnvVariable, fix error discrimination, add log-level tests - Deduplicate extractEnvVariable calls: resolve apiKey/baseURL once, reuse for both the entry and isUserProvided checks (Finding A) - Move ResolvedEndpoint interface from function closure to module scope (Finding B) - Replace fragile msg.includes('NO_USER_KEY') with ErrorTypes.NO_USER_KEY enum check against actual error message format (Finding C). Also handle ErrorTypes.INVALID_USER_KEY as an expected "no key" case. - Add test asserting logger.warn is called for infra errors (not debug) - Add test asserting logger.debug is called for NO_USER_KEY errors (not warn) * fix: Preserve numeric assistants version via String() coercion * 🐛 fix: Address secondary review — Ollama cache bypass, cache tests, type safety - Fix Ollama success path bypassing cache write in fetchModels (CRITICAL): store result before returning so Ollama models benefit from 2-minute TTL - Add 4 fetchModels cache behavior tests: cache write with TTL, cache hit short-circuits HTTP, skipCache bypasses read+write, empty results not cached - Type-safe OVERRIDE_KEY_MAP: Partial<Record<keyof TCustomConfig, keyof AppConfig>> so compiler catches future field rename mismatches - Fix import ordering in config/models.ts (package types longest→shortest) - Rename ToolCacheDeps → MCPToolCacheDeps for naming consistency - Expand getModelsConfig JSDoc to explain caching granularity * fix: Narrow OVERRIDE_KEY_MAP index to satisfy strict tsconfig * 🧩 fix: Add allowedProviders to TConfig, remove Record<string, unknown> from PartialEndpointEntry The agents endpoint config includes allowedProviders (used by the frontend AgentPanel to filter available providers), but it was missing from TConfig. This forced PartialEndpointEntry to use & Record<string, unknown> as an escape hatch, violating AGENTS.md type policy. - Add allowedProviders?: (string | EModelEndpoint)[] to TConfig - Remove Record<string, unknown> from PartialEndpointEntry — now just Partial<TConfig> * 🛡️ fix: Isolate Ollama cache write from fetch try-catch, add Ollama cache tests - Separate Ollama fetch and cache write into distinct scopes so a cache failure (e.g., Redis down) doesn't misattribute the error as an Ollama API failure and fall through to the OpenAI-compatible path (Issue A) - Add 2 Ollama-specific cache tests: models written with TTL on fetch, cached models returned without hitting server (Issue B) - Replace hardcoded 120000 with Time.TWO_MINUTES constant in cache TTL test assertion (Issue C) - Fix OVERRIDE_KEY_MAP JSDoc to accurately describe runtime vs compile-time type enforcement (Issue D) - Add global beforeEach for cache mock reset to prevent cross-test leakage * 🧪 fix: Address third review — DI consistency, cache key width, MCP tests - Inject loadCustomEndpointsConfig via EndpointsConfigDeps with default fallback, matching loadDefaultEndpointsConfig DI pattern (Finding 3) - Widen modelsCacheKey from 64-bit (.slice(0,16)) to 128-bit (.slice(0,32)) for collision-sensitive cross-credential cache key (Finding 4) - Add fetchModels.mockReset() in loadConfigModels.spec beforeEach to prevent mock implementation leaks across tests (Finding 5) - Add 11 unit tests for createMCPToolCacheService covering all three functions: null/empty input, successful ops, error propagation, cold-cache merge (Finding 2) - Simplify getModelsConfig JSDoc to @see reference (Finding 10) * ♻️ refactor: Address remaining follow-ups from reviews OVERRIDE_KEY_MAP completeness: - Add missing turnstile→turnstileConfig mapping - Add exhaustiveness test verifying all three renamed keys are remapped and original YAML keys don't leak through Import role context: - Pass userRole through importConversations job → importLibreChatConvo so role-based endpoint overrides are honored during conversation import - Update convos.js route to include req.user.role in the job payload createEndpointsConfigService unit tests: - Add 8 tests covering: default+custom merge, Azure/AzureAssistants/ Anthropic Vertex/Bedrock config enrichment, assistants version coercion, agents allowedProviders, req.config bypass Plugins/tools efficiency: - Use Set for includedTools/filteredTools lookups (O(1) vs O(n) per plugin) - Combine auth check + filter into single pass (eliminates intermediate array) - Pre-compute toolDefKeys Set for O(1) tool definition lookups * fix: Scope model query cache by user when userIdQuery is enabled * fix: Skip model cache for userIdQuery endpoints, fix endpoints test types - When userIdQuery is true, skip caching entirely (like user_provided keys) to avoid cross-user model list leakage without duplicating cache data - Fix AgentCapabilities type error in endpoints.spec.ts — use enum values and appConfig() helper for partial mock typing * 🐛 fix: Restore filteredTools+includedTools composition, add checkCapability tests - Fix filteredTools regression: whitelist and blacklist are now applied independently (two flat guards), matching original behavior where includedTools=['a','b'] + filteredTools=['b'] produces ['a'] (Finding A) - Fix Set spread in toolkit loop: pre-compute toolDefKeysList array once alongside the Set, reuse for .some() without per-plugin allocation (Finding B) - Add 2 filteredTools tests: blacklist-only path and combined whitelist+blacklist composition (Finding C) - Add 3 checkCapability tests: capability present, capability absent, fallback to defaultAgentCapabilities for non-agents endpoints (Finding D) * 🔑 fix: Include config-override MCP servers in filterAuthorizedTools Config-override MCP servers (defined via admin config overrides for roles/groups) were rejected by filterAuthorizedTools because it called getAllServerConfigs(userId) without the configServers parameter. Only YAML and DB-backed user servers were included in the access check. - Add configServers parameter to filterAuthorizedTools - Resolve config servers via resolveConfigServers(req) at all 4 callsites (create, update, duplicate, revert) using parallel Promise.all - Pass configServers through to getAllServerConfigs(userId, configServers) so the registry merges config-source servers into the access check - Update filterAuthorizedTools.spec.js mock for resolveConfigServers * fix: Skip model cache for userIdQuery endpoints, fix endpoints test types For user-provided key endpoints (userProvide: true), skip the full model list re-fetch during message validation — the user already selected from a list we served them, and re-fetching with skipCache:true on every message send is both slow and fragile (5s provider timeout = rejected model). Instead, validate the model string format only: - Must be a string, max 256 chars - Must match [a-zA-Z0-9][a-zA-Z0-9_.:\-/@+ ]* (covers all known provider model ID formats while rejecting injection attempts) System-configured endpoints still get full model list validation as before. * 🧪 test: Add regression tests for filterAuthorizedTools configServers and validateModel filterAuthorizedTools: - Add test verifying configServers is passed to getAllServerConfigs and config-override server tools are allowed through - Guard resolveConfigServers in createAgentHandler to only run when MCP tools are present (skip for tool-free agent creates) validateModel (12 new tests): - Format validation: missing model, non-string, length overflow, leading special char, script injection, standard model ID acceptance - userProvide early-return: next() called immediately, getModelsConfig not invoked (regression guard for the exact bug this fixes) - System endpoint list validation: reject unknown model, accept known model, handle null/missing models config Also fix unnecessary backslash escape in MODEL_PATTERN regex. * 🧹 fix: Remove space from MODEL_PATTERN, trim input, clean up nits - Remove space character from MODEL_PATTERN regex — no real model ID uses spaces; prevents spurious violation logs from whitespace artifacts - Add model.trim() before validation to handle accidental whitespace - Remove redundant filterUniquePlugins call on already-deduplicated output - Add comment documenting intentional whitelist+blacklist composition - Add getUserKeyValues.mockReset() in loadConfigModels.spec beforeEach - Remove narrating JSDoc from getModelsConfig one-liner - Add 2 tests: trim whitespace handling, reject spaces in model ID * fix: Match startup tool loader semantics — includedTools takes precedence over filteredTools The startup tool loader (loadAndFormatTools) explicitly ignores filteredTools when includedTools is set, with a warning log. The PluginController was applying both independently, creating inconsistent behavior where the same config produced different results at startup vs plugin listing time. Restored mutually exclusive semantics: when includedTools is non-empty, filteredTools is not evaluated. * 🧹 chore: Simplify validateModel flow, note auth requirement on endpoints route - Separate missing-model from invalid-model checks cleanly: type+presence guard first, then trim+format guard (reviewer NIT) - Add route comment noting auth is required for role/tenant scoping * fix: Write trimmed model back to req.body.model for downstream consumers	2026-03-30 16:49:48 -04:00
Dustin Healy	a4a17ac771	⛩️ feat: Admin Grants API Endpoints (#12438) ... * feat: add System Grants handler factory with tests Handler factory with 4 endpoints: getEffectiveCapabilities (expanded capability set for authenticated user), getPrincipalGrants (list grants for a specific principal), assignGrant, and revokeGrant. Write ops dynamically check MANAGE_ROLES/GROUPS/USERS based on target principal type. 31 unit tests covering happy paths, validation, 403, and errors. * feat: wire System Grants REST routes Mount /api/admin/grants with requireJwtAuth + ACCESS_ADMIN gate. Add barrel export for createAdminGrantsHandlers and AdminGrantsDeps. * fix: cascade grant cleanup on role deletion Add deleteGrantsForPrincipal to AdminRolesDeps and call it in deleteRoleHandler via Promise.allSettled after successful deletion, matching the groups cleanup pattern. 3 tests added for cleanup call, skip on 404, and resilience to cleanup failure. * fix: simplify cascade grant cleanup on role deletion Replace Promise.allSettled wrapper with a direct try/catch for the single deleteGrantsForPrincipal call. * fix: harden grant handlers with auth, validation, types, and RESTful revoke - Add per-handler auth checks (401) and granular capability gates (READ_* for getPrincipalGrants, possession check for assignGrant) - Extract validatePrincipal helper; rewrite validateGrantBody to use direct type checks instead of unsafe `as string` casts - Align DI types with data layer (ResolvedPrincipal.principalType widened to string, getUserPrincipals role made optional) - Switch revoke route from DELETE body to RESTful URL params - Return 201 for assignGrant to match roles/groups create convention - Handle null grantCapability return with 500 - Add comprehensive test coverage for new auth/validation paths * fix: deduplicate ResolvedPrincipal, typed body, defensive auth checks - Remove duplicate ResolvedPrincipal from capabilities.ts; import the canonical export from grants.ts - Replace Record<string, unknown> with explicit GrantRequestBody interface - Add defensive 403 when READ_CAPABILITY_BY_TYPE lookup misses - Document revoke asymmetry (no possession check) with JSDoc - Use _id only in resolveUser (avoid Mongoose virtual reliance) - Improve null-grant error message - Complete logger mock in tests * refactor: move ResolvedPrincipal to shared types to fix circular dep Extract ResolvedPrincipal from admin/grants.ts to types/principal.ts so middleware/capabilities.ts imports from shared types rather than depending upward on the admin handler layer. * chore: remove dead re-export, align logger mocks across admin tests - Remove unused ResolvedPrincipal re-export from grants.ts (canonical source is types/principal.ts) - Align logger mocks in roles.spec.ts and groups.spec.ts to include all log levels (error, warn, info, debug) matching grants.spec.ts * fix: cascade Config and AclEntry cleanup on role deletion Add deleteConfig and deleteAclEntries to role deletion cascade, matching the group deletion pattern. Previously only grants were cleaned up, leaving orphaned config overrides and ACL entries. * perf: single-query batch for getEffectiveCapabilities Add getCapabilitiesForPrincipals (plural) to the data layer — a single $or query across all principals instead of N+1 parallel queries. Wire it into the grants handler so getEffectiveCapabilities hits the DB once regardless of how many principals the user has. * fix: defer SystemCapabilities access to factory call time Move all SystemCapabilities usage (VALID_CAPABILITIES, MANAGE_CAPABILITY_BY_TYPE, READ_CAPABILITY_BY_TYPE) inside the createAdminGrantsHandlers factory. External test suites that mock @librechat/data-schemas without providing SystemCapabilities crashed at import time when grants.ts was loaded transitively. * test: add data-layer and handler test coverage for review findings - Add 6 mongodb-memory-server tests for getCapabilitiesForPrincipals: multi-principal batch, empty array, filtering, tenant scoping - Add handler test: all principals filtered (only PUBLIC) - Add handler test: granting an implied capability succeeds - Add handler test: all cascade cleanup operations fail simultaneously - Document platform-scope-only tenantId behavior in JSDoc * fix: resolveUser fallback to user.id, early-return empty principals - Match capabilities middleware pattern: _id?.toString() ?? user.id to handle JWT-deserialized users without Mongoose _id - Move empty-array guard before principals.map() to skip unnecessary normalizePrincipalId calls - Add comment explaining VALID_PRINCIPAL_TYPES module-scope asymmetry * refactor: derive VALID_PRINCIPAL_TYPES from capability maps Make MANAGE_CAPABILITY_BY_TYPE and READ_CAPABILITY_BY_TYPE non-Partial Records over a shared GrantPrincipalType union, then derive VALID_PRINCIPAL_TYPES from the map keys. This makes divergence between the three data structures structurally impossible. * feat: add GET /api/admin/grants list-all-grants endpoint Add listAllGrants data-layer method and handler so the admin panel can fetch all grants in a single request instead of fanning out N+M calls per role and group. Response is filtered to only include grants for principal types the caller has read access to. * fix: update principalType to use GrantPrincipalType for consistency in grants handling - Refactor principalType in createAdminGrantsHandlers to use GrantPrincipalType instead of PrincipalType for better type accuracy. - Ensure type consistency across the grants handling logic in the API. * fix: address admin grants review findings — tenantId propagation, capability validation, pagination, and test coverage Propagate tenantId through all grant operations for multi-tenancy support. Extract isValidCapability to accept full SystemCapability union (base, section, assign) and reuse it in both Mongoose schema validation and handler input checks. Replace listAllGrants with paginated listGrants + countGrants. Filter PUBLIC principals from getCapabilitiesForPrincipals queries. Export getCachedPrincipals from ALS store for fast-path principal resolution. Move DELETE capability param to query string to avoid colon-in-URL issues. Remove dead code and add comprehensive handler and data-layer test coverage. * refactor: harden admin grants — FilterQuery types, auth-first ordering, DELETE path param, isValidCapability tests Replace Record<string, unknown> with FilterQuery<ISystemGrant> across all data-layer query filters. Refactor buildTenantFilter to a pure tenantCondition function that returns a composable FilterQuery fragment, eliminating the $or collision between tenant and principal queries. Move auth check before input validation in getPrincipalGrantsHandler, assignGrantHandler, and revokeGrantHandler to avoid leaking valid type names to unauthenticated callers. Switch DELETE route from query param back to path param (/:capability) with encodeURIComponent per project conventions. Add compound index for listGrants sort. Type VALID_PRINCIPAL_TYPES as Set<GrantPrincipalType>. Remove unused GetCachedPrincipalsFn type export. Add dedicated isValidCapability unit tests and revokeGrant idempotency test. * refactor: batch capability checks in listGrantsHandler via getHeldCapabilities Replace 3 parallel hasCapabilityForPrincipals DB calls with a single getHeldCapabilities query that returns the subset of capabilities any principal holds. Also: defensive limit(0) clamp, parallelized assignGrant auth checks, principalId type-vs-required error split, tenantCondition hoisted to factory top, JSDoc on cascade deps, DELETE route encoding note. * fix: normalize principalId and filter undefined in getHeldCapabilities Add normalizePrincipalId + null guard to getHeldCapabilities, matching the contract of getCapabilitiesForPrincipals. Simplify allCaps build with flatMap, add no-tenantId cross-check and undefined-principalId test cases. * refactor: use concrete types in GrantRequestBody, rename encoding test Replace unknown fields with explicit string types in GrantRequestBody, matching the established pattern in roles/groups/config handlers. Rename misleading 'encoded' test to 'with colons' since Express auto-decodes req.params. * fix: support hierarchical parent capabilities in possession checks hasCapabilityForPrincipals and getHeldCapabilities now resolve parent base capabilities for section/assignment grants. An admin holding manage:configs can now grant manage:configs:<section> and transitively read:configs:<section>. Fixes anti-escalation 403 blocking config capability delegation. * perf: use getHeldCapabilities in assignGrant to halve DB round-trips assignGrantHandler was making two parallel hasCapabilityForPrincipals calls to check manage + capability possession. getHeldCapabilities was introduced in this PR specifically for this pattern. Replace with a single batched call. Update corresponding spec assertions. * fix: validate role existence before granting capabilities Grants for non-existent role names were silently persisted, creating orphaned grants that could surprise-activate if a role with that name was later created. Add optional checkRoleExists dep to assignGrant and wire it to getRoleByName in the route file. * refactor: tighten principalType typing and use grantCapability in tests Narrow getCapabilitiesForPrincipals parameter from string to PrincipalType, removing the redundant cast. Replace direct SystemGrant.create() calls in getCapabilitiesForPrincipals tests with methods.grantCapability() to honor the schema's normalization invariant. Add getHeldCapabilities extended capability tests. * test: rename misleading cascade cleanup test name The test only injects failure into deleteGrantsForPrincipal, not all cascade operations. Rename from 'cascade cleanup fails' to 'grant cleanup fails' to match the actual scope. * fix: reorder role check after permission guard, add tenantId to index Move checkRoleExists after the getHeldCapabilities permission check so that a sub-MANAGE_ROLES admin cannot probe role name existence via 400 vs 403 response codes. Add tenantId to the { principalType, capability } index so listGrants queries in multi-tenant deployments can use a covering index instead of post-scanning for tenant condition. Add missing test for checkRoleExists throwing. * fix: scope deleteGrantsForPrincipal to tenant on role deletion deleteGrantsForPrincipal previously filtered only on principalType + principalId, deleting grants across all tenants. Since the role schema supports multi-tenancy (compound unique index on name + tenantId), two tenants can share a role name like 'editor'. Deleting that role in one tenant would wipe grants for identically-named roles in other tenants. Add optional tenantId parameter to deleteGrantsForPrincipal. When provided, scopes the delete to that tenant plus platform-level grants. Propagate req.user.tenantId through the role deletion cascade. * fix: scope grant cleanup to tenant on group deletion Same cross-tenant gap as the role deletion path: deleteGroupHandler called deleteGrantsForPrincipal without tenantId, so deleting a group would wipe its grants across all tenants. Extract req.user.tenantId and pass it through. * test: add HTTP integration test for admin grants routes Supertest-based test with real MongoMemoryServer exercising the full Express wiring: route registration, injected auth middleware, handler DI deps, and real DB round-trips. Covers GET /, GET /effective, POST / + DELETE / lifecycle, role existence validation, and 401 for unauthenticated callers. Also documents the expandImplications scope: the /effective endpoint returns base-level capabilities only; section-level resolution is handled at authorization check time by getParentCapabilities. * fix: use exact tenant match in deleteGrantsForPrincipal, normalize principalId, harden API CRITICAL: deleteGrantsForPrincipal was using tenantCondition (a read-query helper) for deleteMany, which includes the { tenantId: { $exists: false } } arm. This silently destroyed platform-level grants when a tenant-scoped role/group deletion occurred. Replace with exact { tenantId } match for deletes so platform-level grants survive tenant-scoped cascade cleanup. Refactor deleteGrantsForPrincipal signature from fragile positional overload (sessionOrTenantId union + maybeSession) to a clean options object: { tenantId?, session? }. Update all callers and test assertions. Add normalizePrincipalId to hasCapabilityForPrincipals to match the pattern already used by getHeldCapabilities — prevents string/ObjectId type mismatch on USER/GROUP principal queries. Also: export GrantPrincipalType from barrel, add upper-bound cap to listGrants, document GROUP/USER existence check trade-off, add integration tests for tenant-isolation property of deleteGrantsForPrincipal. * fix: forward tenantId to getUserPrincipals in resolvePrincipals resolvePrincipals had tenantId available from the caller but only forwarded it to getCachedPrincipals (cache lookup). The DB fallback via getUserPrincipals omitted it. While the Group schema's applyTenantIsolation Mongoose plugin handles scoping via AsyncLocalStorage in HTTP request context, explicitly passing tenantId makes the contract visible and prevents silent cross-tenant group resolution if called outside request context. * fix: remove unused import and add assertion to 401 integration test Remove unused SystemCapabilities import flagged by ESLint. Add explicit body assertion to the 401 test so it has a jest expect() call. * chore: hoist grant limit constants to scope, remove dead isolateModules Move GRANTS_DEFAULT_LIMIT / GRANTS_MAX_LIMIT from inside listGrants function body to createSystemGrantMethods scope so they are evaluated once at module load. Remove dead jest.isolateModules + jest.doMock block in integration test — the ~/models mock was never exercised since handlers are built with explicit DI deps. --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-03-30 16:49:23 -04:00
Danny Avila	0d94881c2d	🧹 refactor: Tighten Config Schema Typing and Remove Deprecated Fields (#12452) ... * refactor: Remove deprecated and unused fields from endpoint schemas - Remove summarize, summaryModel from endpointSchema and azureEndpointSchema - Remove plugins from azureEndpointSchema - Remove customOrder from endpointSchema and azureEndpointSchema - Remove baseURL from all and agents endpoint schemas - Type paramDefinitions with full SettingDefinition-based schema - Clean up summarize/summaryModel references in initialize.ts and config.spec.ts * refactor: Improve MCP transport schema typing - Add defaults to transport type discriminators (stdio, websocket, sse) - Type stderr field as IOType union instead of z.any() * refactor: Add narrowed preset schema for model specs - Create tModelSpecPresetSchema omitting system/DB/deprecated fields - Update tModelSpecSchema to use the narrowed preset schema * test: Add explicit type field to MCP test fixtures Add transport type discriminator to test objects that construct MCPOptions/ParsedServerConfig directly, required after type field changed from optional to default in schema definitions. * chore: Bump librechat-data-provider to 0.8.404 * refactor: Tighten z.record(z.any()) fields to precise value types - Type headers fields as z.record(z.string()) in endpoint, assistant, and azure schemas - Type addParams as z.record(z.union([z.string(), z.number(), z.boolean(), z.null()])) - Type azure additionalHeaders as z.record(z.string()) - Type memory model_parameters as z.record(z.union([z.string(), z.number(), z.boolean()])) - Type firecrawl changeTrackingOptions.schema as z.record(z.string()) * refactor: Type supportedMimeTypes schema as z.array(z.string()) Replace z.array(z.any()).refine() with z.array(z.string()) since config input is always strings that get converted to RegExp via convertStringsToRegex() after parsing. Destructure supportedMimeTypes from spreads to avoid string[]/RegExp[] type mismatch. * refactor: Tighten enum, role, and numeric constraint schemas - Type engineSTT as enum ['openai', 'azureOpenAI'] - Type engineTTS as enum ['openai', 'azureOpenAI', 'elevenlabs', 'localai'] - Constrain playbackRate to 0.25–4 range - Type titleMessageRole as enum ['system', 'user', 'assistant'] - Add int().nonnegative() to MCP timeout and firecrawl timeout * chore: Bump librechat-data-provider to 0.8.405 * fix: Accept both string and RegExp in supportedMimeTypes schema The schema must accept both string[] (config input) and RegExp[] (post-merge runtime) since tests validate merged output against the schema. Use z.union([z.string(), z.instanceof(RegExp)]) to handle both. * refactor: Address review findings for schema tightening PR - Revert changeTrackingOptions.schema to z.record(z.unknown()) (JSON Schema is nested, not flat strings) - Remove dead contextStrategy code from BaseClient.js and cleanup.js - Extract paramDefinitionSchema to named exported constant - Add .int() constraint to columnSpan and columns - Apply consistent .int().nonnegative() to initTimeout, sseReadTimeout, scraperTimeout - Update stale stderr JSDoc to match actual accepted types - Add comprehensive tests for paramDefinitionSchema, tModelSpecPresetSchema, endpointSchema deprecated field stripping, and azureEndpointSchema * fix: Address second review pass findings - Revert supportedMimeTypesSchema to z.array(z.string()) and remove as string[] casts — fix tests to not validate merged RegExp[] output against the config input schema - Remove unused tModelSpecSchema import from test file - Consolidate duplicate '../src/schemas' imports - Add expiredAt coverage to tModelSpecPresetSchema test - Assert plugins is absent in azureEndpointSchema test - Add sync comments for engineSTT/engineTTS enum literals * refactor: Omit preset-management fields from tModelSpecPresetSchema Omit conversationId, presetId, title, defaultPreset, and order from the model spec preset schema — these are preset-management fields that don't belong in model spec configuration.	2026-03-29 01:10:57 -04:00
Danny Avila	f82d4300a4	🧹 chore: Remove Deprecated Gemini 2.0 Models & Fix Mistral-Large-3 Context Window (#12453) ... * chore: remove deprecated Gemini 2.0 models from default models list Remove gemini-2.0-flash-001 and gemini-2.0-flash-lite from the Google default models array, as they have been deprecated by Google. Closes #12444 * fix: add mistral-large-3 max context tokens (256k) Add mistral-large-3 with 255000 max context tokens to the mistralModels map. Without this entry, the model falls back to the generic mistral-large key (131k), causing context window errors when using tools with Azure AI Foundry deployments. Closes #12429 * test: add mistral-large-3 token resolution tests and fix key ordering Add test coverage for mistral-large-3 context token resolution, verifying exact match, suffixed variants, and longest-match precedence over the generic mistral-large key. Reorder the mistral-large-3 entry after mistral-large to follow the file's documented convention of listing newer models last for reverse-scan performance.	2026-03-28 23:44:58 -04:00
Danny Avila	fda1bfc3cc	🔬 ci: Add TypeScript Type Checks to Backend Workflow and Fix All Type Errors (#12451) ... * fix(data-schemas): resolve TypeScript strict type check errors in source files - Constrain ConfigSection to string keys via `string & keyof TCustomConfig` - Replace broken `z` import from data-provider with TCustomConfig derivation - Add `_id: Types.ObjectId` to IUser matching other Document interfaces - Add `federatedTokens` and `openidTokens` optional fields to IUser - Type mongoose model accessors as `Model<IRole>` and `Model<IUser>` - Widen `getPremiumRate` param to accept `number | null` - Widen `bulkWriteAclEntries` ops to untyped `AnyBulkWriteOperation[]` - Fix `getUserPrincipals` return type to use `PrincipalType` enum - Add non-null assertions for `connection.db` in migration files - Import DailyRotateFile constructor directly instead of relying on broken module augmentation across mismatched node_modules trees - Add winston-daily-rotate-file as devDependency for type resolution * fix(data-schemas): resolve TypeScript type errors in test files - Replace arbitrary test keys with valid TCustomConfig properties in config.spec - Use non-null assertions for permission objects in role.methods.spec - Replace `.SHARED_GLOBAL` access with `.not.toHaveProperty()` for legacy field - Add non-null assertions for balance, writeRate, readRate in spendTokens.spec - Update mock user _id to use ObjectId in user.test - Remove unused Schema import in tenantIndexes.spec * fix(api): resolve TypeScript strict type check errors across source and test files - Widen getUserPrincipals dep type in capabilities middleware - Fix federatedTokens type in createSafeUser return - Use proper mock req type for read-only properties in preAuthTenant.spec - Replace `as IUser` casts with ObjectId-typed mocks in openid/oidc specs - Use TokenExchangeMethodEnum values instead of string literals in MCP specs - Fix SessionStore type compatibility in sessionCache specs - Replace `catch (error: any)` with `(error as Error)` in redis specs - Remove invalid properties from test data in initialize and MCP specs - Add String.prototype.isWellFormed declaration for sanitizeTitle spec * fix(client): resolve TypeScript type errors in shared client components - Add default values for destructured bindings in OGDialogTemplate - Replace broken ExtendedFile import with inline type in FileIcon * ci: add TypeScript type-check job to backend review workflow Add a `typecheck` job that runs `tsc --noEmit` on all four TypeScript workspaces (data-provider, data-schemas, @librechat/api, @librechat/client) after the build step. Catches type errors that rollup builds may miss. * fix(data-schemas): add local type declaration for DailyRotateFile transport The `winston-daily-rotate-file` package ships a module augmentation for `winston/lib/winston/transports`, but it fails when winston and winston-daily-rotate-file resolve from different node_modules trees (which happens in this monorepo due to npm hoisting). Add a local `.d.ts` declaration that augments the same module path from within data-schemas' compilation unit, so `tsc --noEmit` passes while keeping the original runtime pattern (`new winston.transports.DailyRotateFile`). * fix: address code review findings from PR #12451 - Restore typed `AnyBulkWriteOperation<AclEntry>[]` on bulkWriteAclEntries, cast to untyped only at the tenantSafeBulkWrite call site (Finding 1) - Type `findUser` model accessor consistently with `findUsers` (Finding 2) - Replace inline `import('mongoose').ClientSession` with top-level import type - Use `toHaveLength` for spy assertions in playwright-expect spec file - Replace numbered Record casts with `.not.toHaveProperty()` in role.methods.spec for SHARED_GLOBAL assertions - Use per-test ObjectIds instead of shared testUserId in openid.spec - Replace inline `import()` type annotations with top-level SessionData import in sessionCache spec - Remove extraneous blank line in user.ts searchUsers * refactor: address remaining review findings (4–7) - Extract OIDCTokens interface in user.ts; deduplicate across IUser fields and oidc.ts FederatedTokens (Finding 4) - Move String.isWellFormed declaration from spec file to project-level src/types/es2024-string.d.ts (Finding 5) - Replace verbose `= undefined` defaults in OGDialogTemplate with null coalescing pattern (Finding 6) - Replace `Record<string, unknown>` TestConfig with named interface containing explicit test fields (Finding 7)	2026-03-28 21:06:39 -04:00
Danny Avila	877c2efc85	🏗️ feat: bulkWrite isolation, pre-auth context, strict-mode fixes (#12445) ... * fix: wrap seedDatabase() in runAsSystem() for strict tenant mode seedDatabase() was called without tenant context at startup, causing every Mongoose operation inside it to throw when TENANT_ISOLATION_STRICT=true. Wrapping in runAsSystem() gives it the SYSTEM_TENANT_ID sentinel so the isolation plugin skips filtering, matching the pattern already used for performStartupChecks and updateInterfacePermissions. * fix: chain tenantContextMiddleware in optionalJwtAuth optionalJwtAuth populated req.user but never established ALS tenant context, unlike requireJwtAuth which chains tenantContextMiddleware after successful auth. Authenticated users hitting routes with optionalJwtAuth (e.g. /api/banner) had no tenant isolation. * feat: tenant-safe bulkWrite wrapper and call-site migration Mongoose's bulkWrite() does not trigger schema-level middleware hooks, so the applyTenantIsolation plugin cannot intercept it. This adds a tenantSafeBulkWrite() utility that injects the current ALS tenant context into every operation's filter/document before delegating to native bulkWrite. Migrates all 8 runtime bulkWrite call sites: - agentCategory (seedCategories, ensureDefaultCategories) - conversation (bulkSaveConvos) - message (bulkSaveMessages) - file (batchUpdateFiles) - conversationTag (updateTagsForConversation, bulkIncrementTagCounts) - aclEntry (bulkWriteAclEntries) systemGrant.seedSystemGrants is intentionally not migrated — it uses explicit tenantId: { $exists: false } filters and is exempt from the isolation plugin. * feat: pre-auth tenant middleware and tenant-scoped config cache Adds preAuthTenantMiddleware that reads X-Tenant-Id from the request header and wraps downstream in tenantStorage ALS context. Wired onto /oauth, /api/auth, /api/config, and /api/share — unauthenticated routes that need tenant scoping before JWT auth runs. The /api/config cache key is now tenant-scoped (STARTUP_CONFIG:${tenantId}) so multi-tenant deployments serve the correct login page config per tenant. The middleware is intentionally minimal — no subdomain parsing, no OIDC claim extraction. The private fork's reverse proxy or auth gateway sets the header. * feat: accept optional tenantId in updateInterfacePermissions When tenantId is provided, the function re-enters inside tenantStorage.run({ tenantId }) so all downstream Mongoose queries target that tenant's roles instead of the system context. This lets the private fork's tenant provisioning flow call updateInterfacePermissions per-tenant after creating tenant-scoped ADMIN/USER roles. * fix: tenant-filter $lookup in getPromptGroup aggregation The $lookup stage in getPromptGroup() queried the prompts collection without tenant filtering. While the outer PromptGroup aggregate is protected by the tenantIsolation plugin's pre('aggregate') hook, $lookup runs as an internal MongoDB operation that bypasses Mongoose hooks entirely. Converts from simple field-based $lookup to pipeline-based $lookup with an explicit tenantId match when tenant context is active. * fix: replace field-level unique indexes with tenant-scoped compounds Field-level unique:true creates a globally-unique single-field index in MongoDB, which would cause insert failures across tenants sharing the same ID values. - agent.id: removed field-level unique, added { id, tenantId } compound - convo.conversationId: removed field-level unique (compound at line 50 already exists: { conversationId, user, tenantId }) - message.messageId: removed field-level unique (compound at line 165 already exists: { messageId, user, tenantId }) - preset.presetId: removed field-level unique, added { presetId, tenantId } compound * fix: scope MODELS_CONFIG, ENDPOINT_CONFIG, PLUGINS, TOOLS caches by tenant These caches store per-tenant configuration (available models, endpoint settings, plugin availability, tool definitions) but were using global cache keys. In multi-tenant mode, one tenant's cached config would be served to all tenants. Appends :${tenantId} to cache keys when tenant context is active. Falls back to the unscoped key when no tenant context exists (backward compatible for single-tenant OSS deployments). Covers all read, write, and delete sites: - ModelController.js: get/set MODELS_CONFIG - PluginController.js: get/set PLUGINS, get/set TOOLS - getEndpointsConfig.js: get/set/delete ENDPOINT_CONFIG - app.js: delete ENDPOINT_CONFIG (clearEndpointConfigCache) - mcp.js: delete TOOLS (updateMCPTools, mergeAppTools) - importers.js: get ENDPOINT_CONFIG * fix: add getTenantId to PluginController spec mock The data-schemas mock was missing getTenantId, causing all PluginController tests to throw when the controller calls getTenantId() for tenant-scoped cache keys. * fix: address review findings — migration, strict-mode, DRY, types Addresses all CRITICAL, MAJOR, and MINOR review findings: F1 (CRITICAL): Add agents, conversations, messages, presets to SUPERSEDED_INDEXES in tenantIndexes.ts so dropSupersededTenantIndexes() drops the old single-field unique indexes that block multi-tenant inserts. F2 (CRITICAL): Unknown bulkWrite op types now throw in strict mode instead of silently passing through without tenant injection. F3 (MAJOR): Replace wildcard export with named export for tenantSafeBulkWrite, hiding _resetBulkWriteStrictCache from the public package API. F5 (MAJOR): Restore AnyBulkWriteOperation<IAclEntry>[] typing on bulkWriteAclEntries — the unparameterized wrapper accepts parameterized ops as a subtype. F7 (MAJOR): Fix config.js tenant precedence — JWT-derived req.user.tenantId now takes priority over the X-Tenant-Id header for authenticated requests. F8 (MINOR): Extract scopedCacheKey() helper into tenantContext.ts and replace all 11 inline occurrences across 7 files. F9 (MINOR): Use simple localField/foreignField $lookup for the non-tenant getPromptGroup path (more efficient index seeks). F12 (NIT): Remove redundant BulkOp type alias. F13 (NIT): Remove debug log that leaked raw tenantId. * fix: add new superseded indexes to tenantIndexes test fixture The test creates old indexes to verify the migration drops them. Missing fixture entries for agents.id_1, conversations.conversationId_1, messages.messageId_1, and presets.presetId_1 caused the count assertion to fail (expected 22, got 18). * fix: restore logger.warn for unknown bulk op types in non-strict mode * fix: block SYSTEM_TENANT_ID sentinel from external header input CRITICAL: preAuthTenantMiddleware accepted any string as X-Tenant-Id, including '__SYSTEM__'. The tenantIsolation plugin treats SYSTEM_TENANT_ID as an explicit bypass — skipping ALL query filters. A client sending X-Tenant-Id: __SYSTEM__ to pre-auth routes (/api/share, /api/config, /api/auth, /oauth) would execute Mongoose operations without tenant isolation. Fixes: - preAuthTenantMiddleware rejects SYSTEM_TENANT_ID in header - scopedCacheKey returns the base key (not key:__SYSTEM__) in system context, preventing stale cache entries during runAsSystem() - updateInterfacePermissions guards tenantId against SYSTEM_TENANT_ID - $lookup pipeline separates $expr join from constant tenantId match for better index utilization - Regression test for sentinel rejection in preAuthTenant.spec.ts - Remove redundant getTenantId() call in config.js * test: add missing deleteMany/replaceOne coverage, fix vacuous ALS assertions bulkWrite spec: - deleteMany: verifies tenant-scoped deletion leaves other tenants untouched - replaceOne: verifies tenantId injected into both filter and replacement - replaceOne overwrite: verifies a conflicting tenantId in the replacement document is overwritten by the ALS tenant (defense-in-depth) - empty ops array: verifies graceful handling preAuthTenant spec: - All negative-case tests now use the capturedNext pattern to verify getTenantId() inside the middleware's execution context, not the test runner's outer frame (which was always undefined regardless) * feat: tenant-isolate MESSAGES cache, FLOWS cache, and GenerationJobManager MESSAGES cache (streamAudio.js): - Cache key now uses scopedCacheKey(messageId) to prefix with tenantId, preventing cross-tenant message content reads during TTS streaming. FLOWS cache (FlowStateManager): - getFlowKey() now generates ${type}:${tenantId}:${flowId} when tenant context is active, isolating OAuth flow state per tenant. GenerationJobManager: - tenantId added to SerializableJobData and GenerationJobMetadata - createJob() captures the current ALS tenant context (excluding SYSTEM_TENANT_ID) and stores it in job metadata - SSE subscription endpoint validates job.metadata.tenantId matches req.user.tenantId, blocking cross-tenant stream access - Both InMemoryJobStore and RedisJobStore updated to accept tenantId * fix: add getTenantId and SYSTEM_TENANT_ID to MCP OAuth test mocks FlowStateManager.getFlowKey() now calls getTenantId() for tenant-scoped flow keys. The 4 MCP OAuth test files mock @librechat/data-schemas without these exports, causing TypeError at runtime. * fix: correct import ordering per AGENTS.md conventions Package imports sorted shortest to longest line length, local imports sorted longest to shortest — fixes ordering violations introduced by our new imports across 8 files. * fix: deserialize tenantId in RedisJobStore — cross-tenant SSE guard was no-op in Redis mode serializeJob() writes tenantId to the Redis hash via Object.entries, but deserializeJob() manually enumerates fields and omitted tenantId. Every getJob() from Redis returned tenantId: undefined, causing the SSE route's cross-tenant guard to short-circuit (undefined && ... → false). * test: SSE tenant guard, FlowStateManager key consistency, ALS scope docs SSE stream tenant tests (streamTenant.spec.js): - Cross-tenant user accessing another tenant's stream → 403 - Same-tenant user accessing own stream → allowed - OSS mode (no tenantId on job) → tenant check skipped FlowStateManager tenant tests (manager.tenant.spec.ts): - completeFlow finds flow created under same tenant context - completeFlow does NOT find flow under different tenant context - Unscoped flows are separate from tenant-scoped flows Documentation: - JSDoc on getFlowKey documenting ALS context consistency requirement - Comment on streamAudio.js scopedCacheKey capture site * fix: SSE stream tests hang on success path, remove internal fork references The success-path tests entered the SSE streaming code which never closes, causing timeout. Mock subscribe() to end the response immediately. Restructured assertions to verify non-403/non-404. Removed "private fork" and "OSS" references from code and test descriptions — replaced with "deployment layer", "multi-tenant deployments", and "single-tenant mode". * fix: address review findings — test rigor, tenant ID validation, docs F1: SSE stream tests now mock subscribe() with correct signature (streamId, writeEvent, onDone, onError) and assert 200 status, verifying the tenant guard actually allows through same-tenant users. F2: completeFlow logs the attempted key and ALS tenantId when flow is not found, so reverse proxy misconfiguration (missing X-Tenant-Id on OAuth callback) produces an actionable warning. F3/F10: preAuthTenantMiddleware validates tenant ID format — rejects colons, special characters, and values exceeding 128 chars. Trims whitespace. Prevents cache key collisions via crafted headers. F4: Documented cache invalidation scope limitation in clearEndpointConfigCache — only the calling tenant's key is cleared; other tenants expire via TTL. F7: getFlowKey JSDoc now lists all 8 methods requiring consistent ALS context. F8: Added dedicated scopedCacheKey unit tests — base key without context, base key in system context, scoped key with tenant, no ALS leakage across scope boundaries. * fix: revert flow key tenant scoping, fix SSE test timing FlowStateManager: Reverts tenant-scoped flow keys. OAuth callbacks arrive without tenant ALS context (provider redirects don't carry X-Tenant-Id), so completeFlow/failFlow would never find flows created under tenant context. Flow IDs are random UUIDs with no collision risk, and flow data is ephemeral (TTL-bounded). SSE tests: Use process.nextTick for onDone callback so Express response headers are flushed before res.write/res.end are called. * fix: restore getTenantId import for completeFlow diagnostic log * fix: correct completeFlow warning message, add missing flow test The warning referenced X-Tenant-Id header consistency which was only relevant when flow keys were tenant-scoped (since reverted). Updated to list actual causes: TTL expiry, missing flow, or routing to a different instance without shared Keyv storage. Removed the getTenantId() call and import — no longer needed since flow keys are unscoped. Added test for the !flowState branch in completeFlow — verifies return false and logger.warn on nonexistent flow ID. * fix: add explicit return type to recursive updateInterfacePermissions The recursive call (tenantId branch calls itself without tenantId) causes TypeScript to infer circular return type 'any'. Adding explicit Promise<void> satisfies the rollup typescript plugin. * fix: update MCPOAuthRaceCondition test to match new completeFlow warning * fix: clearEndpointConfigCache deletes both scoped and unscoped keys Unauthenticated /api/endpoints requests populate the unscoped ENDPOINT_CONFIG key. Admin config mutations clear only the tenant-scoped key, leaving the unscoped entry stale indefinitely. Now deletes both when in tenant context. * fix: tenant guard on abort/status endpoints, warn logs, test coverage F1: Add tenant guard to /chat/status/:conversationId and /chat/abort matching the existing guard on /chat/stream/:streamId. The status endpoint exposes aggregatedContent (AI response text) which requires tenant-level access control. F2: preAuthTenantMiddleware now logs warn for rejected __SYSTEM__ sentinel and malformed tenant IDs, providing observability for bypass probing attempts. F3: Abort fallback path (getActiveJobIdsForUser) now has tenant check after resolving the job. F4: Test for strict mode + SYSTEM_TENANT_ID — verifies runAsSystem bypasses tenantSafeBulkWrite without throwing in strict mode. F5: Test for job with tenantId + user without tenantId → 403. F10: Regex uses idiomatic hyphen-at-start form. F11: Test descriptions changed from "rejects" to "ignores" since middleware calls next() (not 4xx). Also fixes MCPOAuthRaceCondition test assertion to match updated completeFlow warning message. * fix: test coverage for logger.warn, status/abort guards, consistency A: preAuthTenant spec now mocks logger and asserts warn calls for __SYSTEM__ sentinel, malformed characters, and oversized headers. B: streamTenant spec expanded with status and abort endpoint tests — cross-tenant status returns 403, same-tenant returns 200 with body, cross-tenant abort returns 403. C: Abort endpoint uses req.user.tenantId (not req.user?.tenantId) matching stream/status pattern — requireJwtAuth guarantees req.user. D: Malformed header warning now includes ip in log metadata, matching the sentinel warning for consistent SOC correlation. * fix: assert ip field in malformed header warn tests * fix: parallelize cache deletes, document tenant guard, fix import order - clearEndpointConfigCache uses Promise.all for independent cache deletes instead of sequential awaits - SSE stream tenant guard has inline comment explaining backward-compat behavior for untenanted legacy jobs - conversation.ts local imports reordered longest-to-shortest per AGENTS.md * fix: tenant-qualify userJobs keys, document tenant guard backward-compat Job store userJobs keys now include tenantId when available: - Redis: stream:user:{tenantId:userId}:jobs (falls back to stream:user:{userId}:jobs when no tenant) - InMemory: composite key tenantId:userId in userJobMap getActiveJobIdsByUser/getActiveJobIdsForUser accept optional tenantId parameter, threaded through from req.user.tenantId at all call sites (/chat/active and /chat/abort fallback). Added inline comments on all three SSE tenant guards explaining the backward-compat design: untenanted legacy jobs remain accessible when the userId check passes. * fix: parallelize cache deletes, document tenant guard, fix import order Fix InMemoryJobStore.getActiveJobIdsByUser empty-set cleanup to use the tenant-qualified userKey instead of bare userId — prevents orphaned empty Sets accumulating in userJobMap for multi-tenant users. Document cross-tenant staleness in clearEndpointConfigCache JSDoc — other tenants' scoped keys expire via TTL, not active invalidation. * fix: cleanup userJobMap leak, startup warning, DRY tenant guard, docs F1: InMemoryJobStore.cleanup() now removes entries from userJobMap before calling deleteJob, preventing orphaned empty Sets from accumulating with tenant-qualified composite keys. F2: Startup warning when TENANT_ISOLATION_STRICT is active — reminds operators to configure reverse proxy to control X-Tenant-Id header. F3: mergeAppTools JSDoc documents that tenant-scoped TOOLS keys are not actively invalidated (matching clearEndpointConfigCache pattern). F5: Abort handler getActiveJobIdsForUser call uses req.user.tenantId (not req.user?.tenantId) — consistent with stream/status handlers. F6: updateInterfacePermissions JSDoc clarifies SYSTEM_TENANT_ID behavior — falls through to caller's ALS context. F7: Extracted hasTenantMismatch() helper, replacing three identical inline tenant guard blocks across stream/status/abort endpoints. F9: scopedCacheKey JSDoc documents both passthrough cases (no context and SYSTEM_TENANT_ID context). * fix: clean userJobMap in evictOldest — same leak as cleanup()	2026-03-28 16:43:50 -04:00
Danny Avila	935288f841	🏗️ feat: 3-Tier MCP Server Architecture with Config-Source Lazy Init (#12435) ... * feat: add MCPServerSource type, tenantMcpPolicy schema, and source-based dbSourced wiring - Add `tenantMcpPolicy` to `mcpSettings` in YAML config schema with `enabled`, `maxServersPerTenant`, `allowedTransports`, and `allowedDomains` - Add `MCPServerSource` type ('yaml' | 'config' | 'user') and `source` field to `ParsedServerConfig` - Change `dbSourced` determination from `!!config.dbId` to `config.source === 'user'` across MCPManager, ConnectionsRepository, UserConnectionManager, and MCPServerInspector - Set `source: 'user'` on all DB-sourced servers in ServerConfigsDB * feat: three-layer MCPServersRegistry with config cache and lazy init - Add `configCacheRepo` as third repository layer between YAML cache and DB for admin-defined config-source MCP servers - Implement `ensureConfigServers()` that identifies config-override servers from resolved `getAppConfig()` mcpConfig, lazily inspects them, and caches parsed configs with `source: 'config'` - Add `lazyInitConfigServer()` with timeout, stub-on-failure, and concurrent-init deduplication via `pendingConfigInits` map - Extend `getAllServerConfigs()` with optional `configServers` param for three-way merge: YAML → Config → User - Add `getServerConfig()` lookup through config cache layer - Add `invalidateConfigCache()` for clearing config-source inspection results on admin config mutations - Tag `source: 'yaml'` on CACHE-stored servers and `source: 'user'` on DB-stored servers in `addServer()` and `addServerStub()` * feat: wire tenant context into MCP controllers, services, and cache invalidation - Resolve config-source servers via `getAppConfig({ role, tenantId })` in `getMCPTools()` and `getMCPServersList()` controllers - Pass `ensureConfigServers()` results through `getAllServerConfigs()` for three-way merge of YAML + Config + User servers - Add tenant/role context to `getMCPSetupData()` and connection status routes via `getTenantId()` from ALS - Add `clearMcpConfigCache()` to `invalidateConfigCaches()` so admin config mutations trigger re-inspection of config-source MCP servers * feat: enforce tenantMcpPolicy on admin config mcpServers mutations - Add `validateMcpServerPolicy()` helper that checks mcpServers against operator-defined `tenantMcpPolicy` (enabled, maxServersPerTenant, allowedTransports, allowedDomains) - Wire validation into `upsertConfigOverrides` and `patchConfigField` handlers — rejects with 403 when policy is violated - Infer transport type from config shape (command → stdio, url protocol → websocket/sse, type field → streamable-http) - Validate server domains against policy allowlist when configured * revert: remove tenantMcpPolicy schema and enforcement The existing admin config CRUD routes already provide the mechanism for granular MCP server prepopulation (groups, roles, users). The tenantMcpPolicy gating adds unnecessary complexity that can be revisited if needed in the future. - Remove tenantMcpPolicy from mcpSettings Zod schema - Remove validateMcpServerPolicy helper and TenantMcpPolicy interface - Remove policy enforcement from upsertConfigOverrides and patchConfigField handlers * test: update test assertions for source field and config-server wiring - Use objectContaining in MCPServersRegistry reset test to account for new source: 'yaml' field on CACHE-stored configs - Add getTenantId and ensureConfigServers mocks to MCP route tests - Add getAppConfig mock to route test Config service mock - Update getMCPSetupData assertion to expect second options argument - Update getAllServerConfigs assertions for new configServers parameter * fix: disconnect active connections when config-source servers are evicted When admin config overrides change and config-source MCP servers are removed, the invalidation now proactively disconnects active connections for evicted servers instead of leaving them lingering until timeout. - Return evicted server names from invalidateConfigCache() - Disconnect app-level connections for evicted servers in clearMcpConfigCache() via MCPManager.appConnections.disconnect() * fix: address code review findings (CRITICAL, MAJOR, MINOR) CRITICAL fixes: - Scope configCacheRepo keys by config content hash to prevent cross-tenant cache poisoning when two tenants define the same server name with different configurations - Change dbSourced checks from `source === 'user'` to `source !== 'yaml' && source !== 'config'` so undefined source (pre-upgrade cached configs) fails closed to restricted mode MAJOR fixes: - Derive OAuth servers from already-computed mcpConfig instead of calling getOAuthServers() separately — config-source OAuth servers are now properly detected - Add parseInt radix (10) and NaN guard with fallback to 30_000 for CONFIG_SERVER_INIT_TIMEOUT_MS - Add CONFIG_CACHE_NAMESPACE to aggregate-key branch in ServerConfigsCacheFactory to avoid SCAN-based Redis stalls - Remove `if (role || tenantId)` guard in getMCPSetupData — config servers now always resolve regardless of tenant context MINOR fixes: - Extract resolveAllMcpConfigs() helper in mcp controller to eliminate 3x copy-pasted config resolution boilerplate - Distinguish "not initialized" from real errors in clearMcpConfigCache — log actual failures instead of swallowing - Remove narrative inline comments per style guide - Remove dead try/catch inside Promise.allSettled in ensureConfigServers (inner method never throws) - Memoize YAML server names to avoid repeated cacheConfigsRepo.getAll() calls per request Test updates: - Add ensureConfigServers mock to registry test fixtures - Update getMCPSetupData assertions for inline OAuth derivation * fix: address code review findings (CRITICAL, MAJOR, MINOR) CRITICAL fixes: - Break circular dependency: move CONFIG_CACHE_NAMESPACE from MCPServersRegistry to ServerConfigsCacheFactory - Fix dbSourced fail-closed: use source field when present, fall back to legacy dbId check when absent (backward-compatible with pre-upgrade cached configs that lack source field) MAJOR fixes: - Add CONFIG_CACHE_NAMESPACE to aggregate-key set in ServerConfigsCacheFactory to avoid SCAN-based Redis stalls - Add comprehensive test suite (ensureConfigServers.test.ts, 18 tests) covering lazy init, stub-on-failure, cross-tenant isolation via config hash keys, concurrent deduplication, merge order, and cache invalidation MINOR fixes: - Update MCPServerInspector test assertion for dbSourced change * fix: restore getServerConfig lookup for config-source servers (NEW-1) Add configNameToKey map that indexes server name → hash-based cache key for O(1) lookup by name in getServerConfig. This restores the config cache layer that was dropped when hash-based keys were introduced. Without this fix, config-source servers appeared in tool listings (via getAllServerConfigs) but getServerConfig returned undefined, breaking all connection and tool call paths. - Populate configNameToKey in ensureSingleConfigServer - Clear configNameToKey in invalidateConfigCache and reset - Clear stale read-through cache entries after lazy init - Remove dead code in invalidateConfigCache (config.title, key parsing) - Add getServerConfig tests for config-source server lookup * fix: eliminate configNameToKey race via caller-provided configServers param Replace the process-global configNameToKey map (last-writer-wins under concurrent multi-tenant load) with a configServers parameter on getServerConfig. Callers pass the pre-resolved config servers map directly — no shared mutable state, no cross-tenant race. - Add optional configServers param to getServerConfig; when provided, returns matching config directly without any global lookup - Remove configNameToKey map entirely (was the source of the race) - Extract server names from cache keys via lastIndexOf in invalidateConfigCache (safe for names containing colons) - Use mcpConfig[serverName] directly in getMCPTools instead of a redundant getServerConfig call - Add cross-tenant isolation test for getServerConfig * fix: populate read-through cache after config server lazy init After lazyInitConfigServer succeeds, write the parsed config to readThroughCache keyed by serverName so that getServerConfig calls from ConnectionsRepository, UserConnectionManager, and MCPManager.callTool find the config without needing configServers. Without this, config-source servers appeared in tool listings but every connection attempt and tool call returned undefined. * fix: user-scoped getServerConfig fallback to server-only cache key When getServerConfig is called with a userId (e.g., from callTool or UserConnectionManager), the cache key is serverName::userId. Config-source servers are cached under the server-only key (no userId). Add a fallback so user-scoped lookups find config-source servers in the read-through cache. * fix: configCacheRepo fallback, isUserSourced DRY, cross-process race CRITICAL: Add findInConfigCache fallback in getServerConfig so config-source servers remain reachable after readThroughCache TTL expires (5s). Without this, every tool call after 5s returned undefined for config-source servers. MAJOR: Extract isUserSourced() helper to mcp/utils.ts and replace all 5 inline dbSourced ternary expressions (MCPManager x2, ConnectionsRepository, UserConnectionManager, MCPServerInspector). MAJOR: Fix cross-process Redis race in lazyInitConfigServer — when configCacheRepo.add throws (key exists from another process), fall back to reading the existing entry instead of returning undefined. MINOR: Parallelize invalidateConfigCache awaits with Promise.all. Remove redundant .catch(() => {}) inside Promise.allSettled. Tighten dedup test assertion to toBe(1). Add TTL-expiry tests for getServerConfig (with and without userId). * feat: thread configServers through getAppToolFunctions and formatInstructionsForContext Add optional configServers parameter to getAppToolFunctions, getInstructions, and formatInstructionsForContext so config-source server tools and instructions are visible to agent initialization and context injection paths. Existing callers (boot-time init, tests) pass no argument and continue to work unchanged. Agent runtime paths can now thread resolved config servers from request context. * fix: stale failure stubs retry after 5 min, upsert for cross-process races - Add CONFIG_STUB_RETRY_MS (5 min) — stale failure stubs are retried instead of permanently disabling config-source servers after transient errors (DNS outage, cold-start race) - Extract upsertConfigCache() helper that tries add then falls back to update, preventing cross-process Redis races where a second instance's successful inspection result was discarded - Add test for stale-stub retry after CONFIG_STUB_RETRY_MS * fix: stamp updatedAt on failure stubs, null-guard callTool config, test cleanup - Add updatedAt: Date.now() to failure stubs in lazyInitConfigServer so CONFIG_STUB_RETRY_MS (5 min) window works correctly — without it, stubs were always considered stale (updatedAt ?? 0 → epoch → always expired) - Add null guard for rawConfig in MCPManager.callTool before passing to preProcessGraphTokens — prevents unsafe `as` cast on undefined - Log double-failure in upsertConfigCache instead of silently swallowing - Replace module-scope Date.now monkey-patch with jest.useFakeTimers / jest.setSystemTime / jest.useRealTimers in ensureConfigServers tests * fix: server-only readThrough fallback only returns truthy values Prevents a cached undefined from a prior no-userId lookup from short-circuiting the DB query on a subsequent userId-scoped lookup. * fix: remove findInConfigCache to eliminate cross-tenant config leakage The findInConfigCache prefix scan (serverName:*) could return any tenant's config after readThrough TTL expires, violating tenant isolation. Config-source servers are now ONLY resolvable through: 1. The configServers param (callers with tenant context from ALS) 2. The readThrough cache (populated by ensureSingleConfigServer, 5s TTL, repopulated on every HTTP request via resolveAllMcpConfigs) Connection/tool-call paths without tenant context rely exclusively on the readThrough cache. If it expires before the next HTTP request repopulates it, the server is not found — which is correct because there is no tenant context to determine which config to return. - Remove findInConfigCache method and its call in getServerConfig - Update server-only readThrough fallback to only return truthy values (prevents cached undefined from short-circuiting user-scoped DB lookup) - Update tests to document tenant isolation behavior after cache expiry * style: fix import order per AGENTS.md conventions Sort package imports shortest-to-longest, local imports longest-to-shortest across MCPServersRegistry, ConnectionsRepository, MCPManager, UserConnectionManager, and MCPServerInspector. * fix: eliminate cross-tenant readThrough contamination and TTL-expiry tool failures Thread pre-resolved serverConfig from tool creation context into callTool, removing dependency on the readThrough cache for config-source servers. This fixes two issues: - Cross-tenant contamination: the readThrough cache key was unscoped (just serverName), so concurrent multi-tenant requests for same-named servers would overwrite each other's entries - TTL expiry: tool calls happening >5s after config resolution would fail with "Configuration not found" because the readThrough entry had expired Changes: - Add optional serverConfig param to MCPManager.callTool — uses provided config directly, falling back to getServerConfig lookup for YAML/user servers - Thread serverConfig from createMCPTool through createToolInstance closure to callTool - Remove readThrough write from ensureSingleConfigServer — config-source servers are only accessible via configServers param (tenant-scoped) - Remove server-only readThrough fallback from getServerConfig - Increase config cache hash from 8 to 16 hex chars (64-bit) - Add isUserSourced boundary tests for all source/dbId combinations - Fix double Object.keys call in getMCPTools controller - Update test assertions for new getServerConfig behavior * fix: cache base configs for config-server users; narrow upsertConfigCache error handling - Refactor getAllServerConfigs to separate base config fetch (YAML + DB) from config-server layering. Base configs are cached via readThroughCacheAll regardless of whether configServers is provided, eliminating uncached MongoDB queries per request for config-server users - Narrow upsertConfigCache catch to duplicate-key errors only; infrastructure errors (Redis timeouts, network failures) now propagate instead of being silently swallowed, preventing inspection storms during outages * fix: restore correct merge order and document upsert error matching - Restore YAML → Config → User DB precedence in getAllServerConfigs (user DB servers have highest precedence, matching the JSDoc contract) - Add source comment on upsertConfigCache duplicate-key detection linking to the two cache implementations that define the error message * feat: complete config-source server support across all execution paths Wire configServers through the entire agent execution pipeline so config-source MCP servers are fully functional — not just visible in listings but executable in agent sessions. - Thread configServers into handleTools.js agent tool pipeline: resolve config servers from tenant context before MCP tool iteration, pass to getServerConfig, createMCPTools, and createMCPTool - Thread configServers into agent instructions pipeline: applyContextToAgent → getMCPInstructionsForServers → formatInstructionsForContext, resolved in client.js before agent context application - Add configServers param to createMCPTool and createMCPTools for reconnect path fallback - Add source field to redactServerSecrets allowlist for client UI differentiation of server tiers - Narrow invalidateConfigCache to only clear readThroughCacheAll (merged results), preserving YAML individual-server readThrough entries - Update context.spec.ts assertions for new configServers parameter * fix: add missing mocks for config-source server dependencies in client.test.js Mock getMCPServersRegistry, getAppConfig, and getTenantId that were added to client.js but not reflected in the test file's jest.mock declarations. * fix: update formatInstructionsForContext assertions for configServers param The test assertions expected formatInstructionsForContext to be called with only the server names array, but it now receives configServers as a second argument after the config-source server feature wiring. * fix: move configServers resolution before MCP tool loop to avoid TDZ configServers was declared with `let` after the first tool loop but referenced inside it via getServerConfig(), causing a ReferenceError temporal dead zone. Move declaration and resolution before the loop, using tools.some(mcpToolPattern) to gate the async resolution. * fix: address review findings — cache bypass, discoverServerTools gap, DRY - #2: getAllServerConfigs now always uses getBaseServerConfigs (cached via readThroughCacheAll) instead of bypassing it when configServers is present. Extracts user-DB entries from cached base by diffing against YAML keys to maintain YAML → Config → User DB merge order without extra MongoDB calls. - #3: Add configServers param to ToolDiscoveryOptions and thread it through discoverServerTools → getServerConfig so config-source servers are discoverable during OAuth reconnection flows. - #6: Replace inline import() type annotations in context.ts with proper import type { ParsedServerConfig } per AGENTS.md conventions. - #7: Extract resolveConfigServers(req) helper in MCP.js and use it from handleTools.js and client.js, eliminating the duplicated 6-line config resolution pattern. - #10: Restore removed "why" comment explaining getLoaded() vs getAll() choice in getMCPSetupData — documents non-obvious correctness constraint. - #11: Fix incomplete JSDoc param type on resolveAllMcpConfigs. * fix: consolidate imports, reorder constants, fix YAML-DB merge edge case - Merge duplicate @librechat/data-schemas requires in MCP.js into one - Move resolveConfigServers after module-level constants - Fix getAllServerConfigs edge case where user-DB entry overriding a YAML entry with the same name was excluded from userDbConfigs; now uses reference equality check to detect DB-overwritten YAML keys * fix: replace fragile string-match error detection with proper upsert method Add upsert() to IServerConfigsRepositoryInterface and all implementations (InMemory, Redis, RedisAggregateKey, DB). This eliminates the brittle error message string match ('already exists in cache') in upsertConfigCache that was the only thing preventing cross-process init races from silently discarding inspection results. Each implementation handles add-or-update atomically: - InMemory: direct Map.set() - Redis: direct cache.set() - RedisAggregateKey: read-modify-write under write lock - DB: delegates to update() (DB servers use explicit add() with ACL setup) * fix: wire configServers through remaining HTTP endpoints - getMCPServerById: use resolveAllMcpConfigs instead of bare getServerConfig - reinitialize route: resolve configServers before getServerConfig - auth-values route: resolve configServers before getServerConfig - getOAuthHeaders: accept configServers param, thread from callers - Update mcp.spec.js tests to mock getAllServerConfigs for GET by name * fix: thread serverConfig through getConnection for config-source servers Config-source servers exist only in configCacheRepo, not in YAML cache or DB. When callTool → getConnection → getUserConnection → getServerConfig runs without configServers, it returns undefined and throws. Fix by threading the pre-resolved serverConfig (providedConfig) from callTool through getConnection → getUserConnection → createUserConnectionInternal, using it as a fallback before the registry lookup. * fix: thread configServers through reinit, reconnect, and tool definition paths Wire configServers through every remaining call chain that creates or reconnects MCP server connections: - reinitMCPServer: accepts serverConfig and configServers, uses them for getServerConfig fallback, getConnection, and discoverServerTools - reconnectServer: accepts and passes configServers to reinitMCPServer - createMCPTools/createMCPTool: pass configServers to reconnectServer - ToolService.loadToolDefinitionsWrapper: resolves configServers from req, passes to both reinitMCPServer call sites - reinitialize route: passes serverConfig and configServers to reinitMCPServer * fix: address review findings — simplify merge, harden error paths, fix log labels - Simplify getAllServerConfigs merge: replace fragile reference-equality loop with direct spread { ...yamlConfigs, ...configServers, ...base } - Guard upsertConfigCache in lazyInitConfigServer catch block so cache failures don't mask the original inspection error - Deduplicate getYamlServerNames cold-start with promise dedup pattern - Remove dead `if (!mcpConfig)` guard in getMCPSetupData - Fix hardcoded "App server" in ServerConfigsCacheRedisAggregateKey error messages — now uses this.namespace for correct Config/App labeling - Remove misleading OAuth callback comment about readThrough cache - Move resolveConfigServers after module-level constants in MCP.js * fix: clear rejected yamlServerNames promise, fix config-source reinspect, fix reset log label - Clear yamlServerNamesPromise on rejection so transient cache errors don't permanently prevent ensureConfigServers from working - Skip reinspectServer for config-source servers (source: 'config') in reinitMCPServer — they lack a CACHE/DB storage location; retry is handled by CONFIG_STUB_RETRY_MS in ensureConfigServers - Use source field instead of dbId for storageLocation derivation - Fix remaining hardcoded "App" in reset() leaderCheck message * fix: persist oauthHeaders in flow state for config-source OAuth servers The OAuth callback route has no JWT auth context and cannot resolve config-source server configs. Previously, getOAuthHeaders would silently return {} for config-source servers, dropping custom token exchange headers. Now oauthHeaders are persisted in MCPOAuthFlowMetadata during flow initiation (which has auth context), and the callback reads them from the stored flow state with a fallback to the registry lookup for YAML/user-DB servers. * fix: update tests for getMCPSetupData null guard removal and ToolService mock - MCP.spec.js: update test to expect graceful handling of null mcpConfig instead of a throw (getAllServerConfigs always returns an object) - MCP.js: add defensive || {} for Object.entries(mcpConfig) in case of null from test mocks - ToolService.spec.js: add missing mock for ~/server/services/MCP (resolveConfigServers) * fix: address review findings — DRY, naming, logging, dead code, defensive guards - #1: Simplify getAllServerConfigs to single getBaseServerConfigs call, eliminating redundant double-fetch of cacheConfigsRepo.getAll() - #2: Add warning log when oauthHeaders absent from OAuth callback flow state - #3: Extract resolveAllMcpConfigs to MCP.js service layer; controller imports shared helper instead of reimplementing - #4: Rename _serverConfig/_provider to capturedServerConfig/capturedProvider in createToolInstance — these are actively used, not unused - #5: Log rejected results from ensureConfigServers Promise.allSettled so cache errors are visible instead of silently dropped - #6: Remove dead 'MCP config not found' error handlers from routes - #7: Document circular-dependency reason for dynamic require in clearMcpConfigCache - #8: Remove logger.error from withTimeout to prevent double-logging timeouts - #10: Add explicit userId guard in ServerConfigsDB.upsert with clear error message - #12: Use spread instead of mutation in addServer for immutability consistency - Add upsert mock to ensureConfigServers.test.ts DB mock - Update route tests for resolveAllMcpConfigs import change * fix: restore correct merge priority, use immutable spread, fix test mock - getAllServerConfigs: { ...configServers, ...base } so userDB wins over configServers, matching documented "User DB (highest)" priority - lazyInitConfigServer: use immutable spread instead of direct mutation for parsedConfig.source, consistent with addServer fix - Fix test to mock getAllServerConfigs as {} instead of null, remove unnecessary || {} defensive guard in getMCPSetupData * fix: error handling, stable hashing, flatten nesting, remove dead param - Wrap resolveConfigServers/resolveAllMcpConfigs in try/catch with graceful {} fallback so transient DB/cache errors don't crash tool pipeline - Sort keys in configCacheKey JSON.stringify for deterministic hashing regardless of object property insertion order - Flatten clearMcpConfigCache from 3 nested try-catch to early returns; document that user connections are cleaned up lazily (accepted tradeoff) - Remove dead configServers param from getAppToolFunctions (never passed) - Add security rationale comment for source field in redactServerSecrets * fix: use recursive key-sorting replacer in configCacheKey to prevent cross-tenant cache collision The array replacer in JSON.stringify acts as a property allowlist at every nesting depth, silently dropping nested keys like headers['X-API-Key'], oauth.client_secret, etc. Two configs with different nested values but identical top-level structure produced the same hash, causing cross-tenant cache hits and potential credential contamination. Switch to a function replacer that recursively sorts keys at all depths without dropping any properties. Also document the known gap in getOAuthServers: config-source OAuth servers are not covered by auto-reconnection or uninstall cleanup because callers lack request context. * fix: move clearMcpConfigCache to packages/api to eliminate circular dependency The function only depends on MCPServersRegistry and MCPManager, both of which live in packages/api. Import it directly from @librechat/api in the CJS layer instead of using dynamic require('~/config'). * chore: imports/fields ordering * fix: address review findings — error handling, targeted lookup, test gaps - Narrow resolveAllMcpConfigs catch to only wrap ensureConfigServers so getAppConfig/getAllServerConfigs failures propagate instead of masking infrastructure errors as empty server lists. - Use targeted getServerConfig in getMCPServerById instead of fetching all server configs for a single-server lookup. - Forward configServers to inner createMCPTool calls so reconnect path works for config-source servers. - Update getAllServerConfigs JSDoc to document disjoint-key design. - Add OAuth callback oauthHeaders fallback tests (flow state present vs registry fallback). - Add resolveConfigServers/resolveAllMcpConfigs unit tests covering happy path and error propagation. * fix: add getOAuthReconnectionManager mock to OAuth callback tests * chore: imports ordering	2026-03-28 10:36:43 -04:00
Danny Avila	77712c825f	🏢 feat: Tenant-Scoped App Config in Auth Login Flows (#12434) ... * feat: add resolveAppConfigForUser utility for tenant-scoped auth config TypeScript utility in packages/api that wraps getAppConfig in tenantStorage.run() when the user has a tenantId, falling back to baseOnly for new users or non-tenant deployments. Uses DI pattern (getAppConfig passed as parameter) for testability. Auth flows apply role-level overrides only (userId not passed) because user/group principal resolution is deferred to post-auth. * feat: tenant-scoped app config in auth login flows All auth strategies (LDAP, SAML, OpenID, social login) now use a two-phase domain check consistent with requestPasswordReset: 1. Fast-fail with base config (memory-cached, zero DB queries) 2. DB user lookup 3. Tenant-scoped re-check via resolveAppConfigForUser (only when user has a tenantId; otherwise reuse base config) This preserves the original fast-fail protection against globally blocked domains while enabling tenant-specific config overrides. OpenID error ordering preserved: AUTH_FAILED checked before domain re-check so users with wrong providers get the correct error type. registerUser unchanged (baseOnly, no user identity yet). * test: add tenant-scoped config tests for auth strategies Add resolveAppConfig.spec.ts in packages/api with 8 tests: - baseOnly fallback for null/undefined/no-tenant users - tenant-scoped config with role and tenantId - ALS context propagation verified inside getAppConfig callback - undefined role with tenantId edge case Update strategy and AuthService tests to mock resolveAppConfigForUser via @librechat/api. Tests verify two-phase domain check behavior: fast-fail before DB, tenant re-check after. Non-tenant users reuse base config without calling resolveAppConfigForUser. * refactor: skip redundant domain re-check for non-tenant users Guard the second isEmailDomainAllowed call with appConfig !== baseConfig in SAML, OpenID, and social strategies. For non-tenant users the tenant config is the same base config object, so the second check is a no-op. Narrow eslint-disable in resolveAppConfig.spec.ts to the specific require line instead of blanket file-level suppression. * fix: address review findings — consistency, tests, and ordering - Consolidate duplicate require('@librechat/api') in AuthService.js - Add two-phase domain check to LDAP (base fast-fail before findUser), making all strategies consistent with PR description - Add appConfig !== baseConfig guard to requestPasswordReset second domain check, consistent with SAML/OpenID/social strategies - Move SAML provider check before tenant config resolution to avoid unnecessary resolveAppConfigForUser call for wrong-provider users - Add tenant domain rejection tests to SAML, OpenID, and social specs verifying that tenant config restrictions actually block login - Add error propagation tests to resolveAppConfig.spec.ts - Remove redundant mockTenantStorage alias in resolveAppConfig.spec.ts - Narrow eslint-disable to specific require line * test: add tenant domain rejection test for LDAP strategy Covers the appConfig !== baseConfig && !isEmailDomainAllowed path, consistent with SAML, OpenID, and social strategy specs. * refactor: rename resolveAppConfig to app/resolve per AGENTS.md Rename resolveAppConfig.ts → resolve.ts and resolveAppConfig.spec.ts → resolve.spec.ts to align with the project's concise naming convention. * fix: remove fragile reference-equality guard, add logging and docs Remove appConfig !== baseConfig guard from all strategies and requestPasswordReset. The guard relied on implicit cache-backend identity semantics (in-memory Keyv returns same object reference) that would silently break with Redis or cloned configs. The second isEmailDomainAllowed call is a cheap synchronous check — always running it is clearer and eliminates the coupling. Add audit logging to requestPasswordReset domain blocks (base and tenant), consistent with all auth strategies. Extract duplicated error construction into makeDomainDeniedError(). Wrap resolveAppConfigForUser in requestPasswordReset with try/catch to prevent DB errors from leaking to the client via the controller's generic catch handler. Document the dual tenantId propagation (ALS for DB isolation, explicit param for cache key) in resolveAppConfigForUser JSDoc. Add comment documenting the LDAP error-type ordering change (cross-provider users from blocked domains now get 'domain not allowed' instead of AUTH_FAILED). Assert resolveAppConfigForUser is not called on LDAP provider mismatch path. * fix: return generic response for tenant domain block in password reset Tenant-scoped domain rejection in requestPasswordReset now returns the same generic "If an account with that email exists..." response instead of an Error. This prevents user-enumeration: an attacker cannot distinguish between "email not found" and "tenant blocks this domain" by comparing HTTP responses. The base-config fast-fail (pre-user-lookup) still returns an Error since it fires before any user existence is revealed. * docs: document phase 1 vs phase 2 domain check behavior in JSDoc Phase 1 (base config, pre-findUser) intentionally returns Error/400 to reveal globally blocked domains without confirming user existence. Phase 2 (tenant config, post-findUser) returns generic 200 to prevent user-enumeration. This distinction is now explicit in the JSDoc.	2026-03-27 16:08:43 -04:00
Dustin Healy	5972a21479	🪪 feat: Admin Roles API Endpoints (#12400) ... * feat: add createRole and deleteRole methods to role * feat: add admin roles handler factory and Express routes * fix: address convention violations in admin roles handlers * fix: rename createRole/deleteRole to avoid AccessRole name collision The existing accessRole.ts already exports createRole/deleteRole for the AccessRole model. In createMethods index.ts, these are spread after roleMethods, overwriting them. Renamed our Role methods to createRoleByName/deleteRoleByName to match the existing pattern (getRoleByName, updateRoleByName) and avoid the collision. * feat: add description field to Role model - Add description to IRole, CreateRoleRequest, UpdateRoleRequest types - Add description field to Mongoose roleSchema (default: '') - Wire description through createRoleHandler and updateRoleHandler - Include description in listRoles select clause so it appears in list * fix: address Copilot review findings in admin roles handlers * test: add unit tests for admin roles and groups handlers * test: add data-layer tests for createRoleByName, deleteRoleByName, listUsersByRole * fix: allow system role updates when name is unchanged The updateRoleHandler guard rejected any request where body.name matched a system role, even when the name was not being changed. This blocked editing a system role's description. Compare against the URL param to only reject actual renames to reserved names. * fix: address external review findings for admin roles - Block renaming system roles (ADMIN/USER) and add user migration on rename - Add input validation: name max-length, trim on update, duplicate name check - Replace fragile String.includes error matching with prefix-based classification - Catch MongoDB 11000 duplicate key in createRoleByName - Add pagination (limit/offset/total) to getRoleMembersHandler - Reverse delete order in deleteRoleByName — reassign users before deletion - Add role existence check in removeRoleMember; drop unused createdAt select - Add Array.isArray guard for permissions input; use consistent ?? coalescing - Fix import ordering per AGENTS.md conventions - Type-cast mongoose.models.User as Model<IUser> for proper TS inference - Add comprehensive tests: rename guards, pagination, validation, 500 paths * fix: address re-review findings for admin roles - Gate deleteRoleByName on existence check — skip user reassignment and cache invalidation when role doesn't exist (fixes test mismatch) - Reverse rename order: migrate users before renaming role so a migration failure leaves the system in a consistent state - Add .sort({ _id: 1 }) to listUsersByRole for deterministic pagination - Import shared AdminMember type from data-schemas instead of local copy; make joinedAt optional since neither groups nor roles populate it - Change IRole.description from optional to required to match schema default - Add data-layer tests for updateUsersByRole and countUsersByRole - Add handler test verifying users-first rename ordering and migration failure safety * fix: add rollback on rename failure and update PR description - Roll back user migration if updateRoleByName returns null during a rename (race: role deleted between existence check and update) - Add test verifying rollback calls updateUsersByRole in reverse - Update PR #12400 description to reflect current test counts (56 handler tests, 40 data-layer tests) and safety features * fix: rollback on rename throw, description validation, delete/DRY cleanup - Hoist isRename/trimmedName above try block so catch can roll back user migration when updateRoleByName throws (not just returns null) - Add description type + max-length (2000) validation in create and update, consistent with groups handler - Remove redundant getRoleByName existence check in deleteRoleHandler — use deleteRoleByName return value directly - Skip no-op name write when body.name equals current name (use isRename) - Extract getUserModel() accessor to DRY repeated Model<IUser> casts - Use name.trim() consistently in createRoleByName error messages - Add tests: rename-throw rollback, description validation (create+update), update delete test mocks to match simplified handler * fix: guard spurious rollback, harden createRole error path, validate before DB calls - Add migrationRan flag to prevent rollback of user migration that never ran - Return generic message on 500 in createRoleHandler, specific only for 409 - Move description validation before DB queries in updateRoleHandler - Return existing role early when update body has no changes - Wrap cache.set in createRoleByName with try/catch to prevent masking DB success - Add JSDoc on 11000 catch explaining compound unique index - Add tests: spurious rollback guard, empty update body, description validation ordering, listUsersByRole pagination * fix: validate permissions in create, RoleConflictError, rollback safety, cache consistency - Add permissions type/array validation in createRoleHandler - Introduce RoleConflictError class replacing fragile string-prefix matching - Wrap rollback in !role null path with try/catch for correct 404 response - Wrap deleteRoleByName cache.set in try/catch matching createRoleByName - Narrow updateRoleHandler body type to { name?, description? } - Add tests: non-string description in create, rollback failure logging, permissions array rejection, description max-length assertion fix * feat: prevent removing the last admin user Add guard in removeRoleMember that checks countUsersByRole before demoting an ADMIN user, returning 400 if they are the last one. * fix: move interleaved export below imports, add await to countUsersByRole * fix: paginate listRoles, null-guard permissions handler, fix export ordering - Add limit/offset/total pagination to listRoles matching the groups pattern - Add countRoles data-layer method - Omit permissions from listRoles select (getRole returns full document) - Null-guard re-fetched role in updateRolePermissionsHandler - Move interleaved export below all imports in methods/index.ts * fix: address review findings — race safety, validation DRY, type accuracy, test coverage - Add post-write admin count verification in removeRoleMember to prevent zero-admin race condition (TOCTOU → rollback if count hits 0) - Make IRole.description optional; backfill in initializeRoles for pre-existing roles that lack the field (.lean() bypasses defaults) - Extract parsePagination, validateNameParam, validateRoleName, and validateDescription helpers to eliminate duplicated validation - Add validateNameParam guard to all 7 handlers reading req.params.name - Catch 11000 in updateRoleByName and surface as 409 via RoleConflictError - Add idempotent skip in addRoleMember when user already has target role - Verify updateRolePermissions test asserts response body - Add data-layer tests: listRoles sort/pagination/projection, countRoles, and createRoleByName 11000 duplicate key race * fix: defensive rollback in removeRoleMember, type/style cleanup, test coverage - Wrap removeRoleMember post-write admin rollback in try/catch so a transient DB failure cannot leave the system with zero administrators - Replace double `as unknown[] as IRole[]` cast with `.lean<IRole[]>()` - Type parsePagination param explicitly; extract DEFAULT/MAX page constants - Preserve original error cause in updateRoleByName re-throw - Add test for rollback failure path in removeRoleMember (returns 400) - Add test for pre-existing roles missing description field (.lean()) * chore: bump @librechat/data-schemas to 0.0.47 * fix: stale cache on rename, extract renameRole helper, shared pagination, cleanup - Fix updateRoleByName cache bug: invalidate old key and populate new key when updates.name differs from roleName (prevents stale cache after rename) - Extract renameRole helper to eliminate mutable outer-scope state flags (isRename, trimmedName, migrationRan) in updateRoleHandler - Unify system-role protection to 403 for both rename-from and rename-to - Extract parsePagination to shared admin/pagination.ts; use in both roles.ts and groups.ts - Extract name.trim() to local const in createRoleByName (was called 5×) - Remove redundant findOne pre-check in deleteRoleByName - Replace getUserModel closure with local const declarations - Remove redundant description ?? '' in createRoleHandler (schema default) - Add doc comment on updateRolePermissionsHandler noting cache dependency - Add data-layer tests for cache rename behavior (old key null, new key set) * fix: harden role guards, add User.role index, validate names, improve tests - Add index on User.role field for efficient member queries at scale - Replace fragile SystemRoles key lookup with value-based Set check (6 sites) - Elevate rename rollback failure logging to CRITICAL (matches removeRoleMember) - Guard removeRoleMember against non-ADMIN system roles (403 for USER) - Fix parsePagination limit=0 gotcha: use parseInt + NaN check instead of || - Add control character and reserved path segment validation to role names - Simplify validateRoleName: remove redundant casts and dead conditions - Add JSDoc to deleteRoleByName documenting non-atomic window - Split mixed value+type import in methods/index.ts per AGENTS.md - Add 9 new tests: permissions assertion, combined rename+desc, createRole with permissions, pagination edge cases, control char/reserved name rejection, system role removeRoleMember guard * fix: exact-case reserved name check, consistent validation, cleaner createRole - Remove .toLowerCase() from reserved name check so only exact matches (members, permissions) are rejected, not legitimate names like "Members" - Extract trimmed const in validateRoleName for consistent validation - Add control char check to validateNameParam for parity with body validation - Build createRole roleData conditionally to avoid passing description: undefined - Expand deleteRoleByName JSDoc documenting self-healing design and no-op trade-off * fix: scope rename rollback to only migrated users, prevent cross-role corruption Capture user IDs before forward migration so the rollback path only reverts users this request actually moved. Previously the rollback called updateUsersByRole(newName, currentName) which would sweep all users with the new role — including any independently assigned by a concurrent admin request — causing silent cross-role data corruption. Adds findUserIdsByRole and updateUsersRoleByIds to the data layer. Extracts rollbackMigratedUsers helper to deduplicate rollback sites. * fix: guard last admin in addRoleMember to prevent zero-admin lockout Since each user has exactly one role, addRoleMember implicitly removes the user from their current role. Without a guard, reassigning the sole admin to a non-admin role leaves zero admins and locks out admin management. Adds the same countUsersByRole check used in removeRoleMember. * fix: wire findUserIdsByRole and updateUsersRoleByIds into roles route The scoped rollback deps added in c89b5db were missing from the route DI wiring, causing renameRole to call undefined and return a 500. * fix: post-write admin guard in addRoleMember, compound role index, review cleanup - Add post-write admin count check + rollback to addRoleMember to match removeRoleMember's two-phase TOCTOU protection (prevents zero-admin via concurrent requests) - Replace single-field User.role index with compound { role: 1, tenantId: 1 } to align with existing multi-tenant index pattern (email, OAuth IDs) - Narrow listRoles dep return type to RoleListItem (projected fields only) - Refactor validateDescription to early-return style per AGENTS.md - Remove redundant double .lean() in updateRoleByName - Document rename snapshot race window in renameRole JSDoc - Document cache null-set behavior in deleteRoleByName - Add routing-coupling comment on RESERVED_ROLE_NAMES - Add test for addRoleMember post-write rollback * fix: review cleanup — system-role guard, type safety, JSDoc accuracy, tests - Add system-role guard to addRoleMember: block direct assignment to non-ADMIN system roles (403), symmetric with removeRoleMember - Fix RESERVED_ROLE_NAMES comment: explain semantic URL ambiguity, not a routing conflict (Express resolves single vs multi-segment correctly) - Replace _id: unknown with Types.ObjectId | string per AGENTS.md - Narrow listRoles data-layer return type to Pick<IRole, 'name' | 'description'> to match the actual .select() projection - Move updateRoleHandler param check inside try/catch for consistency - Include user IDs in all CRITICAL rollback failure logs for operator recovery - Clarify deleteRoleByName JSDoc: replace "self-healing" with "idempotent", document that recovery requires caller retry - Add tests: system-role guard, promote non-admin to ADMIN, findUserIdsByRole throw prevents migration * fix: include _id in listRoles return type to match RoleListItem Pick<IRole, 'name' | 'description'> omits _id, making it incompatible with the handler dep's RoleListItem which requires _id. * fix: case-insensitive system role guard, reject null permissions, check updateUser result - System role name checks now use case-insensitive comparison via toUpperCase() — prevents creating 'admin' or 'user' which would collide with the legacy roles route that uppercases params - Reject permissions: null in createRole (typeof null === 'object' was bypassing the validation) - Check updateUser return in addRoleMember — return 404 if the user was deleted between the findUser and updateUser calls * fix: check updateUser return in removeRoleMember for concurrent delete safety --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-03-27 15:44:47 -04:00
Dustin Healy	2e3d66cfe2	👥 feat: Admin Groups API Endpoints (#12387) ... * feat: add listGroups and deleteGroup methods to userGroup * feat: add admin groups handler factory and Express routes * fix: address convention violations in admin groups handlers * fix: address Copilot review findings in admin groups handlers - Escape regex in listGroups to prevent injection/ReDoS - Validate ObjectId format in all handlers accepting id/userId params - Replace N+1 findUser loop with batched findUsers query - Remove unused findGroupsByMemberId from dep interface - Map Mongoose ValidationError to 400 in create/update handlers - Validate name in updateGroupHandler (reject empty/whitespace) - Handle null updateGroupById result (race condition) - Tighten error message matching in add/remove member handlers * test: add unit tests for admin groups handlers * fix: address code review findings for admin groups Atomic delete/update handlers (single DB trip), pass through idOnTheSource, add removeMemberById for non-ObjectId members, deduplicate member results, fix error message exposure, add hard cap/sort to listGroups, replace GroupListFilter with Pick of GroupFilterOptions, validate memberIds as array, trim name in update, fix import order, and improve test hygiene with fresh IDs per test. * fix: cascade cleanup, pagination, and test coverage for admin groups Add deleteGrantsForPrincipal to systemGrant data layer and wire cascade cleanup (Config, AclEntry, SystemGrant) into deleteGroupHandler. Add limit/offset pagination to getGroupMembers. Guard empty PATCH bodies with 400. Remove dead type guard and unnecessary type cast. Add 11 new tests covering cascade delete, idempotent member removal, empty update, search filter, 500 error paths, and pagination. * fix: harden admin groups with cascade resilience, type safety, and fallback removal Wrap cascade cleanup in inner try/catch so partial failure logs but still returns 200 (group is already deleted). Replace Record<string, unknown> on deleteAclEntries with proper typed filter. Log warning for unmapped user ObjectIds in createGroup memberIds. Add removeMemberById fallback when removeUserFromGroup throws User not found for ObjectId-format userId. Extract VALID_GROUP_SOURCES constant. Add 3 new tests (60 total). * refactor: add countGroups, pagination, and projection type to data layer Extract buildGroupQuery helper, add countGroups method, support limit/offset/skip in listGroups, standardize session handling to .session(session ?? null), and tighten projection parameter from Record<string, unknown> to Record<string, 0 | 1>. * fix: cascade resilience, pagination, validation, and error clarity for admin groups - Use Promise.allSettled for cascade cleanup so all steps run even if one fails; log individual rejections - Echo deleted group id in delete response - Add countGroups dep and wire limit/offset pagination for listGroups - Deduplicate memberIds before computing total in getGroupMembers - Use { memberIds: 1 } projection in getGroupMembers - Cap memberIds at 500 entries in createGroup - Reject search queries exceeding 200 characters - Clarify addGroupMember error for non-ObjectId userId - Document deleted-user fallback limitation in removeGroupMember * test: extend handler and DB-layer test coverage for admin groups Handler tests: projection assertion, dedup total, memberIds cap, search max length, non-ObjectId memberIds passthrough, cascade partial failure resilience, dedup scenarios, echo id in delete response. DB-layer tests: listGroups sort/filter/pagination, countGroups, deleteGroup, removeMemberById, deleteGrantsForPrincipal. * fix: cast group principalId to ObjectId for ACL entry cleanup deleteAclEntries is a thin deleteMany wrapper with no type casting, but grantPermission stores group principalId as ObjectId. Passing the raw string from req.params would leave orphaned ACL entries on group deletion. * refactor: remove redundant pagination clamping from DB listGroups Handler already clamps limit/offset at the API boundary. The DB method is a general-purpose building block and should not re-validate. * fix: add source and name validation, import order, and test coverage for admin groups - Validate source against VALID_GROUP_SOURCES in createGroupHandler - Cap name at 500 characters in both create and update handlers - Document total as upper bound in getGroupMembers response - Document ObjectId requirement for deleteAclEntries in cascade - Fix import ordering in test file (local value after type imports) - Add tests for updateGroup with description, email, avatar fields - Add tests for invalid source and name max-length in both handlers * fix: add field length caps, flatten nested try/catch, and fix logger level in admin groups Add max-length validation for description, email, avatar, and idOnTheSource in create/update handlers. Extract removeObjectIdMember helper to flatten nested try/catch per never-nesting convention. Downgrade unmapped-memberIds log from error to warn. Fix type import ordering and add missing await in removeMemberById for consistency.	2026-03-26 17:36:18 -04:00
Danny Avila	9f6d8c6e93	🧵 feat: ALS Context Middleware, Tenant Threading, and Config Cache Invalidation (#12407) ... * feat: add tenant context middleware for ALS-based isolation Introduces tenantContextMiddleware that propagates req.user.tenantId into AsyncLocalStorage, activating the Mongoose applyTenantIsolation plugin for all downstream DB queries within a request. - Strict mode (TENANT_ISOLATION_STRICT=true) returns 403 if no tenantId - Non-strict mode passes through for backward compatibility - No-op for unauthenticated requests - Includes 6 unit tests covering all paths * feat: register tenant middleware and wrap startup/auth in runAsSystem() - Register tenantContextMiddleware in Express app after capability middleware - Wrap server startup initialization in runAsSystem() for strict mode compat - Wrap auth strategy getAppConfig() calls in runAsSystem() since they run before user context is established (LDAP, SAML, OpenID, social login, AuthService) * feat: thread tenantId through all getAppConfig callers Pass tenantId from req.user to getAppConfig() across all callers that have request context, ensuring correct per-tenant cache key resolution. Also fixes getBaseConfig admin endpoint to scope to requesting admin's tenant instead of returning the unscoped base config. Files updated: - Controllers: UserController, PluginController - Middleware: checkDomainAllowed, balance - Routes: config - Services: loadConfigModels, loadDefaultModels, getEndpointsConfig, MCP - Audio services: TTSService, STTService, getVoices, getCustomConfigSpeech - Admin: getBaseConfig endpoint * feat: add config cache invalidation on admin mutations - Add clearOverrideCache(tenantId?) to flush per-principal override caches by enumerating Keyv store keys matching _OVERRIDE_: prefix - Add invalidateConfigCaches() helper that clears base config, override caches, tool caches, and endpoint config cache in one call - Wire invalidation into all 5 admin config mutation handlers (upsert, patch, delete field, delete overrides, toggle active) - Add strict mode warning when __default__ tenant fallback is used - Add 3 new tests for clearOverrideCache (all/scoped/base-preserving) * chore: update getUserPrincipals comment to reflect ALS-based tenant filtering The TODO(#12091) about missing tenantId filtering is resolved by the tenant context middleware + applyTenantIsolation Mongoose plugin. Group queries are now automatically scoped by tenantId via ALS. * fix: replace runAsSystem with baseOnly for pre-tenant code paths App configs are tenant-owned — runAsSystem() would bypass tenant isolation and return cross-tenant DB overrides. Instead, add baseOnly option to getAppConfig() that returns YAML-derived config only, with zero DB queries. All startup code, auth strategies, and MCP initialization now use getAppConfig({ baseOnly: true }) to get the YAML config without touching the Config collection. * fix: address PR review findings — middleware ordering, types, cache safety - Chain tenantContextMiddleware inside requireJwtAuth after passport auth instead of global app.use() where req.user is always undefined (Finding 1) - Remove global tenantContextMiddleware registration from index.js - Update BalanceMiddlewareOptions to include tenantId, remove redundant cast (Finding 4) - Add warning log when clearOverrideCache cannot enumerate keys on Redis (Finding 3) - Use startsWith instead of includes for cache key filtering (Finding 12) - Use generator loop instead of Array.from for key enumeration (Finding 3) - Selective barrel export — exclude _resetTenantMiddlewareStrictCache (Finding 5) - Move isMainThread check to module level, remove per-request check (Finding 9) - Move mid-file require to top of app.js (Finding 8) - Parallelize invalidateConfigCaches with Promise.all (Finding 10) - Remove clearOverrideCache from public app.js exports (internal only) - Strengthen getUserPrincipals comment re: ALS dependency (Finding 2) * fix: restore runAsSystem for startup DB ops, consolidate require, clarify baseOnly - Restore runAsSystem() around performStartupChecks, updateInterfacePermissions, initializeMCPs, and initializeOAuthReconnectManager — these make Mongoose queries that need system context in strict tenant mode (NEW-3) - Consolidate duplicate require('@librechat/api') in requireJwtAuth.js (NEW-1) - Document that baseOnly ignores role/userId/tenantId in JSDoc (NEW-2) * test: add requireJwtAuth tenant chaining + invalidateConfigCaches tests - requireJwtAuth: 5 tests verifying ALS tenant context is set after passport auth, isolated between concurrent requests, and not set when user has no tenantId (Finding 6) - invalidateConfigCaches: 4 tests verifying all four caches are cleared, tenantId is threaded through, partial failure is handled gracefully, and operations run in parallel via Promise.all (Finding 11) * fix: address Copilot review — passport errors, namespaced cache keys, /base scoping - Forward passport errors in requireJwtAuth before entering tenant middleware — prevents silent auth failures from reaching handlers (P1) - Account for Keyv namespace prefix in clearOverrideCache — stored keys are namespaced as "APP_CONFIG:_OVERRIDE_:..." not "_OVERRIDE_:...", so override caches were never actually matched/cleared (P2) - Remove role from getBaseConfig — /base should return tenant-scoped base config, not role-merged config that drifts per admin role (P2) - Return tenantStorage.run() for cleaner async semantics - Update mock cache in service.spec.ts to simulate Keyv namespacing * fix: address second review — cache safety, code quality, test reliability - Decouple cache invalidation from mutation response: fire-and-forget with logging so DB mutation success is not masked by cache failures - Extract clearEndpointConfigCache helper from inline IIFE - Move isMainThread check to lazy once-per-process guard (no import side effect) - Memoize process.env read in overrideCacheKey to avoid per-request env lookups and log flooding in strict mode - Remove flaky timer-based parallelism assertion, use structural check - Merge orphaned double JSDoc block on getUserPrincipals - Fix stale [getAppConfig] log prefix → [ensureBaseConfig] - Fix import order in tenant.spec.ts (package types before local values) - Replace "Finding 1" reference with self-contained description - Use real tenantStorage primitives in requireJwtAuth spec mock * fix: move JSDoc to correct function after clearEndpointConfigCache extraction * refactor: remove Redis SCAN from clearOverrideCache, rely on TTL expiry Redis SCAN causes 60s+ stalls under concurrent load (see #12410). APP_CONFIG defaults to FORCED_IN_MEMORY_CACHE_NAMESPACES, so the in-memory store.keys() path handles the standard case. When APP_CONFIG is Redis-backed, overrides expire naturally via overrideCacheTtl (60s default) — an acceptable window for admin config mutations. * fix: remove return from tenantStorage.run to satisfy void middleware signature * fix: address second review — cache safety, code quality, test reliability - Switch invalidateConfigCaches from Promise.all to Promise.allSettled so partial failures are logged individually instead of producing one undifferentiated error (Finding 3) - Gate overrideCacheKey strict-mode warning behind a once-per-process flag to prevent log flooding under load (Finding 4) - Add test for passport error forwarding in requireJwtAuth — the if (err) { return next(err) } branch now has coverage (Finding 5) - Add test for real partial failure in invalidateConfigCaches where clearAppConfigCache rejects (not just the swallowed endpoint error) * chore: reorder imports in index.js and app.js for consistency - Moved logger and runAsSystem imports to maintain a consistent import order across files. - Improved code readability by ensuring related imports are grouped together.	2026-03-26 17:35:00 -04:00
Danny Avila	5e3b7bcde3	🌊 refactor: Local Snapshot for Aggregate Key Cache to Avoid Redundant Redis GETs (#12422) ... * perf: Add local snapshot to aggregate key cache to avoid redundant Redis GETs getAll() was being called 20+ times per chat request (once per tool, per server config lookup, per connection check). Each call hit Redis even though the data doesn't change within a request cycle. Add an in-memory snapshot with 5s TTL that collapses all reads within the window into a single Redis GET. Writes (add/update/remove/reset) invalidate the snapshot immediately so mutations are never stale. Also removes the debug logger that was producing noisy per-call logs. * fix: Prevent snapshot mutation and guarantee cleanup on write failure - Never mutate the snapshot object in-place during writes. Build a new object (spread) so concurrent readers never observe uncommitted state. - Move invalidateLocalSnapshot() into withWriteLock's finally block so cleanup is guaranteed even when successCheck throws on Redis failure. - After successful writes, populate the snapshot with the committed state to avoid an unnecessary Redis GET on the next read. - Use Date.now() after the await in getAll() so the TTL window isn't shortened by Redis latency. - Strengthen tests: spy on underlying Keyv cache to verify N getAll() calls collapse into 1 Redis GET, verify snapshot reference immutability. * fix: Remove dead populateLocalSnapshot calls from write callbacks populateLocalSnapshot was called inside withWriteLock callbacks, but the finally block in withWriteLock always calls invalidateLocalSnapshot immediately after — undoing the populate on every execution path. Remove the dead method and its three call sites. The snapshot is correctly cleared by finally on both success and failure paths. The next getAll() after a write hits Redis once to fetch the committed state, which is acceptable since writes only occur during init and rare manual reinspection. * fix: Derive local snapshot TTL from MCP_REGISTRY_CACHE_TTL config Use cacheConfig.MCP_REGISTRY_CACHE_TTL (default 5000ms) instead of a hardcoded 5s constant. When TTL is 0 (operator explicitly wants no caching), the snapshot is disabled entirely — every getAll() hits Redis. * fix: Add TTL expiry test, document 2×TTL staleness, clarify comments - Add missing test for snapshot TTL expiry path (force-expire via localSnapshotExpiry mutation, verify Redis is hit again) - Document 2×TTL max cross-instance staleness in localSnapshot JSDoc - Document reset() intentionally bypasses withWriteLock - Add inline comments explaining why early invalidateLocalSnapshot() in write callbacks is distinct from the finally-block cleanup - Update cacheConfig.MCP_REGISTRY_CACHE_TTL JSDoc to reflect both use sites and the staleness implication - Rename misleading test name for snapshot reference immutability - Add epoch sentinel comment on localSnapshotExpiry initialization	2026-03-26 16:39:09 -04:00
Danny Avila	8e2721011e	🔑 fix: Robust MCP OAuth Detection in Tool-Call Flow (#12418) ... * fix(api): add buildOAuthToolCallName utility for MCP OAuth flows Extract a shared utility that builds the synthetic tool-call name used during MCP OAuth flows (oauth_mcp_{normalizedServerName}). Uses startsWith on the raw serverName (not the normalized form) to guard against double-wrapping, so names that merely normalize to start with oauth_mcp_ (e.g., oauth@mcp@server) are correctly prefixed while genuinely pre-wrapped names are left as-is. Add 8 unit tests covering normal names, pre-wrapped names, _mcp_ substrings, special characters, non-ASCII, and empty string inputs. * fix(backend): use buildOAuthToolCallName in MCP OAuth flows Replace inline tool-call name construction in both reconnectServer (MCP.js) and createOAuthEmitter (ToolService.js) with the shared buildOAuthToolCallName utility. Remove unused normalizeServerName import from ToolService.js. Fix import ordering in both files. This ensures the oauth_mcp_ prefix is consistently applied so the client correctly identifies MCP OAuth flows and binds the CSRF cookie to the right server. * fix(client): robust MCP OAuth detection and split handling in ToolCall - Fix split() destructuring to preserve tail segments for server names containing _mcp_ (e.g., foo_mcp_bar no longer truncated to foo). - Add auth URL redirect_uri fallback: when the tool-call name lacks the _mcp_ delimiter, parse redirect_uri for the MCP callback path. Set function_name to the extracted server name so progress text shows the server, not the raw tool-call ID. - Display server name instead of literal "oauth" as function_name, gated on auth presence to avoid misidentifying real tools named "oauth". - Consolidate three independent new URL(auth) parses into a single parsedAuthUrl useMemo shared across detection, actionId, and authDomain hooks. - Replace any type on ProgressText test mock with structural type. - Add 8 tests covering delimiter detection, multi-segment names, function_name display, redirect_uri fallback, normalized _mcp_ server names, and non-MCP action auth exclusion. * chore: fix import order in utils.test.ts * fix(client): drop auth gate on OAuth displayName so completed flows show server name The createOAuthEnd handler re-emits the toolCall delta without auth, so auth is cleared on the client after OAuth completes. Gating displayName on `func === 'oauth' && auth` caused completed OAuth steps to render "Completed oauth" instead of "Completed my-server". Remove the `&& auth` gate — within the MCP delimiter branch the func="oauth" check alone is sufficient. Also remove `auth` from the useMemo dep array since only `parsedAuthUrl` is referenced. Update the test to assert correct post-completion display.	2026-03-26 14:45:13 -04:00
Danny Avila	359cc63b41	⚡ refactor: Use in-memory cache for App MCP configs to avoid Redis SCAN (#12410) ... * ⚡ perf: Use in-memory cache for App MCP configs to avoid Redis SCAN The 'App' namespace holds static YAML-loaded configs identical on every instance. Storing them in Redis and retrieving via SCAN + batch-GET caused 60s+ stalls under concurrent load (#11624). Since these configs are already loaded into memory at startup, bypass Redis entirely by always returning ServerConfigsCacheInMemory for the 'App' namespace. * ♻️ refactor: Extract APP_CACHE_NAMESPACE constant and harden tests - Extract magic string 'App' to a shared `APP_CACHE_NAMESPACE` constant used by both ServerConfigsCacheFactory and MCPServersRegistry - Document that `leaderOnly` is ignored for the App namespace - Reset `cacheConfig.USE_REDIS` in test `beforeEach` to prevent ordering-dependent flakiness - Fix import order in test file (longest to shortest) * 🐛 fix: Populate App cache on follower instances in cluster mode In cluster deployments, only the leader runs MCPServersInitializer to inspect and cache MCP server configs. Followers previously read these from Redis, but with the App namespace now using in-memory storage, followers would have an empty cache. Add populateLocalCache() so follower processes independently initialize their own in-memory App cache from the same YAML configs after the leader signals completion. The method is idempotent — if the cache is already populated (leader case), it's a no-op. * 🐛 fix: Use static flag for populateLocalCache idempotency Replace getAllServerConfigs() idempotency check with a static localCachePopulated flag. The previous check merged App + DB caches, causing false early returns in deployments with publicly shared DB configs, and poisoned the TTL read-through cache with stale results. The static flag is zero-cost (no async/Redis/DB calls), immune to DB config interference, and is reset alongside hasInitializedThisProcess in resetProcessFlag() for test teardown. Also set localCachePopulated=true after leader initialization completes, so subsequent calls on the leader don't redundantly re-run populateLocalCache. * 📝 docs: Document process-local reset() semantics for App cache With the App namespace using in-memory storage, reset() only clears the calling process's cache. Add JSDoc noting this behavioral change so callers in cluster deployments know each instance must reset independently. * ✅ test: Add follower cache population tests for MCPServersInitializer Cover the populateLocalCache code path: - Follower populates its own App cache after leader signals completion - localCachePopulated flag prevents redundant re-initialization - Fresh follower process independently initializes all servers * 🧹 style: Fix import order to longest-to-shortest convention * 🔬 test: Add Redis perf benchmark to isolate getAll() bottleneck Benchmarks that run against a live Redis instance to measure: 1. SCAN vs batched GET phases independently 2. SCAN cost scaling with total keyspace size (noise keys) 3. Concurrent getAll() at various concurrency levels (1/10/50/100) 4. Alternative: single aggregate key vs SCAN+GET 5. Alternative: raw MGET vs Keyv batch GET (serialization overhead) Run with: npx jest --config packages/api/jest.config.mjs \ --testPathPatterns="perf_benchmark" --coverage=false * ⚡ feat: Add aggregate-key Redis cache for MCP App configs ServerConfigsCacheRedisAggregateKey stores all configs under a single Redis key, making getAll() a single GET instead of SCAN + N GETs. This eliminates the O(keyspace_size) SCAN that caused 60s+ stalls in large deployments while preserving cross-instance visibility — all instances read/write the same Redis key, so reinspection results propagate automatically after readThroughCache TTL expiry. * ♻️ refactor: Use aggregate-key cache for App namespace in factory Update ServerConfigsCacheFactory to return ServerConfigsCacheRedisAggregateKey for the App namespace when Redis is enabled, instead of ServerConfigsCacheInMemory. This preserves cross-instance visibility (reinspection results propagate through Redis) while eliminating SCAN. Non-App namespaces still use the standard per-key ServerConfigsCacheRedis. * 🗑️ revert: Remove populateLocalCache — no longer needed with aggregate key With App configs stored under a single Redis key (aggregate approach), followers read from Redis like before. The populateLocalCache mechanism and its localCachePopulated flag are no longer necessary. Also reverts the process-local reset() JSDoc since reset() is now cluster-wide again via Redis. * 🐛 fix: Add write mutex to aggregate cache and exclude perf benchmark from CI - Add promise-based write lock to ServerConfigsCacheRedisAggregateKey to prevent concurrent read-modify-write races during parallel initialization (Promise.allSettled runs multiple addServer calls concurrently, causing last-write-wins data loss on the aggregate key) - Rename perf benchmark to cache_integration pattern so CI skips it (requires live Redis) * 🔧 fix: Rename perf benchmark to *.manual.spec.ts to exclude from all CI The cache_integration pattern is picked up by test:cache-integration:mcp in CI. Rename to *.manual.spec.ts which isn't matched by any CI runner. * ✅ test: Add cache integration tests for ServerConfigsCacheRedisAggregateKey Tests against a live Redis instance covering: - CRUD operations (add, get, update, remove) - getAll with empty/populated cache - Duplicate add rejection, missing update/remove errors - Concurrent write safety (20 parallel adds without data loss) - Concurrent read safety (50 parallel getAll calls) - Reset clears all configs * 🔧 fix: Rename perf benchmark to *.manual.spec.ts to exclude from all CI The perf benchmark file was renamed to *.manual.spec.ts but no testPathIgnorePatterns existed for that convention. Add .*manual\.spec\. to both test and test:ci scripts, plus jest.config.mjs, so manual-only tests never run in CI unit test jobs. * fix: Address review findings for aggregate key cache - Add successCheck() to all write paths (add/update/remove) so Redis SET failures throw instead of being silently swallowed - Override reset() to use targeted cache.delete(AGGREGATE_KEY) instead of inherited SCAN-based cache.clear() — consistent with eliminating SCAN operations - Document cross-instance write race invariant in class JSDoc: the promise-based writeLock is process-local only; callers must enforce single-writer semantics externally (leader-only init) - Use definite-assignment assertion (let resolve!:) instead of non-null assertion at call site - Fix import type convention in integration test - Verify Promise.allSettled rejections explicitly in concurrent write test - Fix broken run command in benchmark file header * style: Fix import ordering per AGENTS.md convention Local/project imports sorted longest to shortest. * chore: Update import ordering and clean up unused imports in MCPServersRegistry.ts * chore: import order * chore: import order	2026-03-26 14:44:31 -04:00
Danny Avila	4b6d68b3b5	🎛️ feat: DB-Backed Per-Principal Config System (#12354) ... * ✨ feat: Add Config schema, model, and methods for role-based DB config overrides Add the database foundation for principal-based configuration overrides (user, group, role) in data-schemas. Includes schema with tenantId and tenant isolation, CRUD methods, and barrel exports. * 🔧 fix: Add shebang and enforce LF line endings for git hooks The pre-commit hook was missing #!/bin/sh, and core.autocrlf=true was converting it to CRLF, both causing "Exec format error" on Windows. Add .gitattributes to force LF for .husky/* and *.sh files. * ✨ feat: Add admin config API routes with section-level capability checks Add /api/admin/config endpoints for managing per-principal config overrides (user, group, role). Handlers in @librechat/api use DI pattern with section-level hasConfigCapability checks for granular access control. Supports full overrides replacement, per-field PATCH via dot-paths, field deletion, toggle active, and listing. * 🐛 fix: Move deleteConfigField fieldPath from URL param to request body The path-to-regexp wildcard syntax (:fieldPath(*)) is not supported by the version used in Express. Send fieldPath in the DELETE request body instead, which also avoids URL-encoding issues with dotted paths. * ✨ feat: Wire config resolution into getAppConfig with override caching Add mergeConfigOverrides utility in data-schemas for deep-merging DB config overrides into base AppConfig by priority order. Update getAppConfig to query DB for applicable configs when role/userId is provided, with short-TTL caching and a hasAnyConfigs feature flag for zero-cost when no DB configs exist. Also: add unique compound index on Config schema, pass userId from config middleware, and signal config changes from admin API handlers. * 🔄 refactor: Extract getAppConfig logic into packages/api as TS service Move override resolution, caching strategy, and signalConfigChange from api/server/services/Config/app.js into packages/api/src/app/appConfigService.ts using the DI factory pattern (createAppConfigService). The JS file becomes a thin wiring layer injecting loadBaseConfig, cache, and DB dependencies. * 🧹 chore: Rename configResolution.ts to resolution.ts * ✨ feat: Move admin types & capabilities to librechat-data-provider Move SystemCapabilities, CapabilityImplications, and utility functions (hasImpliedCapability, expandImplications) from data-schemas to data-provider so they are available to external consumers like the admin panel without a data-schemas dependency. Add API-friendly admin types: TAdminConfig, TAdminSystemGrant, TAdminAuditLogEntry, TAdminGroup, TAdminMember, TAdminUserSearchResult, TCapabilityCategory, and CAPABILITY_CATEGORIES. data-schemas re-exports these from data-provider and extends with config-schema-derived types (ConfigSection, SystemCapability union). Bump version to 0.8.500. * feat: Add JSON-serializable admin config API response types to data-schemas Add AdminConfig, AdminConfigListResponse, AdminConfigResponse, and AdminConfigDeleteResponse types so both LibreChat API handlers and the admin panel can share the same response contract. Bump version to 0.0.41. * refactor: Move admin capabilities & types from data-provider to data-schemas SystemCapabilities, CapabilityImplications, utility functions, CAPABILITY_CATEGORIES, and admin API response types should not be in data-provider as it gets compiled into the frontend bundle, exposing the capability surface. Moved everything to data-schemas (server-only). All consumers already import from @librechat/data-schemas, so no import changes needed elsewhere. Consolidated duplicate AdminConfig type (was in both config.ts and admin.ts). * chore: Bump @librechat/data-schemas to 0.0.42 * refactor: Reorganize admin capabilities into admin/ and types/admin.ts Split systemCapabilities.ts following data-schemas conventions: - Types (BaseSystemCapability, SystemCapability, AdminConfig, etc.) → src/types/admin.ts - Runtime code (SystemCapabilities, CapabilityImplications, utilities) → src/admin/capabilities.ts Revert data-provider version to 0.8.401 (no longer modified). * chore: Fix import ordering, rename appConfigService to service - Rename app/appConfigService.ts → app/service.ts (directory provides context) - Fix import order in admin/config.ts, types/admin.ts, types/config.ts - Add naming convention to AGENTS.md * feat: Add DB base config support (role/__base__) - Add BASE_CONFIG_PRINCIPAL_ID constant for reserved base config doc - getApplicableConfigs always includes __base__ in queries - getAppConfig queries DB even without role/userId when DB configs exist - Bump @librechat/data-schemas to 0.0.43 * fix: Address PR review issues for admin config - Add listAllConfigs method; listConfigs endpoint returns all active configs instead of only __base__ - Normalize principalId to string in all config methods to prevent ObjectId vs string mismatch on user/group lookups - Block __proto__ and all dunder-prefixed segments in field path validation to prevent prototype pollution - Fix configVersion off-by-one: default to 0, guard pre('save') with !isNew, use $inc on findOneAndUpdate - Remove unused getApplicableConfigs from admin handler deps * fix: Enable tree-shaking for data-schemas, bump packages - Switch data-schemas Rollup output to preserveModules so each source file becomes its own chunk; consumers (admin panel) can now import just the modules they need without pulling in winston/mongoose/etc. - Add sideEffects: false to data-schemas package.json - Bump data-schemas to 0.0.44, data-provider to 0.8.402 * feat: add capabilities subpath export to data-schemas Adds `@librechat/data-schemas/capabilities` subpath export so browser consumers can import BASE_CONFIG_PRINCIPAL_ID and capability constants without pulling in Node.js-only modules (winston, async_hooks, etc.). Bump version to 0.0.45. * fix: include dist/ in data-provider npm package Add explicit files field so npm includes dist/types/ in the published package. Without this, the root .gitignore exclusion of dist/ causes npm to omit type declarations, breaking TypeScript consumers. * chore: bump librechat-data-provider to 0.8.403 * feat: add GET /api/admin/config/base for raw AppConfig Returns the full AppConfig (YAML + DB base merged) so the admin panel can display actual config field values and structure. The startup config endpoint (/api/config) returns TStartupConfig which is a different shape meant for the frontend app. * chore: imports order * fix: address code review findings for admin config Critical: - Fix clearAppConfigCache: was deleting from wrong cache store (CONFIG_STORE instead of APP_CONFIG), now clears BASE and HAS_DB_CONFIGS keys - Eliminate race condition: patchConfigField and deleteConfigField now use atomic MongoDB $set/$unset with dot-path notation instead of read-modify-write cycles, removing the lost-update bug entirely - Add patchConfigFields and unsetConfigField atomic DB methods Major: - Reorder cache check before principal resolution in getAppConfig so getUserPrincipals DB query only fires on cache miss - Replace '' as ConfigSection with typed BROAD_CONFIG_ACCESS constant - Parallelize capability checks with Promise.all instead of sequential awaits in for loops - Use loose equality (== null) for cache miss check to handle both null and undefined returns from cache implementations - Set HAS_DB_CONFIGS_KEY to true on successful config fetch Minor: - Remove dead pre('save') hook from config schema (all writes use findOneAndUpdate which bypasses document hooks) - Consolidate duplicate type imports in resolution.ts - Remove dead deepGet/deepSet/deepUnset functions (replaced by atomic ops) - Add .sort({ priority: 1 }) to getApplicableConfigs query - Rename _impliedBy to impliedByMap * fix: self-referencing BROAD_CONFIG_ACCESS constant * fix: replace type-cast sentinel with proper null parameter Update hasConfigCapability to accept ConfigSection | null where null means broad access check (MANAGE_CONFIGS or READ_CONFIGS only). Removes the '' as ConfigSection type lie from admin config handlers. * fix: remaining review findings + add tests - listAllConfigs accepts optional { isActive } filter so admin listing can show inactive configs (#9) - Standardize session application to .session(session ?? null) across all config DB methods (#15) - Export isValidFieldPath and getTopLevelSection for testability - Add 38 tests across 3 spec files: - config.spec.ts (api): path validation, prototype pollution rejection - resolution.spec.ts: deep merge, priority ordering, array replacement - config.spec.ts (data-schemas): full CRUD, ObjectId normalization, atomic $set/$unset, configVersion increment, toggle, __base__ query * fix: address second code review findings - Fix cross-user cache contamination: overrideCacheKey now handles userId-without-role case with its own cache key (#1) - Add broad capability check before DB lookup in getConfig to prevent config existence enumeration (#2/#3) - Move deleteConfigField fieldPath from request body to query parameter for proxy/load balancer compatibility (#5) - Derive BaseSystemCapability from SystemCapabilities const instead of manual string union (#6) - Return 201 on upsert creation, 200 on update (#11) - Remove inline narration comments per AGENTS.md (#12) - Type overrides as Partial<TCustomConfig> in DB methods and handler deps (#13) - Replace double as-unknown-as casts in resolution.ts with generic deepMerge<T> (#14) - Make override cache TTL injectable via AppConfigServiceDeps (#16) - Add exhaustive never check in principalModel switch (#17) * fix: remaining review findings — tests, rename, semantics - Rename signalConfigChange → markConfigsDirty with JSDoc documenting the stale-window tradeoff and overrideCacheTtl knob - Fix DEFAULT_OVERRIDE_CACHE_TTL naming convention - Add createAppConfigService tests (14 cases): cache behavior, feature flag, cross-user key isolation, fallback on error, markConfigsDirty - Add admin handler integration tests (13 cases): auth ordering, 201/200 on create/update, fieldPath from query param, markConfigsDirty calls, capability checks * fix: global flag corruption + empty overrides auth bypass - Remove HAS_DB_CONFIGS_KEY=false optimization: a scoped query returning no configs does not mean no configs exist globally. Setting the flag false from a per-principal query short-circuited all subsequent users. - Add broad manage capability check before section checks in upsertConfigOverrides: empty overrides {} no longer bypasses auth. * test: add regression and invariant tests for config system Regression tests: - Bug 1: User A's empty result does not short-circuit User B's overrides - Bug 2: Empty overrides {} returns 403 without MANAGE_CONFIGS Invariant tests (applied across ALL handlers): - All 5 mutation handlers call markConfigsDirty on success - All 5 mutation handlers return 401 without auth - All 5 mutation handlers return 403 without capability - All 3 read handlers return 403 without capability * fix: third review pass — all findings addressed Service (service.ts): - Restore HAS_DB_CONFIGS=false for base-only queries (no role/userId) so deployments with zero DB configs skip DB queries (#1) - Resolve cache once at factory init instead of per-invocation (#8) - Use BASE_CONFIG_PRINCIPAL_ID constant in overrideCacheKey (#10) - Add JSDoc to clearAppConfigCache documenting stale-window (#4) - Fix log message to not say "from YAML" (#14) Admin handlers (config.ts): - Use configVersion===1 for 201 vs 200, eliminating TOCTOU race (#2) - Add Array.isArray guard on overrides body (#5) - Import CapabilityUser from capabilities.ts, remove duplicate (#6) - Replace as-unknown-as cast with targeted type assertion (#7) - Add MAX_PATCH_ENTRIES=100 cap on entries array (#15) - Reorder deleteConfigField to validate principalType first (#12) - Export CapabilityUser from middleware/capabilities.ts DB methods (config.ts): - Remove isActive:true from patchConfigFields to prevent silent reactivation of disabled configs (#3) Schema (config.ts): - Change principalId from Schema.Types.Mixed to String (#11) Tests: - Add patchConfigField unsafe fieldPath rejection test (#9) - Add base-only HAS_DB_CONFIGS=false test (#1) - Update 201/200 tests to use configVersion instead of findConfig (#2) * fix: add read handler 401 invariant tests + document flag behavior - Add invariant: all 3 read handlers return 401 without auth - Document on markConfigsDirty that HAS_DB_CONFIGS stays true after all configs are deleted until clearAppConfigCache or restart * fix: remove HAS_DB_CONFIGS false optimization entirely getApplicableConfigs([]) only queries for __base__, not all configs. A deployment with role/group configs but no __base__ doc gets the flag poisoned to false by a base-only query, silently ignoring all scoped overrides. The optimization is not safe without a comprehensive Config.exists() check, which adds its own DB cost. Removed entirely. The flag is now write-once-true (set when configs are found or by markConfigsDirty) and only cleared by clearAppConfigCache/restart. * chore: reorder import statements in app.js for clarity * refactor: remove HAS_DB_CONFIGS_KEY machinery entirely The three-state flag (false/null/true) was the source of multiple bugs across review rounds. Every attempt to safely set it to false was defeated by getApplicableConfigs querying only a subset of principals. Removed: HAS_DB_CONFIGS_KEY constant, all reads/writes of the flag, markConfigsDirty (now a no-op concept), notifyChange wrapper, and all tests that seeded false manually. The per-user/role TTL cache (overrideCacheTtl, default 60s) is the sole caching mechanism. On cache miss, getApplicableConfigs queries the DB. This is one indexed query per user per TTL window — acceptable for the config override use case. * docs: rewrite admin panel remaining work with current state * perf: cache empty override results to avoid repeated DB queries When getApplicableConfigs returns no configs for a principal, cache baseConfig under their override key with TTL. Without this, every user with no per-principal overrides hits MongoDB on every request after the 60s cache window expires. * fix: add tenantId to cache keys + reject PUBLIC principal type - Include tenantId in override cache keys to prevent cross-tenant config contamination. Single-tenant deployments (tenantId undefined) use '_' as placeholder — no behavior change for them. - Reject PrincipalType.PUBLIC in admin config validation — PUBLIC has no PrincipalModel and is never resolved by getApplicableConfigs, so config docs for it would be dead data. - Config middleware passes req.user.tenantId to getAppConfig. * fix: fourth review pass findings DB methods (config.ts): - findConfigByPrincipal accepts { includeInactive } option so admin GET can retrieve inactive configs (#5) - upsertConfig catches E11000 duplicate key on concurrent upserts and retries without upsert flag (#2) - unsetConfigField no longer filters isActive:true, consistent with patchConfigFields (#11) - Typed filter objects replace Record<string, unknown> (#12) Admin handlers (config.ts): - patchConfigField: serial broad capability check before Promise.all to pre-warm ALS principal cache, preventing N parallel DB calls (#3) - isValidFieldPath rejects leading/trailing dots and consecutive dots (#7) - Duplicate fieldPaths in patch entries return 400 (#8) - DEFAULT_PRIORITY named constant replaces hardcoded 10 (#14) - Admin getConfig and patchConfigField pass includeInactive to findConfigByPrincipal (#5) - Route import uses barrel instead of direct file path (#13) Resolution (resolution.ts): - deepMerge has MAX_MERGE_DEPTH=10 guard to prevent stack overflow from crafted deeply nested configs (#4) * fix: final review cleanup - Remove ADMIN_PANEL_REMAINING.md (local dev notes with Windows paths) - Add empty-result caching regression test - Add tenantId to AdminConfigDeps.getAppConfig type - Restore exhaustive never check in principalModel switch - Standardize toggleConfigActive session handling to options pattern * fix: validate priority in patchConfigField handler Add the same non-negative number validation for priority that upsertConfigOverrides already has. Without this, invalid priority values could be stored via PATCH and corrupt merge ordering. * chore: remove planning doc from PR * fix: correct stale cache key strings in service tests * fix: clean up service tests and harden tenant sentinel - Remove no-op cache delete lines from regression tests - Change no-tenant sentinel from '_' to '__default__' to avoid collision with a real tenant ID when multi-tenancy is enabled - Remove unused CONFIG_STORE from AppConfigServiceDeps * chore: bump @librechat/data-schemas to 0.0.46 * fix: block prototype-poisoning keys in deepMerge Skip __proto__, constructor, and prototype keys during config merge to prevent prototype pollution via PUT /api/admin/config overrides.	2026-03-25 19:39:29 -04:00
ESJavadex	abaf9b3e13	🗝️ fix: Resolve User-Provided API Key in Agents API Flow (#12390) ... * fix: resolve user-provided API key in Agents API flow When the Agents API calls initializeCustom, req.body follows the OpenAI-compatible format (model, messages, stream) and does not include the `key` field that the regular UI chat flow sends. Previously, getUserKeyValues was only called when expiresAt (from req.body.key) was truthy, causing the Agents API to always fail with NO_USER_KEY for custom endpoints using apiKey: "user_provided". This fix decouples the key fetch from the expiry check: - If expiresAt is present (UI flow): checks expiry AND fetches key - If expiresAt is absent (Agents API): skips expiry check, still fetches key Fixes #12389 * address review feedback from @danny-avila - Flatten nested if into two sibling statements (never-nesting style) - Add inline comment explaining why expiresAt may be absent - Add negative assertion: checkUserKeyExpiry NOT called in Agents API flow - Add regression test: expired key still throws EXPIRED_USER_KEY - Add test for userProvidesURL=true variant in Agents API flow - Remove unnecessary undefined cast in test params * fix: CI failure + address remaining review items - Fix mock leak: use mockImplementationOnce instead of mockImplementation to prevent checkUserKeyExpiry throwing impl from leaking into SSRF tests (clearAllMocks does not reset implementations) - Use ErrorTypes.EXPIRED_USER_KEY constant instead of raw string - Add test: system-defined key/URL should NOT call getUserKeyValues	2026-03-25 14:17:11 -04:00
Danny Avila	221e49222d	⚡ refactor: Fast-Fail MCP Tool Discovery on 401 for Non-OAuth Servers (#12395) ... * fix: fast-fail MCP discovery for non-OAuth servers on auth errors Always attach oauthHandler in discoverToolsInternal regardless of useOAuth flag. Previously, non-OAuth servers hitting 401 would hang for 30s because connectClient's oauthHandledPromise had no listener to emit oauthFailed, waiting until withTimeout killed it. * chore: import order	2026-03-25 13:18:02 -04:00
Marco Beretta	0c66823c26	🧩 feat: Redesign Tool Call UI with Contextual Icons, Smart Grouping, and Rich Output Rendering (#12163) ... * feat: redesign tool call UI with type-specific icons, smart grouping, and rich output rendering Replace the generic spinner/checkmark tool call UI with a modern, Cursor-inspired design: - Add per-tool-type icons (Plug for MCP, Terminal for code, Globe for web search, etc.) - Group 2+ consecutive tool calls into collapsible "Used N tools" sections - Stack unique tool icons in grouped headers with overlapping circle design - Replace raw JSON output with intelligent renderers (table, error, text) - Restructure ToolCallInfo: output first, parameters collapsible at bottom - Add shared useExpandCollapse hook for consistent animations - Add CodeWindowHeader for ExecuteCode windowed view - Remove FinishedIcon (purple checkmark) entirely * feat: display custom MCP server icons in tool calls Add useMCPIconMap hook to resolve MCP server names to their configured icon paths. ToolIcon and StackedToolIcons now accept custom icon URLs, showing actual server logos (e.g., Home Assistant, GitHub) instead of the generic Plug icon for MCP tool calls. * refactor: unify container styling across code blocks, mermaid, and tool output Replace hardcoded gray colors with theme tokens throughout: - CodeBlock: bg-gray-900/700 -> bg-surface-secondary/tertiary + border-border-light - Mermaid dialog: bg-gray-700 -> bg-surface-secondary, text-gray-200 -> text-text-secondary - Mermaid containers: rounded-xl -> rounded-lg, remove shadow-md for consistency - ResultSwitcher: bg-gray-700 -> bg-surface-secondary with border separator - RunCode: hover:bg-gray-700 -> hover:bg-surface-hover - ErrorOutput: add border for visual consistency - MermaidHeader/CodeWindowHeader: consistent focus outlines using border-heavy * refactor: simplify tool output to plain text, remove custom renderers Remove over-engineered tool output system (TableOutput, ErrorOutput, detectOutputType) in favor of simple text extraction. Tool output now extracts the text content from MCP content blocks and displays it as clean readable text — no tables, no error styling, no JSON formatting. Parameters only show key-value badges for simple objects; complex JSON is hidden instead of dumped raw. Matches Cursor-style simplicity. * fix: handle error messages and format JSON arrays in tool output - Strip verbose MCP error prefixes (Error: [MCP][server][tool] tool call failed: Error POSTing...) and show just the meaningful error message - Display errors in red text - Format uniform JSON arrays as readable lists (name — path) instead of raw JSON dumps - Format plain JSON objects as key: value lines * feat: improve JSON display in tool call output and parameters - Replace flat formatObject with recursive formatValue for proper indented display of nested JSON structures - Add ComplexInput component for tool parameters with nested objects, arrays, or long strings (previously hidden) - Broaden hasParams check to show parameters for all object types - Add font-mono to output renderer for better alignment * feat: add localization keys for tool errors, web search, and code UI * refactor: move Mermaid components into dedicated directory module * refactor: extract CodeBar, FloatingCodeBar, and copy utilities from CodeBlock * feat: replace manual SVG icons with @icons-pack/react-simple-icons Supports 50+ programming languages with tree-shaken brand icons instead of hand-crafted SVGs for 19 languages. * refactor: simplify code execution UI with persistent code toggle * refactor: use useExpandCollapse hook in Thinking and Reasoning * feat: improve tool call error states, subtitles, and group summaries * feat: redesign web search with inline source display * feat: improve agent handoff with keyboard accessibility * feat: reorganize exports order in hooks and utils * refactor: unify CopyCodeButton with animated icon transitions and iconOnly support * feat: add run code state machine with animated success/error feedback * refactor: improve ResultSwitcher with lucide icons and accessibility * refactor: update CopyButton component * refactor: replace CopyCodeButton with CopyButton component across multiple files * test: add ImageGen test stubs * test: add RetrievalCall test stubs * feat: merge ImageGen with ToolIcon and localized progress text * feat: modernize RetrievalCall with ToolIcon and collapsible output * test: add getToolIconType action delimiter tests * test: add ImageGen collapsible output tests * feat: add action ToolIcon type with Zap icon * fix: replace AgentHandoff div with semantic button * feat: add aria-live regions to tool components * feat: redesign execute_code tool UI with syntax highlighting and language icons - Remove filename labels (script.py, main.rs) and line counter from CodeWindowHeader - Replace generic FileCode icon with language-specific LangIcon - Add syntax highlighting via highlight.js to code blocks - Add SquareTerminal icon to ExecuteCode progress text - Use shared CopyButton component in CodeWindowHeader - Remove active:scale-95 press animation from CopyButton and RunCode * feat: dynamic tool status text sizing based on markdown font-size variable - Add tool-status-text CSS class using calc(0.9 * --markdown-font-size) - Update progress-text-wrapper to use dynamic sizing instead of base size - Apply tool-status-text to WebSearch, ToolCallGroup, AgentHandoff, ImageGen - Replace hardcoded text-sm/text-xs with dynamic class across all tools - Animate chevron rotation in ProgressText and ToolCallGroup - Update subtitle text color from tertiary to secondary * fix: consistent spacing and text styles across all tool components - Standardize tool status row spacing to my-1/my-1.5 across all components - Update ToolCallInfo text from tertiary to secondary, add vertical padding - Animate ToolCallInfo parameters chevron rotation - Update OutputRenderer link colors from tertiary to secondary * feat: unify tool call grouping for all tool types All consecutive tool calls (MCP, execute_code, web_search, image_gen, file_search, code_interpreter) are now grouped under a single collapsible "Used N tools" header instead of only grouping generic tool calls. - Remove SPECIAL_TOOL_NAMES blacklist from groupToolCalls - Replace getToolCallData with getToolMeta to handle all tool types - Use renderPart callback in ToolCallGroup for proper component routing - Add file_search and code_interpreter mappings to getToolIconType * feat: friendly tool group labels, more icons, and output copy button - Show friendly names in group summary (Code, Web Search, Image Generation) instead of raw tool names - Display MCP server names instead of individual function names - Deduplicate labels and show up to 3 with +N overflow - Increase stacked icons from 3 to 4 - Add icon-only copy button to tool output (OutputRenderer) * fix: execute_code spacing and syntax-highlighted code visibility Match ToolCall spacing by using my-1.5 on status line and moving my-2 inside overflow-hidden. Replace broken hljs.highlight() with lowlight (same engine used by rehype-highlight for markdown code blocks) to render syntax-highlighted code as React elements. Handle object args in useParseArgs to support both string and Record arg formats. * feat: replace showCode with auto-expand tools setting Replace the execute_code-only "Always show code when using code interpreter" global toggle with a new "Auto-expand tool details" setting that controls all tool types. Each tool instance now uses independent local state initialized from the setting, so expanding one tool no longer affects others. Applies to ToolCall, ExecuteCode, ToolCallGroup, and CodeAnalyze components. * fix: apply auto-expand tools setting to WebSearch and RetrievalCall * fix: only auto-expand tools when content is available Defer auto-expansion until tool output or content arrives, preventing empty bordered containers from showing while tools are still running. Uses useEffect to expand when output becomes available during streaming. * feat: redesign file_search tool output, citations, and file preview - Redesign RetrievalCall with per-file cards using OutputRenderer (truncated content with show more/less, copy button) matching MCP tool pattern - Route file_search tool calls from Agents API to RetrievalCall instead of generic ToolCall - Add FilePreviewDialog for viewing files (PDF iframe, text content) with download option, opened from clickable filenames - Redesign file citations: FileText icon in badge, relevance and page numbers in hovercard, click opens file preview instead of downloading - Add file preview to message file attachments (Files.tsx) - Fix hovercard animation to slide top-to-bottom and dismiss instantly on file click to prevent glitching over dialog - Add localization keys for relevance, extracted content, preview - Add top margin to ToolCallGroup * chore: remove leftover .planning files * fix: polish FilePreviewDialog, CodeBlock, LangIcon, and Sources * fix: prevent keyboard focus on collapsed tool content Add inert attribute to all expand/collapse wrapper divs so collapsed content is removed from tab order and hidden from assistive technology. Skip disabled ProgressText buttons from tab order with tabIndex={-1}. * feat: integrate file metadata into file_search UI Pass fileType (MIME) and fileBytes from backend file records through to the frontend. Add file-type-specific icons, file size display, pages sorted by relevance, multi-snippet content per file, smart preview detection by MIME type, and copy button in file preview dialog. * fix: review fixes — inverted type check, wrong dimension, missing import, fail-open perms, timer leaks, dead code cleanup * fix: update CodeBlock styling for improved visual consistency * fix(chat): open composite file citations in preview * fix(chat): restore file previews for parsed search results * chore(git): ignore bg-shell artifacts * fix(chat): restore readable code content in light theme * style(chat): align code and output surfaces by theme * chore(i18n): remove 6 unused translation keys * fix(deps): replace private registry URL with public npm registry in lockfile * fix: CI lint, build, and test failures - Add missing scaleImage utility (fixes Vite build error) - Export scaleImage from utils/index.ts - Remove unused imports from Part.tsx (FunctionToolCall, CodeToolCall, Agents) - Fix prettier formatting in Part.tsx (multi-line → single-line imports, conditions) - Remove excess blank lines in Part.tsx - Remove unused CodeEditorRef import from Artifacts.tsx - Add useProgress mock to OpenAIImageGen.test.tsx - Add scaleImage mock to OpenAIImageGen.test.tsx - Update OpenAIImageGen tests to match redesigned component structure - Remove dead collapsible output panel tests from ImageGen.test.tsx - Add @icons-pack/react-simple-icons to Jest transformIgnorePatterns (ESM fix) * refactor: reorganize imports order across multiple components for consistency * fix: add scaleImage tests, delete dead ImageGen wrapper, wire up onUIAction in ToolCallInfo - Add 7 unit tests for scaleImage utility covering null ref, scaling, no-upscale, height clamping, landscape, and panoramic images - Delete unused Content/ImageGen.tsx re-export wrapper (ImageGen is imported from Parts/OpenAIImageGen via the Parts barrel) - Wire up onUIAction in ToolCallInfo to use handleUIAction + ask from useMessagesOperations, matching UIResourceCarousel's behavior (was previously a silent no-op) * refactor: optimize imports and enhance lazy loading for language icons * fix: address review findings for tool call UI redesign - Fix unstable array-index keys in ToolCallGroup (streaming state corruption) - Add plain-text fallback in InputRenderer for non-JSON tool args - Localize FRIENDLY_NAMES via translation keys instead of hardcoded English - Guard autoCollapse against user-initiated manual expansion - Fix CODE_INTERPRETER hasOutput to check actual outputs instead of hardcoding true - Add logger.warn for Citations fail-closed behavior on permission errors - Add Terminal icon to CodeAnalyze ProgressText for visual consistency - Fix getMCPServerName to use indexOf instead of fragile split - Use useLayoutEffect for inert attribute in useExpandCollapse (a11y) - Memoize style object in useExpandCollapse to avoid defeating React.memo - Memoize groupSequentialToolCalls in ContentParts to avoid recomputation - Use source.link as stable key instead of array index in WebSearch - Hoist rehypePlugins outside CodeMarkdown to prevent per-render recreation * fix: revert useMemo after conditional returns in ContentParts The useMemo placed after early returns violated React Rules of Hooks — hook call count would change when transitioning between edit/view mode. Reverted to the original plain forEach which is correct and equally performant since content changes on every streaming token anyway. * chore: remove unused com_ui_variables_info translation key * fix: update tests and jest config for ESM compatibility after rebase - Add ESM-only packages to transformIgnorePatterns (@dicebear, unified ecosystem, react-dnd, lowlight, etc.) to fix Jest parse failures introduced by dev rebase - Update ToolCall.test.tsx to match new component API (CSS expand/collapse instead of conditional rendering, simplified props) - Update ToolCallInfo.test.tsx to mock OutputRenderer (avoids ESM chain), align with current component interface (input/output/attachments) * refactor: replace @icons-pack/react-simple-icons with inline SVGs Inline the 51 Simple Icons SVG paths used by LangIcon directly into langIconPaths.ts, eliminating the runtime dependency on @icons-pack/react-simple-icons (which requires Node >= 24). - LangIcon now renders a plain <svg> with the path data instead of lazy-loading React components from the package - Removes Suspense/React.lazy overhead for code block language icons - SVG paths sourced from Simple Icons (CC0 1.0 license) - Package kept in package.json for now (will be removed separately) * fix: replace Plug icon with Wrench for MCP tools, remove unused i18n keys - MCP tools without a custom iconPath now show Wrench instead of Plug, matching the generic tool fallback and avoiding the "plugin" metaphor - Remove unused translation keys: com_assistants_action_attempt, com_assistants_attempt_info, com_assistants_domain_info, com_ui_ui_resources * fix: address second review findings - Combine 3x getToolMeta loop into single toolMetadata pass (ToolCallGroup) - Extract sortPagesByRelevance to shared util (was duplicated in FilePreviewDialog and RetrievalCall) - Deduplicate AGENT_STYLE_TOOLS Set (export from OpenAIImageGen/index.ts) - Localize "source/sources" in WebSearch aria-label - Add autoExpand useEffect to CodeAnalyze for live setting changes - Log download errors in FilePreviewDialog instead of silently swallowing - Replace @ts-ignore with @ts-expect-error + explanation in Code.tsx - Remove dead currentContent alias in CodeMarkdown * chore: remove @icons-pack/react-simple-icons dependency from package.json and package-lock.json - Deleted the @icons-pack/react-simple-icons entry from both package.json and package-lock.json, following the previous refactor to use inline SVGs for icons. * fix: address triage audit findings - Remove unused gIdx variable (ESLint error) - Fix singular/plural in web search sources aria-label - Separate inline type import in ToolCallGroup per AGENTS.md * fix: remove invalid placeholderDimensions prop from Image component * chore: import order * chore: import order * fix: resolve TypeScript errors in PR-touched files - Remove non-existent placeholderDimensions prop from Image in Files.tsx - Fix localize count param type (number, not string) in WebSearch.tsx - Pass full resource object instead of partial in UIResourceCarousel.tsx - Add 'as const' to toggleSwitchConfigs localizationKey in General.tsx - Fix SearchResultData type in Citation.test.tsx - Fix TAttachment and UIResource test fixture types across test files * docs: document formatBytes difference in FilePreviewDialog The local formatBytes returns a human-readable string with units ("1.5 MB"), while ~/utils/formatBytes returns a raw number. They serve different purposes, so the local copy is retained with a JSDoc comment explaining the distinction. * fix: address remaining review items - Replace cancelled IIFE with documented ternary in OpenAIImageGen, explaining the agent vs legacy path distinction - Add .catch() fallback to loadLowlight() in useLazyHighlight — falls back to plain text if the chunk fails to load - Fix import ordering in ToolCallGroup.tsx (type imports grouped before local value imports per AGENTS.md) * fix: blob URL leak and useGetFiles over-fetch - FilePreviewDialog: add cancelledRef guard to loadPreview so blob URLs are never created after the dialog closes (prevents orphaned object URLs on unmount during async PDF fetch) - RetrievalCall: filter useGetFiles by fileIds from fileSources instead of fetching the entire user file corpus for display-only name matching * chore: fix com_nav_auto_expand_tools alphabetical order in translation.json * fix: render non-object JSON params instead of returning null in InputRenderer * refactor: render JSON tool output as syntax-highlighted code block Replace the custom YAML-ish formatValue/formatObjectArray rendering with JSON.stringify + hljs language-json styling. Structured API responses (like GitHub search results) now display as proper syntax-highlighted JSON with indentation instead of a flat key-value text dump. - Remove formatValue, formatObjectArray, isUniformObjectArray helpers - Add isJson flag to extractText return type - JSON output rendered in <code class="hljs language-json"> block - Text content blocks (type: "text") still extracted and rendered as plain text - Error output unchanged * fix: extract cancelled IIFE to named function in OpenAIImageGen Replace nested ternary with a named computeCancelled() function that documents the agent vs legacy path branching. Resolves eslint no-nested-ternary warning. --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-03-25 12:31:39 -04:00
Marco Beretta	ccd049d8ce	📁 refactor: Prompts UI (#11570) ... * style: enhance prompts UI with new components and improved structure; add CreatePromptButton and AutoSendPrompt; refactor GroupSidePanel and PromptsAccordion * refactor(Prompts): move button components to buttons/ subdirectory * refactor(Prompts): move dialog components to dialogs/ subdirectory * refactor(Prompts): move display components to display/ subdirectory * refactor(Prompts): move editor components to editor/ subdirectory * refactor(Prompts): move field components to fields/ subdirectory * refactor(Prompts): move form components to forms/ subdirectory * refactor(Prompts): move layout components to layouts/ subdirectory * refactor(Prompts): move list components to lists/ subdirectory * refactor(Prompts): move sidebar components to sidebar/ subdirectory * refactor(Prompts): move utility components to utils/ subdirectory * refactor(Prompts): update main exports and external imports * refactor(Prompts): fix class name typo in AutoSendPrompt * refactor(Prompts): reorganize exports and imports order across components * refactor(Prompts): reorder exports for better organization and clarity * refactor(Buttons): enhance prompts accessibility with aria-labels and update translations * refactor(AdminSettings): reorganize imports and improve form structure for clarity * refactor(Dialogs): reorganize imports for consistency and clarity across DeleteVersion, SharePrompt, and VariableDialog components * refactor(Dialogs): enhance prompts accessibility with aria-labels * refactor(Display): enhance prompt components and accessibility features * refactor(.gitignore): add Playwright MCP directory * refactor(Preview): enhance prompt components, improve layout, and add accessibility features * refactor(Prompts): enhance variable handling, improve accessibility, and update UI components * refactor(Prompts): enhance loading state handling and improve accessibility in PromptName component * refactor(Prompts): streamline special variable handling, improve icon management, and enhance UI components * refactor(Prompts): update AdvancedSwitch component to use Radio for mode selection, enhance PromptName with tooltips, and improve layout in PromptForm * refactor(Prompts): enhance VersionCard and VersionBadge components for improved UI and accessibility, update loading state handling in VersionsPanel * refactor(Prompts): improve layout and styling of VersionCard component for better visual alignment and clarity * refactor(DeleteVersion): update text color for confirmation prompt in DeleteConfirmDialog * refactor(Prompts): add configurations for always make production and auto-send prompts, update localization strings for clarity * refactor(Prompts): enhance layout and styling in CategorySelector, CreatePromptForm, and List components for improved responsiveness and clarity * refactor(Prompts): enhance PromptDetailHeader and ChatGroupItem components, add shared prompt indication, and remove unused PromptMetadata component * refactor(Prompts): implement prompt group usage tracking, update sorting logic, and enhance related components * fix(Prompts): security, performance, and pagination fixes - Fix cursor pagination skipping/duplicating items by including numberOfGenerations in cursor condition to match sort order - Close NoSQL injection vector via otherFilters rest spread in GET /all, GET /groups, and buildPromptGroupFilter - Validate groupId as ObjectId before passing to query (GET /) - Add prompt body validation in addPromptToGroup (type + text) - Return 404 instead of 500 for missing group in POST /use - Combine data + count into single $facet aggregation - Add compound index {numberOfGenerations, updatedAt, _id} - Add index on prompt.author for deleteUserPrompts - Update useRecordPromptUsage to refresh client caches - Replace console.error with logger.error * refactor(PromptForm): remove console warning for unselected prompt in VersionsPanel * refactor(Prompts): improve error handling for groupId and streamline usage tracking * refactor(.gitignore): add CLAUDE.md to ignore list * refactor(Prompts): streamline prompt components by removing unused variables and enhancing props structure * refactor(Prompts): fix sort stability, keyboard handling, and remove dead code Add _id tiebreaker to prompt group sort pipelines for deterministic pagination ordering. Prevent default browser scroll on Space key in PromptEditor preview mode. Remove unused blurTimeoutRef and its onMutate callback from DashGroupItem. * refactor(Prompts): enhance groupId validation and improve prompt group aggregation handling * fix: aria-hidden, API fixes, accessibility improvements * fix: ACL author filter, mobile guard, semantic HTML, and add useFocusTrap hook - Remove author filter from patchPromptGroup so ACL-granted editors can update prompt groups (aligns with deletePromptGroupController) - Add missing group guard to mobile HeaderActions in PromptForm - Replace div with article in DashGroupItem, remove redundant stopPropagation and onClick on outer container - Add useFocusTrap hook for keyboard focus management - Add numberOfGenerations to default projection - Deduplicate ObjectId validation, remove console.warn, fix aria-labelledby, localize search announcements * refactor(Prompts): adjust UI and improve a11y * refactor(Prompts): reorder imports for consistency and clarity * refactor(Prompts): implement updateFieldsInPlace for efficient data updates and add related tests * refactor(Prompts): reorder imports to include updateFieldsInPlace for better organization * refactor(Prompts): enhance DashGroupItem with toast notifications for prompt updates and add click-to-edit functionality in PromptEditor * style: use self-closing TooltipAnchor in CreatePromptButton Replace ></TooltipAnchor> with /> for consistency with the rest of the Prompts directory. * fix(i18n): replace placeholder text for com_ui_global_group translation key The value was left as 'something needs to go here. was empty' which would be visible to users as an aria-label in DashGroupItem. * fix(DashGroupItem): sync rename input with group.name on external changes nameInputValue was initialized via useState(group.name) but never synced when group.name changed from a background refetch. Added useEffect that updates the input when the dialog is closed. * perf(useFocusTrap): store onEscape in ref to avoid listener churn onEscape was in the useEffect dependency array, causing the keydown listener to be torn down and re-attached on every render when callers passed an inline function. Now stored in a ref so the effect only re-runs when active or containerRef changes. * fix(a11y): replace role=button div with layered button overlay in ListCard The card used role='button' on a div that contained nested Button elements — an invalid ARIA pattern. Replaced with a hidden button at z-0 for the card action while child interactive elements sit at z-10, eliminating nested interactive element violations. * fix(PromptForm): reset selectionIndex on route change, guard auto-save, and fix a11y - Reset selectionIndex to 0 and isEditing to false when promptId changes, preventing out-of-bounds index when navigating between groups with different version counts. - Track selectedPrompt in a ref so the auto-save effect doesn't fire against a stale prompt when the selection changed mid-edit. - Stabilize useFocusTrap onEscape via useCallback to avoid unnecessary listener re-attachment. - Conditionally render mobile overlay instead of always-present button with aria-hidden/pointer-events toggling. * refactor: extract isValidObjectIdString to shared utility in data-schemas The same regex helper was duplicated in api/server/routes/prompts.js and packages/data-schemas/src/methods/prompt.ts. Moved to packages/data-schemas/src/utils/objectId.ts and imported from both consumers. Also removed a duplicate router.use block introduced during the extraction. * perf(updateFieldsInPlace): replace JSON deep clone with targeted spread Instead of JSON.parse(JSON.stringify(data)) which serializes the entire paginated data structure, use targeted immutable spreads that only copy the affected page and collection array. Returns the original data reference unchanged when the item is not found. * perf(VariablesDropdown): memoize items array and stabilize handleAddVariable The items array containing JSX elements was rebuilt on every render. Wrapped in useMemo keyed on usedVariables and localize. Also wrapped handleAddVariable in useCallback and memoized usedCount to avoid redundant array filtering. * perf(DashGroupItem): stabilize mutation callbacks via refs handleSaveRename and handleDelete had updateGroup/deleteGroup mutation objects in their useCallback dependency arrays. Since mutation objects are new references each render, the callbacks were recreated every render, defeating memoization. Now store mutation objects in refs and call via ref.current in the callbacks. * fix(security): validate groupId in incrementPromptGroupUsage The data-schema method passed the groupId string directly to findByIdAndUpdate without validation. If called from a different entrypoint without the route-level check, Mongoose would throw a CastError. Now validates with isValidObjectIdString before the DB call and throws a clean 'Invalid groupId' error. * fix(security): add rate limiter to prompt usage tracking endpoint POST /groups/:groupId/use had no rate limiting — a user could spam it to inflate numberOfGenerations, which controls sort order for all users. Added promptUsageLimiter (30 req/user/min) following the same pattern as toolCallLimiter. Also handle 'Invalid groupId' error from the data layer in the route error handler. * fix(updateFieldsInPlace): guard against undefined identifier value If updatedItem[identifierField] is null/undefined, findIndex could match unintended items where that field is also undefined. Added early return when the identifier value is nullish. * fix(a11y): use React useId for stable unique IDs in ListCard aria-describedby/id values were derived from prompt name which can contain spaces and special characters, producing invalid HTML IDs and potential collisions. Now uses React.useId() for guaranteed unique, valid IDs per component instance. * fix: Align prompts panel styling with other sidebar panels and fix test - Match FilterPrompts first row to Memory/Bookmark pattern (items-center gap-2) - Remove items-stretch override from PromptsAccordion - Add missing promptUsageLimiter mock to prompts route test * fix: Address code review findings for prompts refactor PR - Fix #5: Gate DeletePrompt in HeaderActions behind canDelete permission - Fix #8: BackToChat navigates to last conversation instead of /c/new - Fix #7: Restore useLiveAnnouncer for screen reader feedback on delete/rename - Fix #1: Use isPublic (set by API) instead of deprecated projectIds for globe icon - Fix #4: Optimistic cache update in useRecordPromptUsage instead of full invalidation - Fix #6: Add migration to drop superseded { createdAt, updatedAt } compound index - Fix #9: Single-pass reduce in PromptVariables instead of triple filter - Fix #10: Rename PromptLabelsForm internal component to avoid collision with PromptForm - Fix #14: Remove redundant aria-label from aria-hidden Checkbox in AutoSendPrompt * fix: Align prompts panel filter row element sizes with other panels - Override Dropdown trigger to size-9 (36px) to match FilterInput height - Set CreatePromptButton to size-9 shrink-0 bg-transparent matching Memory/Bookmark panel button pattern * fix(prompts): Shared Prompts filter ignores direct shares, only returns PUBLIC Folds fix from PR #11882 into the refactored codebase. Bug A: filterAccessibleIdsBySharedLogic now accepts ownedPromptGroupIds: - MY_PROMPTS: accessible intersect owned - SHARED_PROMPTS: (accessible union public) minus owned - ALL: accessible union public (deduplicated) Legacy fallback preserved when ownedPromptGroupIds is omitted. Bug B: getPromptGroup uses $lookup aggregation to populate productionPrompt, fixing empty text on direct URL navigation to shared prompts. Also adds getOwnedPromptGroupIds to data-schemas methods and passes it from both /all and /groups route handlers. * fix: Add missing canDelete to mobile HeaderActions, remove dead instanceProjectId prop - Pass canDelete to mobile HeaderActions row (was only on desktop) - Remove instanceProjectId prop from ChatGroupItem and DashGroupItem since global check now uses group.isPublic - Remove useGetStartupConfig from List.tsx (no longer needed) * fix: Use runtime ObjectId instead of type-only Types.ObjectId, fix i18next interpolation - getPromptGroup and getOwnedPromptGroupIds were using Types.ObjectId (imported as type-only), which is erased at compile time. Use the runtime ObjectId from mongoose.Types (already destructured at line 20). This fixes the 404s in PATCH /groups/:groupId tests. - Fix com_ui_prompt_deleted_group translation to use {{0}} (i18next double-brace syntax) instead of {0}. * chore: Fix translation key ordering, add sideEffects: false to data-provider - Reorder new translation keys to maintain alphabetical order: com_ui_click_to_edit, com_ui_labels, com_ui_live, com_ui_prompt_delete_confirm, com_ui_prompt_deleted_group, com_ui_prompt_details, com_ui_prompt_renamed, com_ui_prompt_update_error, com_ui_prompt_variables_list - Add "sideEffects": false to librechat-data-provider package.json to enable tree-shaking of unused exports (types, constants, pure functions) * fix: Reduce prompts panel spacing, align memory toggle with checkbox pattern - Remove unnecessary wrapper div around AutoSendPrompt in PromptsAccordion, reducing vertical space between the toggle and the first prompt item - Replace Memory panel's Switch toggle with Checkbox+Button pattern matching the prompts panel's AutoSendPrompt for visual consistency * fix: Reduce gap between AutoSendPrompt and first prompt item Change ChatGroupItem margin from my-2 to mb-2 to eliminate the doubled spacing (gap-2 from parent + top margin from first item). Restore wrapper div around AutoSendPrompt for right-alignment. * fix: Restore prompt name on empty save, remove dead bodyProps from checkGlobalPromptShare - PromptName: reset newName to name when save is cancelled due to empty or unchanged input, preventing blank title in read mode - checkGlobalPromptShare: remove dead bodyProps config — Permissions.SHARE was not in the permissions array so the bodyProps rule was never evaluated. Per-resource share checks are handled by canAccessPromptGroupResource. --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-03-22 16:56:22 -04:00
Danny Avila	676641f3da	🔄 refactor: Migrate to react-resizable-panels v4 with Artifacts Header polish (#12356) ... * chore: Update react-resizable-panels dependency to version 4.7.4 - Upgraded the "react-resizable-panels" package in package-lock.json, package.json, and client package.json files to ensure compatibility with the latest features and improvements. - Adjusted peer dependencies for React and ReactDOM to align with the new version requirements. * refactor: Update Share and SidePanel components to `react-resizable-panels` v4 - Refactored the ShareArtifactsContainer to utilize a new layout change handler, enhancing artifact panel resizing functionality. - Updated ArtifactsPanel to use the new `usePanelRef` hook, improving panel reference management. - Simplified SidePanelGroup by removing unnecessary layout normalization and integrating default layout handling with localStorage. - Removed the deprecated `normalizeLayout` utility function to streamline the codebase. - Adjusted Resizable components to ensure consistent sizing and layout behavior across panels. * style: Enhance scrollbar appearance across application - Added custom scrollbar styles to both artifacts and markdown files, improving aesthetics and user experience. - Implemented dark mode adjustments for scrollbar visibility, ensuring consistency across different color schemes. * style: Standardize button sizes and layout in Artifacts components - Updated button dimensions to a consistent height of 9 units across various components including Artifacts, Code, and DownloadArtifact. - Adjusted padding and layout properties in the Artifacts header for improved visual consistency. - Enhanced the Radio component to accept a new `buttonClassName` prop for better customization of button styles. * chore: import order	2026-03-22 02:21:27 -04:00
Marco Beretta	733a9364c0	🎨 refactor: Redesign Sidebar with Unified Icon Strip Layout (#12013) ... * fix: Graceful SidePanelContext handling when ChatContext unavailable The UnifiedSidebar component is rendered at the Root level before ChatContext is provided (which happens only in ChatRoute). This caused an error when useSidePanelContext tried to call useChatContext before it was available. Changes: - Made SidePanelProvider gracefully handle missing ChatContext with try/catch - Changed useSidePanelContext to return a safe default instead of throwing - Prevents render error on application load and improves robustness * fix: Provide default context value for ChatContext to prevent setFilesLoading errors The ChatContext was initialized with an empty object as default, causing 'setFilesLoading is not a function' errors when components tried to call functions from the context. This fix provides a proper default context with no-op functions for all expected properties. Fixes FileRow component errors that occurred when navigating to sections with file upload functionality (Agent Builder, Attach Files, etc.). * fix: Move ChatFormProvider to Root to fix Prompts sidebar rendering The ChatFormProvider was only wrapping ChatView, but the sidebar (including Prompts) renders separately and needs access to the ChatFormContext. ChatGroupItem uses useSubmitMessage which calls useChatFormContext, causing a React error when Prompts were accessed. This fix moves the ChatFormProvider to the Root component to wrap both the sidebar and the main chat view, ensuring the form context is available throughout the entire application. * fix: Active section switching and dead code cleanup Sync ActivePanelProvider state when defaultActive prop changes so clicking a collapsed-bar icon actually switches the expanded section. Remove the now-unused hideSidePanel atom and its Settings toggle. * style: Redesign sidebar layout with optimized spacing and positioning - Remove duplicate new chat button from sidebar, keep it in main header - Reposition account settings to bottom of expanded sidebar - Simplify collapsed bar padding and alignment - Clean up unused framer-motion imports from Header - Optimize vertical space usage in expanded panel - Align search bar icon color with sidebar theme * fix: Chat history not showing in sidebar Add h-full to ConversationsSection outer div so it fills the Nav content panel, giving react-virtualized's AutoSizer a measurable height. Change Nav content panel from overflow-y-auto to overflow-hidden since the virtualized list handles its own scrolling. * refactor: Move nav icons to fixed icon strip alongside sidebar toggle Extract section icons from the Nav content panel into the ExpandedPanel icon strip, matching the CollapsedBar layout. Both states now share identical button styling and 50px width, eliminating layout shift on toggle. Nav.tsx simplified to content-only rendering. Set text-text-primary on Nav content for consistent child text color. * refactor: sidebar components and remove unused NewChat component * refactor: streamline sidebar components and introduce NewChat button * refactor: enhance sidebar functionality with expanded state management and improved layout * fix: re-implement sidebar resizing functionality with mouse events * feat: enhance sidebar layout responsiveness on mobile * refactor: remove unused components and streamline sidebar functionality * feat: enhance sidebar behavior with responsive transformations for small screens * feat: add new chat button for small screens with message cache clearing * feat: improve state management in sidebar and marketplace components * feat: enhance scrolling behavior in AgentPanel and Nav components * fix: normalize sidebar panel font sizes and default panel selection Set text-sm as base font size on the shared Nav container so all panels render consistently. Guard against empty localStorage value when restoring the active sidebar panel. * fix: adjust avatar size and class for collapsed state in AccountSettings component * style: adjust padding and class names in Nav, Parameters, and ConversationsSection components * fix: close mobile sidebar on pinned favorite selection * refactor: remove unused key in translation file * fix: Address review findings for unified sidebar - Restore ChatFormProvider per-ChatView to fix multi-conversation input isolation; add separate ChatFormProvider in UnifiedSidebar for Prompts panel access - Add inert attribute on mobile sidebar (when collapsed) and main content (when sidebar overlay is open) to prevent keyboard focus leaking - Replace unsafe `as unknown as TChatContext` cast with null-based context that throws descriptively when used outside a provider - Throttle mousemove resize handler with requestAnimationFrame to prevent React state updates at 120Hz during sidebar drag - Unify active panel state: remove split between activeSection in UnifiedSidebar and internal state in ActivePanelContext; single source of truth with localStorage sync on every write - Delete orphaned SidePanelProvider/useSidePanelContext (no consumers after SidePanel.tsx removal) - Add data-testid="new-chat-button" to NewChat component - Add includeHidePanel option to useSideNavLinks; remove no-op hidePanel callback and post-hoc filter in useUnifiedSidebarLinks - Close sidebar on first mobile visit when localStorage has no prior state - Remove unnecessary min-width/max-width CSS transitions (only width needed) - Remove dead SideNav re-export from SidePanel/index.ts - Remove duplicate aria-label from Sidebar nav element - Fix trailing blank line in mobile.css * style: fix prettier formatting in Root.tsx * fix: Address remaining review findings and re-render isolation - Extract useChatHelpers(0) into SidebarChatProvider child component so Recoil atom subscriptions (streaming tokens, latestMessage, etc.) only re-render the active panel — not the sidebar shell, resize logic, or icon strip (Finding 4) - Fix prompt pre-fill when sidebar form context differs from chat form: useSubmitMessage now reads the actual textarea DOM value via mainTextareaId as fallback for the currentText newline check (Finding 1) - Add id="close-sidebar-button" and data-testid to ExpandedPanel toggle so OpenSidebar focus management works after expand (Finding 10/N3) - Replace Dispatch<SetStateAction<number>> prop with onResizeKeyboard(direction) callback on Sidebar (Finding 13) - Fix first-mobile-visit sidebar flash: atom default now checks window.matchMedia at init time instead of defaulting to true then correcting in a useEffect; removes eslint-disable suppression (N1/N2) - Add tests for ActivePanelContext and ChatContext (Finding 8) * refactor: remove no-op memo from SidebarChatProvider memo(SidebarChatProvider) provided no memoization because its only prop (children) is inline JSX — a new reference on every parent render. The streaming isolation works through Recoil subscription scoping, not memo. Clarified in the JSDoc comment. * fix: add shebang to pre-commit hook for Windows compatibility Git on Windows cannot spawn hook scripts without a shebang line, causing 'cannot spawn .husky/pre-commit: No such file or directory'. * style: fix sidebar panel styling inconsistencies - Remove inner overflow-y-auto from AgentPanel form to eliminate double scrollbars when nested inside Nav.tsx's scroll container - Add consistent padding (px-3 py-2) to Nav.tsx panel container - Remove hardcoded 150px cell widths from Files PanelTable; widen date column from 25% to 35% so dates are no longer cut off - Compact pagination row with flex-wrap and smaller text - Add px-1 padding to Parameters panel for consistency - Change overflow-x-visible to overflow-x-hidden on Files and Bookmarks panels to prevent horizontal overflow * fix: Restore panel styling regressions in unified sidebar - Nav.tsx wrapper: remove extra px-3 padding, add hide-scrollbar class, restore py-1 to match old ResizablePanel wrapper - AgentPanel: restore scrollbar-gutter-stable and mx-1 margin (was px-1) - Parameters/Panel: restore p-3 padding (was px-1 pt-1) - Files/Panel: restore overflow-x-visible (was hidden, clipping content) - Files/PanelTable: restore 75/25 column widths, restore 150px cell width constraints, restore pagination text-sm and gap-2 - Bookmarks/Panel: restore overflow-x-visible * style: initial improvements post-sidenav change * style: update text size in DynamicTextarea for improved readability * style: update FilterPrompts alignment in PromptsAccordion for better layout consistency * style: adjust component heights and padding for consistency across SidePanel elements - Updated height from 40px to 36px in AgentSelect for uniformity - Changed button size class from "bg-transparent" to "size-9" in BookmarkTable, MCPBuilderPanel, and MemoryPanel - Added padding class "py-1.5" in DynamicDropdown for improved spacing - Reduced height from 10 to 9 in DynamicInput and DynamicTags for a cohesive look - Adjusted padding in Parameters/Panel for better layout * style: standardize button sizes and icon dimensions across chat components - Updated button size class from 'size-10' to 'size-9' in multiple components for consistency. - Adjusted icon sizes from 'icon-lg' to 'icon-md' in various locations to maintain uniformity. - Modified header height for better alignment with design specifications. * style: enhance layout consistency and component structure across various panels - Updated ActivePanelContext to utilize useCallback and useMemo for improved performance. - Adjusted padding and layout in multiple SidePanel components for better visual alignment. - Standardized icon sizes and button dimensions in AddMultiConvo and other components. - Improved overall spacing and structure in PromptsAccordion, MemoryPanel, and FilesPanel for a cohesive design. * style: standardize component heights and text sizes across various panels - Adjusted button heights from 10px to 9px in multiple components for consistency. - Updated text sizes from 'text-sm' to 'text-xs' in DynamicCheckbox, DynamicCombobox, DynamicDropdown, DynamicInput, DynamicSlider, DynamicSwitch, DynamicTags, and DynamicTextarea for improved readability. - Added portal prop to account settings menu for better rendering behavior. * refactor: optimize Tooltip component structure for performance - Introduced a new memoized TooltipPopup component to prevent unnecessary re-renders of the TooltipAnchor when the tooltip mounts/unmounts. - Updated TooltipAnchor to utilize the new TooltipPopup, improving separation of concerns and enhancing performance. - Maintained existing functionality while improving code clarity and maintainability. * refactor: improve sidebar transition handling for better responsiveness - Enhanced the transition properties in the UnifiedSidebar component to include min-width and max-width adjustments, ensuring smoother resizing behavior. - Cleaned up import statements by removing unnecessary lines for better code clarity. * fix: prevent text selection during sidebar resizing - Added functionality to disable text selection on the body while the sidebar is being resized, enhancing user experience during interactions. - Restored text selection capability once resizing is complete, ensuring normal behavior resumes. * fix: ensure Header component is always rendered in ChatView - Removed conditional rendering of the Header component to ensure it is always displayed, improving the consistency of the chat interface. * refactor: add NewChatButton to ExpandedPanel for improved user interaction - Introduced a NewChatButton component in the ExpandedPanel, allowing users to initiate new conversations easily. - Implemented functionality to clear message cache and invalidate queries upon button click, enhancing performance and user experience. - Restored import statements for OpenSidebar and PresetsMenu in Header component for better organization. --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-03-22 01:15:20 -04:00
Danny Avila	04e65bb21a	🧹 chore: Move direct model usage from PermissionsController to data-schemas	2026-03-21 15:20:15 -04:00
Danny Avila	a78865b5e0	🔄 refactor: Update Token Deletion Logic to Use AND Semantics ... - Refactored the `deleteTokens` method to delete tokens based on all provided fields using AND conditions instead of OR, enhancing precision in token management. - Updated related tests to reflect the new logic, ensuring that only tokens matching all specified criteria are deleted. - Added new test cases for deleting tokens by type and userId, and for preventing cross-user token deletions, improving overall test coverage and robustness. - Introduced a new `type` field in the `TokenQuery` interface to support the updated deletion functionality.	2026-03-21 14:45:53 -04:00
Danny Avila	150361273f	🧽 chore: Resolve TypeScript errors and test failures in agent/prompt deletion methods ... Type AclEntry model as Model<IAclEntry> instead of Model<unknown> in deleteUserAgents and deleteUserPrompts, wire getSoleOwnedResourceIds into agent.spec.ts via createAclEntryMethods, replace permissionService calls with direct AclEntry.create, and add missing principalModel field.	2026-03-21 14:30:46 -04:00
Danny Avila	7829fa9eca	🪄 refactor: Simplify MCP Tool Content Formatting to Unified String Output (#12352) ... * refactor: Simplify content formatting in MCP service and parser - Consolidated content handling in `formatToolContent` to return a plain-text string instead of an array for all providers, enhancing clarity and consistency. - Removed unnecessary checks for content array providers, streamlining the logic for handling text and image artifacts. - Updated related tests to reflect changes in expected output format, ensuring comprehensive coverage for the new implementation. * fix: Return empty string for image-only tool responses instead of '(No response)' When artifacts exist (images/UI resources) but no text content is present, return an empty string rather than the misleading '(No response)' fallback. Adds missing test assertions for image-only content and standardizes length checks to explicit `> 0` comparisons.	2026-03-21 14:28:56 -04:00
Danny Avila	b5c097e5c7	⚗️ feat: Agent Context Compaction/Summarization (#12287) ... * chore: imports/types Add summarization config and package-level summarize handler contracts Register summarize handlers across server controller paths Port cursor dual-read/dual-write summary support and UI status handling Selectively merge cursor branch files for BaseClient summary content block detection (last-summary-wins), dual-write persistence, summary block unit tests, and on_summarize_status SSE event handling with started/completed/failed branches. Co-authored-by: Cursor <cursoragent@cursor.com> refactor: type safety feat: add localization for summarization status messages refactor: optimize summary block detection in BaseClient Updated the logic for identifying existing summary content blocks to use a reverse loop for improved efficiency. Added a new test case to ensure the last summary content block is updated correctly when multiple summary blocks exist. chore: add runName to chainOptions in AgentClient refactor: streamline summarization configuration and handler integration Removed the deprecated summarizeNotConfigured function and replaced it with a more flexible createSummarizeFn. Updated the summarization handler setup across various controllers to utilize the new function, enhancing error handling and configuration resolution. Improved overall code clarity and maintainability by consolidating summarization logic. feat(summarization): add staged chunk-and-merge fallback feat(usage): track summarization usage separately from messages feat(summarization): resolve prompt from config in runtime fix(endpoints): use @librechat/api provider config loader refactor(agents): import getProviderConfig from @librechat/api chore: code order feat(app-config): auto-enable summarization when configured feat: summarization config refactor(summarization): streamline persist summary handling and enhance configuration validation Removed the deprecated createDeferredPersistSummary function and integrated a new createPersistSummary function for MongoDB persistence. Updated summarization handlers across various controllers to utilize the new persistence method. Enhanced validation for summarization configuration to ensure provider, model, and prompt are properly set, improving error handling and overall robustness. refactor(summarization): update event handling and remove legacy summarize handlers Replaced the deprecated summarization handlers with new event-driven handlers for summarization start and completion across multiple controllers. This change enhances the clarity of the summarization process and improves the integration of summarization events in the application. Additionally, removed unused summarization functions and streamlined the configuration loading process. refactor(summarization): standardize event names in handlers Updated event names in the summarization handlers to use constants from GraphEvents for consistency and clarity. This change improves maintainability and reduces the risk of errors related to string literals in event handling. feat(summarization): enhance usage tracking for summarization events Added logic to track summarization usage in multiple controllers by checking the current node type. If the node indicates a summarization task, the usage type is set accordingly. This change improves the granularity of usage data collected during summarization processes. feat(summarization): integrate SummarizationConfig into AppSummarizationConfig type Enhanced the AppSummarizationConfig type by extending it with the SummarizationConfig type from librechat-data-provider. This change improves type safety and consistency in the summarization configuration structure. test: add end-to-end tests for summarization functionality Introduced a comprehensive suite of end-to-end tests for the summarization feature, covering the full LibreChat pipeline from message creation to summarization. This includes a new setup file for environment configuration and a Jest configuration specifically for E2E tests. The tests utilize real API keys and ensure proper integration with the summarization process, enhancing overall test coverage and reliability. refactor(summarization): include initial summary in formatAgentMessages output Updated the formatAgentMessages function to return an initial summary alongside messages and index token count map. This change is reflected in multiple controllers and the corresponding tests, enhancing the summarization process by providing additional context for each agent's response. refactor: move hydrateMissingIndexTokenCounts to tokenMap utility Extracted the hydrateMissingIndexTokenCounts function from the AgentClient and related tests into a new tokenMap utility file. This change improves code organization and reusability, allowing for better management of token counting logic across the application. refactor(summarization): standardize step event handling and improve summary rendering Refactored the step event handling in the useStepHandler and related components to utilize constants for event names, enhancing consistency and maintainability. Additionally, improved the rendering logic in the Summary component to conditionally display the summary text based on its availability, providing a better user experience during the summarization process. feat(summarization): introduce baseContextTokens and reserveTokensRatio for improved context management Added baseContextTokens to the InitializedAgent type to calculate the context budget based on agentMaxContextNum and maxOutputTokensNum. Implemented reserveTokensRatio in the createRun function to allow configurable context token management. Updated related tests to validate these changes and ensure proper functionality. feat(summarization): add minReserveTokens, context pruning, and overflow recovery configurations Introduced new configuration options for summarization, including minReserveTokens, context pruning settings, and overflow recovery parameters. Updated the createRun function to accommodate these new options and added a comprehensive test suite to validate their functionality and integration within the summarization process. feat(summarization): add updatePrompt and reserveTokensRatio to summarization configuration Introduced an updatePrompt field for updating existing summaries with new messages, enhancing the flexibility of the summarization process. Additionally, added reserveTokensRatio to the configuration schema, allowing for improved management of token allocation during summarization. Updated related tests to validate these new features. feat(logging): add on_agent_log event handler for structured logging Implemented an on_agent_log event handler in both the agents' callbacks and responses to facilitate structured logging of agent activities. This enhancement allows for better tracking and debugging of agent interactions by logging messages with associated metadata. Updated the summarization process to ensure proper handling of log events. fix: remove duplicate IBalanceUpdate interface declaration perf(usage): single-pass partition of collectedUsage Replace two Array.filter() passes with a single for-of loop that partitions message vs. summarization usages in one iteration. fix(BaseClient): shallow-copy message content before mutating and preserve string content Avoid mutating the original message.content array in-place when appending a summary block. Also convert string content to a text content part instead of silently discarding it. fix(ui): fix Part.tsx indentation and useStepHandler summarize-complete handling - Fix SUMMARY else-if branch indentation in Part.tsx to match chain level - Guard ON_SUMMARIZE_COMPLETE with didFinalize flag to avoid unnecessary re-renders when no summarizing parts exist - Protect against undefined completeData.summary instead of unsafe spread fix(agents): use strict enabled check for summarization handlers Change summarizationConfig?.enabled !== false to === true so handlers are not registered when summarizationConfig is undefined. chore: fix initializeClient JSDoc and move DEFAULT_RESERVE_RATIO to module scope refactor(Summary): align collapse/expand behavior with Reasoning component - Single render path instead of separate streaming vs completed branches - Use useMessageContext for isSubmitting/isLatestMessage awareness so the "Summarizing..." label only shows during active streaming - Default to collapsed (matching Reasoning), user toggles to expand - Add proper aria attributes (aria-hidden, role, aria-controls, contentId) - Hide copy button while actively streaming feat(summarization): default to self-summarize using agent's own provider/model When no summarization config is provided (neither in librechat.yaml nor on the agent), automatically enable summarization using the agent's own provider and model. The agents package already provides default prompts, so no prompt configuration is needed. Also removes the dead resolveSummarizationLLMConfig in summarize.ts (and its spec) — run.ts buildAgentContext is the single source of truth for summarization config resolution. Removes the duplicate RuntimeSummarizationConfig local type in favor of the canonical SummarizationConfig from data-provider. chore: schema and type cleanup for summarization - Add trigger field to summarizationAgentOverrideSchema so per-agent trigger overrides in librechat.yaml are not silently stripped by Zod - Remove unused SummarizationStatus type from runs.ts - Make AppSummarizationConfig.enabled non-optional to reflect the invariant that loadSummarizationConfig always sets it refactor(responses): extract duplicated on_agent_log handler refactor(run): use agents package types for summarization config Import SummarizationConfig, ContextPruningConfig, and OverflowRecoveryConfig from @librechat/agents and use them to type-check the translation layer in buildAgentContext. This ensures the config object passed to the agent graph matches what it expects. - Use `satisfies AgentSummarizationConfig` on the config object - Cast contextPruningConfig and overflowRecoveryConfig to agents types - Properly narrow trigger fields from DeepPartial to required shape feat(config): add maxToolResultChars to base endpoint schema Add maxToolResultChars to baseEndpointSchema so it can be configured on any endpoint in librechat.yaml. Resolved during agent initialization using getProviderConfig's endpoint resolution: custom endpoint config takes precedence, then the provider-specific endpoint config, then the shared `all` config. Passed through to the agents package ToolNode, which uses it to cap tool result length before it enters the context window. When not configured, the agents package computes a sensible default from maxContextTokens. fix(summarization): forward agent model_parameters in self-summarize default When no explicit summarization config exists, the self-summarize default now forwards the agent's model_parameters as the summarization parameters. This ensures provider-specific settings (e.g. Bedrock region, credentials, endpoint host) are available when the agents package constructs the summarization LLM. fix(agents): register summarization handlers by default Change the enabled gate from === true to !== false so handlers register when no explicit summarization config exists. This aligns with the self-summarize default where summarization is always on unless explicitly disabled via enabled: false. refactor(summarization): let agents package inherit clientOptions for self-summarize Remove model_parameters forwarding from the self-summarize default. The agents package now reuses the agent's own clientOptions when the summarization provider matches the agent's provider, inheriting all provider-specific settings (region, credentials, proxy, etc.) automatically. refactor(summarization): use MessageContentComplex[] for summary content Unify summary content to always use MessageContentComplex[] arrays, matching the pattern used by on_message_delta. No more string | array unions — content is always an array of typed blocks ({ type: 'text', text: '...' } for text, { type: 'reasoning_content', ... } for reasoning). Agents package: - SummaryContentBlock.content: MessageContentComplex[] (was string) - tokenCount now optional (not sent on deltas) - Removed reasoning field — reasoning is now a content block type - streamAndCollect normalizes all chunks to content block arrays - Delta events pass content blocks directly LibreChat: - SummaryContentPart.content: Agents.MessageContentComplex[] - Updated Part.tsx, Summary.tsx, useStepHandler.ts, BaseClient.js - Summary.tsx derives display text from content blocks via useMemo - Aggregator uses simple array spread refactor(summarization): enhance summary handling and text extraction - Updated BaseClient.js to improve summary text extraction, accommodating both legacy and new content formats. - Modified summarization logic to ensure consistent handling of summary content across different message formats. - Adjusted test cases in summarization.e2e.spec.js to utilize the new summary text extraction method. - Refined SSE useStepHandler to initialize summary content as an array. - Updated configuration schema by removing unused minReserveTokens field. - Cleaned up SummaryContentPart type by removing rangeHash property. These changes streamline the summarization process and ensure compatibility with various content structures. refactor(summarization): streamline usage tracking and logging - Removed direct checks for summarization nodes in ModelEndHandler and replaced them with a dedicated markSummarizationUsage function for better readability and maintainability. - Updated OpenAIChatCompletionController and responses handlers to utilize the new markSummarizationUsage function for setting usage types. - Enhanced logging functionality by ensuring the logger correctly handles different log levels. - Introduced a new useCopyToClipboard hook in the Summary component to encapsulate clipboard copy logic, improving code reusability and clarity. These changes improve the overall structure and efficiency of the summarization handling and logging processes. refactor(summarization): update summary content block documentation - Removed outdated comment regarding the last summary content block in BaseClient.js. - Added a new comment to clarify the purpose of the findSummaryContentBlock method, ensuring consistency in documentation. These changes enhance code clarity and maintainability by providing accurate descriptions of the summarization logic. refactor(summarization): update summary content structure in tests - Modified the summarization content structure in e2e tests to use an array format for text, aligning with recent changes in summary handling. - Updated test descriptions to clarify the behavior of context token calculations, ensuring consistency and clarity in the tests. These changes enhance the accuracy and maintainability of the summarization tests by reflecting the updated content structure. refactor(summarization): remove legacy E2E test setup and configuration - Deleted the e2e-setup.js and jest.e2e.config.js files, which contained legacy configurations for E2E tests using real API keys. - Introduced a new summarization.e2e.ts file that implements comprehensive E2E backend integration tests for the summarization process, utilizing real AI providers and tracking summaries throughout the run. These changes streamline the testing framework by consolidating E2E tests into a single, more robust file while removing outdated configurations. refactor(summarization): enhance E2E tests and error handling - Added a cleanup step to force exit after all tests to manage Redis connections. - Updated the summarization model to 'claude-haiku-4-5-20251001' for consistency across tests. - Improved error handling in the processStream function to capture and return processing errors. - Enhanced logging for cross-run tests and tight context scenarios to provide better insights into test execution. These changes improve the reliability and clarity of the E2E tests for the summarization process. refactor(summarization): enhance test coverage for maxContextTokens behavior - Updated run-summarization.test.ts to include a new test case ensuring that maxContextTokens does not exceed user-defined limits, even when calculated ratios suggest otherwise. - Modified summarization.e2e.ts to replace legacy UsageMetadata type with a more appropriate type for collectedUsage, improving type safety and clarity in the test setup. These changes improve the robustness of the summarization tests by validating context token constraints and refining type definitions. feat(summarization): add comprehensive E2E tests for summarization process - Introduced a new summarization.e2e.test.ts file that implements extensive end-to-end integration tests for the summarization pipeline, covering the full flow from LibreChat to agents. - The tests utilize real AI providers and include functionality to track summaries during and between runs. - Added necessary cleanup steps to manage Redis connections post-tests and ensure proper exit. These changes enhance the testing framework by providing robust coverage for the summarization process, ensuring reliability and performance under real-world conditions. fix(service): import logger from winston configuration - Removed the import statement for logger from '@librechat/data-schemas' and replaced it with an import from '~/config/winston'. - This change ensures that the logger is correctly sourced from the updated configuration, improving consistency in logging practices across the application. refactor(summary): simplify Summary component and enhance token display - Removed the unused `meta` prop from the `SummaryButton` component to streamline its interface. - Updated the token display logic to use a localized string for better internationalization support. - Adjusted the rendering of the `meta` information to improve its visibility within the `Summary` component. These changes enhance the clarity and usability of the Summary component while ensuring better localization practices. feat(summarization): add maxInputTokens configuration for summarization - Introduced a new `maxInputTokens` property in the summarization configuration schema to control the amount of conversation context sent to the summarizer, with a default value of 10000. - Updated the `createRun` function to utilize the new `maxInputTokens` setting, allowing for more flexible summarization based on agent context. These changes enhance the summarization capabilities by providing better control over input token limits, improving the overall summarization process. refactor(summarization): simplify maxInputTokens logic in createRun function - Updated the logic for the `maxInputTokens` property in the `createRun` function to directly use the agent's base context tokens when the resolved summarization configuration does not specify a value. - This change streamlines the configuration process and enhances clarity in how input token limits are determined for summarization. These modifications improve the maintainability of the summarization configuration by reducing complexity in the token calculation logic. feat(summary): enhance Summary component to display meta information - Updated the SummaryContent component to accept an optional `meta` prop, allowing for additional contextual information to be displayed above the main content. - Adjusted the rendering logic in the Summary component to utilize the new `meta` prop, improving the visibility of supplementary details. These changes enhance the user experience by providing more context within the Summary component, making it clearer and more informative. refactor(summarization): standardize reserveRatio configuration in summarization logic - Replaced instances of `reserveTokensRatio` with `reserveRatio` in the `createRun` function and related tests to unify the terminology across the codebase. - Updated the summarization configuration schema to reflect this change, ensuring consistency in how the reserve ratio is defined and utilized. - Removed the per-agent override logic for summarization configuration, simplifying the overall structure and enhancing clarity. These modifications improve the maintainability and readability of the summarization logic by standardizing the configuration parameters. * fix: circular dependency of `~/models` * chore: update logging scope in agent log handlers Changed log scope from `[agentus:${data.scope}]` to `[agents:${data.scope}]` in both the callbacks and responses controllers to ensure consistent logging format across the application. * feat: calibration ratio * refactor(tests): update summarizationConfig tests to reflect changes in enabled property Modified tests to check for the new `summarizationEnabled` property instead of the deprecated `enabled` field in the summarization configuration. This change ensures that the tests accurately validate the current configuration structure and behavior of the agents. * feat(tests): add markSummarizationUsage mock for improved test coverage Introduced a mock for the markSummarizationUsage function in the responses unit tests to enhance the testing of summarization usage tracking. This addition supports better validation of summarization-related functionalities and ensures comprehensive test coverage for the agents' response handling. * refactor(tests): simplify event handler setup in createResponse tests Removed redundant mock implementations for event handlers in the createResponse unit tests, streamlining the setup process. This change enhances test clarity and maintainability while ensuring that the tests continue to validate the correct behavior of usage tracking during on_chat_model_end events. * refactor(agents): move calibration ratio capture to finally block Reorganized the logic for capturing the calibration ratio in the AgentClient class to ensure it is executed in the finally block. This change guarantees that the ratio is captured even if the run is aborted, enhancing the reliability of the response message persistence. Removed redundant code and improved clarity in the handling of context metadata. * refactor(agents): streamline bulk write logic in recordCollectedUsage function Removed redundant bulk write operations and consolidated document handling in the recordCollectedUsage function. The logic now combines all documents into a single bulk write operation, improving efficiency and reducing error handling complexity. Updated logging to provide consistent error messages for bulk write failures. * refactor(agents): enhance summarization configuration resolution in createRun function Streamlined the summarization configuration logic by introducing a base configuration and allowing for overrides from agent-specific settings. This change improves clarity and maintainability, ensuring that the summarization configuration is consistently applied while retaining flexibility for customization. Updated the handling of summarization parameters to ensure proper integration with the agent's model and provider settings. * refactor(agents): remove unused tokenCountMap and streamline calibration ratio handling Eliminated the unused tokenCountMap variable from the AgentClient class to enhance code clarity. Additionally, streamlined the logic for capturing the calibration ratio by using optional chaining and a fallback value, ensuring that context metadata is consistently defined. This change improves maintainability and reduces potential confusion in the codebase. * refactor(agents): extract agent log handler for improved clarity and reusability Refactored the agent log handling logic by extracting it into a dedicated function, `agentLogHandler`, enhancing code clarity and reusability across different modules. Updated the event handlers in both the OpenAI and responses controllers to utilize the new handler, ensuring consistent logging behavior throughout the application. * test: add summarization event tests for useStepHandler Implemented a series of tests for the summarization events in the useStepHandler hook. The tests cover scenarios for ON_SUMMARIZE_START, ON_SUMMARIZE_DELTA, and ON_SUMMARIZE_COMPLETE events, ensuring proper handling of summarization logic, including message accumulation and finalization. This addition enhances test coverage and validates the correct behavior of the summarization process within the application. * refactor(config): update summarizationTriggerSchema to use enum for type validation Changed the type of the `type` field in the summarizationTriggerSchema from a string to an enum with a single value 'token_count'. This modification enhances type safety and ensures that only valid types are accepted in the configuration, improving overall clarity and maintainability of the schema. * test(usage): add bulk write tests for message and summarization usage Implemented tests for the bulk write functionality in the recordCollectedUsage function, covering scenarios for combined message and summarization usage, summarization-only usage, and message-only usage. These tests ensure correct document handling and token rollup calculations, enhancing test coverage and validating the behavior of the usage tracking logic. * refactor(Chat): enhance clipboard copy functionality and type definitions in Summary component Updated the Summary component to improve the clipboard copy functionality by handling clipboard permission errors. Refactored type definitions for SummaryProps to use a more specific type, enhancing type safety. Adjusted the SummaryButton and FloatingSummaryBar components to accept isCopied and onCopy props, promoting better separation of concerns and reusability. * chore(translations): remove unused "Expand Summary" key from English translations Deleted the "Expand Summary" key from the English translation file to streamline the localization resources and improve clarity in the user interface. This change helps maintain an organized and efficient translation structure. * refactor: adjust token counting for Claude model to account for API discrepancies Implemented a correction factor for token counting when using the Claude model, addressing discrepancies between Anthropic's API and local tokenizer results. This change ensures accurate token counts by applying a scaling factor, improving the reliability of token-related functionalities. * refactor(agents): implement token count adjustment for Claude model messages Added a method to adjust token counts for messages processed by the Claude model, applying a correction factor to align with API expectations. This enhancement improves the accuracy of token counting, ensuring reliable functionality when interacting with the Claude model. * refactor(agents): token counting for media content in messages Introduced a new method to estimate token costs for image and document blocks in messages, improving the accuracy of token counting. This enhancement ensures that media content is properly accounted for, particularly for the Claude model, by integrating additional token estimation logic for various content types. Updated the token counting function to utilize this new method, enhancing overall reliability and functionality. * chore: fix missing import * fix(agents): clamp baseContextTokens and document reserve ratio change Prevent negative baseContextTokens when maxOutputTokens exceeds the context window (misconfigured models). Document the 10%→5% default reserve ratio reduction introduced alongside summarization. * fix(agents): include media tokens in hydrated token counts Add estimateMediaTokensForMessage to createTokenCounter so the hydration path (used by hydrateMissingIndexTokenCounts) matches the precomputed path in AgentClient.getTokenCountForMessage. Without this, messages containing images or documents were systematically undercounted during hydration, risking context window overflow. Add 34 unit tests covering all block-type branches of estimateMediaTokensForMessage. * fix(agents): include summarization output tokens in usage return value The returned output_tokens from recordCollectedUsage now reflects all billed LLM calls (message + summarization). Previously, summarization completions were billed but excluded from the returned metadata, causing a discrepancy between what users were charged and what the response message reported. * fix(tests): replace process.exit with proper Redis cleanup in e2e test The summarization E2E test used process.exit(0) to work around a Redis connection opened at import time, which killed the Jest runner and bypassed teardown. Use ioredisClient.quit() and keyvRedisClient.disconnect() for graceful cleanup instead. * fix(tests): update getConvo imports in OpenAI and response tests Refactor test files to import getConvo from the main models module instead of the Conversation submodule. This change ensures consistency across tests and simplifies the import structure, enhancing maintainability. * fix(clients): improve summary text validation in BaseClient Refactor the summary extraction logic to ensure that only non-empty summary texts are considered valid. This change enhances the robustness of the message processing by utilizing a dedicated method for summary text retrieval, improving overall reliability. * fix(config): replace z.any() with explicit union in summarization schema Model parameters (temperature, top_p, etc.) are constrained to primitive types rather than the policy-violating z.any(). * refactor(agents): deduplicate CLAUDE_TOKEN_CORRECTION constant Export from the TS source in packages/api and import in the JS client, eliminating the static class property that could drift out of sync. * refactor(agents): eliminate duplicate selfProvider in buildAgentContext selfProvider and provider were derived from the same expression with different type casts. Consolidated to a single provider variable. * refactor(agents): extract shared SSE handlers and restrict log levels - buildSummarizationHandlers() factory replaces triplicated handler blocks across responses.js and openai.js - agentLogHandlerObj exported from callbacks.js for consistent reuse - agentLogHandler restricted to an allowlist of safe log levels (debug, info, warn, error) instead of accepting arbitrary strings * fix(SSE): batch summarize deltas, add exhaustiveness check, conditional error announcement - ON_SUMMARIZE_DELTA coalesces rapid-fire renders via requestAnimationFrame instead of calling setMessages per chunk - Exhaustive never-check on TStepEvent catches unhandled variants at compile time when new StepEvents are added - ON_SUMMARIZE_COMPLETE error announcement only fires when a summary part was actually present and removed * feat(agents): persist instruction overhead in contextMeta and seed across runs Extend contextMeta with instructionOverhead and toolCount so the provider-observed instruction overhead is persisted on the response message and seeded into the pruner on subsequent runs. This enables the pruner to use a calibrated budget from the first call instead of waiting for a provider observation, preventing the ratio collapse caused by local tokenizer overestimating tool schema tokens. The seeded overhead is only used when encoding and tool count match between runs, ensuring stale values from different configurations are discarded. * test(agents): enhance OpenAI test mocks for summarization handlers Updated the OpenAI test suite to include additional mock implementations for summarization handlers, including buildSummarizationHandlers, markSummarizationUsage, and agentLogHandlerObj. This improves test coverage and ensures consistent behavior during testing. * fix(agents): address review findings for summarization v2 Cancel rAF on unmount to prevent stale Recoil writes from dead component context. Clear orphaned summarizing:true parts when ON_SUMMARIZE_COMPLETE arrives without a summary payload. Add null guard and safe spread to agentLogHandler. Handle Anthropic-format base64 image/* documents in estimateMediaTokensForMessage. Use role="region" for expandable summary content. Add .describe() to contextMeta Zod fields. Extract duplicate usage loop into helper. * refactor: simplify contextMeta to calibrationRatio + encoding only Remove instructionOverhead and toolCount from cross-run persistence — instruction tokens change too frequently between runs (prompt edits, tool changes) for a persisted seed to be reliable. The intra-run calibration in the pruner still self-corrects via provider observations. contextMeta now stores only the tokenizer-bias ratio and encoding, which are stable across instruction changes. * test(SSE): enhance useStepHandler tests for ON_SUMMARIZE_COMPLETE behavior Updated the test for ON_SUMMARIZE_COMPLETE to clarify that it finalizes the existing part with summarizing set to false when the summary is undefined. Added assertions to verify the correct behavior of message updates and the state of summary parts. * refactor(BaseClient): remove handleContextStrategy and truncateToolCallOutputs functions Eliminated the handleContextStrategy method from BaseClient to streamline message handling. Also removed the truncateToolCallOutputs function from the prompts module, simplifying the codebase and improving maintainability. * refactor: add AGENT_DEBUG_LOGGING option and refactor token count handling in BaseClient Introduced AGENT_DEBUG_LOGGING to .env.example for enhanced debugging capabilities. Refactored token count handling in BaseClient by removing the handleTokenCountMap method and simplifying token count updates. Updated AgentClient to log detailed token count recalculations and adjustments, improving traceability during message processing. * chore: update dependencies in package-lock.json and package.json files Bumped versions of several dependencies, including @librechat/agents to ^3.1.62 and various AWS SDK packages to their latest versions. This ensures compatibility and incorporates the latest features and fixes. * chore: imports order * refactor: extract summarization config resolution from buildAgentContext * refactor: rename and simplify summarization configuration shaping function * refactor: replace AgentClient token counting methods with single-pass pure utility Extract getTokenCount() and getTokenCountForMessage() from AgentClient into countFormattedMessageTokens(), a pure function in packages/api that handles text, tool_call, image, and document content types in one loop. - Decompose estimateMediaTokensForMessage into block-level helpers (estimateImageDataTokens, estimateImageBlockTokens, estimateDocumentBlockTokens) shared by both estimateMediaTokensForMessage and the new single-pass function - Remove redundant per-call getEncoding() resolution (closure captures once) - Remove deprecated gpt-3.5-turbo-0301 model branching - Drop this.getTokenCount guard from BaseClient.sendMessage * refactor: streamline token counting in createTokenCounter function Simplified the createTokenCounter function by removing the media token estimation and directly calculating the token count. This change enhances clarity and performance by consolidating the token counting logic into a single pass, while maintaining compatibility with Claude's token correction. * refactor: simplify summarization configuration types Removed the AppSummarizationConfig type and directly used SummarizationConfig in the AppConfig interface. This change streamlines the type definitions and enhances consistency across the codebase. * chore: import order * fix: summarization event handling in useStepHandler - Cancel pending summarizeDeltaRaf in clearStepMaps to prevent stale frames firing after map reset or component unmount - Move announcePolite('summarize_completed') inside the didFinalize guard so screen readers only announce when finalization actually occurs - Remove dead cleanup closure returned from stepHandler useCallback body that was never invoked by any caller * fix: estimate tokens for non-PDF/non-image base64 document blocks Previously estimateDocumentBlockTokens returned 0 for unrecognized MIME types (e.g. text/plain, application/json), silently underestimating context budget. Fall back to character-based heuristic or countTokens. * refactor: return cloned usage from markSummarizationUsage Avoid mutating LangChain's internal usage_metadata object by returning a shallow clone with the usage_type tag. Update all call sites in callbacks, openai, and responses controllers to use the returned value. * refactor: consolidate debug logging loops in buildMessages Merge the two sequential O(n) debug-logging passes over orderedMessages into a single pass inside the map callback where all data is available. * refactor: narrow SummaryContentPart.content type Replace broad Agents.MessageContentComplex[] with the specific Array<{ type: ContentTypes.TEXT; text: string }> that all producers and consumers already use, improving compile-time safety. * refactor: use single output array in recordCollectedUsage Have processUsageGroup append to a shared array instead of returning separate arrays that are spread into a third, reducing allocations. * refactor: use for...in in hydrateMissingIndexTokenCounts Replace Object.entries with for...in to avoid allocating an intermediate tuple array during token map hydration.	2026-03-21 14:28:56 -04:00
Atef Bellaaj	a0fed6173c	🗂️ refactor: Migrate S3 Storage to TypeScript in packages/api (#11947) ... * Migrate S3 storage module with unit and integration tests - Migrate S3 CRUD and image operations to packages/api/src/storage/s3/ - Add S3ImageService class with dependency injection - Add unit tests using aws-sdk-client-mock - Add integration tests with real s3 bucket (condition presence of AWS_TEST_BUCKET_NAME) * AI Review Findings Fixes * chore: tests and refactor S3 storage types - Added mock implementations for the 'sharp' library in various test files to improve image processing testing. - Updated type references in S3 storage files from MongoFile to TFile for consistency and type safety. - Refactored S3 CRUD operations to ensure proper handling of file types and improve code clarity. - Enhanced integration tests to validate S3 file operations and error handling more effectively. * chore: rename test file * Remove duplicate import of refreshS3Url * chore: imports order * fix: remove duplicate imports for S3 URL handling in UserController * fix: remove duplicate import of refreshS3FileUrls in files.js * test: Add mock implementations for 'sharp' and '@librechat/api' in UserController tests - Introduced mock functions for the 'sharp' library to facilitate image processing tests, including metadata retrieval and buffer conversion. - Enhanced mocking for '@librechat/api' to ensure consistent behavior in tests, particularly for the needsRefresh and getNewS3URL functions. --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-03-21 14:28:55 -04:00
Danny Avila	e4e468840e	🏢 feat: Multi-Tenant Data Isolation Infrastructure (#12091) ... * chore: imports * chore: optional chaining in `spendTokens.spec.ts` * feat: Add tenantId field to all MongoDB schemas for multi-tenant isolation - Add AsyncLocalStorage-based tenant context (`tenantContext.ts`) for request-scoped tenantId propagation without modifying method signatures - Add Mongoose `applyTenantIsolation` plugin that injects `{ tenantId }` into all query filters when tenant context is present, with `TENANT_ISOLATION_STRICT` env var for fail-closed production mode - Add optional `tenantId` field to all 28 collection schemas - Update all compound unique indexes to include tenantId (email, OAuth IDs, role names, serverName, conversationId+user, messageId+user, etc.) - Apply tenant isolation plugin in all 28 model factories - Add `tenantId?: string` to all TypeScript document interfaces Behaviorally inert — transitional mode (default) passes through all queries unchanged. No migration required for existing deployments. * refactor: Update tenant context and enhance tenant isolation plugin - Changed `tenantId` in `TenantContext` to be optional, allowing for more flexible usage. - Refactored `runAsSystem` function to accept synchronous functions, improving usability. - Introduced comprehensive tests for the `applyTenantIsolation` plugin, ensuring correct tenant filtering in various query scenarios. - Enhanced the plugin to handle aggregate queries and save operations with tenant context, improving data isolation capabilities. * docs: tenant context documentation and improve tenant isolation tests - Added detailed documentation for the `tenantStorage` AsyncLocalStorage instance in `tenantContext.ts`, clarifying its usage for async tenant context propagation. - Updated tests in `tenantIsolation.spec.ts` to improve clarity and coverage, including new tests for strict mode behavior and tenant context propagation through await boundaries. - Refactored existing test cases for better readability and consistency, ensuring robust validation of tenant isolation functionality. * feat: Enhance tenant isolation by preventing tenantId mutations in update operations - Added a new function to assert that tenantId cannot be modified through update operators in Mongoose queries. - Implemented middleware to enforce this restriction during findOneAndUpdate, updateOne, and updateMany operations. - Updated documentation to reflect the new behavior regarding tenantId modifications, ensuring clarity on tenant isolation rules. * feat: Enhance tenant isolation tests and enforce tenantId restrictions - Updated existing tests to clarify behavior regarding tenantId preservation during save and insertMany operations. - Introduced new tests to validate that tenantId cannot be modified through update operations, ensuring strict adherence to tenant isolation rules. - Added checks for mismatched tenantId scenarios, reinforcing the integrity of tenant context propagation. - Enhanced test coverage for async context propagation and mutation guards, improving overall robustness of tenant isolation functionality. * fix: Remove duplicate re-exports in utils/index.ts Merge artifact caused `string` and `tempChatRetention` to be exported twice, which produces TypeScript compile errors for duplicate bindings. * fix: Resolve admin capability gap in multi-tenant mode (TODO #12091) - hasCapabilityForPrincipals now queries both tenant-scoped AND platform-level grants when tenantId is set, so seeded ADMIN grants remain effective in tenant mode. - Add applyTenantIsolation to SystemGrant model factory. * fix: Harden tenant isolation plugin - Add replaceGuard for replaceOne/findOneAndReplace to prevent cross-tenant document reassignment via replacement documents. - Cache isStrict() result to avoid process.env reads on every query. Export _resetStrictCache() for test teardown. - Replace console.warn with project logger (winston). - Add 5 new tests for replace guard behavior (46 total). * style: Fix import ordering in convo.ts and message.ts Move type imports after value imports per project style guide. * fix: Remove tenant isolation from SystemGrant, stamp tenantId in replaceGuard - SystemGrant is a cross-tenant control plane whose methods handle tenantId conditions explicitly. Applying the isolation plugin injects a hard equality filter that overrides the $and/$or logic in hasCapabilityForPrincipals, making platform-level ADMIN grants invisible in tenant mode. - replaceGuard now stamps tenantId into replacement documents when absent, preventing replaceOne from silently stripping tenant context. Replacements with a matching tenantId are allowed; mismatched tenantId still throws. * test: Add multi-tenant unique constraint and replace stamping tests - Verify same name/email can exist in different tenants (compound unique index allows it). - Verify duplicate within same tenant is rejected (E11000). - Verify tenant-scoped query returns only the correct document. - Update replaceOne test to assert tenantId is stamped into replacement document. - Add test for replacement with matching tenantId. * style: Reorder imports in message.ts to align with project style guide * feat: Add migration to drop superseded unique indexes for multi-tenancy Existing deployments have single-field unique indexes (e.g. { email: 1 }) that block multi-tenant operation — same email in different tenants triggers E11000. Mongoose autoIndex creates the new compound indexes but never drops the old ones. dropSupersededTenantIndexes() drops all 19 superseded indexes across 11 collections. It is idempotent, skips missing indexes/collections, and is a no-op on fresh databases. Must be called before enabling multi-tenant middleware on an existing deployment. Single-tenant deployments are unaffected (old indexes coexist harmlessly until migration runs). Includes 11 tests covering: - Full upgrade simulation (create old indexes, drop them, verify gone) - Multi-tenant writes work after migration (same email, different tenant) - Intra-tenant uniqueness preserved (duplicate within tenant rejected) - Fresh database (no-op, no errors) - Partial migration (some collections exist, some don't) - SUPERSEDED_INDEXES coverage validation * fix: Update systemGrant test — platform grants now satisfy tenant queries The TODO #12091 fix intentionally changed hasCapabilityForPrincipals to match both tenant-scoped AND platform-level grants. The test expected the old behavior (platform grant invisible to tenant query). Updated test name and expectation to match the new semantics. * fix: Align getCapabilitiesForPrincipal with hasCapabilityForPrincipals tenant query getCapabilitiesForPrincipal used a hard tenantId equality filter while hasCapabilityForPrincipals uses $and/$or to match both tenant-scoped and platform-level grants. This caused the two functions to disagree on what grants a principal holds in tenant mode. Apply the same $or pattern: when tenantId is provided, match both { tenantId } and { tenantId: { $exists: false } }. Adds test verifying platform-level ADMIN grants appear in getCapabilitiesForPrincipal when called with a tenantId. * fix: Remove categories from tenant index migration categoriesSchema is exported but never used to create a Mongoose model. No Category model factory exists, no code constructs a model from it, and no categories collection exists in production databases. Including it in the migration would attempt to drop indexes from a non-existent collection (harmlessly skipped) but implies the collection is managed. * fix: Restrict runAsSystem to async callbacks only Sync callbacks returning Mongoose thenables silently lose ALS context — the system bypass does nothing and strict mode throws with no indication runAsSystem was involved. Narrowing to () => Promise<T> makes the wrong pattern a compile error. All existing call sites already use async. * fix: Use next(err) consistently in insertMany pre-hook The hook accepted a next callback but used throw for errors. Standardize on next(err) for all error paths so the hook speaks one language — callback-style throughout. * fix: Replace optional chaining with explicit null assertions in spendTokens tests Optional chaining on test assertions masks failures with unintelligible error messages. Add expect(result).not.toBeNull() before accessing properties, so a null result produces a clear diagnosis instead of "received value must be a number".	2026-03-21 14:28:54 -04:00
Danny Avila	9e0592a236	📜 feat: Implement System Grants for Capability-Based Authorization (#11896) ... * feat: Implement System Grants for Role-Based Capabilities - Added a new `systemGrant` model and associated methods to manage role-based capabilities within the application. - Introduced middleware functions `hasCapability` and `requireCapability` to check user permissions based on their roles. - Updated the database seeding process to include system grants for the ADMIN role, ensuring all necessary capabilities are assigned on startup. - Enhanced type definitions and schemas to support the new system grant functionality, improving overall type safety and clarity in the codebase. * test: Add unit tests for capabilities middleware and system grant methods - Introduced comprehensive unit tests for the capabilities middleware, including `hasCapability` and `requireCapability`, ensuring proper permission checks based on user roles. - Added tests for the `SystemGrant` methods, verifying the seeding of system grants, capability granting, and revocation processes. - Enhanced test coverage for edge cases, including idempotency of grant operations and handling of unexpected errors in middleware. - Utilized mocks for database interactions to isolate tests and improve reliability. * refactor: Transition to Capability-Based Access Control - Replaced role-based access checks with capability-based checks across various middleware and routes, enhancing permission management. - Introduced `hasCapability` and `requireCapability` functions to streamline capability verification for user actions. - Updated relevant routes and middleware to utilize the new capability system, ensuring consistent permission enforcement. - Enhanced type definitions and added tests for the new capability functions, improving overall code reliability and maintainability. * test: Enhance capability-based access tests for ADMIN role - Updated tests to reflect the new capability-based access control, specifically for the ADMIN role. - Modified test descriptions to clarify that users with the MANAGE_AGENTS capability can bypass permission checks. - Seeded capabilities for the ADMIN role in multiple test files to ensure consistent permission checks across different routes and middleware. - Improved overall test coverage for capability verification, ensuring robust permission management. * test: Update capability tests for MCP server access - Renamed test to reflect the correct capability for bypassing permission checks, changing from MANAGE_AGENTS to MANAGE_MCP_SERVERS. - Updated seeding of capabilities for the ADMIN role to align with the new capability structure. - Ensured consistency in capability definitions across tests and middleware for improved permission management. * feat: Add hasConfigCapability for enhanced config access control - Introduced `hasConfigCapability` function to check user permissions for managing or reading specific config sections. - Updated middleware to export the new capability function, ensuring consistent access control across the application. - Enhanced unit tests to cover various scenarios for the new capability, improving overall test coverage and reliability. * fix: Update tenantId filter in createSystemGrantMethods - Added a condition to set tenantId filter to { $exists: false } when tenantId is null, ensuring proper handling of cases where tenantId is not provided. - This change improves the robustness of the system grant methods by explicitly managing the absence of tenantId in the filter logic. * fix: account deletion capability check - Updated the `canDeleteAccount` middleware to ensure that the `hasManageUsers` capability check only occurs if a user is present, preventing potential errors when the user object is undefined. - This change improves the robustness of the account deletion logic by ensuring proper handling of user permissions. * refactor: Optimize seeding of system grants for ADMIN role - Replaced sequential capability granting with parallel execution using Promise.all in the seedSystemGrants function. - This change improves performance and efficiency during the initialization of system grants, ensuring all capabilities are granted concurrently. * refactor: Simplify systemGrantSchema index definition - Removed the sparse option from the unique index on principalType, principalId, capability, and tenantId in the systemGrantSchema. - This change streamlines the index definition, potentially improving query performance and clarity in the schema design. * refactor: Reorganize role capability check in roles route - Moved the capability check for reading roles to occur after parsing the roleName, improving code clarity and structure. - This change ensures that the authorization logic is consistently applied before fetching role details, enhancing overall permission management. * refactor: Remove unused ISystemGrant interface from systemCapabilities.ts - Deleted the ISystemGrant interface as it was no longer needed, streamlining the code and improving clarity. - This change helps reduce clutter in the file and focuses on relevant capabilities for the system. * refactor: Migrate SystemCapabilities to data-schemas - Replaced imports of SystemCapabilities from 'librechat-data-provider' with imports from '@librechat/data-schemas' across multiple files. - This change centralizes the management of system capabilities, improving code organization and maintainability. * refactor: Update account deletion middleware and capability checks - Modified the `canDeleteAccount` middleware to ensure that the account deletion permission is only granted to users with the `MANAGE_USERS` capability, improving security and clarity in permission management. - Enhanced error logging for unauthorized account deletion attempts, providing better insights into permission issues. - Updated the `capabilities.ts` file to ensure consistent handling of user authentication checks, improving robustness in capability verification. - Refined type definitions in `systemGrant.ts` and `systemGrantMethods.ts` to utilize the `PrincipalType` enum, enhancing type safety and code clarity. * refactor: Extract principal ID normalization into a separate function - Introduced `normalizePrincipalId` function to streamline the normalization of principal IDs based on their type, enhancing code clarity and reusability. - Updated references in `createSystemGrantMethods` to utilize the new normalization function, improving maintainability and reducing code duplication. * test: Add unit tests for principalId normalization in systemGrant - Introduced tests for the `grantCapability`, `revokeCapability`, and `getCapabilitiesForPrincipal` methods to verify correct handling of principalId normalization between string and ObjectId formats. - Enhanced the `capabilities.ts` middleware to utilize the `PrincipalType` enum for improved type safety. - Added a new utility function `normalizePrincipalId` to streamline principal ID normalization logic, ensuring consistent behavior across the application. * feat: Introduce capability implications and enhance system grant methods - Added `CapabilityImplications` to define relationships between broader and implied capabilities, allowing for more intuitive permission checks. - Updated `createSystemGrantMethods` to expand capability queries to include implied capabilities, improving authorization logic. - Enhanced `systemGrantSchema` to include an `expiresAt` field for future TTL enforcement of grants, and added validation to ensure `tenantId` is not set to null. - Documented authorization requirements for prompt group and prompt deletion methods to clarify access control expectations. * test: Add unit tests for canDeleteAccount middleware - Introduced unit tests for the `canDeleteAccount` middleware to verify account deletion permissions based on user roles and capabilities. - Covered scenarios for both allowed and blocked account deletions, including checks for ADMIN users with the `MANAGE_USERS` capability and handling of undefined user cases. - Enhanced test structure to ensure clarity and maintainability of permission checks in the middleware. * fix: Add principalType enum validation to SystemGrant schema Without enum validation, any string value was accepted for principalType and silently stored. Invalid documents would never match capability queries, creating phantom grants impossible to diagnose without raw DB inspection. All other ACL models in the codebase validate this field. * fix: Replace seedSystemGrants Promise.all with bulkWrite for concurrency safety When two server instances start simultaneously (K8s rolling deploy, PM2 cluster), both call seedSystemGrants. With Promise.all + findOneAndUpdate upsert, both instances may attempt to insert the same documents, causing E11000 duplicate key errors that crash server startup. bulkWrite with ordered:false handles concurrent upserts gracefully and reduces 17 individual round trips to a single network call. The returned documents (previously discarded) are no longer fetched. * perf: Add AsyncLocalStorage per-request cache for capability checks Every hasCapability call previously required 2 DB round trips (getUserPrincipals + SystemGrant.exists) — replacing what were O(1) string comparisons. Routes like patchPromptGroup triggered this twice, and hasConfigCapability's fallback path resolved principals twice. This adds a per-request AsyncLocalStorage cache that: - Caches resolved principals (same for all checks within one request) - Caches capability check results (same user+cap = same answer) - Automatically scoped to request lifetime (no stale grants) - Falls through to DB when no store exists (background jobs, tests) - Requires no signature changes to hasCapability The capabilityContextMiddleware is registered at the app level before all routes, initializing a fresh store per request. * fix: Add error handling for inline hasCapability calls canDeleteAccount, fetchAssistants, and validateAuthor all call hasCapability without try-catch. These were previously O(1) string comparisons that could never throw. Now they hit the database and can fail on connection timeout or transient errors. Wrap each call in try-catch, defaulting to deny (false) on error. This ensures a DB hiccup returns a clean 403 instead of an unhandled 500 with a stack trace. * test: Add canDeleteAccount DB-error resilience test Tests that hasCapability rejection (e.g., DB timeout) results in a clean 403 rather than an unhandled exception. Validates the error handling added in the previous commit. * refactor: Use barrel import for hasCapability in validateAuthor Import from ~/server/middleware barrel instead of directly from ~/server/middleware/roles/capabilities for consistency with other non-middleware consumers. Files within the middleware barrel itself must continue using direct imports to avoid circular requires. * refactor: Remove misleading pre('save') hook from SystemGrant schema The pre('save') hook normalized principalId for USER/GROUP principals, but the primary write path (grantCapability) uses findOneAndUpdate — which does not trigger save hooks. The normalization was already handled explicitly in grantCapability itself. The hook created a false impression of schema-level enforcement that only covered save()/create() paths. Replace with a comment documenting that all writes must go through grantCapability. * feat: Add READ_ASSISTANTS capability to complete manage/read pair Every other managed resource had a paired READ_X / MANAGE_X capability except assistants. This adds READ_ASSISTANTS and registers the MANAGE_ASSISTANTS → READ_ASSISTANTS implication in CapabilityImplications, enabling future read-only assistant visibility grants. * chore: Reorder systemGrant methods for clarity Moved hasCapabilityForPrincipals to a more logical position in the returned object of createSystemGrantMethods, improving code readability. This change also maintains the inclusion of seedSystemGrants in the export, ensuring all necessary methods are available. * fix: Wrap seedSystemGrants in try-catch to avoid blocking startup Seeding capabilities is idempotent and will succeed on the next restart. A transient DB error during seeding should not prevent the server from starting — log the error and continue. * refactor: Improve capability check efficiency and add audit logging Move hasCapability calls after cheap early-exits in validateAuthor and fetchAssistants so the DB check only runs when its result matters. Add logger.debug on every capability bypass grant across all 7 call sites for auditability, and log errors in catch blocks instead of silently swallowing them. * test: Add integration tests for AsyncLocalStorage capability caching Exercises the full vertical — ALS context, generateCapabilityCheck, real getUserPrincipals, real hasCapabilityForPrincipals, real MongoDB via MongoMemoryServer. Covers per-request caching, cross-context isolation, concurrent request isolation, negative caching, capability implications, tenant scoping, group-based grants, and requireCapability middleware. * test: Add systemGrant data-layer and ALS edge-case integration tests systemGrant.spec.ts (51 tests): Full integration tests for all systemGrant methods against real MongoDB — grant/revoke lifecycle, principalId normalization (string→ObjectId for USER/GROUP, string for ROLE), capability implications (both directions), tenant scoping, schema validation (null tenantId, invalid enum, required fields, unique compound index). capabilities.integration.spec.ts (27 tests): Adds ALS edge cases — missing context degrades gracefully with no caching (background jobs, child processes), nested middleware creates independent inner context, optional-chaining safety when store is undefined, mid-request grant changes are invisible due to result caching, requireCapability works without ALS, and interleaved concurrent contexts maintain isolation. * fix: Add worker thread guards to capability ALS usage Detect when hasCapability or capabilityContextMiddleware is called from a worker thread (where ALS context does not propagate from the parent). hasCapability logs a warn-once per factory instance; the middleware logs an error since mounting Express middleware in a worker is likely a misconfiguration. Both continue to function correctly — the guard is observability, not a hard block. * fix: Include tenantId in ALS principal cache key for tenant isolation The principal cache key was user.id:user.role, which would reuse cached principals across tenants for the same user within a request. When getUserPrincipals gains tenant-scoped group resolution, principals from tenant-a would incorrectly serve tenant-b checks. Changed to user.id:user.role:user.tenantId to prevent cross-tenant cache hits. Adds integration test proving separate principal lookups per tenantId. * test: Remove redundant mocked capabilities.spec.js The JS wrapper test (7 tests, all mocked) is a strict subset of capabilities.integration.spec.ts (28 tests, real MongoDB). Every scenario it covered — hasCapability true/false, tenantId passthrough, requireCapability 403/500, error handling — is tested with higher fidelity in the integration suite. * test: Replace mocked canDeleteAccount tests with real MongoDB integration Remove hasCapability mock — tests now exercise the full capability chain against real MongoDB (getUserPrincipals, hasCapabilityForPrincipals, SystemGrant collection). Only mocks remaining are logger and cache. Adds new coverage: admin role without grant is blocked, user-level grant bypasses deletion restriction, null user handling. * test: Add comprehensive tests for ACL entry management and user group methods Introduces new tests for `deleteAclEntries`, `bulkWriteAclEntries`, and `findPublicResourceIds` in `aclEntry.spec.ts`, ensuring proper functionality for deleting and bulk managing ACL entries. Additionally, enhances `userGroup.spec.ts` with tests for finding groups by ID and name pattern, including external ID matching and source filtering. These changes improve coverage and validate the integrity of ACL and user group operations against real MongoDB interactions. * refactor: Update capability checks and logging for better clarity and error handling Replaced `MANAGE_USERS` with `ACCESS_ADMIN` in the `canDeleteAccount` middleware and related tests to align with updated permission structure. Enhanced logging in various middleware functions to use `logger.warn` for capability check failures, providing clearer error messages. Additionally, refactored capability checks in the `patchPromptGroup` and `validateAuthor` functions to improve readability and maintainability. This commit also includes adjustments to the `systemGrant` methods to implement retry logic for transient failures during capability seeding, ensuring robustness in the face of database errors. * refactor: Enhance logging and retry logic in seedSystemGrants method Updated the logging format in the seedSystemGrants method to include error messages for better clarity. Improved the retry mechanism by explicitly mocking multiple failures in tests, ensuring robust error handling during transient database issues. Additionally, refined imports in the systemGrant schema for better type management. * refactor: Consolidate imports in canDeleteAccount middleware Merged logger and SystemCapabilities imports from the data-schemas module into a single line for improved readability and maintainability of the code. This change streamlines the import statements in the canDeleteAccount middleware. * test: Enhance systemGrant tests for error handling and capability validation Added tests to the systemGrant methods to handle various error scenarios, including E11000 race conditions, invalid ObjectId strings for USER and GROUP principals, and invalid capability strings. These enhancements improve the robustness of the capability granting and revoking logic, ensuring proper error propagation and validation of inputs. * fix: Wrap hasCapability calls in deny-by-default try-catch at remaining sites canAccessResource, files.js, and roles.js all had hasCapability inside outer try-catch blocks that returned 500 on DB failure instead of falling through to the regular ACL check. This contradicts the deny-by-default pattern used everywhere else. Also removes raw error.message from the roles.js 500 response to prevent internal host/connection info leaking to clients. * fix: Normalize user ID in canDeleteAccount before passing to hasCapability requireCapability normalizes req.user.id via _id?.toString() fallback, but canDeleteAccount passed raw req.user directly. If req.user.id is absent (some auth layers only populate _id), getUserPrincipals received undefined, silently returning empty principals and blocking the bypass. * fix: Harden systemGrant schema and type safety - Reject empty string tenantId in schema validator (was only blocking null; empty string silently orphaned documents) - Fix reverseImplications to use BaseSystemCapability[] instead of string[], preserving the narrow discriminated type - Document READ_ASSISTANTS as reserved/unenforced * test: Use fake timers for seedSystemGrants retry tests and add tenantId validation - Switch retry tests to jest.useFakeTimers() to eliminate 3+ seconds of real setTimeout delays per test run - Add regression test for empty-string tenantId rejection * docs: Add TODO(#12091) comments for tenant-scoped capability gaps In multi-tenant mode, platform-level grants (no tenantId) won't match tenant-scoped queries, breaking admin access. getUserPrincipals also returns cross-tenant group memberships. Both need fixes in #12091.	2026-03-21 14:28:54 -04:00
Danny Avila	0412f05daf	🪢 chore: Consolidate Pricing and Tx Imports After tx.js Module Removal (#12086) ... * 🧹 chore: resolve imports due to rebase * chore: Update model mocks in unit tests for consistency - Consolidated model mock implementations across various test files to streamline setup and reduce redundancy. - Removed duplicate mock definitions for `getMultiplier` and `getCacheMultiplier`, ensuring a unified approach in `recordCollectedUsage.spec.js`, `openai.spec.js`, `responses.unit.spec.js`, and `abortMiddleware.spec.js`. - Enhanced clarity and maintainability of test files by aligning mock structures with the latest model updates. * fix: Safeguard token credit checks in transaction tests - Updated assertions in `transaction.spec.ts` to handle potential null values for `updatedBalance` by using optional chaining. - Enhanced robustness of tests related to token credit calculations, ensuring they correctly account for scenarios where the balance may not be found. * chore: transaction methods with bulk insert functionality - Introduced `bulkInsertTransactions` method in `transaction.ts` to facilitate batch insertion of transaction documents. - Updated test file `transactions.bulk-parity.spec.ts` to utilize new pricing function assignments and handle potential null values in calculations, improving test robustness. - Refactored pricing function initialization for clarity and consistency. * refactor: Enhance type definitions and introduce new utility functions for model matching - Added `findMatchingPattern` and `matchModelName` utility functions to improve model name matching logic in transaction methods. - Updated type definitions for `findMatchingPattern` to accept a more specific tokensMap structure, enhancing type safety. - Refactored `dbMethods` initialization in `transactions.bulk-parity.spec.ts` to include the new utility functions, improving test clarity and functionality. * refactor: Update database method imports and enhance transaction handling - Refactored `abortMiddleware.js` to utilize centralized database methods for message handling and conversation retrieval, improving code consistency. - Enhanced `bulkInsertTransactions` in `transaction.ts` to handle empty document arrays gracefully and added error logging for better debugging. - Updated type definitions in `transactions.ts` to enforce stricter typing for token types, enhancing type safety across transaction methods. - Improved test setup in `transactions.bulk-parity.spec.ts` by refining pricing function assignments and ensuring robust handling of potential null values. * refactor: Update database method references and improve transaction multiplier handling - Refactored `client.js` to update database method references for `bulkInsertTransactions` and `updateBalance`, ensuring consistency in method usage. - Enhanced transaction multiplier calculations in `transaction.spec.ts` to provide fallback values for write and read multipliers, improving robustness in cost calculations across structured token spending tests.	2026-03-21 14:28:53 -04:00
Danny Avila	8ba2bde5c1	📦 refactor: Consolidate DB models, encapsulating Mongoose usage in data-schemas (#11830) ... * chore: move database model methods to /packages/data-schemas * chore: add TypeScript ESLint rule to warn on unused variables * refactor: model imports to streamline access - Consolidated model imports across various files to improve code organization and reduce redundancy. - Updated imports for models such as Assistant, Message, Conversation, and others to a unified import path. - Adjusted middleware and service files to reflect the new import structure, ensuring functionality remains intact. - Enhanced test files to align with the new import paths, maintaining test coverage and integrity. * chore: migrate database models to packages/data-schemas and refactor all direct Mongoose Model usage outside of data-schemas * test: update agent model mocks in unit tests - Added `getAgent` mock to `client.test.js` to enhance test coverage for agent-related functionality. - Removed redundant `getAgent` and `getAgents` mocks from `openai.spec.js` and `responses.unit.spec.js` to streamline test setup and reduce duplication. - Ensured consistency in agent mock implementations across test files. * fix: update types in data-schemas * refactor: enhance type definitions in transaction and spending methods - Updated type definitions in `checkBalance.ts` to use specific request and response types. - Refined `spendTokens.ts` to utilize a new `SpendTxData` interface for better clarity and type safety. - Improved transaction handling in `transaction.ts` by introducing `TransactionResult` and `TxData` interfaces, ensuring consistent data structures across methods. - Adjusted unit tests in `transaction.spec.ts` to accommodate new type definitions and enhance robustness. * refactor: streamline model imports and enhance code organization - Consolidated model imports across various controllers and services to a unified import path, improving code clarity and reducing redundancy. - Updated multiple files to reflect the new import structure, ensuring all functionalities remain intact. - Enhanced overall code organization by removing duplicate import statements and optimizing the usage of model methods. * feat: implement loadAddedAgent and refactor agent loading logic - Introduced `loadAddedAgent` function to handle loading agents from added conversations, supporting multi-convo parallel execution. - Created a new `load.ts` file to encapsulate agent loading functionalities, including `loadEphemeralAgent` and `loadAgent`. - Updated the `index.ts` file to export the new `load` module instead of the deprecated `loadAgent`. - Enhanced type definitions and improved error handling in the agent loading process. - Adjusted unit tests to reflect changes in the agent loading structure and ensure comprehensive coverage. * refactor: enhance balance handling with new update interface - Introduced `IBalanceUpdate` interface to streamline balance update operations across the codebase. - Updated `upsertBalanceFields` method signatures in `balance.ts`, `transaction.ts`, and related tests to utilize the new interface for improved type safety. - Adjusted type imports in `balance.spec.ts` to include `IBalanceUpdate`, ensuring consistency in balance management functionalities. - Enhanced overall code clarity and maintainability by refining type definitions related to balance operations. * feat: add unit tests for loadAgent functionality and enhance agent loading logic - Introduced comprehensive unit tests for the `loadAgent` function, covering various scenarios including null and empty agent IDs, loading of ephemeral agents, and permission checks. - Enhanced the `initializeClient` function by moving `getConvoFiles` to the correct position in the database method exports, ensuring proper functionality. - Improved test coverage for agent loading, including handling of non-existent agents and user permissions. * chore: reorder memory method exports for consistency - Moved `deleteAllUserMemories` to the correct position in the exported memory methods, ensuring a consistent and logical order of method exports in `memory.ts`.	2026-03-21 14:28:53 -04:00
Danny Avila	58f128bee7	🗑️ chore: Remove Deprecated Project Model and Associated Fields (#11773) ... * chore: remove projects and projectIds usage * chore: empty line linting * chore: remove isCollaborative property across agent models and related tests - Removed the isCollaborative property from agent models, controllers, and tests, as it is deprecated in favor of ACL permissions. - Updated related validation schemas and data provider types to reflect this change. - Ensured all references to isCollaborative were stripped from the codebase to maintain consistency and clarity.	2026-03-21 14:28:53 -04:00
Danny Avila	38521381f4	🐘 feat: FerretDB Compatibility (#11769) ... * feat: replace unsupported MongoDB aggregation operators for FerretDB compatibility Replace $lookup, $unwind, $sample, $replaceRoot, and $addFields aggregation stages which are unsupported on FerretDB v2.x (postgres-documentdb backend). - Prompt.js: Replace $lookup/$unwind/$project pipelines with find().select().lean() + attachProductionPrompts() batch helper. Replace $group/$replaceRoot/$sample in getRandomPromptGroups with distinct() + Fisher-Yates shuffle. - Agent/Prompt migration scripts: Replace $lookup anti-join pattern with distinct() + $nin two-step queries for finding un-migrated resources. All replacement patterns verified against FerretDB v2.7.0. * fix: use $pullAll for simple array removals, fix memberIds type mismatches Replace $pull with $pullAll for exact-value scalar array removals. Both operators work on MongoDB and FerretDB, but $pullAll is more explicit for exact matching (no condition expressions). Fix critical type mismatch bugs where ObjectId values were used against String[] memberIds arrays in Group queries: - config/delete-user.js: use string uid instead of ObjectId user._id - e2e/setup/cleanupUser.ts: convert userId.toString() before query Harden PermissionService.bulkUpdateResourcePermissions abort handling to prevent crash when abortTransaction is called after commitTransaction. All changes verified against FerretDB v2.7.0 and MongoDB Memory Server. * fix: harden transaction support probe for FerretDB compatibility Commit the transaction before aborting in supportsTransactions probe, and wrap abortTransaction in try-catch to prevent crashes when abort is called after a successful commit (observed behavior on FerretDB). * feat: add FerretDB compatibility test suite, retry utilities, and CI config Add comprehensive FerretDB integration test suite covering: - $pullAll scalar array operations - $pull with subdocument conditions - $lookup replacement (find + manual join) - $sample replacement (distinct + Fisher-Yates) - $bit and $bitsAllSet operations - Migration anti-join pattern - Multi-tenancy (useDb, scaling, write amplification) - Sharding proof-of-concept - Production operations (backup/restore, schema migration, deadlock retry) Add production retryWithBackoff utility for deadlock recovery during concurrent index creation on FerretDB/DocumentDB backends. Add UserController.spec.js tests for deleteUserController (runs in CI). Configure jest and eslint to isolate FerretDB tests from CI pipelines: - packages/data-schemas/jest.config.mjs: ignore misc/ directory - eslint.config.mjs: ignore packages/data-schemas/misc/ Include Docker Compose config for local FerretDB v2.7 + postgres-documentdb, dedicated jest/tsconfig for the test files, and multi-tenancy findings doc. * style: brace formatting in aclEntry.ts modifyPermissionBits * refactor: reorganize retry utilities and update imports - Moved retryWithBackoff utility to a new file `retry.ts` for better structure. - Updated imports in `orgOperations.ferretdb.spec.ts` to reflect the new location of retry utilities. - Removed old import statement for retryWithBackoff from index.ts to streamline exports. * test: add $pullAll coverage for ConversationTag and PermissionService Add integration tests for deleteConversationTag verifying $pullAll removes tags from conversations correctly, and for syncUserEntraGroupMemberships verifying $pullAll removes user from non-matching Entra groups while preserving local group membership. ---------	2026-03-21 14:28:49 -04:00
crossagent	290984c514	🔑 fix: Type-Safe User Context Forwarding for Non-OAuth Tool Discovery (#12348) ... * fix(mcp): pass missing customUserVars and user during unauthenticated tool discovery * fix(mcp): type-safe user context forwarding for non-OAuth tool discovery Extract UserConnectionContext from OAuthConnectionOptions to properly model the non-OAuth case where user/customUserVars/requestBody need placeholder resolution without requiring OAuth-specific fields. - Remove prohibited `as unknown as` double-cast - Forward requestBody and connectionTimeout (previously omitted) - Add unit tests for argument forwarding at Manager and Factory layers - Add integration test exercising real processMCPEnv substitution --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-03-21 12:46:23 -04:00
Danny Avila	0736ff2668	✨ v0.8.4 (#12339) ... * 🔖 chore: Bump version to v0.8.4 - App version: v0.8.4-rc1 → v0.8.4 - @librechat/api: 1.7.26 → 1.7.27 - @librechat/client: 0.4.55 → 0.4.56 - librechat-data-provider: 0.8.400 → 0.8.401 - @librechat/data-schemas: 0.0.39 → 0.0.40 * chore: bun.lock file bumps	2026-03-20 18:01:00 -04:00
Danny Avila	365a0dc0f6	🩺 refactor: Surface Descriptive OCR Error Messages to Client (#12344) ... * fix: pass along error message when OCR fails Right now, if OCR fails, it just says "Error processing file" which isn't very helpful. The `error.message` does has helpful information in it, but our filter wasn't including the right case to pass it along. Now it does! * fix: extract shared upload error filter, apply to images route The 'Unable to extract text from' error was only allowlisted in the files route but not the images route, which also calls processAgentFileUpload. Extract the duplicated error filter logic into a shared resolveUploadErrorMessage utility in packages/api so both routes stay in sync. --------- Co-authored-by: Dan Lew <daniel@mightyacorn.com>	2026-03-20 17:10:25 -04:00
mfish911	4e5ae28fa9	📡 feat: Support Unauthenticated SMTP Relays (#12322) ... * allow smtp server that does not have authentication * fix: align checkEmailConfig with optional SMTP credentials and add tests Remove EMAIL_USERNAME/EMAIL_PASSWORD requirements from the hasSMTPConfig predicate in checkEmailConfig() so the rest of the codebase (login, startup checks, invite-user) correctly recognizes unauthenticated SMTP as a valid email configuration. Add a warning when only one of the two credential env vars is set, in both sendEmail.js and checkEmailConfig(), to catch partial misconfigurations early. Add test coverage for both the transporter auth assembly in sendEmail.js and the checkEmailConfig predicate in packages/api. Document in .env.example that credentials are optional for unauthenticated SMTP relays. --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-03-20 13:07:39 -04:00
Danny Avila	594d9470d5	🪤 fix: Avoid express-rate-limit v8 ERR_ERL_KEY_GEN_IPV6 False Positive (#12333) ... * fix: avoid express-rate-limit v8 ERR_ERL_KEY_GEN_IPV6 false positive express-rate-limit v8 calls keyGenerator.toString() and throws ERR_ERL_KEY_GEN_IPV6 if the source contains the literal substring "req.ip" without "ipKeyGenerator". When packages/api compiles req?.ip to older JS targets, the output contains "req.ip", triggering the heuristic. Bracket notation (req?.['ip']) produces identical runtime behavior but never emits the literal "req.ip" substring regardless of compilation target. Closes #12321 * fix: add toString regression test and clean up redundant annotation Add a test that verifies removePorts.toString() does not contain "req.ip", guarding against reintroduction of the ERR_ERL_KEY_GEN_IPV6 false positive. Fix a misleading test description and remove a redundant type annotation on a trivially-inferred local.	2026-03-20 12:32:55 -04:00
Danny Avila	e442984364	💣 fix: Harden against falsified ZIP metadata in ODT parsing (#12320) ... * security: replace JSZip metadata guard with yauzl streaming decompression The ODT decompressed-size guard was checking JSZip's private _data.uncompressedSize fields, which are populated from the ZIP central directory — attacker-controlled metadata. A crafted ODT with falsified uncompressedSize values bypassed the 50MB cap entirely, allowing content.xml decompression to exhaust Node.js heap memory (DoS). Replace JSZip with yauzl for ODT extraction. The new extractOdtContentXml function uses yauzl's streaming API: it lazily iterates ZIP entries, opens a decompression stream for content.xml, and counts real bytes as they arrive from the inflate stream. The stream is destroyed the moment the byte count crosses ODT_MAX_DECOMPRESSED_SIZE, aborting the inflate before the full payload is materialised in memory. - Remove jszip from direct dependencies (still transitive via mammoth) - Add yauzl + @types/yauzl - Update zip-bomb test to verify streaming abort with DEFLATE payload * fix: close file descriptor leaks and declare jszip test dependency - Use a shared `finish()` helper in extractOdtContentXml that calls zipfile.close() on every exit path (success, size cap, missing entry, openReadStream errors, zipfile errors). Without this, any error path leaked one OS file descriptor permanently — uploading many malformed ODTs could exhaust the process FD limit (a distinct DoS vector). - Add jszip to devDependencies so the zip-bomb test has an explicit dependency rather than relying on mammoth's transitive jszip. - Update JSDoc to document that all exit paths close the zipfile. * fix: move yauzl from dependencies to peerDependencies Matches the established pattern for runtime parser libraries in packages/api: mammoth, pdfjs-dist, and xlsx are all peerDependencies (provided by the consuming /api workspace) with devDependencies for testing. yauzl was incorrectly placed in dependencies. * fix: add yauzl to /api dependencies to satisfy peer dep packages/api declares yauzl as a peerDependency; /api is the consuming workspace that must provide it at runtime, matching the pattern used for mammoth, pdfjs-dist, and xlsx.	2026-03-19 22:13:40 -04:00
Brad Russell	ecd6d76bc8	🚦 fix: ERR_ERL_INVALID_IP_ADDRESS and IPv6 Key Collisions in IP Rate Limiters (#12319) ... * fix: Add removePorts keyGenerator to all IP-based rate limiters Six IP-based rate limiters are missing the `keyGenerator: removePorts` option that is already used by the auth-related limiters (login, register, resetPassword, verifyEmail). Without it, reverse proxies that include ports in X-Forwarded-For headers cause ERR_ERL_INVALID_IP_ADDRESS errors from express-rate-limit. Fixes #12318 * fix: make removePorts IPv6-safe to prevent rate-limit key collisions The original regex `/:\d+[^:]*$/` treated the last colon-delimited segment of bare IPv6 addresses as a port, mangling valid IPs (e.g. `::1` → `::`, `2001:db8::1` → `2001:db8::`). Distinct IPv6 clients could collapse into the same rate-limit bucket. Use `net.isIP()` as a fast path for already-valid IPs, then match bracketed IPv6+port and IPv4+port explicitly. Bare IPv6 addresses are now returned unchanged. Also fixes pre-existing property ordering inconsistency in ttsLimiters.js userLimiterOptions (keyGenerator before store). * refactor: move removePorts to packages/api as TypeScript, fix import order - Move removePorts implementation to packages/api/src/utils/removePorts.ts with proper Express Request typing - Reduce api/server/utils/removePorts.js to a thin re-export from @librechat/api for backward compatibility - Consolidate removePorts import with limiterCache from @librechat/api in all 6 limiter files, fixing import order (package imports shortest to longest, local imports longest to shortest) - Remove narrating inline comments per code style guidelines --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-03-19 21:48:03 -04:00
Danny Avila	3abad53c16	📦 chore: Bump @dicebear dependencies to v9.4.1 (#12315) ... - Bump @dicebear/collection and @dicebear/core to version 9.4.1 across multiple package files for consistency and improved functionality. - Update related dependencies in the client and packages/client directories to ensure compatibility with the new versions.	2026-03-19 16:44:38 -04:00
Danny Avila	11ab5f6ee5	🛂 fix: Reject OpenID Email Fallback When Stored openidId Mismatches Token Sub (#12312) ... * 🔐 fix: Reject OpenID email fallback when stored openidId mismatches token sub When `findOpenIDUser` falls back to email lookup after the primary `openidId`/`idOnTheSource` query fails, it now rejects any user whose stored `openidId` differs from the incoming JWT subject claim. This closes an account-takeover vector where a valid IdP JWT containing a victim's email but a different `sub` could authenticate as the victim when OPENID_REUSE_TOKENS is enabled. The migration path (user has no `openidId` yet) is unaffected. * test: Validate openidId mismatch guard in email fallback path Update `findOpenIDUser` unit tests to assert that email-based lookups returning a user with a different `openidId` are rejected with AUTH_FAILED. Add matching integration test in `openIdJwtStrategy.spec` exercising the full verify callback with the real `findOpenIDUser`. * 🔐 fix: Remove redundant `openidId` truthiness check from mismatch guard The `&& openidId` middle term in the guard condition caused it to be bypassed when the incoming token `sub` was empty or undefined. Since the JS callers can pass `payload?.sub` (which may be undefined), this created a path where the guard never fired and the email fallback returned the victim's account. Removing the term ensures the guard rejects whenever the stored openidId differs from the incoming value, regardless of whether the incoming value is falsy. * test: Cover falsy openidId bypass and openidStrategy mismatch rejection Add regression test for the guard bypass when `openidId` is an empty string and the email lookup finds a user with a stored openidId. Add integration test in openidStrategy.spec.js exercising the mismatch rejection through the full processOpenIDAuth callback, ensuring both OIDC paths (JWT reuse and standard callback) are covered. Restore intent-documenting comment on the no-provider fixture.	2026-03-19 16:42:57 -04:00
Danny Avila	39f5f83a8a	🔌 fix: Isolate Code-Server HTTP Agents to Prevent Socket Pool Contamination (#12311) ... * 🔧 fix: Isolate HTTP agents for code-server axios requests Prevents socket hang up after 5s on Node 19+ when code executor has file attachments. follow-redirects (axios dep) leaks `socket.destroy` as a timeout listener on TCP sockets; with Node 19+ defaulting to keepAlive: true, tainted sockets re-enter the global pool and destroy active node-fetch requests in CodeExecutor after the idle timeout. Uses dedicated http/https agents with keepAlive: false for all axios calls targeting CODE_BASEURL in crud.js and process.js. Closes #12298 * ♻️ refactor: Extract code-server HTTP agents to shared module - Move duplicated agent construction from crud.js and process.js into a shared agents.js module to eliminate DRY violation - Switch process.js from raw `require('axios')` to `createAxiosInstance()` for proxy configuration parity with crud.js - Fix import ordering in process.js (agent constants no longer split imports) - Add 120s timeout to uploadCodeEnvFile (was the only code-server call without a timeout) * ✅ test: Add regression tests for code-server socket isolation - Add crud.spec.js covering getCodeOutputDownloadStream and uploadCodeEnvFile (agent options, timeout, URL, error handling) - Add socket pool isolation tests to process.spec.js asserting keepAlive:false agents are forwarded to axios - Update process.spec.js mocks for createAxiosInstance() migration * ♻️ refactor: Move code-server agents to packages/api Relocate agents.js from api/server/services/Files/Code/ to packages/api/src/utils/code.ts per workspace conventions. Consumers now import codeServerHttpAgent/codeServerHttpsAgent from @librechat/api.	2026-03-19 16:16:57 -04:00
Pol Burkardt Freire	7e74165c3c	📖 feat: Add Native ODT Document Parser Support (#12303) ... * fix: add ODT support to native document parser * fix: replace execSync with jszip for ODT parsing * docs: update documentParserMimeTypes comment to include odt * fix: improve ODT XML extraction and add empty.odt fixture - Scope extraction to <office:body> to exclude metadata/style nodes - Map </text:p> and </text:h> closings to newlines, preserving paragraph structure instead of collapsing everything to a single line - Handle <text:line-break/> as explicit newlines - Strip remaining tags, normalize horizontal whitespace, cap consecutive blank lines at one - Regenerate sample.odt as a two-paragraph fixture so the test exercises multi-paragraph output - Add empty.odt fixture and test asserting 'No text found in document' * fix: address review findings in ODT parser - Use static `import JSZip from 'jszip'` instead of dynamic import; jszip is CommonJS-only with no ESM/Jest-isolation concern (F1) - Decode the five standard XML entities after tag-stripping so documents with &, <, >, ", ' send correct text to the LLM (F2) - Remove @types/jszip devDependency; jszip ships bundled declarations and @types/jszip is a stale 2020 stub that would shadow them (F3) - Handle <text:tab/> → \t and <text:s .../> → ' ' before the generic tag stripper so tab-aligned and multi-space content is preserved (F4) - Add sample-entities.odt fixture and test covering entity decoding, tab, and spacing-element handling (F5) - Rename 'throws for empty odt' → 'throws for odt with no extractable text' to distinguish from a zero-byte/corrupt file case (F8) * fix: add decompressed content size cap to odtToText (F6) Reads uncompressed entry sizes from the JSZip internal metadata before extracting any content. Throws if the total exceeds 50MB, preventing a crafted ODT with a high-ratio compressed payload from exhausting heap. Adds a corresponding test using a real DEFLATE-compressed ZIP (~51KB on disk, 51MB uncompressed) to verify the guard fires before any extraction. * fix: add java to codeTypeMapping for file upload support .java files were rejected with "Unable to determine file type" because browsers send an empty MIME type for them and codeTypeMapping had no 'java' entry for inferMimeType() to fall back on. text/x-java was already present in all five validation lists (fullMimeTypesList, codeInterpreterMimeTypesList, retrievalMimeTypesList, textMimeTypes, retrievalMimeTypes), so mapping to it (not text/plain) ensures .java uploads work for both File Search and Code Interpreter. Closes #12307 * fix: address follow-up review findings (A-E) A: regenerate package-lock.json after removing @types/jszip from package.json; without this npm ci was still installing the stale 2020 type stubs and TypeScript was resolving against them B: replace dynamic import('jszip') in the zip-bomb test with the same static import already used in production; jszip is CJS-only with no ESM/Jest isolation concern C: document that the _data.uncompressedSize guard fails open if jszip renames the private field (accepted limitation, test would catch it) D: rename 'preserves tabs' test to 'normalizes tab and spacing elements to spaces' since <text:tab> is collapsed to a space, not kept as \t E: fix test.each([ formatting artifact (missing newline after '[') --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-03-19 15:49:52 -04:00
Danny Avila	b189972381	🎭 fix: Set Explicit Permission Defaults for USER Role in roleDefaults (#12308) ... * fix: set explicit permission defaults for USER role in roleDefaults Previously several permission types for the USER role had empty objects in roleDefaults, causing the getPermissionValue fallback to resolve SHARE/CREATE via the zod schema defaults on fresh installs. This silently granted users MCP server creation ability and left share permissions ambiguous. Sets explicit defaults for all multi-field permission types: - PROMPTS/AGENTS: USE and CREATE true, SHARE false - MCP_SERVERS: USE true, CREATE/SHARE false - REMOTE_AGENTS: all false Adds regression tests covering the exact reported scenarios (fresh install with `agents: { use: true }`, restart preserving admin-panel overrides) and structural guards against future permission schema expansions missing explicit USER defaults. Closes #12306. * fix: guard MCP_SERVERS.CREATE against configDefaults fallback + add migration The roleDefaults fix alone was insufficient: loadDefaultInterface propagates configDefaults.mcpServers.create=true as tier-1 in getPermissionValue, overriding the roleDefault of false. This commit: - Adds conditional guards for MCP_SERVERS.CREATE and REMOTE_AGENTS.CREATE matching the existing AGENTS/PROMPTS pattern (only include CREATE when explicitly configured in yaml OR on fresh install) - Uses raw interfaceConfig for MCP_SERVERS.CREATE tier-1 instead of loadedInterface (which includes configDefaults fallback) - Adds one-time migration backfill: corrects existing MCP_SERVERS.CREATE=true for USER role in DB when no explicit yaml config is present - Adds restart-scenario and migration regression tests for MCP_SERVERS - Cleans up roles.spec.ts: for..of loops, Permissions[] typing, Set for lookups, removes unnecessary aliases, improves JSDoc for exclusion list - Fixes misleading test name for agents regression test - Removes redundant not.toHaveProperty assertions after strict toEqual * fix: use raw interfaceConfig for REMOTE_AGENTS.CREATE tier-1 (consistency) Aligns REMOTE_AGENTS.CREATE with the MCP_SERVERS.CREATE fix — reads from raw interfaceConfig instead of loadedInterface to prevent a future configDefaults fallback from silently overriding the roleDefault.	2026-03-19 14:52:06 -04:00
Danny Avila	9cb5ac63f8	🫧 refactor: Clear Drafts and Surface Error on Expired SSE Stream (#12309) ... * refactor: error handling in useResumableSSE for 404 responses - Added logic to clear drafts from localStorage when a 404 error occurs. - Integrated errorHandler to notify users of the error condition. - Introduced comprehensive tests to validate the new behavior, ensuring drafts are cleared and error handling is triggered correctly.C * feat: add STREAM_EXPIRED error handling and message localization - Introduced handling for STREAM_EXPIRED errors in useResumableSSE, updating errorHandler to provide relevant feedback. - Added a new error message for STREAM_EXPIRED in translation files for user notifications. - Updated tests to ensure proper error handling and message verification for STREAM_EXPIRED scenarios. * refactor: replace clearDraft with clearAllDrafts utility - Removed the clearDraft function from useResumableSSE and useSSE hooks, replacing it with the new clearAllDrafts utility for better draft management. - Updated localStorage interactions to ensure both text and file drafts are cleared consistently for a conversation. - Enhanced code readability and maintainability by centralizing draft clearing logic.	2026-03-19 14:51:28 -04:00
Danny Avila	b5a55b23a4	📦 chore: NPM audit packages (#12286) ... * 🔧 chore: Update dependencies in package-lock.json and package.json - Bump @aws-sdk/client-bedrock-runtime from 3.980.0 to 3.1011.0 and update related dependencies. - Update fast-xml-parser version from 5.3.8 to 5.5.6 in package.json. - Adjust various @aws-sdk and @smithy packages to their latest versions for improved functionality and security. * 🔧 chore: Update @librechat/agents dependency to version 3.1.57 in package.json and package-lock.json - Bump @librechat/agents from 3.1.56 to 3.1.57 across multiple package files for consistency. - Remove axios dependency from package.json as it is no longer needed.	2026-03-17 17:04:18 -04:00
Danny Avila	1e1a3a8f8d	✨ v0.8.4-rc1 (#12285) ... - App version: v0.8.3 → v0.8.4-rc1 - @librechat/api: 1.7.25 → 1.7.26 - @librechat/client: 0.4.54 → 0.4.55 - librechat-data-provider: 0.8.302 → 0.8.400 - @librechat/data-schemas: 0.0.38 → 0.0.39	2026-03-17 16:08:48 -04:00
Danny Avila	2f09d29c71	🛂 fix: Validate types Query Param in People Picker Access Middleware (#12276) ... * 🛂 fix: Validate `types` query param in people picker access middleware checkPeoplePickerAccess only inspected `req.query.type` (singular), allowing callers to bypass type-specific permission checks by using the `types` (plural) parameter accepted by the controller. Now both `type` and `types` are collected and each requested principal type is validated against the caller's role permissions. * 🛂 refactor: Hoist valid types constant, improve logging, and add edge-case tests - Hoist VALID_PRINCIPAL_TYPES to module-level Set to avoid per-request allocation - Include both `type` and `types` in error log for debuggability - Restore detailed JSDoc documenting per-type permission requirements - Add missing .json() assertion on partial-denial test - Add edge-case tests: all-invalid types, empty string types, PrincipalType.PUBLIC * 🏷️ fix: Align TPrincipalSearchParams with actual controller API The stale type used `type` (singular) but the controller and all callers use `types` (plural array). Aligns with PrincipalSearchParams in types/queries.ts.	2026-03-17 02:46:11 -04:00
Danny Avila	68435cdcd0	🧯 fix: Add Pre-Parse File Size Guard to Document Parser (#12275) ... Prevent memory exhaustion DoS by rejecting documents exceeding 15MB before reading them into memory, closing the gap between the 512MB upload limit and unbounded in-memory parsing.	2026-03-17 02:36:18 -04:00