🏗️ refactor: Remove Redundant Caching, Migrate Config Services to TypeScript (#12466)
* ♻️ refactor: Remove redundant scopedCacheKey caching, support user-provided key model fetching
Remove redundant cache layers that used `scopedCacheKey()` (tenant-only scoping)
on top of `getAppConfig()` which already caches per-principal (role+user+tenant).
This caused config overrides for different principals within the same tenant to
be invisible due to stale cached data.
Changes:
- Add `requireJwtAuth` to `/api/endpoints` route for proper user context
- Remove ENDPOINT_CONFIG, STARTUP_CONFIG, PLUGINS, TOOLS, and MODELS_CONFIG
cache layers — all derive from `getAppConfig()` with cheap computation
- Enhance MODEL_QUERIES cache: hash(baseURL+apiKey) keys, 2-minute TTL,
caching centralized in `fetchModels()` base function
- Support fetching models with user-provided API keys in `loadConfigModels`
via `getUserKeyValues` lookup (no caching for user keys)
- Update all affected tests
Closes #1028
* ♻️ refactor: Migrate config services to TypeScript in packages/api
Move core config logic from CJS /api wrappers to typed TypeScript in
packages/api using dependency injection factories:
- `createEndpointsConfigService` — endpoint config merging + checkCapability
- `createLoadConfigModels` — custom endpoint model loading with user key support
- `createMCPToolCacheService` — MCP tool cache operations (update, merge, cache)
/api files become thin wrappers that wire dependencies (getAppConfig,
loadDefaultEndpointsConfig, getUserKeyValues, getCachedTools, etc.)
into the typed factories.
Also moves existing `endpoints/config.ts` → `endpoints/config/providers.ts`
to accommodate the new `config/` directory structure.
* 🔄 fix: Invalidate models query when user API key is set or revoked
Without this, users had to refresh the page after entering their API key
to see the updated model list fetched with their credentials.
- Invalidate QueryKeys.models in useUpdateUserKeysMutation onSuccess
- Invalidate QueryKeys.models in useRevokeUserKeyMutation onSuccess
- Invalidate QueryKeys.models in useRevokeAllUserKeysMutation onSuccess
* 🗺️ fix: Remap YAML-level override keys to AppConfig equivalents in mergeConfigOverrides
Config overrides stored in the DB use YAML-level keys (TCustomConfig),
but they're merged into the already-processed AppConfig where some fields
have been renamed by AppService. This caused mcpServers overrides to land
on a nonexistent key instead of mcpConfig, so config-override MCP servers
never appeared in the UI.
- Add OVERRIDE_KEY_MAP to remap mcpServers→mcpConfig, interface→interfaceConfig
- Apply remapping before deep merge in mergeConfigOverrides
- Add test for YAML-level key remapping behavior
- Update existing tests to use AppConfig field names in assertions
* 🧪 test: Update service.spec to use AppConfig field names after override key remapping
* 🛡️ fix: Address code review findings — reliability, types, tests, and performance
- Pass tenant context (getTenantId) in importers.js getEndpointsConfig call
- Add 5 tests for user-provided API key model fetching (key found, no key,
DB error, missing userId, apiKey-only with fixed baseURL)
- Distinguish NO_USER_KEY (debug) from infrastructure errors (warn) in catch
- Switch fetchPromisesMap from Promise.all to Promise.allSettled so one
failing provider doesn't kill the entire model config
- Parallelize getUserKeyValues DB lookups via batched Promise.allSettled
instead of sequential awaits in the loop
- Hoist standardCache instance in fetchModels to avoid double instantiation
- Replace Record<string, unknown> types with Partial<TConfig>-based types;
remove as unknown as T double-cast in endpoints config
- Narrow Bedrock availableRegions to typed destructure
- Narrow version field from string|number|undefined to string|undefined
- Fix import ordering in mcp/tools.ts and config/models.ts per AGENTS.md
- Add JSDoc to getModelsConfig alias clarifying caching semantics
* fix: Guard against null getCachedTools in mergeAppTools
* 🔍 fix: Address follow-up review — deduplicate extractEnvVariable, fix error discrimination, add log-level tests
- Deduplicate extractEnvVariable calls: resolve apiKey/baseURL once, reuse
for both the entry and isUserProvided checks (Finding A)
- Move ResolvedEndpoint interface from function closure to module scope (Finding B)
- Replace fragile msg.includes('NO_USER_KEY') with ErrorTypes.NO_USER_KEY
enum check against actual error message format (Finding C). Also handle
ErrorTypes.INVALID_USER_KEY as an expected "no key" case.
- Add test asserting logger.warn is called for infra errors (not debug)
- Add test asserting logger.debug is called for NO_USER_KEY errors (not warn)
* fix: Preserve numeric assistants version via String() coercion
* 🐛 fix: Address secondary review — Ollama cache bypass, cache tests, type safety
- Fix Ollama success path bypassing cache write in fetchModels (CRITICAL):
store result before returning so Ollama models benefit from 2-minute TTL
- Add 4 fetchModels cache behavior tests: cache write with TTL, cache hit
short-circuits HTTP, skipCache bypasses read+write, empty results not cached
- Type-safe OVERRIDE_KEY_MAP: Partial<Record<keyof TCustomConfig, keyof AppConfig>>
so compiler catches future field rename mismatches
- Fix import ordering in config/models.ts (package types longest→shortest)
- Rename ToolCacheDeps → MCPToolCacheDeps for naming consistency
- Expand getModelsConfig JSDoc to explain caching granularity
* fix: Narrow OVERRIDE_KEY_MAP index to satisfy strict tsconfig
* 🧩 fix: Add allowedProviders to TConfig, remove Record<string, unknown> from PartialEndpointEntry
The agents endpoint config includes allowedProviders (used by the frontend
AgentPanel to filter available providers), but it was missing from TConfig.
This forced PartialEndpointEntry to use & Record<string, unknown> as an
escape hatch, violating AGENTS.md type policy.
- Add allowedProviders?: (string | EModelEndpoint)[] to TConfig
- Remove Record<string, unknown> from PartialEndpointEntry — now just Partial<TConfig>
* 🛡️ fix: Isolate Ollama cache write from fetch try-catch, add Ollama cache tests
- Separate Ollama fetch and cache write into distinct scopes so a cache
failure (e.g., Redis down) doesn't misattribute the error as an Ollama
API failure and fall through to the OpenAI-compatible path (Issue A)
- Add 2 Ollama-specific cache tests: models written with TTL on fetch,
cached models returned without hitting server (Issue B)
- Replace hardcoded 120000 with Time.TWO_MINUTES constant in cache TTL
test assertion (Issue C)
- Fix OVERRIDE_KEY_MAP JSDoc to accurately describe runtime vs compile-time
type enforcement (Issue D)
- Add global beforeEach for cache mock reset to prevent cross-test leakage
* 🧪 fix: Address third review — DI consistency, cache key width, MCP tests
- Inject loadCustomEndpointsConfig via EndpointsConfigDeps with default
fallback, matching loadDefaultEndpointsConfig DI pattern (Finding 3)
- Widen modelsCacheKey from 64-bit (.slice(0,16)) to 128-bit (.slice(0,32))
for collision-sensitive cross-credential cache key (Finding 4)
- Add fetchModels.mockReset() in loadConfigModels.spec beforeEach to
prevent mock implementation leaks across tests (Finding 5)
- Add 11 unit tests for createMCPToolCacheService covering all three
functions: null/empty input, successful ops, error propagation,
cold-cache merge (Finding 2)
- Simplify getModelsConfig JSDoc to @see reference (Finding 10)
* ♻️ refactor: Address remaining follow-ups from reviews
OVERRIDE_KEY_MAP completeness:
- Add missing turnstile→turnstileConfig mapping
- Add exhaustiveness test verifying all three renamed keys are remapped
and original YAML keys don't leak through
Import role context:
- Pass userRole through importConversations job → importLibreChatConvo
so role-based endpoint overrides are honored during conversation import
- Update convos.js route to include req.user.role in the job payload
createEndpointsConfigService unit tests:
- Add 8 tests covering: default+custom merge, Azure/AzureAssistants/
Anthropic Vertex/Bedrock config enrichment, assistants version
coercion, agents allowedProviders, req.config bypass
Plugins/tools efficiency:
- Use Set for includedTools/filteredTools lookups (O(1) vs O(n) per plugin)
- Combine auth check + filter into single pass (eliminates intermediate array)
- Pre-compute toolDefKeys Set for O(1) tool definition lookups
* fix: Scope model query cache by user when userIdQuery is enabled
* fix: Skip model cache for userIdQuery endpoints, fix endpoints test types
- When userIdQuery is true, skip caching entirely (like user_provided keys)
to avoid cross-user model list leakage without duplicating cache data
- Fix AgentCapabilities type error in endpoints.spec.ts — use enum values
and appConfig() helper for partial mock typing
* 🐛 fix: Restore filteredTools+includedTools composition, add checkCapability tests
- Fix filteredTools regression: whitelist and blacklist are now applied
independently (two flat guards), matching original behavior where
includedTools=['a','b'] + filteredTools=['b'] produces ['a'] (Finding A)
- Fix Set spread in toolkit loop: pre-compute toolDefKeysList array once
alongside the Set, reuse for .some() without per-plugin allocation (Finding B)
- Add 2 filteredTools tests: blacklist-only path and combined
whitelist+blacklist composition (Finding C)
- Add 3 checkCapability tests: capability present, capability absent,
fallback to defaultAgentCapabilities for non-agents endpoints (Finding D)
* 🔑 fix: Include config-override MCP servers in filterAuthorizedTools
Config-override MCP servers (defined via admin config overrides for
roles/groups) were rejected by filterAuthorizedTools because it called
getAllServerConfigs(userId) without the configServers parameter. Only
YAML and DB-backed user servers were included in the access check.
- Add configServers parameter to filterAuthorizedTools
- Resolve config servers via resolveConfigServers(req) at all 4 callsites
(create, update, duplicate, revert) using parallel Promise.all
- Pass configServers through to getAllServerConfigs(userId, configServers)
so the registry merges config-source servers into the access check
- Update filterAuthorizedTools.spec.js mock for resolveConfigServers
* fix: Skip model cache for userIdQuery endpoints, fix endpoints test types
For user-provided key endpoints (userProvide: true), skip the full model
list re-fetch during message validation — the user already selected from
a list we served them, and re-fetching with skipCache:true on every
message send is both slow and fragile (5s provider timeout = rejected model).
Instead, validate the model string format only:
- Must be a string, max 256 chars
- Must match [a-zA-Z0-9][a-zA-Z0-9_.:\-/@+ ]* (covers all known provider
model ID formats while rejecting injection attempts)
System-configured endpoints still get full model list validation as before.
* 🧪 test: Add regression tests for filterAuthorizedTools configServers and validateModel
filterAuthorizedTools:
- Add test verifying configServers is passed to getAllServerConfigs and
config-override server tools are allowed through
- Guard resolveConfigServers in createAgentHandler to only run when
MCP tools are present (skip for tool-free agent creates)
validateModel (12 new tests):
- Format validation: missing model, non-string, length overflow, leading
special char, script injection, standard model ID acceptance
- userProvide early-return: next() called immediately, getModelsConfig
not invoked (regression guard for the exact bug this fixes)
- System endpoint list validation: reject unknown model, accept known
model, handle null/missing models config
Also fix unnecessary backslash escape in MODEL_PATTERN regex.
* 🧹 fix: Remove space from MODEL_PATTERN, trim input, clean up nits
- Remove space character from MODEL_PATTERN regex — no real model ID
uses spaces; prevents spurious violation logs from whitespace artifacts
- Add model.trim() before validation to handle accidental whitespace
- Remove redundant filterUniquePlugins call on already-deduplicated output
- Add comment documenting intentional whitelist+blacklist composition
- Add getUserKeyValues.mockReset() in loadConfigModels.spec beforeEach
- Remove narrating JSDoc from getModelsConfig one-liner
- Add 2 tests: trim whitespace handling, reject spaces in model ID
* fix: Match startup tool loader semantics — includedTools takes precedence over filteredTools
The startup tool loader (loadAndFormatTools) explicitly ignores
filteredTools when includedTools is set, with a warning log. The
PluginController was applying both independently, creating inconsistent
behavior where the same config produced different results at startup
vs plugin listing time.
Restored mutually exclusive semantics: when includedTools is non-empty,
filteredTools is not evaluated.
* 🧹 chore: Simplify validateModel flow, note auth requirement on endpoints route
- Separate missing-model from invalid-model checks cleanly: type+presence
guard first, then trim+format guard (reviewer NIT)
- Add route comment noting auth is required for role/tenant scoping
* fix: Write trimmed model back to req.body.model for downstream consumers
2026-03-30 16:49:48 -04:00
|
|
|
const { logger } = require('@librechat/data-schemas');
|
💫 feat: Config File & Custom Endpoints (#1474)
* WIP(backend/api): custom endpoint
* WIP(frontend/client): custom endpoint
* chore: adjust typedefs for configs
* refactor: use data-provider for cache keys and rename enums and custom endpoint for better clarity and compatibility
* feat: loadYaml utility
* refactor: rename back to from and proof-of-concept for creating schemas from user-defined defaults
* refactor: remove custom endpoint from default endpointsConfig as it will be exclusively managed by yaml config
* refactor(EndpointController): rename variables for clarity
* feat: initial load custom config
* feat(server/utils): add simple `isUserProvided` helper
* chore(types): update TConfig type
* refactor: remove custom endpoint handling from model services as will be handled by config, modularize fetching of models
* feat: loadCustomConfig, loadConfigEndpoints, loadConfigModels
* chore: reorganize server init imports, invoke loadCustomConfig
* refactor(loadConfigEndpoints/Models): return each custom endpoint as standalone endpoint
* refactor(Endpoint/ModelController): spread config values after default (temporary)
* chore(client): fix type issues
* WIP: first pass for multiple custom endpoints
- add endpointType to Conversation schema
- add update zod schemas for both convo/presets to allow non-EModelEndpoint value as endpoint (also using type assertion)
- use `endpointType` value as `endpoint` where mapping to type is necessary using this field
- use custom defined `endpoint` value and not type for mapping to modelsConfig
- misc: add return type to `getDefaultEndpoint`
- in `useNewConvo`, add the endpointType if it wasn't already added to conversation
- EndpointsMenu: use user-defined endpoint name as Title in menu
- TODO: custom icon via custom config, change unknown to robot icon
* refactor(parseConvo): pass args as an object and change where used accordingly; chore: comment out 'create schema' code
* chore: remove unused availableModels field in TConfig type
* refactor(parseCompactConvo): pass args as an object and change where used accordingly
* feat: chat through custom endpoint
* chore(message/convoSchemas): avoid saving empty arrays
* fix(BaseClient/saveMessageToDatabase): save endpointType
* refactor(ChatRoute): show Spinner if endpointsQuery or modelsQuery are still loading, which is apparent with slow fetching of models/remote config on first serve
* fix(useConversation): assign endpointType if it's missing
* fix(SaveAsPreset): pass real endpoint and endpointType when saving Preset)
* chore: recorganize types order for TConfig, add `iconURL`
* feat: custom endpoint icon support:
- use UnknownIcon in all icon contexts
- add mistral and openrouter as known endpoints, and add their icons
- iconURL support
* fix(presetSchema): move endpointType to default schema definitions shared between convoSchema and defaults
* refactor(Settings/OpenAI): remove legacy `isOpenAI` flag
* fix(OpenAIClient): do not invoke abortCompletion on completion error
* feat: add responseSender/label support for custom endpoints:
- use defaultModelLabel field in endpointOption
- add model defaults for custom endpoints in `getResponseSender`
- add `useGetSender` hook which uses EndpointsQuery to determine `defaultModelLabel`
- include defaultModelLabel from endpointConfig in custom endpoint client options
- pass `endpointType` to `getResponseSender`
* feat(OpenAIClient): use custom options from config file
* refactor: rename `defaultModelLabel` to `modelDisplayLabel`
* refactor(data-provider): separate concerns from `schemas` into `parsers`, `config`, and fix imports elsewhere
* feat: `iconURL` and extract environment variables from custom endpoint config values
* feat: custom config validation via zod schema, rename and move to `./projectRoot/librechat.yaml`
* docs: custom config docs and examples
* fix(OpenAIClient/mistral): mistral does not allow singular system message, also add `useChatCompletion` flag to use openai-node for title completions
* fix(custom/initializeClient): extract env var and use `isUserProvided` function
* Update librechat.example.yaml
* feat(InputWithLabel): add className props, and forwardRef
* fix(streamResponse): handle error edge case where either messages or convos query throws an error
* fix(useSSE): handle errorHandler edge cases where error response is and is not properly formatted from API, especially when a conversationId is not yet provided, which ensures stream is properly closed on error
* feat: user_provided keys for custom endpoints
* fix(config/endpointSchema): do not allow default endpoint values in custom endpoint `name`
* feat(loadConfigModels): extract env variables and optimize fetching models
* feat: support custom endpoint iconURL for messages and Nav
* feat(OpenAIClient): add/dropParams support
* docs: update docs with default params, add/dropParams, and notes to use config file instead of `OPENAI_REVERSE_PROXY`
* docs: update docs with additional notes
* feat(maxTokensMap): add mistral models (32k context)
* docs: update openrouter notes
* Update ai_setup.md
* docs(custom_config): add table of contents and fix note about custom name
* docs(custom_config): reorder ToC
* Update custom_config.md
* Add note about `max_tokens` field in custom_config.md
2024-01-03 09:22:48 -05:00
|
|
|
const { loadDefaultModels, loadConfigModels } = require('~/server/services/Config');
|
WIP: Update UI to match Official Style; Vision and Assistants 👷🏽 (#1190)
* wip: initial client side code
* wip: initial api code
* refactor: export query keys from own module, export assistant hooks
* refactor(SelectDropDown): more customization via props
* feat: create Assistant and render real Assistants
* refactor: major refactor of UI components to allow multi-chat, working alongside CreationPanel
* refactor: move assistant routes to own directory
* fix(CreationHeader): state issue with assistant select
* refactor: style changes for form, fix setSiblingIdx from useChatHelpers to use latestMessageParentId, fix render issue with ChatView and change location
* feat: parseCompactConvo: begin refactor of slimmer JSON payloads between client/api
* refactor(endpoints): add assistant endpoint, also use EModelEndpoint as much as possible
* refactor(useGetConversationsQuery): use object to access query data easily
* fix(MultiMessage): react warning of bad state set, making use of effect during render (instead of useEffect)
* fix(useNewConvo): use correct atom key (index instead of convoId) for reset latestMessageFamily
* refactor: make routing navigation/conversation change simpler
* chore: add removeNullishValues for smaller payloads, remove unused fields, setup frontend pinging of assistant endpoint
* WIP: initial complete assistant run handling
* fix: CreationPanel form correctly setting internal state
* refactor(api/assistants/chat): revise functions to working run handling strategy
* refactor(UI): initial major refactor of ChatForm and options
* feat: textarea hook
* refactor: useAuthRedirect hook and change directory name
* feat: add ChatRoute (/c/), make optionsBar absolute and change on textarea height, add temp header
* feat: match new toggle Nav open button to ChatGPT's
* feat: add OpenAI custom classnames
* feat: useOriginNavigate
* feat: messages loading view
* fix: conversation navigation and effects
* refactor: make toggle change nav opacity
* WIP: new endpoint menu
* feat: NewEndpointsMenu complete
* fix: ensure set key dialog shows on endpoint change, and new conversation resets messages
* WIP: textarea styling fix, add temp footer, create basic file handling component
* feat: image file handling (UI)
* feat: PopOver and ModelSelect in Header, remove GenButtons
* feat: drop file handling
* refactor: bug fixes
use SSE at route level
add opts to useOriginNavigate
delay render of unfinishedMessage to avoid flickering
pass params (convoId) to chatHelpers to set messages query data based on param when the route is new (fixes can't continue convo on /new/)
style(MessagesView): matches height to official
fix(SSE): pass paramId and invalidate convos
style(Message): make bg uniform
* refactor(useSSE): setStorage within setConversation updates
* feat: conversationKeysAtom, allConversationsSelector, update convos query data on created message (if new), correctly handle convo deletion (individual)
* feat: add popover select dropdowns to allow options in header while allowing horizontal scroll for mobile
* style(pluginsSelect): styling changes
* refactor(NewEndpointsMenu): make UI components modular
* feat: Presets complete
* fix: preset editing, make by index
* fix: conversations not setting on inital navigation, fix getMessages() based on query param
* fix: changing preset no longer resets latestMessage
* feat: useOnClickOutside for OptionsPopover and fix bug that causes selection of preset when deleting
* fix: revert /chat/ switchToConvo, also use NewDeleteButton in Convo
* fix: Popover correctly closes on close Popover button using custom condition for useOnClickOutside
* style: new message and nav styling
* style: hover/sibling buttons and preset menu scrolling
* feat: new convo header button
* style(Textarea): minor style changes to textarea buttons
* feat: stop/continue generating and hide hoverbuttons when submitting
* feat: compact AI Provider schemas to make json payloads and db saves smaller
* style: styling changes for consistency on chat route
* fix: created usePresetIndexOptions to prevent bugs between /c/ and /chat/ routes when editing presets, removed redundant code from the new dialog
* chore: make /chat/ route default for now since we still lack full image support
2023-11-16 10:42:24 -05:00
|
|
|
|
🏗️ refactor: Remove Redundant Caching, Migrate Config Services to TypeScript (#12466)
* ♻️ refactor: Remove redundant scopedCacheKey caching, support user-provided key model fetching
Remove redundant cache layers that used `scopedCacheKey()` (tenant-only scoping)
on top of `getAppConfig()` which already caches per-principal (role+user+tenant).
This caused config overrides for different principals within the same tenant to
be invisible due to stale cached data.
Changes:
- Add `requireJwtAuth` to `/api/endpoints` route for proper user context
- Remove ENDPOINT_CONFIG, STARTUP_CONFIG, PLUGINS, TOOLS, and MODELS_CONFIG
cache layers — all derive from `getAppConfig()` with cheap computation
- Enhance MODEL_QUERIES cache: hash(baseURL+apiKey) keys, 2-minute TTL,
caching centralized in `fetchModels()` base function
- Support fetching models with user-provided API keys in `loadConfigModels`
via `getUserKeyValues` lookup (no caching for user keys)
- Update all affected tests
Closes #1028
* ♻️ refactor: Migrate config services to TypeScript in packages/api
Move core config logic from CJS /api wrappers to typed TypeScript in
packages/api using dependency injection factories:
- `createEndpointsConfigService` — endpoint config merging + checkCapability
- `createLoadConfigModels` — custom endpoint model loading with user key support
- `createMCPToolCacheService` — MCP tool cache operations (update, merge, cache)
/api files become thin wrappers that wire dependencies (getAppConfig,
loadDefaultEndpointsConfig, getUserKeyValues, getCachedTools, etc.)
into the typed factories.
Also moves existing `endpoints/config.ts` → `endpoints/config/providers.ts`
to accommodate the new `config/` directory structure.
* 🔄 fix: Invalidate models query when user API key is set or revoked
Without this, users had to refresh the page after entering their API key
to see the updated model list fetched with their credentials.
- Invalidate QueryKeys.models in useUpdateUserKeysMutation onSuccess
- Invalidate QueryKeys.models in useRevokeUserKeyMutation onSuccess
- Invalidate QueryKeys.models in useRevokeAllUserKeysMutation onSuccess
* 🗺️ fix: Remap YAML-level override keys to AppConfig equivalents in mergeConfigOverrides
Config overrides stored in the DB use YAML-level keys (TCustomConfig),
but they're merged into the already-processed AppConfig where some fields
have been renamed by AppService. This caused mcpServers overrides to land
on a nonexistent key instead of mcpConfig, so config-override MCP servers
never appeared in the UI.
- Add OVERRIDE_KEY_MAP to remap mcpServers→mcpConfig, interface→interfaceConfig
- Apply remapping before deep merge in mergeConfigOverrides
- Add test for YAML-level key remapping behavior
- Update existing tests to use AppConfig field names in assertions
* 🧪 test: Update service.spec to use AppConfig field names after override key remapping
* 🛡️ fix: Address code review findings — reliability, types, tests, and performance
- Pass tenant context (getTenantId) in importers.js getEndpointsConfig call
- Add 5 tests for user-provided API key model fetching (key found, no key,
DB error, missing userId, apiKey-only with fixed baseURL)
- Distinguish NO_USER_KEY (debug) from infrastructure errors (warn) in catch
- Switch fetchPromisesMap from Promise.all to Promise.allSettled so one
failing provider doesn't kill the entire model config
- Parallelize getUserKeyValues DB lookups via batched Promise.allSettled
instead of sequential awaits in the loop
- Hoist standardCache instance in fetchModels to avoid double instantiation
- Replace Record<string, unknown> types with Partial<TConfig>-based types;
remove as unknown as T double-cast in endpoints config
- Narrow Bedrock availableRegions to typed destructure
- Narrow version field from string|number|undefined to string|undefined
- Fix import ordering in mcp/tools.ts and config/models.ts per AGENTS.md
- Add JSDoc to getModelsConfig alias clarifying caching semantics
* fix: Guard against null getCachedTools in mergeAppTools
* 🔍 fix: Address follow-up review — deduplicate extractEnvVariable, fix error discrimination, add log-level tests
- Deduplicate extractEnvVariable calls: resolve apiKey/baseURL once, reuse
for both the entry and isUserProvided checks (Finding A)
- Move ResolvedEndpoint interface from function closure to module scope (Finding B)
- Replace fragile msg.includes('NO_USER_KEY') with ErrorTypes.NO_USER_KEY
enum check against actual error message format (Finding C). Also handle
ErrorTypes.INVALID_USER_KEY as an expected "no key" case.
- Add test asserting logger.warn is called for infra errors (not debug)
- Add test asserting logger.debug is called for NO_USER_KEY errors (not warn)
* fix: Preserve numeric assistants version via String() coercion
* 🐛 fix: Address secondary review — Ollama cache bypass, cache tests, type safety
- Fix Ollama success path bypassing cache write in fetchModels (CRITICAL):
store result before returning so Ollama models benefit from 2-minute TTL
- Add 4 fetchModels cache behavior tests: cache write with TTL, cache hit
short-circuits HTTP, skipCache bypasses read+write, empty results not cached
- Type-safe OVERRIDE_KEY_MAP: Partial<Record<keyof TCustomConfig, keyof AppConfig>>
so compiler catches future field rename mismatches
- Fix import ordering in config/models.ts (package types longest→shortest)
- Rename ToolCacheDeps → MCPToolCacheDeps for naming consistency
- Expand getModelsConfig JSDoc to explain caching granularity
* fix: Narrow OVERRIDE_KEY_MAP index to satisfy strict tsconfig
* 🧩 fix: Add allowedProviders to TConfig, remove Record<string, unknown> from PartialEndpointEntry
The agents endpoint config includes allowedProviders (used by the frontend
AgentPanel to filter available providers), but it was missing from TConfig.
This forced PartialEndpointEntry to use & Record<string, unknown> as an
escape hatch, violating AGENTS.md type policy.
- Add allowedProviders?: (string | EModelEndpoint)[] to TConfig
- Remove Record<string, unknown> from PartialEndpointEntry — now just Partial<TConfig>
* 🛡️ fix: Isolate Ollama cache write from fetch try-catch, add Ollama cache tests
- Separate Ollama fetch and cache write into distinct scopes so a cache
failure (e.g., Redis down) doesn't misattribute the error as an Ollama
API failure and fall through to the OpenAI-compatible path (Issue A)
- Add 2 Ollama-specific cache tests: models written with TTL on fetch,
cached models returned without hitting server (Issue B)
- Replace hardcoded 120000 with Time.TWO_MINUTES constant in cache TTL
test assertion (Issue C)
- Fix OVERRIDE_KEY_MAP JSDoc to accurately describe runtime vs compile-time
type enforcement (Issue D)
- Add global beforeEach for cache mock reset to prevent cross-test leakage
* 🧪 fix: Address third review — DI consistency, cache key width, MCP tests
- Inject loadCustomEndpointsConfig via EndpointsConfigDeps with default
fallback, matching loadDefaultEndpointsConfig DI pattern (Finding 3)
- Widen modelsCacheKey from 64-bit (.slice(0,16)) to 128-bit (.slice(0,32))
for collision-sensitive cross-credential cache key (Finding 4)
- Add fetchModels.mockReset() in loadConfigModels.spec beforeEach to
prevent mock implementation leaks across tests (Finding 5)
- Add 11 unit tests for createMCPToolCacheService covering all three
functions: null/empty input, successful ops, error propagation,
cold-cache merge (Finding 2)
- Simplify getModelsConfig JSDoc to @see reference (Finding 10)
* ♻️ refactor: Address remaining follow-ups from reviews
OVERRIDE_KEY_MAP completeness:
- Add missing turnstile→turnstileConfig mapping
- Add exhaustiveness test verifying all three renamed keys are remapped
and original YAML keys don't leak through
Import role context:
- Pass userRole through importConversations job → importLibreChatConvo
so role-based endpoint overrides are honored during conversation import
- Update convos.js route to include req.user.role in the job payload
createEndpointsConfigService unit tests:
- Add 8 tests covering: default+custom merge, Azure/AzureAssistants/
Anthropic Vertex/Bedrock config enrichment, assistants version
coercion, agents allowedProviders, req.config bypass
Plugins/tools efficiency:
- Use Set for includedTools/filteredTools lookups (O(1) vs O(n) per plugin)
- Combine auth check + filter into single pass (eliminates intermediate array)
- Pre-compute toolDefKeys Set for O(1) tool definition lookups
* fix: Scope model query cache by user when userIdQuery is enabled
* fix: Skip model cache for userIdQuery endpoints, fix endpoints test types
- When userIdQuery is true, skip caching entirely (like user_provided keys)
to avoid cross-user model list leakage without duplicating cache data
- Fix AgentCapabilities type error in endpoints.spec.ts — use enum values
and appConfig() helper for partial mock typing
* 🐛 fix: Restore filteredTools+includedTools composition, add checkCapability tests
- Fix filteredTools regression: whitelist and blacklist are now applied
independently (two flat guards), matching original behavior where
includedTools=['a','b'] + filteredTools=['b'] produces ['a'] (Finding A)
- Fix Set spread in toolkit loop: pre-compute toolDefKeysList array once
alongside the Set, reuse for .some() without per-plugin allocation (Finding B)
- Add 2 filteredTools tests: blacklist-only path and combined
whitelist+blacklist composition (Finding C)
- Add 3 checkCapability tests: capability present, capability absent,
fallback to defaultAgentCapabilities for non-agents endpoints (Finding D)
* 🔑 fix: Include config-override MCP servers in filterAuthorizedTools
Config-override MCP servers (defined via admin config overrides for
roles/groups) were rejected by filterAuthorizedTools because it called
getAllServerConfigs(userId) without the configServers parameter. Only
YAML and DB-backed user servers were included in the access check.
- Add configServers parameter to filterAuthorizedTools
- Resolve config servers via resolveConfigServers(req) at all 4 callsites
(create, update, duplicate, revert) using parallel Promise.all
- Pass configServers through to getAllServerConfigs(userId, configServers)
so the registry merges config-source servers into the access check
- Update filterAuthorizedTools.spec.js mock for resolveConfigServers
* fix: Skip model cache for userIdQuery endpoints, fix endpoints test types
For user-provided key endpoints (userProvide: true), skip the full model
list re-fetch during message validation — the user already selected from
a list we served them, and re-fetching with skipCache:true on every
message send is both slow and fragile (5s provider timeout = rejected model).
Instead, validate the model string format only:
- Must be a string, max 256 chars
- Must match [a-zA-Z0-9][a-zA-Z0-9_.:\-/@+ ]* (covers all known provider
model ID formats while rejecting injection attempts)
System-configured endpoints still get full model list validation as before.
* 🧪 test: Add regression tests for filterAuthorizedTools configServers and validateModel
filterAuthorizedTools:
- Add test verifying configServers is passed to getAllServerConfigs and
config-override server tools are allowed through
- Guard resolveConfigServers in createAgentHandler to only run when
MCP tools are present (skip for tool-free agent creates)
validateModel (12 new tests):
- Format validation: missing model, non-string, length overflow, leading
special char, script injection, standard model ID acceptance
- userProvide early-return: next() called immediately, getModelsConfig
not invoked (regression guard for the exact bug this fixes)
- System endpoint list validation: reject unknown model, accept known
model, handle null/missing models config
Also fix unnecessary backslash escape in MODEL_PATTERN regex.
* 🧹 fix: Remove space from MODEL_PATTERN, trim input, clean up nits
- Remove space character from MODEL_PATTERN regex — no real model ID
uses spaces; prevents spurious violation logs from whitespace artifacts
- Add model.trim() before validation to handle accidental whitespace
- Remove redundant filterUniquePlugins call on already-deduplicated output
- Add comment documenting intentional whitelist+blacklist composition
- Add getUserKeyValues.mockReset() in loadConfigModels.spec beforeEach
- Remove narrating JSDoc from getModelsConfig one-liner
- Add 2 tests: trim whitespace handling, reject spaces in model ID
* fix: Match startup tool loader semantics — includedTools takes precedence over filteredTools
The startup tool loader (loadAndFormatTools) explicitly ignores
filteredTools when includedTools is set, with a warning log. The
PluginController was applying both independently, creating inconsistent
behavior where the same config produced different results at startup
vs plugin listing time.
Restored mutually exclusive semantics: when includedTools is non-empty,
filteredTools is not evaluated.
* 🧹 chore: Simplify validateModel flow, note auth requirement on endpoints route
- Separate missing-model from invalid-model checks cleanly: type+presence
guard first, then trim+format guard (reviewer NIT)
- Add route comment noting auth is required for role/tenant scoping
* fix: Write trimmed model back to req.body.model for downstream consumers
2026-03-30 16:49:48 -04:00
|
|
|
const getModelsConfig = (req) => loadModels(req);
|
2024-03-06 00:04:52 -05:00
|
|
|
|
2024-02-20 12:57:58 -05:00
|
|
|
async function loadModels(req) {
|
2024-02-08 10:06:58 -05:00
|
|
|
const defaultModelsConfig = await loadDefaultModels(req);
|
|
|
|
|
const customModelsConfig = await loadConfigModels(req);
|
🏗️ refactor: Remove Redundant Caching, Migrate Config Services to TypeScript (#12466)
* ♻️ refactor: Remove redundant scopedCacheKey caching, support user-provided key model fetching
Remove redundant cache layers that used `scopedCacheKey()` (tenant-only scoping)
on top of `getAppConfig()` which already caches per-principal (role+user+tenant).
This caused config overrides for different principals within the same tenant to
be invisible due to stale cached data.
Changes:
- Add `requireJwtAuth` to `/api/endpoints` route for proper user context
- Remove ENDPOINT_CONFIG, STARTUP_CONFIG, PLUGINS, TOOLS, and MODELS_CONFIG
cache layers — all derive from `getAppConfig()` with cheap computation
- Enhance MODEL_QUERIES cache: hash(baseURL+apiKey) keys, 2-minute TTL,
caching centralized in `fetchModels()` base function
- Support fetching models with user-provided API keys in `loadConfigModels`
via `getUserKeyValues` lookup (no caching for user keys)
- Update all affected tests
Closes #1028
* ♻️ refactor: Migrate config services to TypeScript in packages/api
Move core config logic from CJS /api wrappers to typed TypeScript in
packages/api using dependency injection factories:
- `createEndpointsConfigService` — endpoint config merging + checkCapability
- `createLoadConfigModels` — custom endpoint model loading with user key support
- `createMCPToolCacheService` — MCP tool cache operations (update, merge, cache)
/api files become thin wrappers that wire dependencies (getAppConfig,
loadDefaultEndpointsConfig, getUserKeyValues, getCachedTools, etc.)
into the typed factories.
Also moves existing `endpoints/config.ts` → `endpoints/config/providers.ts`
to accommodate the new `config/` directory structure.
* 🔄 fix: Invalidate models query when user API key is set or revoked
Without this, users had to refresh the page after entering their API key
to see the updated model list fetched with their credentials.
- Invalidate QueryKeys.models in useUpdateUserKeysMutation onSuccess
- Invalidate QueryKeys.models in useRevokeUserKeyMutation onSuccess
- Invalidate QueryKeys.models in useRevokeAllUserKeysMutation onSuccess
* 🗺️ fix: Remap YAML-level override keys to AppConfig equivalents in mergeConfigOverrides
Config overrides stored in the DB use YAML-level keys (TCustomConfig),
but they're merged into the already-processed AppConfig where some fields
have been renamed by AppService. This caused mcpServers overrides to land
on a nonexistent key instead of mcpConfig, so config-override MCP servers
never appeared in the UI.
- Add OVERRIDE_KEY_MAP to remap mcpServers→mcpConfig, interface→interfaceConfig
- Apply remapping before deep merge in mergeConfigOverrides
- Add test for YAML-level key remapping behavior
- Update existing tests to use AppConfig field names in assertions
* 🧪 test: Update service.spec to use AppConfig field names after override key remapping
* 🛡️ fix: Address code review findings — reliability, types, tests, and performance
- Pass tenant context (getTenantId) in importers.js getEndpointsConfig call
- Add 5 tests for user-provided API key model fetching (key found, no key,
DB error, missing userId, apiKey-only with fixed baseURL)
- Distinguish NO_USER_KEY (debug) from infrastructure errors (warn) in catch
- Switch fetchPromisesMap from Promise.all to Promise.allSettled so one
failing provider doesn't kill the entire model config
- Parallelize getUserKeyValues DB lookups via batched Promise.allSettled
instead of sequential awaits in the loop
- Hoist standardCache instance in fetchModels to avoid double instantiation
- Replace Record<string, unknown> types with Partial<TConfig>-based types;
remove as unknown as T double-cast in endpoints config
- Narrow Bedrock availableRegions to typed destructure
- Narrow version field from string|number|undefined to string|undefined
- Fix import ordering in mcp/tools.ts and config/models.ts per AGENTS.md
- Add JSDoc to getModelsConfig alias clarifying caching semantics
* fix: Guard against null getCachedTools in mergeAppTools
* 🔍 fix: Address follow-up review — deduplicate extractEnvVariable, fix error discrimination, add log-level tests
- Deduplicate extractEnvVariable calls: resolve apiKey/baseURL once, reuse
for both the entry and isUserProvided checks (Finding A)
- Move ResolvedEndpoint interface from function closure to module scope (Finding B)
- Replace fragile msg.includes('NO_USER_KEY') with ErrorTypes.NO_USER_KEY
enum check against actual error message format (Finding C). Also handle
ErrorTypes.INVALID_USER_KEY as an expected "no key" case.
- Add test asserting logger.warn is called for infra errors (not debug)
- Add test asserting logger.debug is called for NO_USER_KEY errors (not warn)
* fix: Preserve numeric assistants version via String() coercion
* 🐛 fix: Address secondary review — Ollama cache bypass, cache tests, type safety
- Fix Ollama success path bypassing cache write in fetchModels (CRITICAL):
store result before returning so Ollama models benefit from 2-minute TTL
- Add 4 fetchModels cache behavior tests: cache write with TTL, cache hit
short-circuits HTTP, skipCache bypasses read+write, empty results not cached
- Type-safe OVERRIDE_KEY_MAP: Partial<Record<keyof TCustomConfig, keyof AppConfig>>
so compiler catches future field rename mismatches
- Fix import ordering in config/models.ts (package types longest→shortest)
- Rename ToolCacheDeps → MCPToolCacheDeps for naming consistency
- Expand getModelsConfig JSDoc to explain caching granularity
* fix: Narrow OVERRIDE_KEY_MAP index to satisfy strict tsconfig
* 🧩 fix: Add allowedProviders to TConfig, remove Record<string, unknown> from PartialEndpointEntry
The agents endpoint config includes allowedProviders (used by the frontend
AgentPanel to filter available providers), but it was missing from TConfig.
This forced PartialEndpointEntry to use & Record<string, unknown> as an
escape hatch, violating AGENTS.md type policy.
- Add allowedProviders?: (string | EModelEndpoint)[] to TConfig
- Remove Record<string, unknown> from PartialEndpointEntry — now just Partial<TConfig>
* 🛡️ fix: Isolate Ollama cache write from fetch try-catch, add Ollama cache tests
- Separate Ollama fetch and cache write into distinct scopes so a cache
failure (e.g., Redis down) doesn't misattribute the error as an Ollama
API failure and fall through to the OpenAI-compatible path (Issue A)
- Add 2 Ollama-specific cache tests: models written with TTL on fetch,
cached models returned without hitting server (Issue B)
- Replace hardcoded 120000 with Time.TWO_MINUTES constant in cache TTL
test assertion (Issue C)
- Fix OVERRIDE_KEY_MAP JSDoc to accurately describe runtime vs compile-time
type enforcement (Issue D)
- Add global beforeEach for cache mock reset to prevent cross-test leakage
* 🧪 fix: Address third review — DI consistency, cache key width, MCP tests
- Inject loadCustomEndpointsConfig via EndpointsConfigDeps with default
fallback, matching loadDefaultEndpointsConfig DI pattern (Finding 3)
- Widen modelsCacheKey from 64-bit (.slice(0,16)) to 128-bit (.slice(0,32))
for collision-sensitive cross-credential cache key (Finding 4)
- Add fetchModels.mockReset() in loadConfigModels.spec beforeEach to
prevent mock implementation leaks across tests (Finding 5)
- Add 11 unit tests for createMCPToolCacheService covering all three
functions: null/empty input, successful ops, error propagation,
cold-cache merge (Finding 2)
- Simplify getModelsConfig JSDoc to @see reference (Finding 10)
* ♻️ refactor: Address remaining follow-ups from reviews
OVERRIDE_KEY_MAP completeness:
- Add missing turnstile→turnstileConfig mapping
- Add exhaustiveness test verifying all three renamed keys are remapped
and original YAML keys don't leak through
Import role context:
- Pass userRole through importConversations job → importLibreChatConvo
so role-based endpoint overrides are honored during conversation import
- Update convos.js route to include req.user.role in the job payload
createEndpointsConfigService unit tests:
- Add 8 tests covering: default+custom merge, Azure/AzureAssistants/
Anthropic Vertex/Bedrock config enrichment, assistants version
coercion, agents allowedProviders, req.config bypass
Plugins/tools efficiency:
- Use Set for includedTools/filteredTools lookups (O(1) vs O(n) per plugin)
- Combine auth check + filter into single pass (eliminates intermediate array)
- Pre-compute toolDefKeys Set for O(1) tool definition lookups
* fix: Scope model query cache by user when userIdQuery is enabled
* fix: Skip model cache for userIdQuery endpoints, fix endpoints test types
- When userIdQuery is true, skip caching entirely (like user_provided keys)
to avoid cross-user model list leakage without duplicating cache data
- Fix AgentCapabilities type error in endpoints.spec.ts — use enum values
and appConfig() helper for partial mock typing
* 🐛 fix: Restore filteredTools+includedTools composition, add checkCapability tests
- Fix filteredTools regression: whitelist and blacklist are now applied
independently (two flat guards), matching original behavior where
includedTools=['a','b'] + filteredTools=['b'] produces ['a'] (Finding A)
- Fix Set spread in toolkit loop: pre-compute toolDefKeysList array once
alongside the Set, reuse for .some() without per-plugin allocation (Finding B)
- Add 2 filteredTools tests: blacklist-only path and combined
whitelist+blacklist composition (Finding C)
- Add 3 checkCapability tests: capability present, capability absent,
fallback to defaultAgentCapabilities for non-agents endpoints (Finding D)
* 🔑 fix: Include config-override MCP servers in filterAuthorizedTools
Config-override MCP servers (defined via admin config overrides for
roles/groups) were rejected by filterAuthorizedTools because it called
getAllServerConfigs(userId) without the configServers parameter. Only
YAML and DB-backed user servers were included in the access check.
- Add configServers parameter to filterAuthorizedTools
- Resolve config servers via resolveConfigServers(req) at all 4 callsites
(create, update, duplicate, revert) using parallel Promise.all
- Pass configServers through to getAllServerConfigs(userId, configServers)
so the registry merges config-source servers into the access check
- Update filterAuthorizedTools.spec.js mock for resolveConfigServers
* fix: Skip model cache for userIdQuery endpoints, fix endpoints test types
For user-provided key endpoints (userProvide: true), skip the full model
list re-fetch during message validation — the user already selected from
a list we served them, and re-fetching with skipCache:true on every
message send is both slow and fragile (5s provider timeout = rejected model).
Instead, validate the model string format only:
- Must be a string, max 256 chars
- Must match [a-zA-Z0-9][a-zA-Z0-9_.:\-/@+ ]* (covers all known provider
model ID formats while rejecting injection attempts)
System-configured endpoints still get full model list validation as before.
* 🧪 test: Add regression tests for filterAuthorizedTools configServers and validateModel
filterAuthorizedTools:
- Add test verifying configServers is passed to getAllServerConfigs and
config-override server tools are allowed through
- Guard resolveConfigServers in createAgentHandler to only run when
MCP tools are present (skip for tool-free agent creates)
validateModel (12 new tests):
- Format validation: missing model, non-string, length overflow, leading
special char, script injection, standard model ID acceptance
- userProvide early-return: next() called immediately, getModelsConfig
not invoked (regression guard for the exact bug this fixes)
- System endpoint list validation: reject unknown model, accept known
model, handle null/missing models config
Also fix unnecessary backslash escape in MODEL_PATTERN regex.
* 🧹 fix: Remove space from MODEL_PATTERN, trim input, clean up nits
- Remove space character from MODEL_PATTERN regex — no real model ID
uses spaces; prevents spurious violation logs from whitespace artifacts
- Add model.trim() before validation to handle accidental whitespace
- Remove redundant filterUniquePlugins call on already-deduplicated output
- Add comment documenting intentional whitelist+blacklist composition
- Add getUserKeyValues.mockReset() in loadConfigModels.spec beforeEach
- Remove narrating JSDoc from getModelsConfig one-liner
- Add 2 tests: trim whitespace handling, reject spaces in model ID
* fix: Match startup tool loader semantics — includedTools takes precedence over filteredTools
The startup tool loader (loadAndFormatTools) explicitly ignores
filteredTools when includedTools is set, with a warning log. The
PluginController was applying both independently, creating inconsistent
behavior where the same config produced different results at startup
vs plugin listing time.
Restored mutually exclusive semantics: when includedTools is non-empty,
filteredTools is not evaluated.
* 🧹 chore: Simplify validateModel flow, note auth requirement on endpoints route
- Separate missing-model from invalid-model checks cleanly: type+presence
guard first, then trim+format guard (reviewer NIT)
- Add route comment noting auth is required for role/tenant scoping
* fix: Write trimmed model back to req.body.model for downstream consumers
2026-03-30 16:49:48 -04:00
|
|
|
return { ...defaultModelsConfig, ...customModelsConfig };
|
2024-02-20 12:57:58 -05:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
async function modelController(req, res) {
|
2025-03-05 12:04:26 -05:00
|
|
|
try {
|
|
|
|
|
const modelConfig = await loadModels(req);
|
|
|
|
|
res.send(modelConfig);
|
|
|
|
|
} catch (error) {
|
|
|
|
|
logger.error('Error fetching models:', error);
|
|
|
|
|
res.status(500).send({ error: error.message });
|
|
|
|
|
}
|
2023-09-18 12:55:51 -04:00
|
|
|
}
|
|
|
|
|
|
2024-03-06 00:04:52 -05:00
|
|
|
module.exports = { modelController, loadModels, getModelsConfig };
|