💫 feat: Config File & Custom Endpoints (#1474)

* WIP(backend/api): custom endpoint

* WIP(frontend/client): custom endpoint

* chore: adjust typedefs for configs

* refactor: use data-provider for cache keys and rename enums and custom endpoint for better clarity and compatibility

* feat: loadYaml utility

* refactor: rename back to  from  and proof-of-concept for creating schemas from user-defined defaults

* refactor: remove custom endpoint from default endpointsConfig as it will be exclusively managed by yaml config

* refactor(EndpointController): rename variables for clarity

* feat: initial load custom config

* feat(server/utils): add simple `isUserProvided` helper

* chore(types): update TConfig type

* refactor: remove custom endpoint handling from model services as will be handled by config, modularize fetching of models

* feat: loadCustomConfig, loadConfigEndpoints, loadConfigModels

* chore: reorganize server init imports, invoke loadCustomConfig

* refactor(loadConfigEndpoints/Models): return each custom endpoint as standalone endpoint

* refactor(Endpoint/ModelController): spread config values after default (temporary)

* chore(client): fix type issues

* WIP: first pass for multiple custom endpoints
- add endpointType to Conversation schema
- add update zod schemas for both convo/presets to allow non-EModelEndpoint value as endpoint (also using type assertion)
- use `endpointType` value as `endpoint` where mapping to type is necessary using this field
- use custom defined `endpoint` value and not type for mapping to modelsConfig
- misc: add return type to `getDefaultEndpoint`
- in `useNewConvo`, add the endpointType if it wasn't already added to conversation
- EndpointsMenu: use user-defined endpoint name as Title in menu
- TODO: custom icon via custom config, change unknown to robot icon

* refactor(parseConvo): pass args as an object and change where used accordingly; chore: comment out 'create schema' code

* chore: remove unused availableModels field in TConfig type

* refactor(parseCompactConvo): pass args as an object and change where used accordingly

* feat: chat through custom endpoint

* chore(message/convoSchemas): avoid saving empty arrays

* fix(BaseClient/saveMessageToDatabase): save endpointType

* refactor(ChatRoute): show Spinner if endpointsQuery or modelsQuery are still loading, which is apparent with slow fetching of models/remote config on first serve

* fix(useConversation): assign endpointType if it's missing

* fix(SaveAsPreset): pass real endpoint and endpointType when saving Preset)

* chore: recorganize types order for TConfig, add `iconURL`

* feat: custom endpoint icon support:
- use UnknownIcon in all icon contexts
- add mistral and openrouter as known endpoints, and add their icons
- iconURL support

* fix(presetSchema): move endpointType to default schema definitions shared between convoSchema and defaults

* refactor(Settings/OpenAI): remove legacy `isOpenAI` flag

* fix(OpenAIClient): do not invoke abortCompletion on completion error

* feat: add responseSender/label support for custom endpoints:
- use defaultModelLabel field in endpointOption
- add model defaults for custom endpoints in `getResponseSender`
- add `useGetSender` hook which uses EndpointsQuery to determine `defaultModelLabel`
- include defaultModelLabel from endpointConfig in custom endpoint client options
- pass `endpointType` to `getResponseSender`

* feat(OpenAIClient): use custom options from config file

* refactor: rename `defaultModelLabel` to `modelDisplayLabel`

* refactor(data-provider): separate concerns from `schemas` into `parsers`, `config`, and fix imports elsewhere

* feat: `iconURL` and extract environment variables from custom endpoint config values

* feat: custom config validation via zod schema, rename and move to `./projectRoot/librechat.yaml`

* docs: custom config docs and examples

* fix(OpenAIClient/mistral): mistral does not allow singular system message, also add `useChatCompletion` flag to use openai-node for title completions

* fix(custom/initializeClient): extract env var and use `isUserProvided` function

* Update librechat.example.yaml

* feat(InputWithLabel): add className props, and forwardRef

* fix(streamResponse): handle error edge case where either messages or convos query throws an error

* fix(useSSE): handle errorHandler edge cases where error response is and is not properly formatted from API, especially when a conversationId is not yet provided, which ensures stream is properly closed on error

* feat: user_provided keys for custom endpoints

* fix(config/endpointSchema): do not allow default endpoint values in custom endpoint `name`

* feat(loadConfigModels): extract env variables and optimize fetching models

* feat: support custom endpoint iconURL for messages and Nav

* feat(OpenAIClient): add/dropParams support

* docs: update docs with default params, add/dropParams, and notes to use config file instead of `OPENAI_REVERSE_PROXY`

* docs: update docs with additional notes

* feat(maxTokensMap): add mistral models (32k context)

* docs: update openrouter notes

* Update ai_setup.md

* docs(custom_config): add table of contents and fix note about custom name

* docs(custom_config): reorder ToC

* Update custom_config.md

* Add note about `max_tokens` field in custom_config.md

client/src/components/Chat/Input/ChatForm.tsx

@@ -30,6 +30,8 @@ export default function ChatForm({ index = 0 }) {
   };
 
   const { requiresKey } = useRequiresKey();
+  const { endpoint: _endpoint, endpointType } = conversation ?? { endpoint: null };
+  const endpoint = endpointType ?? _endpoint;
 
   return (
     <form
@@ -49,9 +51,9 @@ export default function ChatForm({ index = 0 }) {
               onChange={(e: ChangeEvent<HTMLTextAreaElement>) => setText(e.target.value)}
               setText={setText}
               submitMessage={submitMessage}
-              endpoint={conversation?.endpoint}
+              endpoint={endpoint}
             />
-            <AttachFile endpoint={conversation?.endpoint ?? ''} disabled={requiresKey} />
+            <AttachFile endpoint={endpoint ?? ''} disabled={requiresKey} />
             {isSubmitting && showStopButton ? (
               <StopButton stop={handleStopGenerating} setShowStopButton={setShowStopButton} />
             ) : (

client/src/components/Chat/Input/HeaderOptions.tsx

@@ -2,7 +2,8 @@ import { useRecoilState } from 'recoil';
 import { Settings2 } from 'lucide-react';
 import { Root, Anchor } from '@radix-ui/react-popover';
 import { useState, useEffect, useMemo } from 'react';
-import { tPresetSchema, EModelEndpoint } from 'librechat-data-provider';
+import { tPresetUpdateSchema, EModelEndpoint } from 'librechat-data-provider';
+import type { TPreset } from 'librechat-data-provider';
 import { EndpointSettings, SaveAsPresetDialog } from '~/components/Endpoints';
 import { ModelSelect } from '~/components/Input/ModelSelect';
 import { PluginStoreDialog } from '~/components';
@@ -106,7 +107,11 @@ export default function OptionsBar() {
             <SaveAsPresetDialog
               open={saveAsDialogShow}
               onOpenChange={setSaveAsDialogShow}
-              preset={tPresetSchema.parse({ ...conversation })}
+              preset={
+                tPresetUpdateSchema.parse({
+                  ...conversation,
+                }) as TPreset
+              }
             />
             <PluginStoreDialog
               isOpen={showPluginStoreDialog}

client/src/components/Chat/Input/OptionsBar.tsx

@@ -1,7 +1,8 @@
 import { useRecoilState } from 'recoil';
 import { Settings2 } from 'lucide-react';
 import { useState, useEffect, useMemo } from 'react';
-import { tPresetSchema, EModelEndpoint } from 'librechat-data-provider';
+import { tPresetUpdateSchema, EModelEndpoint } from 'librechat-data-provider';
+import type { TPreset } from 'librechat-data-provider';
 import { PluginStoreDialog } from '~/components';
 import {
   EndpointSettings,
@@ -24,14 +25,8 @@ export default function OptionsBar({ messagesTree }) {
     store.showPluginStoreDialog,
   );
 
-  const {
-    showPopover,
-    conversation,
-    latestMessage,
-    setShowPopover,
-    setShowBingToneSetting,
-    textareaHeight,
-  } = useChatContext();
+  const { showPopover, conversation, latestMessage, setShowPopover, setShowBingToneSetting } =
+    useChatContext();
   const { setOption } = useSetIndexOptions();
 
   const { endpoint, conversationId, jailbreak } = conversation ?? {};
@@ -81,14 +76,7 @@ export default function OptionsBar({ messagesTree }) {
     ? altSettings[endpoint]
     : () => setShowPopover((prev) => !prev);
   return (
-    <div
-      className="absolute left-0 right-0 mx-auto mb-2 last:mb-2 md:mx-4 md:last:mb-6 lg:mx-auto lg:max-w-2xl xl:max-w-3xl"
-      style={{
-        // TODO: option to hide footer and handle this
-        // bottom: `${80 + (textareaHeight - 56)}px`, // without footer
-        bottom: `${85 + (textareaHeight - 56)}px`, // with footer
-      }}
-    >
+    <div className="absolute left-0 right-0 mx-auto mb-2 last:mb-2 md:mx-4 md:last:mb-6 lg:mx-auto lg:max-w-2xl xl:max-w-3xl">
       <GenerationButtons
         endpoint={endpoint}
         showPopover={showPopover}
@@ -151,7 +139,7 @@ export default function OptionsBar({ messagesTree }) {
           visible={showPopover}
           saveAsPreset={saveAsPreset}
           closePopover={() => setShowPopover(false)}
-          PopoverButtons={<PopoverButtons endpoint={endpoint} />}
+          PopoverButtons={<PopoverButtons />}
         >
           <div className="px-4 py-4">
             <EndpointSettings
@@ -164,7 +152,11 @@ export default function OptionsBar({ messagesTree }) {
         <SaveAsPresetDialog
           open={saveAsDialogShow}
           onOpenChange={setSaveAsDialogShow}
-          preset={tPresetSchema.parse({ ...conversation })}
+          preset={
+            tPresetUpdateSchema.parse({
+              ...conversation,
+            }) as TPreset
+          }
         />
         <PluginStoreDialog isOpen={showPluginStoreDialog} setIsOpen={setShowPluginStoreDialog} />
       </span>

client/src/components/Chat/Input/PopoverButtons.tsx

@@ -27,7 +27,8 @@ export default function PopoverButtons({
     setShowAgentSettings,
   } = useChatContext();
 
-  const { model, endpoint } = conversation ?? {};
+  const { model, endpoint: _endpoint, endpointType } = conversation ?? {};
+  const endpoint = endpointType ?? _endpoint;
   const isGenerativeModel = model?.toLowerCase()?.includes('gemini');
   const isChatModel = !isGenerativeModel && model?.toLowerCase()?.includes('chat');
   const isTextModel = !isGenerativeModel && !isChatModel && /code|text/.test(model ?? '');