Andreas/LibreChat
1
0
Fork 0
mirror of https://github.com/danny-avila/LibreChat.git synced 2025-12-19 18:00:15 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

💫 feat: Config File & Custom Endpoints (#1474)

Browse source 
* WIP(backend/api): custom endpoint

* WIP(frontend/client): custom endpoint

* chore: adjust typedefs for configs

* refactor: use data-provider for cache keys and rename enums and custom endpoint for better clarity and compatibility

* feat: loadYaml utility

* refactor: rename back to  from  and proof-of-concept for creating schemas from user-defined defaults

* refactor: remove custom endpoint from default endpointsConfig as it will be exclusively managed by yaml config

* refactor(EndpointController): rename variables for clarity

* feat: initial load custom config

* feat(server/utils): add simple `isUserProvided` helper

* chore(types): update TConfig type

* refactor: remove custom endpoint handling from model services as will be handled by config, modularize fetching of models

* feat: loadCustomConfig, loadConfigEndpoints, loadConfigModels

* chore: reorganize server init imports, invoke loadCustomConfig

* refactor(loadConfigEndpoints/Models): return each custom endpoint as standalone endpoint

* refactor(Endpoint/ModelController): spread config values after default (temporary)

* chore(client): fix type issues

* WIP: first pass for multiple custom endpoints
- add endpointType to Conversation schema
- add update zod schemas for both convo/presets to allow non-EModelEndpoint value as endpoint (also using type assertion)
- use `endpointType` value as `endpoint` where mapping to type is necessary using this field
- use custom defined `endpoint` value and not type for mapping to modelsConfig
- misc: add return type to `getDefaultEndpoint`
- in `useNewConvo`, add the endpointType if it wasn't already added to conversation
- EndpointsMenu: use user-defined endpoint name as Title in menu
- TODO: custom icon via custom config, change unknown to robot icon

* refactor(parseConvo): pass args as an object and change where used accordingly; chore: comment out 'create schema' code

* chore: remove unused availableModels field in TConfig type

* refactor(parseCompactConvo): pass args as an object and change where used accordingly

* feat: chat through custom endpoint

* chore(message/convoSchemas): avoid saving empty arrays

* fix(BaseClient/saveMessageToDatabase): save endpointType

* refactor(ChatRoute): show Spinner if endpointsQuery or modelsQuery are still loading, which is apparent with slow fetching of models/remote config on first serve

* fix(useConversation): assign endpointType if it's missing

* fix(SaveAsPreset): pass real endpoint and endpointType when saving Preset)

* chore: recorganize types order for TConfig, add `iconURL`

* feat: custom endpoint icon support:
- use UnknownIcon in all icon contexts
- add mistral and openrouter as known endpoints, and add their icons
- iconURL support

* fix(presetSchema): move endpointType to default schema definitions shared between convoSchema and defaults

* refactor(Settings/OpenAI): remove legacy `isOpenAI` flag

* fix(OpenAIClient): do not invoke abortCompletion on completion error

* feat: add responseSender/label support for custom endpoints:
- use defaultModelLabel field in endpointOption
- add model defaults for custom endpoints in `getResponseSender`
- add `useGetSender` hook which uses EndpointsQuery to determine `defaultModelLabel`
- include defaultModelLabel from endpointConfig in custom endpoint client options
- pass `endpointType` to `getResponseSender`

* feat(OpenAIClient): use custom options from config file

* refactor: rename `defaultModelLabel` to `modelDisplayLabel`

* refactor(data-provider): separate concerns from `schemas` into `parsers`, `config`, and fix imports elsewhere

* feat: `iconURL` and extract environment variables from custom endpoint config values

* feat: custom config validation via zod schema, rename and move to `./projectRoot/librechat.yaml`

* docs: custom config docs and examples

* fix(OpenAIClient/mistral): mistral does not allow singular system message, also add `useChatCompletion` flag to use openai-node for title completions

* fix(custom/initializeClient): extract env var and use `isUserProvided` function

* Update librechat.example.yaml

* feat(InputWithLabel): add className props, and forwardRef

* fix(streamResponse): handle error edge case where either messages or convos query throws an error

* fix(useSSE): handle errorHandler edge cases where error response is and is not properly formatted from API, especially when a conversationId is not yet provided, which ensures stream is properly closed on error

* feat: user_provided keys for custom endpoints

* fix(config/endpointSchema): do not allow default endpoint values in custom endpoint `name`

* feat(loadConfigModels): extract env variables and optimize fetching models

* feat: support custom endpoint iconURL for messages and Nav

* feat(OpenAIClient): add/dropParams support

* docs: update docs with default params, add/dropParams, and notes to use config file instead of `OPENAI_REVERSE_PROXY`

* docs: update docs with additional notes

* feat(maxTokensMap): add mistral models (32k context)

* docs: update openrouter notes

* Update ai_setup.md

* docs(custom_config): add table of contents and fix note about custom name

* docs(custom_config): reorder ToC

* Update custom_config.md

* Add note about `max_tokens` field in custom_config.md
This commit is contained in:
Danny Avila 2024-01-03 09:22:48 -05:00 • committed by GitHub
parent 3f98f92d4c
commit 29473a72db
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
100 changed files with 2146 additions and 627 deletions
Download patch file Download diff file Expand all files Collapse all files

89
client/src/components/Endpoints/Settings/OpenAI.tsx
View file

@ -1,6 +1,4 @@
import TextareaAutosize from 'react-textarea-autosize';
import type { TModelSelectProps } from '~/common';
import { ESide } from '~/common';
import {
SelectDropDown,
Input,
@ -10,9 +8,11 @@ import {
HoverCard,
HoverCardTrigger,
} from '~/components/ui';
import OptionHover from './OptionHover';
import { cn, defaultTextProps, optionText, removeFocusOutlines } from '~/utils/';
import type { TModelSelectProps } from '~/common';
import OptionHover from './OptionHover';
import { useLocalize } from '~/hooks';
import { ESide } from '~/common';
export default function Settings({ conversation, setOption, models, readonly }: TModelSelectProps) {
const localize = useLocalize();
@ -28,9 +28,6 @@ export default function Settings({ conversation, setOption, models, readonly }:
frequency_penalty: freqP,
presence_penalty: presP,
} = conversation;
const endpoint = conversation.endpoint || 'openAI';
const isOpenAI = endpoint === 'openAI' || endpoint === 'azureOpenAI';
const setModel = setOption('model');
const setChatGptLabel = setOption('chatGptLabel');
const setPromptPrefix = setOption('promptPrefix');
@ -52,47 +49,43 @@ export default function Settings({ conversation, setOption, models, readonly }:
containerClassName="flex w-full resize-none"
/>
</div>
{isOpenAI && (
<>
<div className="grid w-full items-center gap-2">
<Label htmlFor="chatGptLabel" className="text-left text-sm font-medium">
{localize('com_endpoint_custom_name')}{' '}
<small className="opacity-40">({localize('com_endpoint_default_blank')})</small>
</Label>
<Input
id="chatGptLabel"
disabled={readonly}
value={chatGptLabel || ''}
onChange={(e) => setChatGptLabel(e.target.value ?? null)}
placeholder={localize('com_endpoint_openai_custom_name_placeholder')}
className={cn(
defaultTextProps,
'dark:bg-gray-700 dark:hover:bg-gray-700/60 dark:focus:bg-gray-700',
'flex h-10 max-h-10 w-full resize-none px-3 py-2',
removeFocusOutlines,
)}
/>
</div>
<div className="grid w-full items-center gap-2">
<Label htmlFor="promptPrefix" className="text-left text-sm font-medium">
{localize('com_endpoint_prompt_prefix')}{' '}
<small className="opacity-40">({localize('com_endpoint_default_blank')})</small>
</Label>
<TextareaAutosize
id="promptPrefix"
disabled={readonly}
value={promptPrefix || ''}
onChange={(e) => setPromptPrefix(e.target.value ?? null)}
placeholder={localize('com_endpoint_openai_prompt_prefix_placeholder')}
className={cn(
defaultTextProps,
'dark:bg-gray-700 dark:hover:bg-gray-700/60 dark:focus:bg-gray-700',
'flex max-h-[138px] min-h-[100px] w-full resize-none px-3 py-2 ',
)}
/>
</div>
</>
)}
<div className="grid w-full items-center gap-2">
<Label htmlFor="chatGptLabel" className="text-left text-sm font-medium">
{localize('com_endpoint_custom_name')}{' '}
<small className="opacity-40">({localize('com_endpoint_default_blank')})</small>
</Label>
<Input
id="chatGptLabel"
disabled={readonly}
value={chatGptLabel || ''}
onChange={(e) => setChatGptLabel(e.target.value ?? null)}
placeholder={localize('com_endpoint_openai_custom_name_placeholder')}
className={cn(
defaultTextProps,
'dark:bg-gray-700 dark:hover:bg-gray-700/60 dark:focus:bg-gray-700',
'flex h-10 max-h-10 w-full resize-none px-3 py-2',
removeFocusOutlines,
)}
/>
</div>
<div className="grid w-full items-center gap-2">
<Label htmlFor="promptPrefix" className="text-left text-sm font-medium">
{localize('com_endpoint_prompt_prefix')}{' '}
<small className="opacity-40">({localize('com_endpoint_default_blank')})</small>
</Label>
<TextareaAutosize
id="promptPrefix"
disabled={readonly}
value={promptPrefix || ''}
onChange={(e) => setPromptPrefix(e.target.value ?? null)}
placeholder={localize('com_endpoint_openai_prompt_prefix_placeholder')}
className={cn(
defaultTextProps,
'dark:bg-gray-700 dark:hover:bg-gray-700/60 dark:focus:bg-gray-700',
'flex max-h-[138px] min-h-[100px] w-full resize-none px-3 py-2 ',
)}
/>
</div>
</div>
<div className="col-span-5 flex flex-col items-center justify-start gap-6 px-3 sm:col-span-2">
<HoverCard openDelay={300}>
@ -101,7 +94,7 @@ export default function Settings({ conversation, setOption, models, readonly }:
<Label htmlFor="temp-int" className="text-left text-sm font-medium">
{localize('com_endpoint_temperature')}{' '}
<small className="opacity-40">
({localize('com_endpoint_default_with_num', isOpenAI ? '1' : '0')})
({localize('com_endpoint_default_with_num', '1')})
</small>
</Label>
<InputNumber