mirror of
https://github.com/danny-avila/LibreChat.git
synced 2025-12-23 11:50:14 +01:00
✨ feat: Assistants API, General File Support, Side Panel, File Explorer (#1696)
* feat: assistant name/icon in Landing & Header * feat: assistname in textarea placeholder, and use `Assistant` as default name * feat: display non-image files in user messages * fix: only render files if files.length is > 0 * refactor(config -> file-config): move file related configuration values to separate module, add excel types * chore: spreadsheet file rendering * fix(Landing): dark mode style for Assistant Name * refactor: move progress incrementing to own hook, start smaller, cap near limit \(1\) * refactor(useContentHandler): add empty Text part if last part was completed tool or image * chore: add accordion trigger border styling for dark mode * feat: Assistant Builder model selection * chore: use Spinner when Assistant is mutating * fix(get/assistants): return correct response object `AssistantListResponse` * refactor(Spinner): pass size as prop * refactor: make assistant crud mutations optimistic, add types for options * chore: remove assistants route and view * chore: move assistant builder components to separate directory * feat(ContextButton): delete Assistant via context button/dialog, add localization * refactor: conditionally show use and context menu buttons, add localization for create assistant * feat: save side panel states to localStorage * style(SidePanel): improve avatar menu and assistant select styling for dark mode * refactor: make NavToggle reusable for either side (left or right), add SidePanel Toggle with ability to close it completely * fix: resize handle and navToggle behavior * fix(/avatar/:assistant_id): await `deleteFile` and assign unique name to uploaded image * WIP: file UI components from PR #576 * refactor(OpenAIMinimalIcon): pass className * feat: formatDate helper fn * feat: DataTableColumnHeader * feat: add row selection, formatted row values, number of rows selected * WIP: add files to Side panel temporarily * feat: `LB_QueueAsyncCall`: Leaky Bucket queue for external APIs, use in `processDeleteRequest` * fix(TFile): correct `source` type with `FileSources` * fix(useFileHandling): use `continue` instead of return when iterating multiple files, add file type to extendedFile * chore: add generic setter type * refactor(processDeleteRequest): settle promises to prevent rejections from processing deletions, log errors * feat: `useFileDeletion` to reuse file deletion logic * refactor(useFileDeletion): make `setFiles` an optional param and use object as param * feat: useDeleteFilesFromTable * feat: use real `files` data and add deletion action to data table * fix(Table): make headers sticky * feat: add dynamic filtering for columns; only show to user Host or OpenAI storage type * style(DropdownMenu): replace `slate` with `gray` * style(DataTable): apply dark mode themes and other misc styling * style(Columns): add color to OpenAI Storage option * refactor(FileContainer): make file preview reusable * refactor(Images): make image preview reusable * refactor(FilePreview): make file prop optional for FileIcon and FilePreview, fix relative style * feat(Columns): add file/image previews, set a minimum size to show for file size in bytes * WIP: File Panel with real files and formatted * feat: open files dialog from panel * style: file data table mobile and general column styling fixes * refactor(api/files): return files sorted by the most recently updated * refactor: provide fileMap through context to prevent re-selecting files to map in different areas; remove unused imports commented out in PanelColumns * refactor(ExtendFile): make File type optional, add `attached` to prevent attached files from being deleted on remove, make Message.files a partial TFile type * feat: attach files through file panel * refactor(useFileHandling): move files to the start of cache list when uploaded * refactor(useDeleteFilesMutation): delete files from cache when successfully deleted from server * fix(FileRow): handle possible edge case of duplication due to attaching recently uploaded file * style(SidePanel): make resize grip border transparent, remove unnecessary styling on close sidepanel button * feat: action utilities and tests * refactor(actions): add `ValidationResult` type and change wording for no server URL found * refactor(actions): check for empty server URL * fix(data-provider): revert tsconfig to fix type issue resolution * feat(client): first pass of actions input for assistants * refactor(FunctionSignature): change method to output object instead of string * refactor(models/Assistant): add actions field to schema, use searchParams object for methods, and add `getAssistant` * feat: post actions input first pass - create new Action document - add actions to Assistant DB document - create /action/:assistant_id POST route - pass more props down from PanelSwitcher, derive assistant_id from switcher - move privacy policy to ActionInput - reset data on input change/validation - add `useUpdateAction` - conform FunctionSignature type to FunctionTool - add action, assistant doc, update hook related types * refactor: optimize assistant/actions relationship - past domain in metadata as hostname and not a URL - include domain in tool name - add `getActions` for actions retrieval by user - add `getAssistants` for assistant docs retrieval by user - add `assistant_id` to Action schema - move actions to own module as a subroute to `api/assistants` - add `useGetActionsQuery` and `useGetAssistantDocsQuery` hooks - fix Action type def * feat: show assistant actions in assistant builder * feat: switch to actions on action click, editing action styling * fix: add Assistant state for builder panel to allow immediate selection of newly created assistants as well as retaining the current assistant when switching to a different panel within the builder * refactor(SidePanel/NavToggle): offset less from right when SidePanel is completely collapsed * chore: rename `processActions` -> `processRequiredActions` * chore: rename Assistant API Action to RequiredAction * refactor(actions): avoid nesting actual API params under generic `requestBody` to optimize LLM token usage * fix(handleTools): avoid calling `validTool` if not defined, add optional param to skip the loading of specs, which throws an error in the context of assistants * WIP: working first pass of toolCalls generated from openapi specs * WIP: first pass ToolCall styling * feat: programmatic iv encryption/decryption helpers * fix: correct ActionAuth types/enums, and define type for AuthForm * feat: encryption/decryption helpers for Action AuthMetadata * refactor(getActions): remove sensitive fields from query response * refactor(POST/actions): encrypt and remove sensitive fields from mutation response * fix(ActionService): change ESM import to CJS * feat: frontend auth handling for actions + optimistic update on action update/creation * refactor(actions): use the correct variables and types for setAuth method * refactor: POST /:assistant_id action can now handle updating an existing action, add `saved_auth_fields` to determine when user explicitly saves new auth creds. only send auth metadata if user explicitly saved fields * refactor(createActionTool): catch errors and send back meaningful error message, add flag to `getActions` to determine whether to retrieve sensitive values or not * refactor(ToolService): add `action` property to ToolCall PartMetadata to determine if the tool call was an action, fix parsing function name issue with actionDelimiter * fix(ActionRequest): use URL class to correctly join endpoint parts for `execute` call * feat: delete assistant actions * refactor: conditionally show Available actions * refactor: show `retrieval` and `code_interpreter` as Capabilities, swap `Switch` for `Checkbox` * chore: remove shadow-stroke from messages * WIP: first pass of Assistants Knowledge attachments * refactor: remove AssistantsProvider in favor of FormProvider, fix selectedAssistant re-render bug, map Assistant file_ids to files via fileMap, initialize Knowledge component with mapped files if any exist * fix: prevent deleting files on assistant file upload * chore: remove console.log * refactor(useUploadFileMutation): update files and assistants cache on upload * chore: disable oauth option as not supported yet * feat: cancel assistant runs * refactor: initialize OpenAI client with helper function, resolve all related circular dependencies * fix(DALL-E): initialization * fix(process): openai client initialization * fix: select an existing Assistant when the active one is deleted * chore: allow attaching files for assistant endpoint, send back relevant OpenAI error message when uploading, deconstruct openAI initialization correctly, add `message_file` to formData when a file is attached to the message but not the assistant * fix: add assistant_id on newConvo * fix(initializeClient): import fix * chore: swap setAssistant for setOption in useEffect * fix(DALL-E): add processFileURL to loadTools call * chore: add customConfig to debug logs * feat: delete threads on convo delete * chore: replace Assistants icon * chore: remove console.dir() in `abortRun` * feat(AssistantService): accumulate text values from run in openai.responseText * feat: titling for assistants endpoint * chore: move panel file components to appropriate directory, add file checks for attaching files, change icon for Attach Files * refactor: add localizations to tools, plugins, add condition for adding/remove user plugins so tool selections don't affect this value * chore: disable `import from url` action for now * chore: remove textMimeTypes from default fileConfig for now * fix: catch tool errors and send as outputs with error messages * fix: React warning about button as descendant of button * style: retrieval and cancelled icon * WIP: pass isSubmitting to Parts, use InProgressCall to display cancelled tool calls correctly, show domain/function name * fix(meilisearch): fix `postSaveHook` issue where indexing expects a mongo document, and join all text content parts for meili indexing * ci: fix dall-e tests * ci: fix client tests * fix: button types in actions panel * fix: plugin auth form persisting across tool selections * fix(ci): update AppService spec with `loadAndFormatTools` * fix(clearConvos): add id check earlier on * refactor(AssistantAvatar): set previewURL dynamically when emtadata.avatar changes * feat(assistants): addTitle cache setting * fix(useSSE): resolve rebase conflicts * fix: delete mutation * style(SidePanel): make grip visible on active and hover, invisible otherwise * ci: add data-provider tests to workflow, also update eslint/tsconfig to recognize specs, and add `text/csv` to fileConfig * fix: handle edge case where auth object is undefined, and log errors * refactor(actions): resolve schemas, add tests for resolving refs, import specs from separate file for tests * chore: remove comment * fix(ActionsInput): re-render bug when initializing states with action fields * fix(patch/assistant): filter undefined tools * chore: add logging for errors in assistants routes * fix(updateAssistant): map actions to functions to avoid overwriting * fix(actions): properly handle GET paths * fix(convos): unhandled delete thread exception * refactor(AssistantService): pass both thread_id and conversationId when sending intermediate assistant messages, remove `mapMessagesToSteps` from AssistantService * refactor(useSSE): replace all messages with runMessages and pass latestMessageId to abortRun; fix(checkMessageGaps): include tool calls when syncing messages * refactor(assistants/chat): invoke `createOnTextProgress` after thread creation * chore: add typing * style: sidepanel styling * style: action tool call domain styling * feat(assistants): default models, limit retrieval to certain models, add env variables to to env.example * feat: assistants api key in EndpointService * refactor: set assistant model to conversation on assistant switch * refactor: set assistant model to conversation on assistant select from panel * fix(retrieveAndProcessFile): catch attempt to download file with `assistant` purpose which is not allowed; add logging * feat: retrieval styling, handling, and logging * chore: rename ASSISTANTS_REVERSE_PROXY to ASSISTANTS_BASE_URL * feat: FileContext for file metadata * feat: context file mgmt and filtering * style(Select): hover/rounded changes * refactor: explicit conversation switch, endpoint dependent, through `useSelectAssistant`, which does not create new chat if current endpoint is assistant endpoint * fix(AssistantAvatar): make empty previewURL if no avatar present * refactor: side panel mobile styling * style: merge tool and action section, optimize mobile styling for action/tool buttons * fix: localStorage issues * fix(useSelectAssistant): invoke react query hook directly in select hook as Map was not being updated in time * style: light mode fixes * fix: prevent sidepanel nav styling from shifting layout up * refactor: change default layout (collapsed by default) * style: mobile optimization of DataTable * style: datatable * feat: client-side hide right-side panel * chore(useNewConvo): add partial typing for preset * fix(useSelectAssistant): pass correct model name by using template as preset * WIP: assistant presets * refactor(ToolService): add native solution for `TavilySearchResults` and log tool output errors * refactor: organize imports and use native TavilySearchResults * fix(TavilySearchResults): stringify result * fix(ToolCall): show tool call outputs when not an action * chore: rename Prompt Prefix to custom instructions (in user facing text only) * refactor(EditPresetDialog): Optimize setting title by debouncing, reset preset on dialog close to avoid state mixture * feat: add `presetOverride` to overwrite active conversation settings when saving a Preset (relevant for client side updates only) * feat: Assistant preset settings (client-side) * fix(Switcher): only set assistant_id and model if current endpoint is Assistants * feat: use `useDebouncedInput` for updating conversation settings, starting with EditPresetDialog title setting and Assistant instructions setting * feat(Assistants): add instructions field to settings * feat(chat/assistants): pass conversation settings to run body * wip: begin localization and only allow actions if the assistant is created * refactor(AssistantsPanel): knowledge localization, allow tools on creation * feat: experimental: allow 'priming' values before assistant is created, that would normally require an assistant_id to be defined * chore: trim console logs and make more meaningful * chore: toast messages * fix(ci): date test * feat: create file when uploading Assistant Avatar * feat: file upload rate limiting from custom config with dynamic file route initialization * refactor: use file upload limiters on post routes only * refactor(fileConfig): add endpoints field for endpoint specific fileconfigs, add mergeConfig function, add tests * refactor: fileConfig route, dynamic multer instances used on all '/' and '/images' POST routes, data service and query hook * feat: supportedMimeTypesSchema, test for array of regex * feat: configurable file config limits * chore: clarify assistants file knowledge prereq. * chore(useTextarea): default to localized 'Assistant' if assistant name is empty * feat: configurable file limits and toggle file upload per endpoint * fix(useUploadFileMutation): prevent updating assistant.files cache if file upload is a message_file attachment * fix(AssistantSelect): set last selected assistant only when timeout successfully runs * refactor(queries): disable assistant queries if assistants endpoint is not enabled * chore(Switcher): add localization * chore: pluralize `assistant` for `EModelEndpoint key and value * feat: show/hide assistant UI components based on endpoint availability; librechat.yaml config for disabling builder section and setting polling/timeout intervals * fix(compactEndpointSchemas): use EModelEndpoint for schema access * feat(runAssistant): use configured values from `librechat.yaml` for `pollIntervalMs` and `timeout` * fix: naming issue * wip: revert landing * 🎉 happy birthday LibreChat (#1768) * happy birthday LibreChat * Refactor endpoint condition in Landing component * Update birthday message in Eng.tsx * fix(/config): avoid nesting ternaries * refactor(/config): check birthday --------- Co-authored-by: Danny Avila <messagedaniel@protonmail.com> * fix: landing * fix: landing * fix(useMessageHelpers): hardcoded check to use EModelEndpoint instead * fix(ci): convo test revert to main * fix(assistants/chat): fix issue where assistant_id was being saved as model for convo * chore: added logging, promises racing to prevent longer timeouts, explicit setting of maxRetries and timeouts, robust catching of invalid abortRun params * refactor: use recoil state for `showStopButton` and only show for assistants endpoint after syncing conversation data * refactor: optimize abortRun strategy using localStorage, refactor `abortConversation` to use async/await and await the result, refactor how the abortKey cache is set for runs * fix(checkMessageGaps): assign `assistant_id` to synced messages if defined; prevents UI from showing blank assistant for cancelled messages * refactor: re-order sequence of chat route, only allow aborting messages after run is created, cancel abortRun if there was a cancelling error (likely due already cancelled in chat route), and add extra logging * chore(typedefs): add httpAgent type to OpenAIClient * refactor: use custom implementation of retrieving run with axios to allow for timing out run query * fix(waitForRun): handle timed out run retrieval query * refactor: update preset conditions: - presets will retain settings when a different endpoint is selected; for existing convos, either when modular or is assistant switch - no longer use `navigateToConvo` on preset select * fix: temporary calculator hack as expects string input when invoked * fix: cancel abortRun only when cancelling error is a result of the run already being cancelled * chore: remove use of `fileMaxSizeMB` and total counterpart (redundant) * docs: custom config documentation update * docs: assistants api setup and dotenv, new custom config fields * refactor(Switcher): make Assistant switcher sticky in SidePanel * chore(useSSE): remove console log of data and message index * refactor(AssistantPanel): button styling and add secondary select button to bottom of panel * refactor(OpenAIClient): allow passing conversationId to RunManager through titleConvo and initializeLLM to properly record title context tokens used in cases where conversationId was not defined by the client * feat(assistants): token tracking for assistant runs * chore(spendTokens): improve logging * feat: support/exclude specific assistant Ids * chore: add update `librechat.example.yaml`, optimize `AppService` handling, new tests for `AppService`, optimize missing/outdate config logging * chore: mount docker logs to root of project * chore: condense axios errors * chore: bump vite * chore: vite hot reload fix using latest version * chore(getOpenAIModels): sort instruct models to the end of models list * fix(assistants): user provided key * fix(assistants): user provided key, invalidate more queries on revoke --------- Co-authored-by: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
This commit is contained in:
Danny Avila
GitHub
parent
cd2786441a
commit
ecd63eb9f1
316 changed files with 21873 additions and 6315 deletions
|
|
@ -91,12 +91,56 @@ To get your OpenAI API key, you need to:
|
|||
- Add a payment method to your account (this is not free, sorry 😬)
|
||||
- Copy your secret key (sk-...) and save it in ./.env as OPENAI_API_KEY
|
||||
|
||||
Notes:
|
||||
**Notes:**
|
||||
|
||||
- Selecting a vision model for messages with attachments is not necessary as it will be switched behind the scenes for you. If you didn't outright select a vision model, it will only be used for the vision request and you should still see the non-vision model you had selected after the request is successful
|
||||
- OpenAI Vision models allow for messages without attachments
|
||||
|
||||
---
|
||||
|
||||
## Assistants
|
||||
|
||||
- The [Assistants API by OpenAI](https://platform.openai.com/docs/assistants/overview) has a dedicated endpoint.
|
||||
- The Assistants API enables the creation of AI assistants, offering functionalities like code interpreter, knowledge retrieval of files, and function execution.
|
||||
- [Read here for an in-depth documentation](https://platform.openai.com/docs/assistants/overview) of the feature, how it works, what it's capable of.
|
||||
- As with the regular [OpenAI API](#openai), go to **[https://platform.openai.com/account/api-keys](https://platform.openai.com/account/api-keys)** to get a key.
|
||||
- You will need to set the following environment variable to your key or you can set it to `user_provided` for users to provide their own.
|
||||
|
||||
```bash
|
||||
ASSISTANTS_API_KEY=your-key
|
||||
```
|
||||
|
||||
- You can determine which models you would like to have available with `ASSISTANTS_MODELS`; otherwise, the models list fetched from OpenAI will be used (only Assistants API compatible models will be shown).
|
||||
|
||||
```bash
|
||||
# without spaces
|
||||
ASSISTANTS_MODELS=gpt-3.5-turbo-0125,gpt-3.5-turbo-16k-0613,gpt-3.5-turbo-16k,gpt-3.5-turbo,gpt-4,gpt-4-0314,gpt-4-32k-0314,gpt-4-0613,gpt-3.5-turbo-0613,gpt-3.5-turbo-1106,gpt-4-0125-preview,gpt-4-turbo-preview,gpt-4-1106-preview
|
||||
```
|
||||
|
||||
- If necessary, you can also set an alternate base URL instead of the official one with `ASSISTANTS_BASE_URL`, which is similar to the OpenAI counterpart `OPENAI_REVERSE_PROXY`
|
||||
|
||||
```bash
|
||||
ASSISTANTS_BASE_URL=http://your-alt-baseURL:3080/
|
||||
```
|
||||
|
||||
- There is additional, optional configuration, depending on your needs, such as disabling the assistant builder UI, that are available via the [`librechat.yaml` custom config file](./custom_config.md#assistants-endpoint-object-structure):
|
||||
- Control the visibility and use of the builder interface for assistants. [More info](./custom_config.md#disablebuilder)
|
||||
- Specify the polling interval in milliseconds for checking run updates or changes in assistant run states. [More info](./custom_config.md#pollintervalms)
|
||||
- Set the timeout period in milliseconds for assistant runs. Helps manage system load by limiting total run operation time. [More info](./custom_config.md#timeoutMs)
|
||||
- Specify which assistant Ids are supported or excluded [More info](./custom_config.md#supportedIds)
|
||||
|
||||
**Notes:**
|
||||
|
||||
- At the time of writing, only the following models support the [Retrieval](https://platform.openai.com/docs/assistants/tools/knowledge-retrieval) capability:
|
||||
- gpt-3.5-turbo-0125
|
||||
- gpt-4-0125-preview
|
||||
- gpt-4-turbo-preview
|
||||
- gpt-4-1106-preview
|
||||
- gpt-3.5-turbo-1106
|
||||
- Vision capability is not yet supported.
|
||||
|
||||
---
|
||||
|
||||
## Anthropic
|
||||
|
||||
- Create an account at **[https://console.anthropic.com/](https://console.anthropic.com/)**
|
||||
|
|
@ -334,6 +378,7 @@ Alternatively, you can set the [required variables](#required-variables) to expl
|
|||
|
||||
|
||||
**Notes:**
|
||||
|
||||
- If using `AZURE_OPENAI_BASEURL`, you should not specify instance and deployment names instead of placeholders as the vision request will fail.
|
||||
- As of December 18th, 2023, Vision models seem to have degraded performance with Azure OpenAI when compared to [OpenAI](#openai)
|
||||
|
||||
|
|
@ -403,6 +448,7 @@ To use Azure with the Plugins endpoint, make sure the following environment vari
|
|||
* `AZURE_API_KEY`: Your Azure API key must be set with an environment variable.
|
||||
|
||||
**Important:**
|
||||
|
||||
- If using `AZURE_OPENAI_BASEURL`, you should not specify instance and deployment names instead of placeholders as the plugin request will fail.
|
||||
|
||||
---
|
||||
|
|
|
|||
|
|
@ -20,41 +20,7 @@ Stay tuned for ongoing enhancements to customize your LibreChat instance!
|
|||
|
||||
> Note: To verify your YAML config, you can use online tools like [yamlchecker.com](https://yamlchecker.com/)
|
||||
|
||||
<!-- # Table of Contents
|
||||
|
||||
- [LibreChat Configuration Guide](#librechat-configuration-guide)
|
||||
- [Table of Contents](#table-of-contents)
|
||||
- [Setup](#setup)
|
||||
- [Docker Setup](#docker-setup)
|
||||
- [Config Structure](#config-structure)
|
||||
- [Version](#version)
|
||||
- [Cache Settings](#cache-settings)
|
||||
- [File Strategy](#file-strategy)
|
||||
- [Registration](#registration)
|
||||
- [Endpoints](#endpoints)
|
||||
- [Registration Object Structure](#registration-object-structure)
|
||||
- [**socialLogins**:](#socialLogins)
|
||||
- [**allowedDomains**:](#allowedDomains)
|
||||
- [Custom Endpoint Object Structure](#custom-endpoint-object-structure)
|
||||
- [**name**:](#name)
|
||||
- [**apiKey**:](#apikey)
|
||||
- [**baseURL**:](#baseurl)
|
||||
- [**iconURL**:](#iconurl)
|
||||
- [**models**:](#models)
|
||||
- [**titleConvo**:](#titleconvo)
|
||||
- [**titleMethod**:](#titlemethod)
|
||||
- [**titleModel**:](#titlemodel)
|
||||
- [**summarize**:](#summarize)
|
||||
- [**summaryModel**:](#summarymodel)
|
||||
- [**forcePrompt**:](#forceprompt)
|
||||
- [**modelDisplayLabel**:](#modeldisplaylabel)
|
||||
- [**addParams**:](#addparams)
|
||||
- [**dropParams**:](#dropparams)
|
||||
- [**headers**:](#headers)
|
||||
- [Additional Notes](#additional-notes)
|
||||
- [Default Parameters](#default-parameters)
|
||||
- [Breakdown of Default Params](#breakdown-of-default-params)
|
||||
- [Example Config](#example-config) -->
|
||||
**Note:** To verify your YAML config, you can use online tools like [yamlchecker.com](https://yamlchecker.com/)
|
||||
|
||||
## Setup
|
||||
|
||||
|
|
@ -88,6 +54,105 @@ services:
|
|||
docker compose up # no need to rebuild
|
||||
```
|
||||
|
||||
## Example Config
|
||||
|
||||
```yaml
|
||||
version: 1.0.3
|
||||
cache: true
|
||||
# fileStrategy: "firebase" # If using Firebase CDN
|
||||
fileConfig:
|
||||
endpoints:
|
||||
assistants:
|
||||
fileLimit: 5
|
||||
fileSizeLimit: 10 # Maximum size for an individual file in MB
|
||||
totalSizeLimit: 50 # Maximum total size for all files in a single request in MB
|
||||
supportedMimeTypes:
|
||||
- "image/.*"
|
||||
- "application/pdf"
|
||||
openAI:
|
||||
disabled: true # Disables file uploading to the OpenAI endpoint
|
||||
default:
|
||||
totalSizeLimit: 20
|
||||
YourCustomEndpointName:
|
||||
fileLimit: 2
|
||||
fileSizeLimit: 5
|
||||
serverFileSizeLimit: 100 # Global server file size limit in MB
|
||||
avatarSizeLimit: 2 # Limit for user avatar image size in MB
|
||||
rateLimits:
|
||||
fileUploads:
|
||||
ipMax: 100
|
||||
ipWindowInMinutes: 60 # Rate limit window for file uploads per IP
|
||||
userMax: 50
|
||||
userWindowInMinutes: 60 # Rate limit window for file uploads per user
|
||||
registration:
|
||||
socialLogins: ["google", "facebook", "github", "discord", "openid"]
|
||||
allowedDomains:
|
||||
- "example.com"
|
||||
- "anotherdomain.com"
|
||||
endpoints:
|
||||
assistants:
|
||||
disableBuilder: false # Disable Assistants Builder Interface by setting to `true`
|
||||
pollIntervalMs: 750 # Polling interval for checking assistant updates
|
||||
timeoutMs: 180000 # Timeout for assistant operations
|
||||
# Should only be one or the other, either `supportedIds` or `excludedIds`
|
||||
supportedIds: ["asst_supportedAssistantId1", "asst_supportedAssistantId2"]
|
||||
# excludedIds: ["asst_excludedAssistantId"]
|
||||
custom:
|
||||
- name: "Mistral"
|
||||
apiKey: "${MISTRAL_API_KEY}"
|
||||
baseURL: "https://api.mistral.ai/v1"
|
||||
models:
|
||||
default: ["mistral-tiny", "mistral-small", "mistral-medium"]
|
||||
fetch: true # Attempt to dynamically fetch available models
|
||||
userIdQuery: false
|
||||
iconURL: "https://example.com/mistral-icon.png"
|
||||
titleConvo: true
|
||||
titleMethod: "completion"
|
||||
titleModel: "mistral-tiny"
|
||||
summarize: true
|
||||
summaryModel: "mistral-summary"
|
||||
forcePrompt: false
|
||||
modelDisplayLabel: "Mistral AI"
|
||||
addParams:
|
||||
safe_prompt: true
|
||||
dropParams:
|
||||
- "stop"
|
||||
- "user"
|
||||
- "presence_penalty"
|
||||
- "frequency_penalty"
|
||||
# headers:
|
||||
# x-custom-header: "${CUSTOM_HEADER_VALUE}"
|
||||
- name: "OpenRouter"
|
||||
apiKey: "${OPENROUTER_API_KEY}"
|
||||
baseURL: "https://openrouter.ai/api/v1"
|
||||
models:
|
||||
default: ["gpt-3.5-turbo"]
|
||||
fetch: false
|
||||
titleConvo: true
|
||||
titleModel: "gpt-3.5-turbo"
|
||||
summarize: false
|
||||
forcePrompt: false
|
||||
modelDisplayLabel: "OpenRouter"
|
||||
dropParams:
|
||||
- "frequency_penalty"
|
||||
```
|
||||
|
||||
This example configuration file sets up LibreChat with detailed options across several key areas:
|
||||
|
||||
- **Caching**: Enabled to improve performance.
|
||||
- **File Handling**:
|
||||
- **File Strategy**: Commented out but hints at possible integration with Firebase for file storage.
|
||||
- **File Configurations**: Customizes file upload limits and allowed MIME types for different endpoints, including a global server file size limit and a specific limit for user avatar images.
|
||||
- **Rate Limiting**: Defines thresholds for the maximum number of file uploads allowed per IP and user within a specified time window, aiming to prevent abuse.
|
||||
- **Registration**:
|
||||
- Allows registration from specified social login providers and email domains, enhancing security and user management.
|
||||
- **Endpoints**:
|
||||
- **Assistants**: Configures the assistants' endpoint with a polling interval and a timeout for operations, and provides an option to disable the builder interface.
|
||||
- **Custom Endpoints**:
|
||||
- Configures two external AI service endpoints, Mistral and OpenRouter, including API keys, base URLs, model handling, and specific feature toggles like conversation titles, summarization, and parameter adjustments.
|
||||
- For Mistral, it enables dynamic model fetching, applies additional parameters for safe prompts, and explicitly drops unsupported parameters.
|
||||
- For OpenRouter, it sets up a basic configuration without dynamic model fetching and specifies a model for conversation titles.
|
||||
|
||||
## Config Structure
|
||||
|
||||
**Note:** Fields not specifically mentioned as required are optional.
|
||||
|
|
@ -111,28 +176,236 @@ docker compose up # no need to rebuild
|
|||
- **Description**: Determines where to save user uploaded/generated files. Defaults to `"local"` if omitted.
|
||||
- **Example**: `fileStrategy: "firebase"`
|
||||
|
||||
### File Configuration
|
||||
- **Key**: `fileConfig`
|
||||
- **Type**: Object
|
||||
- **Description**: Configures file handling settings for the application, including size limits and MIME type restrictions.
|
||||
|
||||
- **Sub-Key**: `endpoints`
|
||||
- **Type**: Record/Object
|
||||
- **Description**: Specifies file handling configurations for individual endpoints, allowing customization per endpoint basis.
|
||||
- [Endpoint File Config Object Structure](#endpoint-file-config-object-structure)
|
||||
|
||||
- **Sub-Key**: `serverFileSizeLimit`
|
||||
- **Type**: Number
|
||||
- **Description**: The maximum file size (in MB) that the server will accept. Applies globally across all endpoints unless overridden by endpoint-specific settings.
|
||||
|
||||
- **Sub-Key**: `avatarSizeLimit`
|
||||
- **Type**: Number
|
||||
- **Description**: Maximum size (in MB) for user avatar images.
|
||||
|
||||
### Rate Limiting
|
||||
|
||||
- **Key**: `rateLimits`
|
||||
- **Type**: Object
|
||||
- **Description**: Defines rate limiting policies to prevent abuse by limiting the number of requests.
|
||||
- **Sub-Key**: `fileUploads`
|
||||
- **Type**: Object
|
||||
- **Description**: Configures rate limits specifically for file upload operations.
|
||||
- **Sub-Key**: `ipMax`
|
||||
- **Type**: Number
|
||||
- **Description**: Maximum number of uploads allowed per IP address per window.
|
||||
- **Sub-Key**: `ipWindowInMinutes`
|
||||
- **Type**: Number
|
||||
- **Description**: Time window in minutes for the IP-based upload limit.
|
||||
- **Sub-Key**: `userMax`
|
||||
- **Type**: Number
|
||||
- **Description**: Maximum number of uploads allowed per user per window.
|
||||
- **Sub-Key**: `userWindowInMinutes`
|
||||
- **Type**: Number
|
||||
- **Description**: Time window in minutes for the user-based upload limit.
|
||||
|
||||
- **Example**:
|
||||
```yaml
|
||||
rateLimits:
|
||||
fileUploads:
|
||||
ipMax: 100
|
||||
ipWindowInMinutes: 60
|
||||
userMax: 50
|
||||
userWindowInMinutes: 60
|
||||
```
|
||||
|
||||
### Registration
|
||||
- **Key**: `registration`
|
||||
- **Type**: Object
|
||||
- **Description**: Configures registration-related settings for the application.
|
||||
- **Sub-Key**: `socialLogins`
|
||||
- **Type**: Array of Strings (`"google"`, `"facebook"`, `"openid"`, `"github"`, `"discord"`)
|
||||
- **Description**: Determines both the available social login providers and their arranged order to list on the login/registration page, from top to bottom (first to last values). Note: a login option will not appear even if listed if not [properly configured.](./user_auth_system.md#social-authentication-setup-and-configuration)
|
||||
- [More info](#socialLogins)
|
||||
- **Sub-Key**: `allowedDomains`
|
||||
- **Type**: Array of Strings
|
||||
- **Description**: Specifies a list of allowed email domains for user registration. Users attempting to register with email domains not listed here will be restricted from registering.
|
||||
- [More info](#allowedDomains)
|
||||
- [Registration Object Structure](#registration-object-structure)
|
||||
|
||||
### Endpoints
|
||||
- **Key**: `endpoints`
|
||||
- **Type**: Object
|
||||
- **Description**: Defines custom API endpoints for the application.
|
||||
- **Sub-Key**: `assistants`
|
||||
- **Type**: Object
|
||||
- **Description**: Assistants endpoint-specific configuration.
|
||||
- **Sub-Key**: `disableBuilder`
|
||||
- **Description**: Controls the visibility and use of the builder interface for assistants.
|
||||
- [More info](#disablebuilder)
|
||||
- **Sub-Key**: `pollIntervalMs`
|
||||
- **Description**: Specifies the polling interval in milliseconds for checking run updates or changes in assistant run states.
|
||||
- [More info](#pollintervalms)
|
||||
- **Sub-Key**: `timeoutMs`
|
||||
- **Description**: Sets a timeout in milliseconds for assistant runs. Helps manage system load by limiting total run operation time.
|
||||
- [More info](#timeoutMs)
|
||||
- **Sub-Key**: `supportedIds`
|
||||
- **Description**: List of supported assistant Ids. Use this or `excludedIds` but not both.
|
||||
- [More info](#supportedIds)
|
||||
- **Sub-Key**: `excludedIds`
|
||||
- **Description**: List of excluded assistant Ids. Use this or `supportedIds` but not both (the `excludedIds` field will be ignored if so).
|
||||
- [More info](#excludedIds)
|
||||
- [Full Assistants Endpoint Object Structure](#assistants-endpoint-object-structure)
|
||||
- **Sub-Key**: `custom`
|
||||
- **Type**: Array of Objects
|
||||
- **Description**: Each object in the array represents a unique endpoint configuration.
|
||||
- [Custom Endpoint Object Structure](#custom-endpoint-object-structure)
|
||||
- [Full Custom Endpoint Object Structure](#custom-endpoint-object-structure)
|
||||
- **Required**
|
||||
|
||||
## Endpoint File Config Object Structure
|
||||
|
||||
### **Overview**
|
||||
|
||||
- `disabled`: Whether file handling is disabled for the endpoint.
|
||||
- `fileLimit`: The maximum number of files allowed per upload request.
|
||||
- `fileSizeLimit`: The maximum size for a single file. In units of MB (e.g. use `20` for 20 megabytes)
|
||||
- `totalSizeLimit`: The total maximum size for all files in a single request. In units of MB (e.g. use `20` for 20 megabytes)
|
||||
- `supportedMimeTypes`: A list of [Regular Expressions](https://en.wikipedia.org/wiki/Regular_expression) specifying what MIME types are allowed for upload. This can be customized to restrict file types.
|
||||
|
||||
**Notes:**
|
||||
|
||||
- At the time of writing, the Assistants endpoint [supports filetypes from this list](https://platform.openai.com/docs/assistants/tools/supported-files).
|
||||
- The OpenAI, Azure OpenAI, Google, and Custom endpoints only suppport images.
|
||||
- Any other endpoints not mentioned, like Plugins, do not support file uploads (yet).
|
||||
- The Assistants endpoint has a defined endpoint value of `assistants`. All other endpoints use the defined value `default`
|
||||
- For non-assistants endpoints, you can adjust file settings for all of them under `default`
|
||||
- If you'd like to adjust settings for a specific endpoint, you can list their corresponding endpoint names:
|
||||
- `assistants` (does not use `default` as it has defined defaults separate from the others.)
|
||||
- `openAI`
|
||||
- `azureOpenAI`
|
||||
- `google`
|
||||
- `YourCustomEndpointName`
|
||||
- You can omit values, in which case, the app will use the default values as defined per endpoint type listed below.
|
||||
- LibreChat counts 1 megabyte as follows: `1 x 1024 x 1024`
|
||||
|
||||
### Example
|
||||
|
||||
```yaml
|
||||
fileConfig:
|
||||
endpoints:
|
||||
assistants:
|
||||
fileLimit: 5
|
||||
fileSizeLimit: 10
|
||||
totalSizeLimit: 50
|
||||
supportedMimeTypes:
|
||||
- "image/.*"
|
||||
- "application/pdf"
|
||||
openAI:
|
||||
disabled: true
|
||||
default:
|
||||
totalSizeLimit: 20
|
||||
YourCustomEndpointName:
|
||||
fileLimit: 5
|
||||
fileSizeLimit: 1000
|
||||
supportedMimeTypes:
|
||||
- "image/.*"
|
||||
serverFileSizeLimit: 1000
|
||||
avatarSizeLimit: 2
|
||||
```
|
||||
|
||||
### **disabled**:
|
||||
|
||||
> Indicates whether file uploading is disabled for a specific endpoint.
|
||||
|
||||
- Type: Boolean
|
||||
- Default: `false` (i.e., uploading is enabled by default)
|
||||
- Example:
|
||||
```yaml
|
||||
openAI:
|
||||
disabled: true
|
||||
```
|
||||
- **Note**: Setting this to `true` prevents any file uploads to the specified endpoint, overriding any other file-related settings.
|
||||
|
||||
### **fileLimit**:
|
||||
|
||||
> The maximum number of files allowed in a single upload request.
|
||||
|
||||
- Type: Integer
|
||||
- Default: Varies by endpoint
|
||||
- Example:
|
||||
```yaml
|
||||
assistants:
|
||||
fileLimit: 5
|
||||
```
|
||||
- **Note**: Helps control the volume of uploads and manage server load.
|
||||
|
||||
### **fileSizeLimit**:
|
||||
|
||||
> The maximum size allowed for each individual file, specified in megabytes (MB).
|
||||
|
||||
- Type: Integer
|
||||
- Default: Varies by endpoint
|
||||
- Example:
|
||||
```yaml
|
||||
YourCustomEndpointName:
|
||||
fileSizeLimit: 1000
|
||||
```
|
||||
- **Note**: This limit ensures that no single file exceeds the specified size, allowing for better resource allocation and management.
|
||||
|
||||
### **totalSizeLimit**:
|
||||
|
||||
> The total maximum size allowed for all files in a single request, specified in megabytes (MB).
|
||||
|
||||
- Type: Integer
|
||||
- Default: Varies by endpoint
|
||||
- Example:
|
||||
```yaml
|
||||
assistants:
|
||||
totalSizeLimit: 50
|
||||
```
|
||||
- **Note**: This setting is crucial for preventing excessive bandwidth and storage usage by any single upload request.
|
||||
|
||||
### **supportedMimeTypes**:
|
||||
|
||||
> A list of regular expressions defining the MIME types permitted for upload.
|
||||
|
||||
- Type: Array of Strings
|
||||
- Default: Varies by endpoint
|
||||
- Example:
|
||||
```yaml
|
||||
assistants:
|
||||
supportedMimeTypes:
|
||||
- "image/.*"
|
||||
- "application/pdf"
|
||||
```
|
||||
- **Note**: This allows for precise control over the types of files that can be uploaded. Invalid regex is ignored.
|
||||
|
||||
### **serverFileSizeLimit**:
|
||||
|
||||
> The global maximum size for any file uploaded to the server, specified in megabytes (MB).
|
||||
|
||||
- Type: Integer
|
||||
- Example:
|
||||
```yaml
|
||||
fileConfig:
|
||||
serverFileSizeLimit: 1000
|
||||
```
|
||||
- **Note**: Acts as an overarching limit for file uploads across all endpoints, ensuring that no file exceeds this size server-wide.
|
||||
|
||||
### **avatarSizeLimit**:
|
||||
|
||||
> The maximum size allowed for avatar images, specified in megabytes (MB).
|
||||
|
||||
- Type: Integer
|
||||
- Example:
|
||||
```yaml
|
||||
fileConfig:
|
||||
avatarSizeLimit: 2
|
||||
```
|
||||
- **Note**: Specifically tailored for user avatar uploads, allowing for control over image sizes to maintain consistent quality and loading times.
|
||||
|
||||
## Registration Object Structure
|
||||
|
||||
```yaml
|
||||
|
|
@ -169,9 +442,74 @@ registration:
|
|||
- **Required**
|
||||
- **Note**: Users with email domains not listed will be restricted from registering.
|
||||
|
||||
Given the additional details and correction regarding `supportedMimeTypes` being a list of regex strings and the omission of the `assistantEndpoint` configuration, let's revise and add the necessary documentation sections.
|
||||
|
||||
## Assistants Endpoint Object Structure
|
||||
|
||||
### Example
|
||||
|
||||
```yaml
|
||||
endpoints:
|
||||
assistants:
|
||||
disableBuilder: false
|
||||
pollIntervalMs: 500
|
||||
timeoutMs: 10000
|
||||
# Use either `supportedIds` or `excludedIds` but not both
|
||||
supportedIds: ["asst_supportedAssistantId1", "asst_supportedAssistantId2"]
|
||||
# excludedIds: ["asst_excludedAssistantId"]
|
||||
```
|
||||
> This configuration enables the builder interface for assistants, sets a polling interval of 500ms to check for run updates, and establishes a timeout of 10 seconds for assistant run operations.
|
||||
|
||||
In addition to custom endpoints, you can configure settings specific to the assistants endpoint.
|
||||
|
||||
### **disableBuilder**:
|
||||
|
||||
> Controls the visibility and use of the builder interface for assistants.
|
||||
|
||||
- **Type**: Boolean
|
||||
- **Example**: `disableBuilder: false`
|
||||
- **Description**: When set to `true`, disables the builder interface for the assistant, limiting direct manual interaction.
|
||||
- **Note**: Defaults to `false` if omitted.
|
||||
|
||||
### **pollIntervalMs**:
|
||||
|
||||
> Specifies the polling interval in milliseconds for checking run updates or changes in assistant run states.
|
||||
|
||||
- **Type**: Integer
|
||||
- **Example**: `pollIntervalMs: 500`
|
||||
- **Description**: Specifies the polling interval in milliseconds for checking assistant run updates.
|
||||
- **Note**: Defaults to `750` if omitted.
|
||||
|
||||
### **timeoutMs**:
|
||||
|
||||
> Defines the maximum time in milliseconds that an assistant can run before the request is cancelled.
|
||||
|
||||
- **Type**: Integer
|
||||
- **Example**: `timeoutMs: 10000`
|
||||
- **Description**: Sets a timeout in milliseconds for assistant runs. Helps manage system load by limiting total run operation time.
|
||||
- **Note**: Defaults to 3 minutes (180,000 ms). Run operation times can range between 50 seconds to 2 minutes but also exceed this. If the `timeoutMs` value is exceeded, the run will be cancelled.
|
||||
|
||||
### **supportedIds**:
|
||||
|
||||
> List of supported assistant Ids
|
||||
|
||||
- Type: Array/List of Strings
|
||||
- **Description**: List of supported assistant Ids. Use this or `excludedIds` but not both (the `excludedIds` field will be ignored if so).
|
||||
- **Example**: `supportedIds: ["asst_supportedAssistantId1", "asst_supportedAssistantId2"]`
|
||||
|
||||
### **excludedIds**:
|
||||
|
||||
> List of excluded assistant Ids
|
||||
|
||||
- Type: Array/List of Strings
|
||||
- **Description**: List of excluded assistant Ids. Use this or `supportedIds` but not both (the `excludedIds` field will be ignored if so).
|
||||
- **Example**: `excludedIds: ["asst_excludedAssistantId1", "asst_excludedAssistantId2"]`
|
||||
|
||||
## Custom Endpoint Object Structure
|
||||
Each endpoint in the `custom` array should have the following structure:
|
||||
|
||||
### Example
|
||||
|
||||
```yaml
|
||||
# Example Endpoint Object Structure
|
||||
endpoints:
|
||||
|
|
@ -343,10 +681,10 @@ endpoints:
|
|||
Content-Type: "application/json"
|
||||
```
|
||||
|
||||
## Additional Notes
|
||||
### Additional Notes
|
||||
- Ensure that all URLs and keys are correctly specified to avoid connectivity issues.
|
||||
|
||||
## Default Parameters
|
||||
### Default Parameters
|
||||
|
||||
Custom endpoints share logic with the OpenAI endpoint, and thus have default parameters tailored to the OpenAI API.
|
||||
|
||||
|
|
@ -385,46 +723,3 @@ Custom endpoints share logic with the OpenAI endpoint, and thus have default par
|
|||
|
||||
**Note:** The `max_tokens` field is not sent to use the maximum amount of tokens available, which is default OpenAI API behavior. Some alternate APIs require this field, or it may default to a very low value and your responses may appear cut off; in this case, you should add it to `addParams` field as shown in the [Endpoint Object Structure](#endpoint-object-structure).
|
||||
|
||||
## Example Config
|
||||
|
||||
```yaml
|
||||
version: 1.0.2
|
||||
cache: true
|
||||
# Example Registration Object Structure
|
||||
registration:
|
||||
allowedDomains:
|
||||
- "gmail.com"
|
||||
endpoints:
|
||||
custom:
|
||||
# Mistral AI API
|
||||
- name: "Mistral"
|
||||
apiKey: "your_api_key"
|
||||
baseURL: "https://api.mistral.ai/v1"
|
||||
models:
|
||||
default: ["mistral-tiny", "mistral-small", "mistral-medium"]
|
||||
titleConvo: true
|
||||
titleModel: "mistral-tiny"
|
||||
summarize: false
|
||||
summaryModel: "mistral-tiny"
|
||||
forcePrompt: false
|
||||
modelDisplayLabel: "Mistral"
|
||||
addParams:
|
||||
safe_prompt: true
|
||||
# NOTE: For Mistral, it is necessary to drop the following parameters or you will encounter a 422 Error:
|
||||
dropParams: ["stop", "user", "frequency_penalty", "presence_penalty"]
|
||||
|
||||
# OpenRouter.ai API
|
||||
- name: "OpenRouter"
|
||||
# Known issue: you should not use `OPENROUTER_API_KEY` as it will then override the `openAI` endpoint to use OpenRouter as well.
|
||||
apiKey: "${OPENROUTER_KEY}"
|
||||
baseURL: "https://openrouter.ai/api/v1"
|
||||
models:
|
||||
default: ["gpt-3.5-turbo"]
|
||||
fetch: true
|
||||
titleConvo: true
|
||||
titleModel: "gpt-3.5-turbo"
|
||||
summarize: false
|
||||
summaryModel: "gpt-3.5-turbo"
|
||||
forcePrompt: false
|
||||
modelDisplayLabel: "OpenRouter"
|
||||
```
|
||||
|
|
|
|||
|
|
@ -350,6 +350,35 @@ OPENAI_REVERSE_PROXY=
|
|||
OPENAI_FORCE_PROMPT=true
|
||||
```
|
||||
|
||||
### Assistants
|
||||
|
||||
- The [Assistants API by OpenAI](https://platform.openai.com/docs/assistants/overview) has a dedicated endpoint.
|
||||
- To get your OpenAI API key, you need to:
|
||||
- Go to [https://platform.openai.com/account/api-keys](https://platform.openai.com/account/api-keys)
|
||||
- Create an account or log in with your existing one
|
||||
- Add a payment method to your account (this is not free, sorry 😬)
|
||||
- Copy your secret key (sk-...) to `ASSISTANTS_API_KEY`
|
||||
|
||||
- Leave `ASSISTANTS_API_KEY=` blank to disable this endpoint
|
||||
- Set `ASSISTANTS_API_KEY=` to `user_provided` to allow users to provide their own API key from the WebUI
|
||||
|
||||
- Customize the available models, separated by commas, **without spaces**.
|
||||
- The first will be default.
|
||||
- Leave it blank or commented out to use internal settings:
|
||||
- The models list will be fetched from OpenAI but only Assistants-API-compatible models will be shown; at the time of writing, they are as shown in the example below.
|
||||
|
||||
```bash
|
||||
ASSISTANTS_MODELS=gpt-3.5-turbo-0125,gpt-3.5-turbo-16k-0613,gpt-3.5-turbo-16k,gpt-3.5-turbo,gpt-4,gpt-4-0314,gpt-4-32k-0314,gpt-4-0613,gpt-3.5-turbo-0613,gpt-3.5-turbo-1106,gpt-4-0125-preview,gpt-4-turbo-preview,gpt-4-1106-preview
|
||||
```
|
||||
|
||||
- If necessary, you can also set an alternate base URL instead of the official one with `ASSISTANTS_BASE_URL`, which is similar to the OpenAI counterpart `OPENAI_REVERSE_PROXY`
|
||||
|
||||
```bash
|
||||
ASSISTANTS_BASE_URL=http://your-alt-baseURL:3080/
|
||||
```
|
||||
|
||||
- There is additional, optional configuration, depending on your needs, such as disabling the assistant builder UI, and determining which assistants can be used, that are available via the [`librechat.yaml` custom config file](./custom_config.md#assistants-endpoint-object-structure).
|
||||
|
||||
### OpenRouter
|
||||
See [OpenRouter](./free_ai_apis.md#openrouter-preferred) for more info.
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue