mirror of
https://github.com/danny-avila/LibreChat.git
synced 2025-09-22 08:12:00 +02:00
29473a72db
* WIP(backend/api): custom endpoint * WIP(frontend/client): custom endpoint * chore: adjust typedefs for configs * refactor: use data-provider for cache keys and rename enums and custom endpoint for better clarity and compatibility * feat: loadYaml utility * refactor: rename back to from and proof-of-concept for creating schemas from user-defined defaults * refactor: remove custom endpoint from default endpointsConfig as it will be exclusively managed by yaml config * refactor(EndpointController): rename variables for clarity * feat: initial load custom config * feat(server/utils): add simple `isUserProvided` helper * chore(types): update TConfig type * refactor: remove custom endpoint handling from model services as will be handled by config, modularize fetching of models * feat: loadCustomConfig, loadConfigEndpoints, loadConfigModels * chore: reorganize server init imports, invoke loadCustomConfig * refactor(loadConfigEndpoints/Models): return each custom endpoint as standalone endpoint * refactor(Endpoint/ModelController): spread config values after default (temporary) * chore(client): fix type issues * WIP: first pass for multiple custom endpoints - add endpointType to Conversation schema - add update zod schemas for both convo/presets to allow non-EModelEndpoint value as endpoint (also using type assertion) - use `endpointType` value as `endpoint` where mapping to type is necessary using this field - use custom defined `endpoint` value and not type for mapping to modelsConfig - misc: add return type to `getDefaultEndpoint` - in `useNewConvo`, add the endpointType if it wasn't already added to conversation - EndpointsMenu: use user-defined endpoint name as Title in menu - TODO: custom icon via custom config, change unknown to robot icon * refactor(parseConvo): pass args as an object and change where used accordingly; chore: comment out 'create schema' code * chore: remove unused availableModels field in TConfig type * refactor(parseCompactConvo): pass args as an object and change where used accordingly * feat: chat through custom endpoint * chore(message/convoSchemas): avoid saving empty arrays * fix(BaseClient/saveMessageToDatabase): save endpointType * refactor(ChatRoute): show Spinner if endpointsQuery or modelsQuery are still loading, which is apparent with slow fetching of models/remote config on first serve * fix(useConversation): assign endpointType if it's missing * fix(SaveAsPreset): pass real endpoint and endpointType when saving Preset) * chore: recorganize types order for TConfig, add `iconURL` * feat: custom endpoint icon support: - use UnknownIcon in all icon contexts - add mistral and openrouter as known endpoints, and add their icons - iconURL support * fix(presetSchema): move endpointType to default schema definitions shared between convoSchema and defaults * refactor(Settings/OpenAI): remove legacy `isOpenAI` flag * fix(OpenAIClient): do not invoke abortCompletion on completion error * feat: add responseSender/label support for custom endpoints: - use defaultModelLabel field in endpointOption - add model defaults for custom endpoints in `getResponseSender` - add `useGetSender` hook which uses EndpointsQuery to determine `defaultModelLabel` - include defaultModelLabel from endpointConfig in custom endpoint client options - pass `endpointType` to `getResponseSender` * feat(OpenAIClient): use custom options from config file * refactor: rename `defaultModelLabel` to `modelDisplayLabel` * refactor(data-provider): separate concerns from `schemas` into `parsers`, `config`, and fix imports elsewhere * feat: `iconURL` and extract environment variables from custom endpoint config values * feat: custom config validation via zod schema, rename and move to `./projectRoot/librechat.yaml` * docs: custom config docs and examples * fix(OpenAIClient/mistral): mistral does not allow singular system message, also add `useChatCompletion` flag to use openai-node for title completions * fix(custom/initializeClient): extract env var and use `isUserProvided` function * Update librechat.example.yaml * feat(InputWithLabel): add className props, and forwardRef * fix(streamResponse): handle error edge case where either messages or convos query throws an error * fix(useSSE): handle errorHandler edge cases where error response is and is not properly formatted from API, especially when a conversationId is not yet provided, which ensures stream is properly closed on error * feat: user_provided keys for custom endpoints * fix(config/endpointSchema): do not allow default endpoint values in custom endpoint `name` * feat(loadConfigModels): extract env variables and optimize fetching models * feat: support custom endpoint iconURL for messages and Nav * feat(OpenAIClient): add/dropParams support * docs: update docs with default params, add/dropParams, and notes to use config file instead of `OPENAI_REVERSE_PROXY` * docs: update docs with additional notes * feat(maxTokensMap): add mistral models (32k context) * docs: update openrouter notes * Update ai_setup.md * docs(custom_config): add table of contents and fix note about custom name * docs(custom_config): reorder ToC * Update custom_config.md * Add note about `max_tokens` field in custom_config.md
339 lines
17 KiB
JavaScript
339 lines
17 KiB
JavaScript
/**
|
|
* @namespace typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports OpenAI
|
|
* @typedef {import('openai').OpenAI} OpenAI
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports Assistant
|
|
* @typedef {import('librechat-data-provider').Assistant} Assistant
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports OpenAIFile
|
|
* @typedef {import('librechat-data-provider').File} OpenAIFile
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports TConfig
|
|
* @typedef {import('librechat-data-provider').TConfig} TConfig
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports ImageMetadata
|
|
* @typedef {Object} ImageMetadata
|
|
* @property {string} file_id - The identifier of the file.
|
|
* @property {string} [temp_file_id] - The temporary identifier of the file.
|
|
* @property {number} width - The width of the image.
|
|
* @property {number} height - The height of the image.
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports MongoFile
|
|
* @typedef {import('~/models/schema/fileSchema.js').MongoFile} MongoFile
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports AssistantCreateParams
|
|
* @typedef {import('librechat-data-provider').AssistantCreateParams} AssistantCreateParams
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports AssistantUpdateParams
|
|
* @typedef {import('librechat-data-provider').AssistantUpdateParams} AssistantUpdateParams
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports AssistantListParams
|
|
* @typedef {import('librechat-data-provider').AssistantListParams} AssistantListParams
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports AssistantListResponse
|
|
* @typedef {import('librechat-data-provider').AssistantListResponse} AssistantListResponse
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports ThreadMessage
|
|
* @typedef {import('openai').OpenAI.Beta.Threads.ThreadMessage} ThreadMessage
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports RequiredActionFunctionToolCall
|
|
* @typedef {import('openai').OpenAI.Beta.Threads.RequiredActionFunctionToolCall} RequiredActionFunctionToolCall
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports RunManager
|
|
* @typedef {import('./server/services/Runs/RunMananger.js').RunManager} RunManager
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports Thread
|
|
* @typedef {Object} Thread
|
|
* @property {string} id - The identifier of the thread.
|
|
* @property {string} object - The object type, always 'thread'.
|
|
* @property {number} created_at - The Unix timestamp (in seconds) for when the thread was created.
|
|
* @property {Object} [metadata] - Optional metadata associated with the thread.
|
|
* @property {Message[]} [messages] - An array of messages associated with the thread.
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports Message
|
|
* @typedef {Object} Message
|
|
* @property {string} id - The identifier of the message.
|
|
* @property {string} object - The object type, always 'thread.message'.
|
|
* @property {number} created_at - The Unix timestamp (in seconds) for when the message was created.
|
|
* @property {string} thread_id - The thread ID that this message belongs to.
|
|
* @property {string} role - The entity that produced the message. One of 'user' or 'assistant'.
|
|
* @property {Object[]} content - The content of the message in an array of text and/or images.
|
|
* @property {string} content[].type - The type of content, either 'text' or 'image_file'.
|
|
* @property {Object} [content[].text] - The text content, present if type is 'text'.
|
|
* @property {string} content[].text.value - The data that makes up the text.
|
|
* @property {Object[]} [content[].text.annotations] - Annotations for the text content.
|
|
* @property {Object} [content[].image_file] - The image file content, present if type is 'image_file'.
|
|
* @property {string} content[].image_file.file_id - The File ID of the image in the message content.
|
|
* @property {string[]} [file_ids] - Optional list of File IDs for the message.
|
|
* @property {string|null} [assistant_id] - If applicable, the ID of the assistant that authored this message.
|
|
* @property {string|null} [run_id] - If applicable, the ID of the run associated with the authoring of this message.
|
|
* @property {Object} [metadata] - Optional metadata for the message, a map of key-value pairs.
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports UserMessageContent
|
|
* @typedef {Object} UserMessageContent
|
|
* @property {Object[]} content - The content of the message in an array of text and/or images.
|
|
* @property {string} content[].type - The type of content, either 'text' or 'image_file'.
|
|
* @property {Object} [content[].text] - The text content, present if type is 'text'.
|
|
* @property {string} content[].text.value - The data that makes up the text.
|
|
* @property {Object} [content[].image_url] - The image file content, present if type is 'image_file'.
|
|
* @property {string} content[].image_url.url - The File ID of the image in the message content.
|
|
* @property {'auto' | 'low' | 'high'} content[].image_url.detail: 'auto' - the quality to use for the image, either 'auto', 'low', or 'high'.
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* Represents a message payload with various potential properties,
|
|
* including roles, sender information, and content.
|
|
*
|
|
* @typedef {Object} PayloadMessage
|
|
* @property {string} [role] - The role of the message sender (e.g., 'user', 'assistant').
|
|
* @property {string} [name] - The name associated with the message.
|
|
* @property {string} [sender] - The sender of the message.
|
|
* @property {string} [text] - The text content of the message.
|
|
* @property {(string|Array<UserMessageContent>)} [content] - The content of the message, which could be a string or an array of the 'content' property from the Message type.
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports FunctionTool
|
|
* @typedef {Object} FunctionTool
|
|
* @property {string} type - The type of tool, 'function'.
|
|
* @property {Object} function - The function definition.
|
|
* @property {string} function.description - A description of what the function does.
|
|
* @property {string} function.name - The name of the function to be called.
|
|
* @property {Object} function.parameters - The parameters the function accepts, described as a JSON Schema object.
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports Tool
|
|
* @typedef {Object} Tool
|
|
* @property {string} type - The type of tool, can be 'code_interpreter', 'retrieval', or 'function'.
|
|
* @property {FunctionTool} [function] - The function tool, present if type is 'function'.
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports Run
|
|
* @typedef {Object} Run
|
|
* @property {string} id - The identifier of the run.
|
|
* @property {string} object - The object type, always 'thread.run'.
|
|
* @property {number} created_at - The Unix timestamp (in seconds) for when the run was created.
|
|
* @property {string} thread_id - The ID of the thread that was executed on as a part of this run.
|
|
* @property {string} assistant_id - The ID of the assistant used for execution of this run.
|
|
* @property {string} status - The status of the run (e.g., 'queued', 'completed').
|
|
* @property {Object} [required_action] - Details on the action required to continue the run.
|
|
* @property {string} required_action.type - The type of required action, always 'submit_tool_outputs'.
|
|
* @property {Object} required_action.submit_tool_outputs - Details on the tool outputs needed for the run to continue.
|
|
* @property {Object[]} required_action.submit_tool_outputs.tool_calls - A list of the relevant tool calls.
|
|
* @property {string} required_action.submit_tool_outputs.tool_calls[].id - The ID of the tool call.
|
|
* @property {string} required_action.submit_tool_outputs.tool_calls[].type - The type of tool call the output is required for, always 'function'.
|
|
* @property {Object} required_action.submit_tool_outputs.tool_calls[].function - The function definition.
|
|
* @property {string} required_action.submit_tool_outputs.tool_calls[].function.name - The name of the function.
|
|
* @property {string} required_action.submit_tool_outputs.tool_calls[].function.arguments - The arguments that the model expects you to pass to the function.
|
|
* @property {Object} [last_error] - The last error associated with this run.
|
|
* @property {string} last_error.code - One of 'server_error' or 'rate_limit_exceeded'.
|
|
* @property {string} last_error.message - A human-readable description of the error.
|
|
* @property {number} [expires_at] - The Unix timestamp (in seconds) for when the run will expire.
|
|
* @property {number} [started_at] - The Unix timestamp (in seconds) for when the run was started.
|
|
* @property {number} [cancelled_at] - The Unix timestamp (in seconds) for when the run was cancelled.
|
|
* @property {number} [failed_at] - The Unix timestamp (in seconds) for when the run failed.
|
|
* @property {number} [completed_at] - The Unix timestamp (in seconds) for when the run was completed.
|
|
* @property {string} [model] - The model that the assistant used for this run.
|
|
* @property {string} [instructions] - The instructions that the assistant used for this run.
|
|
* @property {Tool[]} [tools] - The list of tools used for this run.
|
|
* @property {string[]} [file_ids] - The list of File IDs used for this run.
|
|
* @property {Object} [metadata] - Metadata associated with this run.
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports RunStep
|
|
* @typedef {Object} RunStep
|
|
* @property {string} id - The identifier of the run step.
|
|
* @property {string} object - The object type, always 'thread.run.step'.
|
|
* @property {number} created_at - The Unix timestamp (in seconds) for when the run step was created.
|
|
* @property {string} assistant_id - The ID of the assistant associated with the run step.
|
|
* @property {string} thread_id - The ID of the thread that was run.
|
|
* @property {string} run_id - The ID of the run that this run step is a part of.
|
|
* @property {string} type - The type of run step, either 'message_creation' or 'tool_calls'.
|
|
* @property {string} status - The status of the run step, can be 'in_progress', 'cancelled', 'failed', 'completed', or 'expired'.
|
|
* @property {Object} step_details - The details of the run step.
|
|
* @property {Object} [last_error] - The last error associated with this run step.
|
|
* @property {string} last_error.code - One of 'server_error' or 'rate_limit_exceeded'.
|
|
* @property {string} last_error.message - A human-readable description of the error.
|
|
* @property {number} [expired_at] - The Unix timestamp (in seconds) for when the run step expired.
|
|
* @property {number} [cancelled_at] - The Unix timestamp (in seconds) for when the run step was cancelled.
|
|
* @property {number} [failed_at] - The Unix timestamp (in seconds) for when the run step failed.
|
|
* @property {number} [completed_at] - The Unix timestamp (in seconds) for when the run step completed.
|
|
* @property {Object} [metadata] - Metadata associated with this run step, a map of up to 16 key-value pairs.
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports StepMessage
|
|
* @typedef {Object} StepMessage
|
|
* @property {Message} message - The complete message object created by the step.
|
|
* @property {string} id - The identifier of the run step.
|
|
* @property {string} object - The object type, always 'thread.run.step'.
|
|
* @property {number} created_at - The Unix timestamp (in seconds) for when the run step was created.
|
|
* @property {string} assistant_id - The ID of the assistant associated with the run step.
|
|
* @property {string} thread_id - The ID of the thread that was run.
|
|
* @property {string} run_id - The ID of the run that this run step is a part of.
|
|
* @property {string} type - The type of run step, either 'message_creation' or 'tool_calls'.
|
|
* @property {string} status - The status of the run step, can be 'in_progress', 'cancelled', 'failed', 'completed', or 'expired'.
|
|
* @property {Object} step_details - The details of the run step.
|
|
* @property {Object} [last_error] - The last error associated with this run step.
|
|
* @property {string} last_error.code - One of 'server_error' or 'rate_limit_exceeded'.
|
|
* @property {string} last_error.message - A human-readable description of the error.
|
|
* @property {number} [expired_at] - The Unix timestamp (in seconds) for when the run step expired.
|
|
* @property {number} [cancelled_at] - The Unix timestamp (in seconds) for when the run step was cancelled.
|
|
* @property {number} [failed_at] - The Unix timestamp (in seconds) for when the run step failed.
|
|
* @property {number} [completed_at] - The Unix timestamp (in seconds) for when the run step completed.
|
|
* @property {Object} [metadata] - Metadata associated with this run step, a map of up to 16 key-value pairs.
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports AgentAction
|
|
* @typedef {Object} AgentAction
|
|
* @property {string} tool - The name of the tool used.
|
|
* @property {string} toolInput - The input provided to the tool.
|
|
* @property {string} log - A log or message associated with the action.
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports AgentFinish
|
|
* @typedef {Object} AgentFinish
|
|
* @property {Record<string, any>} returnValues - The return values of the agent's execution.
|
|
* @property {string} log - A log or message associated with the finish.
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports OpenAIAssistantFinish
|
|
* @typedef {AgentFinish & { run_id: string; thread_id: string; }} OpenAIAssistantFinish
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports OpenAIAssistantAction
|
|
* @typedef {AgentAction & { toolCallId: string; run_id: string; thread_id: string; }} OpenAIAssistantAction
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports EndpointServiceConfig
|
|
* @typedef {Object} EndpointServiceConfig
|
|
* @property {string} openAIApiKey - The API key for OpenAI.
|
|
* @property {string} azureOpenAIApiKey - The API key for Azure OpenAI.
|
|
* @property {boolean} useAzurePlugins - Flag to indicate if Azure plugins are used.
|
|
* @property {boolean} userProvidedOpenAI - Flag to indicate if OpenAI API key is user provided.
|
|
* @property {string} googleKey - The Palm key.
|
|
* @property {boolean|{userProvide: boolean}} [openAI] - Flag to indicate if OpenAI endpoint is user provided, or its configuration.
|
|
* @property {boolean|{userProvide: boolean}} [assistant] - Flag to indicate if Assistant endpoint is user provided, or its configuration.
|
|
* @property {boolean|{userProvide: boolean}} [azureOpenAI] - Flag to indicate if Azure OpenAI endpoint is user provided, or its configuration.
|
|
* @property {boolean|{userProvide: boolean}} [chatGPTBrowser] - Flag to indicate if ChatGPT Browser endpoint is user provided, or its configuration.
|
|
* @property {boolean|{userProvide: boolean}} [anthropic] - Flag to indicate if Anthropic endpoint is user provided, or its configuration.
|
|
* @property {boolean|{userProvide: boolean}} [bingAI] - Flag to indicate if BingAI endpoint is user provided, or its configuration.
|
|
* @property {boolean|{userProvide: boolean}} [google] - Flag to indicate if BingAI endpoint is user provided, or its configuration.
|
|
* @property {boolean|{userProvide: boolean, userProvideURL: boolean, name: string}} [custom] - Custom Endpoint configuration.
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports Plugin
|
|
* @typedef {Object} Plugin
|
|
* @property {string} pluginKey - The key of the plugin.
|
|
* @property {string} name - The name of the plugin.
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports GptPlugins
|
|
* @typedef {Object} GptPlugins
|
|
* @property {Plugin[]} plugins - An array of plugins available.
|
|
* @property {string[]} availableAgents - Available agents, 'classic' or 'functions'.
|
|
* @property {boolean} userProvide - A flag indicating if the user has provided the data.
|
|
* @property {boolean} azure - A flag indicating if azure plugins are used.
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports DefaultConfig
|
|
* @typedef {Object} DefaultConfig
|
|
* @property {boolean|{userProvide: boolean}} [openAI] - Flag to indicate if OpenAI endpoint is user provided, or its configuration.
|
|
* @property {boolean|{userProvide: boolean}} [assistant] - Flag to indicate if Assistant endpoint is user provided, or its configuration.
|
|
* @property {boolean|{userProvide: boolean}} [azureOpenAI] - Flag to indicate if Azure OpenAI endpoint is user provided, or its configuration.
|
|
* @property {boolean|{userProvide: boolean}} [chatGPTBrowser] - Flag to indicate if ChatGPT Browser endpoint is user provided, or its configuration.
|
|
* @property {boolean|{userProvide: boolean}} [anthropic] - Flag to indicate if Anthropic endpoint is user provided, or its configuration.
|
|
* @property {boolean|{userProvide: boolean}} [bingAI] - Flag to indicate if BingAI endpoint is user provided, or its configuration.
|
|
* @property {boolean|{userProvide: boolean}} [google] - Flag to indicate if Google endpoint is user provided, or its configuration.
|
|
* @property {boolean|{userProvide: boolean, userProvideURL: boolean, name: string}} [custom] - Custom Endpoint configuration.
|
|
* @property {boolean|GptPlugins} [gptPlugins] - Configuration for GPT plugins.
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports EndpointConfig
|
|
* @typedef {boolean|TConfig} EndpointConfig
|
|
* @memberof typedefs
|
|
*/
|
|
|
|
/**
|
|
* @exports EndpointWithOrder
|
|
* @typedef {Object} EndpointWithOrder
|
|
* @property {EndpointConfig} config - The configuration of the endpoint.
|
|
* @property {number} order - The order of the endpoint.
|
|
* @memberof typedefs
|
|
*/
|