LibreChat/api/server/routes/ask/index.js

49 lines
1.4 KiB
JavaScript
Raw Normal View History

2023-02-12 16:38:33 -05:00
const express = require('express');
refactor: Client Classes & Azure OpenAI as a separate Endpoint (#532) * refactor: start new client classes, test localAi support * feat: create base class, extend chatgpt from base * refactor(BaseClient.js): change userId parameter to user refactor(BaseClient.js): change userId parameter to user feat(OpenAIClient.js): add sendMessage method refactor(OpenAIClient.js): change getConversation method to use user parameter instead of userId refactor(OpenAIClient.js): change saveMessageToDatabase method to use user parameter instead of userId refactor(OpenAIClient.js): change buildPrompt method to use messages parameter instead of orderedMessages feat(index.js): export client classes refactor(askGPTPlugins.js): use req.body.token or process.env.OPENAI_API_KEY as OpenAI API key refactor(index.js): comment out askOpenAI route feat(index.js): add openAI route feat(openAI.js): add new route for OpenAI API requests with support for progress updates and aborting requests. * refactor(BaseClient.js): use optional chaining operator to access messageId property refactor(OpenAIClient.js): use orderedMessages instead of messages to build prompt refactor(OpenAIClient.js): use optional chaining operator to access messageId property refactor(fetch-polyfill.js): remove fetch polyfill refactor(openAI.js): comment out debug option in clientOptions * refactor: update import statements and remove unused imports in several files feat: add getAzureCredentials function to azureUtils module docs: update comments in azureUtils module * refactor(utils): rename migrateConversations to migrateDataToFirstUser for clarity and consistency * feat(chatgpt-client.js): add getAzureCredentials function to retrieve Azure credentials feat(chatgpt-client.js): use getAzureCredentials function to generate reverseProxyUrl feat(OpenAIClient.js): add isChatCompletion property to determine if chat completion model is used feat(OpenAIClient.js): add saveOptions parameter to sendMessage and buildPrompt methods feat(OpenAIClient.js): modify buildPrompt method to handle chat completion model feat(openAI.js): modify endpointOption to include modelOptions instead of individual options refactor(OpenAIClient.js): modify getDelta property to use isChatCompletion property instead of isChatGptModel property refactor(OpenAIClient.js): modify sendMessage method to use saveOptions parameter instead of modelOptions parameter refactor(OpenAIClient.js): modify buildPrompt method to use saveOptions parameter instead of modelOptions parameter refactor(OpenAIClient.js): modify ask method to include endpointOption parameter * chore: delete draft file * refactor(OpenAIClient.js): extract sendCompletion method from sendMessage method for reusability * refactor(BaseClient.js): move sendMessage method to BaseClient class feat(OpenAIClient.js): inherit from BaseClient class and implement necessary methods and properties for OpenAIClient class. * refactor(BaseClient.js): rename getBuildPromptOptions to getBuildMessagesOptions feat(BaseClient.js): add buildMessages method to BaseClient class fix(ChatGPTClient.js): use message.text instead of message.message refactor(ChatGPTClient.js): rename buildPromptBody to buildMessagesBody refactor(ChatGPTClient.js): remove console.debug statement and add debug log for prompt variable refactor(OpenAIClient.js): move setOptions method to the bottom of the class feat(OpenAIClient.js): add support for cl100k_base encoding feat(OpenAIClient.js): add support for unofficial chat GPT models feat(OpenAIClient.js): add support for custom modelOptions feat(OpenAIClient.js): add caching for tokenizers feat(OpenAIClient.js): add freeAndInitializeEncoder method to free and reinitialize tokenizers refactor(OpenAIClient.js): rename getBuildPromptOptions to getBuildMessagesOptions refactor(OpenAIClient.js): rename buildPrompt to buildMessages refactor(OpenAIClient.js): remove endpointOption from ask function arguments in openAI.js * refactor(ChatGPTClient.js, OpenAIClient.js): improve code readability and consistency - In ChatGPTClient.js, update the roleLabel and messageString variables to handle cases where the message object does not have an isCreatedByUser property or a role property with a value of 'user'. - In OpenAIClient.js, rename the freeAndInitializeEncoder method to freeAndResetEncoder to better reflect its functionality. Also, update the method calls to reflect the new name. Additionally, update the getTokenCount method to handle errors by calling the freeAndResetEncoder method instead of the now-renamed freeAndInitializeEncoder method. * refactor(OpenAIClient.js): extract instructions object to a separate variable and add it to payload after formatted messages fix(OpenAIClient.js): handle cases where progressMessage.choices is undefined or empty * refactor(BaseClient.js): extract addInstructions method from sendMessage method feat(OpenAIClient.js): add maxTokensMap object to map maximum tokens for each model refactor(OpenAIClient.js): use addInstructions method in buildMessages method instead of manually building the payload list * refactor(OpenAIClient.js): remove unnecessary condition for modelOptions.model property in buildMessages method * feat(BaseClient.js): add support for token count tracking and context strategy feat(OpenAIClient.js): add support for token count tracking and context strategy feat(Message.js): add tokenCount field to Message schema and updateMessage function * refactor(BaseClient.js): add support for refining messages based on token limit feat(OpenAIClient.js): add support for context refinement strategy refactor(OpenAIClient.js): use context refinement strategy in message sending refactor(server/index.js): improve code readability by breaking long lines * refactor(BaseClient.js): change `remainingContext` to `remainingContextTokens` for clarity feat(BaseClient.js): add `refinePrompt` and `refinePromptTemplate` to handle message refinement feat(BaseClient.js): add `refineMessages` method to refine messages feat(BaseClient.js): add `handleContextStrategy` method to handle context strategy feat(OpenAIClient.js): add `abortController` to `buildPrompt` method options refactor(OpenAIClient.js): change `payload` and `tokenCountMap` to let variables in `handleContextStrategy` method refactor(BaseClient.js): change `remainingContext` to `remainingContextTokens` in `handleContextStrategy` method for consistency refactor(BaseClient.js): change `remainingContext` to `remainingContextTokens` in `getMessagesWithinTokenLimit` method for consistency refactor(BaseClient.js): change `remainingContext` to `remainingContext * chore(openAI.js): comment out contextStrategy option in clientOptions * chore(openAI.js): comment out debug option in clientOptions object * test: BaseClient tests in progress * test: Complete OpenAIClient & BaseClient tests * fix(OpenAIClient.js): remove unnecessary whitespace fix(OpenAIClient.js): remove unused variables and comments fix(OpenAIClient.test.js): combine getTokenCount and freeAndResetEncoder tests * chore(.eslintrc.js): add rule for maximum of 1 empty line feat(ask/openAI.js): add abortMessage utility function fix(ask/openAI.js): handle error and abort message if partial text is less than 2 characters feat(utils/index.js): export abortMessage utility function * test: complete additional tests * feat: Azure OpenAI as a separate endpoint * chore: remove extraneous console logs * fix(azureOpenAI): use chatCompletion endpoint * chore(initializeClient.js): delete initializeClient.js file chore(askOpenAI.js): delete old OpenAI route handler chore(handlers.js): remove trailing whitespace in thought variable assignment * chore(chatgpt-client.js): remove unused chatgpt-client.js file refactor(index.js): remove askClient import and export from index.js * chore(chatgpt-client.tokens.js): update test script for memory usage and encoding performance The test script in `chatgpt-client.tokens.js` has been updated to measure the memory usage and encoding performance of the client. The script now includes information about the initial memory usage, peak memory usage, final memory usage, and memory usage after a timeout. It also provides insights into the number of encoding requests that can be processed per second. The script has been modified to use the `OpenAIClient` class instead of the `ChatGPTClient` class. Additionally, the number of iterations for the encoding loop has been reduced to 10,000. A timeout function has been added to simulate a delay of 15 seconds. After the timeout, the memory usage is measured again. The script now handles uncaught exceptions and logs any errors that occur, except for errors related to failed fetch requests. Note: This is a test script and should not be used in production * feat(FakeClient.js): add a new class `FakeClient` that extends `BaseClient` and implements methods for a fake client feat(FakeClient.js): implement the `setOptions` method to handle options for the fake client feat(FakeClient.js): implement the `initializeFakeClient` function to initialize a fake client with options and fake messages fix(OpenAIClient.js): remove duplicate `maxTokensMap` import and use the one from utils feat(BaseClient): return promptTokens and completionTokens * refactor(gptPlugins): refactor ChatAgent to PluginsClient, which extends OpenAIClient * refactor: client paths * chore(jest.config.js): remove jest.config.js file * fix(PluginController.js): update file path to manifest.json feat(gptPlugins.js): add support for aborting messages refactor(ask/index.js): rename askGPTPlugins to gptPlugins for consistency * fix(BaseClient.js): fix spacing in generateTextStream function signature refactor(BaseClient.js): remove unnecessary push to currentMessages in generateUserMessage function refactor(BaseClient.js): remove unnecessary push to currentMessages in handleStartMethods function refactor(PluginsClient.js): remove unused variables and date formatting in constructor refactor(PluginsClient.js): simplify mapping of pastMessages in getCompletionPayload function * refactor(GoogleClient): GoogleClient now extends BaseClient * chore(.env.example): add AZURE_OPENAI_MODELS variable fix(api/routes/ask/gptPlugins.js): enable Azure integration if PLUGINS_USE_AZURE is true fix(api/routes/endpoints.js): getOpenAIModels function now accepts options, use AZURE_OPENAI_MODELS if PLUGINS_USE_AZURE is true fix(client/components/Endpoints/OpenAI/Settings.jsx): remove console.log statement docs(features/azure.md): add documentation for Azure OpenAI integration and environment variables * fix(e2e:popup): includes the icon + endpoint names in role, name property
2023-07-03 16:51:12 -04:00
const openAI = require('./openAI');
💫 feat: Config File & Custom Endpoints (#1474) * WIP(backend/api): custom endpoint * WIP(frontend/client): custom endpoint * chore: adjust typedefs for configs * refactor: use data-provider for cache keys and rename enums and custom endpoint for better clarity and compatibility * feat: loadYaml utility * refactor: rename back to from and proof-of-concept for creating schemas from user-defined defaults * refactor: remove custom endpoint from default endpointsConfig as it will be exclusively managed by yaml config * refactor(EndpointController): rename variables for clarity * feat: initial load custom config * feat(server/utils): add simple `isUserProvided` helper * chore(types): update TConfig type * refactor: remove custom endpoint handling from model services as will be handled by config, modularize fetching of models * feat: loadCustomConfig, loadConfigEndpoints, loadConfigModels * chore: reorganize server init imports, invoke loadCustomConfig * refactor(loadConfigEndpoints/Models): return each custom endpoint as standalone endpoint * refactor(Endpoint/ModelController): spread config values after default (temporary) * chore(client): fix type issues * WIP: first pass for multiple custom endpoints - add endpointType to Conversation schema - add update zod schemas for both convo/presets to allow non-EModelEndpoint value as endpoint (also using type assertion) - use `endpointType` value as `endpoint` where mapping to type is necessary using this field - use custom defined `endpoint` value and not type for mapping to modelsConfig - misc: add return type to `getDefaultEndpoint` - in `useNewConvo`, add the endpointType if it wasn't already added to conversation - EndpointsMenu: use user-defined endpoint name as Title in menu - TODO: custom icon via custom config, change unknown to robot icon * refactor(parseConvo): pass args as an object and change where used accordingly; chore: comment out 'create schema' code * chore: remove unused availableModels field in TConfig type * refactor(parseCompactConvo): pass args as an object and change where used accordingly * feat: chat through custom endpoint * chore(message/convoSchemas): avoid saving empty arrays * fix(BaseClient/saveMessageToDatabase): save endpointType * refactor(ChatRoute): show Spinner if endpointsQuery or modelsQuery are still loading, which is apparent with slow fetching of models/remote config on first serve * fix(useConversation): assign endpointType if it's missing * fix(SaveAsPreset): pass real endpoint and endpointType when saving Preset) * chore: recorganize types order for TConfig, add `iconURL` * feat: custom endpoint icon support: - use UnknownIcon in all icon contexts - add mistral and openrouter as known endpoints, and add their icons - iconURL support * fix(presetSchema): move endpointType to default schema definitions shared between convoSchema and defaults * refactor(Settings/OpenAI): remove legacy `isOpenAI` flag * fix(OpenAIClient): do not invoke abortCompletion on completion error * feat: add responseSender/label support for custom endpoints: - use defaultModelLabel field in endpointOption - add model defaults for custom endpoints in `getResponseSender` - add `useGetSender` hook which uses EndpointsQuery to determine `defaultModelLabel` - include defaultModelLabel from endpointConfig in custom endpoint client options - pass `endpointType` to `getResponseSender` * feat(OpenAIClient): use custom options from config file * refactor: rename `defaultModelLabel` to `modelDisplayLabel` * refactor(data-provider): separate concerns from `schemas` into `parsers`, `config`, and fix imports elsewhere * feat: `iconURL` and extract environment variables from custom endpoint config values * feat: custom config validation via zod schema, rename and move to `./projectRoot/librechat.yaml` * docs: custom config docs and examples * fix(OpenAIClient/mistral): mistral does not allow singular system message, also add `useChatCompletion` flag to use openai-node for title completions * fix(custom/initializeClient): extract env var and use `isUserProvided` function * Update librechat.example.yaml * feat(InputWithLabel): add className props, and forwardRef * fix(streamResponse): handle error edge case where either messages or convos query throws an error * fix(useSSE): handle errorHandler edge cases where error response is and is not properly formatted from API, especially when a conversationId is not yet provided, which ensures stream is properly closed on error * feat: user_provided keys for custom endpoints * fix(config/endpointSchema): do not allow default endpoint values in custom endpoint `name` * feat(loadConfigModels): extract env variables and optimize fetching models * feat: support custom endpoint iconURL for messages and Nav * feat(OpenAIClient): add/dropParams support * docs: update docs with default params, add/dropParams, and notes to use config file instead of `OPENAI_REVERSE_PROXY` * docs: update docs with additional notes * feat(maxTokensMap): add mistral models (32k context) * docs: update openrouter notes * Update ai_setup.md * docs(custom_config): add table of contents and fix note about custom name * docs(custom_config): reorder ToC * Update custom_config.md * Add note about `max_tokens` field in custom_config.md
2024-01-03 09:22:48 -05:00
const custom = require('./custom');
refactor: Client Classes & Azure OpenAI as a separate Endpoint (#532) * refactor: start new client classes, test localAi support * feat: create base class, extend chatgpt from base * refactor(BaseClient.js): change userId parameter to user refactor(BaseClient.js): change userId parameter to user feat(OpenAIClient.js): add sendMessage method refactor(OpenAIClient.js): change getConversation method to use user parameter instead of userId refactor(OpenAIClient.js): change saveMessageToDatabase method to use user parameter instead of userId refactor(OpenAIClient.js): change buildPrompt method to use messages parameter instead of orderedMessages feat(index.js): export client classes refactor(askGPTPlugins.js): use req.body.token or process.env.OPENAI_API_KEY as OpenAI API key refactor(index.js): comment out askOpenAI route feat(index.js): add openAI route feat(openAI.js): add new route for OpenAI API requests with support for progress updates and aborting requests. * refactor(BaseClient.js): use optional chaining operator to access messageId property refactor(OpenAIClient.js): use orderedMessages instead of messages to build prompt refactor(OpenAIClient.js): use optional chaining operator to access messageId property refactor(fetch-polyfill.js): remove fetch polyfill refactor(openAI.js): comment out debug option in clientOptions * refactor: update import statements and remove unused imports in several files feat: add getAzureCredentials function to azureUtils module docs: update comments in azureUtils module * refactor(utils): rename migrateConversations to migrateDataToFirstUser for clarity and consistency * feat(chatgpt-client.js): add getAzureCredentials function to retrieve Azure credentials feat(chatgpt-client.js): use getAzureCredentials function to generate reverseProxyUrl feat(OpenAIClient.js): add isChatCompletion property to determine if chat completion model is used feat(OpenAIClient.js): add saveOptions parameter to sendMessage and buildPrompt methods feat(OpenAIClient.js): modify buildPrompt method to handle chat completion model feat(openAI.js): modify endpointOption to include modelOptions instead of individual options refactor(OpenAIClient.js): modify getDelta property to use isChatCompletion property instead of isChatGptModel property refactor(OpenAIClient.js): modify sendMessage method to use saveOptions parameter instead of modelOptions parameter refactor(OpenAIClient.js): modify buildPrompt method to use saveOptions parameter instead of modelOptions parameter refactor(OpenAIClient.js): modify ask method to include endpointOption parameter * chore: delete draft file * refactor(OpenAIClient.js): extract sendCompletion method from sendMessage method for reusability * refactor(BaseClient.js): move sendMessage method to BaseClient class feat(OpenAIClient.js): inherit from BaseClient class and implement necessary methods and properties for OpenAIClient class. * refactor(BaseClient.js): rename getBuildPromptOptions to getBuildMessagesOptions feat(BaseClient.js): add buildMessages method to BaseClient class fix(ChatGPTClient.js): use message.text instead of message.message refactor(ChatGPTClient.js): rename buildPromptBody to buildMessagesBody refactor(ChatGPTClient.js): remove console.debug statement and add debug log for prompt variable refactor(OpenAIClient.js): move setOptions method to the bottom of the class feat(OpenAIClient.js): add support for cl100k_base encoding feat(OpenAIClient.js): add support for unofficial chat GPT models feat(OpenAIClient.js): add support for custom modelOptions feat(OpenAIClient.js): add caching for tokenizers feat(OpenAIClient.js): add freeAndInitializeEncoder method to free and reinitialize tokenizers refactor(OpenAIClient.js): rename getBuildPromptOptions to getBuildMessagesOptions refactor(OpenAIClient.js): rename buildPrompt to buildMessages refactor(OpenAIClient.js): remove endpointOption from ask function arguments in openAI.js * refactor(ChatGPTClient.js, OpenAIClient.js): improve code readability and consistency - In ChatGPTClient.js, update the roleLabel and messageString variables to handle cases where the message object does not have an isCreatedByUser property or a role property with a value of 'user'. - In OpenAIClient.js, rename the freeAndInitializeEncoder method to freeAndResetEncoder to better reflect its functionality. Also, update the method calls to reflect the new name. Additionally, update the getTokenCount method to handle errors by calling the freeAndResetEncoder method instead of the now-renamed freeAndInitializeEncoder method. * refactor(OpenAIClient.js): extract instructions object to a separate variable and add it to payload after formatted messages fix(OpenAIClient.js): handle cases where progressMessage.choices is undefined or empty * refactor(BaseClient.js): extract addInstructions method from sendMessage method feat(OpenAIClient.js): add maxTokensMap object to map maximum tokens for each model refactor(OpenAIClient.js): use addInstructions method in buildMessages method instead of manually building the payload list * refactor(OpenAIClient.js): remove unnecessary condition for modelOptions.model property in buildMessages method * feat(BaseClient.js): add support for token count tracking and context strategy feat(OpenAIClient.js): add support for token count tracking and context strategy feat(Message.js): add tokenCount field to Message schema and updateMessage function * refactor(BaseClient.js): add support for refining messages based on token limit feat(OpenAIClient.js): add support for context refinement strategy refactor(OpenAIClient.js): use context refinement strategy in message sending refactor(server/index.js): improve code readability by breaking long lines * refactor(BaseClient.js): change `remainingContext` to `remainingContextTokens` for clarity feat(BaseClient.js): add `refinePrompt` and `refinePromptTemplate` to handle message refinement feat(BaseClient.js): add `refineMessages` method to refine messages feat(BaseClient.js): add `handleContextStrategy` method to handle context strategy feat(OpenAIClient.js): add `abortController` to `buildPrompt` method options refactor(OpenAIClient.js): change `payload` and `tokenCountMap` to let variables in `handleContextStrategy` method refactor(BaseClient.js): change `remainingContext` to `remainingContextTokens` in `handleContextStrategy` method for consistency refactor(BaseClient.js): change `remainingContext` to `remainingContextTokens` in `getMessagesWithinTokenLimit` method for consistency refactor(BaseClient.js): change `remainingContext` to `remainingContext * chore(openAI.js): comment out contextStrategy option in clientOptions * chore(openAI.js): comment out debug option in clientOptions object * test: BaseClient tests in progress * test: Complete OpenAIClient & BaseClient tests * fix(OpenAIClient.js): remove unnecessary whitespace fix(OpenAIClient.js): remove unused variables and comments fix(OpenAIClient.test.js): combine getTokenCount and freeAndResetEncoder tests * chore(.eslintrc.js): add rule for maximum of 1 empty line feat(ask/openAI.js): add abortMessage utility function fix(ask/openAI.js): handle error and abort message if partial text is less than 2 characters feat(utils/index.js): export abortMessage utility function * test: complete additional tests * feat: Azure OpenAI as a separate endpoint * chore: remove extraneous console logs * fix(azureOpenAI): use chatCompletion endpoint * chore(initializeClient.js): delete initializeClient.js file chore(askOpenAI.js): delete old OpenAI route handler chore(handlers.js): remove trailing whitespace in thought variable assignment * chore(chatgpt-client.js): remove unused chatgpt-client.js file refactor(index.js): remove askClient import and export from index.js * chore(chatgpt-client.tokens.js): update test script for memory usage and encoding performance The test script in `chatgpt-client.tokens.js` has been updated to measure the memory usage and encoding performance of the client. The script now includes information about the initial memory usage, peak memory usage, final memory usage, and memory usage after a timeout. It also provides insights into the number of encoding requests that can be processed per second. The script has been modified to use the `OpenAIClient` class instead of the `ChatGPTClient` class. Additionally, the number of iterations for the encoding loop has been reduced to 10,000. A timeout function has been added to simulate a delay of 15 seconds. After the timeout, the memory usage is measured again. The script now handles uncaught exceptions and logs any errors that occur, except for errors related to failed fetch requests. Note: This is a test script and should not be used in production * feat(FakeClient.js): add a new class `FakeClient` that extends `BaseClient` and implements methods for a fake client feat(FakeClient.js): implement the `setOptions` method to handle options for the fake client feat(FakeClient.js): implement the `initializeFakeClient` function to initialize a fake client with options and fake messages fix(OpenAIClient.js): remove duplicate `maxTokensMap` import and use the one from utils feat(BaseClient): return promptTokens and completionTokens * refactor(gptPlugins): refactor ChatAgent to PluginsClient, which extends OpenAIClient * refactor: client paths * chore(jest.config.js): remove jest.config.js file * fix(PluginController.js): update file path to manifest.json feat(gptPlugins.js): add support for aborting messages refactor(ask/index.js): rename askGPTPlugins to gptPlugins for consistency * fix(BaseClient.js): fix spacing in generateTextStream function signature refactor(BaseClient.js): remove unnecessary push to currentMessages in generateUserMessage function refactor(BaseClient.js): remove unnecessary push to currentMessages in handleStartMethods function refactor(PluginsClient.js): remove unused variables and date formatting in constructor refactor(PluginsClient.js): simplify mapping of pastMessages in getCompletionPayload function * refactor(GoogleClient): GoogleClient now extends BaseClient * chore(.env.example): add AZURE_OPENAI_MODELS variable fix(api/routes/ask/gptPlugins.js): enable Azure integration if PLUGINS_USE_AZURE is true fix(api/routes/endpoints.js): getOpenAIModels function now accepts options, use AZURE_OPENAI_MODELS if PLUGINS_USE_AZURE is true fix(client/components/Endpoints/OpenAI/Settings.jsx): remove console.log statement docs(features/azure.md): add documentation for Azure OpenAI integration and environment variables * fix(e2e:popup): includes the icon + endpoint names in role, name property
2023-07-03 16:51:12 -04:00
const google = require('./google');
const bingAI = require('./bingAI');
feat(Google): Support all Text/Chat Models, Response streaming, `PaLM` -> `Google` 🤖 (#1316) * feat: update PaLM icons * feat: add additional google models * POC: formatting inputs for Vertex AI streaming * refactor: move endpoints services outside of /routes dir to /services/Endpoints * refactor: shorten schemas import * refactor: rename PALM to GOOGLE * feat: make Google editable endpoint * feat: reusable Ask and Edit controllers based off Anthropic * chore: organize imports/logic * fix(parseConvo): include examples in googleSchema * fix: google only allows odd number of messages to be sent * fix: pass proxy to AnthropicClient * refactor: change `google` altName to `Google` * refactor: update getModelMaxTokens and related functions to handle maxTokensMap with nested endpoint model key/values * refactor: google Icon and response sender changes (Codey and Google logo instead of PaLM in all cases) * feat: google support for maxTokensMap * feat: google updated endpoints with Ask/Edit controllers, buildOptions, and initializeClient * feat(GoogleClient): now builds prompt for text models and supports real streaming from Vertex AI through langchain * chore(GoogleClient): remove comments, left before for reference in git history * docs: update google instructions (WIP) * docs(apis_and_tokens.md): add images to google instructions * docs: remove typo apis_and_tokens.md * Update apis_and_tokens.md * feat(Google): use default settings map, fully support context for both text and chat models, fully support examples for chat models * chore: update more PaLM references to Google * chore: move playwright out of workflows to avoid failing tests
2023-12-10 14:54:13 -05:00
const anthropic = require('./anthropic');
refactor: Client Classes & Azure OpenAI as a separate Endpoint (#532) * refactor: start new client classes, test localAi support * feat: create base class, extend chatgpt from base * refactor(BaseClient.js): change userId parameter to user refactor(BaseClient.js): change userId parameter to user feat(OpenAIClient.js): add sendMessage method refactor(OpenAIClient.js): change getConversation method to use user parameter instead of userId refactor(OpenAIClient.js): change saveMessageToDatabase method to use user parameter instead of userId refactor(OpenAIClient.js): change buildPrompt method to use messages parameter instead of orderedMessages feat(index.js): export client classes refactor(askGPTPlugins.js): use req.body.token or process.env.OPENAI_API_KEY as OpenAI API key refactor(index.js): comment out askOpenAI route feat(index.js): add openAI route feat(openAI.js): add new route for OpenAI API requests with support for progress updates and aborting requests. * refactor(BaseClient.js): use optional chaining operator to access messageId property refactor(OpenAIClient.js): use orderedMessages instead of messages to build prompt refactor(OpenAIClient.js): use optional chaining operator to access messageId property refactor(fetch-polyfill.js): remove fetch polyfill refactor(openAI.js): comment out debug option in clientOptions * refactor: update import statements and remove unused imports in several files feat: add getAzureCredentials function to azureUtils module docs: update comments in azureUtils module * refactor(utils): rename migrateConversations to migrateDataToFirstUser for clarity and consistency * feat(chatgpt-client.js): add getAzureCredentials function to retrieve Azure credentials feat(chatgpt-client.js): use getAzureCredentials function to generate reverseProxyUrl feat(OpenAIClient.js): add isChatCompletion property to determine if chat completion model is used feat(OpenAIClient.js): add saveOptions parameter to sendMessage and buildPrompt methods feat(OpenAIClient.js): modify buildPrompt method to handle chat completion model feat(openAI.js): modify endpointOption to include modelOptions instead of individual options refactor(OpenAIClient.js): modify getDelta property to use isChatCompletion property instead of isChatGptModel property refactor(OpenAIClient.js): modify sendMessage method to use saveOptions parameter instead of modelOptions parameter refactor(OpenAIClient.js): modify buildPrompt method to use saveOptions parameter instead of modelOptions parameter refactor(OpenAIClient.js): modify ask method to include endpointOption parameter * chore: delete draft file * refactor(OpenAIClient.js): extract sendCompletion method from sendMessage method for reusability * refactor(BaseClient.js): move sendMessage method to BaseClient class feat(OpenAIClient.js): inherit from BaseClient class and implement necessary methods and properties for OpenAIClient class. * refactor(BaseClient.js): rename getBuildPromptOptions to getBuildMessagesOptions feat(BaseClient.js): add buildMessages method to BaseClient class fix(ChatGPTClient.js): use message.text instead of message.message refactor(ChatGPTClient.js): rename buildPromptBody to buildMessagesBody refactor(ChatGPTClient.js): remove console.debug statement and add debug log for prompt variable refactor(OpenAIClient.js): move setOptions method to the bottom of the class feat(OpenAIClient.js): add support for cl100k_base encoding feat(OpenAIClient.js): add support for unofficial chat GPT models feat(OpenAIClient.js): add support for custom modelOptions feat(OpenAIClient.js): add caching for tokenizers feat(OpenAIClient.js): add freeAndInitializeEncoder method to free and reinitialize tokenizers refactor(OpenAIClient.js): rename getBuildPromptOptions to getBuildMessagesOptions refactor(OpenAIClient.js): rename buildPrompt to buildMessages refactor(OpenAIClient.js): remove endpointOption from ask function arguments in openAI.js * refactor(ChatGPTClient.js, OpenAIClient.js): improve code readability and consistency - In ChatGPTClient.js, update the roleLabel and messageString variables to handle cases where the message object does not have an isCreatedByUser property or a role property with a value of 'user'. - In OpenAIClient.js, rename the freeAndInitializeEncoder method to freeAndResetEncoder to better reflect its functionality. Also, update the method calls to reflect the new name. Additionally, update the getTokenCount method to handle errors by calling the freeAndResetEncoder method instead of the now-renamed freeAndInitializeEncoder method. * refactor(OpenAIClient.js): extract instructions object to a separate variable and add it to payload after formatted messages fix(OpenAIClient.js): handle cases where progressMessage.choices is undefined or empty * refactor(BaseClient.js): extract addInstructions method from sendMessage method feat(OpenAIClient.js): add maxTokensMap object to map maximum tokens for each model refactor(OpenAIClient.js): use addInstructions method in buildMessages method instead of manually building the payload list * refactor(OpenAIClient.js): remove unnecessary condition for modelOptions.model property in buildMessages method * feat(BaseClient.js): add support for token count tracking and context strategy feat(OpenAIClient.js): add support for token count tracking and context strategy feat(Message.js): add tokenCount field to Message schema and updateMessage function * refactor(BaseClient.js): add support for refining messages based on token limit feat(OpenAIClient.js): add support for context refinement strategy refactor(OpenAIClient.js): use context refinement strategy in message sending refactor(server/index.js): improve code readability by breaking long lines * refactor(BaseClient.js): change `remainingContext` to `remainingContextTokens` for clarity feat(BaseClient.js): add `refinePrompt` and `refinePromptTemplate` to handle message refinement feat(BaseClient.js): add `refineMessages` method to refine messages feat(BaseClient.js): add `handleContextStrategy` method to handle context strategy feat(OpenAIClient.js): add `abortController` to `buildPrompt` method options refactor(OpenAIClient.js): change `payload` and `tokenCountMap` to let variables in `handleContextStrategy` method refactor(BaseClient.js): change `remainingContext` to `remainingContextTokens` in `handleContextStrategy` method for consistency refactor(BaseClient.js): change `remainingContext` to `remainingContextTokens` in `getMessagesWithinTokenLimit` method for consistency refactor(BaseClient.js): change `remainingContext` to `remainingContext * chore(openAI.js): comment out contextStrategy option in clientOptions * chore(openAI.js): comment out debug option in clientOptions object * test: BaseClient tests in progress * test: Complete OpenAIClient & BaseClient tests * fix(OpenAIClient.js): remove unnecessary whitespace fix(OpenAIClient.js): remove unused variables and comments fix(OpenAIClient.test.js): combine getTokenCount and freeAndResetEncoder tests * chore(.eslintrc.js): add rule for maximum of 1 empty line feat(ask/openAI.js): add abortMessage utility function fix(ask/openAI.js): handle error and abort message if partial text is less than 2 characters feat(utils/index.js): export abortMessage utility function * test: complete additional tests * feat: Azure OpenAI as a separate endpoint * chore: remove extraneous console logs * fix(azureOpenAI): use chatCompletion endpoint * chore(initializeClient.js): delete initializeClient.js file chore(askOpenAI.js): delete old OpenAI route handler chore(handlers.js): remove trailing whitespace in thought variable assignment * chore(chatgpt-client.js): remove unused chatgpt-client.js file refactor(index.js): remove askClient import and export from index.js * chore(chatgpt-client.tokens.js): update test script for memory usage and encoding performance The test script in `chatgpt-client.tokens.js` has been updated to measure the memory usage and encoding performance of the client. The script now includes information about the initial memory usage, peak memory usage, final memory usage, and memory usage after a timeout. It also provides insights into the number of encoding requests that can be processed per second. The script has been modified to use the `OpenAIClient` class instead of the `ChatGPTClient` class. Additionally, the number of iterations for the encoding loop has been reduced to 10,000. A timeout function has been added to simulate a delay of 15 seconds. After the timeout, the memory usage is measured again. The script now handles uncaught exceptions and logs any errors that occur, except for errors related to failed fetch requests. Note: This is a test script and should not be used in production * feat(FakeClient.js): add a new class `FakeClient` that extends `BaseClient` and implements methods for a fake client feat(FakeClient.js): implement the `setOptions` method to handle options for the fake client feat(FakeClient.js): implement the `initializeFakeClient` function to initialize a fake client with options and fake messages fix(OpenAIClient.js): remove duplicate `maxTokensMap` import and use the one from utils feat(BaseClient): return promptTokens and completionTokens * refactor(gptPlugins): refactor ChatAgent to PluginsClient, which extends OpenAIClient * refactor: client paths * chore(jest.config.js): remove jest.config.js file * fix(PluginController.js): update file path to manifest.json feat(gptPlugins.js): add support for aborting messages refactor(ask/index.js): rename askGPTPlugins to gptPlugins for consistency * fix(BaseClient.js): fix spacing in generateTextStream function signature refactor(BaseClient.js): remove unnecessary push to currentMessages in generateUserMessage function refactor(BaseClient.js): remove unnecessary push to currentMessages in handleStartMethods function refactor(PluginsClient.js): remove unused variables and date formatting in constructor refactor(PluginsClient.js): simplify mapping of pastMessages in getCompletionPayload function * refactor(GoogleClient): GoogleClient now extends BaseClient * chore(.env.example): add AZURE_OPENAI_MODELS variable fix(api/routes/ask/gptPlugins.js): enable Azure integration if PLUGINS_USE_AZURE is true fix(api/routes/endpoints.js): getOpenAIModels function now accepts options, use AZURE_OPENAI_MODELS if PLUGINS_USE_AZURE is true fix(client/components/Endpoints/OpenAI/Settings.jsx): remove console.log statement docs(features/azure.md): add documentation for Azure OpenAI integration and environment variables * fix(e2e:popup): includes the icon + endpoint names in role, name property
2023-07-03 16:51:12 -04:00
const gptPlugins = require('./gptPlugins');
2023-03-31 03:22:57 +08:00
const askChatGPTBrowser = require('./askChatGPTBrowser');
feat(Google): Support all Text/Chat Models, Response streaming, `PaLM` -> `Google` 🤖 (#1316) * feat: update PaLM icons * feat: add additional google models * POC: formatting inputs for Vertex AI streaming * refactor: move endpoints services outside of /routes dir to /services/Endpoints * refactor: shorten schemas import * refactor: rename PALM to GOOGLE * feat: make Google editable endpoint * feat: reusable Ask and Edit controllers based off Anthropic * chore: organize imports/logic * fix(parseConvo): include examples in googleSchema * fix: google only allows odd number of messages to be sent * fix: pass proxy to AnthropicClient * refactor: change `google` altName to `Google` * refactor: update getModelMaxTokens and related functions to handle maxTokensMap with nested endpoint model key/values * refactor: google Icon and response sender changes (Codey and Google logo instead of PaLM in all cases) * feat: google support for maxTokensMap * feat: google updated endpoints with Ask/Edit controllers, buildOptions, and initializeClient * feat(GoogleClient): now builds prompt for text models and supports real streaming from Vertex AI through langchain * chore(GoogleClient): remove comments, left before for reference in git history * docs: update google instructions (WIP) * docs(apis_and_tokens.md): add images to google instructions * docs: remove typo apis_and_tokens.md * Update apis_and_tokens.md * feat(Google): use default settings map, fully support context for both text and chat models, fully support examples for chat models * chore: update more PaLM references to Google * chore: move playwright out of workflows to avoid failing tests
2023-12-10 14:54:13 -05:00
const { isEnabled } = require('~/server/utils');
const { EModelEndpoint } = require('librechat-data-provider');
feat: Message Rate Limiters, Violation Logging, & Ban System 🔨 (#903) * refactor: require Auth middleware in route index files * feat: concurrent message limiter * feat: complete concurrent message limiter with caching * refactor: SSE response methods separated from handleText * fix(abortMiddleware): fix req and res order to standard, use endpointOption in req.body * chore: minor name changes * refactor: add isUUID condition to saveMessage * fix(concurrentLimiter): logic correctly handles the max number of concurrent messages and res closing/finalization * chore: bump keyv and remove console.log from Message * fix(concurrentLimiter): ensure messages are only saved in later message children * refactor(concurrentLimiter): use KeyvFile instead, could make other stores configurable in the future * feat: add denyRequest function for error responses * feat(utils): add isStringTruthy function Introduce the isStringTruthy function to the utilities module to check if a string value is a case-insensitive match for 'true' * feat: add optional message rate limiters by IP and userId * feat: add optional message rate limiters by IP and userId to edit route * refactor: rename isStringTruthy to isTrue for brevity * refactor(getError): use map to make code cleaner * refactor: use memory for concurrent rate limiter to prevent clearing on startup/exit, add multiple log files, fix error message for concurrent violation * feat: check if errorMessage is object, stringify if so * chore: send object to denyRequest which will stringify it * feat: log excessive requests * fix(getError): correctly pluralize messages * refactor(limiters): make type consistent between logs and errorMessage * refactor(cache): move files out of lib/db into separate cache dir >> feat: add getLogStores function so Keyv instance is not redundantly created on every violation feat: separate violation logging to own function with logViolation * fix: cache/index.js export, properly record userViolations * refactor(messageLimiters): use new logging method, add logging to registrations * refactor(logViolation): make userLogs an array of logs per user * feat: add logging to login limiter * refactor: pass req as first param to logViolation and record offending IP * refactor: rename isTrue helper fn to isEnabled * feat: add simple non_browser check and log violation * fix: open handles in unit tests, remove KeyvMongo as not used and properly mock global fetch * chore: adjust nodemon ignore paths to properly ignore logs * feat: add math helper function for safe use of eval * refactor(api/convos): use middleware at top of file to avoid redundancy * feat: add delete all static method for Sessions * fix: redirect to login on refresh if user is not found, or the session is not found but hasn't expired (ban case) * refactor(getLogStores): adjust return type * feat: add ban violation and check ban logic refactor(logViolation): pass both req and res objects * feat: add removePorts helper function * refactor: rename getError to getMessageError and add getLoginError for displaying different login errors * fix(AuthContext): fix type issue and remove unused code * refactor(bans): ban by ip and user id, send response based on origin * chore: add frontend ban messages * refactor(routes/oauth): add ban check to handler, also consolidate logic to avoid redundancy * feat: add ban check to AI messaging routes * feat: add ban check to login/registration * fix(ci/api): mock KeyvMongo to avoid tests hanging * docs: update .env.example > refactor(banViolation): calculate interval rate crossover, early return if duration is invalid ci(banViolation): add tests to ensure users are only banned when expected * docs: improve wording for mod system * feat: add configurable env variables for violation scores * chore: add jsdoc for uaParser.js * chore: improve ban text log * chore: update bun test scripts * refactor(math.js): add fallback values * fix(KeyvMongo/banLogs): refactor keyv instances to top of files to avoid memory leaks, refactor ban logic to use getLogStores instead refactor(getLogStores): get a single log store by type * fix(ci): refactor tests due to banLogs changes, also make sure to clear and revoke sessions even if ban duration is 0 * fix(banViolation.js): getLogStores import * feat: handle 500 code error at login * fix(middleware): handle case where user.id is _id and not just id * ci: add ban secrets for backend unit tests * refactor: logout user upon ban * chore: log session delete message only if deletedCount > 0 * refactor: change default ban duration (2h) and make logic more clear in JSDOC * fix: login and registration limiters will now return rate limiting error * fix: userId not parsable as non ObjectId string * feat: add useTimeout hook to properly clear timeouts when invoking functions within them refactor(AuthContext): cleanup code by using new hook and defining types in ~/common * fix: login error message for rate limits * docs: add info for automated mod system and rate limiters, update other docs accordingly * chore: bump data-provider version
2023-09-13 10:57:07 -04:00
const {
uaParser,
checkBan,
requireJwtAuth,
concurrentLimiter,
messageIpLimiter,
messageUserLimiter,
feat(Google): Support all Text/Chat Models, Response streaming, `PaLM` -> `Google` 🤖 (#1316) * feat: update PaLM icons * feat: add additional google models * POC: formatting inputs for Vertex AI streaming * refactor: move endpoints services outside of /routes dir to /services/Endpoints * refactor: shorten schemas import * refactor: rename PALM to GOOGLE * feat: make Google editable endpoint * feat: reusable Ask and Edit controllers based off Anthropic * chore: organize imports/logic * fix(parseConvo): include examples in googleSchema * fix: google only allows odd number of messages to be sent * fix: pass proxy to AnthropicClient * refactor: change `google` altName to `Google` * refactor: update getModelMaxTokens and related functions to handle maxTokensMap with nested endpoint model key/values * refactor: google Icon and response sender changes (Codey and Google logo instead of PaLM in all cases) * feat: google support for maxTokensMap * feat: google updated endpoints with Ask/Edit controllers, buildOptions, and initializeClient * feat(GoogleClient): now builds prompt for text models and supports real streaming from Vertex AI through langchain * chore(GoogleClient): remove comments, left before for reference in git history * docs: update google instructions (WIP) * docs(apis_and_tokens.md): add images to google instructions * docs: remove typo apis_and_tokens.md * Update apis_and_tokens.md * feat(Google): use default settings map, fully support context for both text and chat models, fully support examples for chat models * chore: update more PaLM references to Google * chore: move playwright out of workflows to avoid failing tests
2023-12-10 14:54:13 -05:00
} = require('~/server/middleware');
feat: Message Rate Limiters, Violation Logging, & Ban System 🔨 (#903) * refactor: require Auth middleware in route index files * feat: concurrent message limiter * feat: complete concurrent message limiter with caching * refactor: SSE response methods separated from handleText * fix(abortMiddleware): fix req and res order to standard, use endpointOption in req.body * chore: minor name changes * refactor: add isUUID condition to saveMessage * fix(concurrentLimiter): logic correctly handles the max number of concurrent messages and res closing/finalization * chore: bump keyv and remove console.log from Message * fix(concurrentLimiter): ensure messages are only saved in later message children * refactor(concurrentLimiter): use KeyvFile instead, could make other stores configurable in the future * feat: add denyRequest function for error responses * feat(utils): add isStringTruthy function Introduce the isStringTruthy function to the utilities module to check if a string value is a case-insensitive match for 'true' * feat: add optional message rate limiters by IP and userId * feat: add optional message rate limiters by IP and userId to edit route * refactor: rename isStringTruthy to isTrue for brevity * refactor(getError): use map to make code cleaner * refactor: use memory for concurrent rate limiter to prevent clearing on startup/exit, add multiple log files, fix error message for concurrent violation * feat: check if errorMessage is object, stringify if so * chore: send object to denyRequest which will stringify it * feat: log excessive requests * fix(getError): correctly pluralize messages * refactor(limiters): make type consistent between logs and errorMessage * refactor(cache): move files out of lib/db into separate cache dir >> feat: add getLogStores function so Keyv instance is not redundantly created on every violation feat: separate violation logging to own function with logViolation * fix: cache/index.js export, properly record userViolations * refactor(messageLimiters): use new logging method, add logging to registrations * refactor(logViolation): make userLogs an array of logs per user * feat: add logging to login limiter * refactor: pass req as first param to logViolation and record offending IP * refactor: rename isTrue helper fn to isEnabled * feat: add simple non_browser check and log violation * fix: open handles in unit tests, remove KeyvMongo as not used and properly mock global fetch * chore: adjust nodemon ignore paths to properly ignore logs * feat: add math helper function for safe use of eval * refactor(api/convos): use middleware at top of file to avoid redundancy * feat: add delete all static method for Sessions * fix: redirect to login on refresh if user is not found, or the session is not found but hasn't expired (ban case) * refactor(getLogStores): adjust return type * feat: add ban violation and check ban logic refactor(logViolation): pass both req and res objects * feat: add removePorts helper function * refactor: rename getError to getMessageError and add getLoginError for displaying different login errors * fix(AuthContext): fix type issue and remove unused code * refactor(bans): ban by ip and user id, send response based on origin * chore: add frontend ban messages * refactor(routes/oauth): add ban check to handler, also consolidate logic to avoid redundancy * feat: add ban check to AI messaging routes * feat: add ban check to login/registration * fix(ci/api): mock KeyvMongo to avoid tests hanging * docs: update .env.example > refactor(banViolation): calculate interval rate crossover, early return if duration is invalid ci(banViolation): add tests to ensure users are only banned when expected * docs: improve wording for mod system * feat: add configurable env variables for violation scores * chore: add jsdoc for uaParser.js * chore: improve ban text log * chore: update bun test scripts * refactor(math.js): add fallback values * fix(KeyvMongo/banLogs): refactor keyv instances to top of files to avoid memory leaks, refactor ban logic to use getLogStores instead refactor(getLogStores): get a single log store by type * fix(ci): refactor tests due to banLogs changes, also make sure to clear and revoke sessions even if ban duration is 0 * fix(banViolation.js): getLogStores import * feat: handle 500 code error at login * fix(middleware): handle case where user.id is _id and not just id * ci: add ban secrets for backend unit tests * refactor: logout user upon ban * chore: log session delete message only if deletedCount > 0 * refactor: change default ban duration (2h) and make logic more clear in JSDOC * fix: login and registration limiters will now return rate limiting error * fix: userId not parsable as non ObjectId string * feat: add useTimeout hook to properly clear timeouts when invoking functions within them refactor(AuthContext): cleanup code by using new hook and defining types in ~/common * fix: login error message for rate limits * docs: add info for automated mod system and rate limiters, update other docs accordingly * chore: bump data-provider version
2023-09-13 10:57:07 -04:00
const { LIMIT_CONCURRENT_MESSAGES, LIMIT_MESSAGE_IP, LIMIT_MESSAGE_USER } = process.env ?? {};
feat(Google): Support all Text/Chat Models, Response streaming, `PaLM` -> `Google` 🤖 (#1316) * feat: update PaLM icons * feat: add additional google models * POC: formatting inputs for Vertex AI streaming * refactor: move endpoints services outside of /routes dir to /services/Endpoints * refactor: shorten schemas import * refactor: rename PALM to GOOGLE * feat: make Google editable endpoint * feat: reusable Ask and Edit controllers based off Anthropic * chore: organize imports/logic * fix(parseConvo): include examples in googleSchema * fix: google only allows odd number of messages to be sent * fix: pass proxy to AnthropicClient * refactor: change `google` altName to `Google` * refactor: update getModelMaxTokens and related functions to handle maxTokensMap with nested endpoint model key/values * refactor: google Icon and response sender changes (Codey and Google logo instead of PaLM in all cases) * feat: google support for maxTokensMap * feat: google updated endpoints with Ask/Edit controllers, buildOptions, and initializeClient * feat(GoogleClient): now builds prompt for text models and supports real streaming from Vertex AI through langchain * chore(GoogleClient): remove comments, left before for reference in git history * docs: update google instructions (WIP) * docs(apis_and_tokens.md): add images to google instructions * docs: remove typo apis_and_tokens.md * Update apis_and_tokens.md * feat(Google): use default settings map, fully support context for both text and chat models, fully support examples for chat models * chore: update more PaLM references to Google * chore: move playwright out of workflows to avoid failing tests
2023-12-10 14:54:13 -05:00
const router = express.Router();
feat: Message Rate Limiters, Violation Logging, & Ban System 🔨 (#903) * refactor: require Auth middleware in route index files * feat: concurrent message limiter * feat: complete concurrent message limiter with caching * refactor: SSE response methods separated from handleText * fix(abortMiddleware): fix req and res order to standard, use endpointOption in req.body * chore: minor name changes * refactor: add isUUID condition to saveMessage * fix(concurrentLimiter): logic correctly handles the max number of concurrent messages and res closing/finalization * chore: bump keyv and remove console.log from Message * fix(concurrentLimiter): ensure messages are only saved in later message children * refactor(concurrentLimiter): use KeyvFile instead, could make other stores configurable in the future * feat: add denyRequest function for error responses * feat(utils): add isStringTruthy function Introduce the isStringTruthy function to the utilities module to check if a string value is a case-insensitive match for 'true' * feat: add optional message rate limiters by IP and userId * feat: add optional message rate limiters by IP and userId to edit route * refactor: rename isStringTruthy to isTrue for brevity * refactor(getError): use map to make code cleaner * refactor: use memory for concurrent rate limiter to prevent clearing on startup/exit, add multiple log files, fix error message for concurrent violation * feat: check if errorMessage is object, stringify if so * chore: send object to denyRequest which will stringify it * feat: log excessive requests * fix(getError): correctly pluralize messages * refactor(limiters): make type consistent between logs and errorMessage * refactor(cache): move files out of lib/db into separate cache dir >> feat: add getLogStores function so Keyv instance is not redundantly created on every violation feat: separate violation logging to own function with logViolation * fix: cache/index.js export, properly record userViolations * refactor(messageLimiters): use new logging method, add logging to registrations * refactor(logViolation): make userLogs an array of logs per user * feat: add logging to login limiter * refactor: pass req as first param to logViolation and record offending IP * refactor: rename isTrue helper fn to isEnabled * feat: add simple non_browser check and log violation * fix: open handles in unit tests, remove KeyvMongo as not used and properly mock global fetch * chore: adjust nodemon ignore paths to properly ignore logs * feat: add math helper function for safe use of eval * refactor(api/convos): use middleware at top of file to avoid redundancy * feat: add delete all static method for Sessions * fix: redirect to login on refresh if user is not found, or the session is not found but hasn't expired (ban case) * refactor(getLogStores): adjust return type * feat: add ban violation and check ban logic refactor(logViolation): pass both req and res objects * feat: add removePorts helper function * refactor: rename getError to getMessageError and add getLoginError for displaying different login errors * fix(AuthContext): fix type issue and remove unused code * refactor(bans): ban by ip and user id, send response based on origin * chore: add frontend ban messages * refactor(routes/oauth): add ban check to handler, also consolidate logic to avoid redundancy * feat: add ban check to AI messaging routes * feat: add ban check to login/registration * fix(ci/api): mock KeyvMongo to avoid tests hanging * docs: update .env.example > refactor(banViolation): calculate interval rate crossover, early return if duration is invalid ci(banViolation): add tests to ensure users are only banned when expected * docs: improve wording for mod system * feat: add configurable env variables for violation scores * chore: add jsdoc for uaParser.js * chore: improve ban text log * chore: update bun test scripts * refactor(math.js): add fallback values * fix(KeyvMongo/banLogs): refactor keyv instances to top of files to avoid memory leaks, refactor ban logic to use getLogStores instead refactor(getLogStores): get a single log store by type * fix(ci): refactor tests due to banLogs changes, also make sure to clear and revoke sessions even if ban duration is 0 * fix(banViolation.js): getLogStores import * feat: handle 500 code error at login * fix(middleware): handle case where user.id is _id and not just id * ci: add ban secrets for backend unit tests * refactor: logout user upon ban * chore: log session delete message only if deletedCount > 0 * refactor: change default ban duration (2h) and make logic more clear in JSDOC * fix: login and registration limiters will now return rate limiting error * fix: userId not parsable as non ObjectId string * feat: add useTimeout hook to properly clear timeouts when invoking functions within them refactor(AuthContext): cleanup code by using new hook and defining types in ~/common * fix: login error message for rate limits * docs: add info for automated mod system and rate limiters, update other docs accordingly * chore: bump data-provider version
2023-09-13 10:57:07 -04:00
router.use(requireJwtAuth);
router.use(checkBan);
router.use(uaParser);
if (isEnabled(LIMIT_CONCURRENT_MESSAGES)) {
router.use(concurrentLimiter);
}
if (isEnabled(LIMIT_MESSAGE_IP)) {
router.use(messageIpLimiter);
}
if (isEnabled(LIMIT_MESSAGE_USER)) {
router.use(messageUserLimiter);
}
2023-03-31 03:22:57 +08:00
WIP: Update UI to match Official Style; Vision and Assistants 👷🏽 (#1190) * wip: initial client side code * wip: initial api code * refactor: export query keys from own module, export assistant hooks * refactor(SelectDropDown): more customization via props * feat: create Assistant and render real Assistants * refactor: major refactor of UI components to allow multi-chat, working alongside CreationPanel * refactor: move assistant routes to own directory * fix(CreationHeader): state issue with assistant select * refactor: style changes for form, fix setSiblingIdx from useChatHelpers to use latestMessageParentId, fix render issue with ChatView and change location * feat: parseCompactConvo: begin refactor of slimmer JSON payloads between client/api * refactor(endpoints): add assistant endpoint, also use EModelEndpoint as much as possible * refactor(useGetConversationsQuery): use object to access query data easily * fix(MultiMessage): react warning of bad state set, making use of effect during render (instead of useEffect) * fix(useNewConvo): use correct atom key (index instead of convoId) for reset latestMessageFamily * refactor: make routing navigation/conversation change simpler * chore: add removeNullishValues for smaller payloads, remove unused fields, setup frontend pinging of assistant endpoint * WIP: initial complete assistant run handling * fix: CreationPanel form correctly setting internal state * refactor(api/assistants/chat): revise functions to working run handling strategy * refactor(UI): initial major refactor of ChatForm and options * feat: textarea hook * refactor: useAuthRedirect hook and change directory name * feat: add ChatRoute (/c/), make optionsBar absolute and change on textarea height, add temp header * feat: match new toggle Nav open button to ChatGPT's * feat: add OpenAI custom classnames * feat: useOriginNavigate * feat: messages loading view * fix: conversation navigation and effects * refactor: make toggle change nav opacity * WIP: new endpoint menu * feat: NewEndpointsMenu complete * fix: ensure set key dialog shows on endpoint change, and new conversation resets messages * WIP: textarea styling fix, add temp footer, create basic file handling component * feat: image file handling (UI) * feat: PopOver and ModelSelect in Header, remove GenButtons * feat: drop file handling * refactor: bug fixes use SSE at route level add opts to useOriginNavigate delay render of unfinishedMessage to avoid flickering pass params (convoId) to chatHelpers to set messages query data based on param when the route is new (fixes can't continue convo on /new/) style(MessagesView): matches height to official fix(SSE): pass paramId and invalidate convos style(Message): make bg uniform * refactor(useSSE): setStorage within setConversation updates * feat: conversationKeysAtom, allConversationsSelector, update convos query data on created message (if new), correctly handle convo deletion (individual) * feat: add popover select dropdowns to allow options in header while allowing horizontal scroll for mobile * style(pluginsSelect): styling changes * refactor(NewEndpointsMenu): make UI components modular * feat: Presets complete * fix: preset editing, make by index * fix: conversations not setting on inital navigation, fix getMessages() based on query param * fix: changing preset no longer resets latestMessage * feat: useOnClickOutside for OptionsPopover and fix bug that causes selection of preset when deleting * fix: revert /chat/ switchToConvo, also use NewDeleteButton in Convo * fix: Popover correctly closes on close Popover button using custom condition for useOnClickOutside * style: new message and nav styling * style: hover/sibling buttons and preset menu scrolling * feat: new convo header button * style(Textarea): minor style changes to textarea buttons * feat: stop/continue generating and hide hoverbuttons when submitting * feat: compact AI Provider schemas to make json payloads and db saves smaller * style: styling changes for consistency on chat route * fix: created usePresetIndexOptions to prevent bugs between /c/ and /chat/ routes when editing presets, removed redundant code from the new dialog * chore: make /chat/ route default for now since we still lack full image support
2023-11-16 10:42:24 -05:00
router.use([`/${EModelEndpoint.azureOpenAI}`, `/${EModelEndpoint.openAI}`], openAI);
router.use(`/${EModelEndpoint.chatGPTBrowser}`, askChatGPTBrowser);
router.use(`/${EModelEndpoint.gptPlugins}`, gptPlugins);
router.use(`/${EModelEndpoint.anthropic}`, anthropic);
feat(Google): Support all Text/Chat Models, Response streaming, `PaLM` -> `Google` 🤖 (#1316) * feat: update PaLM icons * feat: add additional google models * POC: formatting inputs for Vertex AI streaming * refactor: move endpoints services outside of /routes dir to /services/Endpoints * refactor: shorten schemas import * refactor: rename PALM to GOOGLE * feat: make Google editable endpoint * feat: reusable Ask and Edit controllers based off Anthropic * chore: organize imports/logic * fix(parseConvo): include examples in googleSchema * fix: google only allows odd number of messages to be sent * fix: pass proxy to AnthropicClient * refactor: change `google` altName to `Google` * refactor: update getModelMaxTokens and related functions to handle maxTokensMap with nested endpoint model key/values * refactor: google Icon and response sender changes (Codey and Google logo instead of PaLM in all cases) * feat: google support for maxTokensMap * feat: google updated endpoints with Ask/Edit controllers, buildOptions, and initializeClient * feat(GoogleClient): now builds prompt for text models and supports real streaming from Vertex AI through langchain * chore(GoogleClient): remove comments, left before for reference in git history * docs: update google instructions (WIP) * docs(apis_and_tokens.md): add images to google instructions * docs: remove typo apis_and_tokens.md * Update apis_and_tokens.md * feat(Google): use default settings map, fully support context for both text and chat models, fully support examples for chat models * chore: update more PaLM references to Google * chore: move playwright out of workflows to avoid failing tests
2023-12-10 14:54:13 -05:00
router.use(`/${EModelEndpoint.google}`, google);
router.use(`/${EModelEndpoint.bingAI}`, bingAI);
💫 feat: Config File & Custom Endpoints (#1474) * WIP(backend/api): custom endpoint * WIP(frontend/client): custom endpoint * chore: adjust typedefs for configs * refactor: use data-provider for cache keys and rename enums and custom endpoint for better clarity and compatibility * feat: loadYaml utility * refactor: rename back to from and proof-of-concept for creating schemas from user-defined defaults * refactor: remove custom endpoint from default endpointsConfig as it will be exclusively managed by yaml config * refactor(EndpointController): rename variables for clarity * feat: initial load custom config * feat(server/utils): add simple `isUserProvided` helper * chore(types): update TConfig type * refactor: remove custom endpoint handling from model services as will be handled by config, modularize fetching of models * feat: loadCustomConfig, loadConfigEndpoints, loadConfigModels * chore: reorganize server init imports, invoke loadCustomConfig * refactor(loadConfigEndpoints/Models): return each custom endpoint as standalone endpoint * refactor(Endpoint/ModelController): spread config values after default (temporary) * chore(client): fix type issues * WIP: first pass for multiple custom endpoints - add endpointType to Conversation schema - add update zod schemas for both convo/presets to allow non-EModelEndpoint value as endpoint (also using type assertion) - use `endpointType` value as `endpoint` where mapping to type is necessary using this field - use custom defined `endpoint` value and not type for mapping to modelsConfig - misc: add return type to `getDefaultEndpoint` - in `useNewConvo`, add the endpointType if it wasn't already added to conversation - EndpointsMenu: use user-defined endpoint name as Title in menu - TODO: custom icon via custom config, change unknown to robot icon * refactor(parseConvo): pass args as an object and change where used accordingly; chore: comment out 'create schema' code * chore: remove unused availableModels field in TConfig type * refactor(parseCompactConvo): pass args as an object and change where used accordingly * feat: chat through custom endpoint * chore(message/convoSchemas): avoid saving empty arrays * fix(BaseClient/saveMessageToDatabase): save endpointType * refactor(ChatRoute): show Spinner if endpointsQuery or modelsQuery are still loading, which is apparent with slow fetching of models/remote config on first serve * fix(useConversation): assign endpointType if it's missing * fix(SaveAsPreset): pass real endpoint and endpointType when saving Preset) * chore: recorganize types order for TConfig, add `iconURL` * feat: custom endpoint icon support: - use UnknownIcon in all icon contexts - add mistral and openrouter as known endpoints, and add their icons - iconURL support * fix(presetSchema): move endpointType to default schema definitions shared between convoSchema and defaults * refactor(Settings/OpenAI): remove legacy `isOpenAI` flag * fix(OpenAIClient): do not invoke abortCompletion on completion error * feat: add responseSender/label support for custom endpoints: - use defaultModelLabel field in endpointOption - add model defaults for custom endpoints in `getResponseSender` - add `useGetSender` hook which uses EndpointsQuery to determine `defaultModelLabel` - include defaultModelLabel from endpointConfig in custom endpoint client options - pass `endpointType` to `getResponseSender` * feat(OpenAIClient): use custom options from config file * refactor: rename `defaultModelLabel` to `modelDisplayLabel` * refactor(data-provider): separate concerns from `schemas` into `parsers`, `config`, and fix imports elsewhere * feat: `iconURL` and extract environment variables from custom endpoint config values * feat: custom config validation via zod schema, rename and move to `./projectRoot/librechat.yaml` * docs: custom config docs and examples * fix(OpenAIClient/mistral): mistral does not allow singular system message, also add `useChatCompletion` flag to use openai-node for title completions * fix(custom/initializeClient): extract env var and use `isUserProvided` function * Update librechat.example.yaml * feat(InputWithLabel): add className props, and forwardRef * fix(streamResponse): handle error edge case where either messages or convos query throws an error * fix(useSSE): handle errorHandler edge cases where error response is and is not properly formatted from API, especially when a conversationId is not yet provided, which ensures stream is properly closed on error * feat: user_provided keys for custom endpoints * fix(config/endpointSchema): do not allow default endpoint values in custom endpoint `name` * feat(loadConfigModels): extract env variables and optimize fetching models * feat: support custom endpoint iconURL for messages and Nav * feat(OpenAIClient): add/dropParams support * docs: update docs with default params, add/dropParams, and notes to use config file instead of `OPENAI_REVERSE_PROXY` * docs: update docs with additional notes * feat(maxTokensMap): add mistral models (32k context) * docs: update openrouter notes * Update ai_setup.md * docs(custom_config): add table of contents and fix note about custom name * docs(custom_config): reorder ToC * Update custom_config.md * Add note about `max_tokens` field in custom_config.md
2024-01-03 09:22:48 -05:00
router.use(`/${EModelEndpoint.custom}`, custom);
2023-02-12 16:38:33 -05:00
module.exports = router;