LibreChat/api/server/routes/endpoints/openAI/initializeClient.js

54 lines
1.6 KiB
JavaScript
Raw Normal View History

feat(GPT/Anthropic): Continue Regenerating & Generation Buttons (#808) * feat(useMessageHandler.js/ts): Refactor and add features to handle user messages, support multiple endpoints/models, generate placeholder responses, regeneration, and stopGeneration function fix(conversation.ts, buildTree.ts): Import TMessage type, handle null parentMessageId feat(schemas.ts): Update and add schemas for various AI services, add default values, optional fields, and endpoint-to-schema mapping, create parseConvo function chore(useMessageHandler.js, schemas.ts): Remove unused imports, variables, and chatGPT enum * wip: add generation buttons * refactor(cleanupPreset.ts): simplify cleanupPreset function refactor(getDefaultConversation.js): remove unused code and simplify getDefaultConversation function feat(utils): add getDefaultConversation function This commit adds a new utility function called `getDefaultConversation` to the `client/src/utils/getDefaultConversation.ts` file. This function is responsible for generating a default conversation object based on the provided parameters. The `getDefaultConversation` function takes in an object with the following properties: - `conversation`: The conversation object to be used as a base. - `endpointsConfig`: The configuration object containing information about the available endpoints. - `preset`: An optional preset object that can be used to override the default behavior. The function first tries to determine the target endpoint based on the preset object. If a valid endpoint is found, it is used as the target endpoint. If not, the function tries to retrieve the last conversation setup from the local storage and uses its endpoint if it is valid. If neither the preset nor the local storage contains a valid endpoint, the function falls back to a default endpoint. Once the target endpoint is determined, * fix(utils): remove console.error statement in buildDefaultConversation function fix(schemas): add default values for catch blocks in openAISchema, googleSchema, bingAISchema, anthropicSchema, chatGPTBrowserSchema, and gptPluginsSchema * fix: endpoint not changing on change of preset from other endpoint, wip: refactor * refactor: preset items to TSX * refactor: convert resetConvo to TS * refactor(getDefaultConversation.ts): move defaultEndpoints array to the top of the file for better readability refactor(getDefaultConversation.ts): extract getDefaultEndpoint function for better code organization and reusability * feat(svg): add ContinueIcon component feat(svg): add RegenerateIcon component feat(svg): add ContinueIcon and RegenerateIcon components to index.ts * feat(Button.tsx): add onClick and className props to Button component feat(GenerationButtons.tsx): add logic to display Regenerate or StopGenerating button based on isSubmitting and messages feat(Regenerate.tsx): create Regenerate component with RegenerateIcon and handleRegenerate function feat(StopGenerating.tsx): create StopGenerating component with StopGeneratingIcon and handleStopGenerating function * fix(TextChat.jsx): reorder imports and variables for better readability fix(TextChat.jsx): fix typo in condition for isNotAppendable variable fix(TextChat.jsx): remove unused handleStopGenerating function fix(ContinueIcon.tsx): remove unnecessary closing tags for polygon elements fix(useMessageHandler.ts): add missing type annotations for handleStopGenerating and handleRegenerate functions fix(useMessageHandler.ts): remove unused variables in return statement * fix(getDefaultConversation.ts): refactor code to use getLocalStorageItems function feat(getLocalStorageItems.ts): add utility function to retrieve items from local storage * fix(OpenAIClient.js): add support for streaming result in sendCompletion method feat(OpenAIClient.js): add finish_reason metadata to opts in sendCompletion method feat(Message.js): add finish_reason field to Message model feat(messageSchema.js): add finish_reason field to messageSchema feat(openAI.js): parse chatGptLabel and promptPrefix from req.body and pass rest of the modelOptions to endpointOption feat(openAI.js): add addMetadata function to store metadata in ask function feat(openAI.js): add metadata to response if available feat(schemas.ts): add finish_reason field to tMessageSchema * feat(types.ts): add TOnClick and TGenButtonProps types for button components feat(Continue.tsx): create Continue component for generating button feat(GenerationButtons.tsx): update GenerationButtons component to use Continue component feat(Regenerate.tsx): create Regenerate component for regenerating button feat(Stop.tsx): create Stop component for stop generating button * feat(MessageHandler.jsx): add MessageHandler component to handle messages and conversations fix(Root.jsx): fix import paths for Nav and MessageHandler components * feat(useMessageHandler.ts): add support for generation parameter in ask function feat(useMessageHandler.ts): add support for isEdited parameter in ask function feat(useMessageHandler.ts): add support for continueGeneration function fix(createPayload.ts): replace endpoint URL when isEdited parameter is true * chore(client): set skipLibCheck to true in tsconfig.json * fix(useMessageHandler.ts): remove unused clientId variable fix(schemas.ts): make clientId field in tMessageSchema nullable and optional * wip: edit route for continue generation * refactor(api): move handlers to root of routes dir * fix(useMessageHandler.ts): initialize currentMessages to an empty array if messages is null fix(useMessageHandler.ts): update initialResponse text to use responseText variable fix(useMessageHandler.ts): update setMessages logic for isRegenerate case fix(MessageHandler.jsx): update setMessages logic for cancelHandler, createdHandler, and finalHandler * fix(schemas.ts): make createdAt and updatedAt fields optional and set default values using new Date().toISOString() fix(schemas.ts): change type annotation of TMessage from infer to input * refactor(useMessageHandler.ts): rename AskProps type to TAskProps refactor(useMessageHandler.ts): remove generation property from ask function arguments refactor(useMessageHandler.ts): use nullish coalescing operator (??) instead of logical OR (||) refactor(useMessageHandler.ts): pass the responseMessageId to message prop of submission * fix(BaseClient.js): use nullish coalescing operator (??) instead of logical OR (||) for default values * fix(BaseClient.js): fix responseMessageId assignment in handleStartMethods method feat(BaseClient.js): add support for isEdited flag in sendMessage method feat(BaseClient.js): add generation to responseMessage text in sendMessage method * fix(openAI.js): remove unused imports and commented out code feat(openAI.js): add support for generation parameter in request body fix(openAI.js): remove console.log statement fix(openAI.js): remove unused variables and parameters fix(openAI.js): update response text in case of error fix(openAI.js): handle error and abort message in case of error fix(handlers.js): add generation parameter to createOnProgress function fix(useMessageHandler.ts): update responseText variable to use generation parameter * refactor(api/middleware): move inside server dir * refactor: add endpoint specific, modular functions to build options and initialize clients, create server/utils, move middleware, separate utils into api general utils and server specific utils * fix(abortMiddleware.js): import getConvo and getConvoTitle functions from models feat(abortMiddleware.js): add abortAsk function to abortController to handle aborting of requests fix(openAI.js): import buildOptions and initializeClient functions from endpoints/openAI refactor(openAI.js): use getAbortData function to get data for abortAsk function * refactor: move endpoint specific logic to an endpoints dir * refactor(PluginService.js): fix import path for encrypt and decrypt functions in PluginService.js * feat(openAI): add new endpoint for adding a title to a conversation - Added a new file `addTitle.js` in the `api/server/routes/endpoints/openAI` directory. - The `addTitle.js` file exports a function `addTitle` that takes in request parameters and performs the following actions: - If the `parentMessageId` is `'00000000-0000-0000-0000-000000000000'` and `newConvo` is true, it proceeds with the following steps: - Calls the `titleConvo` function from the `titleConvo` module, passing in the necessary parameters. - Calls the `saveConvo` function from the `saveConvo` module, passing in the user ID and conversation details. - Updated the `index.js` file in the `api/server/routes/endpoints/openAI` directory to export the `addTitle` function. - This change adds * fix(abortMiddleware.js): remove console.log statement refactor(gptPlugins.js): update imports and function parameters feat(gptPlugins.js): add support for abortController and getAbortData refactor(openAI.js): update imports and function parameters feat(openAI.js): add support for abortController and getAbortData fix(openAI.js): refactor code to use modularized functions and middleware fix(buildOptions.js): refactor code to use destructuring and update variable names * refactor(askChatGPTBrowser.js, bingAI.js, google.js): remove duplicate code for setting response headers feat(askChatGPTBrowser.js, bingAI.js, google.js): add setHeaders middleware to set response headers * feat(middleware): validateEndpoint, refactor buildOption to only be concerned of endpointOption * fix(abortMiddleware.js): add 'finish_reason' property with value 'incomplete' to responseMessage object fix(abortMessage.js): remove console.log statement for aborted message fix(handlers.js): modify tokens assignment to handle empty generation string and trailing space * fix(BaseClient.js): import addSpaceIfNeeded function from server/utils fix(BaseClient.js): add space before generation in text property fix(index.js): remove getCitations and citeText exports feat(buildEndpointOption.js): add buildEndpointOption middleware fix(index.js): import buildEndpointOption middleware fix(anthropic.js): remove buildOptions function and use endpointOption from req.body fix(gptPlugins.js): remove buildOptions function and use endpointOption from req.body fix(openAI.js): remove buildOptions function and use endpointOption from req.body feat(utils): add citations.js and handleText.js modules fix(utils): fix import statements in index.js module * refactor(gptPlugins.js): use getResponseSender function from librechat-data-provider * feat(gptPlugins): complete 'continue generating' * wip: anthropic continue regen * feat(middleware): add validateRegistration middleware A new middleware function called `validateRegistration` has been added to the list of exported middleware functions in `index.js`. This middleware is responsible for validating registration data before allowing the registration process to proceed. * feat(Anthropic): complete continue regen * chore: add librechat-data-provider to api/package.json * fix(ci): backend-review will mock meilisearch, also installs data-provider as now needed * chore(ci): remove unneeded SEARCH env var * style(GenerationButtons): make text shorter for sake of space economy, even though this diverges from chat.openai.com * style(GenerationButtons/ScrollToBottom): adjust visibility/position based on screen size * chore(client): 'Editting' typo * feat(GenerationButtons.tsx): add support for endpoint prop in GenerationButtons component feat(OptionsBar.tsx): pass endpoint prop to GenerationButtons component feat(useGenerations.ts): create useGenerations hook to handle generation logic fix(schemas.ts): add searchResult field to tMessageSchema * refactor(HoverButtons): convert to TSX and utilize new useGenerations hook * fix(abortMiddleware): handle error with res headers set, or abortController not found, to ensure proper API error is sent to the client, chore(BaseClient): remove console log for onStart message meant for debugging * refactor(api): remove librechat-data-provider dep for now as it complicates deployed docker build stage, re-use code in CJS, located in server/endpoints/schemas * chore: remove console.logs from test files * ci: add backend tests for AnthropicClient, focusing on new buildMessages logic * refactor(FakeClient): use actual BaseClient sendMessage method for testing * test(BaseClient.test.js): add test for loading chat history test(BaseClient.test.js): add test for sendMessage logic with isEdited flag * fix(buildEndpointOption.js): add support for azureOpenAI in buildFunction object wip(endpoints.js): fetch Azure models from Azure OpenAI API if opts.azure is true * fix(Button.tsx): add data-testid attribute to button component fix(SelectDropDown.tsx): add data-testid attribute to Listbox.Button component fix(messages.spec.ts): add waitForServerStream function to consolidate logic for awaiting the server response feat(messages.spec.ts): add test for stopping and continuing message and improve browser/page context order and closing * refactor(onProgress): speed up time to save initial message for editable routes * chore: disable AI message editing (for now), was accidentally allowed * refactor: ensure continue is only supported for latest message style: improve styling in dark mode and across all hover buttons/icons, including making edit icon for AI invisible (for now) * fix: add test id to generation buttons so they never resolve to 2+ items * chore(package.json): add 'packages/' to the list of ignored directories chore(data-provider/package.json): bump version to 0.1.5
2023-08-17 12:50:05 -04:00
const { OpenAIClient } = require('../../../../app');
feat: ConversationSummaryBufferMemory (#973) * refactor: pass model in message edit payload, use encoder in standalone util function * feat: add summaryBuffer helper * refactor(api/messages): use new countTokens helper and add auth middleware at top * wip: ConversationSummaryBufferMemory * refactor: move pre-generation helpers to prompts dir * chore: remove console log * chore: remove test as payload will no longer carry tokenCount * chore: update getMessagesWithinTokenLimit JSDoc * refactor: optimize getMessagesForConversation and also break on summary, feat(ci): getMessagesForConversation tests * refactor(getMessagesForConvo): count '00000000-0000-0000-0000-000000000000' as root message * chore: add newer model to token map * fix: condition was point to prop of array instead of message prop * refactor(BaseClient): use object for refineMessages param, rename 'summary' to 'summaryMessage', add previous_summary refactor(getMessagesWithinTokenLimit): replace text and tokenCount if should summarize, summary, and summaryTokenCount are present fix/refactor(handleContextStrategy): use the right comparison length for context diff, and replace payload first message when a summary is present * chore: log previous_summary if debugging * refactor(formatMessage): assume if role is defined that it's a valid value * refactor(getMessagesWithinTokenLimit): remove summary logic refactor(handleContextStrategy): add usePrevSummary logic in case only summary was pruned refactor(loadHistory): initial message query will return all ordered messages but keep track of the latest summary refactor(getMessagesForConversation): use object for single param, edit jsdoc, edit all files using the method refactor(ChatGPTClient): order messages before buildPrompt is called, TODO: add convoSumBuffMemory logic * fix: undefined handling and summarizing only when shouldRefineContext is true * chore(BaseClient): fix test results omitting system role for summaries and test edge case * chore: export summaryBuffer from index file * refactor(OpenAIClient/BaseClient): move refineMessages to subclass, implement LLM initialization for summaryBuffer * feat: add OPENAI_SUMMARIZE to enable summarizing, refactor: rename client prop 'shouldRefineContext' to 'shouldSummarize', change contextStrategy value to 'summarize' from 'refine' * refactor: rename refineMessages method to summarizeMessages for clarity * chore: clarify summary future intent in .env.example * refactor(initializeLLM): handle case for either 'model' or 'modelName' being passed * feat(gptPlugins): enable summarization for plugins * refactor(gptPlugins): utilize new initializeLLM method and formatting methods for messages, use payload array for currentMessages and assign pastMessages sooner * refactor(agents): use ConversationSummaryBufferMemory for both agent types * refactor(formatMessage): optimize original method for langchain, add helper function for langchain messages, add JSDocs and tests * refactor(summaryBuffer): add helper to createSummaryBufferMemory, and use new formatting helpers * fix: forgot to spread formatMessages also took opportunity to pluralize filename * refactor: pass memory to tools, namely openapi specs. not used and may never be used by new method but added for testing * ci(formatMessages): add more exhaustive checks for langchain messages * feat: add debug env var for OpenAI * chore: delete unnecessary comments * chore: add extra note about summary feature * fix: remove tokenCount from payload instructions * fix: test fail * fix: only pass instructions to payload when defined or not empty object * refactor: fromPromptMessages is deprecated, use renamed method fromMessages * refactor: use 'includes' instead of 'startsWith' for extended OpenRouter compatibility * fix(PluginsClient.buildPromptBody): handle undefined message strings * chore: log langchain titling error * feat: getModelMaxTokens helper * feat: tokenSplit helper * feat: summary prompts updated * fix: optimize _CUT_OFF_SUMMARIZER prompt * refactor(summaryBuffer): use custom summary prompt, allow prompt to be passed, pass humanPrefix and aiPrefix to memory, along with any future variables, rename messagesToRefine to context * fix(summaryBuffer): handle edge case where messagesToRefine exceeds summary context, refactor(BaseClient): allow custom maxContextTokens to be passed to getMessagesWithinTokenLimit, add defined check before unshifting summaryMessage, update shouldSummarize based on this refactor(OpenAIClient): use getModelMaxTokens, use cut-off message method for summary if no messages were left after pruning * fix(handleContextStrategy): handle case where incoming prompt is bigger than model context * chore: rename refinedContent to splitText * chore: remove unnecessary debug log
2023-09-26 21:02:28 -04:00
const { isEnabled } = require('../../../utils');
feat(GPT/Anthropic): Continue Regenerating & Generation Buttons (#808) * feat(useMessageHandler.js/ts): Refactor and add features to handle user messages, support multiple endpoints/models, generate placeholder responses, regeneration, and stopGeneration function fix(conversation.ts, buildTree.ts): Import TMessage type, handle null parentMessageId feat(schemas.ts): Update and add schemas for various AI services, add default values, optional fields, and endpoint-to-schema mapping, create parseConvo function chore(useMessageHandler.js, schemas.ts): Remove unused imports, variables, and chatGPT enum * wip: add generation buttons * refactor(cleanupPreset.ts): simplify cleanupPreset function refactor(getDefaultConversation.js): remove unused code and simplify getDefaultConversation function feat(utils): add getDefaultConversation function This commit adds a new utility function called `getDefaultConversation` to the `client/src/utils/getDefaultConversation.ts` file. This function is responsible for generating a default conversation object based on the provided parameters. The `getDefaultConversation` function takes in an object with the following properties: - `conversation`: The conversation object to be used as a base. - `endpointsConfig`: The configuration object containing information about the available endpoints. - `preset`: An optional preset object that can be used to override the default behavior. The function first tries to determine the target endpoint based on the preset object. If a valid endpoint is found, it is used as the target endpoint. If not, the function tries to retrieve the last conversation setup from the local storage and uses its endpoint if it is valid. If neither the preset nor the local storage contains a valid endpoint, the function falls back to a default endpoint. Once the target endpoint is determined, * fix(utils): remove console.error statement in buildDefaultConversation function fix(schemas): add default values for catch blocks in openAISchema, googleSchema, bingAISchema, anthropicSchema, chatGPTBrowserSchema, and gptPluginsSchema * fix: endpoint not changing on change of preset from other endpoint, wip: refactor * refactor: preset items to TSX * refactor: convert resetConvo to TS * refactor(getDefaultConversation.ts): move defaultEndpoints array to the top of the file for better readability refactor(getDefaultConversation.ts): extract getDefaultEndpoint function for better code organization and reusability * feat(svg): add ContinueIcon component feat(svg): add RegenerateIcon component feat(svg): add ContinueIcon and RegenerateIcon components to index.ts * feat(Button.tsx): add onClick and className props to Button component feat(GenerationButtons.tsx): add logic to display Regenerate or StopGenerating button based on isSubmitting and messages feat(Regenerate.tsx): create Regenerate component with RegenerateIcon and handleRegenerate function feat(StopGenerating.tsx): create StopGenerating component with StopGeneratingIcon and handleStopGenerating function * fix(TextChat.jsx): reorder imports and variables for better readability fix(TextChat.jsx): fix typo in condition for isNotAppendable variable fix(TextChat.jsx): remove unused handleStopGenerating function fix(ContinueIcon.tsx): remove unnecessary closing tags for polygon elements fix(useMessageHandler.ts): add missing type annotations for handleStopGenerating and handleRegenerate functions fix(useMessageHandler.ts): remove unused variables in return statement * fix(getDefaultConversation.ts): refactor code to use getLocalStorageItems function feat(getLocalStorageItems.ts): add utility function to retrieve items from local storage * fix(OpenAIClient.js): add support for streaming result in sendCompletion method feat(OpenAIClient.js): add finish_reason metadata to opts in sendCompletion method feat(Message.js): add finish_reason field to Message model feat(messageSchema.js): add finish_reason field to messageSchema feat(openAI.js): parse chatGptLabel and promptPrefix from req.body and pass rest of the modelOptions to endpointOption feat(openAI.js): add addMetadata function to store metadata in ask function feat(openAI.js): add metadata to response if available feat(schemas.ts): add finish_reason field to tMessageSchema * feat(types.ts): add TOnClick and TGenButtonProps types for button components feat(Continue.tsx): create Continue component for generating button feat(GenerationButtons.tsx): update GenerationButtons component to use Continue component feat(Regenerate.tsx): create Regenerate component for regenerating button feat(Stop.tsx): create Stop component for stop generating button * feat(MessageHandler.jsx): add MessageHandler component to handle messages and conversations fix(Root.jsx): fix import paths for Nav and MessageHandler components * feat(useMessageHandler.ts): add support for generation parameter in ask function feat(useMessageHandler.ts): add support for isEdited parameter in ask function feat(useMessageHandler.ts): add support for continueGeneration function fix(createPayload.ts): replace endpoint URL when isEdited parameter is true * chore(client): set skipLibCheck to true in tsconfig.json * fix(useMessageHandler.ts): remove unused clientId variable fix(schemas.ts): make clientId field in tMessageSchema nullable and optional * wip: edit route for continue generation * refactor(api): move handlers to root of routes dir * fix(useMessageHandler.ts): initialize currentMessages to an empty array if messages is null fix(useMessageHandler.ts): update initialResponse text to use responseText variable fix(useMessageHandler.ts): update setMessages logic for isRegenerate case fix(MessageHandler.jsx): update setMessages logic for cancelHandler, createdHandler, and finalHandler * fix(schemas.ts): make createdAt and updatedAt fields optional and set default values using new Date().toISOString() fix(schemas.ts): change type annotation of TMessage from infer to input * refactor(useMessageHandler.ts): rename AskProps type to TAskProps refactor(useMessageHandler.ts): remove generation property from ask function arguments refactor(useMessageHandler.ts): use nullish coalescing operator (??) instead of logical OR (||) refactor(useMessageHandler.ts): pass the responseMessageId to message prop of submission * fix(BaseClient.js): use nullish coalescing operator (??) instead of logical OR (||) for default values * fix(BaseClient.js): fix responseMessageId assignment in handleStartMethods method feat(BaseClient.js): add support for isEdited flag in sendMessage method feat(BaseClient.js): add generation to responseMessage text in sendMessage method * fix(openAI.js): remove unused imports and commented out code feat(openAI.js): add support for generation parameter in request body fix(openAI.js): remove console.log statement fix(openAI.js): remove unused variables and parameters fix(openAI.js): update response text in case of error fix(openAI.js): handle error and abort message in case of error fix(handlers.js): add generation parameter to createOnProgress function fix(useMessageHandler.ts): update responseText variable to use generation parameter * refactor(api/middleware): move inside server dir * refactor: add endpoint specific, modular functions to build options and initialize clients, create server/utils, move middleware, separate utils into api general utils and server specific utils * fix(abortMiddleware.js): import getConvo and getConvoTitle functions from models feat(abortMiddleware.js): add abortAsk function to abortController to handle aborting of requests fix(openAI.js): import buildOptions and initializeClient functions from endpoints/openAI refactor(openAI.js): use getAbortData function to get data for abortAsk function * refactor: move endpoint specific logic to an endpoints dir * refactor(PluginService.js): fix import path for encrypt and decrypt functions in PluginService.js * feat(openAI): add new endpoint for adding a title to a conversation - Added a new file `addTitle.js` in the `api/server/routes/endpoints/openAI` directory. - The `addTitle.js` file exports a function `addTitle` that takes in request parameters and performs the following actions: - If the `parentMessageId` is `'00000000-0000-0000-0000-000000000000'` and `newConvo` is true, it proceeds with the following steps: - Calls the `titleConvo` function from the `titleConvo` module, passing in the necessary parameters. - Calls the `saveConvo` function from the `saveConvo` module, passing in the user ID and conversation details. - Updated the `index.js` file in the `api/server/routes/endpoints/openAI` directory to export the `addTitle` function. - This change adds * fix(abortMiddleware.js): remove console.log statement refactor(gptPlugins.js): update imports and function parameters feat(gptPlugins.js): add support for abortController and getAbortData refactor(openAI.js): update imports and function parameters feat(openAI.js): add support for abortController and getAbortData fix(openAI.js): refactor code to use modularized functions and middleware fix(buildOptions.js): refactor code to use destructuring and update variable names * refactor(askChatGPTBrowser.js, bingAI.js, google.js): remove duplicate code for setting response headers feat(askChatGPTBrowser.js, bingAI.js, google.js): add setHeaders middleware to set response headers * feat(middleware): validateEndpoint, refactor buildOption to only be concerned of endpointOption * fix(abortMiddleware.js): add 'finish_reason' property with value 'incomplete' to responseMessage object fix(abortMessage.js): remove console.log statement for aborted message fix(handlers.js): modify tokens assignment to handle empty generation string and trailing space * fix(BaseClient.js): import addSpaceIfNeeded function from server/utils fix(BaseClient.js): add space before generation in text property fix(index.js): remove getCitations and citeText exports feat(buildEndpointOption.js): add buildEndpointOption middleware fix(index.js): import buildEndpointOption middleware fix(anthropic.js): remove buildOptions function and use endpointOption from req.body fix(gptPlugins.js): remove buildOptions function and use endpointOption from req.body fix(openAI.js): remove buildOptions function and use endpointOption from req.body feat(utils): add citations.js and handleText.js modules fix(utils): fix import statements in index.js module * refactor(gptPlugins.js): use getResponseSender function from librechat-data-provider * feat(gptPlugins): complete 'continue generating' * wip: anthropic continue regen * feat(middleware): add validateRegistration middleware A new middleware function called `validateRegistration` has been added to the list of exported middleware functions in `index.js`. This middleware is responsible for validating registration data before allowing the registration process to proceed. * feat(Anthropic): complete continue regen * chore: add librechat-data-provider to api/package.json * fix(ci): backend-review will mock meilisearch, also installs data-provider as now needed * chore(ci): remove unneeded SEARCH env var * style(GenerationButtons): make text shorter for sake of space economy, even though this diverges from chat.openai.com * style(GenerationButtons/ScrollToBottom): adjust visibility/position based on screen size * chore(client): 'Editting' typo * feat(GenerationButtons.tsx): add support for endpoint prop in GenerationButtons component feat(OptionsBar.tsx): pass endpoint prop to GenerationButtons component feat(useGenerations.ts): create useGenerations hook to handle generation logic fix(schemas.ts): add searchResult field to tMessageSchema * refactor(HoverButtons): convert to TSX and utilize new useGenerations hook * fix(abortMiddleware): handle error with res headers set, or abortController not found, to ensure proper API error is sent to the client, chore(BaseClient): remove console log for onStart message meant for debugging * refactor(api): remove librechat-data-provider dep for now as it complicates deployed docker build stage, re-use code in CJS, located in server/endpoints/schemas * chore: remove console.logs from test files * ci: add backend tests for AnthropicClient, focusing on new buildMessages logic * refactor(FakeClient): use actual BaseClient sendMessage method for testing * test(BaseClient.test.js): add test for loading chat history test(BaseClient.test.js): add test for sendMessage logic with isEdited flag * fix(buildEndpointOption.js): add support for azureOpenAI in buildFunction object wip(endpoints.js): fetch Azure models from Azure OpenAI API if opts.azure is true * fix(Button.tsx): add data-testid attribute to button component fix(SelectDropDown.tsx): add data-testid attribute to Listbox.Button component fix(messages.spec.ts): add waitForServerStream function to consolidate logic for awaiting the server response feat(messages.spec.ts): add test for stopping and continuing message and improve browser/page context order and closing * refactor(onProgress): speed up time to save initial message for editable routes * chore: disable AI message editing (for now), was accidentally allowed * refactor: ensure continue is only supported for latest message style: improve styling in dark mode and across all hover buttons/icons, including making edit icon for AI invisible (for now) * fix: add test id to generation buttons so they never resolve to 2+ items * chore(package.json): add 'packages/' to the list of ignored directories chore(data-provider/package.json): bump version to 0.1.5
2023-08-17 12:50:05 -04:00
const { getAzureCredentials } = require('../../../../utils');
refactor: Encrypt & Expire User Provided Keys, feat: Rate Limiting (#874) * docs: make_your_own.md formatting fix for mkdocs * feat: add express-mongo-sanitize feat: add login/registration rate limiting * chore: remove unnecessary console log * wip: remove token handling from localStorage to encrypted DB solution * refactor: minor change to UserService * fix mongo query and add keys route to server * fix backend controllers and simplify schema/crud * refactor: rename token to key to separate from access/refresh tokens, setTokenDialog -> setKeyDialog * refactor(schemas): TEndpointOption token -> key * refactor(api): use new encrypted key retrieval system * fix(SetKeyDialog): fix key prop error * fix(abortMiddleware): pass random UUID if messageId is not generated yet for proper error display on frontend * fix(getUserKey): wrong prop passed in arg, adds error handling * fix: prevent message without conversationId from saving to DB, prevents branching on the frontend to a new top-level branch * refactor: change wording of multiple display messages * refactor(checkExpiry -> checkUserKeyExpiry): move to UserService file * fix: type imports from common * refactor(SubmitButton): convert to TS * refactor(key.ts): change localStorage map key name * refactor: add new custom tailwind classes to better match openAI colors * chore: remove unnecessary warning and catch ScreenShot error * refactor: move userKey frontend logic to hooks and remove use of localStorage and instead query the DB * refactor: invalidate correct query key, memoize userKey hook, conditionally render SetKeyDialog to avoid unnecessary calls, refactor SubmitButton props and useEffect for showing 'provide key first' * fix(SetKeyDialog): use enum-like object for expiry values feat(Dropdown): add optionsClassName to dynamically change dropdown options container classes * fix: handle edge case where user had provided a key but the server changes to env variable for keys * refactor(OpenAI/titleConvo): move titling to client to retain authorized credentials in message lifecycle for titling * fix(azure): handle user_provided keys correctly for azure * feat: send user Id to OpenAI to differentiate users in completion requests * refactor(OpenAI/titleConvo): adding tokens helps minimize LLM from using the language in title response * feat: add delete endpoint for keys * chore: remove throttling of title * feat: add 'Data controls' to Settings, add 'Revoke' keys feature in Key Dialog and Data controls * refactor: reorganize PluginsClient files in langchain format * feat: use langchain for titling convos * chore: cleanup titling convo, with fallback to original method, escape braces, use only snippet for language detection * refactor: move helper functions to appropriate langchain folders for reusability * fix: userProvidesKey handling for gptPlugins * fix: frontend handling of plugins key * chore: cleanup logging and ts-ignore SSE * fix: forwardRef misuse in DangerButton * fix(GoogleConfig/FileUpload): localize errors and simplify validation with zod * fix: cleanup google logging and fix user provided key handling * chore: remove titling from google * chore: removing logging from browser endpoint * wip: fix menu flicker * feat: useLocalStorage hook * feat: add Tooltip for UI * refactor(EndpointMenu): utilize Tooltip and useLocalStorage, remove old 'New Chat' slide-over * fix(e2e): use testId for endpoint menu trigger * chore: final touches to EndpointMenu before future refactor to declutter component * refactor(localization): change select endpoint to open menu and add translations * chore: add final prop to error message response * ci: minor edits to facilitate testing * ci: new e2e test which tests for new key setting/revoking features
2023-09-06 10:46:27 -04:00
const { getUserKey, checkUserKeyExpiry } = require('../../../services/UserService');
feat(GPT/Anthropic): Continue Regenerating & Generation Buttons (#808) * feat(useMessageHandler.js/ts): Refactor and add features to handle user messages, support multiple endpoints/models, generate placeholder responses, regeneration, and stopGeneration function fix(conversation.ts, buildTree.ts): Import TMessage type, handle null parentMessageId feat(schemas.ts): Update and add schemas for various AI services, add default values, optional fields, and endpoint-to-schema mapping, create parseConvo function chore(useMessageHandler.js, schemas.ts): Remove unused imports, variables, and chatGPT enum * wip: add generation buttons * refactor(cleanupPreset.ts): simplify cleanupPreset function refactor(getDefaultConversation.js): remove unused code and simplify getDefaultConversation function feat(utils): add getDefaultConversation function This commit adds a new utility function called `getDefaultConversation` to the `client/src/utils/getDefaultConversation.ts` file. This function is responsible for generating a default conversation object based on the provided parameters. The `getDefaultConversation` function takes in an object with the following properties: - `conversation`: The conversation object to be used as a base. - `endpointsConfig`: The configuration object containing information about the available endpoints. - `preset`: An optional preset object that can be used to override the default behavior. The function first tries to determine the target endpoint based on the preset object. If a valid endpoint is found, it is used as the target endpoint. If not, the function tries to retrieve the last conversation setup from the local storage and uses its endpoint if it is valid. If neither the preset nor the local storage contains a valid endpoint, the function falls back to a default endpoint. Once the target endpoint is determined, * fix(utils): remove console.error statement in buildDefaultConversation function fix(schemas): add default values for catch blocks in openAISchema, googleSchema, bingAISchema, anthropicSchema, chatGPTBrowserSchema, and gptPluginsSchema * fix: endpoint not changing on change of preset from other endpoint, wip: refactor * refactor: preset items to TSX * refactor: convert resetConvo to TS * refactor(getDefaultConversation.ts): move defaultEndpoints array to the top of the file for better readability refactor(getDefaultConversation.ts): extract getDefaultEndpoint function for better code organization and reusability * feat(svg): add ContinueIcon component feat(svg): add RegenerateIcon component feat(svg): add ContinueIcon and RegenerateIcon components to index.ts * feat(Button.tsx): add onClick and className props to Button component feat(GenerationButtons.tsx): add logic to display Regenerate or StopGenerating button based on isSubmitting and messages feat(Regenerate.tsx): create Regenerate component with RegenerateIcon and handleRegenerate function feat(StopGenerating.tsx): create StopGenerating component with StopGeneratingIcon and handleStopGenerating function * fix(TextChat.jsx): reorder imports and variables for better readability fix(TextChat.jsx): fix typo in condition for isNotAppendable variable fix(TextChat.jsx): remove unused handleStopGenerating function fix(ContinueIcon.tsx): remove unnecessary closing tags for polygon elements fix(useMessageHandler.ts): add missing type annotations for handleStopGenerating and handleRegenerate functions fix(useMessageHandler.ts): remove unused variables in return statement * fix(getDefaultConversation.ts): refactor code to use getLocalStorageItems function feat(getLocalStorageItems.ts): add utility function to retrieve items from local storage * fix(OpenAIClient.js): add support for streaming result in sendCompletion method feat(OpenAIClient.js): add finish_reason metadata to opts in sendCompletion method feat(Message.js): add finish_reason field to Message model feat(messageSchema.js): add finish_reason field to messageSchema feat(openAI.js): parse chatGptLabel and promptPrefix from req.body and pass rest of the modelOptions to endpointOption feat(openAI.js): add addMetadata function to store metadata in ask function feat(openAI.js): add metadata to response if available feat(schemas.ts): add finish_reason field to tMessageSchema * feat(types.ts): add TOnClick and TGenButtonProps types for button components feat(Continue.tsx): create Continue component for generating button feat(GenerationButtons.tsx): update GenerationButtons component to use Continue component feat(Regenerate.tsx): create Regenerate component for regenerating button feat(Stop.tsx): create Stop component for stop generating button * feat(MessageHandler.jsx): add MessageHandler component to handle messages and conversations fix(Root.jsx): fix import paths for Nav and MessageHandler components * feat(useMessageHandler.ts): add support for generation parameter in ask function feat(useMessageHandler.ts): add support for isEdited parameter in ask function feat(useMessageHandler.ts): add support for continueGeneration function fix(createPayload.ts): replace endpoint URL when isEdited parameter is true * chore(client): set skipLibCheck to true in tsconfig.json * fix(useMessageHandler.ts): remove unused clientId variable fix(schemas.ts): make clientId field in tMessageSchema nullable and optional * wip: edit route for continue generation * refactor(api): move handlers to root of routes dir * fix(useMessageHandler.ts): initialize currentMessages to an empty array if messages is null fix(useMessageHandler.ts): update initialResponse text to use responseText variable fix(useMessageHandler.ts): update setMessages logic for isRegenerate case fix(MessageHandler.jsx): update setMessages logic for cancelHandler, createdHandler, and finalHandler * fix(schemas.ts): make createdAt and updatedAt fields optional and set default values using new Date().toISOString() fix(schemas.ts): change type annotation of TMessage from infer to input * refactor(useMessageHandler.ts): rename AskProps type to TAskProps refactor(useMessageHandler.ts): remove generation property from ask function arguments refactor(useMessageHandler.ts): use nullish coalescing operator (??) instead of logical OR (||) refactor(useMessageHandler.ts): pass the responseMessageId to message prop of submission * fix(BaseClient.js): use nullish coalescing operator (??) instead of logical OR (||) for default values * fix(BaseClient.js): fix responseMessageId assignment in handleStartMethods method feat(BaseClient.js): add support for isEdited flag in sendMessage method feat(BaseClient.js): add generation to responseMessage text in sendMessage method * fix(openAI.js): remove unused imports and commented out code feat(openAI.js): add support for generation parameter in request body fix(openAI.js): remove console.log statement fix(openAI.js): remove unused variables and parameters fix(openAI.js): update response text in case of error fix(openAI.js): handle error and abort message in case of error fix(handlers.js): add generation parameter to createOnProgress function fix(useMessageHandler.ts): update responseText variable to use generation parameter * refactor(api/middleware): move inside server dir * refactor: add endpoint specific, modular functions to build options and initialize clients, create server/utils, move middleware, separate utils into api general utils and server specific utils * fix(abortMiddleware.js): import getConvo and getConvoTitle functions from models feat(abortMiddleware.js): add abortAsk function to abortController to handle aborting of requests fix(openAI.js): import buildOptions and initializeClient functions from endpoints/openAI refactor(openAI.js): use getAbortData function to get data for abortAsk function * refactor: move endpoint specific logic to an endpoints dir * refactor(PluginService.js): fix import path for encrypt and decrypt functions in PluginService.js * feat(openAI): add new endpoint for adding a title to a conversation - Added a new file `addTitle.js` in the `api/server/routes/endpoints/openAI` directory. - The `addTitle.js` file exports a function `addTitle` that takes in request parameters and performs the following actions: - If the `parentMessageId` is `'00000000-0000-0000-0000-000000000000'` and `newConvo` is true, it proceeds with the following steps: - Calls the `titleConvo` function from the `titleConvo` module, passing in the necessary parameters. - Calls the `saveConvo` function from the `saveConvo` module, passing in the user ID and conversation details. - Updated the `index.js` file in the `api/server/routes/endpoints/openAI` directory to export the `addTitle` function. - This change adds * fix(abortMiddleware.js): remove console.log statement refactor(gptPlugins.js): update imports and function parameters feat(gptPlugins.js): add support for abortController and getAbortData refactor(openAI.js): update imports and function parameters feat(openAI.js): add support for abortController and getAbortData fix(openAI.js): refactor code to use modularized functions and middleware fix(buildOptions.js): refactor code to use destructuring and update variable names * refactor(askChatGPTBrowser.js, bingAI.js, google.js): remove duplicate code for setting response headers feat(askChatGPTBrowser.js, bingAI.js, google.js): add setHeaders middleware to set response headers * feat(middleware): validateEndpoint, refactor buildOption to only be concerned of endpointOption * fix(abortMiddleware.js): add 'finish_reason' property with value 'incomplete' to responseMessage object fix(abortMessage.js): remove console.log statement for aborted message fix(handlers.js): modify tokens assignment to handle empty generation string and trailing space * fix(BaseClient.js): import addSpaceIfNeeded function from server/utils fix(BaseClient.js): add space before generation in text property fix(index.js): remove getCitations and citeText exports feat(buildEndpointOption.js): add buildEndpointOption middleware fix(index.js): import buildEndpointOption middleware fix(anthropic.js): remove buildOptions function and use endpointOption from req.body fix(gptPlugins.js): remove buildOptions function and use endpointOption from req.body fix(openAI.js): remove buildOptions function and use endpointOption from req.body feat(utils): add citations.js and handleText.js modules fix(utils): fix import statements in index.js module * refactor(gptPlugins.js): use getResponseSender function from librechat-data-provider * feat(gptPlugins): complete 'continue generating' * wip: anthropic continue regen * feat(middleware): add validateRegistration middleware A new middleware function called `validateRegistration` has been added to the list of exported middleware functions in `index.js`. This middleware is responsible for validating registration data before allowing the registration process to proceed. * feat(Anthropic): complete continue regen * chore: add librechat-data-provider to api/package.json * fix(ci): backend-review will mock meilisearch, also installs data-provider as now needed * chore(ci): remove unneeded SEARCH env var * style(GenerationButtons): make text shorter for sake of space economy, even though this diverges from chat.openai.com * style(GenerationButtons/ScrollToBottom): adjust visibility/position based on screen size * chore(client): 'Editting' typo * feat(GenerationButtons.tsx): add support for endpoint prop in GenerationButtons component feat(OptionsBar.tsx): pass endpoint prop to GenerationButtons component feat(useGenerations.ts): create useGenerations hook to handle generation logic fix(schemas.ts): add searchResult field to tMessageSchema * refactor(HoverButtons): convert to TSX and utilize new useGenerations hook * fix(abortMiddleware): handle error with res headers set, or abortController not found, to ensure proper API error is sent to the client, chore(BaseClient): remove console log for onStart message meant for debugging * refactor(api): remove librechat-data-provider dep for now as it complicates deployed docker build stage, re-use code in CJS, located in server/endpoints/schemas * chore: remove console.logs from test files * ci: add backend tests for AnthropicClient, focusing on new buildMessages logic * refactor(FakeClient): use actual BaseClient sendMessage method for testing * test(BaseClient.test.js): add test for loading chat history test(BaseClient.test.js): add test for sendMessage logic with isEdited flag * fix(buildEndpointOption.js): add support for azureOpenAI in buildFunction object wip(endpoints.js): fetch Azure models from Azure OpenAI API if opts.azure is true * fix(Button.tsx): add data-testid attribute to button component fix(SelectDropDown.tsx): add data-testid attribute to Listbox.Button component fix(messages.spec.ts): add waitForServerStream function to consolidate logic for awaiting the server response feat(messages.spec.ts): add test for stopping and continuing message and improve browser/page context order and closing * refactor(onProgress): speed up time to save initial message for editable routes * chore: disable AI message editing (for now), was accidentally allowed * refactor: ensure continue is only supported for latest message style: improve styling in dark mode and across all hover buttons/icons, including making edit icon for AI invisible (for now) * fix: add test id to generation buttons so they never resolve to 2+ items * chore(package.json): add 'packages/' to the list of ignored directories chore(data-provider/package.json): bump version to 0.1.5
2023-08-17 12:50:05 -04:00
feat: Accurate Token Usage Tracking & Optional Balance (#1018) * refactor(Chains/llms): allow passing callbacks * refactor(BaseClient): accurately count completion tokens as generation only * refactor(OpenAIClient): remove unused getTokenCountForResponse, pass streaming var and callbacks in initializeLLM * wip: summary prompt tokens * refactor(summarizeMessages): new cut-off strategy that generates a better summary by adding context from beginning, truncating the middle, and providing the end wip: draft out relevant providers and variables for token tracing * refactor(createLLM): make streaming prop false by default * chore: remove use of getTokenCountForResponse * refactor(agents): use BufferMemory as ConversationSummaryBufferMemory token usage not easy to trace * chore: remove passing of streaming prop, also console log useful vars for tracing * feat: formatFromLangChain helper function to count tokens for ChatModelStart * refactor(initializeLLM): add role for LLM tracing * chore(formatFromLangChain): update JSDoc * feat(formatMessages): formats langChain messages into OpenAI payload format * chore: install openai-chat-tokens * refactor(formatMessage): optimize conditional langChain logic fix(formatFromLangChain): fix destructuring * feat: accurate prompt tokens for ChatModelStart before generation * refactor(handleChatModelStart): move to callbacks dir, use factory function * refactor(initializeLLM): rename 'role' to 'context' * feat(Balance/Transaction): new schema/models for tracking token spend refactor(Key): factor out model export to separate file * refactor(initializeClient): add req,res objects to client options * feat: add-balance script to add to an existing users' token balance refactor(Transaction): use multiplier map/function, return balance update * refactor(Tx): update enum for tokenType, return 1 for multiplier if no map match * refactor(Tx): add fair fallback value multiplier incase the config result is undefined * refactor(Balance): rename 'tokens' to 'tokenCredits' * feat: balance check, add tx.js for new tx-related methods and tests * chore(summaryPrompts): update prompt token count * refactor(callbacks): pass req, res wip: check balance * refactor(Tx): make convoId a String type, fix(calculateTokenValue) * refactor(BaseClient): add conversationId as client prop when assigned * feat(RunManager): track LLM runs with manager, track token spend from LLM, refactor(OpenAIClient): use RunManager to create callbacks, pass user prop to langchain api calls * feat(spendTokens): helper to spend prompt/completion tokens * feat(checkBalance): add helper to check, log, deny request if balance doesn't have enough funds refactor(Balance): static check method to return object instead of boolean now wip(OpenAIClient): implement use of checkBalance * refactor(initializeLLM): add token buffer to assure summary isn't generated when subsequent payload is too large refactor(OpenAIClient): add checkBalance refactor(createStartHandler): add checkBalance * chore: remove prompt and completion token logging from route handler * chore(spendTokens): add JSDoc * feat(logTokenCost): record transactions for basic api calls * chore(ask/edit): invoke getResponseSender only once per API call * refactor(ask/edit): pass promptTokens to getIds and include in abort data * refactor(getIds -> getReqData): rename function * refactor(Tx): increase value if incomplete message * feat: record tokenUsage when message is aborted * refactor: subtract tokens when payload includes function_call * refactor: add namespace for token_balance * fix(spendTokens): only execute if corresponding token type amounts are defined * refactor(checkBalance): throws Error if not enough token credits * refactor(runTitleChain): pass and use signal, spread object props in create helpers, and use 'call' instead of 'run' * fix(abortMiddleware): circular dependency, and default to empty string for completionTokens * fix: properly cancel title requests when there isn't enough tokens to generate * feat(predictNewSummary): custom chain for summaries to allow signal passing refactor(summaryBuffer): use new custom chain * feat(RunManager): add getRunByConversationId method, refactor: remove run and throw llm error on handleLLMError * refactor(createStartHandler): if summary, add error details to runs * fix(OpenAIClient): support aborting from summarization & showing error to user refactor(summarizeMessages): remove unnecessary operations counting summaryPromptTokens and note for alternative, pass signal to summaryBuffer * refactor(logTokenCost -> recordTokenUsage): rename * refactor(checkBalance): include promptTokens in errorMessage * refactor(checkBalance/spendTokens): move to models dir * fix(createLanguageChain): correctly pass config * refactor(initializeLLM/title): add tokenBuffer of 150 for balance check * refactor(openAPIPlugin): pass signal and memory, filter functions by the one being called * refactor(createStartHandler): add error to run if context is plugins as well * refactor(RunManager/handleLLMError): throw error immediately if plugins, don't remove run * refactor(PluginsClient): pass memory and signal to tools, cleanup error handling logic * chore: use absolute equality for addTitle condition * refactor(checkBalance): move checkBalance to execute after userMessage and tokenCounts are saved, also make conditional * style: icon changes to match official * fix(BaseClient): getTokenCountForResponse -> getTokenCount * fix(formatLangChainMessages): add kwargs as fallback prop from lc_kwargs, update JSDoc * refactor(Tx.create): does not update balance if CHECK_BALANCE is not enabled * fix(e2e/cleanUp): cleanup new collections, import all model methods from index * fix(config/add-balance): add uncaughtException listener * fix: circular dependency * refactor(initializeLLM/checkBalance): append new generations to errorMessage if cost exceeds balance * fix(handleResponseMessage): only record token usage in this method if not error and completion is not skipped * fix(createStartHandler): correct condition for generations * chore: bump postcss due to moderate severity vulnerability * chore: bump zod due to low severity vulnerability * chore: bump openai & data-provider version * feat(types): OpenAI Message types * chore: update bun lockfile * refactor(CodeBlock): add error block formatting * refactor(utils/Plugin): factor out formatJSON and cn to separate files (json.ts and cn.ts), add extractJSON * chore(logViolation): delete user_id after error is logged * refactor(getMessageError -> Error): change to React.FC, add token_balance handling, use extractJSON to determine JSON instead of regex * fix(DALL-E): use latest openai SDK * chore: reorganize imports, fix type issue * feat(server): add balance route * fix(api/models): add auth * feat(data-provider): /api/balance query * feat: show balance if checking is enabled, refetch on final message or error * chore: update docs, .env.example with token_usage info, add balance script command * fix(Balance): fallback to empty obj for balance query * style: slight adjustment of balance element * docs(token_usage): add PR notes
2023-10-05 18:34:10 -04:00
const initializeClient = async ({ req, res, endpointOption }) => {
feat: ConversationSummaryBufferMemory (#973) * refactor: pass model in message edit payload, use encoder in standalone util function * feat: add summaryBuffer helper * refactor(api/messages): use new countTokens helper and add auth middleware at top * wip: ConversationSummaryBufferMemory * refactor: move pre-generation helpers to prompts dir * chore: remove console log * chore: remove test as payload will no longer carry tokenCount * chore: update getMessagesWithinTokenLimit JSDoc * refactor: optimize getMessagesForConversation and also break on summary, feat(ci): getMessagesForConversation tests * refactor(getMessagesForConvo): count '00000000-0000-0000-0000-000000000000' as root message * chore: add newer model to token map * fix: condition was point to prop of array instead of message prop * refactor(BaseClient): use object for refineMessages param, rename 'summary' to 'summaryMessage', add previous_summary refactor(getMessagesWithinTokenLimit): replace text and tokenCount if should summarize, summary, and summaryTokenCount are present fix/refactor(handleContextStrategy): use the right comparison length for context diff, and replace payload first message when a summary is present * chore: log previous_summary if debugging * refactor(formatMessage): assume if role is defined that it's a valid value * refactor(getMessagesWithinTokenLimit): remove summary logic refactor(handleContextStrategy): add usePrevSummary logic in case only summary was pruned refactor(loadHistory): initial message query will return all ordered messages but keep track of the latest summary refactor(getMessagesForConversation): use object for single param, edit jsdoc, edit all files using the method refactor(ChatGPTClient): order messages before buildPrompt is called, TODO: add convoSumBuffMemory logic * fix: undefined handling and summarizing only when shouldRefineContext is true * chore(BaseClient): fix test results omitting system role for summaries and test edge case * chore: export summaryBuffer from index file * refactor(OpenAIClient/BaseClient): move refineMessages to subclass, implement LLM initialization for summaryBuffer * feat: add OPENAI_SUMMARIZE to enable summarizing, refactor: rename client prop 'shouldRefineContext' to 'shouldSummarize', change contextStrategy value to 'summarize' from 'refine' * refactor: rename refineMessages method to summarizeMessages for clarity * chore: clarify summary future intent in .env.example * refactor(initializeLLM): handle case for either 'model' or 'modelName' being passed * feat(gptPlugins): enable summarization for plugins * refactor(gptPlugins): utilize new initializeLLM method and formatting methods for messages, use payload array for currentMessages and assign pastMessages sooner * refactor(agents): use ConversationSummaryBufferMemory for both agent types * refactor(formatMessage): optimize original method for langchain, add helper function for langchain messages, add JSDocs and tests * refactor(summaryBuffer): add helper to createSummaryBufferMemory, and use new formatting helpers * fix: forgot to spread formatMessages also took opportunity to pluralize filename * refactor: pass memory to tools, namely openapi specs. not used and may never be used by new method but added for testing * ci(formatMessages): add more exhaustive checks for langchain messages * feat: add debug env var for OpenAI * chore: delete unnecessary comments * chore: add extra note about summary feature * fix: remove tokenCount from payload instructions * fix: test fail * fix: only pass instructions to payload when defined or not empty object * refactor: fromPromptMessages is deprecated, use renamed method fromMessages * refactor: use 'includes' instead of 'startsWith' for extended OpenRouter compatibility * fix(PluginsClient.buildPromptBody): handle undefined message strings * chore: log langchain titling error * feat: getModelMaxTokens helper * feat: tokenSplit helper * feat: summary prompts updated * fix: optimize _CUT_OFF_SUMMARIZER prompt * refactor(summaryBuffer): use custom summary prompt, allow prompt to be passed, pass humanPrefix and aiPrefix to memory, along with any future variables, rename messagesToRefine to context * fix(summaryBuffer): handle edge case where messagesToRefine exceeds summary context, refactor(BaseClient): allow custom maxContextTokens to be passed to getMessagesWithinTokenLimit, add defined check before unshifting summaryMessage, update shouldSummarize based on this refactor(OpenAIClient): use getModelMaxTokens, use cut-off message method for summary if no messages were left after pruning * fix(handleContextStrategy): handle case where incoming prompt is bigger than model context * chore: rename refinedContent to splitText * chore: remove unnecessary debug log
2023-09-26 21:02:28 -04:00
const {
PROXY,
OPENAI_API_KEY,
AZURE_API_KEY,
OPENAI_REVERSE_PROXY,
OPENAI_SUMMARIZE,
DEBUG_OPENAI,
} = process.env;
refactor: Encrypt & Expire User Provided Keys, feat: Rate Limiting (#874) * docs: make_your_own.md formatting fix for mkdocs * feat: add express-mongo-sanitize feat: add login/registration rate limiting * chore: remove unnecessary console log * wip: remove token handling from localStorage to encrypted DB solution * refactor: minor change to UserService * fix mongo query and add keys route to server * fix backend controllers and simplify schema/crud * refactor: rename token to key to separate from access/refresh tokens, setTokenDialog -> setKeyDialog * refactor(schemas): TEndpointOption token -> key * refactor(api): use new encrypted key retrieval system * fix(SetKeyDialog): fix key prop error * fix(abortMiddleware): pass random UUID if messageId is not generated yet for proper error display on frontend * fix(getUserKey): wrong prop passed in arg, adds error handling * fix: prevent message without conversationId from saving to DB, prevents branching on the frontend to a new top-level branch * refactor: change wording of multiple display messages * refactor(checkExpiry -> checkUserKeyExpiry): move to UserService file * fix: type imports from common * refactor(SubmitButton): convert to TS * refactor(key.ts): change localStorage map key name * refactor: add new custom tailwind classes to better match openAI colors * chore: remove unnecessary warning and catch ScreenShot error * refactor: move userKey frontend logic to hooks and remove use of localStorage and instead query the DB * refactor: invalidate correct query key, memoize userKey hook, conditionally render SetKeyDialog to avoid unnecessary calls, refactor SubmitButton props and useEffect for showing 'provide key first' * fix(SetKeyDialog): use enum-like object for expiry values feat(Dropdown): add optionsClassName to dynamically change dropdown options container classes * fix: handle edge case where user had provided a key but the server changes to env variable for keys * refactor(OpenAI/titleConvo): move titling to client to retain authorized credentials in message lifecycle for titling * fix(azure): handle user_provided keys correctly for azure * feat: send user Id to OpenAI to differentiate users in completion requests * refactor(OpenAI/titleConvo): adding tokens helps minimize LLM from using the language in title response * feat: add delete endpoint for keys * chore: remove throttling of title * feat: add 'Data controls' to Settings, add 'Revoke' keys feature in Key Dialog and Data controls * refactor: reorganize PluginsClient files in langchain format * feat: use langchain for titling convos * chore: cleanup titling convo, with fallback to original method, escape braces, use only snippet for language detection * refactor: move helper functions to appropriate langchain folders for reusability * fix: userProvidesKey handling for gptPlugins * fix: frontend handling of plugins key * chore: cleanup logging and ts-ignore SSE * fix: forwardRef misuse in DangerButton * fix(GoogleConfig/FileUpload): localize errors and simplify validation with zod * fix: cleanup google logging and fix user provided key handling * chore: remove titling from google * chore: removing logging from browser endpoint * wip: fix menu flicker * feat: useLocalStorage hook * feat: add Tooltip for UI * refactor(EndpointMenu): utilize Tooltip and useLocalStorage, remove old 'New Chat' slide-over * fix(e2e): use testId for endpoint menu trigger * chore: final touches to EndpointMenu before future refactor to declutter component * refactor(localization): change select endpoint to open menu and add translations * chore: add final prop to error message response * ci: minor edits to facilitate testing * ci: new e2e test which tests for new key setting/revoking features
2023-09-06 10:46:27 -04:00
const { key: expiresAt, endpoint } = req.body;
feat: ConversationSummaryBufferMemory (#973) * refactor: pass model in message edit payload, use encoder in standalone util function * feat: add summaryBuffer helper * refactor(api/messages): use new countTokens helper and add auth middleware at top * wip: ConversationSummaryBufferMemory * refactor: move pre-generation helpers to prompts dir * chore: remove console log * chore: remove test as payload will no longer carry tokenCount * chore: update getMessagesWithinTokenLimit JSDoc * refactor: optimize getMessagesForConversation and also break on summary, feat(ci): getMessagesForConversation tests * refactor(getMessagesForConvo): count '00000000-0000-0000-0000-000000000000' as root message * chore: add newer model to token map * fix: condition was point to prop of array instead of message prop * refactor(BaseClient): use object for refineMessages param, rename 'summary' to 'summaryMessage', add previous_summary refactor(getMessagesWithinTokenLimit): replace text and tokenCount if should summarize, summary, and summaryTokenCount are present fix/refactor(handleContextStrategy): use the right comparison length for context diff, and replace payload first message when a summary is present * chore: log previous_summary if debugging * refactor(formatMessage): assume if role is defined that it's a valid value * refactor(getMessagesWithinTokenLimit): remove summary logic refactor(handleContextStrategy): add usePrevSummary logic in case only summary was pruned refactor(loadHistory): initial message query will return all ordered messages but keep track of the latest summary refactor(getMessagesForConversation): use object for single param, edit jsdoc, edit all files using the method refactor(ChatGPTClient): order messages before buildPrompt is called, TODO: add convoSumBuffMemory logic * fix: undefined handling and summarizing only when shouldRefineContext is true * chore(BaseClient): fix test results omitting system role for summaries and test edge case * chore: export summaryBuffer from index file * refactor(OpenAIClient/BaseClient): move refineMessages to subclass, implement LLM initialization for summaryBuffer * feat: add OPENAI_SUMMARIZE to enable summarizing, refactor: rename client prop 'shouldRefineContext' to 'shouldSummarize', change contextStrategy value to 'summarize' from 'refine' * refactor: rename refineMessages method to summarizeMessages for clarity * chore: clarify summary future intent in .env.example * refactor(initializeLLM): handle case for either 'model' or 'modelName' being passed * feat(gptPlugins): enable summarization for plugins * refactor(gptPlugins): utilize new initializeLLM method and formatting methods for messages, use payload array for currentMessages and assign pastMessages sooner * refactor(agents): use ConversationSummaryBufferMemory for both agent types * refactor(formatMessage): optimize original method for langchain, add helper function for langchain messages, add JSDocs and tests * refactor(summaryBuffer): add helper to createSummaryBufferMemory, and use new formatting helpers * fix: forgot to spread formatMessages also took opportunity to pluralize filename * refactor: pass memory to tools, namely openapi specs. not used and may never be used by new method but added for testing * ci(formatMessages): add more exhaustive checks for langchain messages * feat: add debug env var for OpenAI * chore: delete unnecessary comments * chore: add extra note about summary feature * fix: remove tokenCount from payload instructions * fix: test fail * fix: only pass instructions to payload when defined or not empty object * refactor: fromPromptMessages is deprecated, use renamed method fromMessages * refactor: use 'includes' instead of 'startsWith' for extended OpenRouter compatibility * fix(PluginsClient.buildPromptBody): handle undefined message strings * chore: log langchain titling error * feat: getModelMaxTokens helper * feat: tokenSplit helper * feat: summary prompts updated * fix: optimize _CUT_OFF_SUMMARIZER prompt * refactor(summaryBuffer): use custom summary prompt, allow prompt to be passed, pass humanPrefix and aiPrefix to memory, along with any future variables, rename messagesToRefine to context * fix(summaryBuffer): handle edge case where messagesToRefine exceeds summary context, refactor(BaseClient): allow custom maxContextTokens to be passed to getMessagesWithinTokenLimit, add defined check before unshifting summaryMessage, update shouldSummarize based on this refactor(OpenAIClient): use getModelMaxTokens, use cut-off message method for summary if no messages were left after pruning * fix(handleContextStrategy): handle case where incoming prompt is bigger than model context * chore: rename refinedContent to splitText * chore: remove unnecessary debug log
2023-09-26 21:02:28 -04:00
const contextStrategy = isEnabled(OPENAI_SUMMARIZE) ? 'summarize' : null;
feat(GPT/Anthropic): Continue Regenerating & Generation Buttons (#808) * feat(useMessageHandler.js/ts): Refactor and add features to handle user messages, support multiple endpoints/models, generate placeholder responses, regeneration, and stopGeneration function fix(conversation.ts, buildTree.ts): Import TMessage type, handle null parentMessageId feat(schemas.ts): Update and add schemas for various AI services, add default values, optional fields, and endpoint-to-schema mapping, create parseConvo function chore(useMessageHandler.js, schemas.ts): Remove unused imports, variables, and chatGPT enum * wip: add generation buttons * refactor(cleanupPreset.ts): simplify cleanupPreset function refactor(getDefaultConversation.js): remove unused code and simplify getDefaultConversation function feat(utils): add getDefaultConversation function This commit adds a new utility function called `getDefaultConversation` to the `client/src/utils/getDefaultConversation.ts` file. This function is responsible for generating a default conversation object based on the provided parameters. The `getDefaultConversation` function takes in an object with the following properties: - `conversation`: The conversation object to be used as a base. - `endpointsConfig`: The configuration object containing information about the available endpoints. - `preset`: An optional preset object that can be used to override the default behavior. The function first tries to determine the target endpoint based on the preset object. If a valid endpoint is found, it is used as the target endpoint. If not, the function tries to retrieve the last conversation setup from the local storage and uses its endpoint if it is valid. If neither the preset nor the local storage contains a valid endpoint, the function falls back to a default endpoint. Once the target endpoint is determined, * fix(utils): remove console.error statement in buildDefaultConversation function fix(schemas): add default values for catch blocks in openAISchema, googleSchema, bingAISchema, anthropicSchema, chatGPTBrowserSchema, and gptPluginsSchema * fix: endpoint not changing on change of preset from other endpoint, wip: refactor * refactor: preset items to TSX * refactor: convert resetConvo to TS * refactor(getDefaultConversation.ts): move defaultEndpoints array to the top of the file for better readability refactor(getDefaultConversation.ts): extract getDefaultEndpoint function for better code organization and reusability * feat(svg): add ContinueIcon component feat(svg): add RegenerateIcon component feat(svg): add ContinueIcon and RegenerateIcon components to index.ts * feat(Button.tsx): add onClick and className props to Button component feat(GenerationButtons.tsx): add logic to display Regenerate or StopGenerating button based on isSubmitting and messages feat(Regenerate.tsx): create Regenerate component with RegenerateIcon and handleRegenerate function feat(StopGenerating.tsx): create StopGenerating component with StopGeneratingIcon and handleStopGenerating function * fix(TextChat.jsx): reorder imports and variables for better readability fix(TextChat.jsx): fix typo in condition for isNotAppendable variable fix(TextChat.jsx): remove unused handleStopGenerating function fix(ContinueIcon.tsx): remove unnecessary closing tags for polygon elements fix(useMessageHandler.ts): add missing type annotations for handleStopGenerating and handleRegenerate functions fix(useMessageHandler.ts): remove unused variables in return statement * fix(getDefaultConversation.ts): refactor code to use getLocalStorageItems function feat(getLocalStorageItems.ts): add utility function to retrieve items from local storage * fix(OpenAIClient.js): add support for streaming result in sendCompletion method feat(OpenAIClient.js): add finish_reason metadata to opts in sendCompletion method feat(Message.js): add finish_reason field to Message model feat(messageSchema.js): add finish_reason field to messageSchema feat(openAI.js): parse chatGptLabel and promptPrefix from req.body and pass rest of the modelOptions to endpointOption feat(openAI.js): add addMetadata function to store metadata in ask function feat(openAI.js): add metadata to response if available feat(schemas.ts): add finish_reason field to tMessageSchema * feat(types.ts): add TOnClick and TGenButtonProps types for button components feat(Continue.tsx): create Continue component for generating button feat(GenerationButtons.tsx): update GenerationButtons component to use Continue component feat(Regenerate.tsx): create Regenerate component for regenerating button feat(Stop.tsx): create Stop component for stop generating button * feat(MessageHandler.jsx): add MessageHandler component to handle messages and conversations fix(Root.jsx): fix import paths for Nav and MessageHandler components * feat(useMessageHandler.ts): add support for generation parameter in ask function feat(useMessageHandler.ts): add support for isEdited parameter in ask function feat(useMessageHandler.ts): add support for continueGeneration function fix(createPayload.ts): replace endpoint URL when isEdited parameter is true * chore(client): set skipLibCheck to true in tsconfig.json * fix(useMessageHandler.ts): remove unused clientId variable fix(schemas.ts): make clientId field in tMessageSchema nullable and optional * wip: edit route for continue generation * refactor(api): move handlers to root of routes dir * fix(useMessageHandler.ts): initialize currentMessages to an empty array if messages is null fix(useMessageHandler.ts): update initialResponse text to use responseText variable fix(useMessageHandler.ts): update setMessages logic for isRegenerate case fix(MessageHandler.jsx): update setMessages logic for cancelHandler, createdHandler, and finalHandler * fix(schemas.ts): make createdAt and updatedAt fields optional and set default values using new Date().toISOString() fix(schemas.ts): change type annotation of TMessage from infer to input * refactor(useMessageHandler.ts): rename AskProps type to TAskProps refactor(useMessageHandler.ts): remove generation property from ask function arguments refactor(useMessageHandler.ts): use nullish coalescing operator (??) instead of logical OR (||) refactor(useMessageHandler.ts): pass the responseMessageId to message prop of submission * fix(BaseClient.js): use nullish coalescing operator (??) instead of logical OR (||) for default values * fix(BaseClient.js): fix responseMessageId assignment in handleStartMethods method feat(BaseClient.js): add support for isEdited flag in sendMessage method feat(BaseClient.js): add generation to responseMessage text in sendMessage method * fix(openAI.js): remove unused imports and commented out code feat(openAI.js): add support for generation parameter in request body fix(openAI.js): remove console.log statement fix(openAI.js): remove unused variables and parameters fix(openAI.js): update response text in case of error fix(openAI.js): handle error and abort message in case of error fix(handlers.js): add generation parameter to createOnProgress function fix(useMessageHandler.ts): update responseText variable to use generation parameter * refactor(api/middleware): move inside server dir * refactor: add endpoint specific, modular functions to build options and initialize clients, create server/utils, move middleware, separate utils into api general utils and server specific utils * fix(abortMiddleware.js): import getConvo and getConvoTitle functions from models feat(abortMiddleware.js): add abortAsk function to abortController to handle aborting of requests fix(openAI.js): import buildOptions and initializeClient functions from endpoints/openAI refactor(openAI.js): use getAbortData function to get data for abortAsk function * refactor: move endpoint specific logic to an endpoints dir * refactor(PluginService.js): fix import path for encrypt and decrypt functions in PluginService.js * feat(openAI): add new endpoint for adding a title to a conversation - Added a new file `addTitle.js` in the `api/server/routes/endpoints/openAI` directory. - The `addTitle.js` file exports a function `addTitle` that takes in request parameters and performs the following actions: - If the `parentMessageId` is `'00000000-0000-0000-0000-000000000000'` and `newConvo` is true, it proceeds with the following steps: - Calls the `titleConvo` function from the `titleConvo` module, passing in the necessary parameters. - Calls the `saveConvo` function from the `saveConvo` module, passing in the user ID and conversation details. - Updated the `index.js` file in the `api/server/routes/endpoints/openAI` directory to export the `addTitle` function. - This change adds * fix(abortMiddleware.js): remove console.log statement refactor(gptPlugins.js): update imports and function parameters feat(gptPlugins.js): add support for abortController and getAbortData refactor(openAI.js): update imports and function parameters feat(openAI.js): add support for abortController and getAbortData fix(openAI.js): refactor code to use modularized functions and middleware fix(buildOptions.js): refactor code to use destructuring and update variable names * refactor(askChatGPTBrowser.js, bingAI.js, google.js): remove duplicate code for setting response headers feat(askChatGPTBrowser.js, bingAI.js, google.js): add setHeaders middleware to set response headers * feat(middleware): validateEndpoint, refactor buildOption to only be concerned of endpointOption * fix(abortMiddleware.js): add 'finish_reason' property with value 'incomplete' to responseMessage object fix(abortMessage.js): remove console.log statement for aborted message fix(handlers.js): modify tokens assignment to handle empty generation string and trailing space * fix(BaseClient.js): import addSpaceIfNeeded function from server/utils fix(BaseClient.js): add space before generation in text property fix(index.js): remove getCitations and citeText exports feat(buildEndpointOption.js): add buildEndpointOption middleware fix(index.js): import buildEndpointOption middleware fix(anthropic.js): remove buildOptions function and use endpointOption from req.body fix(gptPlugins.js): remove buildOptions function and use endpointOption from req.body fix(openAI.js): remove buildOptions function and use endpointOption from req.body feat(utils): add citations.js and handleText.js modules fix(utils): fix import statements in index.js module * refactor(gptPlugins.js): use getResponseSender function from librechat-data-provider * feat(gptPlugins): complete 'continue generating' * wip: anthropic continue regen * feat(middleware): add validateRegistration middleware A new middleware function called `validateRegistration` has been added to the list of exported middleware functions in `index.js`. This middleware is responsible for validating registration data before allowing the registration process to proceed. * feat(Anthropic): complete continue regen * chore: add librechat-data-provider to api/package.json * fix(ci): backend-review will mock meilisearch, also installs data-provider as now needed * chore(ci): remove unneeded SEARCH env var * style(GenerationButtons): make text shorter for sake of space economy, even though this diverges from chat.openai.com * style(GenerationButtons/ScrollToBottom): adjust visibility/position based on screen size * chore(client): 'Editting' typo * feat(GenerationButtons.tsx): add support for endpoint prop in GenerationButtons component feat(OptionsBar.tsx): pass endpoint prop to GenerationButtons component feat(useGenerations.ts): create useGenerations hook to handle generation logic fix(schemas.ts): add searchResult field to tMessageSchema * refactor(HoverButtons): convert to TSX and utilize new useGenerations hook * fix(abortMiddleware): handle error with res headers set, or abortController not found, to ensure proper API error is sent to the client, chore(BaseClient): remove console log for onStart message meant for debugging * refactor(api): remove librechat-data-provider dep for now as it complicates deployed docker build stage, re-use code in CJS, located in server/endpoints/schemas * chore: remove console.logs from test files * ci: add backend tests for AnthropicClient, focusing on new buildMessages logic * refactor(FakeClient): use actual BaseClient sendMessage method for testing * test(BaseClient.test.js): add test for loading chat history test(BaseClient.test.js): add test for sendMessage logic with isEdited flag * fix(buildEndpointOption.js): add support for azureOpenAI in buildFunction object wip(endpoints.js): fetch Azure models from Azure OpenAI API if opts.azure is true * fix(Button.tsx): add data-testid attribute to button component fix(SelectDropDown.tsx): add data-testid attribute to Listbox.Button component fix(messages.spec.ts): add waitForServerStream function to consolidate logic for awaiting the server response feat(messages.spec.ts): add test for stopping and continuing message and improve browser/page context order and closing * refactor(onProgress): speed up time to save initial message for editable routes * chore: disable AI message editing (for now), was accidentally allowed * refactor: ensure continue is only supported for latest message style: improve styling in dark mode and across all hover buttons/icons, including making edit icon for AI invisible (for now) * fix: add test id to generation buttons so they never resolve to 2+ items * chore(package.json): add 'packages/' to the list of ignored directories chore(data-provider/package.json): bump version to 0.1.5
2023-08-17 12:50:05 -04:00
const clientOptions = {
feat: ConversationSummaryBufferMemory (#973) * refactor: pass model in message edit payload, use encoder in standalone util function * feat: add summaryBuffer helper * refactor(api/messages): use new countTokens helper and add auth middleware at top * wip: ConversationSummaryBufferMemory * refactor: move pre-generation helpers to prompts dir * chore: remove console log * chore: remove test as payload will no longer carry tokenCount * chore: update getMessagesWithinTokenLimit JSDoc * refactor: optimize getMessagesForConversation and also break on summary, feat(ci): getMessagesForConversation tests * refactor(getMessagesForConvo): count '00000000-0000-0000-0000-000000000000' as root message * chore: add newer model to token map * fix: condition was point to prop of array instead of message prop * refactor(BaseClient): use object for refineMessages param, rename 'summary' to 'summaryMessage', add previous_summary refactor(getMessagesWithinTokenLimit): replace text and tokenCount if should summarize, summary, and summaryTokenCount are present fix/refactor(handleContextStrategy): use the right comparison length for context diff, and replace payload first message when a summary is present * chore: log previous_summary if debugging * refactor(formatMessage): assume if role is defined that it's a valid value * refactor(getMessagesWithinTokenLimit): remove summary logic refactor(handleContextStrategy): add usePrevSummary logic in case only summary was pruned refactor(loadHistory): initial message query will return all ordered messages but keep track of the latest summary refactor(getMessagesForConversation): use object for single param, edit jsdoc, edit all files using the method refactor(ChatGPTClient): order messages before buildPrompt is called, TODO: add convoSumBuffMemory logic * fix: undefined handling and summarizing only when shouldRefineContext is true * chore(BaseClient): fix test results omitting system role for summaries and test edge case * chore: export summaryBuffer from index file * refactor(OpenAIClient/BaseClient): move refineMessages to subclass, implement LLM initialization for summaryBuffer * feat: add OPENAI_SUMMARIZE to enable summarizing, refactor: rename client prop 'shouldRefineContext' to 'shouldSummarize', change contextStrategy value to 'summarize' from 'refine' * refactor: rename refineMessages method to summarizeMessages for clarity * chore: clarify summary future intent in .env.example * refactor(initializeLLM): handle case for either 'model' or 'modelName' being passed * feat(gptPlugins): enable summarization for plugins * refactor(gptPlugins): utilize new initializeLLM method and formatting methods for messages, use payload array for currentMessages and assign pastMessages sooner * refactor(agents): use ConversationSummaryBufferMemory for both agent types * refactor(formatMessage): optimize original method for langchain, add helper function for langchain messages, add JSDocs and tests * refactor(summaryBuffer): add helper to createSummaryBufferMemory, and use new formatting helpers * fix: forgot to spread formatMessages also took opportunity to pluralize filename * refactor: pass memory to tools, namely openapi specs. not used and may never be used by new method but added for testing * ci(formatMessages): add more exhaustive checks for langchain messages * feat: add debug env var for OpenAI * chore: delete unnecessary comments * chore: add extra note about summary feature * fix: remove tokenCount from payload instructions * fix: test fail * fix: only pass instructions to payload when defined or not empty object * refactor: fromPromptMessages is deprecated, use renamed method fromMessages * refactor: use 'includes' instead of 'startsWith' for extended OpenRouter compatibility * fix(PluginsClient.buildPromptBody): handle undefined message strings * chore: log langchain titling error * feat: getModelMaxTokens helper * feat: tokenSplit helper * feat: summary prompts updated * fix: optimize _CUT_OFF_SUMMARIZER prompt * refactor(summaryBuffer): use custom summary prompt, allow prompt to be passed, pass humanPrefix and aiPrefix to memory, along with any future variables, rename messagesToRefine to context * fix(summaryBuffer): handle edge case where messagesToRefine exceeds summary context, refactor(BaseClient): allow custom maxContextTokens to be passed to getMessagesWithinTokenLimit, add defined check before unshifting summaryMessage, update shouldSummarize based on this refactor(OpenAIClient): use getModelMaxTokens, use cut-off message method for summary if no messages were left after pruning * fix(handleContextStrategy): handle case where incoming prompt is bigger than model context * chore: rename refinedContent to splitText * chore: remove unnecessary debug log
2023-09-26 21:02:28 -04:00
debug: isEnabled(DEBUG_OPENAI),
contextStrategy,
refactor: Encrypt & Expire User Provided Keys, feat: Rate Limiting (#874) * docs: make_your_own.md formatting fix for mkdocs * feat: add express-mongo-sanitize feat: add login/registration rate limiting * chore: remove unnecessary console log * wip: remove token handling from localStorage to encrypted DB solution * refactor: minor change to UserService * fix mongo query and add keys route to server * fix backend controllers and simplify schema/crud * refactor: rename token to key to separate from access/refresh tokens, setTokenDialog -> setKeyDialog * refactor(schemas): TEndpointOption token -> key * refactor(api): use new encrypted key retrieval system * fix(SetKeyDialog): fix key prop error * fix(abortMiddleware): pass random UUID if messageId is not generated yet for proper error display on frontend * fix(getUserKey): wrong prop passed in arg, adds error handling * fix: prevent message without conversationId from saving to DB, prevents branching on the frontend to a new top-level branch * refactor: change wording of multiple display messages * refactor(checkExpiry -> checkUserKeyExpiry): move to UserService file * fix: type imports from common * refactor(SubmitButton): convert to TS * refactor(key.ts): change localStorage map key name * refactor: add new custom tailwind classes to better match openAI colors * chore: remove unnecessary warning and catch ScreenShot error * refactor: move userKey frontend logic to hooks and remove use of localStorage and instead query the DB * refactor: invalidate correct query key, memoize userKey hook, conditionally render SetKeyDialog to avoid unnecessary calls, refactor SubmitButton props and useEffect for showing 'provide key first' * fix(SetKeyDialog): use enum-like object for expiry values feat(Dropdown): add optionsClassName to dynamically change dropdown options container classes * fix: handle edge case where user had provided a key but the server changes to env variable for keys * refactor(OpenAI/titleConvo): move titling to client to retain authorized credentials in message lifecycle for titling * fix(azure): handle user_provided keys correctly for azure * feat: send user Id to OpenAI to differentiate users in completion requests * refactor(OpenAI/titleConvo): adding tokens helps minimize LLM from using the language in title response * feat: add delete endpoint for keys * chore: remove throttling of title * feat: add 'Data controls' to Settings, add 'Revoke' keys feature in Key Dialog and Data controls * refactor: reorganize PluginsClient files in langchain format * feat: use langchain for titling convos * chore: cleanup titling convo, with fallback to original method, escape braces, use only snippet for language detection * refactor: move helper functions to appropriate langchain folders for reusability * fix: userProvidesKey handling for gptPlugins * fix: frontend handling of plugins key * chore: cleanup logging and ts-ignore SSE * fix: forwardRef misuse in DangerButton * fix(GoogleConfig/FileUpload): localize errors and simplify validation with zod * fix: cleanup google logging and fix user provided key handling * chore: remove titling from google * chore: removing logging from browser endpoint * wip: fix menu flicker * feat: useLocalStorage hook * feat: add Tooltip for UI * refactor(EndpointMenu): utilize Tooltip and useLocalStorage, remove old 'New Chat' slide-over * fix(e2e): use testId for endpoint menu trigger * chore: final touches to EndpointMenu before future refactor to declutter component * refactor(localization): change select endpoint to open menu and add translations * chore: add final prop to error message response * ci: minor edits to facilitate testing * ci: new e2e test which tests for new key setting/revoking features
2023-09-06 10:46:27 -04:00
reverseProxyUrl: OPENAI_REVERSE_PROXY ?? null,
proxy: PROXY ?? null,
feat: Accurate Token Usage Tracking & Optional Balance (#1018) * refactor(Chains/llms): allow passing callbacks * refactor(BaseClient): accurately count completion tokens as generation only * refactor(OpenAIClient): remove unused getTokenCountForResponse, pass streaming var and callbacks in initializeLLM * wip: summary prompt tokens * refactor(summarizeMessages): new cut-off strategy that generates a better summary by adding context from beginning, truncating the middle, and providing the end wip: draft out relevant providers and variables for token tracing * refactor(createLLM): make streaming prop false by default * chore: remove use of getTokenCountForResponse * refactor(agents): use BufferMemory as ConversationSummaryBufferMemory token usage not easy to trace * chore: remove passing of streaming prop, also console log useful vars for tracing * feat: formatFromLangChain helper function to count tokens for ChatModelStart * refactor(initializeLLM): add role for LLM tracing * chore(formatFromLangChain): update JSDoc * feat(formatMessages): formats langChain messages into OpenAI payload format * chore: install openai-chat-tokens * refactor(formatMessage): optimize conditional langChain logic fix(formatFromLangChain): fix destructuring * feat: accurate prompt tokens for ChatModelStart before generation * refactor(handleChatModelStart): move to callbacks dir, use factory function * refactor(initializeLLM): rename 'role' to 'context' * feat(Balance/Transaction): new schema/models for tracking token spend refactor(Key): factor out model export to separate file * refactor(initializeClient): add req,res objects to client options * feat: add-balance script to add to an existing users' token balance refactor(Transaction): use multiplier map/function, return balance update * refactor(Tx): update enum for tokenType, return 1 for multiplier if no map match * refactor(Tx): add fair fallback value multiplier incase the config result is undefined * refactor(Balance): rename 'tokens' to 'tokenCredits' * feat: balance check, add tx.js for new tx-related methods and tests * chore(summaryPrompts): update prompt token count * refactor(callbacks): pass req, res wip: check balance * refactor(Tx): make convoId a String type, fix(calculateTokenValue) * refactor(BaseClient): add conversationId as client prop when assigned * feat(RunManager): track LLM runs with manager, track token spend from LLM, refactor(OpenAIClient): use RunManager to create callbacks, pass user prop to langchain api calls * feat(spendTokens): helper to spend prompt/completion tokens * feat(checkBalance): add helper to check, log, deny request if balance doesn't have enough funds refactor(Balance): static check method to return object instead of boolean now wip(OpenAIClient): implement use of checkBalance * refactor(initializeLLM): add token buffer to assure summary isn't generated when subsequent payload is too large refactor(OpenAIClient): add checkBalance refactor(createStartHandler): add checkBalance * chore: remove prompt and completion token logging from route handler * chore(spendTokens): add JSDoc * feat(logTokenCost): record transactions for basic api calls * chore(ask/edit): invoke getResponseSender only once per API call * refactor(ask/edit): pass promptTokens to getIds and include in abort data * refactor(getIds -> getReqData): rename function * refactor(Tx): increase value if incomplete message * feat: record tokenUsage when message is aborted * refactor: subtract tokens when payload includes function_call * refactor: add namespace for token_balance * fix(spendTokens): only execute if corresponding token type amounts are defined * refactor(checkBalance): throws Error if not enough token credits * refactor(runTitleChain): pass and use signal, spread object props in create helpers, and use 'call' instead of 'run' * fix(abortMiddleware): circular dependency, and default to empty string for completionTokens * fix: properly cancel title requests when there isn't enough tokens to generate * feat(predictNewSummary): custom chain for summaries to allow signal passing refactor(summaryBuffer): use new custom chain * feat(RunManager): add getRunByConversationId method, refactor: remove run and throw llm error on handleLLMError * refactor(createStartHandler): if summary, add error details to runs * fix(OpenAIClient): support aborting from summarization & showing error to user refactor(summarizeMessages): remove unnecessary operations counting summaryPromptTokens and note for alternative, pass signal to summaryBuffer * refactor(logTokenCost -> recordTokenUsage): rename * refactor(checkBalance): include promptTokens in errorMessage * refactor(checkBalance/spendTokens): move to models dir * fix(createLanguageChain): correctly pass config * refactor(initializeLLM/title): add tokenBuffer of 150 for balance check * refactor(openAPIPlugin): pass signal and memory, filter functions by the one being called * refactor(createStartHandler): add error to run if context is plugins as well * refactor(RunManager/handleLLMError): throw error immediately if plugins, don't remove run * refactor(PluginsClient): pass memory and signal to tools, cleanup error handling logic * chore: use absolute equality for addTitle condition * refactor(checkBalance): move checkBalance to execute after userMessage and tokenCounts are saved, also make conditional * style: icon changes to match official * fix(BaseClient): getTokenCountForResponse -> getTokenCount * fix(formatLangChainMessages): add kwargs as fallback prop from lc_kwargs, update JSDoc * refactor(Tx.create): does not update balance if CHECK_BALANCE is not enabled * fix(e2e/cleanUp): cleanup new collections, import all model methods from index * fix(config/add-balance): add uncaughtException listener * fix: circular dependency * refactor(initializeLLM/checkBalance): append new generations to errorMessage if cost exceeds balance * fix(handleResponseMessage): only record token usage in this method if not error and completion is not skipped * fix(createStartHandler): correct condition for generations * chore: bump postcss due to moderate severity vulnerability * chore: bump zod due to low severity vulnerability * chore: bump openai & data-provider version * feat(types): OpenAI Message types * chore: update bun lockfile * refactor(CodeBlock): add error block formatting * refactor(utils/Plugin): factor out formatJSON and cn to separate files (json.ts and cn.ts), add extractJSON * chore(logViolation): delete user_id after error is logged * refactor(getMessageError -> Error): change to React.FC, add token_balance handling, use extractJSON to determine JSON instead of regex * fix(DALL-E): use latest openai SDK * chore: reorganize imports, fix type issue * feat(server): add balance route * fix(api/models): add auth * feat(data-provider): /api/balance query * feat: show balance if checking is enabled, refetch on final message or error * chore: update docs, .env.example with token_usage info, add balance script command * fix(Balance): fallback to empty obj for balance query * style: slight adjustment of balance element * docs(token_usage): add PR notes
2023-10-05 18:34:10 -04:00
req,
res,
feat(GPT/Anthropic): Continue Regenerating & Generation Buttons (#808) * feat(useMessageHandler.js/ts): Refactor and add features to handle user messages, support multiple endpoints/models, generate placeholder responses, regeneration, and stopGeneration function fix(conversation.ts, buildTree.ts): Import TMessage type, handle null parentMessageId feat(schemas.ts): Update and add schemas for various AI services, add default values, optional fields, and endpoint-to-schema mapping, create parseConvo function chore(useMessageHandler.js, schemas.ts): Remove unused imports, variables, and chatGPT enum * wip: add generation buttons * refactor(cleanupPreset.ts): simplify cleanupPreset function refactor(getDefaultConversation.js): remove unused code and simplify getDefaultConversation function feat(utils): add getDefaultConversation function This commit adds a new utility function called `getDefaultConversation` to the `client/src/utils/getDefaultConversation.ts` file. This function is responsible for generating a default conversation object based on the provided parameters. The `getDefaultConversation` function takes in an object with the following properties: - `conversation`: The conversation object to be used as a base. - `endpointsConfig`: The configuration object containing information about the available endpoints. - `preset`: An optional preset object that can be used to override the default behavior. The function first tries to determine the target endpoint based on the preset object. If a valid endpoint is found, it is used as the target endpoint. If not, the function tries to retrieve the last conversation setup from the local storage and uses its endpoint if it is valid. If neither the preset nor the local storage contains a valid endpoint, the function falls back to a default endpoint. Once the target endpoint is determined, * fix(utils): remove console.error statement in buildDefaultConversation function fix(schemas): add default values for catch blocks in openAISchema, googleSchema, bingAISchema, anthropicSchema, chatGPTBrowserSchema, and gptPluginsSchema * fix: endpoint not changing on change of preset from other endpoint, wip: refactor * refactor: preset items to TSX * refactor: convert resetConvo to TS * refactor(getDefaultConversation.ts): move defaultEndpoints array to the top of the file for better readability refactor(getDefaultConversation.ts): extract getDefaultEndpoint function for better code organization and reusability * feat(svg): add ContinueIcon component feat(svg): add RegenerateIcon component feat(svg): add ContinueIcon and RegenerateIcon components to index.ts * feat(Button.tsx): add onClick and className props to Button component feat(GenerationButtons.tsx): add logic to display Regenerate or StopGenerating button based on isSubmitting and messages feat(Regenerate.tsx): create Regenerate component with RegenerateIcon and handleRegenerate function feat(StopGenerating.tsx): create StopGenerating component with StopGeneratingIcon and handleStopGenerating function * fix(TextChat.jsx): reorder imports and variables for better readability fix(TextChat.jsx): fix typo in condition for isNotAppendable variable fix(TextChat.jsx): remove unused handleStopGenerating function fix(ContinueIcon.tsx): remove unnecessary closing tags for polygon elements fix(useMessageHandler.ts): add missing type annotations for handleStopGenerating and handleRegenerate functions fix(useMessageHandler.ts): remove unused variables in return statement * fix(getDefaultConversation.ts): refactor code to use getLocalStorageItems function feat(getLocalStorageItems.ts): add utility function to retrieve items from local storage * fix(OpenAIClient.js): add support for streaming result in sendCompletion method feat(OpenAIClient.js): add finish_reason metadata to opts in sendCompletion method feat(Message.js): add finish_reason field to Message model feat(messageSchema.js): add finish_reason field to messageSchema feat(openAI.js): parse chatGptLabel and promptPrefix from req.body and pass rest of the modelOptions to endpointOption feat(openAI.js): add addMetadata function to store metadata in ask function feat(openAI.js): add metadata to response if available feat(schemas.ts): add finish_reason field to tMessageSchema * feat(types.ts): add TOnClick and TGenButtonProps types for button components feat(Continue.tsx): create Continue component for generating button feat(GenerationButtons.tsx): update GenerationButtons component to use Continue component feat(Regenerate.tsx): create Regenerate component for regenerating button feat(Stop.tsx): create Stop component for stop generating button * feat(MessageHandler.jsx): add MessageHandler component to handle messages and conversations fix(Root.jsx): fix import paths for Nav and MessageHandler components * feat(useMessageHandler.ts): add support for generation parameter in ask function feat(useMessageHandler.ts): add support for isEdited parameter in ask function feat(useMessageHandler.ts): add support for continueGeneration function fix(createPayload.ts): replace endpoint URL when isEdited parameter is true * chore(client): set skipLibCheck to true in tsconfig.json * fix(useMessageHandler.ts): remove unused clientId variable fix(schemas.ts): make clientId field in tMessageSchema nullable and optional * wip: edit route for continue generation * refactor(api): move handlers to root of routes dir * fix(useMessageHandler.ts): initialize currentMessages to an empty array if messages is null fix(useMessageHandler.ts): update initialResponse text to use responseText variable fix(useMessageHandler.ts): update setMessages logic for isRegenerate case fix(MessageHandler.jsx): update setMessages logic for cancelHandler, createdHandler, and finalHandler * fix(schemas.ts): make createdAt and updatedAt fields optional and set default values using new Date().toISOString() fix(schemas.ts): change type annotation of TMessage from infer to input * refactor(useMessageHandler.ts): rename AskProps type to TAskProps refactor(useMessageHandler.ts): remove generation property from ask function arguments refactor(useMessageHandler.ts): use nullish coalescing operator (??) instead of logical OR (||) refactor(useMessageHandler.ts): pass the responseMessageId to message prop of submission * fix(BaseClient.js): use nullish coalescing operator (??) instead of logical OR (||) for default values * fix(BaseClient.js): fix responseMessageId assignment in handleStartMethods method feat(BaseClient.js): add support for isEdited flag in sendMessage method feat(BaseClient.js): add generation to responseMessage text in sendMessage method * fix(openAI.js): remove unused imports and commented out code feat(openAI.js): add support for generation parameter in request body fix(openAI.js): remove console.log statement fix(openAI.js): remove unused variables and parameters fix(openAI.js): update response text in case of error fix(openAI.js): handle error and abort message in case of error fix(handlers.js): add generation parameter to createOnProgress function fix(useMessageHandler.ts): update responseText variable to use generation parameter * refactor(api/middleware): move inside server dir * refactor: add endpoint specific, modular functions to build options and initialize clients, create server/utils, move middleware, separate utils into api general utils and server specific utils * fix(abortMiddleware.js): import getConvo and getConvoTitle functions from models feat(abortMiddleware.js): add abortAsk function to abortController to handle aborting of requests fix(openAI.js): import buildOptions and initializeClient functions from endpoints/openAI refactor(openAI.js): use getAbortData function to get data for abortAsk function * refactor: move endpoint specific logic to an endpoints dir * refactor(PluginService.js): fix import path for encrypt and decrypt functions in PluginService.js * feat(openAI): add new endpoint for adding a title to a conversation - Added a new file `addTitle.js` in the `api/server/routes/endpoints/openAI` directory. - The `addTitle.js` file exports a function `addTitle` that takes in request parameters and performs the following actions: - If the `parentMessageId` is `'00000000-0000-0000-0000-000000000000'` and `newConvo` is true, it proceeds with the following steps: - Calls the `titleConvo` function from the `titleConvo` module, passing in the necessary parameters. - Calls the `saveConvo` function from the `saveConvo` module, passing in the user ID and conversation details. - Updated the `index.js` file in the `api/server/routes/endpoints/openAI` directory to export the `addTitle` function. - This change adds * fix(abortMiddleware.js): remove console.log statement refactor(gptPlugins.js): update imports and function parameters feat(gptPlugins.js): add support for abortController and getAbortData refactor(openAI.js): update imports and function parameters feat(openAI.js): add support for abortController and getAbortData fix(openAI.js): refactor code to use modularized functions and middleware fix(buildOptions.js): refactor code to use destructuring and update variable names * refactor(askChatGPTBrowser.js, bingAI.js, google.js): remove duplicate code for setting response headers feat(askChatGPTBrowser.js, bingAI.js, google.js): add setHeaders middleware to set response headers * feat(middleware): validateEndpoint, refactor buildOption to only be concerned of endpointOption * fix(abortMiddleware.js): add 'finish_reason' property with value 'incomplete' to responseMessage object fix(abortMessage.js): remove console.log statement for aborted message fix(handlers.js): modify tokens assignment to handle empty generation string and trailing space * fix(BaseClient.js): import addSpaceIfNeeded function from server/utils fix(BaseClient.js): add space before generation in text property fix(index.js): remove getCitations and citeText exports feat(buildEndpointOption.js): add buildEndpointOption middleware fix(index.js): import buildEndpointOption middleware fix(anthropic.js): remove buildOptions function and use endpointOption from req.body fix(gptPlugins.js): remove buildOptions function and use endpointOption from req.body fix(openAI.js): remove buildOptions function and use endpointOption from req.body feat(utils): add citations.js and handleText.js modules fix(utils): fix import statements in index.js module * refactor(gptPlugins.js): use getResponseSender function from librechat-data-provider * feat(gptPlugins): complete 'continue generating' * wip: anthropic continue regen * feat(middleware): add validateRegistration middleware A new middleware function called `validateRegistration` has been added to the list of exported middleware functions in `index.js`. This middleware is responsible for validating registration data before allowing the registration process to proceed. * feat(Anthropic): complete continue regen * chore: add librechat-data-provider to api/package.json * fix(ci): backend-review will mock meilisearch, also installs data-provider as now needed * chore(ci): remove unneeded SEARCH env var * style(GenerationButtons): make text shorter for sake of space economy, even though this diverges from chat.openai.com * style(GenerationButtons/ScrollToBottom): adjust visibility/position based on screen size * chore(client): 'Editting' typo * feat(GenerationButtons.tsx): add support for endpoint prop in GenerationButtons component feat(OptionsBar.tsx): pass endpoint prop to GenerationButtons component feat(useGenerations.ts): create useGenerations hook to handle generation logic fix(schemas.ts): add searchResult field to tMessageSchema * refactor(HoverButtons): convert to TSX and utilize new useGenerations hook * fix(abortMiddleware): handle error with res headers set, or abortController not found, to ensure proper API error is sent to the client, chore(BaseClient): remove console log for onStart message meant for debugging * refactor(api): remove librechat-data-provider dep for now as it complicates deployed docker build stage, re-use code in CJS, located in server/endpoints/schemas * chore: remove console.logs from test files * ci: add backend tests for AnthropicClient, focusing on new buildMessages logic * refactor(FakeClient): use actual BaseClient sendMessage method for testing * test(BaseClient.test.js): add test for loading chat history test(BaseClient.test.js): add test for sendMessage logic with isEdited flag * fix(buildEndpointOption.js): add support for azureOpenAI in buildFunction object wip(endpoints.js): fetch Azure models from Azure OpenAI API if opts.azure is true * fix(Button.tsx): add data-testid attribute to button component fix(SelectDropDown.tsx): add data-testid attribute to Listbox.Button component fix(messages.spec.ts): add waitForServerStream function to consolidate logic for awaiting the server response feat(messages.spec.ts): add test for stopping and continuing message and improve browser/page context order and closing * refactor(onProgress): speed up time to save initial message for editable routes * chore: disable AI message editing (for now), was accidentally allowed * refactor: ensure continue is only supported for latest message style: improve styling in dark mode and across all hover buttons/icons, including making edit icon for AI invisible (for now) * fix: add test id to generation buttons so they never resolve to 2+ items * chore(package.json): add 'packages/' to the list of ignored directories chore(data-provider/package.json): bump version to 0.1.5
2023-08-17 12:50:05 -04:00
...endpointOption,
};
refactor: Encrypt & Expire User Provided Keys, feat: Rate Limiting (#874) * docs: make_your_own.md formatting fix for mkdocs * feat: add express-mongo-sanitize feat: add login/registration rate limiting * chore: remove unnecessary console log * wip: remove token handling from localStorage to encrypted DB solution * refactor: minor change to UserService * fix mongo query and add keys route to server * fix backend controllers and simplify schema/crud * refactor: rename token to key to separate from access/refresh tokens, setTokenDialog -> setKeyDialog * refactor(schemas): TEndpointOption token -> key * refactor(api): use new encrypted key retrieval system * fix(SetKeyDialog): fix key prop error * fix(abortMiddleware): pass random UUID if messageId is not generated yet for proper error display on frontend * fix(getUserKey): wrong prop passed in arg, adds error handling * fix: prevent message without conversationId from saving to DB, prevents branching on the frontend to a new top-level branch * refactor: change wording of multiple display messages * refactor(checkExpiry -> checkUserKeyExpiry): move to UserService file * fix: type imports from common * refactor(SubmitButton): convert to TS * refactor(key.ts): change localStorage map key name * refactor: add new custom tailwind classes to better match openAI colors * chore: remove unnecessary warning and catch ScreenShot error * refactor: move userKey frontend logic to hooks and remove use of localStorage and instead query the DB * refactor: invalidate correct query key, memoize userKey hook, conditionally render SetKeyDialog to avoid unnecessary calls, refactor SubmitButton props and useEffect for showing 'provide key first' * fix(SetKeyDialog): use enum-like object for expiry values feat(Dropdown): add optionsClassName to dynamically change dropdown options container classes * fix: handle edge case where user had provided a key but the server changes to env variable for keys * refactor(OpenAI/titleConvo): move titling to client to retain authorized credentials in message lifecycle for titling * fix(azure): handle user_provided keys correctly for azure * feat: send user Id to OpenAI to differentiate users in completion requests * refactor(OpenAI/titleConvo): adding tokens helps minimize LLM from using the language in title response * feat: add delete endpoint for keys * chore: remove throttling of title * feat: add 'Data controls' to Settings, add 'Revoke' keys feature in Key Dialog and Data controls * refactor: reorganize PluginsClient files in langchain format * feat: use langchain for titling convos * chore: cleanup titling convo, with fallback to original method, escape braces, use only snippet for language detection * refactor: move helper functions to appropriate langchain folders for reusability * fix: userProvidesKey handling for gptPlugins * fix: frontend handling of plugins key * chore: cleanup logging and ts-ignore SSE * fix: forwardRef misuse in DangerButton * fix(GoogleConfig/FileUpload): localize errors and simplify validation with zod * fix: cleanup google logging and fix user provided key handling * chore: remove titling from google * chore: removing logging from browser endpoint * wip: fix menu flicker * feat: useLocalStorage hook * feat: add Tooltip for UI * refactor(EndpointMenu): utilize Tooltip and useLocalStorage, remove old 'New Chat' slide-over * fix(e2e): use testId for endpoint menu trigger * chore: final touches to EndpointMenu before future refactor to declutter component * refactor(localization): change select endpoint to open menu and add translations * chore: add final prop to error message response * ci: minor edits to facilitate testing * ci: new e2e test which tests for new key setting/revoking features
2023-09-06 10:46:27 -04:00
const isUserProvided =
endpoint === 'openAI' ? OPENAI_API_KEY === 'user_provided' : AZURE_API_KEY === 'user_provided';
feat(GPT/Anthropic): Continue Regenerating & Generation Buttons (#808) * feat(useMessageHandler.js/ts): Refactor and add features to handle user messages, support multiple endpoints/models, generate placeholder responses, regeneration, and stopGeneration function fix(conversation.ts, buildTree.ts): Import TMessage type, handle null parentMessageId feat(schemas.ts): Update and add schemas for various AI services, add default values, optional fields, and endpoint-to-schema mapping, create parseConvo function chore(useMessageHandler.js, schemas.ts): Remove unused imports, variables, and chatGPT enum * wip: add generation buttons * refactor(cleanupPreset.ts): simplify cleanupPreset function refactor(getDefaultConversation.js): remove unused code and simplify getDefaultConversation function feat(utils): add getDefaultConversation function This commit adds a new utility function called `getDefaultConversation` to the `client/src/utils/getDefaultConversation.ts` file. This function is responsible for generating a default conversation object based on the provided parameters. The `getDefaultConversation` function takes in an object with the following properties: - `conversation`: The conversation object to be used as a base. - `endpointsConfig`: The configuration object containing information about the available endpoints. - `preset`: An optional preset object that can be used to override the default behavior. The function first tries to determine the target endpoint based on the preset object. If a valid endpoint is found, it is used as the target endpoint. If not, the function tries to retrieve the last conversation setup from the local storage and uses its endpoint if it is valid. If neither the preset nor the local storage contains a valid endpoint, the function falls back to a default endpoint. Once the target endpoint is determined, * fix(utils): remove console.error statement in buildDefaultConversation function fix(schemas): add default values for catch blocks in openAISchema, googleSchema, bingAISchema, anthropicSchema, chatGPTBrowserSchema, and gptPluginsSchema * fix: endpoint not changing on change of preset from other endpoint, wip: refactor * refactor: preset items to TSX * refactor: convert resetConvo to TS * refactor(getDefaultConversation.ts): move defaultEndpoints array to the top of the file for better readability refactor(getDefaultConversation.ts): extract getDefaultEndpoint function for better code organization and reusability * feat(svg): add ContinueIcon component feat(svg): add RegenerateIcon component feat(svg): add ContinueIcon and RegenerateIcon components to index.ts * feat(Button.tsx): add onClick and className props to Button component feat(GenerationButtons.tsx): add logic to display Regenerate or StopGenerating button based on isSubmitting and messages feat(Regenerate.tsx): create Regenerate component with RegenerateIcon and handleRegenerate function feat(StopGenerating.tsx): create StopGenerating component with StopGeneratingIcon and handleStopGenerating function * fix(TextChat.jsx): reorder imports and variables for better readability fix(TextChat.jsx): fix typo in condition for isNotAppendable variable fix(TextChat.jsx): remove unused handleStopGenerating function fix(ContinueIcon.tsx): remove unnecessary closing tags for polygon elements fix(useMessageHandler.ts): add missing type annotations for handleStopGenerating and handleRegenerate functions fix(useMessageHandler.ts): remove unused variables in return statement * fix(getDefaultConversation.ts): refactor code to use getLocalStorageItems function feat(getLocalStorageItems.ts): add utility function to retrieve items from local storage * fix(OpenAIClient.js): add support for streaming result in sendCompletion method feat(OpenAIClient.js): add finish_reason metadata to opts in sendCompletion method feat(Message.js): add finish_reason field to Message model feat(messageSchema.js): add finish_reason field to messageSchema feat(openAI.js): parse chatGptLabel and promptPrefix from req.body and pass rest of the modelOptions to endpointOption feat(openAI.js): add addMetadata function to store metadata in ask function feat(openAI.js): add metadata to response if available feat(schemas.ts): add finish_reason field to tMessageSchema * feat(types.ts): add TOnClick and TGenButtonProps types for button components feat(Continue.tsx): create Continue component for generating button feat(GenerationButtons.tsx): update GenerationButtons component to use Continue component feat(Regenerate.tsx): create Regenerate component for regenerating button feat(Stop.tsx): create Stop component for stop generating button * feat(MessageHandler.jsx): add MessageHandler component to handle messages and conversations fix(Root.jsx): fix import paths for Nav and MessageHandler components * feat(useMessageHandler.ts): add support for generation parameter in ask function feat(useMessageHandler.ts): add support for isEdited parameter in ask function feat(useMessageHandler.ts): add support for continueGeneration function fix(createPayload.ts): replace endpoint URL when isEdited parameter is true * chore(client): set skipLibCheck to true in tsconfig.json * fix(useMessageHandler.ts): remove unused clientId variable fix(schemas.ts): make clientId field in tMessageSchema nullable and optional * wip: edit route for continue generation * refactor(api): move handlers to root of routes dir * fix(useMessageHandler.ts): initialize currentMessages to an empty array if messages is null fix(useMessageHandler.ts): update initialResponse text to use responseText variable fix(useMessageHandler.ts): update setMessages logic for isRegenerate case fix(MessageHandler.jsx): update setMessages logic for cancelHandler, createdHandler, and finalHandler * fix(schemas.ts): make createdAt and updatedAt fields optional and set default values using new Date().toISOString() fix(schemas.ts): change type annotation of TMessage from infer to input * refactor(useMessageHandler.ts): rename AskProps type to TAskProps refactor(useMessageHandler.ts): remove generation property from ask function arguments refactor(useMessageHandler.ts): use nullish coalescing operator (??) instead of logical OR (||) refactor(useMessageHandler.ts): pass the responseMessageId to message prop of submission * fix(BaseClient.js): use nullish coalescing operator (??) instead of logical OR (||) for default values * fix(BaseClient.js): fix responseMessageId assignment in handleStartMethods method feat(BaseClient.js): add support for isEdited flag in sendMessage method feat(BaseClient.js): add generation to responseMessage text in sendMessage method * fix(openAI.js): remove unused imports and commented out code feat(openAI.js): add support for generation parameter in request body fix(openAI.js): remove console.log statement fix(openAI.js): remove unused variables and parameters fix(openAI.js): update response text in case of error fix(openAI.js): handle error and abort message in case of error fix(handlers.js): add generation parameter to createOnProgress function fix(useMessageHandler.ts): update responseText variable to use generation parameter * refactor(api/middleware): move inside server dir * refactor: add endpoint specific, modular functions to build options and initialize clients, create server/utils, move middleware, separate utils into api general utils and server specific utils * fix(abortMiddleware.js): import getConvo and getConvoTitle functions from models feat(abortMiddleware.js): add abortAsk function to abortController to handle aborting of requests fix(openAI.js): import buildOptions and initializeClient functions from endpoints/openAI refactor(openAI.js): use getAbortData function to get data for abortAsk function * refactor: move endpoint specific logic to an endpoints dir * refactor(PluginService.js): fix import path for encrypt and decrypt functions in PluginService.js * feat(openAI): add new endpoint for adding a title to a conversation - Added a new file `addTitle.js` in the `api/server/routes/endpoints/openAI` directory. - The `addTitle.js` file exports a function `addTitle` that takes in request parameters and performs the following actions: - If the `parentMessageId` is `'00000000-0000-0000-0000-000000000000'` and `newConvo` is true, it proceeds with the following steps: - Calls the `titleConvo` function from the `titleConvo` module, passing in the necessary parameters. - Calls the `saveConvo` function from the `saveConvo` module, passing in the user ID and conversation details. - Updated the `index.js` file in the `api/server/routes/endpoints/openAI` directory to export the `addTitle` function. - This change adds * fix(abortMiddleware.js): remove console.log statement refactor(gptPlugins.js): update imports and function parameters feat(gptPlugins.js): add support for abortController and getAbortData refactor(openAI.js): update imports and function parameters feat(openAI.js): add support for abortController and getAbortData fix(openAI.js): refactor code to use modularized functions and middleware fix(buildOptions.js): refactor code to use destructuring and update variable names * refactor(askChatGPTBrowser.js, bingAI.js, google.js): remove duplicate code for setting response headers feat(askChatGPTBrowser.js, bingAI.js, google.js): add setHeaders middleware to set response headers * feat(middleware): validateEndpoint, refactor buildOption to only be concerned of endpointOption * fix(abortMiddleware.js): add 'finish_reason' property with value 'incomplete' to responseMessage object fix(abortMessage.js): remove console.log statement for aborted message fix(handlers.js): modify tokens assignment to handle empty generation string and trailing space * fix(BaseClient.js): import addSpaceIfNeeded function from server/utils fix(BaseClient.js): add space before generation in text property fix(index.js): remove getCitations and citeText exports feat(buildEndpointOption.js): add buildEndpointOption middleware fix(index.js): import buildEndpointOption middleware fix(anthropic.js): remove buildOptions function and use endpointOption from req.body fix(gptPlugins.js): remove buildOptions function and use endpointOption from req.body fix(openAI.js): remove buildOptions function and use endpointOption from req.body feat(utils): add citations.js and handleText.js modules fix(utils): fix import statements in index.js module * refactor(gptPlugins.js): use getResponseSender function from librechat-data-provider * feat(gptPlugins): complete 'continue generating' * wip: anthropic continue regen * feat(middleware): add validateRegistration middleware A new middleware function called `validateRegistration` has been added to the list of exported middleware functions in `index.js`. This middleware is responsible for validating registration data before allowing the registration process to proceed. * feat(Anthropic): complete continue regen * chore: add librechat-data-provider to api/package.json * fix(ci): backend-review will mock meilisearch, also installs data-provider as now needed * chore(ci): remove unneeded SEARCH env var * style(GenerationButtons): make text shorter for sake of space economy, even though this diverges from chat.openai.com * style(GenerationButtons/ScrollToBottom): adjust visibility/position based on screen size * chore(client): 'Editting' typo * feat(GenerationButtons.tsx): add support for endpoint prop in GenerationButtons component feat(OptionsBar.tsx): pass endpoint prop to GenerationButtons component feat(useGenerations.ts): create useGenerations hook to handle generation logic fix(schemas.ts): add searchResult field to tMessageSchema * refactor(HoverButtons): convert to TSX and utilize new useGenerations hook * fix(abortMiddleware): handle error with res headers set, or abortController not found, to ensure proper API error is sent to the client, chore(BaseClient): remove console log for onStart message meant for debugging * refactor(api): remove librechat-data-provider dep for now as it complicates deployed docker build stage, re-use code in CJS, located in server/endpoints/schemas * chore: remove console.logs from test files * ci: add backend tests for AnthropicClient, focusing on new buildMessages logic * refactor(FakeClient): use actual BaseClient sendMessage method for testing * test(BaseClient.test.js): add test for loading chat history test(BaseClient.test.js): add test for sendMessage logic with isEdited flag * fix(buildEndpointOption.js): add support for azureOpenAI in buildFunction object wip(endpoints.js): fetch Azure models from Azure OpenAI API if opts.azure is true * fix(Button.tsx): add data-testid attribute to button component fix(SelectDropDown.tsx): add data-testid attribute to Listbox.Button component fix(messages.spec.ts): add waitForServerStream function to consolidate logic for awaiting the server response feat(messages.spec.ts): add test for stopping and continuing message and improve browser/page context order and closing * refactor(onProgress): speed up time to save initial message for editable routes * chore: disable AI message editing (for now), was accidentally allowed * refactor: ensure continue is only supported for latest message style: improve styling in dark mode and across all hover buttons/icons, including making edit icon for AI invisible (for now) * fix: add test id to generation buttons so they never resolve to 2+ items * chore(package.json): add 'packages/' to the list of ignored directories chore(data-provider/package.json): bump version to 0.1.5
2023-08-17 12:50:05 -04:00
refactor: Encrypt & Expire User Provided Keys, feat: Rate Limiting (#874) * docs: make_your_own.md formatting fix for mkdocs * feat: add express-mongo-sanitize feat: add login/registration rate limiting * chore: remove unnecessary console log * wip: remove token handling from localStorage to encrypted DB solution * refactor: minor change to UserService * fix mongo query and add keys route to server * fix backend controllers and simplify schema/crud * refactor: rename token to key to separate from access/refresh tokens, setTokenDialog -> setKeyDialog * refactor(schemas): TEndpointOption token -> key * refactor(api): use new encrypted key retrieval system * fix(SetKeyDialog): fix key prop error * fix(abortMiddleware): pass random UUID if messageId is not generated yet for proper error display on frontend * fix(getUserKey): wrong prop passed in arg, adds error handling * fix: prevent message without conversationId from saving to DB, prevents branching on the frontend to a new top-level branch * refactor: change wording of multiple display messages * refactor(checkExpiry -> checkUserKeyExpiry): move to UserService file * fix: type imports from common * refactor(SubmitButton): convert to TS * refactor(key.ts): change localStorage map key name * refactor: add new custom tailwind classes to better match openAI colors * chore: remove unnecessary warning and catch ScreenShot error * refactor: move userKey frontend logic to hooks and remove use of localStorage and instead query the DB * refactor: invalidate correct query key, memoize userKey hook, conditionally render SetKeyDialog to avoid unnecessary calls, refactor SubmitButton props and useEffect for showing 'provide key first' * fix(SetKeyDialog): use enum-like object for expiry values feat(Dropdown): add optionsClassName to dynamically change dropdown options container classes * fix: handle edge case where user had provided a key but the server changes to env variable for keys * refactor(OpenAI/titleConvo): move titling to client to retain authorized credentials in message lifecycle for titling * fix(azure): handle user_provided keys correctly for azure * feat: send user Id to OpenAI to differentiate users in completion requests * refactor(OpenAI/titleConvo): adding tokens helps minimize LLM from using the language in title response * feat: add delete endpoint for keys * chore: remove throttling of title * feat: add 'Data controls' to Settings, add 'Revoke' keys feature in Key Dialog and Data controls * refactor: reorganize PluginsClient files in langchain format * feat: use langchain for titling convos * chore: cleanup titling convo, with fallback to original method, escape braces, use only snippet for language detection * refactor: move helper functions to appropriate langchain folders for reusability * fix: userProvidesKey handling for gptPlugins * fix: frontend handling of plugins key * chore: cleanup logging and ts-ignore SSE * fix: forwardRef misuse in DangerButton * fix(GoogleConfig/FileUpload): localize errors and simplify validation with zod * fix: cleanup google logging and fix user provided key handling * chore: remove titling from google * chore: removing logging from browser endpoint * wip: fix menu flicker * feat: useLocalStorage hook * feat: add Tooltip for UI * refactor(EndpointMenu): utilize Tooltip and useLocalStorage, remove old 'New Chat' slide-over * fix(e2e): use testId for endpoint menu trigger * chore: final touches to EndpointMenu before future refactor to declutter component * refactor(localization): change select endpoint to open menu and add translations * chore: add final prop to error message response * ci: minor edits to facilitate testing * ci: new e2e test which tests for new key setting/revoking features
2023-09-06 10:46:27 -04:00
let key = null;
if (expiresAt && isUserProvided) {
checkUserKeyExpiry(
expiresAt,
'Your OpenAI API key has expired. Please provide your API key again.',
);
key = await getUserKey({ userId: req.user.id, name: endpoint });
}
let openAIApiKey = isUserProvided ? key : OPENAI_API_KEY;
if (process.env.AZURE_API_KEY && endpoint === 'azureOpenAI') {
clientOptions.azure = isUserProvided ? JSON.parse(key) : getAzureCredentials();
feat(GPT/Anthropic): Continue Regenerating & Generation Buttons (#808) * feat(useMessageHandler.js/ts): Refactor and add features to handle user messages, support multiple endpoints/models, generate placeholder responses, regeneration, and stopGeneration function fix(conversation.ts, buildTree.ts): Import TMessage type, handle null parentMessageId feat(schemas.ts): Update and add schemas for various AI services, add default values, optional fields, and endpoint-to-schema mapping, create parseConvo function chore(useMessageHandler.js, schemas.ts): Remove unused imports, variables, and chatGPT enum * wip: add generation buttons * refactor(cleanupPreset.ts): simplify cleanupPreset function refactor(getDefaultConversation.js): remove unused code and simplify getDefaultConversation function feat(utils): add getDefaultConversation function This commit adds a new utility function called `getDefaultConversation` to the `client/src/utils/getDefaultConversation.ts` file. This function is responsible for generating a default conversation object based on the provided parameters. The `getDefaultConversation` function takes in an object with the following properties: - `conversation`: The conversation object to be used as a base. - `endpointsConfig`: The configuration object containing information about the available endpoints. - `preset`: An optional preset object that can be used to override the default behavior. The function first tries to determine the target endpoint based on the preset object. If a valid endpoint is found, it is used as the target endpoint. If not, the function tries to retrieve the last conversation setup from the local storage and uses its endpoint if it is valid. If neither the preset nor the local storage contains a valid endpoint, the function falls back to a default endpoint. Once the target endpoint is determined, * fix(utils): remove console.error statement in buildDefaultConversation function fix(schemas): add default values for catch blocks in openAISchema, googleSchema, bingAISchema, anthropicSchema, chatGPTBrowserSchema, and gptPluginsSchema * fix: endpoint not changing on change of preset from other endpoint, wip: refactor * refactor: preset items to TSX * refactor: convert resetConvo to TS * refactor(getDefaultConversation.ts): move defaultEndpoints array to the top of the file for better readability refactor(getDefaultConversation.ts): extract getDefaultEndpoint function for better code organization and reusability * feat(svg): add ContinueIcon component feat(svg): add RegenerateIcon component feat(svg): add ContinueIcon and RegenerateIcon components to index.ts * feat(Button.tsx): add onClick and className props to Button component feat(GenerationButtons.tsx): add logic to display Regenerate or StopGenerating button based on isSubmitting and messages feat(Regenerate.tsx): create Regenerate component with RegenerateIcon and handleRegenerate function feat(StopGenerating.tsx): create StopGenerating component with StopGeneratingIcon and handleStopGenerating function * fix(TextChat.jsx): reorder imports and variables for better readability fix(TextChat.jsx): fix typo in condition for isNotAppendable variable fix(TextChat.jsx): remove unused handleStopGenerating function fix(ContinueIcon.tsx): remove unnecessary closing tags for polygon elements fix(useMessageHandler.ts): add missing type annotations for handleStopGenerating and handleRegenerate functions fix(useMessageHandler.ts): remove unused variables in return statement * fix(getDefaultConversation.ts): refactor code to use getLocalStorageItems function feat(getLocalStorageItems.ts): add utility function to retrieve items from local storage * fix(OpenAIClient.js): add support for streaming result in sendCompletion method feat(OpenAIClient.js): add finish_reason metadata to opts in sendCompletion method feat(Message.js): add finish_reason field to Message model feat(messageSchema.js): add finish_reason field to messageSchema feat(openAI.js): parse chatGptLabel and promptPrefix from req.body and pass rest of the modelOptions to endpointOption feat(openAI.js): add addMetadata function to store metadata in ask function feat(openAI.js): add metadata to response if available feat(schemas.ts): add finish_reason field to tMessageSchema * feat(types.ts): add TOnClick and TGenButtonProps types for button components feat(Continue.tsx): create Continue component for generating button feat(GenerationButtons.tsx): update GenerationButtons component to use Continue component feat(Regenerate.tsx): create Regenerate component for regenerating button feat(Stop.tsx): create Stop component for stop generating button * feat(MessageHandler.jsx): add MessageHandler component to handle messages and conversations fix(Root.jsx): fix import paths for Nav and MessageHandler components * feat(useMessageHandler.ts): add support for generation parameter in ask function feat(useMessageHandler.ts): add support for isEdited parameter in ask function feat(useMessageHandler.ts): add support for continueGeneration function fix(createPayload.ts): replace endpoint URL when isEdited parameter is true * chore(client): set skipLibCheck to true in tsconfig.json * fix(useMessageHandler.ts): remove unused clientId variable fix(schemas.ts): make clientId field in tMessageSchema nullable and optional * wip: edit route for continue generation * refactor(api): move handlers to root of routes dir * fix(useMessageHandler.ts): initialize currentMessages to an empty array if messages is null fix(useMessageHandler.ts): update initialResponse text to use responseText variable fix(useMessageHandler.ts): update setMessages logic for isRegenerate case fix(MessageHandler.jsx): update setMessages logic for cancelHandler, createdHandler, and finalHandler * fix(schemas.ts): make createdAt and updatedAt fields optional and set default values using new Date().toISOString() fix(schemas.ts): change type annotation of TMessage from infer to input * refactor(useMessageHandler.ts): rename AskProps type to TAskProps refactor(useMessageHandler.ts): remove generation property from ask function arguments refactor(useMessageHandler.ts): use nullish coalescing operator (??) instead of logical OR (||) refactor(useMessageHandler.ts): pass the responseMessageId to message prop of submission * fix(BaseClient.js): use nullish coalescing operator (??) instead of logical OR (||) for default values * fix(BaseClient.js): fix responseMessageId assignment in handleStartMethods method feat(BaseClient.js): add support for isEdited flag in sendMessage method feat(BaseClient.js): add generation to responseMessage text in sendMessage method * fix(openAI.js): remove unused imports and commented out code feat(openAI.js): add support for generation parameter in request body fix(openAI.js): remove console.log statement fix(openAI.js): remove unused variables and parameters fix(openAI.js): update response text in case of error fix(openAI.js): handle error and abort message in case of error fix(handlers.js): add generation parameter to createOnProgress function fix(useMessageHandler.ts): update responseText variable to use generation parameter * refactor(api/middleware): move inside server dir * refactor: add endpoint specific, modular functions to build options and initialize clients, create server/utils, move middleware, separate utils into api general utils and server specific utils * fix(abortMiddleware.js): import getConvo and getConvoTitle functions from models feat(abortMiddleware.js): add abortAsk function to abortController to handle aborting of requests fix(openAI.js): import buildOptions and initializeClient functions from endpoints/openAI refactor(openAI.js): use getAbortData function to get data for abortAsk function * refactor: move endpoint specific logic to an endpoints dir * refactor(PluginService.js): fix import path for encrypt and decrypt functions in PluginService.js * feat(openAI): add new endpoint for adding a title to a conversation - Added a new file `addTitle.js` in the `api/server/routes/endpoints/openAI` directory. - The `addTitle.js` file exports a function `addTitle` that takes in request parameters and performs the following actions: - If the `parentMessageId` is `'00000000-0000-0000-0000-000000000000'` and `newConvo` is true, it proceeds with the following steps: - Calls the `titleConvo` function from the `titleConvo` module, passing in the necessary parameters. - Calls the `saveConvo` function from the `saveConvo` module, passing in the user ID and conversation details. - Updated the `index.js` file in the `api/server/routes/endpoints/openAI` directory to export the `addTitle` function. - This change adds * fix(abortMiddleware.js): remove console.log statement refactor(gptPlugins.js): update imports and function parameters feat(gptPlugins.js): add support for abortController and getAbortData refactor(openAI.js): update imports and function parameters feat(openAI.js): add support for abortController and getAbortData fix(openAI.js): refactor code to use modularized functions and middleware fix(buildOptions.js): refactor code to use destructuring and update variable names * refactor(askChatGPTBrowser.js, bingAI.js, google.js): remove duplicate code for setting response headers feat(askChatGPTBrowser.js, bingAI.js, google.js): add setHeaders middleware to set response headers * feat(middleware): validateEndpoint, refactor buildOption to only be concerned of endpointOption * fix(abortMiddleware.js): add 'finish_reason' property with value 'incomplete' to responseMessage object fix(abortMessage.js): remove console.log statement for aborted message fix(handlers.js): modify tokens assignment to handle empty generation string and trailing space * fix(BaseClient.js): import addSpaceIfNeeded function from server/utils fix(BaseClient.js): add space before generation in text property fix(index.js): remove getCitations and citeText exports feat(buildEndpointOption.js): add buildEndpointOption middleware fix(index.js): import buildEndpointOption middleware fix(anthropic.js): remove buildOptions function and use endpointOption from req.body fix(gptPlugins.js): remove buildOptions function and use endpointOption from req.body fix(openAI.js): remove buildOptions function and use endpointOption from req.body feat(utils): add citations.js and handleText.js modules fix(utils): fix import statements in index.js module * refactor(gptPlugins.js): use getResponseSender function from librechat-data-provider * feat(gptPlugins): complete 'continue generating' * wip: anthropic continue regen * feat(middleware): add validateRegistration middleware A new middleware function called `validateRegistration` has been added to the list of exported middleware functions in `index.js`. This middleware is responsible for validating registration data before allowing the registration process to proceed. * feat(Anthropic): complete continue regen * chore: add librechat-data-provider to api/package.json * fix(ci): backend-review will mock meilisearch, also installs data-provider as now needed * chore(ci): remove unneeded SEARCH env var * style(GenerationButtons): make text shorter for sake of space economy, even though this diverges from chat.openai.com * style(GenerationButtons/ScrollToBottom): adjust visibility/position based on screen size * chore(client): 'Editting' typo * feat(GenerationButtons.tsx): add support for endpoint prop in GenerationButtons component feat(OptionsBar.tsx): pass endpoint prop to GenerationButtons component feat(useGenerations.ts): create useGenerations hook to handle generation logic fix(schemas.ts): add searchResult field to tMessageSchema * refactor(HoverButtons): convert to TSX and utilize new useGenerations hook * fix(abortMiddleware): handle error with res headers set, or abortController not found, to ensure proper API error is sent to the client, chore(BaseClient): remove console log for onStart message meant for debugging * refactor(api): remove librechat-data-provider dep for now as it complicates deployed docker build stage, re-use code in CJS, located in server/endpoints/schemas * chore: remove console.logs from test files * ci: add backend tests for AnthropicClient, focusing on new buildMessages logic * refactor(FakeClient): use actual BaseClient sendMessage method for testing * test(BaseClient.test.js): add test for loading chat history test(BaseClient.test.js): add test for sendMessage logic with isEdited flag * fix(buildEndpointOption.js): add support for azureOpenAI in buildFunction object wip(endpoints.js): fetch Azure models from Azure OpenAI API if opts.azure is true * fix(Button.tsx): add data-testid attribute to button component fix(SelectDropDown.tsx): add data-testid attribute to Listbox.Button component fix(messages.spec.ts): add waitForServerStream function to consolidate logic for awaiting the server response feat(messages.spec.ts): add test for stopping and continuing message and improve browser/page context order and closing * refactor(onProgress): speed up time to save initial message for editable routes * chore: disable AI message editing (for now), was accidentally allowed * refactor: ensure continue is only supported for latest message style: improve styling in dark mode and across all hover buttons/icons, including making edit icon for AI invisible (for now) * fix: add test id to generation buttons so they never resolve to 2+ items * chore(package.json): add 'packages/' to the list of ignored directories chore(data-provider/package.json): bump version to 0.1.5
2023-08-17 12:50:05 -04:00
openAIApiKey = clientOptions.azure.azureOpenAIApiKey;
}
const client = new OpenAIClient(openAIApiKey, clientOptions);
return {
client,
openAIApiKey,
};
};
module.exports = initializeClient;