LibreChat/.env.example
Danny Avila 8819e83d2c
refactor: Client Classes & Azure OpenAI as a separate Endpoint (#532)
* refactor: start new client classes, test localAi support

* feat: create base class, extend chatgpt from base

* refactor(BaseClient.js): change userId parameter to user
refactor(BaseClient.js): change userId parameter to user
feat(OpenAIClient.js): add sendMessage method
refactor(OpenAIClient.js): change getConversation method to use user parameter instead of userId
refactor(OpenAIClient.js): change saveMessageToDatabase method to use user parameter instead of userId
refactor(OpenAIClient.js): change buildPrompt method to use messages parameter instead of orderedMessages
feat(index.js): export client classes
refactor(askGPTPlugins.js): use req.body.token or process.env.OPENAI_API_KEY as OpenAI API key
refactor(index.js): comment out askOpenAI route
feat(index.js): add openAI route

feat(openAI.js): add new route for OpenAI API requests with support for progress updates and aborting requests.

* refactor(BaseClient.js): use optional chaining operator to access messageId property
refactor(OpenAIClient.js): use orderedMessages instead of messages to build prompt
refactor(OpenAIClient.js): use optional chaining operator to access messageId property
refactor(fetch-polyfill.js): remove fetch polyfill
refactor(openAI.js): comment out debug option in clientOptions

* refactor: update import statements and remove unused imports in several files
feat: add getAzureCredentials function to azureUtils module
docs: update comments in azureUtils module

* refactor(utils): rename migrateConversations to migrateDataToFirstUser for clarity and consistency

* feat(chatgpt-client.js): add getAzureCredentials function to retrieve Azure credentials
feat(chatgpt-client.js): use getAzureCredentials function to generate reverseProxyUrl
feat(OpenAIClient.js): add isChatCompletion property to determine if chat completion model is used
feat(OpenAIClient.js): add saveOptions parameter to sendMessage and buildPrompt methods
feat(OpenAIClient.js): modify buildPrompt method to handle chat completion model
feat(openAI.js): modify endpointOption to include modelOptions instead of individual options
refactor(OpenAIClient.js): modify getDelta property to use isChatCompletion property instead of isChatGptModel property
refactor(OpenAIClient.js): modify sendMessage method to use saveOptions parameter instead of modelOptions parameter
refactor(OpenAIClient.js): modify buildPrompt method to use saveOptions parameter instead of modelOptions parameter
refactor(OpenAIClient.js): modify ask method to include endpointOption parameter

* chore: delete draft file

* refactor(OpenAIClient.js): extract sendCompletion method from sendMessage method for reusability

* refactor(BaseClient.js): move sendMessage method to BaseClient class
feat(OpenAIClient.js): inherit from BaseClient class and implement necessary methods and properties for OpenAIClient class.

* refactor(BaseClient.js): rename getBuildPromptOptions to getBuildMessagesOptions
feat(BaseClient.js): add buildMessages method to BaseClient class
fix(ChatGPTClient.js): use message.text instead of message.message
refactor(ChatGPTClient.js): rename buildPromptBody to buildMessagesBody
refactor(ChatGPTClient.js): remove console.debug statement and add debug log for prompt variable

refactor(OpenAIClient.js): move setOptions method to the bottom of the class
feat(OpenAIClient.js): add support for cl100k_base encoding
feat(OpenAIClient.js): add support for unofficial chat GPT models
feat(OpenAIClient.js): add support for custom modelOptions
feat(OpenAIClient.js): add caching for tokenizers
feat(OpenAIClient.js): add freeAndInitializeEncoder method to free and reinitialize tokenizers
refactor(OpenAIClient.js): rename getBuildPromptOptions to getBuildMessagesOptions
refactor(OpenAIClient.js): rename buildPrompt to buildMessages
refactor(OpenAIClient.js): remove endpointOption from ask function arguments in openAI.js

* refactor(ChatGPTClient.js, OpenAIClient.js): improve code readability and consistency

- In ChatGPTClient.js, update the roleLabel and messageString variables to handle cases where the message object does not have an isCreatedByUser property or a role property with a value of 'user'.
- In OpenAIClient.js, rename the freeAndInitializeEncoder method to freeAndResetEncoder to better reflect its functionality. Also, update the method calls to reflect the new name. Additionally, update the getTokenCount method to handle errors by calling the freeAndResetEncoder method instead of the now-renamed freeAndInitializeEncoder method.

* refactor(OpenAIClient.js): extract instructions object to a separate variable and add it to payload after formatted messages
fix(OpenAIClient.js): handle cases where progressMessage.choices is undefined or empty

* refactor(BaseClient.js): extract addInstructions method from sendMessage method
feat(OpenAIClient.js): add maxTokensMap object to map maximum tokens for each model
refactor(OpenAIClient.js): use addInstructions method in buildMessages method instead of manually building the payload list

* refactor(OpenAIClient.js): remove unnecessary condition for modelOptions.model property in buildMessages method

* feat(BaseClient.js): add support for token count tracking and context strategy
feat(OpenAIClient.js): add support for token count tracking and context strategy
feat(Message.js): add tokenCount field to Message schema and updateMessage function

* refactor(BaseClient.js): add support for refining messages based on token limit
feat(OpenAIClient.js): add support for context refinement strategy
refactor(OpenAIClient.js): use context refinement strategy in message sending
refactor(server/index.js): improve code readability by breaking long lines

* refactor(BaseClient.js): change `remainingContext` to `remainingContextTokens` for clarity
feat(BaseClient.js): add `refinePrompt` and `refinePromptTemplate` to handle message refinement
feat(BaseClient.js): add `refineMessages` method to refine messages
feat(BaseClient.js): add `handleContextStrategy` method to handle context strategy
feat(OpenAIClient.js): add `abortController` to `buildPrompt` method options
refactor(OpenAIClient.js): change `payload` and `tokenCountMap` to let variables in `handleContextStrategy` method
refactor(BaseClient.js): change `remainingContext` to `remainingContextTokens` in `handleContextStrategy` method for consistency
refactor(BaseClient.js): change `remainingContext` to `remainingContextTokens` in `getMessagesWithinTokenLimit` method for consistency
refactor(BaseClient.js): change `remainingContext` to `remainingContext

* chore(openAI.js): comment out contextStrategy option in clientOptions

* chore(openAI.js): comment out debug option in clientOptions object

* test: BaseClient tests in progress

* test: Complete OpenAIClient & BaseClient tests

* fix(OpenAIClient.js): remove unnecessary whitespace
fix(OpenAIClient.js): remove unused variables and comments
fix(OpenAIClient.test.js): combine getTokenCount and freeAndResetEncoder tests

* chore(.eslintrc.js): add rule for maximum of 1 empty line
feat(ask/openAI.js): add abortMessage utility function
fix(ask/openAI.js): handle error and abort message if partial text is less than 2 characters
feat(utils/index.js): export abortMessage utility function

* test: complete additional tests

* feat: Azure OpenAI as a separate endpoint

* chore: remove extraneous console logs

* fix(azureOpenAI): use chatCompletion endpoint

* chore(initializeClient.js): delete initializeClient.js file

chore(askOpenAI.js): delete old OpenAI route handler

chore(handlers.js): remove trailing whitespace in thought variable assignment

* chore(chatgpt-client.js): remove unused chatgpt-client.js file
refactor(index.js): remove askClient import and export from index.js

* chore(chatgpt-client.tokens.js): update test script for memory usage and encoding performance

The test script in `chatgpt-client.tokens.js` has been updated to measure the memory usage and encoding performance of the client. The script now includes information about the initial memory usage, peak memory usage, final memory usage, and memory usage after a timeout. It also provides insights into the number of encoding requests that can be processed per second.

The script has been modified to use the `OpenAIClient` class instead of the `ChatGPTClient` class. Additionally, the number of iterations for the encoding loop has been reduced to 10,000.

A timeout function has been added to simulate a delay of 15 seconds. After the timeout, the memory usage is measured again.

The script now handles uncaught exceptions and logs any errors that occur, except for errors related to failed fetch requests.

Note: This is a test script and should not be used in production

* feat(FakeClient.js): add a new class `FakeClient` that extends `BaseClient` and implements methods for a fake client
feat(FakeClient.js): implement the `setOptions` method to handle options for the fake client
feat(FakeClient.js): implement the `initializeFakeClient` function to initialize a fake client with options and fake messages
fix(OpenAIClient.js): remove duplicate `maxTokensMap` import and use the one from utils
feat(BaseClient): return promptTokens and completionTokens

* refactor(gptPlugins): refactor ChatAgent to PluginsClient, which extends OpenAIClient

* refactor: client paths

* chore(jest.config.js): remove jest.config.js file

* fix(PluginController.js): update file path to manifest.json
feat(gptPlugins.js): add support for aborting messages

refactor(ask/index.js): rename askGPTPlugins to gptPlugins for consistency

* fix(BaseClient.js): fix spacing in generateTextStream function signature
refactor(BaseClient.js): remove unnecessary push to currentMessages in generateUserMessage function
refactor(BaseClient.js): remove unnecessary push to currentMessages in handleStartMethods function
refactor(PluginsClient.js): remove unused variables and date formatting in constructor
refactor(PluginsClient.js): simplify mapping of pastMessages in getCompletionPayload function

* refactor(GoogleClient): GoogleClient now extends BaseClient

* chore(.env.example): add AZURE_OPENAI_MODELS variable
fix(api/routes/ask/gptPlugins.js): enable Azure integration if PLUGINS_USE_AZURE is true
fix(api/routes/endpoints.js): getOpenAIModels function now accepts options, use AZURE_OPENAI_MODELS if PLUGINS_USE_AZURE is true
fix(client/components/Endpoints/OpenAI/Settings.jsx): remove console.log statement
docs(features/azure.md): add documentation for Azure OpenAI integration and environment variables

* fix(e2e:popup): includes the icon + endpoint names in role, name property
2023-07-03 16:51:12 -04:00

231 lines
9.4 KiB
Text

##########################
# Server configuration:
##########################
APP_TITLE=LibreChat
# The server will listen to localhost:3080 by default. You can change the target IP as you want.
# If you want to make this server available externally, for example to share the server with others
# or expose this from a Docker container, set host to 0.0.0.0 or your external IP interface.
# Tips: Setting host to 0.0.0.0 means listening on all interfaces. It's not a real IP.
# Use localhost:port rather than 0.0.0.0:port to access the server.
# Set Node env to development if running in dev mode.
HOST=localhost
PORT=3080
# Change this to proxy any API request.
# It's useful if your machine has difficulty calling the original API server.
# PROXY=
# Change this to your MongoDB URI if different. I recommend appending LibreChat.
MONGO_URI=mongodb://127.0.0.1:27018/LibreChat
##########################
# OpenAI Endpoint:
##########################
# Access key from OpenAI platform.
# Leave it blank to disable this feature.
# Set to "user_provided" to allow the user to provide their API key from the UI.
OPENAI_API_KEY="user_provided"
# Identify the available models, separated by commas *without spaces*.
# The first will be default.
# Leave it blank to use internal settings.
OPENAI_MODELS=gpt-3.5-turbo,gpt-3.5-turbo-16k,gpt-3.5-turbo-0301,text-davinci-003,gpt-4,gpt-4-0314,gpt-4-0613
# Reverse proxy settings for OpenAI:
# https://github.com/waylaidwanderer/node-chatgpt-api#using-a-reverse-proxy
# OPENAI_REVERSE_PROXY=
##########################
# AZURE Endpoint:
##########################
# To use Azure with this project, set the following variables. These will be used to build the API URL.
# Chat completion:
# `https://{AZURE_OPENAI_API_INSTANCE_NAME}.openai.azure.com/openai/deployments/{AZURE_OPENAI_API_DEPLOYMENT_NAME}/chat/completions?api-version={AZURE_OPENAI_API_VERSION}`;
# You should also consider changing the `OPENAI_MODELS` variable above to the models available in your instance/deployment.
# Note: I've noticed that the Azure API is much faster than the OpenAI API, so the streaming looks almost instantaneous.
# Note "AZURE_OPENAI_API_COMPLETIONS_DEPLOYMENT_NAME" and "AZURE_OPENAI_API_EMBEDDINGS_DEPLOYMENT_NAME" are optional but might be used in the future
# AZURE_API_KEY=
# AZURE_OPENAI_API_INSTANCE_NAME=
# AZURE_OPENAI_API_DEPLOYMENT_NAME=
# AZURE_OPENAI_API_VERSION=
# AZURE_OPENAI_API_COMPLETIONS_DEPLOYMENT_NAME=
# AZURE_OPENAI_API_EMBEDDINGS_DEPLOYMENT_NAME=
# Identify the available models, separated by commas *without spaces*.
# The first will be default.
# Leave it blank to use internal settings.
AZURE_OPENAI_MODELS=gpt-3.5-turbo,gpt-4
# To use Azure with the Plugins endpoint, you need the variables above, and uncomment the following variable:
# NOTE: This may not work as expected and Azure OpenAI may not support OpenAI Functions yet
# Omit/leave it commented to use the default OpenAI API
# PLUGINS_USE_AZURE="true"
##########################
# BingAI Endpoint:
##########################
# Also used for Sydney and jailbreak
# To get your Access token for Bing, login to https://www.bing.com
# Use dev tools or an extension while logged into the site to copy the content of the _U cookie.
#If this fails, follow these instructions https://github.com/danny-avila/LibreChat/issues/370#issuecomment-1560382302 to provide the full cookie strings.
# Set to "user_provided" to allow the user to provide its token from the UI.
# Leave it blank to disable this endpoint.
BINGAI_TOKEN="user_provided"
# BingAI Host:
# Necessary for some people in different countries, e.g. China (https://cn.bing.com)
# Leave it blank to use default server.
# BINGAI_HOST=https://cn.bing.com
##########################
# ChatGPT Endpoint:
##########################
# ChatGPT Browser Client (free but use at your own risk)
# Access token from https://chat.openai.com/api/auth/session
# Exposes your access token to `CHATGPT_REVERSE_PROXY`
# Set to "user_provided" to allow the user to provide its token from the UI.
# Leave it blank to disable this endpoint
CHATGPT_TOKEN="user_provided"
# Identify the available models, separated by commas. The first will be default.
# Leave it blank to use internal settings.
CHATGPT_MODELS=text-davinci-002-render-sha,gpt-4
# NOTE: you can add gpt-4-plugins, gpt-4-code-interpreter, and gpt-4-browsing to the list above and use the models for these features;
# however, the view/display portion of these features are not supported, but you can use the underlying models, which have higher token context
# Also: text-davinci-002-render-paid is deprecated as of May 2023
# Reverse proxy setting for OpenAI
# https://github.com/waylaidwanderer/node-chatgpt-api#using-a-reverse-proxy
# By default it will use the node-chatgpt-api recommended proxy, (it's a third party server)
# CHATGPT_REVERSE_PROXY=<YOUR REVERSE PROXY>
#############################
# Plugins:
#############################
# Identify the available models, separated by commas *without spaces*.
# The first will be default.
# Leave it blank to use internal settings.
PLUGIN_MODELS=gpt-3.5-turbo,gpt-3.5-turbo-16k,gpt-3.5-turbo-0301,gpt-4,gpt-4-0314,gpt-4-0613
# For securely storing credentials, you need a fixed key and IV. You can set them here for prod and dev environments
# If you don't set them, the app will crash on startup.
# You need a 32-byte key (64 characters in hex) and 16-byte IV (32 characters in hex)
# Use this replit to generate some quickly: https://replit.com/@daavila/crypto#index.js
# Here are some examples (THESE ARE NOT SECURE!)
CREDS_KEY=f34be427ebb29de8d88c107a71546019685ed8b241d8f2ed00c3df97ad2566f0
CREDS_IV=e2341419ec3dd3d19b13a1a87fafcbfb
# AI-Assisted Google Search
# This bot supports searching google for answers to your questions with assistance from GPT!
# See detailed instructions here: https://github.com/danny-avila/LibreChat/blob/main/docs/features/plugins/google_search.md
GOOGLE_API_KEY=
GOOGLE_CSE_ID=
# StableDiffusion WebUI
# This bot supports StableDiffusion WebUI, using it's API to generated requested images.
# See detailed instructions here: https://github.com/danny-avila/LibreChat/blob/main/docs/features/plugins/stable_diffusion.md
# Use "http://127.0.0.1:7860" with local install and "http://host.docker.internal:7860" for docker
SD_WEBUI_URL=http://host.docker.internal:7860
##########################
# PaLM (Google) Endpoint:
##########################
# Follow the instruction here to setup:
# https://github.com/danny-avila/LibreChat/blob/main/docs/install/apis_and_tokens.md
PALM_KEY="user_provided"
# In case you need a reverse proxy for this endpoint:
# GOOGLE_REVERSE_PROXY=
##########################
# Proxy: To be Used by all endpoints
##########################
PROXY=
##########################
# Search:
##########################
# ENABLING SEARCH MESSAGES/CONVOS
# Requires the installation of the free self-hosted Meilisearch or a paid Remote Plan (Remote not tested)
# The easiest setup for this is through docker-compose, which takes care of it for you.
SEARCH=true
# REQUIRED FOR SEARCH: MeiliSearch Host, mainly for the API server to connect to the search server.
# Replace '0.0.0.0' with 'meilisearch' if serving MeiliSearch with docker-compose.
MEILI_HOST=http://0.0.0.0:7700
# REQUIRED FOR SEARCH: MeiliSearch HTTP Address, mainly for docker-compose to expose the search server.
# Replace '0.0.0.0' with 'meilisearch' if serving MeiliSearch with docker-compose.
MEILI_HTTP_ADDR=0.0.0.0:7700
# REQUIRED FOR SEARCH: In production env., a secure key is needed. You can generate your own.
# This master key must be at least 16 bytes, composed of valid UTF-8 characters.
# MeiliSearch will throw an error and refuse to launch if no master key is provided,
# or if it is under 16 bytes. MeiliSearch will suggest a secure autogenerated master key.
# Using docker, it seems recognized as production so use a secure key.
# This is a ready made secure key for docker-compose, you can replace it with your own.
MEILI_MASTER_KEY=DrhYf7zENyR6AlUCKmnz0eYASOQdl6zxH7s7MKFSfFCt
##########################
# User System:
##########################
# Allow Public Registration
ALLOW_REGISTRATION=true
# JWT Secrets
JWT_SECRET=secret
JWT_REFRESH_SECRET=secret
# Google:
# Add your Google Client ID and Secret here, you must register an app with Google Cloud to get these values
# https://cloud.google.com/
GOOGLE_CLIENT_ID=
GOOGLE_CLIENT_SECRET=
GOOGLE_CALLBACK_URL=/oauth/google/callback
# OpenID:
# See OpenID provider to get the below values
# Create random string for OPENID_SESSION_SECRET
# For Azure AD
# ISSUER: https://login.microsoftonline.com/(tenant id)/v2.0/
# SCOPE: openid profile email
OPENID_CLIENT_ID=
OPENID_CLIENT_SECRET=
OPENID_ISSUER=
OPENID_SESSION_SECRET=
OPENID_SCOPE="openid profile email"
OPENID_CALLBACK_URL=/oauth/openid/callback
# If LABEL and URL are left empty, then the default OpenID label and logo are used.
OPENID_BUTTON_LABEL=
OPENID_IMAGE_URL=
# Set the expiration delay for the secure cookie with the JWT token
# Delay is in millisecond e.g. 7 days is 1000*60*60*24*7
SESSION_EXPIRY=(1000 * 60 * 60 * 24) * 7
###########################
# Application Domains
###########################
# Note:
# Server = Backend
# Client = Public (the client is the url you visit)
# For the Google login to work in dev mode, you will need to change DOMAIN_SERVER to localhost:3090 or place it in .env.development
DOMAIN_CLIENT=http://localhost:3080
DOMAIN_SERVER=http://localhost:3080