mirror of
https://github.com/danny-avila/LibreChat.git
synced 2025-12-17 00:40:14 +01:00
feat: Google Gemini ❇️ (#1355)
* refactor: add gemini-pro to google Models list; use defaultModels for central model listing * refactor(SetKeyDialog): create useMultipleKeys hook to use for Azure, export `isJson` from utils, use EModelEndpoint * refactor(useUserKey): change variable names to make keyName setting more clear * refactor(FileUpload): allow passing container className string * feat(GoogleClient): Gemini support * refactor(GoogleClient): alternate stream speed for Gemini models * feat(Gemini): styling/settings configuration for Gemini * refactor(GoogleClient): substract max response tokens from max context tokens if context is above 32k (I/O max is combined between the two) * refactor(tokens): correct google max token counts and subtract max response tokens when input/output count are combined towards max context count * feat(google/initializeClient): handle both local and user_provided credentials and write tests * fix(GoogleClient): catch if credentials are undefined, handle if serviceKey is string or object correctly, handle no examples passed, throw error if not a Generative Language model and no service account JSON key is provided, throw error if it is a Generative m odel, but not google API key was provided * refactor(loadAsyncEndpoints/google): activate Google endpoint if either the service key JSON file is provided in /api/data, or a GOOGLE_KEY is defined. * docs: updated Google configuration * fix(ci): Mock import of Service Account Key JSON file (auth.json) * Update apis_and_tokens.md * feat: increase max output tokens slider for gemini pro * refactor(GoogleSettings): handle max and default maxOutputTokens on model change * chore: add sensitive redact regex * docs: add warning about data privacy * Update apis_and_tokens.md
This commit is contained in:
Danny Avila
GitHub
parent
d259431316
commit
561ce8e86a
37 changed files with 702 additions and 219 deletions
|
|
@ -303,14 +303,14 @@ cd LibreChat/
|
|||
```
|
||||
|
||||
### **2. Create a global environment file.**
|
||||
The default values are enough to get you started and running the app.
|
||||
The default values are enough to get you started and running the app, allowing you to provide your credentials from the web app.
|
||||
|
||||
```bash
|
||||
# Copies the example file as your global env file
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
However, it's highly recommended you use environment variables for any sensitive credentials until we remove use of localStorage for passing credentials from the frontend
|
||||
However, if you'd like to provide any credentials for all users of your instance to consume, you can add them to the .env file as follows:
|
||||
|
||||
```bash
|
||||
nano .env
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
### Logging
|
||||
### General
|
||||
|
||||
LibreChat has central logging built into its backend (api).
|
||||
|
||||
|
|
@ -13,6 +13,8 @@ Errors logs are also saved in the same location: `./api/logs/error-%DATE%.log`.
|
|||
> Note: Logs are rotated on a 14-day basis, so you will generate 1 error log file, 1 debug log file, and 1 meiliSync log file per 14 days.
|
||||
> Errors will also be present in debug log files as well, but provide stack traces and more detail in the error log files.
|
||||
|
||||
### Setup
|
||||
|
||||
Keep debug logs enabled with the following environment variable. Even if you never set this variable, debug logs will be generated, but you have the option to disable them by setting it to `FALSE`.
|
||||
|
||||
```bash
|
||||
|
|
|
|||
|
|
@ -2,6 +2,20 @@
|
|||
|
||||
This doc explains how to setup various tokens and APIs for the project. You will need some of these tokens and APIs to run the app and use its features. You must set up at least one of these tokens or APIs to run the app.
|
||||
|
||||
### Docker notes
|
||||
|
||||
**If you use docker, you should rebuild the docker image each time you update your credentials**
|
||||
|
||||
Rebuild command:
|
||||
```bash
|
||||
npm run update:docker
|
||||
|
||||
# OR, if you don't have npm
|
||||
docker-compose build --no-cache
|
||||
```
|
||||
|
||||
Alternatively, you can create a new file named `docker-compose.override.yml` in the same directory as your main `docker-compose.yml` file for LibreChat, where you can set your .env variables as needed under `environment`. See the [docker docs](https://docs.docker.com/compose/multiple-compose-files/extends/#understanding-multiple-compose-files) for more info, and you can also view an example of an override file for LibreChat in the ["Manage Your Database" section](https://docs.librechat.ai/features/manage_your_database.html)
|
||||
|
||||
## OpenAI API key
|
||||
|
||||
To get your OpenAI API key, you need to:
|
||||
|
|
@ -46,16 +60,57 @@ To get your Bing Access Token, you have a few options:
|
|||
- Go to [https://console.anthropic.com/account/keys](https://console.anthropic.com/account/keys) and get your api key
|
||||
- add it to `ANTHROPIC_API_KEY=` in the `.env` file
|
||||
|
||||
## Google LLMs
|
||||
## Google
|
||||
|
||||
For the Google Endpoint, you can either use the **Generative Language API** (for Gemini models), or the **Vertex AI API** (for PaLM2 & Codey models, Gemini support coming soon).
|
||||
|
||||
The Generative Language API uses an API key, which you can get from **Google AI Studio**.
|
||||
|
||||
For Vertex AI, you need a Service Account JSON key file, with appropriate access configured.
|
||||
|
||||
Instructions for both are given below.
|
||||
|
||||
Setting `GOOGLE_KEY=user_provided` in your .env file will configure both values to be provided from the client (or frontend) like so:
|
||||
|
||||
|
||||
|
||||
### Generative Language API (Gemini)
|
||||
|
||||
**60 Gemini requests/minute are currently free until early next year when it enters general availability.**
|
||||
|
||||
⚠️ Google will be using that free input/output to help improve the model, with data de-identified from your Google Account and API key.
|
||||
⚠️ During this period, your messages “may be accessible to trained reviewers.”
|
||||
|
||||
To use Gemini models, you'll need an API key. If you don't already have one, create a key in Google AI Studio.
|
||||
|
||||
<p><a class="button button-primary" href="https://makersuite.google.com/app/apikey" target="_blank" rel="noopener noreferrer">Get an API key here</a></p>
|
||||
|
||||
Once you have your key, you can either provide it from the frontend by setting the following:
|
||||
|
||||
```bash
|
||||
GOOGLE_KEY=user_provided
|
||||
```
|
||||
|
||||
Or, provide the key in your .env file, which allows all users of your instance to use it.
|
||||
|
||||
```bash
|
||||
GOOGLE_KEY=mY_SeCreT_w9347w8_kEY
|
||||
```
|
||||
|
||||
> Notes:
|
||||
> - As of 12/15/23, Gemini Pro Vision is not yet supported but is planned.
|
||||
> - PaLM2 and Codey models cannot be accessed through the Generative Language API.
|
||||
|
||||
### Vertex AI (PaLM 2 & Codey)
|
||||
|
||||
To setup Google LLMs (via Google Cloud Vertex AI), first, signup for Google Cloud: https://cloud.google.com/
|
||||
|
||||
You can usually get $300 starting credit, which makes this option free for 90 days.
|
||||
You can usually get **$300 starting credit**, which makes this option free for 90 days.
|
||||
|
||||
### Once signed up, Enable the Vertex AI API on Google Cloud:
|
||||
### 1. Once signed up, Enable the Vertex AI API on Google Cloud:
|
||||
- Go to [Vertex AI page on Google Cloud console](https://console.cloud.google.com/vertex-ai)
|
||||
- Click on "Enable API" if prompted
|
||||
### Create a Service Account with Vertex AI role:
|
||||
### 2. Create a Service Account with Vertex AI role:
|
||||
- **[Click here to create a Service Account](https://console.cloud.google.com/projectselector/iam-admin/serviceaccounts/create?walkthrough_id=iam--create-service-account#step_index=1)**
|
||||
- **Select or create a project**
|
||||
- ### Enter a service account ID (required), name and description are optional
|
||||
|
|
@ -63,7 +118,7 @@ You can usually get $300 starting credit, which makes this option free for 90 da
|
|||
- ### Click on "Create and Continue" to give at least the "Vertex AI User" role
|
||||
- 
|
||||
- **Click on "Continue/Done"**
|
||||
### Create a JSON key to Save in Project Directory:
|
||||
### 3. Create a JSON key to Save in your Project Directory:
|
||||
- **Go back to [the Service Accounts page](https://console.cloud.google.com/projectselector/iam-admin/serviceaccounts)**
|
||||
- **Select your service account**
|
||||
- ### Click on "Keys"
|
||||
|
|
@ -75,6 +130,16 @@ You can usually get $300 starting credit, which makes this option free for 90 da
|
|||
- **Save it within the project directory, in `/api/data/`**
|
||||
- 
|
||||
|
||||
**Saving your JSON key file in the project directory which allows all users of your LibreChat instance to use it.**
|
||||
|
||||
Alternatively, Once you have your JSON key file, you can also provide it from the frontend on a user-basis by setting the following:
|
||||
|
||||
```bash
|
||||
GOOGLE_KEY=user_provided
|
||||
```
|
||||
|
||||
> Notes:
|
||||
> - As of 12/15/23, Gemini and Gemini Pro Vision are not yet supported through Vertex AI but are planned.
|
||||
|
||||
## Azure OpenAI
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue