Andreas/LibreChat
1
0
Fork 0
mirror of https://github.com/danny-avila/LibreChat.git synced 2025-12-16 16:30:15 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 6

feat(Google): Support all Text/Chat Models, Response streaming, PaLM -> Google 🤖 (#1316)

Browse source 
* feat: update PaLM icons

* feat: add additional google models

* POC: formatting inputs for Vertex AI streaming

* refactor: move endpoints services outside of /routes dir to /services/Endpoints

* refactor: shorten schemas import

* refactor: rename PALM to GOOGLE

* feat: make Google editable endpoint

* feat: reusable Ask and Edit controllers based off Anthropic

* chore: organize imports/logic

* fix(parseConvo): include examples in googleSchema

* fix: google only allows odd number of messages to be sent

* fix: pass proxy to AnthropicClient

* refactor: change `google` altName to `Google`

* refactor: update getModelMaxTokens and related functions to handle maxTokensMap with nested endpoint model key/values

* refactor: google Icon and response sender changes (Codey and Google logo instead of PaLM in all cases)

* feat: google support for maxTokensMap

* feat: google updated endpoints with Ask/Edit controllers, buildOptions, and initializeClient

* feat(GoogleClient): now builds prompt for text models and supports real streaming from Vertex AI through langchain

* chore(GoogleClient): remove comments, left before for reference in git history

* docs: update google instructions (WIP)

* docs(apis_and_tokens.md): add images to google instructions

* docs: remove typo apis_and_tokens.md

* Update apis_and_tokens.md

* feat(Google): use default settings map, fully support context for both text and chat models, fully support examples for chat models

* chore: update more PaLM references to Google

* chore: move playwright out of workflows to avoid failing tests
This commit is contained in:
Danny Avila 2023-12-10 14:54:13 -05:00 • committed by GitHub
parent 8a1968b2f8
commit 583e978a82
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
90 changed files with 1613 additions and 784 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/deployment/huggingface.md
View file

@ -36,7 +36,7 @@ You will need to fill these values:
| BINGAI_TOKEN | `user_provided` |
| CHATGPT_TOKEN | `user_provided` |
| ANTHROPIC_API_KEY | `user_provided` |
| PALM_KEY | `user_provided` |
| GOOGLE_KEY | `user_provided` |
| CREDS_KEY | * see bellow |
| CREDS_IV | * see bellow |
| JWT_SECRET | * see bellow |

2
docs/deployment/render.md
View file

@ -53,7 +53,7 @@ Also:
| JWT_REFRESH_SECRET | secret |
| JWT_SECRET | secret |
| OPENAI_API_KEY | user_provided |
| PALM_KEY | user_provided |
| GOOGLE_KEY | user_provided |
| PORT | 3080 |
| SESSION_EXPIRY | (1000 * 60 * 60 * 24) * 7 |

2
docs/features/plugins/introduction.md
View file

@ -6,7 +6,7 @@ The plugins endpoint opens the door to prompting LLMs in new ways other than tra
The first step is using chain-of-thought prompting & ["agency"](https://zapier.com/blog/ai-agent/) for using plugins/tools in a fashion mimicing the official ChatGPT Plugins feature.
More than this, you can use this endpoint for changing your conversation settings mid-conversation. Unlike the official ChatGPT site and all other endpoints, you can switch models, presets, and settings mid-convo, even when you have no plugins selected. This is useful if you first want a creative response from GPT-4, and then a deterministic, lower cost response from GPT-3. Soon, you will be able to use PaLM2 and HuggingFace models, all in this endpoint in the same modular manner.
More than this, you can use this endpoint for changing your conversation settings mid-conversation. Unlike the official ChatGPT site and all other endpoints, you can switch models, presets, and settings mid-convo, even when you have no plugins selected. This is useful if you first want a creative response from GPT-4, and then a deterministic, lower cost response from GPT-3. Soon, you will be able to use Google, HuggingFace, local models, all in this or a similar endpoint in the same modular manner.
### Roadmap:
- More plugins and advanced plugin usage (ongoing)

2
docs/index.md
View file

@ -36,7 +36,7 @@
- 🌎 Multilingual UI:
- English, 中文, Deutsch, Español, Français, Italiano, Polski, Português Brasileiro, Русский
- 日本語, Svenska, 한국어, Tiếng Việt, 繁體中文, العربية, Türkçe, Nederlands
- 🤖 AI model selection: OpenAI API, Azure, BingAI, ChatGPT Browser, PaLM2, Anthropic (Claude), Plugins
- 🤖 AI model selection: OpenAI API, Azure, BingAI, ChatGPT, Google Vertex AI, Anthropic (Claude), Plugins
- 💾 Create, Save, & Share Custom Presets
- 🔄 Edit, Resubmit, and Continue messages with conversation branching
- 📤 Export conversations as screenshots, markdown, text, json.

44
docs/install/apis_and_tokens.md
View file

@ -46,27 +46,35 @@ To get your Bing Access Token, you have a few options:
- Go to [https://console.anthropic.com/account/keys](https://console.anthropic.com/account/keys) and get your api key
- add it to `ANTHROPIC_API_KEY=` in the `.env` file
## Google's PaLM 2
## Google LLMs
To setup PaLM 2 (via Google Cloud Vertex AI API), you need to:
To setup Google LLMs (via Google Cloud Vertex AI), first, signup for Google Cloud: https://cloud.google.com/
### Enable the Vertex AI API on Google Cloud:
- Go to [https://console.cloud.google.com/vertex-ai](https://console.cloud.google.com/vertex-ai)
You can usually get $300 starting credit, which makes this option free for 90 days.
### Once signed up, Enable the Vertex AI API on Google Cloud:
- Go to [Vertex AI page on Google Cloud console](https://console.cloud.google.com/vertex-ai)
- Click on "Enable API" if prompted
### Create a Service Account:
- Go to [https://console.cloud.google.com/projectselector/iam-admin/serviceaccounts/create?walkthrough_id=iam--create-service-account#step_index=1](https://console.cloud.google.com/projectselector/iam-admin/serviceaccounts/create?walkthrough_id=iam--create-service-account#step_index=1)
- Select or create a project
- Enter a service account name and description
- Click on "Create and Continue" to give at least the "Vertex AI User" role
- Click on "Done"
### Create a JSON key, rename as 'auth.json' and save it in /api/data/:
- Go back to [https://console.cloud.google.com/projectselector/iam-admin/serviceaccounts](https://console.cloud.google.com/projectselector/iam-admin/serviceaccounts)
- Select your service account
- Click on "Keys"
- Click on "Add Key" and then "Create new key"
- Choose JSON as the key type and click on "Create"
- Download the key file and rename it as 'auth.json'
- Save it in `/api/data/`
### Create a Service Account with Vertex AI role:
- **[Click here to create a Service Account](https://console.cloud.google.com/projectselector/iam-admin/serviceaccounts/create?walkthrough_id=iam--create-service-account#step_index=1)**
- **Select or create a project**
- ### Enter a service account ID (required), name and description are optional
- ![image](https://github.com/danny-avila/LibreChat/assets/110412045/0c5cd177-029b-44fa-a398-a794aeb09de6)
- ### Click on "Create and Continue" to give at least the "Vertex AI User" role
- ![image](https://github.com/danny-avila/LibreChat/assets/110412045/22d3a080-e71e-446e-8485-bcc5bf558dbb)
- **Click on "Continue/Done"**
### Create a JSON key to Save in Project Directory:
- **Go back to [the Service Accounts page](https://console.cloud.google.com/projectselector/iam-admin/serviceaccounts)**
- **Select your service account**
- ### Click on "Keys"
- ![image](https://github.com/danny-avila/LibreChat/assets/110412045/735a7bbe-25a6-4b4c-9bb5-e0d8aa91be3d)
- ### Click on "Add Key" and then "Create new key"
- ![image](https://github.com/danny-avila/LibreChat/assets/110412045/cfbb20d3-94a8-4cd1-ac39-f9cd8c2fceaa)
- **Choose JSON as the key type and click on "Create"**
- **Download the key file and rename it as 'auth.json'**
- **Save it within the project directory, in `/api/data/`**
- ![image](https://github.com/danny-avila/LibreChat/assets/110412045/f5b8bcb5-1b20-4751-81a1-d3757a4b3f2f)
## Azure OpenAI

2
docs/install/dotenv.md
View file

@ -249,7 +249,7 @@ OPENROUTER_API_KEY=
Follow these instruction to setup: [Google PaLM 2](./apis_and_tokens.md#googles-palm-2)
```bash
PALM_KEY=user_provided
GOOGLE_KEY=user_provided
GOOGLE_REVERSE_PROXY=
```