Commit graph

2728 commits

Author SHA1 Message Date
Kai Kreuzer
a0288f1c5c
🧾 docs: Fix Typo in librechat.example.yaml (#2606) 2024-05-02 16:27:11 -04:00
Danny Avila
5d3c90be26
📦 chore: update package.json/package-lock.json (#2600)
* 📦 chore: update package.json/package-lock.json

* 📦 chore: remove unused dependencies
2024-05-02 03:23:38 -04:00
Denis Palnitsky
ab6fbe48f1
📥 feat: Import Conversations from LibreChat, ChatGPT, Chatbot UI (#2355)
* Basic implementation of ChatGPT conversation import

* remove debug code

* Handle citations

* Fix updatedAt in import

* update default model

* Use job scheduler to handle import requests

* import job status endpoint

* Add wrapper around Agenda

* Rate limits for import endpoint

* rename import api path

* Batch save import to mongo

* Improve naming

* Add documenting comments

* Test for importers

* Change button for importing conversations

* Frontend changes

* Import job status endpoint

* Import endpoint response

* Add translations to new phrases

* Fix conversations refreshing

* cleanup unused functions

* set timeout for import job status polling

* Add documentation

* get extra spaces back

* Improve error message

* Fix translation files after merge

* fix translation files 2

* Add zh translation for import functionality

* Sync mailisearch index after import

* chore: add dummy uri for jest tests, as MONGO_URI should only be real for E2E tests

* docs: fix links

* docs: fix conversationsImport section

* fix: user role issue for librechat imports

* refactor: import conversations from json
- organize imports
- add additional jsdocs
- use multer with diskStorage to avoid loading file into memory outside of job
- use filepath instead of loading data string for imports
- replace console logs and some logger.info() with logger.debug
- only use multer for import route

* fix: undefined metadata edge case and replace ChatGtp -> ChatGpt

* Refactor importChatGptConvo function to handle undefined metadata edge case and replace ChatGtp with ChatGpt

* fix: chatgpt importer

* feat: maintain tree relationship for librechat messages

* chore: use enum

* refactor: saveMessage to use single object arg, replace console logs, add userId to log message

* chore: additional comment

* chore: multer edge case

* feat: first pass, maintain tree relationship

* chore: organize

* chore: remove log

* ci: add heirarchy test for chatgpt

* ci: test maintaining of heirarchy for librechat

* wip: allow non-text content type messages

* refactor: import content part object json string

* refactor: more content types to format

* chore: consolidate messageText formatting

* docs: update on changes, bump data-provider/config versions, update readme

* refactor(indexSync): singleton pattern for MeiliSearchClient

* refactor: debug log after batch is done

* chore: add back indexSync error handling

---------

Co-authored-by: jakubmieszczak <jakub.mieszczak@zendesk.com>
Co-authored-by: Danny Avila <danny@librechat.ai>
2024-05-02 02:48:26 -04:00
Yuichi Ohneda
3b44741cf9
🚑 fix(dialog): showCloseButton Prop Warning in DialogContent Component (#2597)
Fix DialogPrimitive.Content to not pass a showCloseButton property that does not exist in it to avoid outputting a warning.
2024-05-02 02:08:37 -04:00
Extremys
d21a05606e
🍎 feat: Apple MLX as Known Endpoint (#2580)
* add integration with Apple MLX

* fix: apple icon + image mkd link

---------

Co-authored-by: “Extremys” <“Extremys@email.com”>
Co-authored-by: Danny Avila <danny@librechat.ai>
2024-05-01 03:27:02 -04:00
Danny Avila
0e50c07e3f
🤖 feat: Model Specs & Save Tools per Convo/Preset (#2578)
* WIP: first pass ModelSpecs

* refactor(onSelectEndpoint): use `getConvoSwitchLogic`

* feat: introduce iconURL, greeting, frontend fields for conversations/presets/messages

* feat: conversation.iconURL & greeting in Landing

* feat: conversation.iconURL & greeting in New Chat button

* feat: message.iconURL

* refactor: ConversationIcon -> ConvoIconURL

* WIP: add spec as a conversation field

* refactor: useAppStartup, set spec on initial load for new chat, allow undefined spec, add localStorage keys enum, additional type fields for spec

* feat: handle `showIconInMenu`, `showIconInHeader`, undefined `iconURL` and no specs on initial load

* chore: handle undefined or empty modelSpecs

* WIP: first pass, modelSpec schema for custom config

* refactor: move default filtered tools definition to ToolService

* feat: pass modelSpecs from backend via startupConfig

* refactor: modelSpecs config, return and define list

* fix: react error and include iconURL in responseMessage

* refactor: add iconURL to responseMessage only

* refactor: getIconEndpoint

* refactor: pass TSpecsConfig

* fix(assistants): differentiate compactAssistantSchema, correctly resets shared conversation state with other endpoints

* refactor: assistant id prefix localStorage key

* refactor: add more LocalStorageKeys and replace hardcoded values

* feat: prioritize spec on new chat behavior: last selected modelSpec behavior (localStorage)

* feat: first pass, interface config

* chore: WIP, todo: add warnings based on config.modelSpecs settings.

* feat: enforce modelSpecs if configured

* feat: show config file yaml errors

* chore: delete unused legacy Plugins component

* refactor: set tools to localStorage from recoil store

* chore: add stable recoil setter to useEffect deps

* refactor: save tools to conversation documents

* style(MultiSelectPop): dynamic height, remove unused import

* refactor(react-query): use localstorage keys and pass config to useAvailablePluginsQuery

* feat(utils): add mapPlugins

* refactor(Convo): use conversation.tools if defined, lastSelectedTools if not

* refactor: remove unused legacy code using `useSetOptions`, remove conditional flag `isMultiChat` for using legacy settings

* refactor(PluginStoreDialog): add exhaustive-deps which are stable react state setters

* fix(HeaderOptions): pass `popover` as true

* refactor(useSetStorage): use project enums

* refactor: use LocalStorageKeys enum

* fix: prevent setConversation from setting falsy values in lastSelectedTools

* refactor: use map for availableTools state and available Plugins query

* refactor(updateLastSelectedModel): organize logic better and add note on purpose

* fix(setAgentOption): prevent reseting last model to secondary model for gptPlugins

* refactor(buildDefaultConvo): use enum

* refactor: remove `useSetStorage` and consolidate areas where conversation state is saved to localStorage

* fix: conversations retain tools on refresh

* fix(gptPlugins): prevent nullish tools from being saved

* chore: delete useServerStream

* refactor: move initial plugins logic to useAppStartup

* refactor(MultiSelectDropDown): add more pass-in className props

* feat: use tools in presets

* chore: delete unused usePresetOptions

* refactor: new agentOptions default handling

* chore: note

* feat: add label and custom instructions to agents

* chore: remove 'disabled with tools' message

* style: move plugins to 2nd column in parameters

* fix: TPreset type for agentOptions

* fix: interface controls

* refactor: add interfaceConfig, use Separator within Switcher

* refactor: hide Assistants panel if interface.parameters are disabled

* fix(Header): only modelSpecs if list is greater than 0

* refactor: separate MessageIcon logic from useMessageHelpers for better react rule-following

* fix(AppService): don't use reserved keyword 'interface'

* feat: set existing Icon for custom endpoints through iconURL

* fix(ci): tests passing for App Service

* docs: refactor custom_config.md for readability and better organization, also include missing values

* docs: interface section and re-organize docs

* docs: update modelSpecs info

* chore: remove unused files

* chore: remove unused files

* chore: move useSetIndexOptions

* chore: remove unused file

* chore: move useConversation(s)

* chore: move useDefaultConvo

* chore: move useNavigateToConvo

* refactor: use plugin install hook so it can be used elsewhere

* chore: import order

* update docs

* refactor(OpenAI/Plugins): allow modelLabel as an initial value for chatGptLabel

* chore: remove unused EndpointOptionsPopover and hide 'Save as Preset' button if preset UI visibility disabled

* feat(loadDefaultInterface): issue warnings based on values

* feat: changelog for custom config file

* docs: add additional changelog note

* fix: prevent unavailable tool selection from preset and update availableTools on Plugin installations

* feat: add `filteredTools` option in custom config

* chore: changelog

* fix(MessageIcon): always overwrite conversation.iconURL in messageSettings

* fix(ModelSpecsMenu): icon edge cases

* fix(NewChat): dynamic icon

* fix(PluginsClient): always include endpoint in responseMessage

* fix: always include endpoint and iconURL in responseMessage across different response methods

* feat: interchangeable keys for modelSpec enforcing
2024-04-30 22:11:48 -04:00
Ventz Petkov
a5cac03fa4
🚅 docs: load LiteLLM into LibreChat (#2573)
Documentation was missing one significant component, on loading librechat.yaml
2024-04-29 19:56:19 -04:00
Fuegovic
ba4fa6150e
🦙 docs: fix litellm.md (#2566) 2024-04-28 08:33:51 -04:00
Neelesh Kumar
463ca5d613
📄 docs: Update apipie fetch.py in ai_endpoints.md (#2547)
* Update apipie fetch.py in ai_endpoints.md

Made the python code more pythonic

* fix bug that caused duplicate model_ids
2024-04-27 18:37:33 -04:00
Neelesh Kumar
039c7ae880
📄 docs: update GOOGLE_MODELS in .env.example (#2506)
* update GOOGLE_MODELS in .env.example

* Update .env.example

* Update .env.example

* Update .env.example

* Update .env.example

---------

Co-authored-by: Danny Avila <danacordially@gmail.com>
2024-04-27 18:36:54 -04:00
Danny Avila
63ef15ab63
🦙 feat: Fetch list of Ollama Models (#2565)
* 🦙 feat: Fetch list of Ollama Models

* style: better Tag text styling for light mode
2024-04-27 18:27:04 -04:00
Ventz Petkov
8a78500fe2
🚅 docs: update LiteLLM config with more models (#2553)
Added all OpenAI, Azure OpenAI, Amazon Bedrock, and Google GCP models available.

Specifically latest: Llama3; Google Gemini 1.5 Pro Preview; Claude Opus.

Here are all of the models:
  - model_name: claude-3-haiku
  - model_name: claude-3-sonnet
  - model_name: claude-3-opus
  - model_name: claude-v2
  - model_name: claude-instant
  - model_name: llama2-13b
  - model_name: llama2-70b
  - model_name: llama3-8b
  - model_name: llama3-70b
  - model_name: mistral-7b-instruct
  - model_name: mixtral-8x7b-instruct
  - model_name: mixtral-large
  - model_name: cohere-command-v14
  - model_name: cohere-command-light-v14
  - model_name: ai21-j2-mid
  - model_name: ai21-j2-ultra
  - model_name: amazon-titan-lite
  - model_name: amazon-titan-express
  - model_name: azure-gpt-4-turbo-preview
  - model_name: azure-gpt-3.5-turbo
  - model_name: azure-gpt-4
  - model_name: azure-gpt-3.5-turbo-16k
  - model_name: azure-gpt-4-32k
  - model_name: gpt-4-turbo
  - model_name: old-gpt-4-turbo-preview
  - model_name: gpt-3.5-turbo
  - model_name: gpt-4
  - model_name: gpt-3.5-turbo-16k
  - model_name: gpt-4-32k
  - model_name: gpt-4-vision-preview
  - model_name: google-chat-bison
  - model_name: google-chat-bison-32k
  - model_name: google-gemini-pro-1.0
  - model_name: google-gemini-pro-1.5-preview
2024-04-27 06:46:20 -04:00
Fuegovic
144fd5f6aa
⚙️ docs: update dotenv.md (#2551)
* ⚙️ docs: update dotenv.md

* Update dotenv.md - formatting
2024-04-26 14:34:50 -04:00
Danny Avila
2720327aa1
👩‍💻 fix: Minor UI fixes (#2548)
* fix(useMessageHelpers): define iconEndpoint

* fix: rely on Assistant Switcher effect for defining `assistant_id`, ensure ChatRoute `newConversation` only fires once
2024-04-26 10:27:49 -04:00
Martin Dahlö
4d0806d3e8
📋 refactor: allow paste in confirm field when resetting passwords (#2542)
* Disabled paste prevention in the confirm password field when resetting passwords.

* chore(ResetPassword): remove comments

---------

Co-authored-by: Danny Avila <danacordially@gmail.com>
2024-04-25 20:26:33 -04:00
Neelesh Kumar
5b5f9b950b
🐋 refactor: Update docker-compose.yml (#2507) 2024-04-25 20:25:51 -04:00
いしかず
3ccff19821
🌍: Update Japanese translation (#2519)
* Update japanese translation

* translate japanese

* Added 5 Japanese translations

---------

Co-authored-by: k-ishii1020 <40662914+8492GitKaz@users.noreply.github.com>
2024-04-25 13:55:59 -04:00
Marco Beretta
11d5e232b3
🧪 refactor(isDomainAllowed): change directory, add tests (#2539) 2024-04-25 13:14:07 -04:00
Danny Avila
099aa9dead
feat: Stop Sequences for Conversations & Presets (#2536)
* feat: `stop` conversation parameter

* feat: Tag primitive

* feat: dynamic tags

* refactor: update tag styling

* feat: add stop sequences to OpenAI settings

* fix(Presentation): prevent `SidePanel` re-renders that flicker side panel

* refactor: use stop placeholder

* feat: type and schema update for `stop` and `TPreset` in generation param related types

* refactor: pass conversation to dynamic settings

* refactor(OpenAIClient): remove default handling for `modelOptions.stop`

* docs: fix Google AI Setup formatting

* feat: current_model

* docs: WIP update

* fix(ChatRoute): prevent default preset override before `hasSetConversation.current` becomes true by including latest conversation state as template

* docs: update docs with more info on `stop`

* chore: bump config_version

* refactor: CURRENT_MODEL handling
2024-04-25 11:40:17 -04:00
Danny Avila
4121818124
✍️ fix(useTextarea): Rich Text Format paste from MS Word (#2530) 2024-04-25 00:10:41 -04:00
Fuegovic
ca9a0fe629
🥧 feat: APIpie support (#2524) 2024-04-24 20:32:18 -04:00
Danny Avila
bde6bb0152
🔀 fix: Remove use of Mongo Transactions (#2525)
* fix: remove use of transactions

* fix: remove use of transactions

* refactor(actions): perform OpenAI API operation first, before attempting database updates
2024-04-24 16:04:46 -04:00
ilsubyeega
667f5f91fe
👤 fix: Use user?.username if user?.name is undefined (#2511)
* Use `user?.username` if `user?.name` is undefined

* Add useLocalize hook to Icon component
2024-04-24 12:34:01 -04:00
Mathieu Breton
75da75be08
🛂 feat(oauth): add domain restriction on social login (#2512) 2024-04-24 12:14:27 -04:00
Danny Avila
cdab1e9cda
🔒 fix: package-lock file for cross-compatibility (#2515) 2024-04-24 10:42:18 -04:00
Danny Avila
3df4fac118
v0.7.1 (#2502)
* chore: make openai package definition explicit

*  v0.7.1

* chore: gpt-4-vision correct context length

* add `llava` to vision models list
2024-04-23 08:57:20 -04:00
Danny Avila
0ae98ff011
🧪 ci: Add .env.test for backend-review.yml (#2501)
* 🧪 ci: Add `.env.test` for `backend-review.yml`

* chore: touch example file
2024-04-23 08:21:12 -04:00
Walber Cardoso
4d05e5b79a
👟 style: WrenchIcon and ImageGen SVG Animations (#2382)
* refactor(WrenchIcon, ImageGen SVG)

* refactor(WrenchIcon SVG)

* refactor(ImageGen SVG)

* refactor(ImageGen SVG) - add CSS
2024-04-23 08:14:28 -04:00
Danny Avila
199f9f32e6
🐋 fix(Dockerfile.multi): Resolve OpenAI SDK @ Latest for Assistants v1 2024-04-22 20:23:28 -04:00
Danny Avila
f94a782b4f
💻 fix(client): Allow Code Filetypes and Suppress Known Vite Warnings (#2492)
* refactor(vite): suppress known warnings

* fix: allow known code filetypes if there is a mismatch of expected type, or originalFile.type is empty

* refactor(useFileHandling): naming, use variable
2024-04-22 20:08:34 -04:00
Fuegovic
738207de50
✏️docs: remove "copilot-gpt4-service" (#2491)
removed "copilot-gpt4-service", the repository has been disabled
2024-04-22 16:35:05 -04:00
Danny Avila
c96f067689
🔧 fix: Resolve Proper Dependencies to fix Application Error (#2488)
* chore: bump data-provider

* feat: script to check recent dependency updates

* fix: override vite/rollup version for vite build fix
- also remove unused vite-plugin-html
- add vite build to file output command

* chore: bump rollup override to last known working version (v4.16.0 is breaking)

* chore(vite): increase file size cache for workbox

* fix: resolve openai to last known version using assistants v1 latest features and default header

* chore: update openrouter examples
2024-04-22 12:52:30 -04:00
Danny Avila
3bfd185cab
feat: Added PWA Setup & Manual Chunks via Vite (#2477)
* added pwa setup via vite config

Added apple status bar meta data

added maskable 512 icon for chrome and android devices

added vite-plugin-pwa

updated vite config to setup the pwa service worker and manifest upon build

* fix(vite): avoid pre-caching generated images

* chore: add manual chunking of larger vendor package

* chore: remove comments

---------

Co-authored-by: davecrab <65996799+davecrab@users.noreply.github.com>
2024-04-21 10:39:15 -04:00
Danny Avila
c937b8cd07
🧑‍💻 refactor: Display Client-facing Errors (#2476)
* fix(Google): allow presets to configure expected maxOutputTokens

* refactor: standardize client-facing errors

* refactor(checkUserKeyExpiry): pass endpoint instead of custom message

* feat(UserService): JSDocs and getUserKeyValues

* refactor: add NO_BASE_URL error type, make use of getUserKeyValues, throw user-specific errors

* ci: update tests with recent changes
2024-04-21 08:31:54 -04:00
Passerby1011
6db91978ca
🎨 style: update CodeSherpa icon (#2417)
The icon address error leads to the icon not displaying.
2024-04-20 15:09:59 -04:00
Danny Avila
8c22bb1d3d
🛠️ fix(Azure/Assistants): Handle Long Domain Names & Other Minor chores (#2475)
* chore: replace violation cache accessors with enum

* chore: fix test

* chore(fileSchema): index timestamps

* fix(ActionService): use encoding/caching strategy for handling assistant function character length limit

* refactor(actions): async `domainParser` also resolve retrieved model (which is deployment name) to user-defined model

* style(AssistantAction): add `whitespace-nowrap` for ellipsis

* refactor(ActionService): if domain is less than or equal to encoded domain fixed length, return domain with replacement of separator

* refactor(actions): use sessions/transactions for updating Assistant Action database records

* chore: remove TTL from ENCODED_DOMAINS cache

* refactor(domainParser): minor optimization and add tests

* fix(spendTokens): use txData.user for token usage logging

* refactor(actions): add helper function `withSession` for database operations with sessions/transactions

* fix(PluginsClient): logger debug `message` field edge case
2024-04-20 15:02:56 -04:00
Noah Ispas
5d642d0187
📙 docs: Remove duplicate information (#2451) 2024-04-19 22:57:40 -04:00
Fuegovic
4196a86fa9
🦙 doc update: llama3 (#2470)
* docs: update breaking_changes.md

* docs: update ai_endpoints.md -> llama3 for Ollama and groq

* librechat.yaml: update groq models

* Update breaking_changes.md

logs location

* Update breaking_changes.md

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2024-04-19 21:40:12 -04:00
Danny Avila
e6310c806a
🤖 fix: Minor Assistants Endpoint Fixes (#2472)
* fix(useCreateAssistantMutation): force re-render of assistants map by avoiding use shallow reference of listRes.data

* fix(AppService): regression by not including azure assistant defaults when no assistant endpoint values are set
2024-04-19 19:05:25 -04:00
Danny Avila
3d1dec62a4
🧼 refactor(AppService): Consolidate Logic & Issue more Warnings (#2468)
* chore: bump example config version

* refactor(AppService): issue warnings from separate modules where possible

* refactor(AppService): consolidate AppService logic to separate modules as much as possible

* chore: bump data-provider

* chore: remove unn. variable definition

* chore: warning wording
2024-04-19 12:05:39 -04:00
Danny Avila
de3987cbaf
🛂 refactor(openidStrategy): Use Strategy Functions for Avatars (#2467) 2024-04-19 09:12:55 -04:00
Fuegovic
f406a85633
🖥️ docs: env changes v0.6.10→v0.7.0+ (#2442)
* docs: env changes v0.6.10→v0.7.0+ | update railway referral

* docs: env changes v0.6.10→v0.7.0+
2024-04-18 08:27:52 -04:00
Danny Avila
692ce3b346
🛠️ fix: Merge Textarea Ref with Form for Simplified Handling (#2456)
* share ref correctly

* chore: remove extraneous textarea handlers and add excel text data
2024-04-18 08:19:52 -04:00
Fuegovic
26ea990045
📘 docs: update docker_compose_install.md (#2447)
* Update docker_compose_install.md

* Update docker_compose_install.md
2024-04-18 07:31:44 -04:00
Fuegovic
265abbc1c8
⚙️ docs: Update .env.example (#2449) 2024-04-18 07:30:46 -04:00
marlonka
0b7da72be6
🌍 : Update German Translations (#2409)
* Improved and additional German translations

* Improved and added german translations and removed language translations here

* Spelling error fixed

* Use single quotes instead of double

* Use single quotes instead of double
2024-04-18 07:26:44 -04:00
Danny Avila
3c184e9410
🛠️ fix: Ensure imageOutputType is Always Defined (#2438)
* avatar fix

* chore: ensure `imageOutputType` is always defined

* ci(AppService): extra test for default value

* chore: replace default value for `desiredFormat` with `EImageOutputType` enum
2024-04-16 16:34:19 -04:00
Danny Avila
bf4e64ce63
🪙 fix(Google): Update maxOutputTokens Condition (#2434)
- generalize condition to fix max output tokens for gemini-1.5
2024-04-16 10:23:13 -04:00
Danny Avila
9d854dac07
🤖 feat: Gemini 1.5 Support (+Vertex AI) (#2383)
* WIP: gemini-1.5 support

* feat: extended vertex ai support

* fix: handle possibly undefined modelName

* fix: gpt-4-turbo-preview invalid vision model

* feat: specify `fileConfig.imageOutputType` and make PNG default image conversion type

* feat: better truncation for errors including base64 strings

* fix: gemini inlineData formatting

* feat: RAG augmented prompt for gemini-1.5

* feat: gemini-1.5 rates and token window

* chore: adjust tokens, update docs, update vision Models

* chore: add back `ChatGoogleVertexAI` for chat models via vertex ai

* refactor: ask/edit controllers to not use `unfinished` field for google endpoint

* chore: remove comment

* chore(ci): fix AppService test

* chore: remove comment

* refactor(GoogleSearch): use `GOOGLE_SEARCH_API_KEY` instead, issue warning for old variable

* chore: bump data-provider to 0.5.4

* chore: update docs

* fix: condition for gemini-1.5 using generative ai lib

* chore: update docs

* ci: add additional AppService test for `imageOutputType`

* refactor: optimize new config value `imageOutputType`

* chore: bump CONFIG_VERSION

* fix(assistants): avatar upload
2024-04-16 08:32:40 -04:00
Danny Avila
fce7246ac1
🔓 refactor: Make Image URL Security Optional (#2415) 2024-04-14 19:34:13 -04:00