Commit graph

1443 commits

Author SHA1 Message Date
Fuegovic
51050cc4d3
🧹📚 docs: refactor and clean up (#1392)
* 📑 update mkdocs

* rename docker override file and add to gitignore

* update .env.example - GOOGLE_MODELS

* update index.md

* doc refactor: split installation and configuration in two sub-folders

* doc update: installation guides

* doc update: configuration guides

* doc: new docker override guide

* doc: new beginner's guide for contributions - Thanks @Berry-13

* doc: update documentation_guidelines.md

* doc: update testing.md

* doc: update deployment guides

* doc: update /dev readme

* doc: update general_info

* doc: add 0 value to doc weight

* doc: add index.md to every doc folders

* doc: add weight to index.md and move openrouter from free_ai_apis.md to ai_setup.md

* doc: update toc so they display properly on the right had side in mkdocs

* doc: update pandoranext.md

* doc: index logging_system.md

* doc: update readme.md

* doc: update litellm.md

* doc: update ./dev/readme.md

* doc:🔖 new presets.md

* doc: minor corrections

* doc update: user_auth_system.md and presets.md, doc feat: add mermaid support to mkdocs

* doc update: add screenshots to presets.md

* doc update: add screenshots to - OpenID with AWS Cognito

* doc update: BingAI cookie instruction

* doc update: discord auth

* doc update: facebook auth

* doc: corrections to user_auth_system.md

* doc update: github auth

* doc update: google auth

* doc update: auth clean up

* doc organization: installation

* doc organization: configuration

* doc organization: features+plugins & update:plugins screenshots

* doc organization: deploymend + general_info  & update: tech_stack.md

* doc organization: contributions

* doc: minor fixes

* doc: minor fixes
2023-12-22 08:36:42 -05:00
Danny Avila
5c27fa304a
Update pull_request_template.md (#1417) 2023-12-22 07:26:24 -05:00
Danny Avila
5b28362282
Release v0.6.5 (#1391)
*  Release v0.6.5

* fix(ci): use dynamic currentDateString
2023-12-19 01:09:42 -05:00
Danny Avila
8d563d61f1
feat: Azure Vision Support & Docs Update (#1389)
* feat(AzureOpenAI): Vision Support

* chore(ci/OpenAIClient.test): update test to reflect Azure now uses chatCompletion method as opposed to getCompletion, while still testing the latter method

* docs: update documentation mainly revolving around Azure setup, but also reformatting the 'Tokens and API' section completely

* docs: add images and links to ai_setup.md

* docs: ai setup reference
2023-12-18 18:43:50 -05:00
Danny Avila
c9d3e0ab6a
🚩 fix: Initialize Conversation Only when Necessary Data is Fetched (#1379)
* fix(ChatRoute): only initialize conversation after all data is fetched (models, endpoints, initialConversationQuery if not `new`)

* chore: remove unnecessary packages for rolling up api

* chore: bump data-provider package.json
2023-12-17 18:56:01 -05:00
Danny Avila
7c2134fb12
fix: revert fonts resolution path in vite.config.ts 2023-12-17 16:36:24 -05:00
Danny Avila
0c326797dd
📸 feat: Gemini vision, Improved Logs and Multi-modal Handling (#1368)
* feat: add GOOGLE_MODELS env var

* feat: add gemini vision support

* refactor(GoogleClient): adjust clientOptions handling depending on model

* fix(logger): fix redact logic and redact errors only

* fix(GoogleClient): do not allow non-multiModal messages when gemini-pro-vision is selected

* refactor(OpenAIClient): use `isVisionModel` client property to avoid calling validateVisionModel multiple times

* refactor: better debug logging by correctly traversing, redacting sensitive info, and logging condensed versions of long values

* refactor(GoogleClient): allow response errors to be thrown/caught above client handling so user receives meaningful error message
debug orderedMessages, parentMessageId, and buildMessages result

* refactor(AskController): use model from client.modelOptions.model when saving intermediate messages, which requires for the progress callback to be initialized after the client is initialized

* feat(useSSE): revert to previous model if the model was auto-switched by backend due to message attachments

* docs: update with google updates, notes about Gemini Pro Vision

* fix: redis should not be initialized without USE_REDIS and increase max listeners to 20
2023-12-16 20:45:27 -05:00
Fuegovic
676f133545
🍏Update mac_install.md (#1373)
The manual installation guide for mac was very outdated. This brings it up to date with the current method
2023-12-16 20:26:47 -05:00
Danny Avila
2dfade1c42
Update package.json 2023-12-15 15:52:56 -05:00
Danny Avila
509b1e5c63
🔄 refactor: Consolidate Ask/Edit Controllers (#1365)
* refactor(Ask/Edit): consolidate ask/edit controllers between the main modules and openAI controllers to reduce repetition of code and increase reusability

* fix(winston/logger): circular dependency issue

* fix(config/scripts): fix script imports

* refactor(indexSync): make not configured message an info log message

* chore: create a rollup script for api/server/index.js to check circular dependencies

* chore: bump @keyv/redis
2023-12-15 15:47:40 -05:00
Danny Avila
0958db3825
fix: Enhance Test Coverage and Fix Compatibility Issues 👷‍♂️ (#1363)
* refactor: only remove conversation states from localStorage on login/logout but not on refresh

* chore: add debugging log for azure completion url

* chore: add api-key to redact regex

* fix: do not show endpoint selector if endpoint is falsy

* chore: remove logger from genAzureChatCompletion

* feat(ci): mock fetchEventSource

* refactor(ci): mock all model methods in BaseClient.test, as well as mock the implementation for getCompletion in FakeClient

* fix(OpenAIClient): consider chatCompletion if model name includes `gpt` as opposed to `gpt-`

* fix(ChatGPTClient/azureOpenAI): Remove 'model' option for Azure compatibility (cannot be sent in payload body)

* feat(ci): write new test suite that significantly increase test coverage for OpenAIClient and BaseClient by covering most of the real implementation of the `sendMessage` method
- test for the azure edge case where model option is appended to modelOptions, ensuring removal before sent to the azure endpoint
- test for expected azure url being passed to SSE POST request
- test for AZURE_OPENAI_DEFAULT_MODEL being set, but is not included in the URL deployment name as expected
- test getCompletion method to have correct payload
fix(ci/OpenAIClient.test.js): correctly mock hanging/async methods

* refactor(addTitle): allow azure to title as it aborts signal on completion
2023-12-15 13:27:13 -05:00
Fuegovic
072a7e5f05
update: docker-compose.yaml 🐋 (#1341) 2023-12-15 02:41:33 -05:00
Danny Avila
ff59a2e41d
fix: Avoid Throwing Errors for Unsupported Token Count Endpoints 🪙 (#1356) 2023-12-15 02:40:15 -05:00
Danny Avila
561ce8e86a
feat: Google Gemini ❇️ (#1355)
* refactor: add gemini-pro to google Models list; use defaultModels for central model listing

* refactor(SetKeyDialog): create useMultipleKeys hook to use for Azure, export `isJson` from utils, use EModelEndpoint

* refactor(useUserKey): change variable names to make keyName setting more clear

* refactor(FileUpload): allow passing container className string

* feat(GoogleClient): Gemini support

* refactor(GoogleClient): alternate stream speed for Gemini models

* feat(Gemini): styling/settings configuration for Gemini

* refactor(GoogleClient): substract max response tokens from max context tokens if context is above 32k (I/O max is combined between the two)

* refactor(tokens): correct google max token counts and subtract max response tokens when input/output count are combined towards max context count

* feat(google/initializeClient): handle both local and user_provided credentials and write tests

* fix(GoogleClient): catch if credentials are undefined, handle if serviceKey is string or object correctly, handle no examples passed, throw error if not a Generative Language model and no service account JSON key is provided, throw error if it is a Generative m
odel, but not google API key was provided

* refactor(loadAsyncEndpoints/google): activate Google endpoint if either the service key JSON file is provided in /api/data, or a GOOGLE_KEY is defined.

* docs: updated Google configuration

* fix(ci): Mock import of Service Account Key JSON file (auth.json)

* Update apis_and_tokens.md

* feat: increase max output tokens slider for gemini pro

* refactor(GoogleSettings): handle max and default maxOutputTokens on model change

* chore: add sensitive redact regex

* docs: add warning about data privacy

* Update apis_and_tokens.md
2023-12-15 02:18:07 -05:00
Danny Avila
d259431316
fix: Add error handling for missing role in OpenAIClient.js (#1352) 2023-12-14 10:53:40 -05:00
Danny Avila
ea1dd59ef4
refactor(api): Central Logging 📜 (#1348)
* WIP: initial logging changes
add several transports in ~/config/winston
omit messages in logs, truncate long strings
add short blurb in dotenv for debug logging
GoogleClient: using logger
OpenAIClient: using logger, handleOpenAIErrors
Adding typedef for payload message
bumped winston and using winston-daily-rotate-file
moved config for server paths to ~/config dir
Added `DEBUG_LOGGING=true` to .env.example

* WIP: Refactor logging statements in code

* WIP: Refactor logging statements and import configurations

* WIP: Refactor logging statements and import configurations

* refactor: broadcast Redis initialization message with `info` not `debug`

* refactor: complete Refactor logging statements and import configurations

* chore: delete unused tools

* fix: circular dependencies due to accessing logger

* refactor(handleText): handle booleans and write tests

* refactor: redact sensitive values, better formatting

* chore: improve log formatting, avoid passing strings to 2nd arg

* fix(ci): fix jest tests due to logger changes

* refactor(getAvailablePluginsController): cache plugins as they are static and avoids async addOpenAPISpecs call every time

* chore: update docs

* chore: update docs

* chore: create separate meiliSync logger, clean up logs to avoid being unnecessarily verbose

* chore: spread objects where they are commonly logged to allow string truncation

* chore: improve error log formatting
2023-12-14 07:49:27 -05:00
Ed Burnette
49571ac635
chore: Get the latest of all github actions (#1335) 2023-12-14 07:44:38 -05:00
Fuegovic
1f5cb71a64
documentation update: mongodb (#1347)
* update dotenv.md

add details about the MONGO_URI connection string format

* update mongodb.md

add details about the MONGO_URI connection string format
2023-12-14 07:44:20 -05:00
Fuegovic
bff365785a
update: issue templates (#1346)
* update BUG-REPORT.yml

remove the contact field asking for the email address since it exposes them to the web when used

* update FEATURE-REQUEST.yml

remove the contact field since it exposes users email address

* Update QUESTION.yml

remove the contact field since it exposes users email address
2023-12-13 12:42:18 -05:00
Fuegovic
f2fc47e741
📚 documentation update (#1340) 2023-12-13 09:45:03 -05:00
Fuegovic
44755c964f
update huggingface.md (#1342)
change the URL of the space to duplicate for a cleaner template I made
2023-12-13 06:52:21 -05:00
Danny Avila
fac2580a19
fix(librechat-data-provider): Update types paths in package.json (#1333)
* fix(librechat-data-provider): Update types paths in package.json

* chore: bump version

* fix(librechat-data-provider): Update types paths in react-query/package.json
2023-12-12 09:16:29 -05:00
Danny Avila
6829d66c1f
Update docker_compose_install.md 2023-12-11 22:44:27 -05:00
Danny Avila
4df6a261d3
Update README.md 2023-12-11 22:43:37 -05:00
Danny Avila
e69644d7b4
fix: docker issues with volume mapping (#1330)
* fix: docker issues with volume mapping

* Update docker-compose.yml

---------

Co-authored-by: stunt_pilot <twitchstuntpilot@gmail.com>
2023-12-11 22:39:24 -05:00
Fuegovic
9db3d792cc
update .env.example (#1327)
remove client_id and secret values for discord and github so they won't show in the UI without being properly configured
2023-12-11 17:23:38 -05:00
Danny Avila
df1dfa7d46
refactor: Use librechat-data-provider app-wide 🔄 (#1326)
* chore: bump vite, vitejs/plugin-react, mark client package as esm, move react-query as a peer dep in data-provider

* chore: import changes due to new data-provider export strategy, also fix type imports where applicable

* chore: export react-query services as separate to avoid react dependencies in /api/

* chore: suppress sourcemap warnings and polyfill node:path which is used by filenamify
TODO: replace filenamify with an alternative and REMOVE polyfill

* chore: /api/ changes to support `librechat-data-provider`

* refactor: rewrite Dockerfile.multi in light of /api/ changes to support `librechat-data-provider`

* chore: remove volume mapping to node_modules directories in default compose file

* chore: remove schemas from /api/ as is no longer needed with use of `librechat-data-provider`

* fix(ci): jest `librechat-data-provider/react-query` module resolution
2023-12-11 14:48:40 -05:00
Ed Burnette
d4c846b543
fix: fonts warning (#1318)
* fix: Fix a fonts warning
    The warning was:
    files in the public directory are served at the root path.
    Instead of /public/fonts/soehne-buch.woff2, use /fonts/soehne-buch.woff2.

* See if it likes /fonts better
2023-12-10 17:19:51 -05:00
Danny Avila
968b8ccdbd
🔒 fix: Robust Cache Reset on User Logout (#1324)
* refactor(Logout): rely on hooks for mutation behavior

* fix: logging out now correctly resets cache, disallowing any cache mixing between the next logged in user on the same browser

* chore: remove additional localStorage values on logout
2023-12-10 17:13:42 -05:00
Danny Avila
583e978a82
feat(Google): Support all Text/Chat Models, Response streaming, PaLM -> Google 🤖 (#1316)
* feat: update PaLM icons

* feat: add additional google models

* POC: formatting inputs for Vertex AI streaming

* refactor: move endpoints services outside of /routes dir to /services/Endpoints

* refactor: shorten schemas import

* refactor: rename PALM to GOOGLE

* feat: make Google editable endpoint

* feat: reusable Ask and Edit controllers based off Anthropic

* chore: organize imports/logic

* fix(parseConvo): include examples in googleSchema

* fix: google only allows odd number of messages to be sent

* fix: pass proxy to AnthropicClient

* refactor: change `google` altName to `Google`

* refactor: update getModelMaxTokens and related functions to handle maxTokensMap with nested endpoint model key/values

* refactor: google Icon and response sender changes (Codey and Google logo instead of PaLM in all cases)

* feat: google support for maxTokensMap

* feat: google updated endpoints with Ask/Edit controllers, buildOptions, and initializeClient

* feat(GoogleClient): now builds prompt for text models and supports real streaming from Vertex AI through langchain

* chore(GoogleClient): remove comments, left before for reference in git history

* docs: update google instructions (WIP)

* docs(apis_and_tokens.md): add images to google instructions

* docs: remove typo apis_and_tokens.md

* Update apis_and_tokens.md

* feat(Google): use default settings map, fully support context for both text and chat models, fully support examples for chat models

* chore: update more PaLM references to Google

* chore: move playwright out of workflows to avoid failing tests
2023-12-10 14:54:13 -05:00
Ed Burnette
8a1968b2f8
chore: Clean up the devcontainer files (#1317)
* Spruce up docker-compose for dev container

* Update devcontainer.json to remove unused lines

* It's /workspaces not /workspace
2023-12-10 09:49:21 -05:00
Marco Beretta
34d2da1ffc
📚 docs: Facebook update (#1312)
* Update user_auth_system.md

* Update user_auth_system.md

* Update user_auth_system.md
2023-12-08 14:05:24 -05:00
eniyiweb
427b05b891
🌎: Additional Turkish translations (#1310)
* Language translation:Turkish translation update

* 🌍: Turkish Translation - Update

* Additional Turkish translations

* Additional Turkish translations
2023-12-08 06:58:47 -05:00
Fuegovic
c2d8ae8616
📚 documentation update (#1307)
* fix .env.example

comment out DALLE_API_KEY by default and remove `user_provided` as default value since it is not supported

* 📚 docs update
2023-12-07 22:44:26 -05:00
Danny Avila
9041fe7472
feat: Make Nav Visible by Default on Clean Install 🧭 (#1309) 2023-12-07 22:42:14 -05:00
Marco Beretta
20b93ad065
feat: open sidebar popup for new users (#1308) 2023-12-07 22:40:23 -05:00
Danny Avila
10ace5fa75
🌎: Add Supported Languages to README.md 2023-12-07 14:37:56 -05:00
Fuegovic
b822cd48d2
bug fix: remove 3rd party code interpreter (#1306) 2023-12-07 14:27:52 -05:00
Ken Taniguchi
4d528efaf6
🇯🇵🗾: Japanese Translations (add and improve) (#1298) 2023-12-07 09:35:22 -05:00
Danny Avila
0bae503a0a
refactor: Speed up Config fetching and Setup Config Groundwork 👷🚧 (#1297)
* refactor: move endpoint services to own directory

* refactor: make endpointconfig handling more concise, separate logic, and cache result for subsequent serving

* refactor: ModelController gets same treatment as EndpointController, draft OverrideController

* wip: flesh out override controller more to return real value

* refactor: client/api changes in anticipation of override
2023-12-06 19:36:57 -05:00
Danny Avila
9b2359fc27
🛠️ refactor: Improve Input Placeholder Handling and Error Management 🔄 (#1296)
* chore: identify new chat buttons with testid

* fix: avoid parsing error in useSSE, which causes errorHandler to fail

* fix: ensure last message isn't setting latestMessage when conversationId is `new` and text is the same due to possible re-renders

* refactor: set placeholder through inputRef and useEffect

* Update useSSE.ts

* Update useSSE.ts
2023-12-06 14:10:06 -05:00
Marco Beretta
2e390596ea
return 404 instead of 200 (#1294) 2023-12-06 14:08:15 -05:00
Danny Avila
ca64efec1b
feat: Implement Default Preset Selection for Conversations 📌 (#1275)
* fix: type issues with icons

* refactor: use react query for presets, show toasts on preset crud, refactor mutations, remove presetsQuery from Root (breaking change)

* refactor: change preset titling

* refactor: update preset schemas and methods for necessary new properties `order` and `defaultPreset`

* feat: add `defaultPreset` Recoil value

* refactor(getPresetTitle): make logic cleaner and more concise

* feat: complete UI portion of defaultPreset feature, with animations added to preset items

* chore: remove console.logs()

* feat: complete default preset handling

* refactor: remove user sensitive values on logout

* fix: allow endpoint selection without default preset overwriting
2023-12-06 14:00:15 -05:00
Marco Beretta
fdb65366d7
📧 feat: disable login (ALLOW_EMAIL_LOGIN) (#1282)
* added ALLOW_EMAIL_LOGIN

* update .env.example

* fix(config) email login true by default

* Update dotenv.md
2023-12-06 07:08:49 -05:00
Fuegovic
1706886a64
doc: PandoraNext guide (#1276) 2023-12-06 06:57:10 -05:00
Danny Avila
00b6af8c74
fix: Ensure Message Send Requires Key 🔑 (#1281)
* fix: only allow message send when key is provided when required
- create useRequiresKey hook
- pass same disabled prop to Textarea, AttachFile, and SendButton
- EndpointItem: add localization, stopPropagation, and remove commented code
- separate some hooks to new Input dir
- completely remove textareaHeight recoil state as is not needed
- update imports for moved hooks
- pass disabled prop to useTextarea

* feat: add localization to textarea placeholders
2023-12-05 09:38:04 -05:00
Fuegovic
f6118879e5
🔎 update: meilisearch (#1269) 2023-12-05 08:38:41 -05:00
MACHINSOFT
270031c783
🪟 Minor UI Styling Changes (#1260)
* Correct the display of the interface when hovering with the mouse

* Replace ring-1 with border completely.

* Fix it so that the text does not overlap with the SVG in mobile navigation.

* Remove the extra class -ml-0.5 -mt-0.5, there is no need to shift the buttons
2023-12-05 08:37:36 -05:00
Danny Avila
f1bc711cd7
🐛 fix: Prevent Node Server Crash Due to Unhandled ChatCompletionMessage Error (#1278)
* refactor(addTitle): avoid generating title when a request was aborted

* chore: bump openai to latest

* fix: catch OpenAIError Uncaught error as last resort

* fix: handle final messages excludes role=assistant

* Update OpenAIClient.js

* chore: fix linting errors
2023-12-04 22:58:23 -05:00
eniyiweb
076a9b9b9c
🇹🇷: Additional Turkish translations (#1265)
* Language translation:Turkish translation update

* 🌍: Turkish Translation - Update

* Additional Turkish translations
2023-12-04 16:50:33 -05:00