* refactor: post-cleanup changes:
- add more unnecessary paths to .dockerignore
- remove librechat.yaml from main compose file (prevents from being required)
- do not create librechat.yaml during build (does nothing)
* docs: make config file instructions easier to read, more info throughout other docs
* docs: add custom config to menu
* Update custom_config.md
* Update docker_compose_install.md
* Improving builds
When adding LibreChat to ansible, it rebuilt way too often, even if I
only changed the configuration.
With this PR, it should build only when the files of the app change.
Also removed the 'volumes' section for the 'api' in the docker-compose.yml.
At least with our installation it works fine like this.
* @danny-avila's comments
- removed 'env_file' from docker-compose.yml
- re-added link to '.env' in volumes
* Adding latest changes from main
* @danny-avila's comments
* Updating installation instructions
* @danny-avila's comments
- Remove unused environment in docker-compose.yml
- Re-add some steps for cleaning docker images
* WIP(backend/api): custom endpoint
* WIP(frontend/client): custom endpoint
* chore: adjust typedefs for configs
* refactor: use data-provider for cache keys and rename enums and custom endpoint for better clarity and compatibility
* feat: loadYaml utility
* refactor: rename back to from and proof-of-concept for creating schemas from user-defined defaults
* refactor: remove custom endpoint from default endpointsConfig as it will be exclusively managed by yaml config
* refactor(EndpointController): rename variables for clarity
* feat: initial load custom config
* feat(server/utils): add simple `isUserProvided` helper
* chore(types): update TConfig type
* refactor: remove custom endpoint handling from model services as will be handled by config, modularize fetching of models
* feat: loadCustomConfig, loadConfigEndpoints, loadConfigModels
* chore: reorganize server init imports, invoke loadCustomConfig
* refactor(loadConfigEndpoints/Models): return each custom endpoint as standalone endpoint
* refactor(Endpoint/ModelController): spread config values after default (temporary)
* chore(client): fix type issues
* WIP: first pass for multiple custom endpoints
- add endpointType to Conversation schema
- add update zod schemas for both convo/presets to allow non-EModelEndpoint value as endpoint (also using type assertion)
- use `endpointType` value as `endpoint` where mapping to type is necessary using this field
- use custom defined `endpoint` value and not type for mapping to modelsConfig
- misc: add return type to `getDefaultEndpoint`
- in `useNewConvo`, add the endpointType if it wasn't already added to conversation
- EndpointsMenu: use user-defined endpoint name as Title in menu
- TODO: custom icon via custom config, change unknown to robot icon
* refactor(parseConvo): pass args as an object and change where used accordingly; chore: comment out 'create schema' code
* chore: remove unused availableModels field in TConfig type
* refactor(parseCompactConvo): pass args as an object and change where used accordingly
* feat: chat through custom endpoint
* chore(message/convoSchemas): avoid saving empty arrays
* fix(BaseClient/saveMessageToDatabase): save endpointType
* refactor(ChatRoute): show Spinner if endpointsQuery or modelsQuery are still loading, which is apparent with slow fetching of models/remote config on first serve
* fix(useConversation): assign endpointType if it's missing
* fix(SaveAsPreset): pass real endpoint and endpointType when saving Preset)
* chore: recorganize types order for TConfig, add `iconURL`
* feat: custom endpoint icon support:
- use UnknownIcon in all icon contexts
- add mistral and openrouter as known endpoints, and add their icons
- iconURL support
* fix(presetSchema): move endpointType to default schema definitions shared between convoSchema and defaults
* refactor(Settings/OpenAI): remove legacy `isOpenAI` flag
* fix(OpenAIClient): do not invoke abortCompletion on completion error
* feat: add responseSender/label support for custom endpoints:
- use defaultModelLabel field in endpointOption
- add model defaults for custom endpoints in `getResponseSender`
- add `useGetSender` hook which uses EndpointsQuery to determine `defaultModelLabel`
- include defaultModelLabel from endpointConfig in custom endpoint client options
- pass `endpointType` to `getResponseSender`
* feat(OpenAIClient): use custom options from config file
* refactor: rename `defaultModelLabel` to `modelDisplayLabel`
* refactor(data-provider): separate concerns from `schemas` into `parsers`, `config`, and fix imports elsewhere
* feat: `iconURL` and extract environment variables from custom endpoint config values
* feat: custom config validation via zod schema, rename and move to `./projectRoot/librechat.yaml`
* docs: custom config docs and examples
* fix(OpenAIClient/mistral): mistral does not allow singular system message, also add `useChatCompletion` flag to use openai-node for title completions
* fix(custom/initializeClient): extract env var and use `isUserProvided` function
* Update librechat.example.yaml
* feat(InputWithLabel): add className props, and forwardRef
* fix(streamResponse): handle error edge case where either messages or convos query throws an error
* fix(useSSE): handle errorHandler edge cases where error response is and is not properly formatted from API, especially when a conversationId is not yet provided, which ensures stream is properly closed on error
* feat: user_provided keys for custom endpoints
* fix(config/endpointSchema): do not allow default endpoint values in custom endpoint `name`
* feat(loadConfigModels): extract env variables and optimize fetching models
* feat: support custom endpoint iconURL for messages and Nav
* feat(OpenAIClient): add/dropParams support
* docs: update docs with default params, add/dropParams, and notes to use config file instead of `OPENAI_REVERSE_PROXY`
* docs: update docs with additional notes
* feat(maxTokensMap): add mistral models (32k context)
* docs: update openrouter notes
* Update ai_setup.md
* docs(custom_config): add table of contents and fix note about custom name
* docs(custom_config): reorder ToC
* Update custom_config.md
* Add note about `max_tokens` field in custom_config.md
* Refactoring opening of DB to config/helpers.js
* Adding two user scripts:
- 'delete-user' to remove a user definitely
- 'list-balances' to show the balances of all the users
* localization + api-endpoint
* docs: added firebase documentation
* chore: icons
* chore: SettingsTabs
* feat: account pannel; fix: gear icons
* docs: position update
* feat: firebase
* feat: plugin support
* route
* fixed bugs with firebase and moved a lot of files
* chore(DALLE3): using UUID v4
* feat: support for social strategies; moved '/images' path
* fix: data ignored
* gitignore update
* docs: update firebase guide
* refactor: Firebase
- use singleton pattern for firebase initialization, initially on server start
- reorganize imports, move firebase specific files to own service under Files
- rename modules to remove 'avatar' redundancy
- fix imports based on changes
* ci(DALLE/DALLE3): fix tests to use logger and new expected outputs, add firebase tests
* refactor(loadToolWithAuth): pass userId to tool as field
* feat(images/parse): feat: Add URL Image Basename Extraction
Implement a new module to extract the basename of an image from a given URL. This addition includes the function, which parses the URL and retrieves the basename using the Node.js 'url' and 'path' modules. The function is documented with JSDoc comments for better maintainability and understanding. This feature enhances the application's ability to handle and process image URLs efficiently.
* refactor(addImages): function to use a more specific regular expression for observedImagePath based on the generated image markdown standard across the app
* refactor(DALLE/DALLE3): utilize `getImageBasename` and `this.userId`; fix: pass correct image path to firebase url helper
* fix(addImages): make more general to match any image markdown descriptor
* fix(parse/getImageBasename): test result of this function for an actual image basename
* ci(DALLE3): mock getImageBasename
* refactor(AuthContext): use Recoil atom state for user
* feat: useUploadAvatarMutation, react-query hook for avatar upload
* fix(Toast): stack z-order of Toast over all components (1000)
* refactor(showToast): add optional status field to avoid importing NotificationSeverity on each use of the function
* refactor(routes/avatar): remove unnecessary get route, get userId from req.user.id, require auth on POST request
* chore(uploadAvatar): TODO: remove direct use of Model, `User`
* fix(client): fix Spinner imports
* refactor(Avatar): use react-query hook, Toast, remove unnecessary states, add optimistic UI to upload
* fix(avatar/localStrategy): correctly save local profile picture and cache bust for immediate rendering; fix: firebase init info message (only show once)
* fix: use `includes` instead of `endsWith` for checking manual query of avatar image path in case more queries are appended (as is done in avatar/localStrategy)
---------
Co-authored-by: Danny Avila <messagedaniel@protonmail.com>
* mkdocs plugins: add plugin for social cards and plugin that allow to exclude a folder
* docs: fix hyperlinks
* mkdocs: social cards (descriptions) for 'contributions' and 'deployment' guides
* mkdocs: social cards (descriptions) for all 'index.md'
* mkdocs: social cards (descriptions) for 'features' and 'plugins'
* mkdocs: social cards (descriptions) for 'general_info'
* mkdocs: social cards (descriptions) for 'configuration'
* mkdocs: social cards (descriptions) for 'installation'
* mkdocs: minor fixes
* update librechat.svg
* update how_to_contribute.md
add reference to the official GitHub documentation
* feat(AzureOpenAI): Vision Support
* chore(ci/OpenAIClient.test): update test to reflect Azure now uses chatCompletion method as opposed to getCompletion, while still testing the latter method
* docs: update documentation mainly revolving around Azure setup, but also reformatting the 'Tokens and API' section completely
* docs: add images and links to ai_setup.md
* docs: ai setup reference
* feat: add GOOGLE_MODELS env var
* feat: add gemini vision support
* refactor(GoogleClient): adjust clientOptions handling depending on model
* fix(logger): fix redact logic and redact errors only
* fix(GoogleClient): do not allow non-multiModal messages when gemini-pro-vision is selected
* refactor(OpenAIClient): use `isVisionModel` client property to avoid calling validateVisionModel multiple times
* refactor: better debug logging by correctly traversing, redacting sensitive info, and logging condensed versions of long values
* refactor(GoogleClient): allow response errors to be thrown/caught above client handling so user receives meaningful error message
debug orderedMessages, parentMessageId, and buildMessages result
* refactor(AskController): use model from client.modelOptions.model when saving intermediate messages, which requires for the progress callback to be initialized after the client is initialized
* feat(useSSE): revert to previous model if the model was auto-switched by backend due to message attachments
* docs: update with google updates, notes about Gemini Pro Vision
* fix: redis should not be initialized without USE_REDIS and increase max listeners to 20
* refactor: add gemini-pro to google Models list; use defaultModels for central model listing
* refactor(SetKeyDialog): create useMultipleKeys hook to use for Azure, export `isJson` from utils, use EModelEndpoint
* refactor(useUserKey): change variable names to make keyName setting more clear
* refactor(FileUpload): allow passing container className string
* feat(GoogleClient): Gemini support
* refactor(GoogleClient): alternate stream speed for Gemini models
* feat(Gemini): styling/settings configuration for Gemini
* refactor(GoogleClient): substract max response tokens from max context tokens if context is above 32k (I/O max is combined between the two)
* refactor(tokens): correct google max token counts and subtract max response tokens when input/output count are combined towards max context count
* feat(google/initializeClient): handle both local and user_provided credentials and write tests
* fix(GoogleClient): catch if credentials are undefined, handle if serviceKey is string or object correctly, handle no examples passed, throw error if not a Generative Language model and no service account JSON key is provided, throw error if it is a Generative m
odel, but not google API key was provided
* refactor(loadAsyncEndpoints/google): activate Google endpoint if either the service key JSON file is provided in /api/data, or a GOOGLE_KEY is defined.
* docs: updated Google configuration
* fix(ci): Mock import of Service Account Key JSON file (auth.json)
* Update apis_and_tokens.md
* feat: increase max output tokens slider for gemini pro
* refactor(GoogleSettings): handle max and default maxOutputTokens on model change
* chore: add sensitive redact regex
* docs: add warning about data privacy
* Update apis_and_tokens.md
* WIP: initial logging changes
add several transports in ~/config/winston
omit messages in logs, truncate long strings
add short blurb in dotenv for debug logging
GoogleClient: using logger
OpenAIClient: using logger, handleOpenAIErrors
Adding typedef for payload message
bumped winston and using winston-daily-rotate-file
moved config for server paths to ~/config dir
Added `DEBUG_LOGGING=true` to .env.example
* WIP: Refactor logging statements in code
* WIP: Refactor logging statements and import configurations
* WIP: Refactor logging statements and import configurations
* refactor: broadcast Redis initialization message with `info` not `debug`
* refactor: complete Refactor logging statements and import configurations
* chore: delete unused tools
* fix: circular dependencies due to accessing logger
* refactor(handleText): handle booleans and write tests
* refactor: redact sensitive values, better formatting
* chore: improve log formatting, avoid passing strings to 2nd arg
* fix(ci): fix jest tests due to logger changes
* refactor(getAvailablePluginsController): cache plugins as they are static and avoids async addOpenAPISpecs call every time
* chore: update docs
* chore: update docs
* chore: create separate meiliSync logger, clean up logs to avoid being unnecessarily verbose
* chore: spread objects where they are commonly logged to allow string truncation
* chore: improve error log formatting
* update dotenv.md
add details about the MONGO_URI connection string format
* update mongodb.md
add details about the MONGO_URI connection string format
* feat: update PaLM icons
* feat: add additional google models
* POC: formatting inputs for Vertex AI streaming
* refactor: move endpoints services outside of /routes dir to /services/Endpoints
* refactor: shorten schemas import
* refactor: rename PALM to GOOGLE
* feat: make Google editable endpoint
* feat: reusable Ask and Edit controllers based off Anthropic
* chore: organize imports/logic
* fix(parseConvo): include examples in googleSchema
* fix: google only allows odd number of messages to be sent
* fix: pass proxy to AnthropicClient
* refactor: change `google` altName to `Google`
* refactor: update getModelMaxTokens and related functions to handle maxTokensMap with nested endpoint model key/values
* refactor: google Icon and response sender changes (Codey and Google logo instead of PaLM in all cases)
* feat: google support for maxTokensMap
* feat: google updated endpoints with Ask/Edit controllers, buildOptions, and initializeClient
* feat(GoogleClient): now builds prompt for text models and supports real streaming from Vertex AI through langchain
* chore(GoogleClient): remove comments, left before for reference in git history
* docs: update google instructions (WIP)
* docs(apis_and_tokens.md): add images to google instructions
* docs: remove typo apis_and_tokens.md
* Update apis_and_tokens.md
* feat(Google): use default settings map, fully support context for both text and chat models, fully support examples for chat models
* chore: update more PaLM references to Google
* chore: move playwright out of workflows to avoid failing tests
* Update azure_cognitive_search.md
* Updated: Azure Cognitive Search Plugin to Azure AI Search Plugin.
Update Docs: Azure Cognitive Search Plugin to Azure AI Search Plugin.
Updated:.env.example Azure Cognitive Search to Azure AI Search
Updated: mkdocs.yml link
Updated: SDK Azure 11.3.2 to 12.0.0
* fix:.env AZURE- to AZURE_
* Update azure_ai_search.md
* Updated:(api/package.json, package-lock.json): updated for new version the
plugin (@azure/search-documents)
* fix:Resolved incorrect file name AzureAISearch
* fix:.env Azure AI Search
* fix:"-" to "_"
* Update Docs: Azure AI Search ith an improved tutorial featuring images and easier-to-understand instructions
fix: Change name of plugin "Azure Ai Search" to "Azure AI Search" i
* Update:Version of REST API versions (Azure AI Search)
* Update azure_ai_search.md
* Update azure_ai_search.md
* Update azure_ai_search.md
* fix: docs Azure AI Seach Images were not appearing.
* fix:Updated to the new repository with working APIs
* Update: Added Compatibility for Previous Environment Variable Names in AzureAISearch Plugin
* Update: Added Compatibility for Previous Environment Variable Names in AzureAISearch Plugin
* Update: Added Compatibility for Previous Environment Variable Names in AzureAISearch Plugin
* Update: Added Compatibility for Previous Environment Variable Names in AzureAISearch Plugin
* Update: o AzureAiSearch.js
* Atualizar o AzureAISearch.js
* Update/fix:EnvironmentVariablesForDeprecation
* fix:The file is outdated and needs to be updated.
* fix:The file is outdated and needs to be updated.
* update: translation portuguese brazilian
* Refactor:Improve Readability and Cleanliness of AzureAISearch Class
* Update AzureAiSearch.js
* Update AzureAISearch.js
* Update heroku.md
Copying the config/install.js expected by RUN npm ci
Heroku CLI would not take the push without this and errored out consistently due to the expected file being missing.
* Adding UID, GID to prevent permission problems when running docker compose
as user and not as root.
* Update docker_install.md
Add comment on pre-creating volume mount directories.
---------
Co-authored-by: Erich Focht <efocht@gmail.com>
Co-authored-by: Erich Focht <efocht@users.noreply.github.com>
* refactor(Chains/llms): allow passing callbacks
* refactor(BaseClient): accurately count completion tokens as generation only
* refactor(OpenAIClient): remove unused getTokenCountForResponse, pass streaming var and callbacks in initializeLLM
* wip: summary prompt tokens
* refactor(summarizeMessages): new cut-off strategy that generates a better summary by adding context from beginning, truncating the middle, and providing the end
wip: draft out relevant providers and variables for token tracing
* refactor(createLLM): make streaming prop false by default
* chore: remove use of getTokenCountForResponse
* refactor(agents): use BufferMemory as ConversationSummaryBufferMemory token usage not easy to trace
* chore: remove passing of streaming prop, also console log useful vars for tracing
* feat: formatFromLangChain helper function to count tokens for ChatModelStart
* refactor(initializeLLM): add role for LLM tracing
* chore(formatFromLangChain): update JSDoc
* feat(formatMessages): formats langChain messages into OpenAI payload format
* chore: install openai-chat-tokens
* refactor(formatMessage): optimize conditional langChain logic
fix(formatFromLangChain): fix destructuring
* feat: accurate prompt tokens for ChatModelStart before generation
* refactor(handleChatModelStart): move to callbacks dir, use factory function
* refactor(initializeLLM): rename 'role' to 'context'
* feat(Balance/Transaction): new schema/models for tracking token spend
refactor(Key): factor out model export to separate file
* refactor(initializeClient): add req,res objects to client options
* feat: add-balance script to add to an existing users' token balance
refactor(Transaction): use multiplier map/function, return balance update
* refactor(Tx): update enum for tokenType, return 1 for multiplier if no map match
* refactor(Tx): add fair fallback value multiplier incase the config result is undefined
* refactor(Balance): rename 'tokens' to 'tokenCredits'
* feat: balance check, add tx.js for new tx-related methods and tests
* chore(summaryPrompts): update prompt token count
* refactor(callbacks): pass req, res
wip: check balance
* refactor(Tx): make convoId a String type, fix(calculateTokenValue)
* refactor(BaseClient): add conversationId as client prop when assigned
* feat(RunManager): track LLM runs with manager, track token spend from LLM,
refactor(OpenAIClient): use RunManager to create callbacks, pass user prop to langchain api calls
* feat(spendTokens): helper to spend prompt/completion tokens
* feat(checkBalance): add helper to check, log, deny request if balance doesn't have enough funds
refactor(Balance): static check method to return object instead of boolean now
wip(OpenAIClient): implement use of checkBalance
* refactor(initializeLLM): add token buffer to assure summary isn't generated when subsequent payload is too large
refactor(OpenAIClient): add checkBalance
refactor(createStartHandler): add checkBalance
* chore: remove prompt and completion token logging from route handler
* chore(spendTokens): add JSDoc
* feat(logTokenCost): record transactions for basic api calls
* chore(ask/edit): invoke getResponseSender only once per API call
* refactor(ask/edit): pass promptTokens to getIds and include in abort data
* refactor(getIds -> getReqData): rename function
* refactor(Tx): increase value if incomplete message
* feat: record tokenUsage when message is aborted
* refactor: subtract tokens when payload includes function_call
* refactor: add namespace for token_balance
* fix(spendTokens): only execute if corresponding token type amounts are defined
* refactor(checkBalance): throws Error if not enough token credits
* refactor(runTitleChain): pass and use signal, spread object props in create helpers, and use 'call' instead of 'run'
* fix(abortMiddleware): circular dependency, and default to empty string for completionTokens
* fix: properly cancel title requests when there isn't enough tokens to generate
* feat(predictNewSummary): custom chain for summaries to allow signal passing
refactor(summaryBuffer): use new custom chain
* feat(RunManager): add getRunByConversationId method, refactor: remove run and throw llm error on handleLLMError
* refactor(createStartHandler): if summary, add error details to runs
* fix(OpenAIClient): support aborting from summarization & showing error to user
refactor(summarizeMessages): remove unnecessary operations counting summaryPromptTokens and note for alternative, pass signal to summaryBuffer
* refactor(logTokenCost -> recordTokenUsage): rename
* refactor(checkBalance): include promptTokens in errorMessage
* refactor(checkBalance/spendTokens): move to models dir
* fix(createLanguageChain): correctly pass config
* refactor(initializeLLM/title): add tokenBuffer of 150 for balance check
* refactor(openAPIPlugin): pass signal and memory, filter functions by the one being called
* refactor(createStartHandler): add error to run if context is plugins as well
* refactor(RunManager/handleLLMError): throw error immediately if plugins, don't remove run
* refactor(PluginsClient): pass memory and signal to tools, cleanup error handling logic
* chore: use absolute equality for addTitle condition
* refactor(checkBalance): move checkBalance to execute after userMessage and tokenCounts are saved, also make conditional
* style: icon changes to match official
* fix(BaseClient): getTokenCountForResponse -> getTokenCount
* fix(formatLangChainMessages): add kwargs as fallback prop from lc_kwargs, update JSDoc
* refactor(Tx.create): does not update balance if CHECK_BALANCE is not enabled
* fix(e2e/cleanUp): cleanup new collections, import all model methods from index
* fix(config/add-balance): add uncaughtException listener
* fix: circular dependency
* refactor(initializeLLM/checkBalance): append new generations to errorMessage if cost exceeds balance
* fix(handleResponseMessage): only record token usage in this method if not error and completion is not skipped
* fix(createStartHandler): correct condition for generations
* chore: bump postcss due to moderate severity vulnerability
* chore: bump zod due to low severity vulnerability
* chore: bump openai & data-provider version
* feat(types): OpenAI Message types
* chore: update bun lockfile
* refactor(CodeBlock): add error block formatting
* refactor(utils/Plugin): factor out formatJSON and cn to separate files (json.ts and cn.ts), add extractJSON
* chore(logViolation): delete user_id after error is logged
* refactor(getMessageError -> Error): change to React.FC, add token_balance handling, use extractJSON to determine JSON instead of regex
* fix(DALL-E): use latest openai SDK
* chore: reorganize imports, fix type issue
* feat(server): add balance route
* fix(api/models): add auth
* feat(data-provider): /api/balance query
* feat: show balance if checking is enabled, refetch on final message or error
* chore: update docs, .env.example with token_usage info, add balance script command
* fix(Balance): fallback to empty obj for balance query
* style: slight adjustment of balance element
* docs(token_usage): add PR notes
* chore(ChatGPTClient.js): add support for OpenRouter API
chore(OpenAIClient.js): add support for OpenRouter API
* chore: comment out token debugging
* chore: add back streamResult assignment
* chore: remove double condition/assignment from merging
* refactor(routes/endpoints): -> controller/services logic
* feat: add openrouter model fetching
* chore: remove unused endpointsConfig in cleanupPreset function
* refactor: separate models concern from endpointsConfig
* refactor(data-provider): add TModels type and make TEndpointsConfig adaptible to new endpoint keys
* refactor: complete models endpoint service in data-provider
* refactor: onMutate for refreshToken and login, invalidate models query
* feat: complete models endpoint logic for frontend
* chore: remove requireJwtAuth from /api/endpoints and /api/models as not implemented yet
* fix: endpoint will not be overwritten and instead use active value
* feat: openrouter support for plugins
* chore(EndpointOptionsDialog): remove unused recoil value
* refactor(schemas/parseConvo): add handling of secondaryModels to use first of defined secondary models, which includes last selected one as first, or default to the convo's secondary model value
* refactor: remove hooks from store and move to hooks
refactor(switchToConversation): make switchToConversation use latest recoil state, which is necessary to get the most up-to-date models list, replace wrapper function
refactor(getDefaultConversation): factor out logic into 3 pieces to reduce complexity.
* fix: backend tests
* feat: optimistic update by calling newConvo when models are fetched
* feat: openrouter support for titling convos
* feat: cache models fetch
* chore: add missing dep to AuthContext useEffect
* chore: fix useTimeout types
* chore: delete old getDefaultConvo file
* chore: remove newConvo logic from Root, remove console log from api models caching
* chore: ensure bun is used for building in b:client script
* fix: default endpoint will not default to null on a completely fresh login (no localStorage/cookies)
* chore: add openrouter docs to free_ai_apis.md and .env.example
* chore: remove openrouter console logs
* feat: add debugging env variable for Plugins
