2023-03-06 10:15:07 -05:00
|
|
|
{
|
2023-06-12 16:50:33 +10:00
|
|
|
"name": "@librechat/backend",
|
2026-03-20 18:01:00 -04:00
|
|
|
"version": "v0.8.4",
|
2023-03-06 10:15:07 -05:00
|
|
|
"description": "",
|
|
|
|
|
"scripts": {
|
feat: Plugins endpoint - Reverse Engineering of official Plugins features (#197)
* components for plugins in progress
* WIP: add langchain client implementation for tools/plugins
feat(langchain): add loadHistory function for loading chat history from database
feat(langchain): add saveMessageToDatabase function for saving chat messages to database
* chore(Memory.js): remove Memory.js file from the project directory.
* WIP: adding plugin functionality
——————————————————
fix(eslintrc.js): change arrow-parens rule to always require parentheses
refactor(agent.js): reorganize imports and add new imports
feat(agent.js): add support for saving and loading chat history
feat(agent.js): add support for saving messages to database
feat(agent.js): add ChatAgent class with initialize and sendMessage methods
fix(langchain): use getConvo and saveMessage functions from models.js instead of Conversation and Message models
feat(langchain): add user parameter to loadHistory and saveMessageToDatabase functions
chore(package.json): update langchain package version to 0.0.59 and add langchain script to run test2.js file
——————————————————
* WIP: testing agent initialization
* WIP: testing various agent methods
feat(agent.js): add CustomChatAgent class and initializeAgentExecutorWithOptions method
feat(customChatAgent.js): add CustomPromptTemplate and CustomOutputParser classes
refactor(langchain): uncomment code for input2 and options
feat(langchain): add input1 to read comments on a youtube video
docs(langchain): remove commented code and add whitespace to package.json
* WIP: feat: plugin endpoint, backend class working
* feat(agent.js): add support for Zapier NLA API key
feat(agent.js): add ZapierToolKit to tools if zapierApiKey is provided
feat(customAgent.js): change prompt prefix and suffix to reflect new task-based prompt
feat(test4.js): add test for new task-based prompt
* style(langchain): improve readability and add comments to code
feat(langchain): update prompt message for custom agent
fix(langchain): update message format in test4.js
* style(customAgent.js): remove unnecessary capitalization and rephrase some sentences
test(langchain): add test2 and test3 scripts to package.json
* chore(customAgent.js): fix typo in comment, change "an" to "identical"
* WIP: gpt-4 testing
* feat(langchain): add AIPluginTool and HumanTool classes
fix(langchain): remove zapierApiKey option from ChatAgent constructor
refactor(langchain): update langchain package to v0.0.64
misc(langchain): update test2, test3, and test4 scripts to use --inspect flag
* feat(langchain): add GoogleSearchAPI tool for searching the web using Google Custom Search API
* feat(askGPTPlugins.js): add support for progress callback in ask function
fix(agent.js): pass progress callback to sendApiMessage function
* refactor(agent.js): load tools from options and initialize them in constructor
feat(agent.js): add support for environment variable SERPAPI_API_KEY
feat(agent.js): add support for environment variable ZAPIER_NLA_API_KEY
docs(agent.js): remove commented out code and add comments to clarify code
* chore(langchain): remove unused files loadHistory.js and saveMessage.js
* feat(validateTools.js): add function to validate API keys for supported tools
* feat(langchain): update langchain package to version 0.0.66
feat(langchain): add support for GPT-4 model
fix(server/index.js): fix uncaughtException handler to ignore 'fetch failed' errors
* refactor(agent.js): remove FORMAT_INSTRUCTIONS and replace with a more concise message
refactor(agent.js): remove unused variable 'errorMessage'
refactor(agent.js): change 'result' variable initialization to an empty object instead of null
refactor(agent.js): change error message when response generation fails
refactor(agent.js): change output message when response generation fails
refactor(agent.js): change output message when response generation succeeds
* chore(langchain): comment out unused model in ChatAgent constructor
feat(langchain): add test5 script to package.json for running test5.js script
* refactor(agent.js): change response to answer and update message
refactor(test3.js, test5.js): remove commented out code and add comments
The changes in agent.js are to improve the message that is returned to the user. The word "response" has been changed to "answer" to better reflect the output of the chatbot. The message has also been updated to provide clearer instructions to the user.
The changes in test3.js and test5.js are to remove commented out code and add comments to improve readability.
* docs: update links to LOCAL_INSTALL.md and defaultSystemMessage.md
fix: fix typo in BingAI/Settings.jsx
feat: add Dockerfile for app containerization
docs(google_search.md): add guide for setting up Google Custom Search API key and ID
* docs: update link to system message guidelines in Bing AI Settings component
docs: update link to system message guidelines in GOOGLE_SEARCH.md
feat: add JAILBREAK_INFO.md guide for Bing AI jailbreak mode system message guidelines
* style(api): remove unnecessary quotes and empty values from .env.example
style(agent.js): refactor getActions method to accept an input parameter
feat(agent.js): add handleChainEnd method to CustomChatAgent class
style(customAgent.js): add a new line to the end of the file
style(test5.js): comment out unused variable and update input1 variable
style(googleSearch.js): change tool name to kebab-case
* chore(langchain): comment out handleChainEnd method in agent.js
feat(langchain): add browser tool to ChatAgent in test2.js
feat(langchain): add modelOptions to ChatAgent in test2.js
feat(langchain): change question in input1 and request article review summary in test5.js
* fix(askGPTPlugins.js): fix syntax error by removing extra comma in parentMessageId field
feat(askGPTPlugins.js): add default value of null to parentMessageId parameter in ask function
* fix(askGPTPlugins.js): change endpoint string from 'GPTPlugins' to 'gptPlugins'
feat(endpoints.js): add support for gptPlugins endpoint
feat(PresetItem.jsx): add support for gptPlugins endpoint
feat(HoverButtons.jsx): add support for gptPlugins endpoint
feat(createPayload.ts): add support for gptPlugins endpoint
feat(types.ts): add gptPlugins endpoint to EModelEndpoint enum
feat(endpoints.js): add gptPlugins endpoint to availableEndpoints selector
feat(cleanupPreset.js): add support for gptPlugins endpoint
feat(getDefaultConversation.js): add support for gptPlugins endpoint
feat(getIcon.jsx): add support for gptPlugins endpoint
feat(handleSubmit.js): add support for gptPlugins endpoint
* refactor(agent.js): remove debug option from options object
refactor(agent.js): change tool name from 'google-search' to 'google'
refactor(agent.js): update description for 'google' tool
feat(agent.js): add support for citing sources when using web links in response message
fix(agent.js): update error message to not mention error to user
feat(agent.js): add unique message ids for user message and response message
feat(agent.js): limit number of search results to 5 in 'google' tool
refactor(validateTools.js): add console log to show valid tools
* feat(askGPTPlugins.js): add support for GPT-3.5-turbo model and validate model option
refactor(askGPTPlugins.js): remove unused imports and variables
refactor(askGPTPlugins.js): remove commented code
refactor(askGPTPlugins.js): remove unused parameters in ask function
feat(ask/index.js): add askGPTPlugins route to router
* feat(NewConversationMenu): add alpha tag to gptPlugins endpoint and rename it to Plugins
* refactor(askGPTPlugins.js): remove commented code and unused imports
feat(askGPTPlugins.js): add support for debug option in endpointOption
feat(askGPTPlugins.js): add support for chatGptLabel, promptPrefix, temperature, top_p, presence_penalty, and frequency_penalty in endpointOption
feat(askGPTPlugins.js): add support for sending plugin and pluginend events
feat(askGPTPlugins.js): add onAgentAction and onChainEnd callbacks to ChatAgent.sendMessage
refactor(titleConvo.js): comment out unused imports
refactor(validateTools.js): comment out console.log statement
refactor(agent.js): change saveMessage to include unfinished property
feat(agent.js): add endpoint property to saveConvo call in saveMessageToDatabase
feat(askGPTPlugins.js): add validateTools import and use it to validate endpointOption.tools before passing to ChatAgent constructor
feat(askGPTPlugins.js
* refactor(MessageHeader.jsx): extract plugins section into a separate variable and add support for gptPlugins endpoint
fix(MessageHeader.jsx): disable clicking on non-clickable endpoints
* components for plugins in progress
* feat(Plugin.jsx): add plugin prop to Plugin component and display plugin name
feat(Plugin.jsx): add loading state and display loading spinner
feat(Plugin.jsx): add Disclosure component to Plugin component
feat(Plugin.jsx): add Disclosure.Panel to Plugin component to display team pricing information
feat(Spinner.jsx): add classProp prop to Spinner component to allow for custom styling
feat(Landing.jsx): add Plugin component to Landing page for testing
testing gpt plugins
feat(plugins): Milestone commit
- Add formatAction function to format plugin actions.
- Add prefix.js file to store the prefix message for ChatAgent.
- Update ask function to include plugin object to store plugin data.
- Update onAgentAction and onChainEnd functions to format plugin data and send intermediate messages.
- Update response object to include plugin data.
The `handlers.js` file now includes a `formatAction` function that formats the action object for display in the UI. The `createOnProgress` function now returns a `sendIntermediateMessage` function that sends intermediate messages to the client.
feat (client): add support for plugins in messages
This commit adds support for plugins in messages. It includes changes to the `handlers.js`, `index.jsx`, `CodeBlock.jsx`, `Message.jsx`, `MessageHeader.jsx`, and `Plugin.jsx` files.
The `index.jsx` file now includes a `plugin` property in the `messageHandler` function.
The `CodeBlock.jsx` file now includes a `plugin` property that determines the language of the code block.
The `Message.jsx` file now includes a `Plugin` component that displays the plugin used in the message.
The `MessageHeader.jsx` file now includes a `Plugins` component that displays the enabled plugins.
feat(langchain): add OpenAICreateImage tool for generating images based on user prompts
fix(langchain): update validateTools to include create-image tool
fix(langchain): save plugin data to messageSchema
fix(server/routes/askGPTPlugins.js): save userMessage and response to messageSchema
feat(langchain): add SelfReflectionTool
Add a new tool to the LangChain agent, SelfReflectionTool, which enhances the agent's self-awareness by reflecting on its thoughts before taking action. The tool provides a space for the agent to explore and organize its ideas in response to the user's message.
Also, update the prefix message to reflect the changes in the agent's behavior and the way it should engage with the user. The prefix message now emphasizes the use of tools when necessary, and relying on the agent's knowledge for creative requests. It also provides clear instructions on how to use the 'Action' input and how to carry out tasks in the sequence written by the human.
Finally, update the OpenAICreateImage tool to return the image URL in markdown format. The tool replaces newlines and spaces in the input text with hyphens to create a valid markdown link.
Milestone commit: better error handling with custom output parser, dir and file re-org
style(langchain): fix formatting and add comments to prefix.js
fix(langchain): remove commented out code in test6.js
feat(langchain): reduce maxAttempts from 3 to 2 in CustomChatAgent's buildPromptPrefix method
feat(langchain): add null check for result.output in CustomChatAgent's buildPromptPrefix method
style(langchain): improve consistency and readability of code
This commit improves the consistency and readability of the code in the langchain directory. Specifically, it:
- Changes the case of the "Thought" output in the CustomChatAgent class to match the "Thought" output in the SelfReflectionTool class.
- Adds a currentDateString property to the CustomChatAgent class to avoid repeating the same code in multiple places.
- Updates the prefix in the prefix.js file to match the current objectives of the ChatGPT model.
- Changes the description of the OpenAICreateImage tool to request a description of the image to be generated.
- Updates the tools used by the ChatAgent in the askGPTPlugins.js file to include the Google and Browser tools instead of the Calculator and Create-Image tools.
feat: add wolfram, improve image creation, rename to dall-e
* refactor(langchain): update language and formatting in various files
- Update tool-based instructions to use proper Markdown syntax for image URLs
- Adjust temperature for modelOptions in CustomChatAgent class
- Comment out console.debug statement in CustomChatAgent class
- Update prefix in initializeCustomAgent function to use proper line breaks
- Update prefix in instructions.js to use proper line breaks and change "user" to "human"
- Update input in test6.js to use Ezra Pound instead of Hemingway
- Update return statement in OpenAICreateImage class to use "generated-image" as alt-text
- Update description in SelfReflectionTool class to provide clearer instructions
- Update tools in ask function in askGPTPlugins.js to use only the DALL-E tool and enable debug mode
feat(ask): add support for DALL-E tool in formatAction function
feat(ask): add support for self-reflection tool in formatAction function
feat(Plugin.jsx): add support for self-reflection tool in Plugin component
fix(Plugin.jsx): fix Plugin component to not display 'None' when latest is not available
* docs(openaiCreateImage.js): update tool description to clarify usage
* feat(agent.js): add message parameter to initialize function
feat(agent.js): pass message parameter to SelfReflectionTool constructor
feat(customAgent.js): add longestToolName variable to CustomOutputParser
feat(openaiCreateImage.js): replace new lines with spaces in prompt parameter
feat(selfReflection.js): add message parameter to SelfReflectionTool constructor
feat(selfReflection.js): add placeholder response to selfReflect function
* feat: frontend plugin selection
* fix: agent updates, available tools via endpoint config
* fix: improve frontend plugin selection
* feat: further customize agent and bypass executor when no tools are provided
* fix: key issue in multiselect and allow setting changes during convo in plugins endpoint
* fix: convo will save modelOptions, fix persistent errors with agent
* fix: add looser final answer parsing and edit action formatting
* fix: handle edge case where stop token is not hit and causes long parsing error
* feat: trying new prompt for image creation
* fix: improvements based on gpt-3.5
* feat: allow setting model options throughout plugin conversation
* fix: agent adjustments
* improve final reply for gpt-4, gpt-3.5 needs a more stable approach
* fix: better context output for gpt-3.5
* fix: added clarification for better context output for gpt-3.5
* feat(PluginsOptions): add advanced mode to show/hide options
style(PluginsOptions): add styles for advanced mode and show/hide options
* minor changes to styling
* refactor(langchain): add support for custom GPT-4 agent
This commit adds support for a custom GPT-4 agent in the langchain
module. The `CustomGpt4Agent` class extends the `ZeroShotAgent` class
and includes a new `createPrompt` method that generates a prompt
template for the agent. The `initializeCustomAgent` function has been
updated to use the `CustomGpt4Agent` class when the model is not GPT-3.
The `instructions.js` file has also been updated to include new
instructions for the GPT-4 agent. The `formatInstructions` method has
been removed and replaced with `gpt4Instructions` and `prefix2` and
`suffix2` have been added to include the new instructions.
feat(langchain): add custom output parser for langchain agents
This commit adds a custom output parser for langchain agents. The new parser is called CustomOutputParser and it extends ZeroShotAgentOutputParser. It takes a fields object as a parameter and sets the tools and longestToolName properties. It also sets the finishToolNameRegex property to match the final answer. The parse method of the CustomOutputParser class takes a text parameter and returns an object with returnValues, log, and toolInput properties.
This commit also adds a Gpt4OutputParser class that extends ZeroShotAgentOutputParser. It takes a fields object as a parameter and sets the tools and longestToolName properties. It also sets the finishToolNameRegex property to match the final answer. The parse method of the Gpt4OutputParser class takes a text parameter and returns an object with returnValues, log, and toolInput properties.
feat(langchain): add isGpt3 parameter to
* Stable Diffusion Plugin (#204)
* Added stable diffusion plugin
* Added example prompt
* Fixed naming
* Removed brackets in the prompt
* fix: improved agent for gpt-3.5
* fix: outparser, gpt3 instructions, and wolfram error handling
* chore: update langchain to 0.0.71
* fix: long parsing action input fix
* fix: make plugin select close on clicking label/button
* fix: make plugin select close on clicking label/button
* fix: wolfram input formatting and gpt-3 payload without plugins
* chore(api): update axios package version to 1.3.4
feat(api): add requireJwtAuth middleware to askGPTPlugins endpoint
fix(api): replace session user with user id in askGPTPlugins endpoint
docs(LOCAL_INSTALL.md): update guide for local installation and testing
This commit updates the guide for local installation and testing of the
ChatGPT-Clone app. It includes instructions for locally running the app,
updating the app version, and running tests. It also includes a new
option for running the app using Docker. The commit also fixes some
typos and formatting issues.
* add reverseProxy to plugins client
* chore(Dockerfile-app): add Dockerfile for building and running the app in a container
docs: remove outdated guides on Google search and Bing jailbreak mode
docs(LOCAL_INSTALL.md): remove outdated Windows installation instructions and update MeiliSearch configuration file
* fix: handle n/a parsing error better, reduce token waste if no agentic behavior is needed
* style: fix formatting and add parentheses around arrow function parameter
style: change hover background color to white and dark hover background color to gray-700
* chore: re-organize agent dir and files
* feat(ChatAgent.js): add support for PlanAndExecuteAgentExecutor
feat(PlanAndExecuteAgentExecutor.js): add PlanAndExecuteAgentExecutor class
feat(planExecutor.js): add demo for PlanAndExecuteAgentExecutor
* feat: add azure support to plugins
* refactor(utils): add basePath endpoint for genAzureEndpoint
feat(api): add support for Azure OpenAI API in various modules and tools
* feat: add plugin api for fetching available tools
* feat: add data service for getting available plugins
* feat: first iteration plugin store UI
* refactor: rename files to follow proper naming convention
* feat: Plugin store UI components
* feat: create separate user routes, service, controller, and add plugins to user model
* feat: create data service for adding and removing plugins per user
* feat: UI for adding and removing plugins, displaying plugins in dropdown based on what user has installed
* fix: merge conflicts from main
* fix: fix plugin items titles
* fix: tool.value -> tool.pluginKey
* fix: testing returnDirect for self-reflection
* fix: add browser tool to manifest
* refactor(outputParser.js): remove commented out code
feat(outputParser.js): add support for thought input when there is no action input
* handling 'use tool' edge case
* merge main to langchain
* fix(User.js, auth.service.js, localStrategy.js): change deprecated Joi.validate() to schema.validate() method (#322)
* fix(auth.service.js): fixes deprecated error callback in mongoose save method (#323)
* chore: run formatting script with new rules
* refactor: add requiresAuth to manifest, fix uninstall button
* version with plugin auth as dialog modal
* feat: Complete frontend for plugin auth
* frontend styling updates
* feat: api for plugin auth
* feat: Add tooltip with field description to plugin auth form
* fix: issue with plugin that has no auth
* feat(tools): add support for user-specific API keys
This commit adds support for user-specific API keys for the following tools:
- Google Search API
- Web Browser
- SerpAPI
- Zapier
- DALL-E
- Wolfram Alpha API
It also adds support for OpenAI API key for the Web Browser tool.
The `validateTools` function now takes a `user` parameter and checks for user-specific API keys before falling back to environment variables.
The `loadTools` function now takes a `user` parameter and initializes the tools with user-specific API keys if available.
The `manifest.json` file has been updated to include the new `authConfig` fields for the tools that support user-specific API keys.
The `askGPTPlugins.js` file has been updated to use the `validateTools` function with the `user` parameter.
refactor(ChatAgent.js): add user parameter to initialize function and pass it to loadTools function
refactor(tools/index.js): set default value for tools parameter in validateTools function
refactor(askGPTPlugins.js): remove duplicate user variable declaration and use the one from req object
* refactor(ChatAgent.js): await validTool() before pushing to this.tools array
refactor(tools/index.js): use Map instead of Set to store valid tools
refactor(tools/index.js): filter availableTools to only validate tools passed in
refactor(PluginController.js): filter out duplicate plugins by pluginKey
refactor(crypto.js): use environment variables for encryption key and initialization vector
feat(PluginService.js): add null check for pluginAuth in getUserPluginAuthValue()
* feat(api): add credentials key and IV to .env.example for securely storing credentials
* Adds testing for handling tools, introducing a test env to the backend
Fixes bugs & optimizes code as revealed through testing, including:
- wolfram.js: fixes bug where wolfram was not handling authentication
- ChatAgent.js: ChatAgent modified to reflect 'handleTools' changes
- handleTools.js: Moves logic out of index file
- handleTools.js: loadTools: returns only requested tools
- handleTools.js: validTools: correctly returns tools based on authentication
* test(index.test.js): add test to validate a tool from an environment variable
* test(tools): add test for initializing an authenticated tool through Environment Variables
* refactor(ChatAgent.js): remove commented out code and unused imports
* refactor(ChatAgent.js): move instructions to a separate file and import them
fix(ChatAgent.js): replace hardcoded instructions with imported ones
* refactor(ChatAgent.js): change import path for TextStream
refactor(stream.js): remove unused TextStream class
* chore(.gitignore): add .env.test to gitignore
refactor(ChatAgent.js): rename CustomChatAgent to ChatAgent
test(ChatAgent.test.js): add tests for ChatAgent class
refactor(outputParser.js): remove OldOutputParser class
refactor(outputParser.js): rename CustomOutputParser to OutputParser
docs(.env.test.example): add comment explaining how to use OPENAI_API_KEY
refactor(jestSetup.js): use dotenv to load environment variables from .env.test file
* Various optimizations and config, add tests for PluginStoreDialog
* test(ChatAgent.test.js): add test to check if chat history is returned correctly
* test: unit tests for plugin store
* test: add frontend-test script to root package.json
* feat(ChatAgent.js, askGPTPlugins.js): add support for aborting chat requests (in progress)
* test: add more client tests
* feat(ChatAgent): allow plugin requests to be cancelled
* feat(ChatAgent): allow message regeneration
* feat(ChatAgent): remember last selected tools
* Remove plugins we don't yet have from manifest.json
* fix(ChatAgent.js): increase maxAttempts from 1 to 2
fix(ChatAgent.js): change error message to 'Cancelled.' if message was aborted mid-generation
fix(openaiCreateImage.js): replace unwanted characters in input string
fix(handlers.js): compare action.tool in lowercase to 'self-reflection'
* fix(ChatAgent): Fix up plugin I/O formatting for n/a actions
* refactor(Plugin.jsx): remove unused import statement
feat(Plugin.jsx): add Plugin component with svg paths and styles
* refactor: simplify credential encryption/decryption by using a single key and IV for all environments. Update crypto.js and .env.example files accordingly.
* fix(ChatAgent.js): reduce maxAttempts from 2 to 1
feat(ChatAgent.js): add model information to responseMessage object
feat(Message.js): add model field to messageSchema
feat(Message.js): add model field to message object
feat(Message.jsx): pass model information to getIcon function
feat(getIcon.jsx): add Plugin component and handle plugin messages differently
* feat(askGPTPlugins.js): add model property to the ask function response object
feat(EndpointItem.jsx): add message property to the EndpointItem component
feat(MessageHeader.jsx): add Plugin icon to the plugins section
feat(MessageHeader.jsx): change alpha to beta in the plugins section
feat(svg): add Plugin, GPTIcon, and BingIcon components to the svg folder
refactor(EndpointItems.jsx): remove unused import statement
* refactor(googleSearch.js, wolfram.js): change error handling to return a message instead of throwing an error
* refactor(CustomAgent): remove commented code and change return object to include returnValues property
* feat(CustomAgent.js): add currentDateString to createPrompt method options
deps(api/package.json): update langchain to v0.0.81
* fix: do not show pagination if the maxPage is 1
* Add Zapier back to manifest (accidentally removed)
* chore(api): update langchain dependency to version 0.0.84
* feat(DALL-E.js): add DALL-E tool for generating images using OpenAI's DALL-E API
refactor(handleTools.js): update import for DALL-E tool
refactor(index.test.js): update import for DALL-E tool
refactor(stablediffusion.js): add check for image directory existence before saving image
* refactor(CustomAgent): rename instructions prefix variable to gpt3 and add gpt4 instructions
feat(CustomAgent): add support for gpt-4 model
fix(initializeCustomAgent.js): pass model name to createPrompt method
fix(outputParser.js): set selectedTool to 'self-reflection' when tool parsing fails
* style(langchain/tools): update guidelines for image creation in DALL-E and StableDiffusion
- Update guidelines for image creation in DALL-E and StableDiffusion tools
- Emphasize the importance of "showing" and not "telling" the imagery in crafting input
- Update formatting for the example prompt for generating a realistic portrait photo of a man
- Generate images only once per human query unless explicitly requested by the user
* docs(tools): update tool descriptions for DALL-E and Stable Diffusion
- Update the description for DALL-E tool to indicate that it is exclusively for visual content and provide guidelines for generating images with a focus on visual attributes.
- Update the description for Stable Diffusion tool to indicate that it is exclusively for visual content and provide guidelines for generating images with a focus on visual attributes.
* chore(api): update "@waylaidwanderer/chatgpt-api" dependency to version "^1.36.3"
* refactor(ChatAgent.js): use environment variable for reverse proxy url
refactor(ChatAgent.js): use environment variable for openai base path
refactor(instructions.js): update gpt3 and gpt3-v2 instructions
refactor(outputParser.js): update finishToolNameRegex in CustomOutputParser class
* refactor(DALL-E.js): change apiKey and azureKey fields to uppercase
refactor(googleSearch.js): change cx and apiKey fields to uppercase
feat(manifest.json): add authConfig field for Stable Diffusion WebUI API URL
refactor(stablediffusion.js): add url field to constructor and change getServerURL() to this.url
refactor(wolfram.js): change apiKey field to uppercase WOLFRAM_APP_ID
* refactor(handleTools.js): simplify tool loading and add support for custom tool constructors and options
* refactor(handleTools.js): remove commented out code and unused imports
* refactor(handleTools.js, index.js): change file name from wolfram.js to Wolfram.js and selfReflection.js to SelfReflection.js to follow PascalCase convention
* refactor(outputParser.js, askGPTPlugins.js): improve code readability and remove unnecessary comments
* feat(GoogleSearch.js): add GoogleSearchAPI tool to allow agents to use the Google Custom Search API
feat(SelfReflection.js): add SelfReflectionTool to allow agents to reflect on their thoughts and actions
feat(StableDiffusion.js): add StableDiffusionAPI tool to allow agents to generate images using stable diffusion webui's api
feat(Wolfram.js): add WolframAlphaAPI tool for computation, math, curated knowledge & real-time data through WolframAlpha.
* testing openai specs
* doc: fix link in .env.example
* package-update
* fix(MultiSelectDropDown.jsx): handle null or undefined values in availableValues array
* refactor(DALL-E.js, StableDiffusion.js): remove 'dist/' from image path
feat(docker-compose.yml): add comments for reverse proxy configuration
* chore(.gitignore): ignore client/public/images/
fix(DALL-E.js, StableDiffusion.js): change image path from dist/ to public/
feat(index.js): add support for serving static files from client/public/ directory
* fix: remove selected tool when uninstalled
* plugin options in progress
* fix: fix issue with uninstalling a plugin that is in use and typescript errors
* feat(gptPlugins): add Preset support for GPT Plugins endpoint
feat(ChatAgent.js): add support for agentOptions object
feat(convoSchema.js): add agentOptions field to conversation schema
feat(defaults.js): add agentOptions object to defaults
feat(presetSchema.js): add agentOptions field to preset schema
feat(askGPTPlugins.js): add support for agentOptions object in request body
feat(EditPresetDialog.jsx): add support for showing/hiding GPT Plugins agent settings
feat(EditPresetDialog.jsx): add support for setting GPT Plugins agent options
fix(EndpointOptionsDialog.jsx): change endpoint name from 'gptPlugins' to 'Plugins'
feat(AgentSettings.jsx): add AgentSettings component for GPT plugins configuration
feat(client): add GPT Plugins settings component and endpoint to Settings component
fix(client): remove unused imports in GoogleOptions component
feat(PluginsOptions): add support for agent settings and refactor code
feat(PluginsOptions): add GPTIcon to show/hide agent settings button
feat(index.ts): export SVG components
feat(GPTIcon.jsx): add className prop to GPTIcon component
feat(GPTIcon.jsx): import cn function from utils
feat(BingIcon.tsx): export BingIcon component
feat(index.ts): export BingIcon component
feat(index.ts): export MessagesSquared component
refactor(cleanupPreset.js): add default values for agentOptions in gptPlugins endpoint
feat(getDefaultConversation.js, handleSubmit.js): add agentOptions object to conversation object for GPT plugins endpoint. Update default temperature value to 0.8. Add chatGptLabel and promptPrefix properties to conversation object.
* fix: set default convo back to null
* refactor(ChatAgent.js, askGPTPlugins.js, AgentSettings.jsx): change variable names for better readability and remove redundant code
* test: add RecoilRoot to layout-test-utils
* refactor(askGPTPlugins.js): remove redundant code and use endpointOption directly
feat(askGPTPlugins.js): add validation for tools in endpointOption before using it
* chore(ChatAgent.js, Settings.jsx): add agentOptions to saveConvo function and adjust Settings component height
The ChatAgent.js file was modified to include the agentOptions object in the saveConvo function. The Settings.jsx file was modified to adjust the height of the component to ensure that all content is visible.
* refactor(ChatAgent.js): extract reverseProxyUrl option to a class property and add support for it
feat(ChatAgent.js): add support for completionMode option in sendApiMessage method
feat(ChatAgent.js): add support for user-provided promptPrefix in buildPrompt method
* feat(plugins): allow preset change mid conversation
* chore: update OPENAI_KEY to OPENAI_API_KEY in .github/playwright.yml and api/.env.example
refactor(chatgpt-client.js): update OPENAI_KEY to OPENAI_API_KEY
feat(langchain): add demo-aiplugin.js and demo-yaml.js, remove test2.js, test3.js, and test4.js
chore: remove unused test files
fix(titleConvo.js): fix typo in environment variable name
fix(askGPTPlugins.js): fix typo in environment variable name
fix(endpoints.js): fix typo in environment variable name
docs: update installation guide to use OPENAI_API_KEY instead of OPENAI_KEY in .env file
* fix(index.test.js): change import of GoogleSearchAPI to use uppercase G in GoogleSearch
* chore(api): bump langchain version
* feat(PluginController.js): authenticate plugins from environment variables if they are set
feat(PluginStoreDialog.tsx): show plugin auth form only if plugin is not authenticated by env var and require authentication
feat(types.ts): add authenticated field to TPlugin type definition
* docs: update google_search.md and add stable_diffusion.md
* Update stable_diffusion.md
* refactor(Wolfram.js): remove newline characters from query before encoding
docs(wolfram.md): add instructions for setting WOLFRAM_APP_ID in api.env to bypass prompt for AppID in plugin
* refactor(Wolfram.js): replace deprecated replaceAll method with replace method
* Update wolfram.md
* fix(askGPTPlugins): error message will reference correct Parent Message
* refactor(chatgpt-client.js, ChatAgent.js): simplify maxContextTokens calculation and add promptPrefix parameter to buildPrompt method
* docs: initial draft of intro to plugins
* Update introduction.md
* Update introduction.md
* Feature: User/Reg cleanup + Install / Upgrade script for langchain (#427)
* test: login tests
* test: finish login tests
* test: initial tests for registration
* test: registration specs
* feature: Init a app config file
- Simplifies the ENV vars too
- Legacy fallbacks for older builds
* refactor(auth): Refactor log in/out controllers
- Moves both login and logout controllers to their own file
* chore(jwt): Throw warning if secret is default
* feature(frontend): Ability to disable registration
* feature(env): Env in the root + version support
ie .env.prod, .env.dev, .env.test
* feature: Upgrade .env script for users
* chore(config): Refactor and remove legacy env refs
* feature(upgrade): Upgrade script for .env changes
* feature: Install script and upgrade script
* bugfix: Uncomment line to remove old .env file
* chore: rename OPENAI_KEY to OPENAI_API_KEY
* chore: Cleanup config changes/bugs
* bugfix: Fix config and node env issues
* bugfix: Config validation logic
* bugfix: Handle unusual env configs gracefully
* bugfix: Revert route changes and fix registration disable calling
* bugfix: Fix env issues in frontend
* bugfix: Fix login
* bugfix: Fix frontend envs
* bugfix: Fix frontend jest tests
* bugfix: Fix upgrade scripts
* bugfix: Allow install in non-tty envs
* bugfix(windows): Use cross-env to set for windows
* bugfix(env): Handle .env being incorrect to begin with for client domain
* chore(merge-conflict): Update to LibreChat
* chore(merge-conflict): Update to package-lock
---------
Co-authored-by: Daniel D Orlando <dan@danorlando.com>
* chore: comment out unused agent options
* Update langchain plugins docs (#461)
* Update: install docs (LibreChat) (#458)
* Release: rename project from ChatGPT Clone to LibreChat
Release: rename project from ChatGPT Clone to LibreChat
* Release: rename project from ChatGPT Clone to LibreChat
Release: rename project from ChatGPT Clone to LibreChat
* Release: rename project from ChatGPT Clone to LibreChat
Release: rename project from ChatGPT Clone to LibreChat
* Release: rename project from ChatGPT Clone to LibreChat
Release: rename project from ChatGPT Clone to LibreChat
* Update documentation_guidelines.md
* Update introduction.md
add link to readme
* Update stable_diffusion.md
add link back to readme
* Update wolfram.md
add link back to readme
* Update README.md
add Plugins to ToC
* feat(ChatAgent.js): add support for langchainProxy configuration option
Add a new configuration option `langchainProxy` to the ChatAgent class. If the option is set, the `basePath` configuration option of the `ChatOpenAI` instance is set to the base path of `langchainProxy`.
* bugfix(errors): Possible workaround for error flashing (#463)
* Test/user auth system client tests (#462)
* test: login tests
* test: finish login tests
* test: initial tests for registration
* test: registration specs
* chore(api): update langchain dependency to version 0.0.91
* Update introduction.md
* Update introduction.md
* Update introduction.md
* fix: no longer renders html in markdown content
fix: patch XSS vulnerability completely by handling cursor on the frontend without css/html
* fix(Content.jsx): fix cursor logic so it never shows for static messages
* bugfix(langchain): Upgrade script, docker, env and docs (#465)
* bugfix(errors): Remove incorrect manual fix from misunderstanding
* chore(env): Lets not make a .env.prod and use the prod values in the default root .env
- .env.dev will still be created
* chore(upgrade.js): Lets tell the user about .env.dev if we create it
* bugfix(env): Move to full name environments for vite
- .env.prod => .env.production
- .env.dev => .env.development
* chore(env-example): Explain how to get google login working in production
* bugfix(oauth): Minor fix to point isProduction to a correct value
* bugfix: Typo in public
* chore(docs): Update docs to note the changes to .env
* chore(docs): Include note on how to get google auth working in dev and how to disable registration
* bugfix: Fix missing env changes
* bugfix: Fix up docker to work with new env / npm changes
* Update .env.example
Cleanup the env of the palm2 instruction and fix to formating
* chore(docker): Simplify Docker deployments
- Needs work to support dev env/hotreload
* bugfix: Remove volume map for client dir
* chore(env-example): Change instructions to be more user centric
---------
Co-authored-by: Fuegovic <32828263+fuegovic@users.noreply.github.com>
* update: install docs (#466)
* Add files via upload
* Update apis-and-tokens.md
* Update apis-and-tokens.md
* Update docker_install.md
* Update linux_install.md
* Rename apis-and-tokens.md to apis_and_tokens.md
* Update docker_install.md
* Update linux_install.md
* Update mac_install.md
* Update linux_install.md
* Update docker_install.md
* Update windows_install.md
* Update apis_and_tokens.md
* Update mac_install.md
* Update linux_install.md
* Update docker_install.md
* Update README.md
* Update README.md : Breaking Changes
---------
Co-authored-by: Danny Avila <110412045+danny-avila@users.noreply.github.com>
* Update README.md (#468)
add new API/Token docs to Toc
* docs: guide on how to create your own plugin
* Update make_your_own.md
* Update make_your_own.md
* feat(docker): add build args for frontend variables in Dockerfile
feat(docker-compose): add build args for frontend variables in docker-compose.yml
* Update docker_install.md
* Update docker_install.md
* Update docker_install.md
* Update docker_install.md
* docs: update (#469)
* Update: make_your_own.md
* Update README.md
add `make_your_own.md` to ToC
* Update linux_install.md
* Update mac_install.md
* Update windows_install.md
* Update apis_and_tokens.md
* Update docker_install.md
* Update docker_install.md
* Update linux_install.md
* Update mac_install.md
* Update windows_install.md
* Update apis_and_tokens.md
* Update user_auth_system.md
* Update docker_install.md
clean up of repeated information
* Update docker_install.md
* Update docker_install.md
typo
* fix: fix issue with pluginstore next and prev buttons going out of bounds
* fix: add icon for web browser plugin
* docs(GoogleSearch.js): update description of GoogleSearchAPI class to be more descriptive of its functionality
* feat(ask/handlers.js): add cursor to indicate ongoing progress of a long-running task
fix(Content.jsx): handle null content in the message stream by replacing it with an empty string (with a space so a text space is rendered)
* Update README.md
* Update README.md
* fix: plugin option stacking order
* update: web browser icon (#470)
* Delete web-browser.png
* update: web browser icon
* Update readme (#472)
* Update README.md
Discord badge now displays the number of online users
Project description has been updated to reflect current status
Feature section has been updated to reflect current capabilities
Sponsors section is now located just above the contributors section
Roadmap has been removed as it was outdated.
* Delete roadmap.md
Roadmap has been removed to streamline document maintenance.
* Update README.md
* Update README.md
* Delete CHANGELOG.md
* fix: pluginstore in mobile view getting clipped and not scrolling
* docs(linux_install.md): remove duplicate git clone command
* chore(Dockerfile): comment out nginx-client build stage
docs(README.md): update installation instructions and mention docker-compose changes
docs(features/plugins/introduction.md): bold plugin names and add emphasis to notes
* feat: add superscript and subscript support to markdown rendering
refactor: support markdown citations for BingAI
* refactor: support markdown citations for BingAI
---------
Co-authored-by: David Shin <42793498+dncc89@users.noreply.github.com>
Co-authored-by: Daniel D Orlando <dan@danorlando.com>
Co-authored-by: LaraClara <2524209+ClaraLeigh@users.noreply.github.com>
Co-authored-by: Fuegovic <32828263+fuegovic@users.noreply.github.com>
2023-06-10 19:10:03 -04:00
|
|
|
"start": "echo 'please run this from the root directory'",
|
|
|
|
|
"server-dev": "echo 'please run this from the root directory'",
|
test: frontend jest ci/cd & minor fixes post-release (#478)
* improve final reply for gpt-4, gpt-3.5 needs a more stable approach
* fix: better context output for gpt-3.5
* fix: added clarification for better context output for gpt-3.5
* feat(PluginsOptions): add advanced mode to show/hide options
style(PluginsOptions): add styles for advanced mode and show/hide options
* minor changes to styling
* refactor(langchain): add support for custom GPT-4 agent
This commit adds support for a custom GPT-4 agent in the langchain
module. The `CustomGpt4Agent` class extends the `ZeroShotAgent` class
and includes a new `createPrompt` method that generates a prompt
template for the agent. The `initializeCustomAgent` function has been
updated to use the `CustomGpt4Agent` class when the model is not GPT-3.
The `instructions.js` file has also been updated to include new
instructions for the GPT-4 agent. The `formatInstructions` method has
been removed and replaced with `gpt4Instructions` and `prefix2` and
`suffix2` have been added to include the new instructions.
feat(langchain): add custom output parser for langchain agents
This commit adds a custom output parser for langchain agents. The new parser is called CustomOutputParser and it extends ZeroShotAgentOutputParser. It takes a fields object as a parameter and sets the tools and longestToolName properties. It also sets the finishToolNameRegex property to match the final answer. The parse method of the CustomOutputParser class takes a text parameter and returns an object with returnValues, log, and toolInput properties.
This commit also adds a Gpt4OutputParser class that extends ZeroShotAgentOutputParser. It takes a fields object as a parameter and sets the tools and longestToolName properties. It also sets the finishToolNameRegex property to match the final answer. The parse method of the Gpt4OutputParser class takes a text parameter and returns an object with returnValues, log, and toolInput properties.
feat(langchain): add isGpt3 parameter to
* Stable Diffusion Plugin (#204)
* Added stable diffusion plugin
* Added example prompt
* Fixed naming
* Removed brackets in the prompt
* fix: improved agent for gpt-3.5
* fix: outparser, gpt3 instructions, and wolfram error handling
* chore: update langchain to 0.0.71
* fix: long parsing action input fix
* fix: make plugin select close on clicking label/button
* fix: make plugin select close on clicking label/button
* fix: wolfram input formatting and gpt-3 payload without plugins
* chore(api): update axios package version to 1.3.4
feat(api): add requireJwtAuth middleware to askGPTPlugins endpoint
fix(api): replace session user with user id in askGPTPlugins endpoint
docs(LOCAL_INSTALL.md): update guide for local installation and testing
This commit updates the guide for local installation and testing of the
ChatGPT-Clone app. It includes instructions for locally running the app,
updating the app version, and running tests. It also includes a new
option for running the app using Docker. The commit also fixes some
typos and formatting issues.
* add reverseProxy to plugins client
* chore(Dockerfile-app): add Dockerfile for building and running the app in a container
docs: remove outdated guides on Google search and Bing jailbreak mode
docs(LOCAL_INSTALL.md): remove outdated Windows installation instructions and update MeiliSearch configuration file
* fix: handle n/a parsing error better, reduce token waste if no agentic behavior is needed
* style: fix formatting and add parentheses around arrow function parameter
style: change hover background color to white and dark hover background color to gray-700
* chore: re-organize agent dir and files
* feat(ChatAgent.js): add support for PlanAndExecuteAgentExecutor
feat(PlanAndExecuteAgentExecutor.js): add PlanAndExecuteAgentExecutor class
feat(planExecutor.js): add demo for PlanAndExecuteAgentExecutor
* feat: add azure support to plugins
* refactor(utils): add basePath endpoint for genAzureEndpoint
feat(api): add support for Azure OpenAI API in various modules and tools
* feat: add plugin api for fetching available tools
* feat: add data service for getting available plugins
* feat: first iteration plugin store UI
* refactor: rename files to follow proper naming convention
* feat: Plugin store UI components
* feat: create separate user routes, service, controller, and add plugins to user model
* feat: create data service for adding and removing plugins per user
* feat: UI for adding and removing plugins, displaying plugins in dropdown based on what user has installed
* fix: merge conflicts from main
* fix: fix plugin items titles
* fix: tool.value -> tool.pluginKey
* fix: testing returnDirect for self-reflection
* fix: add browser tool to manifest
* refactor(outputParser.js): remove commented out code
feat(outputParser.js): add support for thought input when there is no action input
* handling 'use tool' edge case
* merge main to langchain
* fix(User.js, auth.service.js, localStrategy.js): change deprecated Joi.validate() to schema.validate() method (#322)
* fix(auth.service.js): fixes deprecated error callback in mongoose save method (#323)
* chore: run formatting script with new rules
* refactor: add requiresAuth to manifest, fix uninstall button
* version with plugin auth as dialog modal
* feat: Complete frontend for plugin auth
* frontend styling updates
* feat: api for plugin auth
* feat: Add tooltip with field description to plugin auth form
* fix: issue with plugin that has no auth
* feat(tools): add support for user-specific API keys
This commit adds support for user-specific API keys for the following tools:
- Google Search API
- Web Browser
- SerpAPI
- Zapier
- DALL-E
- Wolfram Alpha API
It also adds support for OpenAI API key for the Web Browser tool.
The `validateTools` function now takes a `user` parameter and checks for user-specific API keys before falling back to environment variables.
The `loadTools` function now takes a `user` parameter and initializes the tools with user-specific API keys if available.
The `manifest.json` file has been updated to include the new `authConfig` fields for the tools that support user-specific API keys.
The `askGPTPlugins.js` file has been updated to use the `validateTools` function with the `user` parameter.
refactor(ChatAgent.js): add user parameter to initialize function and pass it to loadTools function
refactor(tools/index.js): set default value for tools parameter in validateTools function
refactor(askGPTPlugins.js): remove duplicate user variable declaration and use the one from req object
* refactor(ChatAgent.js): await validTool() before pushing to this.tools array
refactor(tools/index.js): use Map instead of Set to store valid tools
refactor(tools/index.js): filter availableTools to only validate tools passed in
refactor(PluginController.js): filter out duplicate plugins by pluginKey
refactor(crypto.js): use environment variables for encryption key and initialization vector
feat(PluginService.js): add null check for pluginAuth in getUserPluginAuthValue()
* feat(api): add credentials key and IV to .env.example for securely storing credentials
* Adds testing for handling tools, introducing a test env to the backend
Fixes bugs & optimizes code as revealed through testing, including:
- wolfram.js: fixes bug where wolfram was not handling authentication
- ChatAgent.js: ChatAgent modified to reflect 'handleTools' changes
- handleTools.js: Moves logic out of index file
- handleTools.js: loadTools: returns only requested tools
- handleTools.js: validTools: correctly returns tools based on authentication
* test(index.test.js): add test to validate a tool from an environment variable
* test(tools): add test for initializing an authenticated tool through Environment Variables
* refactor(ChatAgent.js): remove commented out code and unused imports
* refactor(ChatAgent.js): move instructions to a separate file and import them
fix(ChatAgent.js): replace hardcoded instructions with imported ones
* refactor(ChatAgent.js): change import path for TextStream
refactor(stream.js): remove unused TextStream class
* chore(.gitignore): add .env.test to gitignore
refactor(ChatAgent.js): rename CustomChatAgent to ChatAgent
test(ChatAgent.test.js): add tests for ChatAgent class
refactor(outputParser.js): remove OldOutputParser class
refactor(outputParser.js): rename CustomOutputParser to OutputParser
docs(.env.test.example): add comment explaining how to use OPENAI_API_KEY
refactor(jestSetup.js): use dotenv to load environment variables from .env.test file
* Various optimizations and config, add tests for PluginStoreDialog
* test(ChatAgent.test.js): add test to check if chat history is returned correctly
* test: unit tests for plugin store
* test: add frontend-test script to root package.json
* feat(ChatAgent.js, askGPTPlugins.js): add support for aborting chat requests (in progress)
* test: add more client tests
* feat(ChatAgent): allow plugin requests to be cancelled
* feat(ChatAgent): allow message regeneration
* feat(ChatAgent): remember last selected tools
* Remove plugins we don't yet have from manifest.json
* fix(ChatAgent.js): increase maxAttempts from 1 to 2
fix(ChatAgent.js): change error message to 'Cancelled.' if message was aborted mid-generation
fix(openaiCreateImage.js): replace unwanted characters in input string
fix(handlers.js): compare action.tool in lowercase to 'self-reflection'
* fix(ChatAgent): Fix up plugin I/O formatting for n/a actions
* refactor(Plugin.jsx): remove unused import statement
feat(Plugin.jsx): add Plugin component with svg paths and styles
* refactor: simplify credential encryption/decryption by using a single key and IV for all environments. Update crypto.js and .env.example files accordingly.
* fix(ChatAgent.js): reduce maxAttempts from 2 to 1
feat(ChatAgent.js): add model information to responseMessage object
feat(Message.js): add model field to messageSchema
feat(Message.js): add model field to message object
feat(Message.jsx): pass model information to getIcon function
feat(getIcon.jsx): add Plugin component and handle plugin messages differently
* feat(askGPTPlugins.js): add model property to the ask function response object
feat(EndpointItem.jsx): add message property to the EndpointItem component
feat(MessageHeader.jsx): add Plugin icon to the plugins section
feat(MessageHeader.jsx): change alpha to beta in the plugins section
feat(svg): add Plugin, GPTIcon, and BingIcon components to the svg folder
refactor(EndpointItems.jsx): remove unused import statement
* refactor(googleSearch.js, wolfram.js): change error handling to return a message instead of throwing an error
* refactor(CustomAgent): remove commented code and change return object to include returnValues property
* feat(CustomAgent.js): add currentDateString to createPrompt method options
deps(api/package.json): update langchain to v0.0.81
* fix: do not show pagination if the maxPage is 1
* Add Zapier back to manifest (accidentally removed)
* chore(api): update langchain dependency to version 0.0.84
* feat(DALL-E.js): add DALL-E tool for generating images using OpenAI's DALL-E API
refactor(handleTools.js): update import for DALL-E tool
refactor(index.test.js): update import for DALL-E tool
refactor(stablediffusion.js): add check for image directory existence before saving image
* refactor(CustomAgent): rename instructions prefix variable to gpt3 and add gpt4 instructions
feat(CustomAgent): add support for gpt-4 model
fix(initializeCustomAgent.js): pass model name to createPrompt method
fix(outputParser.js): set selectedTool to 'self-reflection' when tool parsing fails
* style(langchain/tools): update guidelines for image creation in DALL-E and StableDiffusion
- Update guidelines for image creation in DALL-E and StableDiffusion tools
- Emphasize the importance of "showing" and not "telling" the imagery in crafting input
- Update formatting for the example prompt for generating a realistic portrait photo of a man
- Generate images only once per human query unless explicitly requested by the user
* docs(tools): update tool descriptions for DALL-E and Stable Diffusion
- Update the description for DALL-E tool to indicate that it is exclusively for visual content and provide guidelines for generating images with a focus on visual attributes.
- Update the description for Stable Diffusion tool to indicate that it is exclusively for visual content and provide guidelines for generating images with a focus on visual attributes.
* chore(api): update "@waylaidwanderer/chatgpt-api" dependency to version "^1.36.3"
* refactor(ChatAgent.js): use environment variable for reverse proxy url
refactor(ChatAgent.js): use environment variable for openai base path
refactor(instructions.js): update gpt3 and gpt3-v2 instructions
refactor(outputParser.js): update finishToolNameRegex in CustomOutputParser class
* refactor(DALL-E.js): change apiKey and azureKey fields to uppercase
refactor(googleSearch.js): change cx and apiKey fields to uppercase
feat(manifest.json): add authConfig field for Stable Diffusion WebUI API URL
refactor(stablediffusion.js): add url field to constructor and change getServerURL() to this.url
refactor(wolfram.js): change apiKey field to uppercase WOLFRAM_APP_ID
* refactor(handleTools.js): simplify tool loading and add support for custom tool constructors and options
* refactor(handleTools.js): remove commented out code and unused imports
* refactor(handleTools.js, index.js): change file name from wolfram.js to Wolfram.js and selfReflection.js to SelfReflection.js to follow PascalCase convention
* refactor(outputParser.js, askGPTPlugins.js): improve code readability and remove unnecessary comments
* feat(GoogleSearch.js): add GoogleSearchAPI tool to allow agents to use the Google Custom Search API
feat(SelfReflection.js): add SelfReflectionTool to allow agents to reflect on their thoughts and actions
feat(StableDiffusion.js): add StableDiffusionAPI tool to allow agents to generate images using stable diffusion webui's api
feat(Wolfram.js): add WolframAlphaAPI tool for computation, math, curated knowledge & real-time data through WolframAlpha.
* testing openai specs
* doc: fix link in .env.example
* package-update
* fix(MultiSelectDropDown.jsx): handle null or undefined values in availableValues array
* refactor(DALL-E.js, StableDiffusion.js): remove 'dist/' from image path
feat(docker-compose.yml): add comments for reverse proxy configuration
* chore(.gitignore): ignore client/public/images/
fix(DALL-E.js, StableDiffusion.js): change image path from dist/ to public/
feat(index.js): add support for serving static files from client/public/ directory
* fix: remove selected tool when uninstalled
* plugin options in progress
* fix: fix issue with uninstalling a plugin that is in use and typescript errors
* feat(gptPlugins): add Preset support for GPT Plugins endpoint
feat(ChatAgent.js): add support for agentOptions object
feat(convoSchema.js): add agentOptions field to conversation schema
feat(defaults.js): add agentOptions object to defaults
feat(presetSchema.js): add agentOptions field to preset schema
feat(askGPTPlugins.js): add support for agentOptions object in request body
feat(EditPresetDialog.jsx): add support for showing/hiding GPT Plugins agent settings
feat(EditPresetDialog.jsx): add support for setting GPT Plugins agent options
fix(EndpointOptionsDialog.jsx): change endpoint name from 'gptPlugins' to 'Plugins'
feat(AgentSettings.jsx): add AgentSettings component for GPT plugins configuration
feat(client): add GPT Plugins settings component and endpoint to Settings component
fix(client): remove unused imports in GoogleOptions component
feat(PluginsOptions): add support for agent settings and refactor code
feat(PluginsOptions): add GPTIcon to show/hide agent settings button
feat(index.ts): export SVG components
feat(GPTIcon.jsx): add className prop to GPTIcon component
feat(GPTIcon.jsx): import cn function from utils
feat(BingIcon.tsx): export BingIcon component
feat(index.ts): export BingIcon component
feat(index.ts): export MessagesSquared component
refactor(cleanupPreset.js): add default values for agentOptions in gptPlugins endpoint
feat(getDefaultConversation.js, handleSubmit.js): add agentOptions object to conversation object for GPT plugins endpoint. Update default temperature value to 0.8. Add chatGptLabel and promptPrefix properties to conversation object.
* fix: set default convo back to null
* refactor(ChatAgent.js, askGPTPlugins.js, AgentSettings.jsx): change variable names for better readability and remove redundant code
* test: add RecoilRoot to layout-test-utils
* refactor(askGPTPlugins.js): remove redundant code and use endpointOption directly
feat(askGPTPlugins.js): add validation for tools in endpointOption before using it
* chore(ChatAgent.js, Settings.jsx): add agentOptions to saveConvo function and adjust Settings component height
The ChatAgent.js file was modified to include the agentOptions object in the saveConvo function. The Settings.jsx file was modified to adjust the height of the component to ensure that all content is visible.
* refactor(ChatAgent.js): extract reverseProxyUrl option to a class property and add support for it
feat(ChatAgent.js): add support for completionMode option in sendApiMessage method
feat(ChatAgent.js): add support for user-provided promptPrefix in buildPrompt method
* feat(plugins): allow preset change mid conversation
* refactor: playwright config
* build: update configs
* test: add automated setup process for playwright
* test: update messages spec to delete message after its been created
* build: add github action for e2e to workflows
* make husky install conditional
* build: try making husky install exit if running in ci
* ignore husky on ci
* chore: update OPENAI_KEY to OPENAI_API_KEY in .github/playwright.yml and api/.env.example
refactor(chatgpt-client.js): update OPENAI_KEY to OPENAI_API_KEY
feat(langchain): add demo-aiplugin.js and demo-yaml.js, remove test2.js, test3.js, and test4.js
chore: remove unused test files
fix(titleConvo.js): fix typo in environment variable name
fix(askGPTPlugins.js): fix typo in environment variable name
fix(endpoints.js): fix typo in environment variable name
docs: update installation guide to use OPENAI_API_KEY instead of OPENAI_KEY in .env file
* fix(index.test.js): change import of GoogleSearchAPI to use uppercase G in GoogleSearch
* backtrack and redo
* add jwt secret
* try manually installing vite
* refactor workflow again
* move vite to root and add frontend:build script
* chore(api): bump langchain version
* feat(PluginController.js): authenticate plugins from environment variables if they are set
feat(PluginStoreDialog.tsx): show plugin auth form only if plugin is not authenticated by env var and require authentication
feat(types.ts): add authenticated field to TPlugin type definition
* docs: update google_search.md and add stable_diffusion.md
* Update stable_diffusion.md
* refactor(Wolfram.js): remove newline characters from query before encoding
docs(wolfram.md): add instructions for setting WOLFRAM_APP_ID in api.env to bypass prompt for AppID in plugin
* refactor(Wolfram.js): replace deprecated replaceAll method with replace method
* Update wolfram.md
* fix(askGPTPlugins): error message will reference correct Parent Message
* refactor(chatgpt-client.js, ChatAgent.js): simplify maxContextTokens calculation and add promptPrefix parameter to buildPrompt method
* docs: initial draft of intro to plugins
* Update introduction.md
* Update introduction.md
* Feature: User/Reg cleanup + Install / Upgrade script for langchain (#427)
* test: login tests
* test: finish login tests
* test: initial tests for registration
* test: registration specs
* feature: Init a app config file
- Simplifies the ENV vars too
- Legacy fallbacks for older builds
* refactor(auth): Refactor log in/out controllers
- Moves both login and logout controllers to their own file
* chore(jwt): Throw warning if secret is default
* feature(frontend): Ability to disable registration
* feature(env): Env in the root + version support
ie .env.prod, .env.dev, .env.test
* feature: Upgrade .env script for users
* chore(config): Refactor and remove legacy env refs
* feature(upgrade): Upgrade script for .env changes
* feature: Install script and upgrade script
* bugfix: Uncomment line to remove old .env file
* chore: rename OPENAI_KEY to OPENAI_API_KEY
* chore: Cleanup config changes/bugs
* bugfix: Fix config and node env issues
* bugfix: Config validation logic
* bugfix: Handle unusual env configs gracefully
* bugfix: Revert route changes and fix registration disable calling
* bugfix: Fix env issues in frontend
* bugfix: Fix login
* bugfix: Fix frontend envs
* bugfix: Fix frontend jest tests
* bugfix: Fix upgrade scripts
* bugfix: Allow install in non-tty envs
* bugfix(windows): Use cross-env to set for windows
* bugfix(env): Handle .env being incorrect to begin with for client domain
* chore(merge-conflict): Update to LibreChat
* chore(merge-conflict): Update to package-lock
---------
Co-authored-by: Daniel D Orlando <dan@danorlando.com>
* chore: comment out unused agent options
* refactor playwright.yml again
* try adding .npmrc
* delete playwright yml
* create new action
* test new playwright action
* Update langchain plugins docs (#461)
* Update: install docs (LibreChat) (#458)
* Release: rename project from ChatGPT Clone to LibreChat
Release: rename project from ChatGPT Clone to LibreChat
* Release: rename project from ChatGPT Clone to LibreChat
Release: rename project from ChatGPT Clone to LibreChat
* Release: rename project from ChatGPT Clone to LibreChat
Release: rename project from ChatGPT Clone to LibreChat
* Release: rename project from ChatGPT Clone to LibreChat
Release: rename project from ChatGPT Clone to LibreChat
* Update documentation_guidelines.md
* Update introduction.md
add link to readme
* Update stable_diffusion.md
add link back to readme
* Update wolfram.md
add link back to readme
* Update README.md
add Plugins to ToC
* feat(ChatAgent.js): add support for langchainProxy configuration option
Add a new configuration option `langchainProxy` to the ChatAgent class. If the option is set, the `basePath` configuration option of the `ChatOpenAI` instance is set to the base path of `langchainProxy`.
* bugfix(errors): Possible workaround for error flashing (#463)
* Test/user auth system client tests (#462)
* test: login tests
* test: finish login tests
* test: initial tests for registration
* test: registration specs
* modify for new secrets
* modify for new secrets
* fix corrupted package-lock
* remove env vars for testing
* chore(api): update langchain dependency to version 0.0.91
* Update introduction.md
* Update introduction.md
* Update introduction.md
* run it again
* try externalizing uuid package in rollupoptions to fix build error
* update authenticate script for LibreChat
* add npm install playwright/test to run
* try adding langchain dependency to root package.json to resolve zod error
* add missing env vars
* fix: no longer renders html in markdown content
fix: patch XSS vulnerability completely by handling cursor on the frontend without css/html
* fix(Content.jsx): fix cursor logic so it never shows for static messages
* bugfix(langchain): Upgrade script, docker, env and docs (#465)
* bugfix(errors): Remove incorrect manual fix from misunderstanding
* chore(env): Lets not make a .env.prod and use the prod values in the default root .env
- .env.dev will still be created
* chore(upgrade.js): Lets tell the user about .env.dev if we create it
* bugfix(env): Move to full name environments for vite
- .env.prod => .env.production
- .env.dev => .env.development
* chore(env-example): Explain how to get google login working in production
* bugfix(oauth): Minor fix to point isProduction to a correct value
* bugfix: Typo in public
* chore(docs): Update docs to note the changes to .env
* chore(docs): Include note on how to get google auth working in dev and how to disable registration
* bugfix: Fix missing env changes
* bugfix: Fix up docker to work with new env / npm changes
* Update .env.example
Cleanup the env of the palm2 instruction and fix to formating
* chore(docker): Simplify Docker deployments
- Needs work to support dev env/hotreload
* bugfix: Remove volume map for client dir
* chore(env-example): Change instructions to be more user centric
---------
Co-authored-by: Fuegovic <32828263+fuegovic@users.noreply.github.com>
* update: install docs (#466)
* Add files via upload
* Update apis-and-tokens.md
* Update apis-and-tokens.md
* Update docker_install.md
* Update linux_install.md
* Rename apis-and-tokens.md to apis_and_tokens.md
* Update docker_install.md
* Update linux_install.md
* Update mac_install.md
* Update linux_install.md
* Update docker_install.md
* Update windows_install.md
* Update apis_and_tokens.md
* Update mac_install.md
* Update linux_install.md
* Update docker_install.md
* Update README.md
* Update README.md : Breaking Changes
---------
Co-authored-by: Danny Avila <110412045+danny-avila@users.noreply.github.com>
* Update README.md (#468)
add new API/Token docs to Toc
* docs: guide on how to create your own plugin
* Update make_your_own.md
* Update make_your_own.md
* feat(docker): add build args for frontend variables in Dockerfile
feat(docker-compose): add build args for frontend variables in docker-compose.yml
* Update docker_install.md
* Update docker_install.md
* Update docker_install.md
* Update docker_install.md
* docs: update (#469)
* Update: make_your_own.md
* Update README.md
add `make_your_own.md` to ToC
* Update linux_install.md
* Update mac_install.md
* Update windows_install.md
* Update apis_and_tokens.md
* Update docker_install.md
* Update docker_install.md
* Update linux_install.md
* Update mac_install.md
* Update windows_install.md
* Update apis_and_tokens.md
* Update user_auth_system.md
* Update docker_install.md
clean up of repeated information
* Update docker_install.md
* Update docker_install.md
typo
* fix: fix issue with pluginstore next and prev buttons going out of bounds
* fix: add icon for web browser plugin
* docs(GoogleSearch.js): update description of GoogleSearchAPI class to be more descriptive of its functionality
* feat(ask/handlers.js): add cursor to indicate ongoing progress of a long-running task
fix(Content.jsx): handle null content in the message stream by replacing it with an empty string (with a space so a text space is rendered)
* Update README.md
* Update README.md
* fix: plugin option stacking order
* update: web browser icon (#470)
* Delete web-browser.png
* update: web browser icon
* Update readme (#472)
* Update README.md
Discord badge now displays the number of online users
Project description has been updated to reflect current status
Feature section has been updated to reflect current capabilities
Sponsors section is now located just above the contributors section
Roadmap has been removed as it was outdated.
* Delete roadmap.md
Roadmap has been removed to streamline document maintenance.
* Update README.md
* Update README.md
* Delete CHANGELOG.md
* add frontend unit test action
* action for backend unit test run
* add env vars for backend tests
* add step for install api deps
* use test:ci script
* make test:ci run from api folder
* add script for installing linux x64 sharp
* move jest setup from root to /api
* fix: pluginstore in mobile view getting clipped and not scrolling
* change scripts to test:client and test:api in root package.json
* test api pipeline sharp installation
* remove sharp installation
* last attempt
* revert
* docs(linux_install.md): remove duplicate git clone command
* chore(Dockerfile): comment out nginx-client build stage
docs(README.md): update installation instructions and mention docker-compose changes
docs(features/plugins/introduction.md): bold plugin names and add emphasis to notes
* feat: add superscript and subscript support to markdown rendering
refactor: support markdown citations for BingAI
* refactor: support markdown citations for BingAI
* chore(client): add cross-env package to dependencies
refactor(client): comment out uuid from external rollup options in vite.config.ts
* feat(package-lock.json): add Jest as a devDependency and remove uuidv4 dependency
* feat(workflows): add jest branch to backend and frontend unit tests on push and pull_request events
* chore(workflows): update Node.js version to 19.x in backend and frontend review workflows
* test(PluginStoreDialog.spec.tsx): update test to reflect new pagination
The test was updated to reflect the new pagination. The test now expects to see Plugin 6 and Plugin 7 on the second page instead of Plugin 3, Plugin 4, and Plugin 5. The previous expectations were commented out.
* chore: update package versions to 0.5.0
chore: remove jest branch from backend-review.yml
feat: add support for feat/playwright-jest-cicd branch in backend-review.yml
fix(api): fix sampleTools variable in index.test.js
* fix(Content.jsx): replace escaped whitespace with invisible character
---------
Co-authored-by: David Shin <42793498+dncc89@users.noreply.github.com>
Co-authored-by: Daniel D Orlando <dan@danorlando.com>
Co-authored-by: LaraClara <2524209+ClaraLeigh@users.noreply.github.com>
Co-authored-by: Fuegovic <32828263+fuegovic@users.noreply.github.com>
2023-06-11 00:00:48 -04:00
|
|
|
"test": "cross-env NODE_ENV=test jest",
|
feat: Message Rate Limiters, Violation Logging, & Ban System 🔨 (#903)
* refactor: require Auth middleware in route index files
* feat: concurrent message limiter
* feat: complete concurrent message limiter with caching
* refactor: SSE response methods separated from handleText
* fix(abortMiddleware): fix req and res order to standard, use endpointOption in req.body
* chore: minor name changes
* refactor: add isUUID condition to saveMessage
* fix(concurrentLimiter): logic correctly handles the max number of concurrent messages and res closing/finalization
* chore: bump keyv and remove console.log from Message
* fix(concurrentLimiter): ensure messages are only saved in later message children
* refactor(concurrentLimiter): use KeyvFile instead, could make other stores configurable in the future
* feat: add denyRequest function for error responses
* feat(utils): add isStringTruthy function
Introduce the isStringTruthy function to the utilities module to check if a string value is a case-insensitive match for 'true'
* feat: add optional message rate limiters by IP and userId
* feat: add optional message rate limiters by IP and userId to edit route
* refactor: rename isStringTruthy to isTrue for brevity
* refactor(getError): use map to make code cleaner
* refactor: use memory for concurrent rate limiter to prevent clearing on startup/exit, add multiple log files, fix error message for concurrent violation
* feat: check if errorMessage is object, stringify if so
* chore: send object to denyRequest which will stringify it
* feat: log excessive requests
* fix(getError): correctly pluralize messages
* refactor(limiters): make type consistent between logs and errorMessage
* refactor(cache): move files out of lib/db into separate cache dir
>> feat: add getLogStores function so Keyv instance is not redundantly created on every violation
feat: separate violation logging to own function with logViolation
* fix: cache/index.js export, properly record userViolations
* refactor(messageLimiters): use new logging method, add logging to registrations
* refactor(logViolation): make userLogs an array of logs per user
* feat: add logging to login limiter
* refactor: pass req as first param to logViolation and record offending IP
* refactor: rename isTrue helper fn to isEnabled
* feat: add simple non_browser check and log violation
* fix: open handles in unit tests, remove KeyvMongo as not used and properly mock global fetch
* chore: adjust nodemon ignore paths to properly ignore logs
* feat: add math helper function for safe use of eval
* refactor(api/convos): use middleware at top of file to avoid redundancy
* feat: add delete all static method for Sessions
* fix: redirect to login on refresh if user is not found, or the session is not found but hasn't expired (ban case)
* refactor(getLogStores): adjust return type
* feat: add ban violation and check ban logic
refactor(logViolation): pass both req and res objects
* feat: add removePorts helper function
* refactor: rename getError to getMessageError and add getLoginError for displaying different login errors
* fix(AuthContext): fix type issue and remove unused code
* refactor(bans): ban by ip and user id, send response based on origin
* chore: add frontend ban messages
* refactor(routes/oauth): add ban check to handler, also consolidate logic to avoid redundancy
* feat: add ban check to AI messaging routes
* feat: add ban check to login/registration
* fix(ci/api): mock KeyvMongo to avoid tests hanging
* docs: update .env.example
> refactor(banViolation): calculate interval rate crossover, early return if duration is invalid
ci(banViolation): add tests to ensure users are only banned when expected
* docs: improve wording for mod system
* feat: add configurable env variables for violation scores
* chore: add jsdoc for uaParser.js
* chore: improve ban text log
* chore: update bun test scripts
* refactor(math.js): add fallback values
* fix(KeyvMongo/banLogs): refactor keyv instances to top of files to avoid memory leaks, refactor ban logic to use getLogStores instead
refactor(getLogStores): get a single log store by type
* fix(ci): refactor tests due to banLogs changes, also make sure to clear and revoke sessions even if ban duration is 0
* fix(banViolation.js): getLogStores import
* feat: handle 500 code error at login
* fix(middleware): handle case where user.id is _id and not just id
* ci: add ban secrets for backend unit tests
* refactor: logout user upon ban
* chore: log session delete message only if deletedCount > 0
* refactor: change default ban duration (2h) and make logic more clear in JSDOC
* fix: login and registration limiters will now return rate limiting error
* fix: userId not parsable as non ObjectId string
* feat: add useTimeout hook to properly clear timeouts when invoking functions within them
refactor(AuthContext): cleanup code by using new hook and defining types in ~/common
* fix: login error message for rate limits
* docs: add info for automated mod system and rate limiters, update other docs accordingly
* chore: bump data-provider version
2023-09-13 10:57:07 -04:00
|
|
|
"b:test": "NODE_ENV=test bun jest",
|
2025-12-25 16:17:17 -05:00
|
|
|
"test:ci": "jest --ci --logHeapUsage",
|
2024-03-31 18:59:07 -04:00
|
|
|
"add-balance": "node ./add-balance.js",
|
|
|
|
|
"list-balances": "node ./list-balances.js",
|
|
|
|
|
"user-stats": "node ./user-stats.js",
|
|
|
|
|
"create-user": "node ./create-user.js",
|
2024-08-18 06:23:38 +02:00
|
|
|
"invite-user": "node ./invite-user.js",
|
2024-03-31 18:59:07 -04:00
|
|
|
"ban-user": "node ./ban-user.js",
|
|
|
|
|
"delete-user": "node ./delete-user.js"
|
2023-03-06 10:15:07 -05:00
|
|
|
},
|
|
|
|
|
"repository": {
|
|
|
|
|
"type": "git",
|
2023-06-05 14:24:08 -04:00
|
|
|
"url": "git+https://github.com/danny-avila/LibreChat.git"
|
2023-03-06 10:15:07 -05:00
|
|
|
},
|
|
|
|
|
"keywords": [],
|
|
|
|
|
"author": "",
|
|
|
|
|
"license": "ISC",
|
feat: Vision Support + New UI (#1203)
* feat: add timer duration to showToast, show toast for preset selection
* refactor: replace old /chat/ route with /c/. e2e tests will fail here
* refactor: move typedefs to root of /api/ and add a few to assistant types in TS
* refactor: reorganize data-provider imports, fix dependency cycle, strategize new plan to separate react dependent packages
* feat: add dataService for uploading images
* feat(data-provider): add mutation keys
* feat: file resizing and upload
* WIP: initial API image handling
* fix: catch JSON.parse of localStorage tools
* chore: experimental: use module-alias for absolute imports
* refactor: change temp_file_id strategy
* fix: updating files state by using Map and defining react query callbacks in a way that keeps them during component unmount, initial delete handling
* feat: properly handle file deletion
* refactor: unexpose complete filepath and resize from server for higher fidelity
* fix: make sure resized height, width is saved, catch bad requests
* refactor: use absolute imports
* fix: prevent setOptions from being called more than once for OpenAIClient, made note to fix for PluginsClient
* refactor: import supportsFiles and models vars from schemas
* fix: correctly replace temp file id
* refactor(BaseClient): use absolute imports, pass message 'opts' to buildMessages method, count tokens for nested objects/arrays
* feat: add validateVisionModel to determine if model has vision capabilities
* chore(checkBalance): update jsdoc
* feat: formatVisionMessage: change message content format dependent on role and image_urls passed
* refactor: add usage to File schema, make create and updateFile, correctly set and remove TTL
* feat: working vision support
TODO: file size, type, amount validations, making sure they are styled right, and making sure you can add images from the clipboard/dragging
* feat: clipboard support for uploading images
* feat: handle files on drop to screen, refactor top level view code to Presentation component so the useDragHelpers hook has ChatContext
* fix(Images): replace uploaded images in place
* feat: add filepath validation to protect sensitive files
* fix: ensure correct file_ids are push and not the Map key values
* fix(ToastContext): type issue
* feat: add basic file validation
* fix(useDragHelpers): correct context issue with `files` dependency
* refactor: consolidate setErrors logic to setError
* feat: add dialog Image overlay on image click
* fix: close endpoints menu on click
* chore: set detail to auto, make note for configuration
* fix: react warning (button desc. of button)
* refactor: optimize filepath handling, pass file_ids to images for easier re-use
* refactor: optimize image file handling, allow re-using files in regen, pass more file metadata in messages
* feat: lazy loading images including use of upload preview
* fix: SetKeyDialog closing, stopPropagation on Dialog content click
* style(EndpointMenuItem): tighten up the style, fix dark theme showing in lightmode, make menu more ux friendly
* style: change maxheight of all settings textareas to 138px from 300px
* style: better styling for textarea and enclosing buttons
* refactor(PresetItems): swap back edit and delete icons
* feat: make textarea placeholder dynamic to endpoint
* style: show user hover buttons only on hover when message is streaming
* fix: ordered list not going past 9, fix css
* feat: add User/AI labels; style: hide loading spinner
* feat: add back custom footer, change original footer text
* feat: dynamic landing icons based on endpoint
* chore: comment out assistants route
* fix: autoScroll to newest on /c/ view
* fix: Export Conversation on new UI
* style: match message style of official more closely
* ci: fix api jest unit tests, comment out e2e tests for now as they will fail until addressed
* feat: more file validation and use blob in preview field, not filepath, to fix temp deletion
* feat: filefilter for multer
* feat: better AI labels based on custom name, model, and endpoint instead of `ChatGPT`
2023-11-21 20:12:48 -05:00
|
|
|
"_moduleAliases": {
|
|
|
|
|
"~": "."
|
|
|
|
|
},
|
|
|
|
|
"imports": {
|
|
|
|
|
"~/*": "./*"
|
|
|
|
|
},
|
2023-03-06 10:15:07 -05:00
|
|
|
"bugs": {
|
2023-06-05 14:24:08 -04:00
|
|
|
"url": "https://github.com/danny-avila/LibreChat/issues"
|
2023-03-06 10:15:07 -05:00
|
|
|
},
|
2024-02-19 09:42:57 -05:00
|
|
|
"homepage": "https://librechat.ai",
|
2023-03-06 10:15:07 -05:00
|
|
|
"dependencies": {
|
2026-02-06 18:35:36 -05:00
|
|
|
"@anthropic-ai/vertex-sdk": "^0.14.3",
|
2026-02-02 10:11:57 -05:00
|
|
|
"@aws-sdk/client-bedrock-runtime": "^3.980.0",
|
|
|
|
|
"@aws-sdk/client-s3": "^3.980.0",
|
2025-03-19 07:04:45 +01:00
|
|
|
"@aws-sdk/s3-request-presigner": "^3.758.0",
|
2025-03-19 15:45:52 +01:00
|
|
|
"@azure/identity": "^4.7.0",
|
2023-11-30 15:50:28 -03:00
|
|
|
"@azure/search-documents": "^12.0.0",
|
2026-02-02 10:11:57 -05:00
|
|
|
"@azure/storage-blob": "^12.30.0",
|
2026-01-03 11:26:46 -05:00
|
|
|
"@google/genai": "^1.19.0",
|
2025-10-04 16:43:22 -04:00
|
|
|
"@keyv/redis": "^4.3.3",
|
2025-12-25 01:43:54 -05:00
|
|
|
"@langchain/core": "^0.3.80",
|
⚗️ feat: Agent Context Compaction/Summarization (#12287)
* chore: imports/types
Add summarization config and package-level summarize handler contracts
Register summarize handlers across server controller paths
Port cursor dual-read/dual-write summary support and UI status handling
Selectively merge cursor branch files for BaseClient summary content
block detection (last-summary-wins), dual-write persistence, summary
block unit tests, and on_summarize_status SSE event handling with
started/completed/failed branches.
Co-authored-by: Cursor <cursoragent@cursor.com>
refactor: type safety
feat: add localization for summarization status messages
refactor: optimize summary block detection in BaseClient
Updated the logic for identifying existing summary content blocks to use a reverse loop for improved efficiency. Added a new test case to ensure the last summary content block is updated correctly when multiple summary blocks exist.
chore: add runName to chainOptions in AgentClient
refactor: streamline summarization configuration and handler integration
Removed the deprecated summarizeNotConfigured function and replaced it with a more flexible createSummarizeFn. Updated the summarization handler setup across various controllers to utilize the new function, enhancing error handling and configuration resolution. Improved overall code clarity and maintainability by consolidating summarization logic.
feat(summarization): add staged chunk-and-merge fallback
feat(usage): track summarization usage separately from messages
feat(summarization): resolve prompt from config in runtime
fix(endpoints): use @librechat/api provider config loader
refactor(agents): import getProviderConfig from @librechat/api
chore: code order
feat(app-config): auto-enable summarization when configured
feat: summarization config
refactor(summarization): streamline persist summary handling and enhance configuration validation
Removed the deprecated createDeferredPersistSummary function and integrated a new createPersistSummary function for MongoDB persistence. Updated summarization handlers across various controllers to utilize the new persistence method. Enhanced validation for summarization configuration to ensure provider, model, and prompt are properly set, improving error handling and overall robustness.
refactor(summarization): update event handling and remove legacy summarize handlers
Replaced the deprecated summarization handlers with new event-driven handlers for summarization start and completion across multiple controllers. This change enhances the clarity of the summarization process and improves the integration of summarization events in the application. Additionally, removed unused summarization functions and streamlined the configuration loading process.
refactor(summarization): standardize event names in handlers
Updated event names in the summarization handlers to use constants from GraphEvents for consistency and clarity. This change improves maintainability and reduces the risk of errors related to string literals in event handling.
feat(summarization): enhance usage tracking for summarization events
Added logic to track summarization usage in multiple controllers by checking the current node type. If the node indicates a summarization task, the usage type is set accordingly. This change improves the granularity of usage data collected during summarization processes.
feat(summarization): integrate SummarizationConfig into AppSummarizationConfig type
Enhanced the AppSummarizationConfig type by extending it with the SummarizationConfig type from librechat-data-provider. This change improves type safety and consistency in the summarization configuration structure.
test: add end-to-end tests for summarization functionality
Introduced a comprehensive suite of end-to-end tests for the summarization feature, covering the full LibreChat pipeline from message creation to summarization. This includes a new setup file for environment configuration and a Jest configuration specifically for E2E tests. The tests utilize real API keys and ensure proper integration with the summarization process, enhancing overall test coverage and reliability.
refactor(summarization): include initial summary in formatAgentMessages output
Updated the formatAgentMessages function to return an initial summary alongside messages and index token count map. This change is reflected in multiple controllers and the corresponding tests, enhancing the summarization process by providing additional context for each agent's response.
refactor: move hydrateMissingIndexTokenCounts to tokenMap utility
Extracted the hydrateMissingIndexTokenCounts function from the AgentClient and related tests into a new tokenMap utility file. This change improves code organization and reusability, allowing for better management of token counting logic across the application.
refactor(summarization): standardize step event handling and improve summary rendering
Refactored the step event handling in the useStepHandler and related components to utilize constants for event names, enhancing consistency and maintainability. Additionally, improved the rendering logic in the Summary component to conditionally display the summary text based on its availability, providing a better user experience during the summarization process.
feat(summarization): introduce baseContextTokens and reserveTokensRatio for improved context management
Added baseContextTokens to the InitializedAgent type to calculate the context budget based on agentMaxContextNum and maxOutputTokensNum. Implemented reserveTokensRatio in the createRun function to allow configurable context token management. Updated related tests to validate these changes and ensure proper functionality.
feat(summarization): add minReserveTokens, context pruning, and overflow recovery configurations
Introduced new configuration options for summarization, including minReserveTokens, context pruning settings, and overflow recovery parameters. Updated the createRun function to accommodate these new options and added a comprehensive test suite to validate their functionality and integration within the summarization process.
feat(summarization): add updatePrompt and reserveTokensRatio to summarization configuration
Introduced an updatePrompt field for updating existing summaries with new messages, enhancing the flexibility of the summarization process. Additionally, added reserveTokensRatio to the configuration schema, allowing for improved management of token allocation during summarization. Updated related tests to validate these new features.
feat(logging): add on_agent_log event handler for structured logging
Implemented an on_agent_log event handler in both the agents' callbacks and responses to facilitate structured logging of agent activities. This enhancement allows for better tracking and debugging of agent interactions by logging messages with associated metadata. Updated the summarization process to ensure proper handling of log events.
fix: remove duplicate IBalanceUpdate interface declaration
perf(usage): single-pass partition of collectedUsage
Replace two Array.filter() passes with a single for-of loop that
partitions message vs. summarization usages in one iteration.
fix(BaseClient): shallow-copy message content before mutating and preserve string content
Avoid mutating the original message.content array in-place when
appending a summary block. Also convert string content to a text
content part instead of silently discarding it.
fix(ui): fix Part.tsx indentation and useStepHandler summarize-complete handling
- Fix SUMMARY else-if branch indentation in Part.tsx to match chain level
- Guard ON_SUMMARIZE_COMPLETE with didFinalize flag to avoid unnecessary
re-renders when no summarizing parts exist
- Protect against undefined completeData.summary instead of unsafe spread
fix(agents): use strict enabled check for summarization handlers
Change summarizationConfig?.enabled !== false to === true so handlers
are not registered when summarizationConfig is undefined.
chore: fix initializeClient JSDoc and move DEFAULT_RESERVE_RATIO to module scope
refactor(Summary): align collapse/expand behavior with Reasoning component
- Single render path instead of separate streaming vs completed branches
- Use useMessageContext for isSubmitting/isLatestMessage awareness so
the "Summarizing..." label only shows during active streaming
- Default to collapsed (matching Reasoning), user toggles to expand
- Add proper aria attributes (aria-hidden, role, aria-controls, contentId)
- Hide copy button while actively streaming
feat(summarization): default to self-summarize using agent's own provider/model
When no summarization config is provided (neither in librechat.yaml nor
on the agent), automatically enable summarization using the agent's own
provider and model. The agents package already provides default prompts,
so no prompt configuration is needed.
Also removes the dead resolveSummarizationLLMConfig in summarize.ts
(and its spec) — run.ts buildAgentContext is the single source of truth
for summarization config resolution. Removes the duplicate
RuntimeSummarizationConfig local type in favor of the canonical
SummarizationConfig from data-provider.
chore: schema and type cleanup for summarization
- Add trigger field to summarizationAgentOverrideSchema so per-agent
trigger overrides in librechat.yaml are not silently stripped by Zod
- Remove unused SummarizationStatus type from runs.ts
- Make AppSummarizationConfig.enabled non-optional to reflect the
invariant that loadSummarizationConfig always sets it
refactor(responses): extract duplicated on_agent_log handler
refactor(run): use agents package types for summarization config
Import SummarizationConfig, ContextPruningConfig, and
OverflowRecoveryConfig from @librechat/agents and use them to
type-check the translation layer in buildAgentContext. This ensures
the config object passed to the agent graph matches what it expects.
- Use `satisfies AgentSummarizationConfig` on the config object
- Cast contextPruningConfig and overflowRecoveryConfig to agents types
- Properly narrow trigger fields from DeepPartial to required shape
feat(config): add maxToolResultChars to base endpoint schema
Add maxToolResultChars to baseEndpointSchema so it can be configured
on any endpoint in librechat.yaml. Resolved during agent initialization
using getProviderConfig's endpoint resolution: custom endpoint config
takes precedence, then the provider-specific endpoint config, then the
shared `all` config.
Passed through to the agents package ToolNode, which uses it to cap
tool result length before it enters the context window. When not
configured, the agents package computes a sensible default from
maxContextTokens.
fix(summarization): forward agent model_parameters in self-summarize default
When no explicit summarization config exists, the self-summarize
default now forwards the agent's model_parameters as the
summarization parameters. This ensures provider-specific settings
(e.g. Bedrock region, credentials, endpoint host) are available
when the agents package constructs the summarization LLM.
fix(agents): register summarization handlers by default
Change the enabled gate from === true to !== false so handlers
register when no explicit summarization config exists. This aligns
with the self-summarize default where summarization is always on
unless explicitly disabled via enabled: false.
refactor(summarization): let agents package inherit clientOptions for self-summarize
Remove model_parameters forwarding from the self-summarize default.
The agents package now reuses the agent's own clientOptions when the
summarization provider matches the agent's provider, inheriting all
provider-specific settings (region, credentials, proxy, etc.)
automatically.
refactor(summarization): use MessageContentComplex[] for summary content
Unify summary content to always use MessageContentComplex[] arrays,
matching the pattern used by on_message_delta. No more string | array
unions — content is always an array of typed blocks ({ type: 'text',
text: '...' } for text, { type: 'reasoning_content', ... } for
reasoning).
Agents package:
- SummaryContentBlock.content: MessageContentComplex[] (was string)
- tokenCount now optional (not sent on deltas)
- Removed reasoning field — reasoning is now a content block type
- streamAndCollect normalizes all chunks to content block arrays
- Delta events pass content blocks directly
LibreChat:
- SummaryContentPart.content: Agents.MessageContentComplex[]
- Updated Part.tsx, Summary.tsx, useStepHandler.ts, BaseClient.js
- Summary.tsx derives display text from content blocks via useMemo
- Aggregator uses simple array spread
refactor(summarization): enhance summary handling and text extraction
- Updated BaseClient.js to improve summary text extraction, accommodating both legacy and new content formats.
- Modified summarization logic to ensure consistent handling of summary content across different message formats.
- Adjusted test cases in summarization.e2e.spec.js to utilize the new summary text extraction method.
- Refined SSE useStepHandler to initialize summary content as an array.
- Updated configuration schema by removing unused minReserveTokens field.
- Cleaned up SummaryContentPart type by removing rangeHash property.
These changes streamline the summarization process and ensure compatibility with various content structures.
refactor(summarization): streamline usage tracking and logging
- Removed direct checks for summarization nodes in ModelEndHandler and replaced them with a dedicated markSummarizationUsage function for better readability and maintainability.
- Updated OpenAIChatCompletionController and responses handlers to utilize the new markSummarizationUsage function for setting usage types.
- Enhanced logging functionality by ensuring the logger correctly handles different log levels.
- Introduced a new useCopyToClipboard hook in the Summary component to encapsulate clipboard copy logic, improving code reusability and clarity.
These changes improve the overall structure and efficiency of the summarization handling and logging processes.
refactor(summarization): update summary content block documentation
- Removed outdated comment regarding the last summary content block in BaseClient.js.
- Added a new comment to clarify the purpose of the findSummaryContentBlock method, ensuring consistency in documentation.
These changes enhance code clarity and maintainability by providing accurate descriptions of the summarization logic.
refactor(summarization): update summary content structure in tests
- Modified the summarization content structure in e2e tests to use an array format for text, aligning with recent changes in summary handling.
- Updated test descriptions to clarify the behavior of context token calculations, ensuring consistency and clarity in the tests.
These changes enhance the accuracy and maintainability of the summarization tests by reflecting the updated content structure.
refactor(summarization): remove legacy E2E test setup and configuration
- Deleted the e2e-setup.js and jest.e2e.config.js files, which contained legacy configurations for E2E tests using real API keys.
- Introduced a new summarization.e2e.ts file that implements comprehensive E2E backend integration tests for the summarization process, utilizing real AI providers and tracking summaries throughout the run.
These changes streamline the testing framework by consolidating E2E tests into a single, more robust file while removing outdated configurations.
refactor(summarization): enhance E2E tests and error handling
- Added a cleanup step to force exit after all tests to manage Redis connections.
- Updated the summarization model to 'claude-haiku-4-5-20251001' for consistency across tests.
- Improved error handling in the processStream function to capture and return processing errors.
- Enhanced logging for cross-run tests and tight context scenarios to provide better insights into test execution.
These changes improve the reliability and clarity of the E2E tests for the summarization process.
refactor(summarization): enhance test coverage for maxContextTokens behavior
- Updated run-summarization.test.ts to include a new test case ensuring that maxContextTokens does not exceed user-defined limits, even when calculated ratios suggest otherwise.
- Modified summarization.e2e.ts to replace legacy UsageMetadata type with a more appropriate type for collectedUsage, improving type safety and clarity in the test setup.
These changes improve the robustness of the summarization tests by validating context token constraints and refining type definitions.
feat(summarization): add comprehensive E2E tests for summarization process
- Introduced a new summarization.e2e.test.ts file that implements extensive end-to-end integration tests for the summarization pipeline, covering the full flow from LibreChat to agents.
- The tests utilize real AI providers and include functionality to track summaries during and between runs.
- Added necessary cleanup steps to manage Redis connections post-tests and ensure proper exit.
These changes enhance the testing framework by providing robust coverage for the summarization process, ensuring reliability and performance under real-world conditions.
fix(service): import logger from winston configuration
- Removed the import statement for logger from '@librechat/data-schemas' and replaced it with an import from '~/config/winston'.
- This change ensures that the logger is correctly sourced from the updated configuration, improving consistency in logging practices across the application.
refactor(summary): simplify Summary component and enhance token display
- Removed the unused `meta` prop from the `SummaryButton` component to streamline its interface.
- Updated the token display logic to use a localized string for better internationalization support.
- Adjusted the rendering of the `meta` information to improve its visibility within the `Summary` component.
These changes enhance the clarity and usability of the Summary component while ensuring better localization practices.
feat(summarization): add maxInputTokens configuration for summarization
- Introduced a new `maxInputTokens` property in the summarization configuration schema to control the amount of conversation context sent to the summarizer, with a default value of 10000.
- Updated the `createRun` function to utilize the new `maxInputTokens` setting, allowing for more flexible summarization based on agent context.
These changes enhance the summarization capabilities by providing better control over input token limits, improving the overall summarization process.
refactor(summarization): simplify maxInputTokens logic in createRun function
- Updated the logic for the `maxInputTokens` property in the `createRun` function to directly use the agent's base context tokens when the resolved summarization configuration does not specify a value.
- This change streamlines the configuration process and enhances clarity in how input token limits are determined for summarization.
These modifications improve the maintainability of the summarization configuration by reducing complexity in the token calculation logic.
feat(summary): enhance Summary component to display meta information
- Updated the SummaryContent component to accept an optional `meta` prop, allowing for additional contextual information to be displayed above the main content.
- Adjusted the rendering logic in the Summary component to utilize the new `meta` prop, improving the visibility of supplementary details.
These changes enhance the user experience by providing more context within the Summary component, making it clearer and more informative.
refactor(summarization): standardize reserveRatio configuration in summarization logic
- Replaced instances of `reserveTokensRatio` with `reserveRatio` in the `createRun` function and related tests to unify the terminology across the codebase.
- Updated the summarization configuration schema to reflect this change, ensuring consistency in how the reserve ratio is defined and utilized.
- Removed the per-agent override logic for summarization configuration, simplifying the overall structure and enhancing clarity.
These modifications improve the maintainability and readability of the summarization logic by standardizing the configuration parameters.
* fix: circular dependency of `~/models`
* chore: update logging scope in agent log handlers
Changed log scope from `[agentus:${data.scope}]` to `[agents:${data.scope}]` in both the callbacks and responses controllers to ensure consistent logging format across the application.
* feat: calibration ratio
* refactor(tests): update summarizationConfig tests to reflect changes in enabled property
Modified tests to check for the new `summarizationEnabled` property instead of the deprecated `enabled` field in the summarization configuration. This change ensures that the tests accurately validate the current configuration structure and behavior of the agents.
* feat(tests): add markSummarizationUsage mock for improved test coverage
Introduced a mock for the markSummarizationUsage function in the responses unit tests to enhance the testing of summarization usage tracking. This addition supports better validation of summarization-related functionalities and ensures comprehensive test coverage for the agents' response handling.
* refactor(tests): simplify event handler setup in createResponse tests
Removed redundant mock implementations for event handlers in the createResponse unit tests, streamlining the setup process. This change enhances test clarity and maintainability while ensuring that the tests continue to validate the correct behavior of usage tracking during on_chat_model_end events.
* refactor(agents): move calibration ratio capture to finally block
Reorganized the logic for capturing the calibration ratio in the AgentClient class to ensure it is executed in the finally block. This change guarantees that the ratio is captured even if the run is aborted, enhancing the reliability of the response message persistence. Removed redundant code and improved clarity in the handling of context metadata.
* refactor(agents): streamline bulk write logic in recordCollectedUsage function
Removed redundant bulk write operations and consolidated document handling in the recordCollectedUsage function. The logic now combines all documents into a single bulk write operation, improving efficiency and reducing error handling complexity. Updated logging to provide consistent error messages for bulk write failures.
* refactor(agents): enhance summarization configuration resolution in createRun function
Streamlined the summarization configuration logic by introducing a base configuration and allowing for overrides from agent-specific settings. This change improves clarity and maintainability, ensuring that the summarization configuration is consistently applied while retaining flexibility for customization. Updated the handling of summarization parameters to ensure proper integration with the agent's model and provider settings.
* refactor(agents): remove unused tokenCountMap and streamline calibration ratio handling
Eliminated the unused tokenCountMap variable from the AgentClient class to enhance code clarity. Additionally, streamlined the logic for capturing the calibration ratio by using optional chaining and a fallback value, ensuring that context metadata is consistently defined. This change improves maintainability and reduces potential confusion in the codebase.
* refactor(agents): extract agent log handler for improved clarity and reusability
Refactored the agent log handling logic by extracting it into a dedicated function, `agentLogHandler`, enhancing code clarity and reusability across different modules. Updated the event handlers in both the OpenAI and responses controllers to utilize the new handler, ensuring consistent logging behavior throughout the application.
* test: add summarization event tests for useStepHandler
Implemented a series of tests for the summarization events in the useStepHandler hook. The tests cover scenarios for ON_SUMMARIZE_START, ON_SUMMARIZE_DELTA, and ON_SUMMARIZE_COMPLETE events, ensuring proper handling of summarization logic, including message accumulation and finalization. This addition enhances test coverage and validates the correct behavior of the summarization process within the application.
* refactor(config): update summarizationTriggerSchema to use enum for type validation
Changed the type of the `type` field in the summarizationTriggerSchema from a string to an enum with a single value 'token_count'. This modification enhances type safety and ensures that only valid types are accepted in the configuration, improving overall clarity and maintainability of the schema.
* test(usage): add bulk write tests for message and summarization usage
Implemented tests for the bulk write functionality in the recordCollectedUsage function, covering scenarios for combined message and summarization usage, summarization-only usage, and message-only usage. These tests ensure correct document handling and token rollup calculations, enhancing test coverage and validating the behavior of the usage tracking logic.
* refactor(Chat): enhance clipboard copy functionality and type definitions in Summary component
Updated the Summary component to improve the clipboard copy functionality by handling clipboard permission errors. Refactored type definitions for SummaryProps to use a more specific type, enhancing type safety. Adjusted the SummaryButton and FloatingSummaryBar components to accept isCopied and onCopy props, promoting better separation of concerns and reusability.
* chore(translations): remove unused "Expand Summary" key from English translations
Deleted the "Expand Summary" key from the English translation file to streamline the localization resources and improve clarity in the user interface. This change helps maintain an organized and efficient translation structure.
* refactor: adjust token counting for Claude model to account for API discrepancies
Implemented a correction factor for token counting when using the Claude model, addressing discrepancies between Anthropic's API and local tokenizer results. This change ensures accurate token counts by applying a scaling factor, improving the reliability of token-related functionalities.
* refactor(agents): implement token count adjustment for Claude model messages
Added a method to adjust token counts for messages processed by the Claude model, applying a correction factor to align with API expectations. This enhancement improves the accuracy of token counting, ensuring reliable functionality when interacting with the Claude model.
* refactor(agents): token counting for media content in messages
Introduced a new method to estimate token costs for image and document blocks in messages, improving the accuracy of token counting. This enhancement ensures that media content is properly accounted for, particularly for the Claude model, by integrating additional token estimation logic for various content types. Updated the token counting function to utilize this new method, enhancing overall reliability and functionality.
* chore: fix missing import
* fix(agents): clamp baseContextTokens and document reserve ratio change
Prevent negative baseContextTokens when maxOutputTokens exceeds the
context window (misconfigured models). Document the 10%→5% default
reserve ratio reduction introduced alongside summarization.
* fix(agents): include media tokens in hydrated token counts
Add estimateMediaTokensForMessage to createTokenCounter so the hydration
path (used by hydrateMissingIndexTokenCounts) matches the precomputed
path in AgentClient.getTokenCountForMessage. Without this, messages
containing images or documents were systematically undercounted during
hydration, risking context window overflow.
Add 34 unit tests covering all block-type branches of
estimateMediaTokensForMessage.
* fix(agents): include summarization output tokens in usage return value
The returned output_tokens from recordCollectedUsage now reflects all
billed LLM calls (message + summarization). Previously, summarization
completions were billed but excluded from the returned metadata, causing
a discrepancy between what users were charged and what the response
message reported.
* fix(tests): replace process.exit with proper Redis cleanup in e2e test
The summarization E2E test used process.exit(0) to work around a Redis
connection opened at import time, which killed the Jest runner and
bypassed teardown. Use ioredisClient.quit() and keyvRedisClient.disconnect()
for graceful cleanup instead.
* fix(tests): update getConvo imports in OpenAI and response tests
Refactor test files to import getConvo from the main models module instead of the Conversation submodule. This change ensures consistency across tests and simplifies the import structure, enhancing maintainability.
* fix(clients): improve summary text validation in BaseClient
Refactor the summary extraction logic to ensure that only non-empty summary texts are considered valid. This change enhances the robustness of the message processing by utilizing a dedicated method for summary text retrieval, improving overall reliability.
* fix(config): replace z.any() with explicit union in summarization schema
Model parameters (temperature, top_p, etc.) are constrained to
primitive types rather than the policy-violating z.any().
* refactor(agents): deduplicate CLAUDE_TOKEN_CORRECTION constant
Export from the TS source in packages/api and import in the JS client,
eliminating the static class property that could drift out of sync.
* refactor(agents): eliminate duplicate selfProvider in buildAgentContext
selfProvider and provider were derived from the same expression with
different type casts. Consolidated to a single provider variable.
* refactor(agents): extract shared SSE handlers and restrict log levels
- buildSummarizationHandlers() factory replaces triplicated handler
blocks across responses.js and openai.js
- agentLogHandlerObj exported from callbacks.js for consistent reuse
- agentLogHandler restricted to an allowlist of safe log levels
(debug, info, warn, error) instead of accepting arbitrary strings
* fix(SSE): batch summarize deltas, add exhaustiveness check, conditional error announcement
- ON_SUMMARIZE_DELTA coalesces rapid-fire renders via requestAnimationFrame
instead of calling setMessages per chunk
- Exhaustive never-check on TStepEvent catches unhandled variants at
compile time when new StepEvents are added
- ON_SUMMARIZE_COMPLETE error announcement only fires when a summary
part was actually present and removed
* feat(agents): persist instruction overhead in contextMeta and seed across runs
Extend contextMeta with instructionOverhead and toolCount so the
provider-observed instruction overhead is persisted on the response message
and seeded into the pruner on subsequent runs. This enables the pruner to
use a calibrated budget from the first call instead of waiting for a
provider observation, preventing the ratio collapse caused by local
tokenizer overestimating tool schema tokens.
The seeded overhead is only used when encoding and tool count match
between runs, ensuring stale values from different configurations
are discarded.
* test(agents): enhance OpenAI test mocks for summarization handlers
Updated the OpenAI test suite to include additional mock implementations for summarization handlers, including buildSummarizationHandlers, markSummarizationUsage, and agentLogHandlerObj. This improves test coverage and ensures consistent behavior during testing.
* fix(agents): address review findings for summarization v2
Cancel rAF on unmount to prevent stale Recoil writes from dead
component context. Clear orphaned summarizing:true parts when
ON_SUMMARIZE_COMPLETE arrives without a summary payload. Add null
guard and safe spread to agentLogHandler. Handle Anthropic-format
base64 image/* documents in estimateMediaTokensForMessage. Use
role="region" for expandable summary content. Add .describe() to
contextMeta Zod fields. Extract duplicate usage loop into helper.
* refactor: simplify contextMeta to calibrationRatio + encoding only
Remove instructionOverhead and toolCount from cross-run persistence —
instruction tokens change too frequently between runs (prompt edits,
tool changes) for a persisted seed to be reliable. The intra-run
calibration in the pruner still self-corrects via provider observations.
contextMeta now stores only the tokenizer-bias ratio and encoding,
which are stable across instruction changes.
* test(SSE): enhance useStepHandler tests for ON_SUMMARIZE_COMPLETE behavior
Updated the test for ON_SUMMARIZE_COMPLETE to clarify that it finalizes the existing part with summarizing set to false when the summary is undefined. Added assertions to verify the correct behavior of message updates and the state of summary parts.
* refactor(BaseClient): remove handleContextStrategy and truncateToolCallOutputs functions
Eliminated the handleContextStrategy method from BaseClient to streamline message handling. Also removed the truncateToolCallOutputs function from the prompts module, simplifying the codebase and improving maintainability.
* refactor: add AGENT_DEBUG_LOGGING option and refactor token count handling in BaseClient
Introduced AGENT_DEBUG_LOGGING to .env.example for enhanced debugging capabilities. Refactored token count handling in BaseClient by removing the handleTokenCountMap method and simplifying token count updates. Updated AgentClient to log detailed token count recalculations and adjustments, improving traceability during message processing.
* chore: update dependencies in package-lock.json and package.json files
Bumped versions of several dependencies, including @librechat/agents to ^3.1.62 and various AWS SDK packages to their latest versions. This ensures compatibility and incorporates the latest features and fixes.
* chore: imports order
* refactor: extract summarization config resolution from buildAgentContext
* refactor: rename and simplify summarization configuration shaping function
* refactor: replace AgentClient token counting methods with single-pass pure utility
Extract getTokenCount() and getTokenCountForMessage() from AgentClient
into countFormattedMessageTokens(), a pure function in packages/api that
handles text, tool_call, image, and document content types in one loop.
- Decompose estimateMediaTokensForMessage into block-level helpers
(estimateImageDataTokens, estimateImageBlockTokens, estimateDocumentBlockTokens)
shared by both estimateMediaTokensForMessage and the new single-pass function
- Remove redundant per-call getEncoding() resolution (closure captures once)
- Remove deprecated gpt-3.5-turbo-0301 model branching
- Drop this.getTokenCount guard from BaseClient.sendMessage
* refactor: streamline token counting in createTokenCounter function
Simplified the createTokenCounter function by removing the media token estimation and directly calculating the token count. This change enhances clarity and performance by consolidating the token counting logic into a single pass, while maintaining compatibility with Claude's token correction.
* refactor: simplify summarization configuration types
Removed the AppSummarizationConfig type and directly used SummarizationConfig in the AppConfig interface. This change streamlines the type definitions and enhances consistency across the codebase.
* chore: import order
* fix: summarization event handling in useStepHandler
- Cancel pending summarizeDeltaRaf in clearStepMaps to prevent stale
frames firing after map reset or component unmount
- Move announcePolite('summarize_completed') inside the didFinalize
guard so screen readers only announce when finalization actually occurs
- Remove dead cleanup closure returned from stepHandler useCallback body
that was never invoked by any caller
* fix: estimate tokens for non-PDF/non-image base64 document blocks
Previously estimateDocumentBlockTokens returned 0 for unrecognized MIME
types (e.g. text/plain, application/json), silently underestimating
context budget. Fall back to character-based heuristic or countTokens.
* refactor: return cloned usage from markSummarizationUsage
Avoid mutating LangChain's internal usage_metadata object by returning
a shallow clone with the usage_type tag. Update all call sites in
callbacks, openai, and responses controllers to use the returned value.
* refactor: consolidate debug logging loops in buildMessages
Merge the two sequential O(n) debug-logging passes over orderedMessages
into a single pass inside the map callback where all data is available.
* refactor: narrow SummaryContentPart.content type
Replace broad Agents.MessageContentComplex[] with the specific
Array<{ type: ContentTypes.TEXT; text: string }> that all producers
and consumers already use, improving compile-time safety.
* refactor: use single output array in recordCollectedUsage
Have processUsageGroup append to a shared array instead of returning
separate arrays that are spread into a third, reducing allocations.
* refactor: use for...in in hydrateMissingIndexTokenCounts
Replace Object.entries with for...in to avoid allocating an
intermediate tuple array during token map hydration.
2026-03-21 12:03:10 -04:00
|
|
|
"@librechat/agents": "^3.1.62",
|
🧠 feat: User Memories for Conversational Context (#7760)
* 🧠 feat: User Memories for Conversational Context
chore: mcp typing, use `t`
WIP: first pass, Memories UI
- Added MemoryViewer component for displaying, editing, and deleting user memories.
- Integrated data provider hooks for fetching, updating, and deleting memories.
- Implemented pagination and loading states for better user experience.
- Created unit tests for MemoryViewer to ensure functionality and interaction with data provider.
- Updated translation files to include new UI strings related to memories.
chore: move mcp-related files to own directory
chore: rename librechat-mcp to librechat-api
WIP: first pass, memory processing and data schemas
chore: linting in fileSearch.js query description
chore: rename librechat-api to @librechat/api across the project
WIP: first pass, functional memory agent
feat: add MemoryEditDialog and MemoryViewer components for managing user memories
- Introduced MemoryEditDialog for editing memory entries with validation and toast notifications.
- Updated MemoryViewer to support editing and deleting memories, including pagination and loading states.
- Enhanced data provider to handle memory updates with optional original key for better management.
- Added new localization strings for memory-related UI elements.
feat: add memory permissions management
- Implemented memory permissions in the backend, allowing roles to have specific permissions for using, creating, updating, and reading memories.
- Added new API endpoints for updating memory permissions associated with roles.
- Created a new AdminSettings component for managing memory permissions in the frontend.
- Integrated memory permissions into the existing roles and permissions schemas.
- Updated the interface to include memory settings and permissions.
- Enhanced the MemoryViewer component to conditionally render admin settings based on user roles.
- Added localization support for memory permissions in the translation files.
feat: move AdminSettings component to a new position in MemoryViewer for better visibility
refactor: clean up commented code in MemoryViewer component
feat: enhance MemoryViewer with search functionality and improve MemoryEditDialog integration
- Added a search input to filter memories in the MemoryViewer component.
- Refactored MemoryEditDialog to accept children for better customization.
- Updated MemoryViewer to utilize the new EditMemoryButton and DeleteMemoryButton components for editing and deleting memories.
- Improved localization support by adding new strings for memory filtering and deletion confirmation.
refactor: optimize memory filtering in MemoryViewer using match-sorter
- Replaced manual filtering logic with match-sorter for improved search functionality.
- Enhanced performance and readability of the filteredMemories computation.
feat: enhance MemoryEditDialog with triggerRef and improve updateMemory mutation handling
feat: implement access control for MemoryEditDialog and MemoryViewer components
refactor: remove commented out code and create runMemory method
refactor: rename role based files
feat: implement access control for memory usage in AgentClient
refactor: simplify checkVisionRequest method in AgentClient by removing commented-out code
refactor: make `agents` dir in api package
refactor: migrate Azure utilities to TypeScript and consolidate imports
refactor: move sanitizeFilename function to a new file and update imports, add related tests
refactor: update LLM configuration types and consolidate Azure options in the API package
chore: linting
chore: import order
refactor: replace getLLMConfig with getOpenAIConfig and remove unused LLM configuration file
chore: update winston-daily-rotate-file to version 5.0.0 and add object-hash dependency in package-lock.json
refactor: move primeResources and optionalChainWithEmptyCheck functions to resources.ts and update imports
refactor: move createRun function to a new run.ts file and update related imports
fix: ensure safeAttachments is correctly typed as an array of TFile
chore: add node-fetch dependency and refactor fetch-related functions into packages/api/utils, removing the old generators file
refactor: enhance TEndpointOption type by using Pick to streamline endpoint fields and add new properties for model parameters and client options
feat: implement initializeOpenAIOptions function and update OpenAI types for enhanced configuration handling
fix: update types due to new TEndpointOption typing
fix: ensure safe access to group parameters in initializeOpenAIOptions function
fix: remove redundant API key validation comment in initializeOpenAIOptions function
refactor: rename initializeOpenAIOptions to initializeOpenAI for consistency and update related documentation
refactor: decouple req.body fields and tool loading from initializeAgentOptions
chore: linting
refactor: adjust column widths in MemoryViewer for improved layout
refactor: simplify agent initialization by creating loadAgent function and removing unused code
feat: add memory configuration loading and validation functions
WIP: first pass, memory processing with config
feat: implement memory callback and artifact handling
feat: implement memory artifacts display and processing updates
feat: add memory configuration options and schema validation for validKeys
fix: update MemoryEditDialog and MemoryViewer to handle memory state and display improvements
refactor: remove padding from BookmarkTable and MemoryViewer headers for consistent styling
WIP: initial tokenLimit config and move Tokenizer to @librechat/api
refactor: update mongoMeili plugin methods to use callback for better error handling
feat: enhance memory management with token tracking and usage metrics
- Added token counting for memory entries to enforce limits and provide usage statistics.
- Updated memory retrieval and update routes to include total token usage and limit.
- Enhanced MemoryEditDialog and MemoryViewer components to display memory usage and token information.
- Refactored memory processing functions to handle token limits and provide feedback on memory capacity.
feat: implement memory artifact handling in attachment handler
- Enhanced useAttachmentHandler to process memory artifacts when receiving updates.
- Introduced handleMemoryArtifact utility to manage memory updates and deletions.
- Updated query client to reflect changes in memory state based on incoming data.
refactor: restructure web search key extraction logic
- Moved the logic for extracting API keys from the webSearchAuth configuration into a dedicated function, getWebSearchKeys.
- Updated webSearchKeys to utilize the new function for improved clarity and maintainability.
- Prevents build time errors
feat: add personalization settings and memory preferences management
- Introduced a new Personalization tab in settings to manage user memory preferences.
- Implemented API endpoints and client-side logic for updating memory preferences.
- Enhanced user interface components to reflect personalization options and memory usage.
- Updated permissions to allow users to opt out of memory features.
- Added localization support for new settings and messages related to personalization.
style: personalization switch class
feat: add PersonalizationIcon and align Side Panel UI
feat: implement memory creation functionality
- Added a new API endpoint for creating memory entries, including validation for key and value.
- Introduced MemoryCreateDialog component for user interface to facilitate memory creation.
- Integrated token limit checks to prevent exceeding user memory capacity.
- Updated MemoryViewer to include a button for opening the memory creation dialog.
- Enhanced localization support for new messages related to memory creation.
feat: enhance message processing with configurable window size
- Updated AgentClient to use a configurable message window size for processing messages.
- Introduced messageWindowSize option in memory configuration schema with a default value of 5.
- Improved logic for selecting messages to process based on the configured window size.
chore: update librechat-data-provider version to 0.7.87 in package.json and package-lock.json
chore: remove OpenAPIPlugin and its associated tests
chore: remove MIGRATION_README.md as migration tasks are completed
ci: fix backend tests
chore: remove unused translation keys from localization file
chore: remove problematic test file and unused var in AgentClient
chore: remove unused import and import directly for JSDoc
* feat: add api package build stage in Dockerfile for improved modularity
* docs: reorder build steps in contributing guide for clarity
2025-06-07 18:52:22 -04:00
|
|
|
"@librechat/api": "*",
|
2025-03-07 12:45:31 -05:00
|
|
|
"@librechat/data-schemas": "*",
|
2025-08-15 15:29:16 -04:00
|
|
|
"@microsoft/microsoft-graph-client": "^3.0.7",
|
2026-02-24 21:10:34 -05:00
|
|
|
"@modelcontextprotocol/sdk": "^1.27.1",
|
2025-07-25 11:26:20 -04:00
|
|
|
"@node-saml/passport-saml": "^5.1.0",
|
2025-12-01 12:23:46 -05:00
|
|
|
"@smithy/node-http-handler": "^4.4.5",
|
⚡ refactor: Replace `tiktoken` with `ai-tokenizer` (#12175)
* chore: Update dependencies by adding ai-tokenizer and removing tiktoken
- Added ai-tokenizer version 1.0.6 to package.json and package-lock.json across multiple packages.
- Removed tiktoken version 1.0.15 from package.json and package-lock.json in the same locations, streamlining dependency management.
* refactor: replace js-tiktoken with ai-tokenizer
- Added support for 'claude' encoding in the AgentClient class to improve model compatibility.
- Updated Tokenizer class to utilize 'ai-tokenizer' for both 'o200k_base' and 'claude' encodings, replacing the previous 'tiktoken' dependency.
- Refactored tests to reflect changes in tokenizer behavior and ensure accurate token counting for both encoding types.
- Removed deprecated references to 'tiktoken' and adjusted related tests for improved clarity and functionality.
* chore: remove tiktoken mocks from DALLE3 tests
- Eliminated mock implementations of 'tiktoken' from DALLE3-related test files to streamline test setup and align with recent dependency updates.
- Adjusted related test structures to ensure compatibility with the new tokenizer implementation.
* chore: Add distinct encoding support for Anthropic Claude models
- Introduced a new method `getEncoding` in the AgentClient class to handle the specific BPE tokenizer for Claude models, ensuring compatibility with the distinct encoding requirements.
- Updated documentation to clarify the encoding logic for Claude and other models.
* docs: Update return type documentation for getEncoding method in AgentClient
- Clarified the return type of the getEncoding method to specify that it can return an EncodingName or undefined, enhancing code readability and type safety.
* refactor: Tokenizer class and error handling
- Exported the EncodingName type for broader usage.
- Renamed encodingMap to encodingData for clarity.
- Improved error handling in getTokenCount method to ensure recovery attempts are logged and return 0 on failure.
- Updated countTokens function documentation to specify the use of 'o200k_base' encoding.
* refactor: Simplify encoding documentation and export type
- Updated the getEncoding method documentation to clarify the default behavior for non-Anthropic Claude models.
- Exported the EncodingName type separately from the Tokenizer module for improved clarity and usage.
* test: Update text processing tests for token limits
- Adjusted test cases to handle smaller text sizes, changing scenarios from ~120k tokens to ~20k tokens for both the real tokenizer and countTokens functions.
- Updated token limits in tests to reflect new constraints, ensuring tests accurately assess performance and call reduction.
- Enhanced console log messages for clarity regarding token counts and reductions in the updated scenarios.
* refactor: Update Tokenizer imports and exports
- Moved Tokenizer and countTokens exports to the tokenizer module for better organization.
- Adjusted imports in memory.ts to reflect the new structure, ensuring consistent usage across the codebase.
- Updated memory.test.ts to mock the Tokenizer from the correct module path, enhancing test accuracy.
* refactor: Tokenizer initialization and error handling
- Introduced an async `initEncoding` method to preload tokenizers, improving performance and accuracy in token counting.
- Updated `getTokenCount` to handle uninitialized tokenizers more gracefully, ensuring proper recovery and logging on errors.
- Removed deprecated synchronous tokenizer retrieval, streamlining the overall tokenizer management process.
* test: Enhance tokenizer tests with initialization and encoding checks
- Added `beforeAll` hooks to initialize tokenizers for 'o200k_base' and 'claude' encodings before running tests, ensuring proper setup.
- Updated tests to validate the loading of encodings and the correctness of token counts for both 'o200k_base' and 'claude'.
- Improved test structure to deduplicate concurrent initialization calls, enhancing performance and reliability.
2026-03-10 23:14:52 -04:00
|
|
|
"ai-tokenizer": "^1.0.6",
|
2026-02-10 20:03:17 -05:00
|
|
|
"axios": "^1.13.5",
|
feat: Auth and User System (#205)
* server-side JWT auth implementation
* move oauth routes and strategies, fix bugs
* backend modifications for wiring up the frontend login and reg forms
* Add frontend data services for login and registration
* Add login and registration forms
* Implment auth context, functional client side auth
* protect routes with jwt auth
* finish local strategy (using local storage)
* Start setting up google auth
* disable token refresh, remove old auth middleware
* refactor client, add ApiErrorBoundary context
* disable google and facebook strategies
* fix: fix presets not displaying specific to user
* fix: fix issue with browser refresh
* fix: casing issue with User.js (#11)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* fix: issue with api error watcher (#12)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* feat: add google auth social login
* fix: make google login url dynamic based on dev/prod
* fix: bug where UI is briefly displayed before redirecting to login
* fix: fix cookie expires value for local auth
* Update README.md
* Update LOCAL_INSTALL structure
* Add local testing instructions
* Only load google strategy if client id and secret are provided
* Update .env.example files with new params
* fix issue with not redirecting to register form
* only show google login button if value is set in .env
* cleanup log messages
* Add label to button for google login on login form
* doc: fix client/server url values in .env.example
* feat: add error message details to registration failure
* Restore preventing paste on confirm password
* auto-login user after registering
* feat: forgot password (#24)
* make login/reg pages look like openai's
* add password reset data services
* new form designs similar to openai, add password reset pages
* add api's for password reset
* email utils for password reset
* remove bcrypt salt rounds from process.env
* refactor: restructure api auth code, consolidate routes (#25)
* add api's for password reset
* remove bcrypt salt rounds from process.env
* refactor: consolidate auth routes, use controller pattern
* refactor: code cleanup
* feat: migrate data to first user (#26)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes after refactor (#27)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: issue with auto-login when logging out then logging in with new browser window (#28)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: Update README and .env.example files with user system information (#29)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: update README and .env.example files
* Fixup: LOCAL_INSTALL.md PS instructions (#200) (#30)
Co-authored-by: alfredo-f <alfredo.fomitchenko@mail.polimi.it>
* feat: send user with completion to protect against abuse (#31)
* Fixup: LOCAL_INSTALL.md PS instructions (#200)
* server-side JWT auth implementation
* move oauth routes and strategies, fix bugs
* backend modifications for wiring up the frontend login and reg forms
* Add frontend data services for login and registration
* Add login and registration forms
* Implment auth context, functional client side auth
* protect routes with jwt auth
* finish local strategy (using local storage)
* Start setting up google auth
* disable token refresh, remove old auth middleware
* refactor client, add ApiErrorBoundary context
* disable google and facebook strategies
* fix: fix presets not displaying specific to user
* fix: fix issue with browser refresh
* fix: casing issue with User.js (#11)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* feat: add google auth social login
* fix: make google login url dynamic based on dev/prod
* fix: bug where UI is briefly displayed before redirecting to login
* fix: fix cookie expires value for local auth
* Only load google strategy if client id and secret are provided
* Update .env.example files with new params
* fix issue with not redirecting to register form
* only show google login button if value is set in .env
* cleanup log messages
* Add label to button for google login on login form
* doc: fix client/server url values in .env.example
* feat: add error message details to registration failure
* Restore preventing paste on confirm password
* auto-login user after registering
* feat: forgot password (#24)
* make login/reg pages look like openai's
* add password reset data services
* new form designs similar to openai, add password reset pages
* add api's for password reset
* email utils for password reset
* remove bcrypt salt rounds from process.env
* refactor: restructure api auth code, consolidate routes (#25)
* add api's for password reset
* remove bcrypt salt rounds from process.env
* refactor: consolidate auth routes, use controller pattern
* refactor: code cleanup
* feat: migrate data to first user (#26)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes after refactor (#27)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: issue with auto-login when logging out then logging in with new browser window (#28)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: Update README and .env.example files with user system information (#29)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: update README and .env.example files
* Send user id to openai to protect against abuse
* add meilisearch to gitignore
* Remove webpack
---------
Co-authored-by: alfredo-f <alfredo.fomitchenko@mail.polimi.it>
---------
Co-authored-by: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Co-authored-by: Alfredo Fomitchenko <alfredo.fomitchenko@mail.polimi.it>
2023-05-07 10:04:51 -07:00
|
|
|
"bcryptjs": "^2.4.3",
|
2025-07-19 13:36:59 -04:00
|
|
|
"compression": "^1.8.1",
|
2025-10-04 16:43:22 -04:00
|
|
|
"connect-redis": "^8.1.0",
|
2024-10-15 20:05:22 -04:00
|
|
|
"cookie": "^0.7.2",
|
|
|
|
|
"cookie-parser": "^1.4.7",
|
2023-03-06 10:15:07 -05:00
|
|
|
"cors": "^2.8.5",
|
2024-08-27 17:03:16 -04:00
|
|
|
"dedent": "^1.5.3",
|
2023-03-06 10:15:07 -05:00
|
|
|
"dotenv": "^16.0.3",
|
2025-02-24 20:08:55 -05:00
|
|
|
"eventsource": "^3.0.2",
|
2025-12-30 18:41:32 -05:00
|
|
|
"express": "^5.2.1",
|
refactor: Encrypt & Expire User Provided Keys, feat: Rate Limiting (#874)
* docs: make_your_own.md formatting fix for mkdocs
* feat: add express-mongo-sanitize
feat: add login/registration rate limiting
* chore: remove unnecessary console log
* wip: remove token handling from localStorage to encrypted DB solution
* refactor: minor change to UserService
* fix mongo query and add keys route to server
* fix backend controllers and simplify schema/crud
* refactor: rename token to key to separate from access/refresh tokens, setTokenDialog -> setKeyDialog
* refactor(schemas): TEndpointOption token -> key
* refactor(api): use new encrypted key retrieval system
* fix(SetKeyDialog): fix key prop error
* fix(abortMiddleware): pass random UUID if messageId is not generated yet for proper error display on frontend
* fix(getUserKey): wrong prop passed in arg, adds error handling
* fix: prevent message without conversationId from saving to DB, prevents branching on the frontend to a new top-level branch
* refactor: change wording of multiple display messages
* refactor(checkExpiry -> checkUserKeyExpiry): move to UserService file
* fix: type imports from common
* refactor(SubmitButton): convert to TS
* refactor(key.ts): change localStorage map key name
* refactor: add new custom tailwind classes to better match openAI colors
* chore: remove unnecessary warning and catch ScreenShot error
* refactor: move userKey frontend logic to hooks and remove use of localStorage and instead query the DB
* refactor: invalidate correct query key, memoize userKey hook, conditionally render SetKeyDialog to avoid unnecessary calls, refactor SubmitButton props and useEffect for showing 'provide key first'
* fix(SetKeyDialog): use enum-like object for expiry values
feat(Dropdown): add optionsClassName to dynamically change dropdown options container classes
* fix: handle edge case where user had provided a key but the server changes to env variable for keys
* refactor(OpenAI/titleConvo): move titling to client to retain authorized credentials in message lifecycle for titling
* fix(azure): handle user_provided keys correctly for azure
* feat: send user Id to OpenAI to differentiate users in completion requests
* refactor(OpenAI/titleConvo): adding tokens helps minimize LLM from using the language in title response
* feat: add delete endpoint for keys
* chore: remove throttling of title
* feat: add 'Data controls' to Settings, add 'Revoke' keys feature in Key Dialog and Data controls
* refactor: reorganize PluginsClient files in langchain format
* feat: use langchain for titling convos
* chore: cleanup titling convo, with fallback to original method, escape braces, use only snippet for language detection
* refactor: move helper functions to appropriate langchain folders for reusability
* fix: userProvidesKey handling for gptPlugins
* fix: frontend handling of plugins key
* chore: cleanup logging and ts-ignore SSE
* fix: forwardRef misuse in DangerButton
* fix(GoogleConfig/FileUpload): localize errors and simplify validation with zod
* fix: cleanup google logging and fix user provided key handling
* chore: remove titling from google
* chore: removing logging from browser endpoint
* wip: fix menu flicker
* feat: useLocalStorage hook
* feat: add Tooltip for UI
* refactor(EndpointMenu): utilize Tooltip and useLocalStorage, remove old 'New Chat' slide-over
* fix(e2e): use testId for endpoint menu trigger
* chore: final touches to EndpointMenu before future refactor to declutter component
* refactor(localization): change select endpoint to open menu and add translations
* chore: add final prop to error message response
* ci: minor edits to facilitate testing
* ci: new e2e test which tests for new key setting/revoking features
2023-09-06 10:46:27 -04:00
|
|
|
"express-mongo-sanitize": "^2.2.0",
|
2026-03-06 19:18:35 -05:00
|
|
|
"express-rate-limit": "^8.3.0",
|
2025-07-19 13:36:59 -04:00
|
|
|
"express-session": "^1.18.2",
|
2025-02-22 23:42:20 +01:00
|
|
|
"express-static-gzip": "^2.2.0",
|
2026-03-14 03:36:03 -04:00
|
|
|
"file-type": "^21.3.2",
|
2024-11-22 16:35:23 -05:00
|
|
|
"firebase": "^11.0.2",
|
2025-07-21 17:37:37 -04:00
|
|
|
"form-data": "^4.0.4",
|
feat: Auth and User System (#205)
* server-side JWT auth implementation
* move oauth routes and strategies, fix bugs
* backend modifications for wiring up the frontend login and reg forms
* Add frontend data services for login and registration
* Add login and registration forms
* Implment auth context, functional client side auth
* protect routes with jwt auth
* finish local strategy (using local storage)
* Start setting up google auth
* disable token refresh, remove old auth middleware
* refactor client, add ApiErrorBoundary context
* disable google and facebook strategies
* fix: fix presets not displaying specific to user
* fix: fix issue with browser refresh
* fix: casing issue with User.js (#11)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* fix: issue with api error watcher (#12)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* feat: add google auth social login
* fix: make google login url dynamic based on dev/prod
* fix: bug where UI is briefly displayed before redirecting to login
* fix: fix cookie expires value for local auth
* Update README.md
* Update LOCAL_INSTALL structure
* Add local testing instructions
* Only load google strategy if client id and secret are provided
* Update .env.example files with new params
* fix issue with not redirecting to register form
* only show google login button if value is set in .env
* cleanup log messages
* Add label to button for google login on login form
* doc: fix client/server url values in .env.example
* feat: add error message details to registration failure
* Restore preventing paste on confirm password
* auto-login user after registering
* feat: forgot password (#24)
* make login/reg pages look like openai's
* add password reset data services
* new form designs similar to openai, add password reset pages
* add api's for password reset
* email utils for password reset
* remove bcrypt salt rounds from process.env
* refactor: restructure api auth code, consolidate routes (#25)
* add api's for password reset
* remove bcrypt salt rounds from process.env
* refactor: consolidate auth routes, use controller pattern
* refactor: code cleanup
* feat: migrate data to first user (#26)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes after refactor (#27)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: issue with auto-login when logging out then logging in with new browser window (#28)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: Update README and .env.example files with user system information (#29)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: update README and .env.example files
* Fixup: LOCAL_INSTALL.md PS instructions (#200) (#30)
Co-authored-by: alfredo-f <alfredo.fomitchenko@mail.polimi.it>
* feat: send user with completion to protect against abuse (#31)
* Fixup: LOCAL_INSTALL.md PS instructions (#200)
* server-side JWT auth implementation
* move oauth routes and strategies, fix bugs
* backend modifications for wiring up the frontend login and reg forms
* Add frontend data services for login and registration
* Add login and registration forms
* Implment auth context, functional client side auth
* protect routes with jwt auth
* finish local strategy (using local storage)
* Start setting up google auth
* disable token refresh, remove old auth middleware
* refactor client, add ApiErrorBoundary context
* disable google and facebook strategies
* fix: fix presets not displaying specific to user
* fix: fix issue with browser refresh
* fix: casing issue with User.js (#11)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* feat: add google auth social login
* fix: make google login url dynamic based on dev/prod
* fix: bug where UI is briefly displayed before redirecting to login
* fix: fix cookie expires value for local auth
* Only load google strategy if client id and secret are provided
* Update .env.example files with new params
* fix issue with not redirecting to register form
* only show google login button if value is set in .env
* cleanup log messages
* Add label to button for google login on login form
* doc: fix client/server url values in .env.example
* feat: add error message details to registration failure
* Restore preventing paste on confirm password
* auto-login user after registering
* feat: forgot password (#24)
* make login/reg pages look like openai's
* add password reset data services
* new form designs similar to openai, add password reset pages
* add api's for password reset
* email utils for password reset
* remove bcrypt salt rounds from process.env
* refactor: restructure api auth code, consolidate routes (#25)
* add api's for password reset
* remove bcrypt salt rounds from process.env
* refactor: consolidate auth routes, use controller pattern
* refactor: code cleanup
* feat: migrate data to first user (#26)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes after refactor (#27)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: issue with auto-login when logging out then logging in with new browser window (#28)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: Update README and .env.example files with user system information (#29)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: update README and .env.example files
* Send user id to openai to protect against abuse
* add meilisearch to gitignore
* Remove webpack
---------
Co-authored-by: alfredo-f <alfredo.fomitchenko@mail.polimi.it>
---------
Co-authored-by: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Co-authored-by: Alfredo Fomitchenko <alfredo.fomitchenko@mail.polimi.it>
2023-05-07 10:04:51 -07:00
|
|
|
"handlebars": "^4.7.7",
|
2025-02-15 18:52:29 -05:00
|
|
|
"https-proxy-agent": "^7.0.6",
|
2025-10-04 16:43:22 -04:00
|
|
|
"ioredis": "^5.3.2",
|
2025-11-16 11:55:18 -05:00
|
|
|
"js-yaml": "^4.1.1",
|
feat: Auth and User System (#205)
* server-side JWT auth implementation
* move oauth routes and strategies, fix bugs
* backend modifications for wiring up the frontend login and reg forms
* Add frontend data services for login and registration
* Add login and registration forms
* Implment auth context, functional client side auth
* protect routes with jwt auth
* finish local strategy (using local storage)
* Start setting up google auth
* disable token refresh, remove old auth middleware
* refactor client, add ApiErrorBoundary context
* disable google and facebook strategies
* fix: fix presets not displaying specific to user
* fix: fix issue with browser refresh
* fix: casing issue with User.js (#11)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* fix: issue with api error watcher (#12)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* feat: add google auth social login
* fix: make google login url dynamic based on dev/prod
* fix: bug where UI is briefly displayed before redirecting to login
* fix: fix cookie expires value for local auth
* Update README.md
* Update LOCAL_INSTALL structure
* Add local testing instructions
* Only load google strategy if client id and secret are provided
* Update .env.example files with new params
* fix issue with not redirecting to register form
* only show google login button if value is set in .env
* cleanup log messages
* Add label to button for google login on login form
* doc: fix client/server url values in .env.example
* feat: add error message details to registration failure
* Restore preventing paste on confirm password
* auto-login user after registering
* feat: forgot password (#24)
* make login/reg pages look like openai's
* add password reset data services
* new form designs similar to openai, add password reset pages
* add api's for password reset
* email utils for password reset
* remove bcrypt salt rounds from process.env
* refactor: restructure api auth code, consolidate routes (#25)
* add api's for password reset
* remove bcrypt salt rounds from process.env
* refactor: consolidate auth routes, use controller pattern
* refactor: code cleanup
* feat: migrate data to first user (#26)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes after refactor (#27)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: issue with auto-login when logging out then logging in with new browser window (#28)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: Update README and .env.example files with user system information (#29)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: update README and .env.example files
* Fixup: LOCAL_INSTALL.md PS instructions (#200) (#30)
Co-authored-by: alfredo-f <alfredo.fomitchenko@mail.polimi.it>
* feat: send user with completion to protect against abuse (#31)
* Fixup: LOCAL_INSTALL.md PS instructions (#200)
* server-side JWT auth implementation
* move oauth routes and strategies, fix bugs
* backend modifications for wiring up the frontend login and reg forms
* Add frontend data services for login and registration
* Add login and registration forms
* Implment auth context, functional client side auth
* protect routes with jwt auth
* finish local strategy (using local storage)
* Start setting up google auth
* disable token refresh, remove old auth middleware
* refactor client, add ApiErrorBoundary context
* disable google and facebook strategies
* fix: fix presets not displaying specific to user
* fix: fix issue with browser refresh
* fix: casing issue with User.js (#11)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* feat: add google auth social login
* fix: make google login url dynamic based on dev/prod
* fix: bug where UI is briefly displayed before redirecting to login
* fix: fix cookie expires value for local auth
* Only load google strategy if client id and secret are provided
* Update .env.example files with new params
* fix issue with not redirecting to register form
* only show google login button if value is set in .env
* cleanup log messages
* Add label to button for google login on login form
* doc: fix client/server url values in .env.example
* feat: add error message details to registration failure
* Restore preventing paste on confirm password
* auto-login user after registering
* feat: forgot password (#24)
* make login/reg pages look like openai's
* add password reset data services
* new form designs similar to openai, add password reset pages
* add api's for password reset
* email utils for password reset
* remove bcrypt salt rounds from process.env
* refactor: restructure api auth code, consolidate routes (#25)
* add api's for password reset
* remove bcrypt salt rounds from process.env
* refactor: consolidate auth routes, use controller pattern
* refactor: code cleanup
* feat: migrate data to first user (#26)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes after refactor (#27)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: issue with auto-login when logging out then logging in with new browser window (#28)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: Update README and .env.example files with user system information (#29)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: update README and .env.example files
* Send user id to openai to protect against abuse
* add meilisearch to gitignore
* Remove webpack
---------
Co-authored-by: alfredo-f <alfredo.fomitchenko@mail.polimi.it>
---------
Co-authored-by: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Co-authored-by: Alfredo Fomitchenko <alfredo.fomitchenko@mail.polimi.it>
2023-05-07 10:04:51 -07:00
|
|
|
"jsonwebtoken": "^9.0.0",
|
2025-05-22 14:19:24 +02:00
|
|
|
"jwks-rsa": "^3.2.0",
|
2025-10-04 16:43:22 -04:00
|
|
|
"keyv": "^5.3.2",
|
|
|
|
|
"keyv-file": "^5.1.2",
|
2023-12-14 07:49:27 -05:00
|
|
|
"klona": "^2.0.6",
|
2023-12-11 14:48:40 -05:00
|
|
|
"librechat-data-provider": "*",
|
2026-01-22 09:46:08 -05:00
|
|
|
"lodash": "^4.17.23",
|
📄 feat: Local Text Extraction for PDF, DOCX, and XLS/XLSX (#11900)
* feat: Added "document parser" OCR strategy
The document parser uses libraries to parse the text out of known document types.
This lets LibreChat handle some complex document types without having to use a
secondary service (like Mistral or standing up a RAG API server).
To enable the document parser, set the ocr strategy to "document_parser" in
librechat.yaml.
We now support:
- PDFs using pdfjs
- DOCX using mammoth
- XLS/XLSX using SheetJS
(The associated packages were also added to the project.)
* fix: applied Copilot code review suggestions
- Properly calculate length of text based on UTF8.
- Avoid issues with loading / blocking PDF parsing.
* fix: improved docs on parseDocument()
* chore: move to packages/api for TS support
* refactor: make document processing the default ocr strategy
- Introduced support for additional document types in the OCR strategy, including PDF, DOCX, and XLS/XLSX.
- Updated the file upload handling to dynamically select the appropriate parsing strategy based on the file type.
- Refactored the document parsing functions to use asynchronous imports for improved performance and maintainability.
* test: add unit tests for processAgentFileUpload functionality
- Introduced a new test suite for the processAgentFileUpload function in process.spec.js.
- Implemented various test cases to validate OCR strategy selection based on file types, including PDF, DOCX, XLSX, and XLS.
- Mocked dependencies to ensure isolated testing of file upload handling and strategy selection logic.
- Enhanced coverage for scenarios involving OCR capability checks and default strategy fallbacks.
* chore: update pdfjs-dist version and enhance document parsing tests
- Bumped pdfjs-dist dependency to version 5.4.624 in both api and packages/api.
- Refactored document parsing tests to use 'originalname' instead of 'filename' for file objects.
- Added a new test case for parsing XLS files to improve coverage of document types supported by the parser.
- Introduced a sample XLS file for testing purposes.
* feat: enforce text size limit and improve OCR fallback handling in processAgentFileUpload
- Added a check to ensure extracted text does not exceed the 15MB storage limit, throwing an error if it does.
- Refactored the OCR handling logic to improve fallback behavior when the configured OCR fails, ensuring a more robust document processing flow.
- Enhanced unit tests to cover scenarios for oversized text and fallback mechanisms, ensuring proper error handling and functionality.
* fix: correct OCR URL construction in performOCR function
- Updated the OCR URL construction to ensure it correctly appends '/ocr' to the base URL if not already present, improving the reliability of the OCR request.
---------
Co-authored-by: Dan Lew <daniel@mightyacorn.com>
2026-02-22 14:22:45 -05:00
|
|
|
"mammoth": "^1.11.0",
|
2025-12-25 12:25:41 -05:00
|
|
|
"mathjs": "^15.1.0",
|
2024-03-27 10:08:20 -04:00
|
|
|
"meilisearch": "^0.38.0",
|
2025-10-04 16:43:22 -04:00
|
|
|
"memorystore": "^1.6.7",
|
✨ feat: Assistants API, General File Support, Side Panel, File Explorer (#1696)
* feat: assistant name/icon in Landing & Header
* feat: assistname in textarea placeholder, and use `Assistant` as default name
* feat: display non-image files in user messages
* fix: only render files if files.length is > 0
* refactor(config -> file-config): move file related configuration values to separate module, add excel types
* chore: spreadsheet file rendering
* fix(Landing): dark mode style for Assistant Name
* refactor: move progress incrementing to own hook, start smaller, cap near limit \(1\)
* refactor(useContentHandler): add empty Text part if last part was completed tool or image
* chore: add accordion trigger border styling for dark mode
* feat: Assistant Builder model selection
* chore: use Spinner when Assistant is mutating
* fix(get/assistants): return correct response object `AssistantListResponse`
* refactor(Spinner): pass size as prop
* refactor: make assistant crud mutations optimistic, add types for options
* chore: remove assistants route and view
* chore: move assistant builder components to separate directory
* feat(ContextButton): delete Assistant via context button/dialog, add localization
* refactor: conditionally show use and context menu buttons, add localization for create assistant
* feat: save side panel states to localStorage
* style(SidePanel): improve avatar menu and assistant select styling for dark mode
* refactor: make NavToggle reusable for either side (left or right), add SidePanel Toggle with ability to close it completely
* fix: resize handle and navToggle behavior
* fix(/avatar/:assistant_id): await `deleteFile` and assign unique name to uploaded image
* WIP: file UI components from PR #576
* refactor(OpenAIMinimalIcon): pass className
* feat: formatDate helper fn
* feat: DataTableColumnHeader
* feat: add row selection, formatted row values, number of rows selected
* WIP: add files to Side panel temporarily
* feat: `LB_QueueAsyncCall`: Leaky Bucket queue for external APIs, use in `processDeleteRequest`
* fix(TFile): correct `source` type with `FileSources`
* fix(useFileHandling): use `continue` instead of return when iterating multiple files, add file type to extendedFile
* chore: add generic setter type
* refactor(processDeleteRequest): settle promises to prevent rejections from processing deletions, log errors
* feat: `useFileDeletion` to reuse file deletion logic
* refactor(useFileDeletion): make `setFiles` an optional param and use object as param
* feat: useDeleteFilesFromTable
* feat: use real `files` data and add deletion action to data table
* fix(Table): make headers sticky
* feat: add dynamic filtering for columns; only show to user Host or OpenAI storage type
* style(DropdownMenu): replace `slate` with `gray`
* style(DataTable): apply dark mode themes and other misc styling
* style(Columns): add color to OpenAI Storage option
* refactor(FileContainer): make file preview reusable
* refactor(Images): make image preview reusable
* refactor(FilePreview): make file prop optional for FileIcon and FilePreview, fix relative style
* feat(Columns): add file/image previews, set a minimum size to show for file size in bytes
* WIP: File Panel with real files and formatted
* feat: open files dialog from panel
* style: file data table mobile and general column styling fixes
* refactor(api/files): return files sorted by the most recently updated
* refactor: provide fileMap through context to prevent re-selecting files to map in different areas; remove unused imports commented out in PanelColumns
* refactor(ExtendFile): make File type optional, add `attached` to prevent attached files from being deleted on remove, make Message.files a partial TFile type
* feat: attach files through file panel
* refactor(useFileHandling): move files to the start of cache list when uploaded
* refactor(useDeleteFilesMutation): delete files from cache when successfully deleted from server
* fix(FileRow): handle possible edge case of duplication due to attaching recently uploaded file
* style(SidePanel): make resize grip border transparent, remove unnecessary styling on close sidepanel button
* feat: action utilities and tests
* refactor(actions): add `ValidationResult` type and change wording for no server URL found
* refactor(actions): check for empty server URL
* fix(data-provider): revert tsconfig to fix type issue resolution
* feat(client): first pass of actions input for assistants
* refactor(FunctionSignature): change method to output object instead of string
* refactor(models/Assistant): add actions field to schema, use searchParams object for methods, and add `getAssistant`
* feat: post actions input first pass
- create new Action document
- add actions to Assistant DB document
- create /action/:assistant_id POST route
- pass more props down from PanelSwitcher, derive assistant_id from switcher
- move privacy policy to ActionInput
- reset data on input change/validation
- add `useUpdateAction`
- conform FunctionSignature type to FunctionTool
- add action, assistant doc, update hook related types
* refactor: optimize assistant/actions relationship
- past domain in metadata as hostname and not a URL
- include domain in tool name
- add `getActions` for actions retrieval by user
- add `getAssistants` for assistant docs retrieval by user
- add `assistant_id` to Action schema
- move actions to own module as a subroute to `api/assistants`
- add `useGetActionsQuery` and `useGetAssistantDocsQuery` hooks
- fix Action type def
* feat: show assistant actions in assistant builder
* feat: switch to actions on action click, editing action styling
* fix: add Assistant state for builder panel to allow immediate selection of newly created assistants as well as retaining the current assistant when switching to a different panel within the builder
* refactor(SidePanel/NavToggle): offset less from right when SidePanel is completely collapsed
* chore: rename `processActions` -> `processRequiredActions`
* chore: rename Assistant API Action to RequiredAction
* refactor(actions): avoid nesting actual API params under generic `requestBody` to optimize LLM token usage
* fix(handleTools): avoid calling `validTool` if not defined, add optional param to skip the loading of specs, which throws an error in the context of assistants
* WIP: working first pass of toolCalls generated from openapi specs
* WIP: first pass ToolCall styling
* feat: programmatic iv encryption/decryption helpers
* fix: correct ActionAuth types/enums, and define type for AuthForm
* feat: encryption/decryption helpers for Action AuthMetadata
* refactor(getActions): remove sensitive fields from query response
* refactor(POST/actions): encrypt and remove sensitive fields from mutation response
* fix(ActionService): change ESM import to CJS
* feat: frontend auth handling for actions + optimistic update on action update/creation
* refactor(actions): use the correct variables and types for setAuth method
* refactor: POST /:assistant_id action can now handle updating an existing action, add `saved_auth_fields` to determine when user explicitly saves new auth creds. only send auth metadata if user explicitly saved fields
* refactor(createActionTool): catch errors and send back meaningful error message, add flag to `getActions` to determine whether to retrieve sensitive values or not
* refactor(ToolService): add `action` property to ToolCall PartMetadata to determine if the tool call was an action, fix parsing function name issue with actionDelimiter
* fix(ActionRequest): use URL class to correctly join endpoint parts for `execute` call
* feat: delete assistant actions
* refactor: conditionally show Available actions
* refactor: show `retrieval` and `code_interpreter` as Capabilities, swap `Switch` for `Checkbox`
* chore: remove shadow-stroke from messages
* WIP: first pass of Assistants Knowledge attachments
* refactor: remove AssistantsProvider in favor of FormProvider, fix selectedAssistant re-render bug, map Assistant file_ids to files via fileMap, initialize Knowledge component with mapped files if any exist
* fix: prevent deleting files on assistant file upload
* chore: remove console.log
* refactor(useUploadFileMutation): update files and assistants cache on upload
* chore: disable oauth option as not supported yet
* feat: cancel assistant runs
* refactor: initialize OpenAI client with helper function, resolve all related circular dependencies
* fix(DALL-E): initialization
* fix(process): openai client initialization
* fix: select an existing Assistant when the active one is deleted
* chore: allow attaching files for assistant endpoint, send back relevant OpenAI error message when uploading, deconstruct openAI initialization correctly, add `message_file` to formData when a file is attached to the message but not the assistant
* fix: add assistant_id on newConvo
* fix(initializeClient): import fix
* chore: swap setAssistant for setOption in useEffect
* fix(DALL-E): add processFileURL to loadTools call
* chore: add customConfig to debug logs
* feat: delete threads on convo delete
* chore: replace Assistants icon
* chore: remove console.dir() in `abortRun`
* feat(AssistantService): accumulate text values from run in openai.responseText
* feat: titling for assistants endpoint
* chore: move panel file components to appropriate directory, add file checks for attaching files, change icon for Attach Files
* refactor: add localizations to tools, plugins, add condition for adding/remove user plugins so tool selections don't affect this value
* chore: disable `import from url` action for now
* chore: remove textMimeTypes from default fileConfig for now
* fix: catch tool errors and send as outputs with error messages
* fix: React warning about button as descendant of button
* style: retrieval and cancelled icon
* WIP: pass isSubmitting to Parts, use InProgressCall to display cancelled tool calls correctly, show domain/function name
* fix(meilisearch): fix `postSaveHook` issue where indexing expects a mongo document, and join all text content parts for meili indexing
* ci: fix dall-e tests
* ci: fix client tests
* fix: button types in actions panel
* fix: plugin auth form persisting across tool selections
* fix(ci): update AppService spec with `loadAndFormatTools`
* fix(clearConvos): add id check earlier on
* refactor(AssistantAvatar): set previewURL dynamically when emtadata.avatar changes
* feat(assistants): addTitle cache setting
* fix(useSSE): resolve rebase conflicts
* fix: delete mutation
* style(SidePanel): make grip visible on active and hover, invisible otherwise
* ci: add data-provider tests to workflow, also update eslint/tsconfig to recognize specs, and add `text/csv` to fileConfig
* fix: handle edge case where auth object is undefined, and log errors
* refactor(actions): resolve schemas, add tests for resolving refs, import specs from separate file for tests
* chore: remove comment
* fix(ActionsInput): re-render bug when initializing states with action fields
* fix(patch/assistant): filter undefined tools
* chore: add logging for errors in assistants routes
* fix(updateAssistant): map actions to functions to avoid overwriting
* fix(actions): properly handle GET paths
* fix(convos): unhandled delete thread exception
* refactor(AssistantService): pass both thread_id and conversationId when sending intermediate assistant messages, remove `mapMessagesToSteps` from AssistantService
* refactor(useSSE): replace all messages with runMessages and pass latestMessageId to abortRun; fix(checkMessageGaps): include tool calls when syncing messages
* refactor(assistants/chat): invoke `createOnTextProgress` after thread creation
* chore: add typing
* style: sidepanel styling
* style: action tool call domain styling
* feat(assistants): default models, limit retrieval to certain models, add env variables to to env.example
* feat: assistants api key in EndpointService
* refactor: set assistant model to conversation on assistant switch
* refactor: set assistant model to conversation on assistant select from panel
* fix(retrieveAndProcessFile): catch attempt to download file with `assistant` purpose which is not allowed; add logging
* feat: retrieval styling, handling, and logging
* chore: rename ASSISTANTS_REVERSE_PROXY to ASSISTANTS_BASE_URL
* feat: FileContext for file metadata
* feat: context file mgmt and filtering
* style(Select): hover/rounded changes
* refactor: explicit conversation switch, endpoint dependent, through `useSelectAssistant`, which does not create new chat if current endpoint is assistant endpoint
* fix(AssistantAvatar): make empty previewURL if no avatar present
* refactor: side panel mobile styling
* style: merge tool and action section, optimize mobile styling for action/tool buttons
* fix: localStorage issues
* fix(useSelectAssistant): invoke react query hook directly in select hook as Map was not being updated in time
* style: light mode fixes
* fix: prevent sidepanel nav styling from shifting layout up
* refactor: change default layout (collapsed by default)
* style: mobile optimization of DataTable
* style: datatable
* feat: client-side hide right-side panel
* chore(useNewConvo): add partial typing for preset
* fix(useSelectAssistant): pass correct model name by using template as preset
* WIP: assistant presets
* refactor(ToolService): add native solution for `TavilySearchResults` and log tool output errors
* refactor: organize imports and use native TavilySearchResults
* fix(TavilySearchResults): stringify result
* fix(ToolCall): show tool call outputs when not an action
* chore: rename Prompt Prefix to custom instructions (in user facing text only)
* refactor(EditPresetDialog): Optimize setting title by debouncing, reset preset on dialog close to avoid state mixture
* feat: add `presetOverride` to overwrite active conversation settings when saving a Preset (relevant for client side updates only)
* feat: Assistant preset settings (client-side)
* fix(Switcher): only set assistant_id and model if current endpoint is Assistants
* feat: use `useDebouncedInput` for updating conversation settings, starting with EditPresetDialog title setting and Assistant instructions setting
* feat(Assistants): add instructions field to settings
* feat(chat/assistants): pass conversation settings to run body
* wip: begin localization and only allow actions if the assistant is created
* refactor(AssistantsPanel): knowledge localization, allow tools on creation
* feat: experimental: allow 'priming' values before assistant is created, that would normally require an assistant_id to be defined
* chore: trim console logs and make more meaningful
* chore: toast messages
* fix(ci): date test
* feat: create file when uploading Assistant Avatar
* feat: file upload rate limiting from custom config with dynamic file route initialization
* refactor: use file upload limiters on post routes only
* refactor(fileConfig): add endpoints field for endpoint specific fileconfigs, add mergeConfig function, add tests
* refactor: fileConfig route, dynamic multer instances used on all '/' and '/images' POST routes, data service and query hook
* feat: supportedMimeTypesSchema, test for array of regex
* feat: configurable file config limits
* chore: clarify assistants file knowledge prereq.
* chore(useTextarea): default to localized 'Assistant' if assistant name is empty
* feat: configurable file limits and toggle file upload per endpoint
* fix(useUploadFileMutation): prevent updating assistant.files cache if file upload is a message_file attachment
* fix(AssistantSelect): set last selected assistant only when timeout successfully runs
* refactor(queries): disable assistant queries if assistants endpoint is not enabled
* chore(Switcher): add localization
* chore: pluralize `assistant` for `EModelEndpoint key and value
* feat: show/hide assistant UI components based on endpoint availability; librechat.yaml config for disabling builder section and setting polling/timeout intervals
* fix(compactEndpointSchemas): use EModelEndpoint for schema access
* feat(runAssistant): use configured values from `librechat.yaml` for `pollIntervalMs` and `timeout`
* fix: naming issue
* wip: revert landing
* 🎉 happy birthday LibreChat (#1768)
* happy birthday LibreChat
* Refactor endpoint condition in Landing component
* Update birthday message in Eng.tsx
* fix(/config): avoid nesting ternaries
* refactor(/config): check birthday
---------
Co-authored-by: Danny Avila <messagedaniel@protonmail.com>
* fix: landing
* fix: landing
* fix(useMessageHelpers): hardcoded check to use EModelEndpoint instead
* fix(ci): convo test revert to main
* fix(assistants/chat): fix issue where assistant_id was being saved as model for convo
* chore: added logging, promises racing to prevent longer timeouts, explicit setting of maxRetries and timeouts, robust catching of invalid abortRun params
* refactor: use recoil state for `showStopButton` and only show for assistants endpoint after syncing conversation data
* refactor: optimize abortRun strategy using localStorage, refactor `abortConversation` to use async/await and await the result, refactor how the abortKey cache is set for runs
* fix(checkMessageGaps): assign `assistant_id` to synced messages if defined; prevents UI from showing blank assistant for cancelled messages
* refactor: re-order sequence of chat route, only allow aborting messages after run is created, cancel abortRun if there was a cancelling error (likely due already cancelled in chat route), and add extra logging
* chore(typedefs): add httpAgent type to OpenAIClient
* refactor: use custom implementation of retrieving run with axios to allow for timing out run query
* fix(waitForRun): handle timed out run retrieval query
* refactor: update preset conditions:
- presets will retain settings when a different endpoint is selected; for existing convos, either when modular or is assistant switch
- no longer use `navigateToConvo` on preset select
* fix: temporary calculator hack as expects string input when invoked
* fix: cancel abortRun only when cancelling error is a result of the run already being cancelled
* chore: remove use of `fileMaxSizeMB` and total counterpart (redundant)
* docs: custom config documentation update
* docs: assistants api setup and dotenv, new custom config fields
* refactor(Switcher): make Assistant switcher sticky in SidePanel
* chore(useSSE): remove console log of data and message index
* refactor(AssistantPanel): button styling and add secondary select button to bottom of panel
* refactor(OpenAIClient): allow passing conversationId to RunManager through titleConvo and initializeLLM to properly record title context tokens used in cases where conversationId was not defined by the client
* feat(assistants): token tracking for assistant runs
* chore(spendTokens): improve logging
* feat: support/exclude specific assistant Ids
* chore: add update `librechat.example.yaml`, optimize `AppService` handling, new tests for `AppService`, optimize missing/outdate config logging
* chore: mount docker logs to root of project
* chore: condense axios errors
* chore: bump vite
* chore: vite hot reload fix using latest version
* chore(getOpenAIModels): sort instruct models to the end of models list
* fix(assistants): user provided key
* fix(assistants): user provided key, invalidate more queries on revoke
---------
Co-authored-by: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
2024-02-13 20:42:27 -05:00
|
|
|
"mime": "^3.0.0",
|
feat: Vision Support + New UI (#1203)
* feat: add timer duration to showToast, show toast for preset selection
* refactor: replace old /chat/ route with /c/. e2e tests will fail here
* refactor: move typedefs to root of /api/ and add a few to assistant types in TS
* refactor: reorganize data-provider imports, fix dependency cycle, strategize new plan to separate react dependent packages
* feat: add dataService for uploading images
* feat(data-provider): add mutation keys
* feat: file resizing and upload
* WIP: initial API image handling
* fix: catch JSON.parse of localStorage tools
* chore: experimental: use module-alias for absolute imports
* refactor: change temp_file_id strategy
* fix: updating files state by using Map and defining react query callbacks in a way that keeps them during component unmount, initial delete handling
* feat: properly handle file deletion
* refactor: unexpose complete filepath and resize from server for higher fidelity
* fix: make sure resized height, width is saved, catch bad requests
* refactor: use absolute imports
* fix: prevent setOptions from being called more than once for OpenAIClient, made note to fix for PluginsClient
* refactor: import supportsFiles and models vars from schemas
* fix: correctly replace temp file id
* refactor(BaseClient): use absolute imports, pass message 'opts' to buildMessages method, count tokens for nested objects/arrays
* feat: add validateVisionModel to determine if model has vision capabilities
* chore(checkBalance): update jsdoc
* feat: formatVisionMessage: change message content format dependent on role and image_urls passed
* refactor: add usage to File schema, make create and updateFile, correctly set and remove TTL
* feat: working vision support
TODO: file size, type, amount validations, making sure they are styled right, and making sure you can add images from the clipboard/dragging
* feat: clipboard support for uploading images
* feat: handle files on drop to screen, refactor top level view code to Presentation component so the useDragHelpers hook has ChatContext
* fix(Images): replace uploaded images in place
* feat: add filepath validation to protect sensitive files
* fix: ensure correct file_ids are push and not the Map key values
* fix(ToastContext): type issue
* feat: add basic file validation
* fix(useDragHelpers): correct context issue with `files` dependency
* refactor: consolidate setErrors logic to setError
* feat: add dialog Image overlay on image click
* fix: close endpoints menu on click
* chore: set detail to auto, make note for configuration
* fix: react warning (button desc. of button)
* refactor: optimize filepath handling, pass file_ids to images for easier re-use
* refactor: optimize image file handling, allow re-using files in regen, pass more file metadata in messages
* feat: lazy loading images including use of upload preview
* fix: SetKeyDialog closing, stopPropagation on Dialog content click
* style(EndpointMenuItem): tighten up the style, fix dark theme showing in lightmode, make menu more ux friendly
* style: change maxheight of all settings textareas to 138px from 300px
* style: better styling for textarea and enclosing buttons
* refactor(PresetItems): swap back edit and delete icons
* feat: make textarea placeholder dynamic to endpoint
* style: show user hover buttons only on hover when message is streaming
* fix: ordered list not going past 9, fix css
* feat: add User/AI labels; style: hide loading spinner
* feat: add back custom footer, change original footer text
* feat: dynamic landing icons based on endpoint
* chore: comment out assistants route
* fix: autoScroll to newest on /c/ view
* fix: Export Conversation on new UI
* style: match message style of official more closely
* ci: fix api jest unit tests, comment out e2e tests for now as they will fail until addressed
* feat: more file validation and use blob in preview field, not filepath, to fix temp deletion
* feat: filefilter for multer
* feat: better AI labels based on custom name, model, and endpoint instead of `ChatGPT`
2023-11-21 20:12:48 -05:00
|
|
|
"module-alias": "^2.2.3",
|
2025-03-11 14:44:54 -04:00
|
|
|
"mongoose": "^8.12.1",
|
2026-03-04 21:49:13 -05:00
|
|
|
"multer": "^2.1.1",
|
2024-08-05 03:34:00 -04:00
|
|
|
"nanoid": "^3.3.7",
|
🧠 feat: User Memories for Conversational Context (#7760)
* 🧠 feat: User Memories for Conversational Context
chore: mcp typing, use `t`
WIP: first pass, Memories UI
- Added MemoryViewer component for displaying, editing, and deleting user memories.
- Integrated data provider hooks for fetching, updating, and deleting memories.
- Implemented pagination and loading states for better user experience.
- Created unit tests for MemoryViewer to ensure functionality and interaction with data provider.
- Updated translation files to include new UI strings related to memories.
chore: move mcp-related files to own directory
chore: rename librechat-mcp to librechat-api
WIP: first pass, memory processing and data schemas
chore: linting in fileSearch.js query description
chore: rename librechat-api to @librechat/api across the project
WIP: first pass, functional memory agent
feat: add MemoryEditDialog and MemoryViewer components for managing user memories
- Introduced MemoryEditDialog for editing memory entries with validation and toast notifications.
- Updated MemoryViewer to support editing and deleting memories, including pagination and loading states.
- Enhanced data provider to handle memory updates with optional original key for better management.
- Added new localization strings for memory-related UI elements.
feat: add memory permissions management
- Implemented memory permissions in the backend, allowing roles to have specific permissions for using, creating, updating, and reading memories.
- Added new API endpoints for updating memory permissions associated with roles.
- Created a new AdminSettings component for managing memory permissions in the frontend.
- Integrated memory permissions into the existing roles and permissions schemas.
- Updated the interface to include memory settings and permissions.
- Enhanced the MemoryViewer component to conditionally render admin settings based on user roles.
- Added localization support for memory permissions in the translation files.
feat: move AdminSettings component to a new position in MemoryViewer for better visibility
refactor: clean up commented code in MemoryViewer component
feat: enhance MemoryViewer with search functionality and improve MemoryEditDialog integration
- Added a search input to filter memories in the MemoryViewer component.
- Refactored MemoryEditDialog to accept children for better customization.
- Updated MemoryViewer to utilize the new EditMemoryButton and DeleteMemoryButton components for editing and deleting memories.
- Improved localization support by adding new strings for memory filtering and deletion confirmation.
refactor: optimize memory filtering in MemoryViewer using match-sorter
- Replaced manual filtering logic with match-sorter for improved search functionality.
- Enhanced performance and readability of the filteredMemories computation.
feat: enhance MemoryEditDialog with triggerRef and improve updateMemory mutation handling
feat: implement access control for MemoryEditDialog and MemoryViewer components
refactor: remove commented out code and create runMemory method
refactor: rename role based files
feat: implement access control for memory usage in AgentClient
refactor: simplify checkVisionRequest method in AgentClient by removing commented-out code
refactor: make `agents` dir in api package
refactor: migrate Azure utilities to TypeScript and consolidate imports
refactor: move sanitizeFilename function to a new file and update imports, add related tests
refactor: update LLM configuration types and consolidate Azure options in the API package
chore: linting
chore: import order
refactor: replace getLLMConfig with getOpenAIConfig and remove unused LLM configuration file
chore: update winston-daily-rotate-file to version 5.0.0 and add object-hash dependency in package-lock.json
refactor: move primeResources and optionalChainWithEmptyCheck functions to resources.ts and update imports
refactor: move createRun function to a new run.ts file and update related imports
fix: ensure safeAttachments is correctly typed as an array of TFile
chore: add node-fetch dependency and refactor fetch-related functions into packages/api/utils, removing the old generators file
refactor: enhance TEndpointOption type by using Pick to streamline endpoint fields and add new properties for model parameters and client options
feat: implement initializeOpenAIOptions function and update OpenAI types for enhanced configuration handling
fix: update types due to new TEndpointOption typing
fix: ensure safe access to group parameters in initializeOpenAIOptions function
fix: remove redundant API key validation comment in initializeOpenAIOptions function
refactor: rename initializeOpenAIOptions to initializeOpenAI for consistency and update related documentation
refactor: decouple req.body fields and tool loading from initializeAgentOptions
chore: linting
refactor: adjust column widths in MemoryViewer for improved layout
refactor: simplify agent initialization by creating loadAgent function and removing unused code
feat: add memory configuration loading and validation functions
WIP: first pass, memory processing with config
feat: implement memory callback and artifact handling
feat: implement memory artifacts display and processing updates
feat: add memory configuration options and schema validation for validKeys
fix: update MemoryEditDialog and MemoryViewer to handle memory state and display improvements
refactor: remove padding from BookmarkTable and MemoryViewer headers for consistent styling
WIP: initial tokenLimit config and move Tokenizer to @librechat/api
refactor: update mongoMeili plugin methods to use callback for better error handling
feat: enhance memory management with token tracking and usage metrics
- Added token counting for memory entries to enforce limits and provide usage statistics.
- Updated memory retrieval and update routes to include total token usage and limit.
- Enhanced MemoryEditDialog and MemoryViewer components to display memory usage and token information.
- Refactored memory processing functions to handle token limits and provide feedback on memory capacity.
feat: implement memory artifact handling in attachment handler
- Enhanced useAttachmentHandler to process memory artifacts when receiving updates.
- Introduced handleMemoryArtifact utility to manage memory updates and deletions.
- Updated query client to reflect changes in memory state based on incoming data.
refactor: restructure web search key extraction logic
- Moved the logic for extracting API keys from the webSearchAuth configuration into a dedicated function, getWebSearchKeys.
- Updated webSearchKeys to utilize the new function for improved clarity and maintainability.
- Prevents build time errors
feat: add personalization settings and memory preferences management
- Introduced a new Personalization tab in settings to manage user memory preferences.
- Implemented API endpoints and client-side logic for updating memory preferences.
- Enhanced user interface components to reflect personalization options and memory usage.
- Updated permissions to allow users to opt out of memory features.
- Added localization support for new settings and messages related to personalization.
style: personalization switch class
feat: add PersonalizationIcon and align Side Panel UI
feat: implement memory creation functionality
- Added a new API endpoint for creating memory entries, including validation for key and value.
- Introduced MemoryCreateDialog component for user interface to facilitate memory creation.
- Integrated token limit checks to prevent exceeding user memory capacity.
- Updated MemoryViewer to include a button for opening the memory creation dialog.
- Enhanced localization support for new messages related to memory creation.
feat: enhance message processing with configurable window size
- Updated AgentClient to use a configurable message window size for processing messages.
- Introduced messageWindowSize option in memory configuration schema with a default value of 5.
- Improved logic for selecting messages to process based on the configured window size.
chore: update librechat-data-provider version to 0.7.87 in package.json and package-lock.json
chore: remove OpenAPIPlugin and its associated tests
chore: remove MIGRATION_README.md as migration tasks are completed
ci: fix backend tests
chore: remove unused translation keys from localization file
chore: remove problematic test file and unused var in AgentClient
chore: remove unused import and import directly for JSDoc
* feat: add api package build stage in Dockerfile for improved modularity
* docs: reorder build steps in contributing guide for clarity
2025-06-07 18:52:22 -04:00
|
|
|
"node-fetch": "^2.7.0",
|
2025-12-02 00:50:15 -05:00
|
|
|
"nodemailer": "^7.0.11",
|
2024-05-08 20:24:40 -04:00
|
|
|
"ollama": "^0.5.0",
|
2025-11-05 19:24:36 -05:00
|
|
|
"openai": "5.8.2",
|
2025-05-22 14:19:24 +02:00
|
|
|
"openid-client": "^6.5.0",
|
feat: Auth and User System (#205)
* server-side JWT auth implementation
* move oauth routes and strategies, fix bugs
* backend modifications for wiring up the frontend login and reg forms
* Add frontend data services for login and registration
* Add login and registration forms
* Implment auth context, functional client side auth
* protect routes with jwt auth
* finish local strategy (using local storage)
* Start setting up google auth
* disable token refresh, remove old auth middleware
* refactor client, add ApiErrorBoundary context
* disable google and facebook strategies
* fix: fix presets not displaying specific to user
* fix: fix issue with browser refresh
* fix: casing issue with User.js (#11)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* fix: issue with api error watcher (#12)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* feat: add google auth social login
* fix: make google login url dynamic based on dev/prod
* fix: bug where UI is briefly displayed before redirecting to login
* fix: fix cookie expires value for local auth
* Update README.md
* Update LOCAL_INSTALL structure
* Add local testing instructions
* Only load google strategy if client id and secret are provided
* Update .env.example files with new params
* fix issue with not redirecting to register form
* only show google login button if value is set in .env
* cleanup log messages
* Add label to button for google login on login form
* doc: fix client/server url values in .env.example
* feat: add error message details to registration failure
* Restore preventing paste on confirm password
* auto-login user after registering
* feat: forgot password (#24)
* make login/reg pages look like openai's
* add password reset data services
* new form designs similar to openai, add password reset pages
* add api's for password reset
* email utils for password reset
* remove bcrypt salt rounds from process.env
* refactor: restructure api auth code, consolidate routes (#25)
* add api's for password reset
* remove bcrypt salt rounds from process.env
* refactor: consolidate auth routes, use controller pattern
* refactor: code cleanup
* feat: migrate data to first user (#26)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes after refactor (#27)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: issue with auto-login when logging out then logging in with new browser window (#28)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: Update README and .env.example files with user system information (#29)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: update README and .env.example files
* Fixup: LOCAL_INSTALL.md PS instructions (#200) (#30)
Co-authored-by: alfredo-f <alfredo.fomitchenko@mail.polimi.it>
* feat: send user with completion to protect against abuse (#31)
* Fixup: LOCAL_INSTALL.md PS instructions (#200)
* server-side JWT auth implementation
* move oauth routes and strategies, fix bugs
* backend modifications for wiring up the frontend login and reg forms
* Add frontend data services for login and registration
* Add login and registration forms
* Implment auth context, functional client side auth
* protect routes with jwt auth
* finish local strategy (using local storage)
* Start setting up google auth
* disable token refresh, remove old auth middleware
* refactor client, add ApiErrorBoundary context
* disable google and facebook strategies
* fix: fix presets not displaying specific to user
* fix: fix issue with browser refresh
* fix: casing issue with User.js (#11)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* feat: add google auth social login
* fix: make google login url dynamic based on dev/prod
* fix: bug where UI is briefly displayed before redirecting to login
* fix: fix cookie expires value for local auth
* Only load google strategy if client id and secret are provided
* Update .env.example files with new params
* fix issue with not redirecting to register form
* only show google login button if value is set in .env
* cleanup log messages
* Add label to button for google login on login form
* doc: fix client/server url values in .env.example
* feat: add error message details to registration failure
* Restore preventing paste on confirm password
* auto-login user after registering
* feat: forgot password (#24)
* make login/reg pages look like openai's
* add password reset data services
* new form designs similar to openai, add password reset pages
* add api's for password reset
* email utils for password reset
* remove bcrypt salt rounds from process.env
* refactor: restructure api auth code, consolidate routes (#25)
* add api's for password reset
* remove bcrypt salt rounds from process.env
* refactor: consolidate auth routes, use controller pattern
* refactor: code cleanup
* feat: migrate data to first user (#26)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes after refactor (#27)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: issue with auto-login when logging out then logging in with new browser window (#28)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: Update README and .env.example files with user system information (#29)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: update README and .env.example files
* Send user id to openai to protect against abuse
* add meilisearch to gitignore
* Remove webpack
---------
Co-authored-by: alfredo-f <alfredo.fomitchenko@mail.polimi.it>
---------
Co-authored-by: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Co-authored-by: Alfredo Fomitchenko <alfredo.fomitchenko@mail.polimi.it>
2023-05-07 10:04:51 -07:00
|
|
|
"passport": "^0.6.0",
|
2025-01-31 15:49:09 +01:00
|
|
|
"passport-apple": "^2.0.2",
|
2023-07-22 07:29:17 -07:00
|
|
|
"passport-discord": "^0.1.4",
|
feat: Auth and User System (#205)
* server-side JWT auth implementation
* move oauth routes and strategies, fix bugs
* backend modifications for wiring up the frontend login and reg forms
* Add frontend data services for login and registration
* Add login and registration forms
* Implment auth context, functional client side auth
* protect routes with jwt auth
* finish local strategy (using local storage)
* Start setting up google auth
* disable token refresh, remove old auth middleware
* refactor client, add ApiErrorBoundary context
* disable google and facebook strategies
* fix: fix presets not displaying specific to user
* fix: fix issue with browser refresh
* fix: casing issue with User.js (#11)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* fix: issue with api error watcher (#12)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* feat: add google auth social login
* fix: make google login url dynamic based on dev/prod
* fix: bug where UI is briefly displayed before redirecting to login
* fix: fix cookie expires value for local auth
* Update README.md
* Update LOCAL_INSTALL structure
* Add local testing instructions
* Only load google strategy if client id and secret are provided
* Update .env.example files with new params
* fix issue with not redirecting to register form
* only show google login button if value is set in .env
* cleanup log messages
* Add label to button for google login on login form
* doc: fix client/server url values in .env.example
* feat: add error message details to registration failure
* Restore preventing paste on confirm password
* auto-login user after registering
* feat: forgot password (#24)
* make login/reg pages look like openai's
* add password reset data services
* new form designs similar to openai, add password reset pages
* add api's for password reset
* email utils for password reset
* remove bcrypt salt rounds from process.env
* refactor: restructure api auth code, consolidate routes (#25)
* add api's for password reset
* remove bcrypt salt rounds from process.env
* refactor: consolidate auth routes, use controller pattern
* refactor: code cleanup
* feat: migrate data to first user (#26)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes after refactor (#27)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: issue with auto-login when logging out then logging in with new browser window (#28)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: Update README and .env.example files with user system information (#29)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: update README and .env.example files
* Fixup: LOCAL_INSTALL.md PS instructions (#200) (#30)
Co-authored-by: alfredo-f <alfredo.fomitchenko@mail.polimi.it>
* feat: send user with completion to protect against abuse (#31)
* Fixup: LOCAL_INSTALL.md PS instructions (#200)
* server-side JWT auth implementation
* move oauth routes and strategies, fix bugs
* backend modifications for wiring up the frontend login and reg forms
* Add frontend data services for login and registration
* Add login and registration forms
* Implment auth context, functional client side auth
* protect routes with jwt auth
* finish local strategy (using local storage)
* Start setting up google auth
* disable token refresh, remove old auth middleware
* refactor client, add ApiErrorBoundary context
* disable google and facebook strategies
* fix: fix presets not displaying specific to user
* fix: fix issue with browser refresh
* fix: casing issue with User.js (#11)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* feat: add google auth social login
* fix: make google login url dynamic based on dev/prod
* fix: bug where UI is briefly displayed before redirecting to login
* fix: fix cookie expires value for local auth
* Only load google strategy if client id and secret are provided
* Update .env.example files with new params
* fix issue with not redirecting to register form
* only show google login button if value is set in .env
* cleanup log messages
* Add label to button for google login on login form
* doc: fix client/server url values in .env.example
* feat: add error message details to registration failure
* Restore preventing paste on confirm password
* auto-login user after registering
* feat: forgot password (#24)
* make login/reg pages look like openai's
* add password reset data services
* new form designs similar to openai, add password reset pages
* add api's for password reset
* email utils for password reset
* remove bcrypt salt rounds from process.env
* refactor: restructure api auth code, consolidate routes (#25)
* add api's for password reset
* remove bcrypt salt rounds from process.env
* refactor: consolidate auth routes, use controller pattern
* refactor: code cleanup
* feat: migrate data to first user (#26)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes after refactor (#27)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: issue with auto-login when logging out then logging in with new browser window (#28)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: Update README and .env.example files with user system information (#29)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: update README and .env.example files
* Send user id to openai to protect against abuse
* add meilisearch to gitignore
* Remove webpack
---------
Co-authored-by: alfredo-f <alfredo.fomitchenko@mail.polimi.it>
---------
Co-authored-by: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Co-authored-by: Alfredo Fomitchenko <alfredo.fomitchenko@mail.polimi.it>
2023-05-07 10:04:51 -07:00
|
|
|
"passport-facebook": "^3.0.0",
|
2023-07-22 07:29:17 -07:00
|
|
|
"passport-github2": "^0.1.12",
|
feat: Auth and User System (#205)
* server-side JWT auth implementation
* move oauth routes and strategies, fix bugs
* backend modifications for wiring up the frontend login and reg forms
* Add frontend data services for login and registration
* Add login and registration forms
* Implment auth context, functional client side auth
* protect routes with jwt auth
* finish local strategy (using local storage)
* Start setting up google auth
* disable token refresh, remove old auth middleware
* refactor client, add ApiErrorBoundary context
* disable google and facebook strategies
* fix: fix presets not displaying specific to user
* fix: fix issue with browser refresh
* fix: casing issue with User.js (#11)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* fix: issue with api error watcher (#12)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* feat: add google auth social login
* fix: make google login url dynamic based on dev/prod
* fix: bug where UI is briefly displayed before redirecting to login
* fix: fix cookie expires value for local auth
* Update README.md
* Update LOCAL_INSTALL structure
* Add local testing instructions
* Only load google strategy if client id and secret are provided
* Update .env.example files with new params
* fix issue with not redirecting to register form
* only show google login button if value is set in .env
* cleanup log messages
* Add label to button for google login on login form
* doc: fix client/server url values in .env.example
* feat: add error message details to registration failure
* Restore preventing paste on confirm password
* auto-login user after registering
* feat: forgot password (#24)
* make login/reg pages look like openai's
* add password reset data services
* new form designs similar to openai, add password reset pages
* add api's for password reset
* email utils for password reset
* remove bcrypt salt rounds from process.env
* refactor: restructure api auth code, consolidate routes (#25)
* add api's for password reset
* remove bcrypt salt rounds from process.env
* refactor: consolidate auth routes, use controller pattern
* refactor: code cleanup
* feat: migrate data to first user (#26)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes after refactor (#27)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: issue with auto-login when logging out then logging in with new browser window (#28)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: Update README and .env.example files with user system information (#29)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: update README and .env.example files
* Fixup: LOCAL_INSTALL.md PS instructions (#200) (#30)
Co-authored-by: alfredo-f <alfredo.fomitchenko@mail.polimi.it>
* feat: send user with completion to protect against abuse (#31)
* Fixup: LOCAL_INSTALL.md PS instructions (#200)
* server-side JWT auth implementation
* move oauth routes and strategies, fix bugs
* backend modifications for wiring up the frontend login and reg forms
* Add frontend data services for login and registration
* Add login and registration forms
* Implment auth context, functional client side auth
* protect routes with jwt auth
* finish local strategy (using local storage)
* Start setting up google auth
* disable token refresh, remove old auth middleware
* refactor client, add ApiErrorBoundary context
* disable google and facebook strategies
* fix: fix presets not displaying specific to user
* fix: fix issue with browser refresh
* fix: casing issue with User.js (#11)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* feat: add google auth social login
* fix: make google login url dynamic based on dev/prod
* fix: bug where UI is briefly displayed before redirecting to login
* fix: fix cookie expires value for local auth
* Only load google strategy if client id and secret are provided
* Update .env.example files with new params
* fix issue with not redirecting to register form
* only show google login button if value is set in .env
* cleanup log messages
* Add label to button for google login on login form
* doc: fix client/server url values in .env.example
* feat: add error message details to registration failure
* Restore preventing paste on confirm password
* auto-login user after registering
* feat: forgot password (#24)
* make login/reg pages look like openai's
* add password reset data services
* new form designs similar to openai, add password reset pages
* add api's for password reset
* email utils for password reset
* remove bcrypt salt rounds from process.env
* refactor: restructure api auth code, consolidate routes (#25)
* add api's for password reset
* remove bcrypt salt rounds from process.env
* refactor: consolidate auth routes, use controller pattern
* refactor: code cleanup
* feat: migrate data to first user (#26)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes after refactor (#27)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: issue with auto-login when logging out then logging in with new browser window (#28)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: Update README and .env.example files with user system information (#29)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: update README and .env.example files
* Send user id to openai to protect against abuse
* add meilisearch to gitignore
* Remove webpack
---------
Co-authored-by: alfredo-f <alfredo.fomitchenko@mail.polimi.it>
---------
Co-authored-by: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Co-authored-by: Alfredo Fomitchenko <alfredo.fomitchenko@mail.polimi.it>
2023-05-07 10:04:51 -07:00
|
|
|
"passport-google-oauth20": "^2.0.0",
|
|
|
|
|
"passport-jwt": "^4.0.1",
|
2024-05-29 14:46:20 -07:00
|
|
|
"passport-ldapauth": "^3.0.1",
|
feat: Auth and User System (#205)
* server-side JWT auth implementation
* move oauth routes and strategies, fix bugs
* backend modifications for wiring up the frontend login and reg forms
* Add frontend data services for login and registration
* Add login and registration forms
* Implment auth context, functional client side auth
* protect routes with jwt auth
* finish local strategy (using local storage)
* Start setting up google auth
* disable token refresh, remove old auth middleware
* refactor client, add ApiErrorBoundary context
* disable google and facebook strategies
* fix: fix presets not displaying specific to user
* fix: fix issue with browser refresh
* fix: casing issue with User.js (#11)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* fix: issue with api error watcher (#12)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* feat: add google auth social login
* fix: make google login url dynamic based on dev/prod
* fix: bug where UI is briefly displayed before redirecting to login
* fix: fix cookie expires value for local auth
* Update README.md
* Update LOCAL_INSTALL structure
* Add local testing instructions
* Only load google strategy if client id and secret are provided
* Update .env.example files with new params
* fix issue with not redirecting to register form
* only show google login button if value is set in .env
* cleanup log messages
* Add label to button for google login on login form
* doc: fix client/server url values in .env.example
* feat: add error message details to registration failure
* Restore preventing paste on confirm password
* auto-login user after registering
* feat: forgot password (#24)
* make login/reg pages look like openai's
* add password reset data services
* new form designs similar to openai, add password reset pages
* add api's for password reset
* email utils for password reset
* remove bcrypt salt rounds from process.env
* refactor: restructure api auth code, consolidate routes (#25)
* add api's for password reset
* remove bcrypt salt rounds from process.env
* refactor: consolidate auth routes, use controller pattern
* refactor: code cleanup
* feat: migrate data to first user (#26)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes after refactor (#27)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: issue with auto-login when logging out then logging in with new browser window (#28)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: Update README and .env.example files with user system information (#29)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: update README and .env.example files
* Fixup: LOCAL_INSTALL.md PS instructions (#200) (#30)
Co-authored-by: alfredo-f <alfredo.fomitchenko@mail.polimi.it>
* feat: send user with completion to protect against abuse (#31)
* Fixup: LOCAL_INSTALL.md PS instructions (#200)
* server-side JWT auth implementation
* move oauth routes and strategies, fix bugs
* backend modifications for wiring up the frontend login and reg forms
* Add frontend data services for login and registration
* Add login and registration forms
* Implment auth context, functional client side auth
* protect routes with jwt auth
* finish local strategy (using local storage)
* Start setting up google auth
* disable token refresh, remove old auth middleware
* refactor client, add ApiErrorBoundary context
* disable google and facebook strategies
* fix: fix presets not displaying specific to user
* fix: fix issue with browser refresh
* fix: casing issue with User.js (#11)
* delete user.js to be renamed
* fix: fix casing issue with User.js
* comment out api error watcher temporarily
* feat: add google auth social login
* fix: make google login url dynamic based on dev/prod
* fix: bug where UI is briefly displayed before redirecting to login
* fix: fix cookie expires value for local auth
* Only load google strategy if client id and secret are provided
* Update .env.example files with new params
* fix issue with not redirecting to register form
* only show google login button if value is set in .env
* cleanup log messages
* Add label to button for google login on login form
* doc: fix client/server url values in .env.example
* feat: add error message details to registration failure
* Restore preventing paste on confirm password
* auto-login user after registering
* feat: forgot password (#24)
* make login/reg pages look like openai's
* add password reset data services
* new form designs similar to openai, add password reset pages
* add api's for password reset
* email utils for password reset
* remove bcrypt salt rounds from process.env
* refactor: restructure api auth code, consolidate routes (#25)
* add api's for password reset
* remove bcrypt salt rounds from process.env
* refactor: consolidate auth routes, use controller pattern
* refactor: code cleanup
* feat: migrate data to first user (#26)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes after refactor (#27)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: issue with auto-login when logging out then logging in with new browser window (#28)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: Update README and .env.example files with user system information (#29)
* refactor: use /api for auth routes
* fix: use user id instead of username
* feat: migrate data to first user on register
* fix: fix social login routes
* fix: fix issue with auto-login in new tab
* doc: update README and .env.example files
* Send user id to openai to protect against abuse
* add meilisearch to gitignore
* Remove webpack
---------
Co-authored-by: alfredo-f <alfredo.fomitchenko@mail.polimi.it>
---------
Co-authored-by: Danny Avila <110412045+danny-avila@users.noreply.github.com>
Co-authored-by: Alfredo Fomitchenko <alfredo.fomitchenko@mail.polimi.it>
2023-05-07 10:04:51 -07:00
|
|
|
"passport-local": "^1.0.0",
|
📄 feat: Local Text Extraction for PDF, DOCX, and XLS/XLSX (#11900)
* feat: Added "document parser" OCR strategy
The document parser uses libraries to parse the text out of known document types.
This lets LibreChat handle some complex document types without having to use a
secondary service (like Mistral or standing up a RAG API server).
To enable the document parser, set the ocr strategy to "document_parser" in
librechat.yaml.
We now support:
- PDFs using pdfjs
- DOCX using mammoth
- XLS/XLSX using SheetJS
(The associated packages were also added to the project.)
* fix: applied Copilot code review suggestions
- Properly calculate length of text based on UTF8.
- Avoid issues with loading / blocking PDF parsing.
* fix: improved docs on parseDocument()
* chore: move to packages/api for TS support
* refactor: make document processing the default ocr strategy
- Introduced support for additional document types in the OCR strategy, including PDF, DOCX, and XLS/XLSX.
- Updated the file upload handling to dynamically select the appropriate parsing strategy based on the file type.
- Refactored the document parsing functions to use asynchronous imports for improved performance and maintainability.
* test: add unit tests for processAgentFileUpload functionality
- Introduced a new test suite for the processAgentFileUpload function in process.spec.js.
- Implemented various test cases to validate OCR strategy selection based on file types, including PDF, DOCX, XLSX, and XLS.
- Mocked dependencies to ensure isolated testing of file upload handling and strategy selection logic.
- Enhanced coverage for scenarios involving OCR capability checks and default strategy fallbacks.
* chore: update pdfjs-dist version and enhance document parsing tests
- Bumped pdfjs-dist dependency to version 5.4.624 in both api and packages/api.
- Refactored document parsing tests to use 'originalname' instead of 'filename' for file objects.
- Added a new test case for parsing XLS files to improve coverage of document types supported by the parser.
- Introduced a sample XLS file for testing purposes.
* feat: enforce text size limit and improve OCR fallback handling in processAgentFileUpload
- Added a check to ensure extracted text does not exceed the 15MB storage limit, throwing an error if it does.
- Refactored the OCR handling logic to improve fallback behavior when the configured OCR fails, ensuring a more robust document processing flow.
- Enhanced unit tests to cover scenarios for oversized text and fallback mechanisms, ensuring proper error handling and functionality.
* fix: correct OCR URL construction in performOCR function
- Updated the OCR URL construction to ensure it correctly appends '/ocr' to the base URL if not already present, improving the reliability of the OCR request.
---------
Co-authored-by: Dan Lew <daniel@mightyacorn.com>
2026-02-22 14:22:45 -05:00
|
|
|
"pdfjs-dist": "^5.4.624",
|
2025-10-04 16:43:22 -04:00
|
|
|
"rate-limit-redis": "^4.2.0",
|
2025-04-01 03:50:32 -04:00
|
|
|
"sharp": "^0.33.5",
|
2023-12-14 07:49:27 -05:00
|
|
|
"traverse": "^0.6.7",
|
feat: Message Rate Limiters, Violation Logging, & Ban System 🔨 (#903)
* refactor: require Auth middleware in route index files
* feat: concurrent message limiter
* feat: complete concurrent message limiter with caching
* refactor: SSE response methods separated from handleText
* fix(abortMiddleware): fix req and res order to standard, use endpointOption in req.body
* chore: minor name changes
* refactor: add isUUID condition to saveMessage
* fix(concurrentLimiter): logic correctly handles the max number of concurrent messages and res closing/finalization
* chore: bump keyv and remove console.log from Message
* fix(concurrentLimiter): ensure messages are only saved in later message children
* refactor(concurrentLimiter): use KeyvFile instead, could make other stores configurable in the future
* feat: add denyRequest function for error responses
* feat(utils): add isStringTruthy function
Introduce the isStringTruthy function to the utilities module to check if a string value is a case-insensitive match for 'true'
* feat: add optional message rate limiters by IP and userId
* feat: add optional message rate limiters by IP and userId to edit route
* refactor: rename isStringTruthy to isTrue for brevity
* refactor(getError): use map to make code cleaner
* refactor: use memory for concurrent rate limiter to prevent clearing on startup/exit, add multiple log files, fix error message for concurrent violation
* feat: check if errorMessage is object, stringify if so
* chore: send object to denyRequest which will stringify it
* feat: log excessive requests
* fix(getError): correctly pluralize messages
* refactor(limiters): make type consistent between logs and errorMessage
* refactor(cache): move files out of lib/db into separate cache dir
>> feat: add getLogStores function so Keyv instance is not redundantly created on every violation
feat: separate violation logging to own function with logViolation
* fix: cache/index.js export, properly record userViolations
* refactor(messageLimiters): use new logging method, add logging to registrations
* refactor(logViolation): make userLogs an array of logs per user
* feat: add logging to login limiter
* refactor: pass req as first param to logViolation and record offending IP
* refactor: rename isTrue helper fn to isEnabled
* feat: add simple non_browser check and log violation
* fix: open handles in unit tests, remove KeyvMongo as not used and properly mock global fetch
* chore: adjust nodemon ignore paths to properly ignore logs
* feat: add math helper function for safe use of eval
* refactor(api/convos): use middleware at top of file to avoid redundancy
* feat: add delete all static method for Sessions
* fix: redirect to login on refresh if user is not found, or the session is not found but hasn't expired (ban case)
* refactor(getLogStores): adjust return type
* feat: add ban violation and check ban logic
refactor(logViolation): pass both req and res objects
* feat: add removePorts helper function
* refactor: rename getError to getMessageError and add getLoginError for displaying different login errors
* fix(AuthContext): fix type issue and remove unused code
* refactor(bans): ban by ip and user id, send response based on origin
* chore: add frontend ban messages
* refactor(routes/oauth): add ban check to handler, also consolidate logic to avoid redundancy
* feat: add ban check to AI messaging routes
* feat: add ban check to login/registration
* fix(ci/api): mock KeyvMongo to avoid tests hanging
* docs: update .env.example
> refactor(banViolation): calculate interval rate crossover, early return if duration is invalid
ci(banViolation): add tests to ensure users are only banned when expected
* docs: improve wording for mod system
* feat: add configurable env variables for violation scores
* chore: add jsdoc for uaParser.js
* chore: improve ban text log
* chore: update bun test scripts
* refactor(math.js): add fallback values
* fix(KeyvMongo/banLogs): refactor keyv instances to top of files to avoid memory leaks, refactor ban logic to use getLogStores instead
refactor(getLogStores): get a single log store by type
* fix(ci): refactor tests due to banLogs changes, also make sure to clear and revoke sessions even if ban duration is 0
* fix(banViolation.js): getLogStores import
* feat: handle 500 code error at login
* fix(middleware): handle case where user.id is _id and not just id
* ci: add ban secrets for backend unit tests
* refactor: logout user upon ban
* chore: log session delete message only if deletedCount > 0
* refactor: change default ban duration (2h) and make logic more clear in JSDOC
* fix: login and registration limiters will now return rate limiting error
* fix: userId not parsable as non ObjectId string
* feat: add useTimeout hook to properly clear timeouts when invoking functions within them
refactor(AuthContext): cleanup code by using new hook and defining types in ~/common
* fix: login error message for rate limits
* docs: add info for automated mod system and rate limiters, update other docs accordingly
* chore: bump data-provider version
2023-09-13 10:57:07 -04:00
|
|
|
"ua-parser-js": "^1.0.36",
|
2026-03-14 03:36:03 -04:00
|
|
|
"undici": "^7.24.1",
|
2023-12-14 07:49:27 -05:00
|
|
|
"winston": "^3.11.0",
|
🧠 feat: User Memories for Conversational Context (#7760)
* 🧠 feat: User Memories for Conversational Context
chore: mcp typing, use `t`
WIP: first pass, Memories UI
- Added MemoryViewer component for displaying, editing, and deleting user memories.
- Integrated data provider hooks for fetching, updating, and deleting memories.
- Implemented pagination and loading states for better user experience.
- Created unit tests for MemoryViewer to ensure functionality and interaction with data provider.
- Updated translation files to include new UI strings related to memories.
chore: move mcp-related files to own directory
chore: rename librechat-mcp to librechat-api
WIP: first pass, memory processing and data schemas
chore: linting in fileSearch.js query description
chore: rename librechat-api to @librechat/api across the project
WIP: first pass, functional memory agent
feat: add MemoryEditDialog and MemoryViewer components for managing user memories
- Introduced MemoryEditDialog for editing memory entries with validation and toast notifications.
- Updated MemoryViewer to support editing and deleting memories, including pagination and loading states.
- Enhanced data provider to handle memory updates with optional original key for better management.
- Added new localization strings for memory-related UI elements.
feat: add memory permissions management
- Implemented memory permissions in the backend, allowing roles to have specific permissions for using, creating, updating, and reading memories.
- Added new API endpoints for updating memory permissions associated with roles.
- Created a new AdminSettings component for managing memory permissions in the frontend.
- Integrated memory permissions into the existing roles and permissions schemas.
- Updated the interface to include memory settings and permissions.
- Enhanced the MemoryViewer component to conditionally render admin settings based on user roles.
- Added localization support for memory permissions in the translation files.
feat: move AdminSettings component to a new position in MemoryViewer for better visibility
refactor: clean up commented code in MemoryViewer component
feat: enhance MemoryViewer with search functionality and improve MemoryEditDialog integration
- Added a search input to filter memories in the MemoryViewer component.
- Refactored MemoryEditDialog to accept children for better customization.
- Updated MemoryViewer to utilize the new EditMemoryButton and DeleteMemoryButton components for editing and deleting memories.
- Improved localization support by adding new strings for memory filtering and deletion confirmation.
refactor: optimize memory filtering in MemoryViewer using match-sorter
- Replaced manual filtering logic with match-sorter for improved search functionality.
- Enhanced performance and readability of the filteredMemories computation.
feat: enhance MemoryEditDialog with triggerRef and improve updateMemory mutation handling
feat: implement access control for MemoryEditDialog and MemoryViewer components
refactor: remove commented out code and create runMemory method
refactor: rename role based files
feat: implement access control for memory usage in AgentClient
refactor: simplify checkVisionRequest method in AgentClient by removing commented-out code
refactor: make `agents` dir in api package
refactor: migrate Azure utilities to TypeScript and consolidate imports
refactor: move sanitizeFilename function to a new file and update imports, add related tests
refactor: update LLM configuration types and consolidate Azure options in the API package
chore: linting
chore: import order
refactor: replace getLLMConfig with getOpenAIConfig and remove unused LLM configuration file
chore: update winston-daily-rotate-file to version 5.0.0 and add object-hash dependency in package-lock.json
refactor: move primeResources and optionalChainWithEmptyCheck functions to resources.ts and update imports
refactor: move createRun function to a new run.ts file and update related imports
fix: ensure safeAttachments is correctly typed as an array of TFile
chore: add node-fetch dependency and refactor fetch-related functions into packages/api/utils, removing the old generators file
refactor: enhance TEndpointOption type by using Pick to streamline endpoint fields and add new properties for model parameters and client options
feat: implement initializeOpenAIOptions function and update OpenAI types for enhanced configuration handling
fix: update types due to new TEndpointOption typing
fix: ensure safe access to group parameters in initializeOpenAIOptions function
fix: remove redundant API key validation comment in initializeOpenAIOptions function
refactor: rename initializeOpenAIOptions to initializeOpenAI for consistency and update related documentation
refactor: decouple req.body fields and tool loading from initializeAgentOptions
chore: linting
refactor: adjust column widths in MemoryViewer for improved layout
refactor: simplify agent initialization by creating loadAgent function and removing unused code
feat: add memory configuration loading and validation functions
WIP: first pass, memory processing with config
feat: implement memory callback and artifact handling
feat: implement memory artifacts display and processing updates
feat: add memory configuration options and schema validation for validKeys
fix: update MemoryEditDialog and MemoryViewer to handle memory state and display improvements
refactor: remove padding from BookmarkTable and MemoryViewer headers for consistent styling
WIP: initial tokenLimit config and move Tokenizer to @librechat/api
refactor: update mongoMeili plugin methods to use callback for better error handling
feat: enhance memory management with token tracking and usage metrics
- Added token counting for memory entries to enforce limits and provide usage statistics.
- Updated memory retrieval and update routes to include total token usage and limit.
- Enhanced MemoryEditDialog and MemoryViewer components to display memory usage and token information.
- Refactored memory processing functions to handle token limits and provide feedback on memory capacity.
feat: implement memory artifact handling in attachment handler
- Enhanced useAttachmentHandler to process memory artifacts when receiving updates.
- Introduced handleMemoryArtifact utility to manage memory updates and deletions.
- Updated query client to reflect changes in memory state based on incoming data.
refactor: restructure web search key extraction logic
- Moved the logic for extracting API keys from the webSearchAuth configuration into a dedicated function, getWebSearchKeys.
- Updated webSearchKeys to utilize the new function for improved clarity and maintainability.
- Prevents build time errors
feat: add personalization settings and memory preferences management
- Introduced a new Personalization tab in settings to manage user memory preferences.
- Implemented API endpoints and client-side logic for updating memory preferences.
- Enhanced user interface components to reflect personalization options and memory usage.
- Updated permissions to allow users to opt out of memory features.
- Added localization support for new settings and messages related to personalization.
style: personalization switch class
feat: add PersonalizationIcon and align Side Panel UI
feat: implement memory creation functionality
- Added a new API endpoint for creating memory entries, including validation for key and value.
- Introduced MemoryCreateDialog component for user interface to facilitate memory creation.
- Integrated token limit checks to prevent exceeding user memory capacity.
- Updated MemoryViewer to include a button for opening the memory creation dialog.
- Enhanced localization support for new messages related to memory creation.
feat: enhance message processing with configurable window size
- Updated AgentClient to use a configurable message window size for processing messages.
- Introduced messageWindowSize option in memory configuration schema with a default value of 5.
- Improved logic for selecting messages to process based on the configured window size.
chore: update librechat-data-provider version to 0.7.87 in package.json and package-lock.json
chore: remove OpenAPIPlugin and its associated tests
chore: remove MIGRATION_README.md as migration tasks are completed
ci: fix backend tests
chore: remove unused translation keys from localization file
chore: remove problematic test file and unused var in AgentClient
chore: remove unused import and import directly for JSDoc
* feat: add api package build stage in Dockerfile for improved modularity
* docs: reorder build steps in contributing guide for clarity
2025-06-07 18:52:22 -04:00
|
|
|
"winston-daily-rotate-file": "^5.0.0",
|
📄 feat: Local Text Extraction for PDF, DOCX, and XLS/XLSX (#11900)
* feat: Added "document parser" OCR strategy
The document parser uses libraries to parse the text out of known document types.
This lets LibreChat handle some complex document types without having to use a
secondary service (like Mistral or standing up a RAG API server).
To enable the document parser, set the ocr strategy to "document_parser" in
librechat.yaml.
We now support:
- PDFs using pdfjs
- DOCX using mammoth
- XLS/XLSX using SheetJS
(The associated packages were also added to the project.)
* fix: applied Copilot code review suggestions
- Properly calculate length of text based on UTF8.
- Avoid issues with loading / blocking PDF parsing.
* fix: improved docs on parseDocument()
* chore: move to packages/api for TS support
* refactor: make document processing the default ocr strategy
- Introduced support for additional document types in the OCR strategy, including PDF, DOCX, and XLS/XLSX.
- Updated the file upload handling to dynamically select the appropriate parsing strategy based on the file type.
- Refactored the document parsing functions to use asynchronous imports for improved performance and maintainability.
* test: add unit tests for processAgentFileUpload functionality
- Introduced a new test suite for the processAgentFileUpload function in process.spec.js.
- Implemented various test cases to validate OCR strategy selection based on file types, including PDF, DOCX, XLSX, and XLS.
- Mocked dependencies to ensure isolated testing of file upload handling and strategy selection logic.
- Enhanced coverage for scenarios involving OCR capability checks and default strategy fallbacks.
* chore: update pdfjs-dist version and enhance document parsing tests
- Bumped pdfjs-dist dependency to version 5.4.624 in both api and packages/api.
- Refactored document parsing tests to use 'originalname' instead of 'filename' for file objects.
- Added a new test case for parsing XLS files to improve coverage of document types supported by the parser.
- Introduced a sample XLS file for testing purposes.
* feat: enforce text size limit and improve OCR fallback handling in processAgentFileUpload
- Added a check to ensure extracted text does not exceed the 15MB storage limit, throwing an error if it does.
- Refactored the OCR handling logic to improve fallback behavior when the configured OCR fails, ensuring a more robust document processing flow.
- Enhanced unit tests to cover scenarios for oversized text and fallback mechanisms, ensuring proper error handling and functionality.
* fix: correct OCR URL construction in performOCR function
- Updated the OCR URL construction to ensure it correctly appends '/ocr' to the base URL if not already present, improving the reliability of the OCR request.
---------
Co-authored-by: Dan Lew <daniel@mightyacorn.com>
2026-02-22 14:22:45 -05:00
|
|
|
"xlsx": "https://cdn.sheetjs.com/xlsx-0.20.3/xlsx-0.20.3.tgz",
|
💣 fix: Harden against falsified ZIP metadata in ODT parsing (#12320)
* security: replace JSZip metadata guard with yauzl streaming decompression
The ODT decompressed-size guard was checking JSZip's private
_data.uncompressedSize fields, which are populated from the ZIP central
directory — attacker-controlled metadata. A crafted ODT with falsified
uncompressedSize values bypassed the 50MB cap entirely, allowing
content.xml decompression to exhaust Node.js heap memory (DoS).
Replace JSZip with yauzl for ODT extraction. The new extractOdtContentXml
function uses yauzl's streaming API: it lazily iterates ZIP entries,
opens a decompression stream for content.xml, and counts real bytes as
they arrive from the inflate stream. The stream is destroyed the moment
the byte count crosses ODT_MAX_DECOMPRESSED_SIZE, aborting the inflate
before the full payload is materialised in memory.
- Remove jszip from direct dependencies (still transitive via mammoth)
- Add yauzl + @types/yauzl
- Update zip-bomb test to verify streaming abort with DEFLATE payload
* fix: close file descriptor leaks and declare jszip test dependency
- Use a shared `finish()` helper in extractOdtContentXml that calls
zipfile.close() on every exit path (success, size cap, missing entry,
openReadStream errors, zipfile errors). Without this, any error path
leaked one OS file descriptor permanently — uploading many malformed
ODTs could exhaust the process FD limit (a distinct DoS vector).
- Add jszip to devDependencies so the zip-bomb test has an explicit
dependency rather than relying on mammoth's transitive jszip.
- Update JSDoc to document that all exit paths close the zipfile.
* fix: move yauzl from dependencies to peerDependencies
Matches the established pattern for runtime parser libraries in
packages/api: mammoth, pdfjs-dist, and xlsx are all peerDependencies
(provided by the consuming /api workspace) with devDependencies for
testing. yauzl was incorrectly placed in dependencies.
* fix: add yauzl to /api dependencies to satisfy peer dep
packages/api declares yauzl as a peerDependency; /api is the consuming
workspace that must provide it at runtime, matching the pattern used
for mammoth, pdfjs-dist, and xlsx.
2026-03-19 22:13:40 -04:00
|
|
|
"yauzl": "^3.2.1",
|
feat: Accurate Token Usage Tracking & Optional Balance (#1018)
* refactor(Chains/llms): allow passing callbacks
* refactor(BaseClient): accurately count completion tokens as generation only
* refactor(OpenAIClient): remove unused getTokenCountForResponse, pass streaming var and callbacks in initializeLLM
* wip: summary prompt tokens
* refactor(summarizeMessages): new cut-off strategy that generates a better summary by adding context from beginning, truncating the middle, and providing the end
wip: draft out relevant providers and variables for token tracing
* refactor(createLLM): make streaming prop false by default
* chore: remove use of getTokenCountForResponse
* refactor(agents): use BufferMemory as ConversationSummaryBufferMemory token usage not easy to trace
* chore: remove passing of streaming prop, also console log useful vars for tracing
* feat: formatFromLangChain helper function to count tokens for ChatModelStart
* refactor(initializeLLM): add role for LLM tracing
* chore(formatFromLangChain): update JSDoc
* feat(formatMessages): formats langChain messages into OpenAI payload format
* chore: install openai-chat-tokens
* refactor(formatMessage): optimize conditional langChain logic
fix(formatFromLangChain): fix destructuring
* feat: accurate prompt tokens for ChatModelStart before generation
* refactor(handleChatModelStart): move to callbacks dir, use factory function
* refactor(initializeLLM): rename 'role' to 'context'
* feat(Balance/Transaction): new schema/models for tracking token spend
refactor(Key): factor out model export to separate file
* refactor(initializeClient): add req,res objects to client options
* feat: add-balance script to add to an existing users' token balance
refactor(Transaction): use multiplier map/function, return balance update
* refactor(Tx): update enum for tokenType, return 1 for multiplier if no map match
* refactor(Tx): add fair fallback value multiplier incase the config result is undefined
* refactor(Balance): rename 'tokens' to 'tokenCredits'
* feat: balance check, add tx.js for new tx-related methods and tests
* chore(summaryPrompts): update prompt token count
* refactor(callbacks): pass req, res
wip: check balance
* refactor(Tx): make convoId a String type, fix(calculateTokenValue)
* refactor(BaseClient): add conversationId as client prop when assigned
* feat(RunManager): track LLM runs with manager, track token spend from LLM,
refactor(OpenAIClient): use RunManager to create callbacks, pass user prop to langchain api calls
* feat(spendTokens): helper to spend prompt/completion tokens
* feat(checkBalance): add helper to check, log, deny request if balance doesn't have enough funds
refactor(Balance): static check method to return object instead of boolean now
wip(OpenAIClient): implement use of checkBalance
* refactor(initializeLLM): add token buffer to assure summary isn't generated when subsequent payload is too large
refactor(OpenAIClient): add checkBalance
refactor(createStartHandler): add checkBalance
* chore: remove prompt and completion token logging from route handler
* chore(spendTokens): add JSDoc
* feat(logTokenCost): record transactions for basic api calls
* chore(ask/edit): invoke getResponseSender only once per API call
* refactor(ask/edit): pass promptTokens to getIds and include in abort data
* refactor(getIds -> getReqData): rename function
* refactor(Tx): increase value if incomplete message
* feat: record tokenUsage when message is aborted
* refactor: subtract tokens when payload includes function_call
* refactor: add namespace for token_balance
* fix(spendTokens): only execute if corresponding token type amounts are defined
* refactor(checkBalance): throws Error if not enough token credits
* refactor(runTitleChain): pass and use signal, spread object props in create helpers, and use 'call' instead of 'run'
* fix(abortMiddleware): circular dependency, and default to empty string for completionTokens
* fix: properly cancel title requests when there isn't enough tokens to generate
* feat(predictNewSummary): custom chain for summaries to allow signal passing
refactor(summaryBuffer): use new custom chain
* feat(RunManager): add getRunByConversationId method, refactor: remove run and throw llm error on handleLLMError
* refactor(createStartHandler): if summary, add error details to runs
* fix(OpenAIClient): support aborting from summarization & showing error to user
refactor(summarizeMessages): remove unnecessary operations counting summaryPromptTokens and note for alternative, pass signal to summaryBuffer
* refactor(logTokenCost -> recordTokenUsage): rename
* refactor(checkBalance): include promptTokens in errorMessage
* refactor(checkBalance/spendTokens): move to models dir
* fix(createLanguageChain): correctly pass config
* refactor(initializeLLM/title): add tokenBuffer of 150 for balance check
* refactor(openAPIPlugin): pass signal and memory, filter functions by the one being called
* refactor(createStartHandler): add error to run if context is plugins as well
* refactor(RunManager/handleLLMError): throw error immediately if plugins, don't remove run
* refactor(PluginsClient): pass memory and signal to tools, cleanup error handling logic
* chore: use absolute equality for addTitle condition
* refactor(checkBalance): move checkBalance to execute after userMessage and tokenCounts are saved, also make conditional
* style: icon changes to match official
* fix(BaseClient): getTokenCountForResponse -> getTokenCount
* fix(formatLangChainMessages): add kwargs as fallback prop from lc_kwargs, update JSDoc
* refactor(Tx.create): does not update balance if CHECK_BALANCE is not enabled
* fix(e2e/cleanUp): cleanup new collections, import all model methods from index
* fix(config/add-balance): add uncaughtException listener
* fix: circular dependency
* refactor(initializeLLM/checkBalance): append new generations to errorMessage if cost exceeds balance
* fix(handleResponseMessage): only record token usage in this method if not error and completion is not skipped
* fix(createStartHandler): correct condition for generations
* chore: bump postcss due to moderate severity vulnerability
* chore: bump zod due to low severity vulnerability
* chore: bump openai & data-provider version
* feat(types): OpenAI Message types
* chore: update bun lockfile
* refactor(CodeBlock): add error block formatting
* refactor(utils/Plugin): factor out formatJSON and cn to separate files (json.ts and cn.ts), add extractJSON
* chore(logViolation): delete user_id after error is logged
* refactor(getMessageError -> Error): change to React.FC, add token_balance handling, use extractJSON to determine JSON instead of regex
* fix(DALL-E): use latest openai SDK
* chore: reorganize imports, fix type issue
* feat(server): add balance route
* fix(api/models): add auth
* feat(data-provider): /api/balance query
* feat: show balance if checking is enabled, refetch on final message or error
* chore: update docs, .env.example with token_usage info, add balance script command
* fix(Balance): fallback to empty obj for balance query
* style: slight adjustment of balance element
* docs(token_usage): add PR notes
2023-10-05 18:34:10 -04:00
|
|
|
"zod": "^3.22.4"
|
2023-03-06 10:15:07 -05:00
|
|
|
},
|
|
|
|
|
"devDependencies": {
|
2025-11-16 11:55:18 -05:00
|
|
|
"jest": "^30.2.0",
|
2025-07-25 11:23:38 -04:00
|
|
|
"mongodb-memory-server": "^10.1.4",
|
2025-02-11 15:55:13 +01:00
|
|
|
"nodemon": "^3.0.3",
|
2025-05-01 15:02:51 -04:00
|
|
|
"supertest": "^7.1.0"
|
2023-03-06 10:15:07 -05:00
|
|
|
}
|
|
|
|
|
}
|