* 🚀 feat: gpt-4o
* update readme.md
* feat: Add new test case for getMultiplier function
* feat: Refactor getMultiplier function to use valueKey variable
* chore: make frequent 'error' log into 'debug' log
* feat: add maxContextTokens as a conversation field
* refactor(settings): increase popover height
* feat: add DynamicInputNumber and maxContextTokens to all endpoints that support it (frontend), fix schema
* feat: maxContextTokens handling (backend)
* style: revert popover height
* feat: max tokens
* fix: Ollama Vision firebase compatibility
* fix: Ollama Vision, use message_file_map to determine multimodal request
* refactor: bring back MobileNav and improve title styling
* 🔧chore: add internationalization labels for archive feature
* ✨ feat: Add function to useArchiveConversationMutation()
This commit adds a new mutation function `useArchiveConversationMutation()` for archiving conversations. This function takes the ID string of the conversation to be archived and returns a mutation result object. Upon successful archiving, it removes and refreshes the conversation from the query data cache.
While ChatGPT PATCHes the archived status by sending `{is_archived: true}` to the URL `/backend-api/conversation/$conversation_id`, this implementation uses the `dataService.updateConversation(payload)` with a POST method, aligning with the existing code conventions.
* ✨ feat(api): add is_archived field to Conversation schema and update getConvosByPage method
This commit adds a new field `is_archived` with a default value of false to the Conversation schema. It also modifies the `getConvosByPage` method within the Conversation API to adjust the query to only target conversations where `is_archived` is set to false or where the `is_archived` field does not exist. The function `getConvosQueried`, which returns conversations for a specified Conversation ID, was determined not to require consideration of whether `is_archived` is true or false, and thus was not modified.
* ♻️ refactor: add className prop to DotsIcon component
To enhance the versatility of the DotsIcon component, this commit introduces the ability to specify a className prop, allowing for greater customization.
* ✨ feat(ui): add Edit Button to group Title change and Conversation delete buttons
Added a new Edit Button to the conversations, similar to the ChatGPT UI, which groups options for editing the conversation title and deleting conversations. This grouping is accessible through a dialogue that appears when the three-dot icon is clicked.
* ♻️ refactor(ui): enhance Delete Button to accept className and label options
Enhanced the Delete Button component to accept a `className` for customization and an optional `appendLabel`. The DeleteButton component is used by both `Convo.tsx` and `Conversation.tsx`, but currently only `Convo.tsx` is active and `Conversation.tsx `is apparently not used; removing `Conversation.tsx` may eliminate the need for the `appendLabel` property in the future.
* ♻️ refactor(ui): enhance RenameButton to accept label options
Added the ability to optionally display labels; the Rename Button component is used by both `Convo.tsx` and `Conversation.tsx`, but currently only `Convo.tsx` is active and `Conversation.tsx `is apparently not used; removing `Conversation.tsx` may eliminate the need for the `appendLabel` property in the future.
* 🔧 chors: additional localization labels
* ♻️ refactor: change is_archived property of conversation to camelCase
* Refactor the is_archived property of conversation to camelCase (isArchived) to adhere to the existing code conventions
* Modify the function that retrieves conversations to accept the isArchived parameter
* ♻️ refactor: add archiveConversation mutation
I thought I could divert dataService.updateConversation, but added a new archiveConversation because the request types are different. It might be better to make them common, but to avoid side effects, I added a new function this time.
Added process to deleteConversationMutation to delete archived conversations
* ✨ feat: Add the function to hide a cancel button in DialogTemplate component
The Cancel button is not needed when displaying the archive list, so I made the Cancel button optional.
* ♻️ refactor: Add support for filtering archived conversations in Nav component
This commit modifies the Nav component to add the ability to filter out archived conversations when fetching data. This is done by adding `isArchived: false` to the query parameters for both the `useConversationsInfiniteQuery()` and `useSearchInfiniteQuery()` hooks, effectively excluding any archived conversations from the results returned.
* ♻️ refactor: add Tooltip to DeleteButton
* Add Tooltip to DeleteButton component
* Display Tooltip when DeleteButton only shows an Icon without text
* ✨ feat(ui): add ArchiveButton component for archiving conversations
To be compatible with the ChatGPT UI, no confirmation dialog is displayed when ArchiveButton is clicked. The basic behavior conforms to DeleteButton and RenameButton.
* ✨ feat(ui): add Archive button to list of conversations
Modify the Nav of the conversation list to include a dropdown that contains the Rename and Delete options, similar to the ChatGPT UI. Additionally, an Archive button has been added adjacent to the dropdown menu.
* ✨ feat: Add ArchivedChatsTable component
Adds the `ArchivedChatsTable` component, which displays a table of archived chats. It has been implemented to be as compatible with the ChatGPT UI as possible.
* 🚑 fix(tooltip): increase z-index to ensure visibility over Dialog
Resolve an issue where tooltips were not visible when displayed over a Dialog. The z-index of `DialogPrimitive.Portal` in `Dialog.tsx` is set to 999. Since the rationale for this value is unclear, the z-index of the tooltip has been increased to 1000 to guarantee its visibility above the Dialog component.
* 🔧 chors: add internationalization labels
* typedef for ImportBatchBuilder
* feat: first pass, fork conversations
* feat: fork - getMessagesUpToTargetLevel
* fix: additional tests and fix getAllMessagesUpToParent
* chore: arrow function return
* refactor: fork 3 options
* chore: remove unused genbuttons
* chore: remove unused hover buttons code
* feat: fork first pass
* wip: fork remember setting
* style: user icon
* chore: move clear chats to data tab
* WIP: fork UI options
* feat: data-provider fork types/services/vars and use generic MutationOptions
* refactor: use single param for fork option, use enum, fix mongo errors, use Date.now(), add records flag for testing, use endpoint from original convo and messages, pass originalConvo to finishConversation
* feat: add fork mutation hook and consolidate type imports
* refactor: use enum
* feat: first pass, fork mutation
* chore: add enum for target level fork option
* chore: add enum for target level fork option
* show toast when checking remember selection
* feat: splitAtTarget
* feat: split at target option
* feat: navigate to new fork, show toasts, set result query data
* feat: hover info for all fork options
* refactor: add Messages settings tab
* fix(Fork): remember text info
* ci: test for single message and is target edge case
* feat: additional tests for getAllMessagesUpToParent
* ci: additional tests and cycle detection for getMessagesUpToTargetLevel
* feat: circular dependency checks for getAllMessagesUpToParent
* fix: getMessagesUpToTargetLevel circular dep. check
* ci: more tests for getMessagesForConversation
* style: hover text for checkbox fork items
* refactor: add statefulness to conversation import
* Basic implementation of ChatGPT conversation import
* remove debug code
* Handle citations
* Fix updatedAt in import
* update default model
* Use job scheduler to handle import requests
* import job status endpoint
* Add wrapper around Agenda
* Rate limits for import endpoint
* rename import api path
* Batch save import to mongo
* Improve naming
* Add documenting comments
* Test for importers
* Change button for importing conversations
* Frontend changes
* Import job status endpoint
* Import endpoint response
* Add translations to new phrases
* Fix conversations refreshing
* cleanup unused functions
* set timeout for import job status polling
* Add documentation
* get extra spaces back
* Improve error message
* Fix translation files after merge
* fix translation files 2
* Add zh translation for import functionality
* Sync mailisearch index after import
* chore: add dummy uri for jest tests, as MONGO_URI should only be real for E2E tests
* docs: fix links
* docs: fix conversationsImport section
* fix: user role issue for librechat imports
* refactor: import conversations from json
- organize imports
- add additional jsdocs
- use multer with diskStorage to avoid loading file into memory outside of job
- use filepath instead of loading data string for imports
- replace console logs and some logger.info() with logger.debug
- only use multer for import route
* fix: undefined metadata edge case and replace ChatGtp -> ChatGpt
* Refactor importChatGptConvo function to handle undefined metadata edge case and replace ChatGtp with ChatGpt
* fix: chatgpt importer
* feat: maintain tree relationship for librechat messages
* chore: use enum
* refactor: saveMessage to use single object arg, replace console logs, add userId to log message
* chore: additional comment
* chore: multer edge case
* feat: first pass, maintain tree relationship
* chore: organize
* chore: remove log
* ci: add heirarchy test for chatgpt
* ci: test maintaining of heirarchy for librechat
* wip: allow non-text content type messages
* refactor: import content part object json string
* refactor: more content types to format
* chore: consolidate messageText formatting
* docs: update on changes, bump data-provider/config versions, update readme
* refactor(indexSync): singleton pattern for MeiliSearchClient
* refactor: debug log after batch is done
* chore: add back indexSync error handling
---------
Co-authored-by: jakubmieszczak <jakub.mieszczak@zendesk.com>
Co-authored-by: Danny Avila <danny@librechat.ai>
* WIP: first pass ModelSpecs
* refactor(onSelectEndpoint): use `getConvoSwitchLogic`
* feat: introduce iconURL, greeting, frontend fields for conversations/presets/messages
* feat: conversation.iconURL & greeting in Landing
* feat: conversation.iconURL & greeting in New Chat button
* feat: message.iconURL
* refactor: ConversationIcon -> ConvoIconURL
* WIP: add spec as a conversation field
* refactor: useAppStartup, set spec on initial load for new chat, allow undefined spec, add localStorage keys enum, additional type fields for spec
* feat: handle `showIconInMenu`, `showIconInHeader`, undefined `iconURL` and no specs on initial load
* chore: handle undefined or empty modelSpecs
* WIP: first pass, modelSpec schema for custom config
* refactor: move default filtered tools definition to ToolService
* feat: pass modelSpecs from backend via startupConfig
* refactor: modelSpecs config, return and define list
* fix: react error and include iconURL in responseMessage
* refactor: add iconURL to responseMessage only
* refactor: getIconEndpoint
* refactor: pass TSpecsConfig
* fix(assistants): differentiate compactAssistantSchema, correctly resets shared conversation state with other endpoints
* refactor: assistant id prefix localStorage key
* refactor: add more LocalStorageKeys and replace hardcoded values
* feat: prioritize spec on new chat behavior: last selected modelSpec behavior (localStorage)
* feat: first pass, interface config
* chore: WIP, todo: add warnings based on config.modelSpecs settings.
* feat: enforce modelSpecs if configured
* feat: show config file yaml errors
* chore: delete unused legacy Plugins component
* refactor: set tools to localStorage from recoil store
* chore: add stable recoil setter to useEffect deps
* refactor: save tools to conversation documents
* style(MultiSelectPop): dynamic height, remove unused import
* refactor(react-query): use localstorage keys and pass config to useAvailablePluginsQuery
* feat(utils): add mapPlugins
* refactor(Convo): use conversation.tools if defined, lastSelectedTools if not
* refactor: remove unused legacy code using `useSetOptions`, remove conditional flag `isMultiChat` for using legacy settings
* refactor(PluginStoreDialog): add exhaustive-deps which are stable react state setters
* fix(HeaderOptions): pass `popover` as true
* refactor(useSetStorage): use project enums
* refactor: use LocalStorageKeys enum
* fix: prevent setConversation from setting falsy values in lastSelectedTools
* refactor: use map for availableTools state and available Plugins query
* refactor(updateLastSelectedModel): organize logic better and add note on purpose
* fix(setAgentOption): prevent reseting last model to secondary model for gptPlugins
* refactor(buildDefaultConvo): use enum
* refactor: remove `useSetStorage` and consolidate areas where conversation state is saved to localStorage
* fix: conversations retain tools on refresh
* fix(gptPlugins): prevent nullish tools from being saved
* chore: delete useServerStream
* refactor: move initial plugins logic to useAppStartup
* refactor(MultiSelectDropDown): add more pass-in className props
* feat: use tools in presets
* chore: delete unused usePresetOptions
* refactor: new agentOptions default handling
* chore: note
* feat: add label and custom instructions to agents
* chore: remove 'disabled with tools' message
* style: move plugins to 2nd column in parameters
* fix: TPreset type for agentOptions
* fix: interface controls
* refactor: add interfaceConfig, use Separator within Switcher
* refactor: hide Assistants panel if interface.parameters are disabled
* fix(Header): only modelSpecs if list is greater than 0
* refactor: separate MessageIcon logic from useMessageHelpers for better react rule-following
* fix(AppService): don't use reserved keyword 'interface'
* feat: set existing Icon for custom endpoints through iconURL
* fix(ci): tests passing for App Service
* docs: refactor custom_config.md for readability and better organization, also include missing values
* docs: interface section and re-organize docs
* docs: update modelSpecs info
* chore: remove unused files
* chore: remove unused files
* chore: move useSetIndexOptions
* chore: remove unused file
* chore: move useConversation(s)
* chore: move useDefaultConvo
* chore: move useNavigateToConvo
* refactor: use plugin install hook so it can be used elsewhere
* chore: import order
* update docs
* refactor(OpenAI/Plugins): allow modelLabel as an initial value for chatGptLabel
* chore: remove unused EndpointOptionsPopover and hide 'Save as Preset' button if preset UI visibility disabled
* feat(loadDefaultInterface): issue warnings based on values
* feat: changelog for custom config file
* docs: add additional changelog note
* fix: prevent unavailable tool selection from preset and update availableTools on Plugin installations
* feat: add `filteredTools` option in custom config
* chore: changelog
* fix(MessageIcon): always overwrite conversation.iconURL in messageSettings
* fix(ModelSpecsMenu): icon edge cases
* fix(NewChat): dynamic icon
* fix(PluginsClient): always include endpoint in responseMessage
* fix: always include endpoint and iconURL in responseMessage across different response methods
* feat: interchangeable keys for modelSpec enforcing
* feat: `stop` conversation parameter
* feat: Tag primitive
* feat: dynamic tags
* refactor: update tag styling
* feat: add stop sequences to OpenAI settings
* fix(Presentation): prevent `SidePanel` re-renders that flicker side panel
* refactor: use stop placeholder
* feat: type and schema update for `stop` and `TPreset` in generation param related types
* refactor: pass conversation to dynamic settings
* refactor(OpenAIClient): remove default handling for `modelOptions.stop`
* docs: fix Google AI Setup formatting
* feat: current_model
* docs: WIP update
* fix(ChatRoute): prevent default preset override before `hasSetConversation.current` becomes true by including latest conversation state as template
* docs: update docs with more info on `stop`
* chore: bump config_version
* refactor: CURRENT_MODEL handling
* chore: replace violation cache accessors with enum
* chore: fix test
* chore(fileSchema): index timestamps
* fix(ActionService): use encoding/caching strategy for handling assistant function character length limit
* refactor(actions): async `domainParser` also resolve retrieved model (which is deployment name) to user-defined model
* style(AssistantAction): add `whitespace-nowrap` for ellipsis
* refactor(ActionService): if domain is less than or equal to encoded domain fixed length, return domain with replacement of separator
* refactor(actions): use sessions/transactions for updating Assistant Action database records
* chore: remove TTL from ENCODED_DOMAINS cache
* refactor(domainParser): minor optimization and add tests
* fix(spendTokens): use txData.user for token usage logging
* refactor(actions): add helper function `withSession` for database operations with sessions/transactions
* fix(PluginsClient): logger debug `message` field edge case
* WIP: gemini-1.5 support
* feat: extended vertex ai support
* fix: handle possibly undefined modelName
* fix: gpt-4-turbo-preview invalid vision model
* feat: specify `fileConfig.imageOutputType` and make PNG default image conversion type
* feat: better truncation for errors including base64 strings
* fix: gemini inlineData formatting
* feat: RAG augmented prompt for gemini-1.5
* feat: gemini-1.5 rates and token window
* chore: adjust tokens, update docs, update vision Models
* chore: add back `ChatGoogleVertexAI` for chat models via vertex ai
* refactor: ask/edit controllers to not use `unfinished` field for google endpoint
* chore: remove comment
* chore(ci): fix AppService test
* chore: remove comment
* refactor(GoogleSearch): use `GOOGLE_SEARCH_API_KEY` instead, issue warning for old variable
* chore: bump data-provider to 0.5.4
* chore: update docs
* fix: condition for gemini-1.5 using generative ai lib
* chore: update docs
* ci: add additional AppService test for `imageOutputType`
* refactor: optimize new config value `imageOutputType`
* chore: bump CONFIG_VERSION
* fix(assistants): avatar upload
* fix(processModelData): handle `openrouter/auto` edge case
* fix(Tx.create): prevent negative multiplier edge case and prevent balance from becoming negative
* fix(NavLinks): render 0 balance properly
* refactor(NavLinks): show only up to 2 decimal places for balance
* fix(OpenAIClient/titleConvo): fix cohere condition and record token usage for `this.options.titleMethod === 'completion'`
* WIP: basic route for file downloads and file strategy for generating readablestream to pipe as res
* chore(DALLE3): add typing for OpenAI client
* chore: add `CONSOLE_JSON` notes to dotenv.md
* WIP: first pass OpenAI Assistants File Output handling
* feat: first pass assistants output file download from openai
* chore: yml vs. yaml variation to .gitignore for `librechat.yml`
* refactor(retrieveAndProcessFile): remove redundancies
* fix(syncMessages): explicit sort of apiMessages to fix message order on abort
* chore: add logs for warnings and errors, show toast on frontend
* chore: add logger where console was still being used
* feat: add claude-3-haiku-20240307 to default anthropic list
* refactor: optimize `saveMessage` calls mid-stream via throttling
* chore: remove addMetadata operations and consolidate in BaseClient
* fix(listAssistantsForAzure): attempt to specify correct model mapping as accurately as possible (#2177)
* refactor(client): update last conversation setup with current assistant model, call newConvo again when assistants load to allow fast initial load and ensure assistant model is always the default, not the last selected model
* refactor(cache): explicitly add TTL of 2 minutes when setting titleCache and add default TTL of 10 minutes to abortKeys cache
* feat(AnthropicClient): conversation titling using Anthropic Function Calling
* chore: remove extraneous token usage logging
* fix(convos): unhandled edge case for conversation grouping (undefined conversation)
* style: Improved style of Search Bar after recent UI update
* chore: remove unused code, content part helpers
* feat: always show code option
* refactor: re-purpose `resendImages` as `resendFiles`
* refactor: re-purpose `resendImages` as `resendFiles`
* feat: upload general files
* feat: embed file during upload
* feat: delete file embeddings on file deletion
* chore(fileConfig): add epub+zip type
* feat(encodeAndFormat): handle non-image files
* feat(createContextHandlers): build context prompt from file attachments and successful RAG
* fix: prevent non-temp files as well as embedded files to be deleted on new conversation
* fix: remove temp_file_id on usage, prevent non-temp files as well as embedded files to be deleted on new conversation
* fix: prevent non-temp files as well as embedded files to be deleted on new conversation
* feat(OpenAI/Anthropic/Google): basic RAG support
* fix: delete `resendFiles` only when true (Default)
* refactor(RAG): update endpoints and pass JWT
* fix(resendFiles): default values
* fix(context/processFile): query unique ids only
* feat: rag-api.yaml
* feat: file upload improved ux for longer uploads
* chore: await embed call and catch embedding errors
* refactor: store augmentedPrompt in Client
* refactor(processFileUpload): throw error if not assistant file upload
* fix(useFileHandling): handle markdown empty mimetype issue
* chore: necessary compose file changes
* fix: remove unique field from assistant_id, which can be shared between different users
* refactor: remove unique user fields from actions/assistant queries
* feat: only allow user who saved action to delete it
* refactor: allow deletions for anyone with builder access
* refactor: update user.id when updating assistants/actions records, instead of searching with it
* fix: stringify response data in case it's an object
* fix: correctly handle path input
* fix(decryptV2): handle edge case where value is already decrypted
* chore: add assistants to supportsBalanceCheck
* feat(Transaction): getTransactions and refactor export of model
* refactor: use enum: ViolationTypes.TOKEN_BALANCE
* feat(assistants): check balance
* refactor(assistants): only add promptBuffer if new convo (for title), and remove endpoint definition
* refactor(assistants): Count tokens up to the current context window
* fix(Switcher): make Select list explicitly controlled
* feat(assistants): use assistant's default model when no model is specified instead of the last selected assistant, prevent assistant_id from being recorded in non-assistant endpoints
* chore(assistants/chat): import order
* chore: bump librechat-data-provider due to changes
* chore: bump anthropic SDK
* chore: update anthropic config settings (fileSupport, default models)
* feat: anthropic multi modal formatting
* refactor: update vision models and use endpoint specific max long side resizing
* feat(anthropic): multimodal messages, retry logic, and messages payload
* chore: add more safety to trimming content due to whitespace error for assistant messages
* feat(anthropic): token accounting and resending multiple images in progress
* chore: bump data-provider
* feat(anthropic): resendImages feature
* chore: optimize Edit/Ask controllers, switch model back to req model
* fix: false positive of invalid model
* refactor(validateVisionModel): use object as arg, pass in additional/available models
* refactor(validateModel): use helper function, `getModelsConfig`
* feat: add modelsConfig to endpointOption so it gets passed to all clients, use for properly validating vision models
* refactor: initialize default vision model and make sure it's available before assigning it
* refactor(useSSE): avoid resetting model if user selected a new model between request and response
* feat: show rate in transaction logging
* fix: return tokenCountMap regardless of payload shape
* feat: assistant name/icon in Landing & Header
* feat: assistname in textarea placeholder, and use `Assistant` as default name
* feat: display non-image files in user messages
* fix: only render files if files.length is > 0
* refactor(config -> file-config): move file related configuration values to separate module, add excel types
* chore: spreadsheet file rendering
* fix(Landing): dark mode style for Assistant Name
* refactor: move progress incrementing to own hook, start smaller, cap near limit \(1\)
* refactor(useContentHandler): add empty Text part if last part was completed tool or image
* chore: add accordion trigger border styling for dark mode
* feat: Assistant Builder model selection
* chore: use Spinner when Assistant is mutating
* fix(get/assistants): return correct response object `AssistantListResponse`
* refactor(Spinner): pass size as prop
* refactor: make assistant crud mutations optimistic, add types for options
* chore: remove assistants route and view
* chore: move assistant builder components to separate directory
* feat(ContextButton): delete Assistant via context button/dialog, add localization
* refactor: conditionally show use and context menu buttons, add localization for create assistant
* feat: save side panel states to localStorage
* style(SidePanel): improve avatar menu and assistant select styling for dark mode
* refactor: make NavToggle reusable for either side (left or right), add SidePanel Toggle with ability to close it completely
* fix: resize handle and navToggle behavior
* fix(/avatar/:assistant_id): await `deleteFile` and assign unique name to uploaded image
* WIP: file UI components from PR #576
* refactor(OpenAIMinimalIcon): pass className
* feat: formatDate helper fn
* feat: DataTableColumnHeader
* feat: add row selection, formatted row values, number of rows selected
* WIP: add files to Side panel temporarily
* feat: `LB_QueueAsyncCall`: Leaky Bucket queue for external APIs, use in `processDeleteRequest`
* fix(TFile): correct `source` type with `FileSources`
* fix(useFileHandling): use `continue` instead of return when iterating multiple files, add file type to extendedFile
* chore: add generic setter type
* refactor(processDeleteRequest): settle promises to prevent rejections from processing deletions, log errors
* feat: `useFileDeletion` to reuse file deletion logic
* refactor(useFileDeletion): make `setFiles` an optional param and use object as param
* feat: useDeleteFilesFromTable
* feat: use real `files` data and add deletion action to data table
* fix(Table): make headers sticky
* feat: add dynamic filtering for columns; only show to user Host or OpenAI storage type
* style(DropdownMenu): replace `slate` with `gray`
* style(DataTable): apply dark mode themes and other misc styling
* style(Columns): add color to OpenAI Storage option
* refactor(FileContainer): make file preview reusable
* refactor(Images): make image preview reusable
* refactor(FilePreview): make file prop optional for FileIcon and FilePreview, fix relative style
* feat(Columns): add file/image previews, set a minimum size to show for file size in bytes
* WIP: File Panel with real files and formatted
* feat: open files dialog from panel
* style: file data table mobile and general column styling fixes
* refactor(api/files): return files sorted by the most recently updated
* refactor: provide fileMap through context to prevent re-selecting files to map in different areas; remove unused imports commented out in PanelColumns
* refactor(ExtendFile): make File type optional, add `attached` to prevent attached files from being deleted on remove, make Message.files a partial TFile type
* feat: attach files through file panel
* refactor(useFileHandling): move files to the start of cache list when uploaded
* refactor(useDeleteFilesMutation): delete files from cache when successfully deleted from server
* fix(FileRow): handle possible edge case of duplication due to attaching recently uploaded file
* style(SidePanel): make resize grip border transparent, remove unnecessary styling on close sidepanel button
* feat: action utilities and tests
* refactor(actions): add `ValidationResult` type and change wording for no server URL found
* refactor(actions): check for empty server URL
* fix(data-provider): revert tsconfig to fix type issue resolution
* feat(client): first pass of actions input for assistants
* refactor(FunctionSignature): change method to output object instead of string
* refactor(models/Assistant): add actions field to schema, use searchParams object for methods, and add `getAssistant`
* feat: post actions input first pass
- create new Action document
- add actions to Assistant DB document
- create /action/:assistant_id POST route
- pass more props down from PanelSwitcher, derive assistant_id from switcher
- move privacy policy to ActionInput
- reset data on input change/validation
- add `useUpdateAction`
- conform FunctionSignature type to FunctionTool
- add action, assistant doc, update hook related types
* refactor: optimize assistant/actions relationship
- past domain in metadata as hostname and not a URL
- include domain in tool name
- add `getActions` for actions retrieval by user
- add `getAssistants` for assistant docs retrieval by user
- add `assistant_id` to Action schema
- move actions to own module as a subroute to `api/assistants`
- add `useGetActionsQuery` and `useGetAssistantDocsQuery` hooks
- fix Action type def
* feat: show assistant actions in assistant builder
* feat: switch to actions on action click, editing action styling
* fix: add Assistant state for builder panel to allow immediate selection of newly created assistants as well as retaining the current assistant when switching to a different panel within the builder
* refactor(SidePanel/NavToggle): offset less from right when SidePanel is completely collapsed
* chore: rename `processActions` -> `processRequiredActions`
* chore: rename Assistant API Action to RequiredAction
* refactor(actions): avoid nesting actual API params under generic `requestBody` to optimize LLM token usage
* fix(handleTools): avoid calling `validTool` if not defined, add optional param to skip the loading of specs, which throws an error in the context of assistants
* WIP: working first pass of toolCalls generated from openapi specs
* WIP: first pass ToolCall styling
* feat: programmatic iv encryption/decryption helpers
* fix: correct ActionAuth types/enums, and define type for AuthForm
* feat: encryption/decryption helpers for Action AuthMetadata
* refactor(getActions): remove sensitive fields from query response
* refactor(POST/actions): encrypt and remove sensitive fields from mutation response
* fix(ActionService): change ESM import to CJS
* feat: frontend auth handling for actions + optimistic update on action update/creation
* refactor(actions): use the correct variables and types for setAuth method
* refactor: POST /:assistant_id action can now handle updating an existing action, add `saved_auth_fields` to determine when user explicitly saves new auth creds. only send auth metadata if user explicitly saved fields
* refactor(createActionTool): catch errors and send back meaningful error message, add flag to `getActions` to determine whether to retrieve sensitive values or not
* refactor(ToolService): add `action` property to ToolCall PartMetadata to determine if the tool call was an action, fix parsing function name issue with actionDelimiter
* fix(ActionRequest): use URL class to correctly join endpoint parts for `execute` call
* feat: delete assistant actions
* refactor: conditionally show Available actions
* refactor: show `retrieval` and `code_interpreter` as Capabilities, swap `Switch` for `Checkbox`
* chore: remove shadow-stroke from messages
* WIP: first pass of Assistants Knowledge attachments
* refactor: remove AssistantsProvider in favor of FormProvider, fix selectedAssistant re-render bug, map Assistant file_ids to files via fileMap, initialize Knowledge component with mapped files if any exist
* fix: prevent deleting files on assistant file upload
* chore: remove console.log
* refactor(useUploadFileMutation): update files and assistants cache on upload
* chore: disable oauth option as not supported yet
* feat: cancel assistant runs
* refactor: initialize OpenAI client with helper function, resolve all related circular dependencies
* fix(DALL-E): initialization
* fix(process): openai client initialization
* fix: select an existing Assistant when the active one is deleted
* chore: allow attaching files for assistant endpoint, send back relevant OpenAI error message when uploading, deconstruct openAI initialization correctly, add `message_file` to formData when a file is attached to the message but not the assistant
* fix: add assistant_id on newConvo
* fix(initializeClient): import fix
* chore: swap setAssistant for setOption in useEffect
* fix(DALL-E): add processFileURL to loadTools call
* chore: add customConfig to debug logs
* feat: delete threads on convo delete
* chore: replace Assistants icon
* chore: remove console.dir() in `abortRun`
* feat(AssistantService): accumulate text values from run in openai.responseText
* feat: titling for assistants endpoint
* chore: move panel file components to appropriate directory, add file checks for attaching files, change icon for Attach Files
* refactor: add localizations to tools, plugins, add condition for adding/remove user plugins so tool selections don't affect this value
* chore: disable `import from url` action for now
* chore: remove textMimeTypes from default fileConfig for now
* fix: catch tool errors and send as outputs with error messages
* fix: React warning about button as descendant of button
* style: retrieval and cancelled icon
* WIP: pass isSubmitting to Parts, use InProgressCall to display cancelled tool calls correctly, show domain/function name
* fix(meilisearch): fix `postSaveHook` issue where indexing expects a mongo document, and join all text content parts for meili indexing
* ci: fix dall-e tests
* ci: fix client tests
* fix: button types in actions panel
* fix: plugin auth form persisting across tool selections
* fix(ci): update AppService spec with `loadAndFormatTools`
* fix(clearConvos): add id check earlier on
* refactor(AssistantAvatar): set previewURL dynamically when emtadata.avatar changes
* feat(assistants): addTitle cache setting
* fix(useSSE): resolve rebase conflicts
* fix: delete mutation
* style(SidePanel): make grip visible on active and hover, invisible otherwise
* ci: add data-provider tests to workflow, also update eslint/tsconfig to recognize specs, and add `text/csv` to fileConfig
* fix: handle edge case where auth object is undefined, and log errors
* refactor(actions): resolve schemas, add tests for resolving refs, import specs from separate file for tests
* chore: remove comment
* fix(ActionsInput): re-render bug when initializing states with action fields
* fix(patch/assistant): filter undefined tools
* chore: add logging for errors in assistants routes
* fix(updateAssistant): map actions to functions to avoid overwriting
* fix(actions): properly handle GET paths
* fix(convos): unhandled delete thread exception
* refactor(AssistantService): pass both thread_id and conversationId when sending intermediate assistant messages, remove `mapMessagesToSteps` from AssistantService
* refactor(useSSE): replace all messages with runMessages and pass latestMessageId to abortRun; fix(checkMessageGaps): include tool calls when syncing messages
* refactor(assistants/chat): invoke `createOnTextProgress` after thread creation
* chore: add typing
* style: sidepanel styling
* style: action tool call domain styling
* feat(assistants): default models, limit retrieval to certain models, add env variables to to env.example
* feat: assistants api key in EndpointService
* refactor: set assistant model to conversation on assistant switch
* refactor: set assistant model to conversation on assistant select from panel
* fix(retrieveAndProcessFile): catch attempt to download file with `assistant` purpose which is not allowed; add logging
* feat: retrieval styling, handling, and logging
* chore: rename ASSISTANTS_REVERSE_PROXY to ASSISTANTS_BASE_URL
* feat: FileContext for file metadata
* feat: context file mgmt and filtering
* style(Select): hover/rounded changes
* refactor: explicit conversation switch, endpoint dependent, through `useSelectAssistant`, which does not create new chat if current endpoint is assistant endpoint
* fix(AssistantAvatar): make empty previewURL if no avatar present
* refactor: side panel mobile styling
* style: merge tool and action section, optimize mobile styling for action/tool buttons
* fix: localStorage issues
* fix(useSelectAssistant): invoke react query hook directly in select hook as Map was not being updated in time
* style: light mode fixes
* fix: prevent sidepanel nav styling from shifting layout up
* refactor: change default layout (collapsed by default)
* style: mobile optimization of DataTable
* style: datatable
* feat: client-side hide right-side panel
* chore(useNewConvo): add partial typing for preset
* fix(useSelectAssistant): pass correct model name by using template as preset
* WIP: assistant presets
* refactor(ToolService): add native solution for `TavilySearchResults` and log tool output errors
* refactor: organize imports and use native TavilySearchResults
* fix(TavilySearchResults): stringify result
* fix(ToolCall): show tool call outputs when not an action
* chore: rename Prompt Prefix to custom instructions (in user facing text only)
* refactor(EditPresetDialog): Optimize setting title by debouncing, reset preset on dialog close to avoid state mixture
* feat: add `presetOverride` to overwrite active conversation settings when saving a Preset (relevant for client side updates only)
* feat: Assistant preset settings (client-side)
* fix(Switcher): only set assistant_id and model if current endpoint is Assistants
* feat: use `useDebouncedInput` for updating conversation settings, starting with EditPresetDialog title setting and Assistant instructions setting
* feat(Assistants): add instructions field to settings
* feat(chat/assistants): pass conversation settings to run body
* wip: begin localization and only allow actions if the assistant is created
* refactor(AssistantsPanel): knowledge localization, allow tools on creation
* feat: experimental: allow 'priming' values before assistant is created, that would normally require an assistant_id to be defined
* chore: trim console logs and make more meaningful
* chore: toast messages
* fix(ci): date test
* feat: create file when uploading Assistant Avatar
* feat: file upload rate limiting from custom config with dynamic file route initialization
* refactor: use file upload limiters on post routes only
* refactor(fileConfig): add endpoints field for endpoint specific fileconfigs, add mergeConfig function, add tests
* refactor: fileConfig route, dynamic multer instances used on all '/' and '/images' POST routes, data service and query hook
* feat: supportedMimeTypesSchema, test for array of regex
* feat: configurable file config limits
* chore: clarify assistants file knowledge prereq.
* chore(useTextarea): default to localized 'Assistant' if assistant name is empty
* feat: configurable file limits and toggle file upload per endpoint
* fix(useUploadFileMutation): prevent updating assistant.files cache if file upload is a message_file attachment
* fix(AssistantSelect): set last selected assistant only when timeout successfully runs
* refactor(queries): disable assistant queries if assistants endpoint is not enabled
* chore(Switcher): add localization
* chore: pluralize `assistant` for `EModelEndpoint key and value
* feat: show/hide assistant UI components based on endpoint availability; librechat.yaml config for disabling builder section and setting polling/timeout intervals
* fix(compactEndpointSchemas): use EModelEndpoint for schema access
* feat(runAssistant): use configured values from `librechat.yaml` for `pollIntervalMs` and `timeout`
* fix: naming issue
* wip: revert landing
* 🎉 happy birthday LibreChat (#1768)
* happy birthday LibreChat
* Refactor endpoint condition in Landing component
* Update birthday message in Eng.tsx
* fix(/config): avoid nesting ternaries
* refactor(/config): check birthday
---------
Co-authored-by: Danny Avila <messagedaniel@protonmail.com>
* fix: landing
* fix: landing
* fix(useMessageHelpers): hardcoded check to use EModelEndpoint instead
* fix(ci): convo test revert to main
* fix(assistants/chat): fix issue where assistant_id was being saved as model for convo
* chore: added logging, promises racing to prevent longer timeouts, explicit setting of maxRetries and timeouts, robust catching of invalid abortRun params
* refactor: use recoil state for `showStopButton` and only show for assistants endpoint after syncing conversation data
* refactor: optimize abortRun strategy using localStorage, refactor `abortConversation` to use async/await and await the result, refactor how the abortKey cache is set for runs
* fix(checkMessageGaps): assign `assistant_id` to synced messages if defined; prevents UI from showing blank assistant for cancelled messages
* refactor: re-order sequence of chat route, only allow aborting messages after run is created, cancel abortRun if there was a cancelling error (likely due already cancelled in chat route), and add extra logging
* chore(typedefs): add httpAgent type to OpenAIClient
* refactor: use custom implementation of retrieving run with axios to allow for timing out run query
* fix(waitForRun): handle timed out run retrieval query
* refactor: update preset conditions:
- presets will retain settings when a different endpoint is selected; for existing convos, either when modular or is assistant switch
- no longer use `navigateToConvo` on preset select
* fix: temporary calculator hack as expects string input when invoked
* fix: cancel abortRun only when cancelling error is a result of the run already being cancelled
* chore: remove use of `fileMaxSizeMB` and total counterpart (redundant)
* docs: custom config documentation update
* docs: assistants api setup and dotenv, new custom config fields
* refactor(Switcher): make Assistant switcher sticky in SidePanel
* chore(useSSE): remove console log of data and message index
* refactor(AssistantPanel): button styling and add secondary select button to bottom of panel
* refactor(OpenAIClient): allow passing conversationId to RunManager through titleConvo and initializeLLM to properly record title context tokens used in cases where conversationId was not defined by the client
* feat(assistants): token tracking for assistant runs
* chore(spendTokens): improve logging
* feat: support/exclude specific assistant Ids
* chore: add update `librechat.example.yaml`, optimize `AppService` handling, new tests for `AppService`, optimize missing/outdate config logging
* chore: mount docker logs to root of project
* chore: condense axios errors
* chore: bump vite
* chore: vite hot reload fix using latest version
* chore(getOpenAIModels): sort instruct models to the end of models list
* fix(assistants): user provided key
* fix(assistants): user provided key, invalidate more queries on revoke
---------
Co-authored-by: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
* Style: Infinite Scroll and Group convos by date
* Style: Infinite Scroll and Group convos by date- Redesign NavBar
* Style: Infinite Scroll and Group convos by date- Redesign NavBar - Clean code
* Style: Infinite Scroll and Group convos by date- Redesign NavBar - Redesign NewChat Component
* Style: Infinite Scroll and Group convos by date- Redesign NavBar - Redesign NewChat Component
* Style: Infinite Scroll and Group convos by date- Redesign NavBar - Redesign NewChat Component
* Including OpenRouter and Mistral icon
* refactor(Conversations): cleanup use of utility functions and typing
* refactor(Nav/NewChat): use localStorage `lastConversationSetup` to determine the endpoint to use, as well as icons -> JSX components, remove use of `endpointSelected`
* refactor: remove use of `isFirstToday`
* refactor(Nav): remove use of `endpointSelected`, consolidate scrolling logic to its own hook `useNavScrolling`, remove use of recoil `conversation`
* refactor: Add spinner to bottom of list, throttle fetching, move query hooks to client workspace
* chore: sort by `updatedAt` field
* refactor: optimize conversation infinite query, use optimistic updates, add conversation helpers for managing pagination, remove unnecessary operations
* feat: gen_title route for generating the title for the conversation
* style(Convo): change hover bg-color
* refactor: memoize groupedConversations and return as array of tuples, correctly update convos pre/post message stream, only call genTitle if conversation is new, make `addConversation` dynamically either add/update depending if convo exists in pages already, reorganize type definitions
* style: rename Header NewChat Button -> HeaderNewChat, add NewChatIcon, closely match main Nav New Chat button to ChatGPT
* style(NewChat): add hover bg color
* style: cleanup comments, match ChatGPT nav styling, redesign search bar, make part of new chat sticky header, move Nav under same parent as outlet/mobilenav, remove legacy code, search only if searchQuery is not empty
* feat: add tests for conversation helpers and ensure no duplicate conversations are ever grouped
* style: hover bg-color
* feat: alt-click on convo item to open conversation in new tab
* chore: send error message when `gen_title` fails
---------
Co-authored-by: Walber Cardoso <walbercardoso@gmail.com>
* chore: fix `endpoint` typescript issues and typo in console info message
* feat(api): files GET endpoint and save only file_id references to messages
* refactor(client): `useGetFiles` query hook, update file types, optimistic update of filesQuery on file upload
* refactor(buildTree): update to use params object and accept fileMap
* feat: map files to messages; refactor(ChatView): messages only available after files are fetched
* fix: fetch files only when authenticated
* feat(api): AppService
- rename app.locals.configs to app.locals.paths
- load custom config use fileStrategy from yaml config in app.locals
* refactor: separate Firebase and Local strategies, call based on config
* refactor: modularize file strategies and employ with use of DALL-E
* refactor(librechat.yaml): add fileStrategy field
* feat: add source to MongoFile schema, as well as BatchFile, and ExtendedFile types
* feat: employ file strategies for upload/delete files
* refactor(deleteFirebaseFile): add user id validation for firebase file deletion
* chore(deleteFirebaseFile): update jsdocs
* feat: employ strategies for vision requests
* fix(client): handle messages with deleted files
* fix(client): ensure `filesToDelete` always saves/sends `file.source`
* feat(openAI): configurable `resendImages` and `imageDetail`
* refactor(getTokenCountForMessage): recursive process only when array of Objects and only their values (not keys) aside from `image_url` types
* feat(OpenAIClient): calculateImageTokenCost
* chore: remove comment
* refactor(uploadAvatar): employ fileStrategy for avatars, from social logins or user upload
* docs: update docs on how to configure fileStrategy
* fix(ci): mock winston and winston related modules, update DALLE3.spec.js with changes made
* refactor(redis): change terminal message to reflect current development state
* fix(DALL-E-2): pass fileStrategy to dall-e
* fix: load all existing conversation settings on refresh
* refactor(buildDefaultConvo): use `lastConversationSetup.endpointType` before `conversation.endpointType`
* refactor(TMessage/messageSchema): add `endpoint` field to messages to differentiate generation origin
* feat(useNewConvo): `keepLatestMessage` param to prevent reseting the `latestMessage` mid-conversation
* style(Settings): adjust height styling to allow more space in dialog for additional settings
* feat: Modular Chat: experimental setting to Enable switching Endpoints mid-conversation
* fix(ChatRoute): fix potential parsing issue with tPresetSchema
* WIP(backend/api): custom endpoint
* WIP(frontend/client): custom endpoint
* chore: adjust typedefs for configs
* refactor: use data-provider for cache keys and rename enums and custom endpoint for better clarity and compatibility
* feat: loadYaml utility
* refactor: rename back to from and proof-of-concept for creating schemas from user-defined defaults
* refactor: remove custom endpoint from default endpointsConfig as it will be exclusively managed by yaml config
* refactor(EndpointController): rename variables for clarity
* feat: initial load custom config
* feat(server/utils): add simple `isUserProvided` helper
* chore(types): update TConfig type
* refactor: remove custom endpoint handling from model services as will be handled by config, modularize fetching of models
* feat: loadCustomConfig, loadConfigEndpoints, loadConfigModels
* chore: reorganize server init imports, invoke loadCustomConfig
* refactor(loadConfigEndpoints/Models): return each custom endpoint as standalone endpoint
* refactor(Endpoint/ModelController): spread config values after default (temporary)
* chore(client): fix type issues
* WIP: first pass for multiple custom endpoints
- add endpointType to Conversation schema
- add update zod schemas for both convo/presets to allow non-EModelEndpoint value as endpoint (also using type assertion)
- use `endpointType` value as `endpoint` where mapping to type is necessary using this field
- use custom defined `endpoint` value and not type for mapping to modelsConfig
- misc: add return type to `getDefaultEndpoint`
- in `useNewConvo`, add the endpointType if it wasn't already added to conversation
- EndpointsMenu: use user-defined endpoint name as Title in menu
- TODO: custom icon via custom config, change unknown to robot icon
* refactor(parseConvo): pass args as an object and change where used accordingly; chore: comment out 'create schema' code
* chore: remove unused availableModels field in TConfig type
* refactor(parseCompactConvo): pass args as an object and change where used accordingly
* feat: chat through custom endpoint
* chore(message/convoSchemas): avoid saving empty arrays
* fix(BaseClient/saveMessageToDatabase): save endpointType
* refactor(ChatRoute): show Spinner if endpointsQuery or modelsQuery are still loading, which is apparent with slow fetching of models/remote config on first serve
* fix(useConversation): assign endpointType if it's missing
* fix(SaveAsPreset): pass real endpoint and endpointType when saving Preset)
* chore: recorganize types order for TConfig, add `iconURL`
* feat: custom endpoint icon support:
- use UnknownIcon in all icon contexts
- add mistral and openrouter as known endpoints, and add their icons
- iconURL support
* fix(presetSchema): move endpointType to default schema definitions shared between convoSchema and defaults
* refactor(Settings/OpenAI): remove legacy `isOpenAI` flag
* fix(OpenAIClient): do not invoke abortCompletion on completion error
* feat: add responseSender/label support for custom endpoints:
- use defaultModelLabel field in endpointOption
- add model defaults for custom endpoints in `getResponseSender`
- add `useGetSender` hook which uses EndpointsQuery to determine `defaultModelLabel`
- include defaultModelLabel from endpointConfig in custom endpoint client options
- pass `endpointType` to `getResponseSender`
* feat(OpenAIClient): use custom options from config file
* refactor: rename `defaultModelLabel` to `modelDisplayLabel`
* refactor(data-provider): separate concerns from `schemas` into `parsers`, `config`, and fix imports elsewhere
* feat: `iconURL` and extract environment variables from custom endpoint config values
* feat: custom config validation via zod schema, rename and move to `./projectRoot/librechat.yaml`
* docs: custom config docs and examples
* fix(OpenAIClient/mistral): mistral does not allow singular system message, also add `useChatCompletion` flag to use openai-node for title completions
* fix(custom/initializeClient): extract env var and use `isUserProvided` function
* Update librechat.example.yaml
* feat(InputWithLabel): add className props, and forwardRef
* fix(streamResponse): handle error edge case where either messages or convos query throws an error
* fix(useSSE): handle errorHandler edge cases where error response is and is not properly formatted from API, especially when a conversationId is not yet provided, which ensures stream is properly closed on error
* feat: user_provided keys for custom endpoints
* fix(config/endpointSchema): do not allow default endpoint values in custom endpoint `name`
* feat(loadConfigModels): extract env variables and optimize fetching models
* feat: support custom endpoint iconURL for messages and Nav
* feat(OpenAIClient): add/dropParams support
* docs: update docs with default params, add/dropParams, and notes to use config file instead of `OPENAI_REVERSE_PROXY`
* docs: update docs with additional notes
* feat(maxTokensMap): add mistral models (32k context)
* docs: update openrouter notes
* Update ai_setup.md
* docs(custom_config): add table of contents and fix note about custom name
* docs(custom_config): reorder ToC
* Update custom_config.md
* Add note about `max_tokens` field in custom_config.md
* fix(Message): avoid overwriting unprovided properties
* fix(OpenAIClient): return intermediateReply on user abort
* fix(AskController): do not send/save final message if abort was triggered
* fix(countTokens): avoid fetching remote registry and exclusively use cl100k_base or p50k_base weights for token counting
* refactor(Message/messageSchema): rely on messageSchema for default values when saving messages
* fix(EditController): do not send/save final message if abort was triggered
* fix(config/helpers): fix module resolution error
* chore: bump langchain to v0.0.213 from v0.0.186
* fix: handle abort edge cases:
- abort message server-side if response experienced error mid-generation
- attempt to recover message if aborting resulted in error
- if abortKey is not provided, use conversationId if it exists
- if headers were already sent, send an Event stream message
- issue warning for possible Google censor/filter
refactor(streamResponse): for `sendError`, allow passing overrides so that error can include partial generation, improve typing for `sendMessage`
* chore(MessageContent): remove eslint warning for unused `i`, rephrase unfinished message text
* fix(useSSE): avoid invoking cancelHandler if the abort response was 404
* chore(TMessage): remove unnecessary, unused legacy message property `submitting`
* chore(TMessage): remove unnecessary legacy message property `cancelled`
* chore(abortMiddleware): remove unused `errorText` property to avoid confusion
* feat: add GOOGLE_MODELS env var
* feat: add gemini vision support
* refactor(GoogleClient): adjust clientOptions handling depending on model
* fix(logger): fix redact logic and redact errors only
* fix(GoogleClient): do not allow non-multiModal messages when gemini-pro-vision is selected
* refactor(OpenAIClient): use `isVisionModel` client property to avoid calling validateVisionModel multiple times
* refactor: better debug logging by correctly traversing, redacting sensitive info, and logging condensed versions of long values
* refactor(GoogleClient): allow response errors to be thrown/caught above client handling so user receives meaningful error message
debug orderedMessages, parentMessageId, and buildMessages result
* refactor(AskController): use model from client.modelOptions.model when saving intermediate messages, which requires for the progress callback to be initialized after the client is initialized
* feat(useSSE): revert to previous model if the model was auto-switched by backend due to message attachments
* docs: update with google updates, notes about Gemini Pro Vision
* fix: redis should not be initialized without USE_REDIS and increase max listeners to 20
* WIP: initial logging changes
add several transports in ~/config/winston
omit messages in logs, truncate long strings
add short blurb in dotenv for debug logging
GoogleClient: using logger
OpenAIClient: using logger, handleOpenAIErrors
Adding typedef for payload message
bumped winston and using winston-daily-rotate-file
moved config for server paths to ~/config dir
Added `DEBUG_LOGGING=true` to .env.example
* WIP: Refactor logging statements in code
* WIP: Refactor logging statements and import configurations
* WIP: Refactor logging statements and import configurations
* refactor: broadcast Redis initialization message with `info` not `debug`
* refactor: complete Refactor logging statements and import configurations
* chore: delete unused tools
* fix: circular dependencies due to accessing logger
* refactor(handleText): handle booleans and write tests
* refactor: redact sensitive values, better formatting
* chore: improve log formatting, avoid passing strings to 2nd arg
* fix(ci): fix jest tests due to logger changes
* refactor(getAvailablePluginsController): cache plugins as they are static and avoids async addOpenAPISpecs call every time
* chore: update docs
* chore: update docs
* chore: create separate meiliSync logger, clean up logs to avoid being unnecessarily verbose
* chore: spread objects where they are commonly logged to allow string truncation
* chore: improve error log formatting
* feat: update PaLM icons
* feat: add additional google models
* POC: formatting inputs for Vertex AI streaming
* refactor: move endpoints services outside of /routes dir to /services/Endpoints
* refactor: shorten schemas import
* refactor: rename PALM to GOOGLE
* feat: make Google editable endpoint
* feat: reusable Ask and Edit controllers based off Anthropic
* chore: organize imports/logic
* fix(parseConvo): include examples in googleSchema
* fix: google only allows odd number of messages to be sent
* fix: pass proxy to AnthropicClient
* refactor: change `google` altName to `Google`
* refactor: update getModelMaxTokens and related functions to handle maxTokensMap with nested endpoint model key/values
* refactor: google Icon and response sender changes (Codey and Google logo instead of PaLM in all cases)
* feat: google support for maxTokensMap
* feat: google updated endpoints with Ask/Edit controllers, buildOptions, and initializeClient
* feat(GoogleClient): now builds prompt for text models and supports real streaming from Vertex AI through langchain
* chore(GoogleClient): remove comments, left before for reference in git history
* docs: update google instructions (WIP)
* docs(apis_and_tokens.md): add images to google instructions
* docs: remove typo apis_and_tokens.md
* Update apis_and_tokens.md
* feat(Google): use default settings map, fully support context for both text and chat models, fully support examples for chat models
* chore: update more PaLM references to Google
* chore: move playwright out of workflows to avoid failing tests
* fix: type issues with icons
* refactor: use react query for presets, show toasts on preset crud, refactor mutations, remove presetsQuery from Root (breaking change)
* refactor: change preset titling
* refactor: update preset schemas and methods for necessary new properties `order` and `defaultPreset`
* feat: add `defaultPreset` Recoil value
* refactor(getPresetTitle): make logic cleaner and more concise
* feat: complete UI portion of defaultPreset feature, with animations added to preset items
* chore: remove console.logs()
* feat: complete default preset handling
* refactor: remove user sensitive values on logout
* fix: allow endpoint selection without default preset overwriting
* feat: add timer duration to showToast, show toast for preset selection
* refactor: replace old /chat/ route with /c/. e2e tests will fail here
* refactor: move typedefs to root of /api/ and add a few to assistant types in TS
* refactor: reorganize data-provider imports, fix dependency cycle, strategize new plan to separate react dependent packages
* feat: add dataService for uploading images
* feat(data-provider): add mutation keys
* feat: file resizing and upload
* WIP: initial API image handling
* fix: catch JSON.parse of localStorage tools
* chore: experimental: use module-alias for absolute imports
* refactor: change temp_file_id strategy
* fix: updating files state by using Map and defining react query callbacks in a way that keeps them during component unmount, initial delete handling
* feat: properly handle file deletion
* refactor: unexpose complete filepath and resize from server for higher fidelity
* fix: make sure resized height, width is saved, catch bad requests
* refactor: use absolute imports
* fix: prevent setOptions from being called more than once for OpenAIClient, made note to fix for PluginsClient
* refactor: import supportsFiles and models vars from schemas
* fix: correctly replace temp file id
* refactor(BaseClient): use absolute imports, pass message 'opts' to buildMessages method, count tokens for nested objects/arrays
* feat: add validateVisionModel to determine if model has vision capabilities
* chore(checkBalance): update jsdoc
* feat: formatVisionMessage: change message content format dependent on role and image_urls passed
* refactor: add usage to File schema, make create and updateFile, correctly set and remove TTL
* feat: working vision support
TODO: file size, type, amount validations, making sure they are styled right, and making sure you can add images from the clipboard/dragging
* feat: clipboard support for uploading images
* feat: handle files on drop to screen, refactor top level view code to Presentation component so the useDragHelpers hook has ChatContext
* fix(Images): replace uploaded images in place
* feat: add filepath validation to protect sensitive files
* fix: ensure correct file_ids are push and not the Map key values
* fix(ToastContext): type issue
* feat: add basic file validation
* fix(useDragHelpers): correct context issue with `files` dependency
* refactor: consolidate setErrors logic to setError
* feat: add dialog Image overlay on image click
* fix: close endpoints menu on click
* chore: set detail to auto, make note for configuration
* fix: react warning (button desc. of button)
* refactor: optimize filepath handling, pass file_ids to images for easier re-use
* refactor: optimize image file handling, allow re-using files in regen, pass more file metadata in messages
* feat: lazy loading images including use of upload preview
* fix: SetKeyDialog closing, stopPropagation on Dialog content click
* style(EndpointMenuItem): tighten up the style, fix dark theme showing in lightmode, make menu more ux friendly
* style: change maxheight of all settings textareas to 138px from 300px
* style: better styling for textarea and enclosing buttons
* refactor(PresetItems): swap back edit and delete icons
* feat: make textarea placeholder dynamic to endpoint
* style: show user hover buttons only on hover when message is streaming
* fix: ordered list not going past 9, fix css
* feat: add User/AI labels; style: hide loading spinner
* feat: add back custom footer, change original footer text
* feat: dynamic landing icons based on endpoint
* chore: comment out assistants route
* fix: autoScroll to newest on /c/ view
* fix: Export Conversation on new UI
* style: match message style of official more closely
* ci: fix api jest unit tests, comment out e2e tests for now as they will fail until addressed
* feat: more file validation and use blob in preview field, not filepath, to fix temp deletion
* feat: filefilter for multer
* feat: better AI labels based on custom name, model, and endpoint instead of `ChatGPT`
* wip: initial client side code
* wip: initial api code
* refactor: export query keys from own module, export assistant hooks
* refactor(SelectDropDown): more customization via props
* feat: create Assistant and render real Assistants
* refactor: major refactor of UI components to allow multi-chat, working alongside CreationPanel
* refactor: move assistant routes to own directory
* fix(CreationHeader): state issue with assistant select
* refactor: style changes for form, fix setSiblingIdx from useChatHelpers to use latestMessageParentId, fix render issue with ChatView and change location
* feat: parseCompactConvo: begin refactor of slimmer JSON payloads between client/api
* refactor(endpoints): add assistant endpoint, also use EModelEndpoint as much as possible
* refactor(useGetConversationsQuery): use object to access query data easily
* fix(MultiMessage): react warning of bad state set, making use of effect during render (instead of useEffect)
* fix(useNewConvo): use correct atom key (index instead of convoId) for reset latestMessageFamily
* refactor: make routing navigation/conversation change simpler
* chore: add removeNullishValues for smaller payloads, remove unused fields, setup frontend pinging of assistant endpoint
* WIP: initial complete assistant run handling
* fix: CreationPanel form correctly setting internal state
* refactor(api/assistants/chat): revise functions to working run handling strategy
* refactor(UI): initial major refactor of ChatForm and options
* feat: textarea hook
* refactor: useAuthRedirect hook and change directory name
* feat: add ChatRoute (/c/), make optionsBar absolute and change on textarea height, add temp header
* feat: match new toggle Nav open button to ChatGPT's
* feat: add OpenAI custom classnames
* feat: useOriginNavigate
* feat: messages loading view
* fix: conversation navigation and effects
* refactor: make toggle change nav opacity
* WIP: new endpoint menu
* feat: NewEndpointsMenu complete
* fix: ensure set key dialog shows on endpoint change, and new conversation resets messages
* WIP: textarea styling fix, add temp footer, create basic file handling component
* feat: image file handling (UI)
* feat: PopOver and ModelSelect in Header, remove GenButtons
* feat: drop file handling
* refactor: bug fixes
use SSE at route level
add opts to useOriginNavigate
delay render of unfinishedMessage to avoid flickering
pass params (convoId) to chatHelpers to set messages query data based on param when the route is new (fixes can't continue convo on /new/)
style(MessagesView): matches height to official
fix(SSE): pass paramId and invalidate convos
style(Message): make bg uniform
* refactor(useSSE): setStorage within setConversation updates
* feat: conversationKeysAtom, allConversationsSelector, update convos query data on created message (if new), correctly handle convo deletion (individual)
* feat: add popover select dropdowns to allow options in header while allowing horizontal scroll for mobile
* style(pluginsSelect): styling changes
* refactor(NewEndpointsMenu): make UI components modular
* feat: Presets complete
* fix: preset editing, make by index
* fix: conversations not setting on inital navigation, fix getMessages() based on query param
* fix: changing preset no longer resets latestMessage
* feat: useOnClickOutside for OptionsPopover and fix bug that causes selection of preset when deleting
* fix: revert /chat/ switchToConvo, also use NewDeleteButton in Convo
* fix: Popover correctly closes on close Popover button using custom condition for useOnClickOutside
* style: new message and nav styling
* style: hover/sibling buttons and preset menu scrolling
* feat: new convo header button
* style(Textarea): minor style changes to textarea buttons
* feat: stop/continue generating and hide hoverbuttons when submitting
* feat: compact AI Provider schemas to make json payloads and db saves smaller
* style: styling changes for consistency on chat route
* fix: created usePresetIndexOptions to prevent bugs between /c/ and /chat/ routes when editing presets, removed redundant code from the new dialog
* chore: make /chat/ route default for now since we still lack full image support
* chore: update model rates with 11/06 rates
* chore: add new models to env.example for OPENAI_MODELS
* chore: reference actual maxTokensMap in ci tests
* refactor(Chains/llms): allow passing callbacks
* refactor(BaseClient): accurately count completion tokens as generation only
* refactor(OpenAIClient): remove unused getTokenCountForResponse, pass streaming var and callbacks in initializeLLM
* wip: summary prompt tokens
* refactor(summarizeMessages): new cut-off strategy that generates a better summary by adding context from beginning, truncating the middle, and providing the end
wip: draft out relevant providers and variables for token tracing
* refactor(createLLM): make streaming prop false by default
* chore: remove use of getTokenCountForResponse
* refactor(agents): use BufferMemory as ConversationSummaryBufferMemory token usage not easy to trace
* chore: remove passing of streaming prop, also console log useful vars for tracing
* feat: formatFromLangChain helper function to count tokens for ChatModelStart
* refactor(initializeLLM): add role for LLM tracing
* chore(formatFromLangChain): update JSDoc
* feat(formatMessages): formats langChain messages into OpenAI payload format
* chore: install openai-chat-tokens
* refactor(formatMessage): optimize conditional langChain logic
fix(formatFromLangChain): fix destructuring
* feat: accurate prompt tokens for ChatModelStart before generation
* refactor(handleChatModelStart): move to callbacks dir, use factory function
* refactor(initializeLLM): rename 'role' to 'context'
* feat(Balance/Transaction): new schema/models for tracking token spend
refactor(Key): factor out model export to separate file
* refactor(initializeClient): add req,res objects to client options
* feat: add-balance script to add to an existing users' token balance
refactor(Transaction): use multiplier map/function, return balance update
* refactor(Tx): update enum for tokenType, return 1 for multiplier if no map match
* refactor(Tx): add fair fallback value multiplier incase the config result is undefined
* refactor(Balance): rename 'tokens' to 'tokenCredits'
* feat: balance check, add tx.js for new tx-related methods and tests
* chore(summaryPrompts): update prompt token count
* refactor(callbacks): pass req, res
wip: check balance
* refactor(Tx): make convoId a String type, fix(calculateTokenValue)
* refactor(BaseClient): add conversationId as client prop when assigned
* feat(RunManager): track LLM runs with manager, track token spend from LLM,
refactor(OpenAIClient): use RunManager to create callbacks, pass user prop to langchain api calls
* feat(spendTokens): helper to spend prompt/completion tokens
* feat(checkBalance): add helper to check, log, deny request if balance doesn't have enough funds
refactor(Balance): static check method to return object instead of boolean now
wip(OpenAIClient): implement use of checkBalance
* refactor(initializeLLM): add token buffer to assure summary isn't generated when subsequent payload is too large
refactor(OpenAIClient): add checkBalance
refactor(createStartHandler): add checkBalance
* chore: remove prompt and completion token logging from route handler
* chore(spendTokens): add JSDoc
* feat(logTokenCost): record transactions for basic api calls
* chore(ask/edit): invoke getResponseSender only once per API call
* refactor(ask/edit): pass promptTokens to getIds and include in abort data
* refactor(getIds -> getReqData): rename function
* refactor(Tx): increase value if incomplete message
* feat: record tokenUsage when message is aborted
* refactor: subtract tokens when payload includes function_call
* refactor: add namespace for token_balance
* fix(spendTokens): only execute if corresponding token type amounts are defined
* refactor(checkBalance): throws Error if not enough token credits
* refactor(runTitleChain): pass and use signal, spread object props in create helpers, and use 'call' instead of 'run'
* fix(abortMiddleware): circular dependency, and default to empty string for completionTokens
* fix: properly cancel title requests when there isn't enough tokens to generate
* feat(predictNewSummary): custom chain for summaries to allow signal passing
refactor(summaryBuffer): use new custom chain
* feat(RunManager): add getRunByConversationId method, refactor: remove run and throw llm error on handleLLMError
* refactor(createStartHandler): if summary, add error details to runs
* fix(OpenAIClient): support aborting from summarization & showing error to user
refactor(summarizeMessages): remove unnecessary operations counting summaryPromptTokens and note for alternative, pass signal to summaryBuffer
* refactor(logTokenCost -> recordTokenUsage): rename
* refactor(checkBalance): include promptTokens in errorMessage
* refactor(checkBalance/spendTokens): move to models dir
* fix(createLanguageChain): correctly pass config
* refactor(initializeLLM/title): add tokenBuffer of 150 for balance check
* refactor(openAPIPlugin): pass signal and memory, filter functions by the one being called
* refactor(createStartHandler): add error to run if context is plugins as well
* refactor(RunManager/handleLLMError): throw error immediately if plugins, don't remove run
* refactor(PluginsClient): pass memory and signal to tools, cleanup error handling logic
* chore: use absolute equality for addTitle condition
* refactor(checkBalance): move checkBalance to execute after userMessage and tokenCounts are saved, also make conditional
* style: icon changes to match official
* fix(BaseClient): getTokenCountForResponse -> getTokenCount
* fix(formatLangChainMessages): add kwargs as fallback prop from lc_kwargs, update JSDoc
* refactor(Tx.create): does not update balance if CHECK_BALANCE is not enabled
* fix(e2e/cleanUp): cleanup new collections, import all model methods from index
* fix(config/add-balance): add uncaughtException listener
* fix: circular dependency
* refactor(initializeLLM/checkBalance): append new generations to errorMessage if cost exceeds balance
* fix(handleResponseMessage): only record token usage in this method if not error and completion is not skipped
* fix(createStartHandler): correct condition for generations
* chore: bump postcss due to moderate severity vulnerability
* chore: bump zod due to low severity vulnerability
* chore: bump openai & data-provider version
* feat(types): OpenAI Message types
* chore: update bun lockfile
* refactor(CodeBlock): add error block formatting
* refactor(utils/Plugin): factor out formatJSON and cn to separate files (json.ts and cn.ts), add extractJSON
* chore(logViolation): delete user_id after error is logged
* refactor(getMessageError -> Error): change to React.FC, add token_balance handling, use extractJSON to determine JSON instead of regex
* fix(DALL-E): use latest openai SDK
* chore: reorganize imports, fix type issue
* feat(server): add balance route
* fix(api/models): add auth
* feat(data-provider): /api/balance query
* feat: show balance if checking is enabled, refetch on final message or error
* chore: update docs, .env.example with token_usage info, add balance script command
* fix(Balance): fallback to empty obj for balance query
* style: slight adjustment of balance element
* docs(token_usage): add PR notes
* refactor: pass model in message edit payload, use encoder in standalone util function
* feat: add summaryBuffer helper
* refactor(api/messages): use new countTokens helper and add auth middleware at top
* wip: ConversationSummaryBufferMemory
* refactor: move pre-generation helpers to prompts dir
* chore: remove console log
* chore: remove test as payload will no longer carry tokenCount
* chore: update getMessagesWithinTokenLimit JSDoc
* refactor: optimize getMessagesForConversation and also break on summary, feat(ci): getMessagesForConversation tests
* refactor(getMessagesForConvo): count '00000000-0000-0000-0000-000000000000' as root message
* chore: add newer model to token map
* fix: condition was point to prop of array instead of message prop
* refactor(BaseClient): use object for refineMessages param, rename 'summary' to 'summaryMessage', add previous_summary
refactor(getMessagesWithinTokenLimit): replace text and tokenCount if should summarize, summary, and summaryTokenCount are present
fix/refactor(handleContextStrategy): use the right comparison length for context diff, and replace payload first message when a summary is present
* chore: log previous_summary if debugging
* refactor(formatMessage): assume if role is defined that it's a valid value
* refactor(getMessagesWithinTokenLimit): remove summary logic
refactor(handleContextStrategy): add usePrevSummary logic in case only summary was pruned
refactor(loadHistory): initial message query will return all ordered messages but keep track of the latest summary
refactor(getMessagesForConversation): use object for single param, edit jsdoc, edit all files using the method
refactor(ChatGPTClient): order messages before buildPrompt is called, TODO: add convoSumBuffMemory logic
* fix: undefined handling and summarizing only when shouldRefineContext is true
* chore(BaseClient): fix test results omitting system role for summaries and test edge case
* chore: export summaryBuffer from index file
* refactor(OpenAIClient/BaseClient): move refineMessages to subclass, implement LLM initialization for summaryBuffer
* feat: add OPENAI_SUMMARIZE to enable summarizing, refactor: rename client prop 'shouldRefineContext' to 'shouldSummarize', change contextStrategy value to 'summarize' from 'refine'
* refactor: rename refineMessages method to summarizeMessages for clarity
* chore: clarify summary future intent in .env.example
* refactor(initializeLLM): handle case for either 'model' or 'modelName' being passed
* feat(gptPlugins): enable summarization for plugins
* refactor(gptPlugins): utilize new initializeLLM method and formatting methods for messages, use payload array for currentMessages and assign pastMessages sooner
* refactor(agents): use ConversationSummaryBufferMemory for both agent types
* refactor(formatMessage): optimize original method for langchain, add helper function for langchain messages, add JSDocs and tests
* refactor(summaryBuffer): add helper to createSummaryBufferMemory, and use new formatting helpers
* fix: forgot to spread formatMessages also took opportunity to pluralize filename
* refactor: pass memory to tools, namely openapi specs. not used and may never be used by new method but added for testing
* ci(formatMessages): add more exhaustive checks for langchain messages
* feat: add debug env var for OpenAI
* chore: delete unnecessary comments
* chore: add extra note about summary feature
* fix: remove tokenCount from payload instructions
* fix: test fail
* fix: only pass instructions to payload when defined or not empty object
* refactor: fromPromptMessages is deprecated, use renamed method fromMessages
* refactor: use 'includes' instead of 'startsWith' for extended OpenRouter compatibility
* fix(PluginsClient.buildPromptBody): handle undefined message strings
* chore: log langchain titling error
* feat: getModelMaxTokens helper
* feat: tokenSplit helper
* feat: summary prompts updated
* fix: optimize _CUT_OFF_SUMMARIZER prompt
* refactor(summaryBuffer): use custom summary prompt, allow prompt to be passed, pass humanPrefix and aiPrefix to memory, along with any future variables, rename messagesToRefine to context
* fix(summaryBuffer): handle edge case where messagesToRefine exceeds summary context,
refactor(BaseClient): allow custom maxContextTokens to be passed to getMessagesWithinTokenLimit, add defined check before unshifting summaryMessage, update shouldSummarize based on this
refactor(OpenAIClient): use getModelMaxTokens, use cut-off message method for summary if no messages were left after pruning
* fix(handleContextStrategy): handle case where incoming prompt is bigger than model context
* chore: rename refinedContent to splitText
* chore: remove unnecessary debug log
* feat: add global teardown to remove test data and add registration/log-out to auth flow
* refactor(models/Conversation): index user field and add JSDoc to deleteConvos
* refactor: add user index to message schema and ensure user is saved to each Message
* refactor: add user to each saveMessage call
* fix: handle case where title is null in zod schema
* feat(e2e): ensure messages are deleted on cleanUp
* fix: set last convo for all endpoints on conversation update
* fix: enable registration for CI env
* chore: cleanup client depend 🧹
* chore: replace joi with zod and remove unused user validator
* chore: move dep from root to api, cleanup other unused api deps
* chore: remove unused dev dep
* chore: update bun lockfile
* fix: bun scripts
* chore: add bun flag to update script
* chore: remove legacy webpack + babel dev deps
* chore: add back dev deps needed for frontend unit testing
* fix(validators): make schemas as expected and more robust with a full test suite of edge cases
* chore: remove axios from root package, remove path from api, update bun
* refactor: require Auth middleware in route index files
* feat: concurrent message limiter
* feat: complete concurrent message limiter with caching
* refactor: SSE response methods separated from handleText
* fix(abortMiddleware): fix req and res order to standard, use endpointOption in req.body
* chore: minor name changes
* refactor: add isUUID condition to saveMessage
* fix(concurrentLimiter): logic correctly handles the max number of concurrent messages and res closing/finalization
* chore: bump keyv and remove console.log from Message
* fix(concurrentLimiter): ensure messages are only saved in later message children
* refactor(concurrentLimiter): use KeyvFile instead, could make other stores configurable in the future
* feat: add denyRequest function for error responses
* feat(utils): add isStringTruthy function
Introduce the isStringTruthy function to the utilities module to check if a string value is a case-insensitive match for 'true'
* feat: add optional message rate limiters by IP and userId
* feat: add optional message rate limiters by IP and userId to edit route
* refactor: rename isStringTruthy to isTrue for brevity
* refactor(getError): use map to make code cleaner
* refactor: use memory for concurrent rate limiter to prevent clearing on startup/exit, add multiple log files, fix error message for concurrent violation
* feat: check if errorMessage is object, stringify if so
* chore: send object to denyRequest which will stringify it
* feat: log excessive requests
* fix(getError): correctly pluralize messages
* refactor(limiters): make type consistent between logs and errorMessage
* refactor(cache): move files out of lib/db into separate cache dir
>> feat: add getLogStores function so Keyv instance is not redundantly created on every violation
feat: separate violation logging to own function with logViolation
* fix: cache/index.js export, properly record userViolations
* refactor(messageLimiters): use new logging method, add logging to registrations
* refactor(logViolation): make userLogs an array of logs per user
* feat: add logging to login limiter
* refactor: pass req as first param to logViolation and record offending IP
* refactor: rename isTrue helper fn to isEnabled
* feat: add simple non_browser check and log violation
* fix: open handles in unit tests, remove KeyvMongo as not used and properly mock global fetch
* chore: adjust nodemon ignore paths to properly ignore logs
* feat: add math helper function for safe use of eval
* refactor(api/convos): use middleware at top of file to avoid redundancy
* feat: add delete all static method for Sessions
* fix: redirect to login on refresh if user is not found, or the session is not found but hasn't expired (ban case)
* refactor(getLogStores): adjust return type
* feat: add ban violation and check ban logic
refactor(logViolation): pass both req and res objects
* feat: add removePorts helper function
* refactor: rename getError to getMessageError and add getLoginError for displaying different login errors
* fix(AuthContext): fix type issue and remove unused code
* refactor(bans): ban by ip and user id, send response based on origin
* chore: add frontend ban messages
* refactor(routes/oauth): add ban check to handler, also consolidate logic to avoid redundancy
* feat: add ban check to AI messaging routes
* feat: add ban check to login/registration
* fix(ci/api): mock KeyvMongo to avoid tests hanging
* docs: update .env.example
> refactor(banViolation): calculate interval rate crossover, early return if duration is invalid
ci(banViolation): add tests to ensure users are only banned when expected
* docs: improve wording for mod system
* feat: add configurable env variables for violation scores
* chore: add jsdoc for uaParser.js
* chore: improve ban text log
* chore: update bun test scripts
* refactor(math.js): add fallback values
* fix(KeyvMongo/banLogs): refactor keyv instances to top of files to avoid memory leaks, refactor ban logic to use getLogStores instead
refactor(getLogStores): get a single log store by type
* fix(ci): refactor tests due to banLogs changes, also make sure to clear and revoke sessions even if ban duration is 0
* fix(banViolation.js): getLogStores import
* feat: handle 500 code error at login
* fix(middleware): handle case where user.id is _id and not just id
* ci: add ban secrets for backend unit tests
* refactor: logout user upon ban
* chore: log session delete message only if deletedCount > 0
* refactor: change default ban duration (2h) and make logic more clear in JSDOC
* fix: login and registration limiters will now return rate limiting error
* fix: userId not parsable as non ObjectId string
* feat: add useTimeout hook to properly clear timeouts when invoking functions within them
refactor(AuthContext): cleanup code by using new hook and defining types in ~/common
* fix: login error message for rate limits
* docs: add info for automated mod system and rate limiters, update other docs accordingly
* chore: bump data-provider version
* feat(api): refresh token logic
* feat(client): refresh token logic
* feat(data-provider): refresh token logic
* fix: SSE uses esm
* chore: add default refresh token expiry to AuthService, add message about env var not set when generating a token
* chore: update scripts to more compatible bun methods, ran bun install again
* chore: update env.example and playwright workflow with JWT_REFRESH_SECRET
* chore: update breaking changes docs
* chore: add timeout to url visit
* chore: add default SESSION_EXPIRY in generateToken logic, add act script for testing github actions
* fix(e2e): refresh automatically in development environment to pass e2e tests
