mirror of
https://github.com/danny-avila/LibreChat.git
synced 2025-12-21 02:40:14 +01:00
66 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Denis Palnitsky
|
ab6fbe48f1
|
📥 feat: Import Conversations from LibreChat, ChatGPT, Chatbot UI (#2355)
* Basic implementation of ChatGPT conversation import * remove debug code * Handle citations * Fix updatedAt in import * update default model * Use job scheduler to handle import requests * import job status endpoint * Add wrapper around Agenda * Rate limits for import endpoint * rename import api path * Batch save import to mongo * Improve naming * Add documenting comments * Test for importers * Change button for importing conversations * Frontend changes * Import job status endpoint * Import endpoint response * Add translations to new phrases * Fix conversations refreshing * cleanup unused functions * set timeout for import job status polling * Add documentation * get extra spaces back * Improve error message * Fix translation files after merge * fix translation files 2 * Add zh translation for import functionality * Sync mailisearch index after import * chore: add dummy uri for jest tests, as MONGO_URI should only be real for E2E tests * docs: fix links * docs: fix conversationsImport section * fix: user role issue for librechat imports * refactor: import conversations from json - organize imports - add additional jsdocs - use multer with diskStorage to avoid loading file into memory outside of job - use filepath instead of loading data string for imports - replace console logs and some logger.info() with logger.debug - only use multer for import route * fix: undefined metadata edge case and replace ChatGtp -> ChatGpt * Refactor importChatGptConvo function to handle undefined metadata edge case and replace ChatGtp with ChatGpt * fix: chatgpt importer * feat: maintain tree relationship for librechat messages * chore: use enum * refactor: saveMessage to use single object arg, replace console logs, add userId to log message * chore: additional comment * chore: multer edge case * feat: first pass, maintain tree relationship * chore: organize * chore: remove log * ci: add heirarchy test for chatgpt * ci: test maintaining of heirarchy for librechat * wip: allow non-text content type messages * refactor: import content part object json string * refactor: more content types to format * chore: consolidate messageText formatting * docs: update on changes, bump data-provider/config versions, update readme * refactor(indexSync): singleton pattern for MeiliSearchClient * refactor: debug log after batch is done * chore: add back indexSync error handling --------- Co-authored-by: jakubmieszczak <jakub.mieszczak@zendesk.com> Co-authored-by: Danny Avila <danny@librechat.ai> |
||
|
Danny Avila
|
5cd5c3bef8
|
🅰️ feat: Azure OpenAI Assistants API Support (#1992)
* chore: rename dir from `assistant` to plural * feat: `assistants` field for azure config, spread options in AppService * refactor: rename constructAzureURL param for azure as `azureOptions` * chore: bump openai and bun * chore(loadDefaultModels): change naming of assistant -> assistants * feat: load azure settings with currect baseURL for assistants' initializeClient * refactor: add `assistants` flags to groups and model configs, add mapGroupToAzureConfig * feat(loadConfigEndpoints): initialize assistants endpoint if azure flag `assistants` is enabled * feat(AppService): determine assistant models on startup, throw Error if none * refactor(useDeleteAssistantMutation): send model along with assistant id for delete mutations * feat: support listing and deleting assistants with azure * feat: add model query to assistant avatar upload * feat: add azure support for retrieveRun method * refactor: update OpenAIClient initialization * chore: update README * fix(ci): tests passing * refactor(uploadOpenAIFile): improve logging and use more efficient REST API method * refactor(useFileHandling): add model to metadata to target Azure region compatible with current model * chore(files): add azure naming pattern for valid file id recognition * fix(assistants): initialize openai with first available assistant model if none provided * refactor(uploadOpenAIFile): add content type for azure, initialize formdata before azure options * refactor(sleep): move sleep function out of Runs and into `~/server/utils` * fix(azureOpenAI/assistants): make sure to only overwrite models with assistant models if `assistants` flag is enabled * refactor(uploadOpenAIFile): revert to old method * chore(uploadOpenAIFile): use enum for file purpose * docs: azureOpenAI update guide with more info, examples * feat: enable/disable assistant capabilities and specify retrieval models * refactor: optional chain conditional statement in loadConfigModels.js * docs: add assistants examples * chore: update librechat.example.yaml * docs(azure): update note of file upload behavior in Azure OpenAI Assistants * chore: update docs and add descriptive message about assistant errors * fix: prevent message submission with invalid assistant or if files loading * style: update Landing icon & text when assistant is not selected * chore: bump librechat-data-provider to 0.4.8 * fix(assistants/azure): assign req.body.model for proper azure init to abort runs |
||
|
Danny Avila
|
74459d6261
|
♾️ style: Infinite Scroll Nav and Sort Convos by Date/Usage (#1708)
* Style: Infinite Scroll and Group convos by date * Style: Infinite Scroll and Group convos by date- Redesign NavBar * Style: Infinite Scroll and Group convos by date- Redesign NavBar - Clean code * Style: Infinite Scroll and Group convos by date- Redesign NavBar - Redesign NewChat Component * Style: Infinite Scroll and Group convos by date- Redesign NavBar - Redesign NewChat Component * Style: Infinite Scroll and Group convos by date- Redesign NavBar - Redesign NewChat Component * Including OpenRouter and Mistral icon * refactor(Conversations): cleanup use of utility functions and typing * refactor(Nav/NewChat): use localStorage `lastConversationSetup` to determine the endpoint to use, as well as icons -> JSX components, remove use of `endpointSelected` * refactor: remove use of `isFirstToday` * refactor(Nav): remove use of `endpointSelected`, consolidate scrolling logic to its own hook `useNavScrolling`, remove use of recoil `conversation` * refactor: Add spinner to bottom of list, throttle fetching, move query hooks to client workspace * chore: sort by `updatedAt` field * refactor: optimize conversation infinite query, use optimistic updates, add conversation helpers for managing pagination, remove unnecessary operations * feat: gen_title route for generating the title for the conversation * style(Convo): change hover bg-color * refactor: memoize groupedConversations and return as array of tuples, correctly update convos pre/post message stream, only call genTitle if conversation is new, make `addConversation` dynamically either add/update depending if convo exists in pages already, reorganize type definitions * style: rename Header NewChat Button -> HeaderNewChat, add NewChatIcon, closely match main Nav New Chat button to ChatGPT * style(NewChat): add hover bg color * style: cleanup comments, match ChatGPT nav styling, redesign search bar, make part of new chat sticky header, move Nav under same parent as outlet/mobilenav, remove legacy code, search only if searchQuery is not empty * feat: add tests for conversation helpers and ensure no duplicate conversations are ever grouped * style: hover bg-color * feat: alt-click on convo item to open conversation in new tab * chore: send error message when `gen_title` fails --------- Co-authored-by: Walber Cardoso <walbercardoso@gmail.com> |
||
|
Marco Beretta
|
f19f5dca8e
|
🔥🚀 feat: CDN (Firebase) & feat: account section (#1438)
* localization + api-endpoint * docs: added firebase documentation * chore: icons * chore: SettingsTabs * feat: account pannel; fix: gear icons * docs: position update * feat: firebase * feat: plugin support * route * fixed bugs with firebase and moved a lot of files * chore(DALLE3): using UUID v4 * feat: support for social strategies; moved '/images' path * fix: data ignored * gitignore update * docs: update firebase guide * refactor: Firebase - use singleton pattern for firebase initialization, initially on server start - reorganize imports, move firebase specific files to own service under Files - rename modules to remove 'avatar' redundancy - fix imports based on changes * ci(DALLE/DALLE3): fix tests to use logger and new expected outputs, add firebase tests * refactor(loadToolWithAuth): pass userId to tool as field * feat(images/parse): feat: Add URL Image Basename Extraction Implement a new module to extract the basename of an image from a given URL. This addition includes the function, which parses the URL and retrieves the basename using the Node.js 'url' and 'path' modules. The function is documented with JSDoc comments for better maintainability and understanding. This feature enhances the application's ability to handle and process image URLs efficiently. * refactor(addImages): function to use a more specific regular expression for observedImagePath based on the generated image markdown standard across the app * refactor(DALLE/DALLE3): utilize `getImageBasename` and `this.userId`; fix: pass correct image path to firebase url helper * fix(addImages): make more general to match any image markdown descriptor * fix(parse/getImageBasename): test result of this function for an actual image basename * ci(DALLE3): mock getImageBasename * refactor(AuthContext): use Recoil atom state for user * feat: useUploadAvatarMutation, react-query hook for avatar upload * fix(Toast): stack z-order of Toast over all components (1000) * refactor(showToast): add optional status field to avoid importing NotificationSeverity on each use of the function * refactor(routes/avatar): remove unnecessary get route, get userId from req.user.id, require auth on POST request * chore(uploadAvatar): TODO: remove direct use of Model, `User` * fix(client): fix Spinner imports * refactor(Avatar): use react-query hook, Toast, remove unnecessary states, add optimistic UI to upload * fix(avatar/localStrategy): correctly save local profile picture and cache bust for immediate rendering; fix: firebase init info message (only show once) * fix: use `includes` instead of `endsWith` for checking manual query of avatar image path in case more queries are appended (as is done in avatar/localStrategy) --------- Co-authored-by: Danny Avila <messagedaniel@protonmail.com> |
||
|
Danny Avila
|
0bae503a0a
|
refactor: Speed up Config fetching and Setup Config Groundwork 👷🚧 (#1297)
* refactor: move endpoint services to own directory * refactor: make endpointconfig handling more concise, separate logic, and cache result for subsequent serving * refactor: ModelController gets same treatment as EndpointController, draft OverrideController * wip: flesh out override controller more to return real value * refactor: client/api changes in anticipation of override |
||
|
Danny Avila
|
317cdd3f77
|
feat: Vision Support + New UI (#1203)
* feat: add timer duration to showToast, show toast for preset selection * refactor: replace old /chat/ route with /c/. e2e tests will fail here * refactor: move typedefs to root of /api/ and add a few to assistant types in TS * refactor: reorganize data-provider imports, fix dependency cycle, strategize new plan to separate react dependent packages * feat: add dataService for uploading images * feat(data-provider): add mutation keys * feat: file resizing and upload * WIP: initial API image handling * fix: catch JSON.parse of localStorage tools * chore: experimental: use module-alias for absolute imports * refactor: change temp_file_id strategy * fix: updating files state by using Map and defining react query callbacks in a way that keeps them during component unmount, initial delete handling * feat: properly handle file deletion * refactor: unexpose complete filepath and resize from server for higher fidelity * fix: make sure resized height, width is saved, catch bad requests * refactor: use absolute imports * fix: prevent setOptions from being called more than once for OpenAIClient, made note to fix for PluginsClient * refactor: import supportsFiles and models vars from schemas * fix: correctly replace temp file id * refactor(BaseClient): use absolute imports, pass message 'opts' to buildMessages method, count tokens for nested objects/arrays * feat: add validateVisionModel to determine if model has vision capabilities * chore(checkBalance): update jsdoc * feat: formatVisionMessage: change message content format dependent on role and image_urls passed * refactor: add usage to File schema, make create and updateFile, correctly set and remove TTL * feat: working vision support TODO: file size, type, amount validations, making sure they are styled right, and making sure you can add images from the clipboard/dragging * feat: clipboard support for uploading images * feat: handle files on drop to screen, refactor top level view code to Presentation component so the useDragHelpers hook has ChatContext * fix(Images): replace uploaded images in place * feat: add filepath validation to protect sensitive files * fix: ensure correct file_ids are push and not the Map key values * fix(ToastContext): type issue * feat: add basic file validation * fix(useDragHelpers): correct context issue with `files` dependency * refactor: consolidate setErrors logic to setError * feat: add dialog Image overlay on image click * fix: close endpoints menu on click * chore: set detail to auto, make note for configuration * fix: react warning (button desc. of button) * refactor: optimize filepath handling, pass file_ids to images for easier re-use * refactor: optimize image file handling, allow re-using files in regen, pass more file metadata in messages * feat: lazy loading images including use of upload preview * fix: SetKeyDialog closing, stopPropagation on Dialog content click * style(EndpointMenuItem): tighten up the style, fix dark theme showing in lightmode, make menu more ux friendly * style: change maxheight of all settings textareas to 138px from 300px * style: better styling for textarea and enclosing buttons * refactor(PresetItems): swap back edit and delete icons * feat: make textarea placeholder dynamic to endpoint * style: show user hover buttons only on hover when message is streaming * fix: ordered list not going past 9, fix css * feat: add User/AI labels; style: hide loading spinner * feat: add back custom footer, change original footer text * feat: dynamic landing icons based on endpoint * chore: comment out assistants route * fix: autoScroll to newest on /c/ view * fix: Export Conversation on new UI * style: match message style of official more closely * ci: fix api jest unit tests, comment out e2e tests for now as they will fail until addressed * feat: more file validation and use blob in preview field, not filepath, to fix temp deletion * feat: filefilter for multer * feat: better AI labels based on custom name, model, and endpoint instead of `ChatGPT` |
||
|
Danny Avila
|
bac1fb67d2
|
WIP: Update UI to match Official Style; Vision and Assistants 👷🏽 (#1190)
* wip: initial client side code * wip: initial api code * refactor: export query keys from own module, export assistant hooks * refactor(SelectDropDown): more customization via props * feat: create Assistant and render real Assistants * refactor: major refactor of UI components to allow multi-chat, working alongside CreationPanel * refactor: move assistant routes to own directory * fix(CreationHeader): state issue with assistant select * refactor: style changes for form, fix setSiblingIdx from useChatHelpers to use latestMessageParentId, fix render issue with ChatView and change location * feat: parseCompactConvo: begin refactor of slimmer JSON payloads between client/api * refactor(endpoints): add assistant endpoint, also use EModelEndpoint as much as possible * refactor(useGetConversationsQuery): use object to access query data easily * fix(MultiMessage): react warning of bad state set, making use of effect during render (instead of useEffect) * fix(useNewConvo): use correct atom key (index instead of convoId) for reset latestMessageFamily * refactor: make routing navigation/conversation change simpler * chore: add removeNullishValues for smaller payloads, remove unused fields, setup frontend pinging of assistant endpoint * WIP: initial complete assistant run handling * fix: CreationPanel form correctly setting internal state * refactor(api/assistants/chat): revise functions to working run handling strategy * refactor(UI): initial major refactor of ChatForm and options * feat: textarea hook * refactor: useAuthRedirect hook and change directory name * feat: add ChatRoute (/c/), make optionsBar absolute and change on textarea height, add temp header * feat: match new toggle Nav open button to ChatGPT's * feat: add OpenAI custom classnames * feat: useOriginNavigate * feat: messages loading view * fix: conversation navigation and effects * refactor: make toggle change nav opacity * WIP: new endpoint menu * feat: NewEndpointsMenu complete * fix: ensure set key dialog shows on endpoint change, and new conversation resets messages * WIP: textarea styling fix, add temp footer, create basic file handling component * feat: image file handling (UI) * feat: PopOver and ModelSelect in Header, remove GenButtons * feat: drop file handling * refactor: bug fixes use SSE at route level add opts to useOriginNavigate delay render of unfinishedMessage to avoid flickering pass params (convoId) to chatHelpers to set messages query data based on param when the route is new (fixes can't continue convo on /new/) style(MessagesView): matches height to official fix(SSE): pass paramId and invalidate convos style(Message): make bg uniform * refactor(useSSE): setStorage within setConversation updates * feat: conversationKeysAtom, allConversationsSelector, update convos query data on created message (if new), correctly handle convo deletion (individual) * feat: add popover select dropdowns to allow options in header while allowing horizontal scroll for mobile * style(pluginsSelect): styling changes * refactor(NewEndpointsMenu): make UI components modular * feat: Presets complete * fix: preset editing, make by index * fix: conversations not setting on inital navigation, fix getMessages() based on query param * fix: changing preset no longer resets latestMessage * feat: useOnClickOutside for OptionsPopover and fix bug that causes selection of preset when deleting * fix: revert /chat/ switchToConvo, also use NewDeleteButton in Convo * fix: Popover correctly closes on close Popover button using custom condition for useOnClickOutside * style: new message and nav styling * style: hover/sibling buttons and preset menu scrolling * feat: new convo header button * style(Textarea): minor style changes to textarea buttons * feat: stop/continue generating and hide hoverbuttons when submitting * feat: compact AI Provider schemas to make json payloads and db saves smaller * style: styling changes for consistency on chat route * fix: created usePresetIndexOptions to prevent bugs between /c/ and /chat/ routes when editing presets, removed redundant code from the new dialog * chore: make /chat/ route default for now since we still lack full image support |
||
|
Danny Avila
|
6cb561abcf
|
fix: getLogStores Property and Handle 401 Error from Refresh Token Request (#1084)
* fix(getLogStores): correct wrong prop passed to keyv opts: duration -> ttl * fix: edge case where we get a blank screen if the initially intercepted 401 error is from a refresh token request; in this case, make explicit to the server that we are retrying from a refreshToken request |
||
|
Danny Avila
|
365c39c405
|
feat: Accurate Token Usage Tracking & Optional Balance (#1018)
* refactor(Chains/llms): allow passing callbacks * refactor(BaseClient): accurately count completion tokens as generation only * refactor(OpenAIClient): remove unused getTokenCountForResponse, pass streaming var and callbacks in initializeLLM * wip: summary prompt tokens * refactor(summarizeMessages): new cut-off strategy that generates a better summary by adding context from beginning, truncating the middle, and providing the end wip: draft out relevant providers and variables for token tracing * refactor(createLLM): make streaming prop false by default * chore: remove use of getTokenCountForResponse * refactor(agents): use BufferMemory as ConversationSummaryBufferMemory token usage not easy to trace * chore: remove passing of streaming prop, also console log useful vars for tracing * feat: formatFromLangChain helper function to count tokens for ChatModelStart * refactor(initializeLLM): add role for LLM tracing * chore(formatFromLangChain): update JSDoc * feat(formatMessages): formats langChain messages into OpenAI payload format * chore: install openai-chat-tokens * refactor(formatMessage): optimize conditional langChain logic fix(formatFromLangChain): fix destructuring * feat: accurate prompt tokens for ChatModelStart before generation * refactor(handleChatModelStart): move to callbacks dir, use factory function * refactor(initializeLLM): rename 'role' to 'context' * feat(Balance/Transaction): new schema/models for tracking token spend refactor(Key): factor out model export to separate file * refactor(initializeClient): add req,res objects to client options * feat: add-balance script to add to an existing users' token balance refactor(Transaction): use multiplier map/function, return balance update * refactor(Tx): update enum for tokenType, return 1 for multiplier if no map match * refactor(Tx): add fair fallback value multiplier incase the config result is undefined * refactor(Balance): rename 'tokens' to 'tokenCredits' * feat: balance check, add tx.js for new tx-related methods and tests * chore(summaryPrompts): update prompt token count * refactor(callbacks): pass req, res wip: check balance * refactor(Tx): make convoId a String type, fix(calculateTokenValue) * refactor(BaseClient): add conversationId as client prop when assigned * feat(RunManager): track LLM runs with manager, track token spend from LLM, refactor(OpenAIClient): use RunManager to create callbacks, pass user prop to langchain api calls * feat(spendTokens): helper to spend prompt/completion tokens * feat(checkBalance): add helper to check, log, deny request if balance doesn't have enough funds refactor(Balance): static check method to return object instead of boolean now wip(OpenAIClient): implement use of checkBalance * refactor(initializeLLM): add token buffer to assure summary isn't generated when subsequent payload is too large refactor(OpenAIClient): add checkBalance refactor(createStartHandler): add checkBalance * chore: remove prompt and completion token logging from route handler * chore(spendTokens): add JSDoc * feat(logTokenCost): record transactions for basic api calls * chore(ask/edit): invoke getResponseSender only once per API call * refactor(ask/edit): pass promptTokens to getIds and include in abort data * refactor(getIds -> getReqData): rename function * refactor(Tx): increase value if incomplete message * feat: record tokenUsage when message is aborted * refactor: subtract tokens when payload includes function_call * refactor: add namespace for token_balance * fix(spendTokens): only execute if corresponding token type amounts are defined * refactor(checkBalance): throws Error if not enough token credits * refactor(runTitleChain): pass and use signal, spread object props in create helpers, and use 'call' instead of 'run' * fix(abortMiddleware): circular dependency, and default to empty string for completionTokens * fix: properly cancel title requests when there isn't enough tokens to generate * feat(predictNewSummary): custom chain for summaries to allow signal passing refactor(summaryBuffer): use new custom chain * feat(RunManager): add getRunByConversationId method, refactor: remove run and throw llm error on handleLLMError * refactor(createStartHandler): if summary, add error details to runs * fix(OpenAIClient): support aborting from summarization & showing error to user refactor(summarizeMessages): remove unnecessary operations counting summaryPromptTokens and note for alternative, pass signal to summaryBuffer * refactor(logTokenCost -> recordTokenUsage): rename * refactor(checkBalance): include promptTokens in errorMessage * refactor(checkBalance/spendTokens): move to models dir * fix(createLanguageChain): correctly pass config * refactor(initializeLLM/title): add tokenBuffer of 150 for balance check * refactor(openAPIPlugin): pass signal and memory, filter functions by the one being called * refactor(createStartHandler): add error to run if context is plugins as well * refactor(RunManager/handleLLMError): throw error immediately if plugins, don't remove run * refactor(PluginsClient): pass memory and signal to tools, cleanup error handling logic * chore: use absolute equality for addTitle condition * refactor(checkBalance): move checkBalance to execute after userMessage and tokenCounts are saved, also make conditional * style: icon changes to match official * fix(BaseClient): getTokenCountForResponse -> getTokenCount * fix(formatLangChainMessages): add kwargs as fallback prop from lc_kwargs, update JSDoc * refactor(Tx.create): does not update balance if CHECK_BALANCE is not enabled * fix(e2e/cleanUp): cleanup new collections, import all model methods from index * fix(config/add-balance): add uncaughtException listener * fix: circular dependency * refactor(initializeLLM/checkBalance): append new generations to errorMessage if cost exceeds balance * fix(handleResponseMessage): only record token usage in this method if not error and completion is not skipped * fix(createStartHandler): correct condition for generations * chore: bump postcss due to moderate severity vulnerability * chore: bump zod due to low severity vulnerability * chore: bump openai & data-provider version * feat(types): OpenAI Message types * chore: update bun lockfile * refactor(CodeBlock): add error block formatting * refactor(utils/Plugin): factor out formatJSON and cn to separate files (json.ts and cn.ts), add extractJSON * chore(logViolation): delete user_id after error is logged * refactor(getMessageError -> Error): change to React.FC, add token_balance handling, use extractJSON to determine JSON instead of regex * fix(DALL-E): use latest openai SDK * chore: reorganize imports, fix type issue * feat(server): add balance route * fix(api/models): add auth * feat(data-provider): /api/balance query * feat: show balance if checking is enabled, refetch on final message or error * chore: update docs, .env.example with token_usage info, add balance script command * fix(Balance): fallback to empty obj for balance query * style: slight adjustment of balance element * docs(token_usage): add PR notes |
||
|
Danny Avila
|
fd70e21732
|
feat: OpenRouter Support & Improve Model Fetching ⇆ (#936)
* chore(ChatGPTClient.js): add support for OpenRouter API chore(OpenAIClient.js): add support for OpenRouter API * chore: comment out token debugging * chore: add back streamResult assignment * chore: remove double condition/assignment from merging * refactor(routes/endpoints): -> controller/services logic * feat: add openrouter model fetching * chore: remove unused endpointsConfig in cleanupPreset function * refactor: separate models concern from endpointsConfig * refactor(data-provider): add TModels type and make TEndpointsConfig adaptible to new endpoint keys * refactor: complete models endpoint service in data-provider * refactor: onMutate for refreshToken and login, invalidate models query * feat: complete models endpoint logic for frontend * chore: remove requireJwtAuth from /api/endpoints and /api/models as not implemented yet * fix: endpoint will not be overwritten and instead use active value * feat: openrouter support for plugins * chore(EndpointOptionsDialog): remove unused recoil value * refactor(schemas/parseConvo): add handling of secondaryModels to use first of defined secondary models, which includes last selected one as first, or default to the convo's secondary model value * refactor: remove hooks from store and move to hooks refactor(switchToConversation): make switchToConversation use latest recoil state, which is necessary to get the most up-to-date models list, replace wrapper function refactor(getDefaultConversation): factor out logic into 3 pieces to reduce complexity. * fix: backend tests * feat: optimistic update by calling newConvo when models are fetched * feat: openrouter support for titling convos * feat: cache models fetch * chore: add missing dep to AuthContext useEffect * chore: fix useTimeout types * chore: delete old getDefaultConvo file * chore: remove newConvo logic from Root, remove console log from api models caching * chore: ensure bun is used for building in b:client script * fix: default endpoint will not default to null on a completely fresh login (no localStorage/cookies) * chore: add openrouter docs to free_ai_apis.md and .env.example * chore: remove openrouter console logs * feat: add debugging env variable for Plugins |
||
|
Danny Avila
|
4ca43fb53d
|
refactor: Encrypt & Expire User Provided Keys, feat: Rate Limiting (#874)
* docs: make_your_own.md formatting fix for mkdocs * feat: add express-mongo-sanitize feat: add login/registration rate limiting * chore: remove unnecessary console log * wip: remove token handling from localStorage to encrypted DB solution * refactor: minor change to UserService * fix mongo query and add keys route to server * fix backend controllers and simplify schema/crud * refactor: rename token to key to separate from access/refresh tokens, setTokenDialog -> setKeyDialog * refactor(schemas): TEndpointOption token -> key * refactor(api): use new encrypted key retrieval system * fix(SetKeyDialog): fix key prop error * fix(abortMiddleware): pass random UUID if messageId is not generated yet for proper error display on frontend * fix(getUserKey): wrong prop passed in arg, adds error handling * fix: prevent message without conversationId from saving to DB, prevents branching on the frontend to a new top-level branch * refactor: change wording of multiple display messages * refactor(checkExpiry -> checkUserKeyExpiry): move to UserService file * fix: type imports from common * refactor(SubmitButton): convert to TS * refactor(key.ts): change localStorage map key name * refactor: add new custom tailwind classes to better match openAI colors * chore: remove unnecessary warning and catch ScreenShot error * refactor: move userKey frontend logic to hooks and remove use of localStorage and instead query the DB * refactor: invalidate correct query key, memoize userKey hook, conditionally render SetKeyDialog to avoid unnecessary calls, refactor SubmitButton props and useEffect for showing 'provide key first' * fix(SetKeyDialog): use enum-like object for expiry values feat(Dropdown): add optionsClassName to dynamically change dropdown options container classes * fix: handle edge case where user had provided a key but the server changes to env variable for keys * refactor(OpenAI/titleConvo): move titling to client to retain authorized credentials in message lifecycle for titling * fix(azure): handle user_provided keys correctly for azure * feat: send user Id to OpenAI to differentiate users in completion requests * refactor(OpenAI/titleConvo): adding tokens helps minimize LLM from using the language in title response * feat: add delete endpoint for keys * chore: remove throttling of title * feat: add 'Data controls' to Settings, add 'Revoke' keys feature in Key Dialog and Data controls * refactor: reorganize PluginsClient files in langchain format * feat: use langchain for titling convos * chore: cleanup titling convo, with fallback to original method, escape braces, use only snippet for language detection * refactor: move helper functions to appropriate langchain folders for reusability * fix: userProvidesKey handling for gptPlugins * fix: frontend handling of plugins key * chore: cleanup logging and ts-ignore SSE * fix: forwardRef misuse in DangerButton * fix(GoogleConfig/FileUpload): localize errors and simplify validation with zod * fix: cleanup google logging and fix user provided key handling * chore: remove titling from google * chore: removing logging from browser endpoint * wip: fix menu flicker * feat: useLocalStorage hook * feat: add Tooltip for UI * refactor(EndpointMenu): utilize Tooltip and useLocalStorage, remove old 'New Chat' slide-over * fix(e2e): use testId for endpoint menu trigger * chore: final touches to EndpointMenu before future refactor to declutter component * refactor(localization): change select endpoint to open menu and add translations * chore: add final prop to error message response * ci: minor edits to facilitate testing * ci: new e2e test which tests for new key setting/revoking features |
||
|
Danny Avila
|
7dc27b10f1
|
feat: Edit AI Messages, Edit Messages in Place (#825)
* refactor: replace lodash import with specific function import fix(api): esm imports to cjs * refactor(Messages.tsx): convert to TS, out-source scrollToDiv logic to a custom hook fix(ScreenshotContext.tsx): change Ref to RefObject in ScreenshotContextType feat(useScrollToRef.ts): add useScrollToRef hook for scrolling to a ref with throttle fix(Chat.tsx): update import path for Messages component fix(Search.tsx): update import path for Messages component * chore(types.ts): add TAskProps and TOptions types refactor(useMessageHandler.ts): use TAskFunction type for ask function signature * refactor(Message/Content): convert to TS, move Plugin component to Content dir * feat(MessageContent.tsx): add MessageContent component for displaying and editing message content feat(index.ts): export MessageContent component from Messages/Content directory * wip(Message.jsx): conversion and use of new component in progress * refactor: convert Message.jsx to TS and fix typing/imports based on changes * refactor: add typed props and refactor MultiMessage to TS, fix typing issues resulting from the conversion * edit message in progress * feat: complete edit AI message logic, refactor continue logic * feat(middleware): add validateMessageReq middleware feat(routes): add validation for message requests using validateMessageReq middleware feat(routes): add create, read, update, and delete routes for messages * feat: complete frontend logic for editing messages in place feat(messages.js): update route for updating a specific message - Change the route for updating a message to include the messageId in the URL - Update the request handler to use the messageId from the request parameters and the text from the request body - Call the updateMessage function with the updated parameters feat(MessageContent.tsx): add functionality to update a message - Import the useUpdateMessageMutation hook from the data provider - Destructure the conversationId, parentMessageId, and messageId from the message object - Create a mutation function using the useUpdateMessageMutation hook - Implement the updateMessage function to call the mutation function with the updated message parameters - Update the messages state to reflect the updated message text feat(api-endpoints.ts): update messages endpoint to include messageId - Update the messages endpoint to include the messageId as an optional parameter feat(data-service.ts): add updateMessage function - Implement the updateMessage function to make a PUT request to * fix(messages.js): make updateMessage function asynchronous and await its execution * style(EditIcon): make icon active for AI message * feat(gptPlugins/anthropic): add edit support * fix(validateMessageReq.js): handle case when conversationId is 'new' and return empty array feat(Message.tsx): pass message prop to SiblingSwitch component refactor(SiblingSwitch.tsx): convert to TS * fix(useMessageHandler.ts): remove message from currentMessages if isContinued is true feat(useMessageHandler.ts): add support for submission messages in setMessages fix(useServerStream.ts): remove unnecessary conditional in setMessages fix(useServerStream.ts): remove isContinued variable from submission * fix(continue): switch to continued message generation when continuing an earlier branch in conversation * fix(abortMiddleware.js): fix condition to check partialText length chore(abortMiddleware.js): add error logging when abortMessage fails * refactor(MessageHeader.tsx): convert to TS fix(Plugin.tsx): add default value for className prop in Plugin component * refactor(MultiMessage.tsx): remove commented out code docs(MultiMessage.tsx): update comment to clarify when siblingIdx is reset * fix(GenerationButtons): optimistic state for continue button * fix(MessageContent.tsx): add data-testid attribute to message text editor fix(messages.spec.ts): update waitForServerStream function to include edit endpoint check feat(messages.spec.ts): add test case for editing messages * fix(HoverButtons & Message & useGenerations): Refactor edit functionality and related conditions - Update enterEdit function signature and prop - Create and utilize hideEditButton variable - Enhance conditions for edit button visibility and active state - Update button event handlers - Introduce isEditableEndpoint in useGenerations and refine continueSupported condition. * fix(useGenerations.ts): fix condition for hideEditButton to include error and searchResult chore(data-provider): bump version to 0.1.6 fix(types.ts): add status property to TError type * chore: bump @dqbd/tiktoken to 1.0.7 * fix(abortMiddleware.js): add required isCreatedByUser property to the error response object * refactor(Message.tsx): remove unnecessary props from SiblingSwitch component, as setLatestMessage is firing on every switch already refactor(SiblingSwitch.tsx): remove unused imports and code * chore(BaseClient.js): move console.debug statements back inside if block |
||
|
Daniel Avila
|
77a76f8511 | chore: add back data-provider | ||
|
Danny Avila
|
369b1f4eba
|
chore: remove data-provider and use npm package instead (#713)
* chore: remove data-provider, install npm package * chore: replace monorepo package with npm package: librechat-data-provider * chore: remove data-provider scripts * chore: remove data-provider from .eslintrc.js |
||
|
Danny Avila
|
e5336039fc
|
ci(backend-review.yml): add linter step to the backend review workflow (#625)
* ci(backend-review.yml): add linter step to the backend review workflow * chore(backend-review.yml): remove prettier from lint-action configuration * chore: apply new linting workflow * chore(lint-staged.config.js): reorder lint-staged tasks for JavaScript and TypeScript files * chore(eslint): update ignorePatterns in .eslintrc.js chore(lint-action): remove prettier option in backend-review.yml chore(package.json): add lint and lint:fix scripts * chore(lint-staged.config.js): remove prettier --write command for js, jsx, ts, tsx files * chore(titleConvo.js): remove unnecessary console.log statement chore(titleConvo.js): add missing comma in options object * chore: apply linting to all files * chore(lint-staged.config.js): update lint-staged configuration to include prettier formatting |
||
|
Dan Orlando
|
04e4259005
|
Move data provider to shared package (#582)
* create data-provider package and move code from data-provider folder to be shared between apps * fix type issues * add packages to ignore * add new data-provider package to apps * refactor: change client imports to use @librechat/data-provider package * include data-provider build script in frontend build * fix type issue after rebasing * delete admin/package.json from this branch * update test ci script to include building of data-provider package * Try using regular build for test action * Switch frontend-review back to build:ci * Remove loginRedirect from Login.tsx * Add ChatGPT back to EModelEndpoint |
Renamed from client/src/data-provider/api-endpoints.ts (Browse further)