Commit graph

186 commits

Author SHA1 Message Date
Danny Avila
16c9aed1bb
🤖 feat: Recognize chatgpt-4o-latest, update default OpenAI Models (#3667)
* 🤖 feat: recognize chatgpt-4o-latest, update default OpenAI models

* chore: Bump data-provider version to 0.7.412
2024-08-16 15:28:17 -04:00
Marco Beretta
96581d56df
🖼️ style: Conversation Menu and Dialogs update (#3601)
* feat: new dropdown

* fix: maintain popover active when open

* fix: update DeleteButton and ShareButton component to use useState for managing dialog state

* BREAKING: style improvement of base Button component

* style: update export button

* a11y: ExportAndShareButton

* add border

* quick style fix

* fix: flick issue on convo

* fix: DropDown opens when renaming

* chore: update radix-ui/react-dropdown-menu to latest

* small fix

* style: bookmarks update

* reorder export modal

* feat: imporved dropdowns

* style: a lot of changes; header, bookmarks, export, nav, convo, convoOptions

* fix: small style issues

* fix: button

* fix: bookmarks header menu

* fix: dropdown close glitch

* feat: Improve accessibility and keyboard navigation in ModelSpec component

* fix: Nav related type issues

* style: ConvoOptions theming and focus ring

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2024-08-16 04:30:14 -04:00
Yuichi Oneda
e47c3f40f0
🔧 fix: Bookmark Order Adjustment When Moving Up (#3634) 2024-08-16 03:01:35 -04:00
Danny Avila
1ff4841603
🧹 chore: pre-release cleanup 2 (#3600)
* refactor: scrollToEnd

* fix(validateConvoAccess): search conversation by ID for proper validation

* feat: Add unique index for conversationId and user in convoSchema

* refactor: Update font sizes 1 rem -> font-size-base in style.css

* fix: Assistants map type issues

* refactor: Remove obsolete scripts

* fix: Update DropdownNoState component to handle both string and OptionType values

* refactor: Remove config/loader.js file

* fix: remove crypto.randomBytes(); refactor: Create reusable function for generating token and hash
2024-08-09 15:17:13 -04:00
Danny Avila
473859b0e2
🧹 chore: pre-release cleanup (#3595)
* fix: Update health endpoint URL

* refactor: use Constants for saved tag default value, do not place Saved as first always

* refactor: check trimmed currentText before appending parsedText in useSubmitMessage

* refactor: move `scrollToEnd()` to `createdHandler` and increase delay before execution slightly

* chore: Add back TypeScript linting rules for unnecessary conditions and strict boolean expressions

* chore: Update librechat-data-provider package.json version to 0.7.4.0
2024-08-09 02:11:56 -04:00
Danny Avila
e05a6d306d
🪙 feat: Update Token Values for gpt-4o-2024-08-06 and AWS Models (#3594)
* feat: gpt-4o-2024-08-06 pricing for tx

* feat: add AWS models to tokenValues in tx.js for pricing transactions

* feat: Update tokenValues in tx.js for AWS models pricing

* refactor: add bedrock prefix values as well (temporary until we update value keys which includes context)
2024-08-08 23:31:07 -04:00
Danny Avila
016ed866a3
🏷️ fix: Address Statefulness Issues for Bookmarks (#3590)
* refactor: optimize tag methods, remove rebuild

* refactor(tags): add lean db operations, fix updateTagsForConversation, remove rebuild button, only send convoId once

* refactor: Update BookmarkMenu to use Constants.NEW_CONVO constant for comparison

* style: Update BookmarkMenu styles and constants, use theming

* refactor: move tags query from package to client workspace

* refactor: optimize ConversationTag document creation and update logic

* style: Update BookmarkMenuItems to use theming

* refactor: JSDocs + try/catch for conversation tags API routes

* refactor: Update BookmarkNav theming classes and new data provider location

* fix: statefulness of conversation bookmarks
- move non-mutation hook to hooks/Conversation
- remove use of deprecated global convo
- update convo infinite data as well as current convo state upon successful tag add

* refactor: Update BookmarkMenu styles and constants, use theming

* refactor: Add lean option to ConversationTag deletion query

* fix(BookmarkTable): position order rendering esp. when new tag is created

* refactor: Update useBookmarkSucess to useBookmarkSuccess for consistency

* refactor: Update ConversationTag creation logic to increment count only if addToConversation is true

* style: theming
2024-08-08 21:25:10 -04:00
Danny Avila
458dc9c88e
🛡️ feat: Optimize and Improve Anonymity of SharedLinks (#3543)
* ci: add spec for validateImageRequest

* chore: install nanoid and run npm audit fix

* refactor: optimize and further anonymize shared links data

* ci: add SharedLink specs

* feat: anonymize asst_id's

* ci: tests actually failing, to revisit later

* fix: do not anonymize authenticated shared links query

* refactor: Update ShareView component styling

* refactor: remove nested ternary

* refactor: no nested ternaries

* fix(ShareView): eslint warnings

* fix(eslint): remove nested terns
2024-08-05 03:34:00 -04:00
matt burnett
3e0f95458f
🕸️ refactor: Migrate from crypto to Web Crypto API (#3357)
* move crypto to async webcrypto

update encrypt/decrypt

forgot await

* chore: import order - openidStrategy.js

* chore: import order - Session.js

* chore: import order - AuthController.js

* Update AuthService.js

---------

Co-authored-by: Danny Avila <danacordially@gmail.com>
2024-08-04 23:59:45 -04:00
Danny Avila
11bfed7126
🤲 feat(a11y): Initial a11y improvements, added linters, tests; fix: close sidebars in mobile view (#3536)
* chore: playwright setup update

* refactor: update ChatRoute component with accessible loading spinner with live region

* chore(Message): typing

* ci: first pass, a11y testing

* refactor: update lang attribute in index.html to "en-US"

* ci: jsx-a11y dev eslint plugin

* ci: jsx plugin

* fix: Exclude 'vite.config.ts' from TypeScript compilation for testing

* fix(a11y): Remove tabIndex from non-interactive element in MessagesView component

* fix(a11y):
- Visible, non-interactive elements with click handlers must have at least one keyboard listener.eslintjsx-a11y/click-events-have-key-events
- Avoid non-native interactive elements. If using native HTML is not possible, add an appropriate role and support for tabbing, mouse, keyboard, and touch inputs to an interactive content element.eslintjsx-a11y/no-static-element-interactions
chore: remove unused bookmarks panel
- fix some "Unexpected nullable boolean value in conditional" warnings

* fix(NewChat): a11y, nested button issue, add aria-label, remove implicit role

* fix(a11y):
- partially address #3515 with `main` landmark
other:
- eslint@typescript-eslint/strict-boolean-expressions

* chore(MenuButton): Use button element instead of div for accessibility

* chore: Update TitleButton to use button element for accessibility

* chore: Update TitleButton to use button element for accessibility

* refactor(ChatMenuItem): Improve focus accessibility and code readability

* chore(MenuButton): Update aria-label to dynamically include primaryText

* fix(a11y): SearchBar
- If a form control does not have a properly associated text label, the function or purpose of that form control may not be presented to screen reader users. Visible form labels also provide visible descriptions and larger clickable targets for form controls which placeholders do not.

* chore: remove duplicate SearchBar twcss

* fix(a11y):
- The edit and copy buttons that are visually hidden are exposed to Assistive technology and are announced to screen reader users.

* fix(a11y): visible focus outline

* fix(a11y): The button to select the LLM Model has the aria-haspopup and aria- expanded attributes which makes its role ambuguous and unclear. It functions like a combobox but doesn't fully support that interaction and also fucntions like a dialog but doesn't completely support that interaction either.

* fix(a11y): fix visible focus outline

* fix(a11y): Scroll to bottom button missing accessible name #3474

* fix(a11y): The page lacks any heading structure. There should be at least one H1 and other headings to help users understand the orgainzation of the page and the contents.

Note: h1 won't be correct here so made it h2

* fix(a11y): LLM controls aria-labels

* fix(a11y): There is no visible focus outline to the 'send message' button

* fix(a11y): fix visible focus outline for Fork button

* refactor(MessageRender): add focus ring to message cards, consolidate complex conditions, add logger for setting latest message, add tabindex for card

* fix: focus border color and fix set latest message card condition

* fix(a11y): Adequate contrast for MessageAudio buttton

* feat: Add GitHub Actions workflow for accessibility linting

* chore: Update GitHub Actions workflow for accessibility linting to include client/src/** path

* fix(Nav): navmask and accessibility

* fix: Update Nav component to handle potential undefined type in SearchContext

* fix(a11y): add focus visibility to attach files button #3475

* fix(a11y): discernible text for NewChat button

* fix(a11y): accessible landmark names, all page content in landmarks, ensures landmarks are unique #3514 #3515

* fix(Prompts): update isChatRoute prop to be required in List component

* fix(a11y): buttons must have discernible text
2024-08-04 20:39:52 -04:00
Danny Avila
b8a6efa649
👷‍♂️ chore: pass Tavily Response error and revert test commit on userSchema.js (#3496)
* chore: revert test commit

* chore: make sure tavily error gets passed
2024-07-31 12:06:27 -04:00
Danny Avila
4ffdefc2a8
🔧 refactor: update userSchema default value for role 2024-07-30 01:48:20 -04:00
Yuichi Oneda
e565e0faab
🔖 feat: Conversation Bookmarks (#3344)
* feat: add tags property in Conversation model

* feat: add ConversationTag model

* feat: add the tags parameter to getConvosByPage

* feat: add API route to ConversationTag

* feat: add types of ConversationTag

* feat: add data access functions for conversation tags

* feat: add Bookmark table component

* feat: Add an action to bookmark

* feat: add Bookmark nav component

* fix: failed test

* refactor: made 'Saved' tag a constant

* feat: add new bookmark to current conversation

* chore: Add comment

* fix: delete tag from conversations when it's deleted

* fix: Update the query cache when the tag title is changed.

* chore: fix typo

* refactor: add description of rebuilding bookmarks

* chore: remove unused variables

* fix: position when adding a new bookmark

* refactor: add comment, rename a function

* refactor: add a unique constraint in ConversationTag

* chore: add localizations
2024-07-29 10:45:59 -04:00
Danny Avila
e5dfa06e6c
🎚️ chore: Update Logging Levels for Conversations and Messages (#3415) 2024-07-21 13:54:47 -04:00
Danny Avila
2ad097647c
fix: Wait for Initial Message Save & Correct Latest Message (#3399)
* chore: assistants, unsupported assistant, better logging

* chore: remove unnecessary logger in validateAssistant middleware

* fix: resolve initial conversation save/promise before saving response

* chore: Import and organize dependencies in Speech component

* fix: conversation statefulness
- Latest Message (at index 0) should not be reset if existing convo
- add debugging context for clearAllLatestMessages
- Added logging concerning latest Message updates (dev mode only)
- update latest message Set logic, also checks for change in conversation Id
- consolidated latest message helpers to client/src/utils/messages.ts
2024-07-20 01:51:59 -04:00
Danny Avila
9e7615f832
⚙️ fix: Plugin Message Handling Errors (#3392)
- Add unique index for messageId and user in messageSchema
- use `updateMessage` for updating the plugins message?
- add better logging for updateMessage
- prevents dupe_key or getKeyIndex error
2024-07-19 08:06:05 -04:00
Marco Beretta
ee4dd1b2e9
🚀 feat: gpt-4o-mini (#3384)
* feat: `gpt-4o-mini`

* feat: retrival

* fix: Update order of model token values for 'gpt-4o' and 'gpt-4o-mini'

* fix: Update order of model token values for 'gpt-4o' and 'gpt-4o-mini'

* fix: Update order of model token values for 'gpt-4o' and 'gpt-4o-mini'

* fix: add jsdoc

* fix: Update order of model token values for 'gpt-4o' and 'gpt-4o-mini'

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2024-07-19 07:59:07 -04:00
Danny Avila
f6125ccd59
🌿 fix: Message Route Improvements pt. 2 (#3379)
* fix: edge case with debugTraverse function in parsers.js

* chore: Update error message in audio stream processing

* refactor: Add ONE_MINUTE and THIRTY_SECONDS options to Time enum

* fix: logging
2024-07-18 13:56:19 -04:00
Danny Avila
1acd47a0f6
🌿 fix: Message Route Improvements (#3378)
* fix: do not throw errors for invalid convo id, simply return undefined

* feat: Add error handling and logging to message route definitions

* feat: Refactor Message.js to improve code organization and readability

* fix: Return undefined instead of throwing error for invalid conversation ID in Message.spec.js
2024-07-18 10:07:10 -04:00
Danny Avila
d5d188eebf
🔐 fix: Enhance Message & Image Access Security (#3363)
* chore: slight refactor

* fix: prevent message updates unless explicitly owned

* refactor: rethrow errors, update deleteMessagesSince (not used), add basic tests

* fix: Add path normalization and validation to image request middleware

* fix: image validation path security
2024-07-17 09:51:03 -04:00
Danny Avila
a87d4e0b75
🛠️ fix: Update Conversation and Message Models to Return Objects Instead of Using Lean() (#3230) 2024-06-28 21:57:53 -04:00
Danny Avila
a2fd975cd5
🚤 refactor: Optimize Request Lifecycle Speeds (#3222)
* refactor: optimize backend operations for client requests

* fix: message styling

* refactor: Improve handleKeyUp logic in StreamRunManager.js and handleText.js

* refactor: Improve handleKeyUp logic in StreamRunManager.js and handleText.js

* fix: clear new convo messages on clear all convos

* fix: forgot to pass userId to getConvo

* refactor: update getPartialText to send basePayload.text
2024-06-28 08:44:47 -04:00
Danny Avila
83619de158
🗨️ feat: Prompt Slash Commands (#3219)
* chore: Update prompt description placeholder text

* fix: promptsPathPattern to not include new

* feat: command input and styling change for prompt views

* fix: intended validation

* feat: prompts slash command

* chore: localizations and fix add command during creation

* refactor(PromptsCommand): better label

* feat: update `allPrompGroups` cache on all promptGroups mutations

* refactor: ensure assistants builder is first within sidepanel

* refactor: allow defining emailVerified via create-user script
2024-06-27 17:34:48 -04:00
Danny Avila
aac01df80c
🧹 chore: remove unnecessary try/catch when creating users (#3153) 2024-06-21 15:14:18 -04:00
Yuichi Oneda
4e4de88faa
🔧 fix(Shared Links): Handling Shared Link Errors (#3118)
* refactor: add error handling in Share model

* refactor: add error handing to API routers

* refactor: display error message when API response is an error

* chore: remove unneccesary JSON.stringify

* chore: revert unintended changes

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2024-06-21 10:12:37 -04:00
Danny Avila
5d8793c5d1
🗨️ refactor: Improve Prompts Query (#3138) 2024-06-20 22:21:17 -04:00
Danny Avila
54db67449a
🧠 feat: claude-3-5-sonnet (#3135)
* 🧠 feat: claude-3-5-sonnet

* chore: bump data-provider
2024-06-20 20:48:15 -04:00
Danny Avila
0cd3c83328
🗨️ feat: Prompts (#3131)
* 🗨️ feat: Prompts (#7)

* WIP: MERGE prompts/frontend (#1)

* added schema for prompt and promptgroup, added model methods for prompts, added routes for prompts

* * updated promptGroup Schema

* updated model methods for prompts (get, add, delete)

* slight fixes in prompt routes

* * Created Files Management components

* Created Vector Stores components

* Added file management route in the routes folder

* Completed UI for Files list, Compeleted UI for vector stores list, Completed UI for upload file modal, Completed UI for preview file, Completed UI for preview vector store

* Fixed style and UI fixes for file dashboard, file list and vector stores list

* added responsiveness classes for vector store page

* fixed responsiveness of file page, dashboard page, and main page

* fixed styling and responsiveness issues on dashboard page, file list page and vector store page

* added queries and mutations for prompts and promptGroups, added relevant endpoints in data-provider, added relevant components prompts, added and updated relevant APIs

* added types on mutation queries data service, updated prompt attributes

* feature: Prompts and prompt groups management, added relevant APIs, added types for data service/queries/mutations, added relevant mutation and queries

* chore: typing clarifications

* added drop down on prompts mgmt dashboard

* Fixes: fixed version switching issue on tags update or labels update, added cross button on create prompt group, fixed list updation on prompt group renaiming, added CSV upload button

* Feature: Added oneliner and category attributes in prompt group, added schema for categories, added schema methods and route for categories

* chore: typing and lint issues

* chore: more type and linter fixes

* chore: linting

* chore: prompt controller and backend typing example; MOVE TO CONTROLLER DIRECTORY

* chore: more type fixes

* style: prompt name changes

* chore: more type changes, and stateful prompt name change without flickering

* fix: Return result of savePrompt in patchPrompt API endpoint

* fix: navigation prompt queries; refactor: name 'prompt-groups' to just 'groups'

* refactor: fetch prompt groups rewrite

* refactor(prompts): query/mutation statefulness

* refactor: remove `isActive` field

* refactor: remove labels, consolidate logic

* style: width, layout shift

* refactor: improve hover toggle behavior and styling

* refactor: add useParams hook to PromptListItem for dynamic rendering and add timeout ref for blur timeout

* chore: hide upload button

* refactor: import Button component from correct location in PromptSidePanel

* style: prompt editor styling

* style: fix more layout shifts

* style: container scroll

* refactor: Rename CreatePrompt component to CreatePromptForm

* refactor: use react-hook-form

* refactor: Add Prompts components and routes to Dashboard

* style: skeletons for loading

* fix: optimize makePromptProduction

* refactor: consolidate variables

* feat: create prompt form validation

* refactor: Consolidate variables and update mutation hooks

* style: minor touchups

* chore: Update lucide-react npm dependency to version 0.394.0 and npm audit fix

* refactor: add a new icon for the Prompts heading.

* style: Update PromptsView heading to use h1 instead of h2 and other minor margin issues

* chore: wording

* refactor: Update PromptsView heading to use h1 instead of h2, consolidate variables, and add new icons

* refactor: Prompts Button for Mobile

* feature: added category field in prompt group, added relevant API and static data on BE to support FE UI for category in prompt group

* chore: template for prompt cards

---------

Co-authored-by: Fawadpot <contactfawada@gmail.com>

* WIP: Prompts/frontend Continued (#2)

* chore: loading style, remove unused component

* feat: Add CategorySelector component for prompt group category selection

* feat: add categories to create prompt

* feat: prompt versions styling

* feat: optimistic updates for prompt production state

* refactor: optimize form state and show if prompt field is dirty with cross icon, also other styling changes

* chore: remove unused code and localizations

* fix: light mode styling

* WIP: SidePanel Prompts

* refactor: move to groups directory

* refactor: rename GroupsSidePanel to GroupSidePanel and update imports

* style: ListCard

* refactor: isProduction changes

* refactor: infinite query with productionPrompt

* refactor: optimize snippets and prompts, and styling

* refactor: Update getSnippet function to accept a length parameter

* chore: localizations

* feat: prompts navigation to chat and vice versa

* fix: create prompt

* feat: remember last selected category for creating prompts

* fix(promptGroups): fix pagination and add usePromptGroupsNav hook

* Prompts/frontend 3 (#3)

* fix: stateful issues with prompt groups

* style: improved layout

* refactor: improve variable naming in Eng.ts

* refactor: theme selector styling improvements

* added prompt cards on chat new page, with dark mode, added API to fetch random prompts, added types for useQuery

Slightly improved usePromptGroupNav logic to fetch updated result for pageSize, updated prompt cards view with darkmode and responsiveness

fixed page size option buttons styling to match the theme

added dark mode on create prompt page and prompt edit/preview page

fixed page size option buttons styling to match the theme

added dark mode on create prompt page and prompt edit/preview page

* WIP: Prompts/frontend (#4)

* fix: optimize and fix paginated query

* fix: remove unique constraint on names

* refactor: button links and styling

* style: menu border light mode

* feat: Add Auto-Send Switch component for prompts groups

* refactor(ChatView): use form context for submission text

* chore: clear convo state on navigation to dashboard routes

* chore: save prompt edit name on tab, remove console log

* feat: basic prompt submission

* refactor: move Auto-Send Switch

* style(ListCard): border styling

* feat: Add function to detect variables in text

* feat: Add OriginalDialog component to UI library

* chore(ui): Update SelectDropDown options list class to use text-xs size

* refactor: submitMessage hook now includes submitPrompt, make compatible to document query selector

* WIP: Variable Dialog

* feat: variable submission working for both auto-send and non-autosend

* feat: dashboard breadcrumbs and prompts/chat navigation

* refactor: dashboard breadcrumb and dashboard link to chat navigation

* refactor: Update VariableDialog and VariableForm styles

* Prompts: Admin features (#5)

* fix: link issue

* fix: usePromptGroupsNav add missing dep.

* style: dashbreadcrumb and sidepanel text color

* temp fix: remove refetch on pageNumber change

* fix: handle multiple variable replacement

* WIP: create project schema and add project groups to fetch

* feat: Add functionality to add prompt group IDs to a project

* feat: Add caching for startup config in config route

* chore: remove prompt landing

* style: Update Skeleton component with additional background styling

* chore: styling and types

* WIP: SharePrompt first draft

* feat(SharePrompt): form validation

* feat: shared global indicators

* refactor: prompt details

* refactor: change NoPromptGroup directory

* feat: preview prompt

* feat: remove/add global prompts, add rbac-related enums

* refactor: manage prompts location

* WIP: first draft admin settings for prompts

* feat: SystemRoles enum

* refactor: update PromptDetails component styling

* style: ellipsis custom class for showing more preview text

* WIP: initial role schema and initialization

* style: improved margins for single unordered lists

* fix: use custom chat form context to prevent re-renders from FormProvider

* feat: Role mutations for Prompt Permissions

* feat: fetch user role

* feat: update AdminSettings form default values from user role values

* refactor: rename PromptPermissions to Permissions for general definitions

* feat: initial role checks

* feat: Add optional `bodyProps` parameter to generateCheckAccess middleware

* refactor: UI access checks

* Prompts: delete (#6)

* Fixed delete prompt version API, fixed types and logic for prompt version deletion, updated prompt delete mutation logic

* chore: Update return type of deletePrompt function in Prompt.js

---------

Co-authored-by: Fawadpot <contactfawada@gmail.com>

* chore: Update package-lock.json version to 0.7.4-rc1 and fast-xml-parser to 4.4.0

* feat: toast for saving admin settings, add timer no-access navigation

* feat: always make prod

* feat: Add localization to category labels in CategorySelector component

* feat: Update category label localization in CategorySelector component

* fix: Enable making prompt production in Prompt API

---------

Co-authored-by: Fawadpot <contactfawada@gmail.com>

* feat: Add helper fn for dark mode detection in ThemeProvider

* style: surface-primary definition

* fix(useHasAccess): utilize user.role and not just USER role

* fix: empty category and role fetch

* refactort: increase max height to options list and use label if no localization is found

* fix: update CategorySelector to handle empty category value and improve localization

* refactor: move prompts to own store/reactquery modules, add in filter WIP

* refactor: Rename AutoSendSwitch to AutoSendPrompt

* style: theming commit

* style: fix slight coloring issue for convos in dark mode

* style: better composition for prompts side panel

* style: remove gray-750 and make it gray-850

* chore: adjust theming

* feat: filter all prompt groups and properly remove prompts from projects

* refactor: optimize delete prompt groups further

* chore: localization

* feat: Add uniqueProperty filtering to normalizeData function

* WIP: filter prompts

* chore: Update FilterPrompts component to include User icon in FilterItem

* feat(FilterPrompts): set categories

* feat: more system filters and show selected category icon

* style: always make prod, flips switch to avoid mis-clicks

* style: ui/ux loading/no prompts

* chore: style FilterPrompts ChatView

* fix: handle missing role edge case

* style: special variables

* feat: special variables

* refactor: improve replaceSpecialVars function in prompts.ts

* feat: simple/advanced editor modes

* chore: bump versions

* feat: localizations and hide production button on simple mode

* fix: error connecting layout shift

* fix: prompts CRUD for admins

* fix: secure single group fetch

* style: sidepanel styling

* style(PromptName): bring edit button closer to name

* style: mobile prompts header

* style: mobile prompts header continued

* style: align send prompts switch right

* feat: description

* Update special variables description in Eng.ts

* feat: update/create/preview oneliner

* fix: allow empty oneliner update

* style: loading improvement and always make selected prompt Production if simple mode

* fix: production index set and remove unused props

* fix(ci): mock initializeRoles

* fix: address #3128

* fix: address #3128

* feat: add deletion confirmation dialog

* fix: mobile UI issues

* style: prompt library UI update

* style: focus, logcal tab order

* style: Refactor SelectDropDown component to improve code readability and maintainability

* chore: bump data-provider

* chore: fix labels

* refactor: confirm delete prompt version

---------

Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
2024-06-20 20:24:32 -04:00
Danny Avila
302b28fc9b
v0.7.4-rc1 (#3099)
* fix(openIdStrategy): return user object on new user creation

*  v0.7.4-rc1
2024-06-17 12:47:28 -04:00
Danny Avila
04502e9525
👤 fix: Create User with timestamps (#3070)
* 👤 fix: Create User with timestamps

* chore: fix lint script to ignore venv

* chore: linting
2024-06-15 10:36:49 -04:00
Danny Avila
4416f69a9b
🚀 refactor: Use Undici Instead of Node-Fetch, prevent Event Close, add Index (#3052)
* feat: Add index to conversationId field in messageSchema

* refactor: prevent immediate event close on error

* refactor: use undici instead of node-fetch in non-Bun environment
2024-06-13 13:38:15 -04:00
Danny Avila
92232afaca
📧 fix: Cancel Signup if Email Issuance Fails (#3010)
* fix: user.id assignment in jwtStrategy.js

* refactor(sendEmail): pass params as object, await email sending to propogate errors and restrict registration flow

* fix(Conversations): handle missing updatedAt field

* refactor: use `processDeleteRequest` when deleting user account for user file deletion

* refactor: delete orphaned files when deleting user account

* fix: remove unnecessary 404 status code in server/index.js
2024-06-08 06:51:29 -04:00
Marco Beretta
ee673d682e
📧 feat: email verification (#2344)
* feat: verification email

* chore: email verification invalid; localize: update

* fix: redirect to login when signup: fix: save emailVerified correctly

* docs: update ALLOW_UNVERIFIED_EMAIL_LOGIN; fix: don't accept login only when ALLOW_UNVERIFIED_EMAIL_LOGIN = true

* fix: user needs to be authenticated

* style: update

* fix: registration success message and redirect logic

* refactor: use `isEnabled` in ALLOW_UNVERIFIED_EMAIL_LOGIN

* refactor: move checkEmailConfig to server/utils

* refactor: use req as param for verifyEmail function

* chore: jsdoc

* chore: remove console log

* refactor: rename `createNewUser` to `createSocialUser`

* refactor: update typing and add expiresAt field to userSchema

* refactor: begin use of user methods over direct model access for User

* refactor: initial email verification rewrite

* chore: typing

* refactor: registration flow rewrite

* chore: remove help center text

* refactor: update getUser to getUserById and add findUser methods. general fixes from recent changes

* refactor: Update updateUser method to remove expiresAt field and use $set and $unset operations, createUser now returns Id only

* refactor: Update openidStrategy to use optional chaining for avatar check, move saveBuffer init to buffer condition

* refactor: logout on deleteUser mutatation

* refactor: Update openidStrategy login success message format

* refactor: Add emailVerified field to Discord and Facebook profile details

* refactor: move limiters to separate middleware dir

* refactor: Add limiters for email verification and password reset

* refactor: Remove getUserController and update routes and controllers accordingly

* refactor: Update getUserById method to exclude password and version fields

* refactor: move verification to user route, add resend verification option

* refactor: Improve email verification process and resend option

* refactor: remove more direct model access of User and remove unused code

* refactor: replace user authentication methods and token generation

* fix: add user.id to jwt user

* refactor: Update AuthContext to include setError function, add resend link to Login Form, make registration redirect shorter

* fix(updateUserPluginsService): ensure userPlugins variable is defined

* refactor: Delete all shared links for a specific user

* fix: remove use of direct User.save() in handleExistingUser

* fix(importLibreChatConvo): handle missing createdAt field in messages

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2024-06-07 15:06:47 -04:00
Marco Beretta
a7f5b57272
🚫👤feat: delete user from UI (#1526)
* initial commit

* fix: UserController bugs; fix: lint errors

* fix: delete files

* language support

* style(DeleteAccount): update to the latest style

* style: fix after merge main

* chore: Add canDeleteAccount middleware for user deletion endpoint

* chore: renamed to ALLOW_ACCOUNT_DELETION

* fix(canDeleteAccount): use uppercase admin role

* chore: imports order

* chore: Enable account deletion by default if omitted/commented out

* chore: Add logging for user account deletion

* chore: Bump data-provider package version to 0.6.6

* chore: Import Transaction model in UserController

* chore: Update CONFIG_VERSION to 1.1.4

* chore: Update user account deletion logging

* chore: Refactor user account deletion logic

---------

Co-authored-by: Berry-13 <root@Berry>
Co-authored-by: Danny Avila <messagedaniel@protonmail.com>
Co-authored-by: Danny Avila <danny@librechat.ai>
2024-06-05 19:35:12 -04:00
Yuichi Oneda
a618266905
🔒 feature(auth): LDAP Authentication (#2859)
* 🔧 chore: npm install passport-ldapauth

*  feat(auth): add ldap authentication support

* chore: merge conflict fix

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2024-05-29 17:46:20 -04:00
Danny Avila
3f600f0d3f
⬇️ fix: JSON LibreChat Imports (#2897)
* chore: remove unused code

* refactor: Update NewChatButtonIcon component to use JSX syntax

The NewChatButtonIcon component in the Nav folder has been updated to use JSX syntax instead of calling the Icon function directly. This change improves code readability and maintainability.

* remove use memo

* refactor: allow passing `select` to messages db query

* fix: initial fix for non-recursive messages

* ci: first pass, importers test rewrite

* fix(groupConversationsByDate): handle edge case of conversation.updatedAt being null

* fix: correctly handle non-recursive uploads

* feat: imports non-recursive conversations with branches correctly

* feat: support retaining original options on import

* refactor: Allow `messageTree` field for Import of non-recursive conversations
2024-05-29 09:15:05 -04:00
Danny Avila
40685f6eb4
🚀 refactor: Enhance Custom Endpoints, Message Logic, and Payload Handling (#2895)
* chore: use node-fetch for OpenAIClient fetch key for non-crashing usage of AbortController in Bun runtime

* chore: variable order

* fix(useSSE): prevent finalHandler call in abortConversation to update messages/conversation after user navigated away

* chore: params order

* refactor: organize intermediate message logic and ensure correct variables are passed

* fix: Add stt and tts routes before upload limiters, prevent bans

* fix(abortRun): temp fix to delete unfinished messages to avoid message thread parent relationship issues

* refactor: Update AnthropicClient to use node-fetch for fetch key and add proxy support

* fix(gptPlugins): ensure parentMessageId/messageId relationship is maintained

* feat(BaseClient): custom fetch function to analyze/edit payloads just before sending (also prevents abortController crash on Bun runtime)

* feat: `directEndpoint` and `titleMessageRole` custom endpoint options

* chore: Bump version to 0.6.6 in data-provider package.json
2024-05-28 14:52:12 -04:00
Danny Avila
5dc5d875ba
🤖 feat: Private Assistants (#2881)
* feat: add configuration for user private assistants

* filter private assistant message requests

* add test for privateAssistants

* add privateAssistants configuration to tests

* fix: destructuring error when assistants config is not added

* chore: revert chat controller changes

* chore: add payload type, add metadata types

* feat: validateAssistant

* refactor(fetchAssistants): allow for flexibility

* feat: validateAuthor

* refactor: return all assistants to ADMIN role

* feat: add assistant doc on assistant creation

* refactor(listAssistants): use `listAllAssistants` to exhaustively fetch all assistants

* chore: add suggestion to tts error

* refactor(validateAuthor): attempt database check first

* refactor: author validation when patching/deleting assistant

---------

Co-authored-by: Leon Juenemann <leon.juenemann@maibornwolff.de>
2024-05-28 08:27:45 -04:00
Danny Avila
1a452121fa
🤖 feat: OpenAI Assistants v2 (initial support) (#2781)
* 🤖 Assistants V2 Support: Part 1

- Separated Azure Assistants to its own endpoint
- File Search / Vector Store integration is incomplete, but can toggle and use storage from playground
- Code Interpreter resource files can be added but not deleted
- GPT-4o is supported
- Many improvements to the Assistants Endpoint overall

data-provider v2 changes

copy existing route as v1

chore: rename new endpoint to reduce comparison operations and add new azure filesource

api: add azureAssistants part 1

force use of version for assistants/assistantsAzure

chore: switch name back to azureAssistants

refactor type version: string | number

Ensure assistants endpoints have version set

fix: isArchived type issue in ConversationListParams

refactor: update assistants mutations/queries with endpoint/version definitions, update Assistants Map structure

chore:  FilePreview component ExtendedFile type assertion

feat: isAssistantsEndpoint helper

chore: remove unused useGenerations

chore(buildTree): type issue

chore(Advanced): type issue (unused component, maybe in future)

first pass for multi-assistant endpoint rewrite

fix(listAssistants): pass params correctly

feat: list separate assistants by endpoint

fix(useTextarea): access assistantMap correctly

fix: assistant endpoint switching, resetting ID

fix: broken during rewrite, selecting assistant mention

fix: set/invalidate assistants endpoint query data correctly

feat: Fix issue with assistant ID not being reset correctly

getOpenAIClient helper function

feat: add toast for assistant deletion

fix: assistants delete right after create issue for azure

fix: assistant patching

refactor: actions to use getOpenAIClient

refactor: consolidate logic into helpers file

fix: issue where conversation data was not initially available

v1 chat support

refactor(spendTokens): only early return if completionTokens isNaN

fix(OpenAIClient): ensure spendTokens has all necessary params

refactor: route/controller logic

fix(assistants/initializeClient): use defaultHeaders field

fix: sanitize default operation id

chore: bump openai package

first pass v2 action service

feat: retroactive domain parsing for actions added via v1

feat: delete db records of actions/assistants on openai assistant deletion

chore: remove vision tools from v2 assistants

feat: v2 upload and delete assistant vision images

WIP first pass, thread attachments

fix: show assistant vision files (save local/firebase copy)

v2 image continue

fix: annotations

fix: refine annotations

show analyze as error if is no longer submitting before progress reaches 1 and show file_search as retrieval tool

fix: abort run, undefined endpoint issue

refactor: consolidate capabilities logic and anticipate versioning

frontend version 2 changes

fix: query selection and filter

add endpoint to unknown filepath

add file ids to resource, deleting in progress

enable/disable file search

remove version log

* 🤖 Assistants V2 Support: Part 2

🎹 fix: Autocompletion Chrome Bug on Action API Key Input

chore: remove `useOriginNavigate`

chore: set correct OpenAI Storage Source

fix: azure file deletions, instantiate clients by source for deletion

update code interpret files info

feat: deleteResourceFileId

chore: increase poll interval as azure easily rate limits

fix: openai file deletions, TODO: evaluate rejected deletion settled promises to determine which to delete from db records

file source icons

update table file filters

chore: file search info and versioning

fix: retrieval update with necessary tool_resources if specified

fix(useMentions): add optional chaining in case listMap value is undefined

fix: force assistant avatar roundedness

fix: azure assistants, check correct flag

chore: bump data-provider

* fix: merge conflict

* ci: fix backend tests due to new updates

* chore: update .env.example

* meilisearch improvements

* localization updates

* chore: update comparisons

* feat: add additional metadata: endpoint, author ID

* chore: azureAssistants ENDPOINTS exclusion warning
2024-05-19 12:56:55 -04:00
Danny Avila
f0e8cca5df
🚀 feat: Shared Links (#2772)
*  feat(types): add necessary types for shared link feature

*  feat: add shared links functions to data service

Added functions for retrieving, creating, updating, and deleting shared links and shared messages.

*  feat: Add useGetSharedMessages hook to fetch shared messages by shareId

Adds a new hook `useGetSharedMessages` which fetches shared messages based on the provided shareId.

*  feat: Add share schema and data access functions to API models

*  feat: Add share endpoint to API

The GET /api/share/${shareId} is exposed to the public, so authentication is not required. Other paths require authentication.

* ♻️ refactor(utils): generalize react-query cache manipulation functions

Introduces generic functions for manipulating react-query cache entries, marking a refinement in how query cache data is managed. It aims to enhance the flexibility and reusability of the cache interaction patterns within our application.

- Replaced specific index names with more generic terms in queries.ts, enhancing consistency across data handling functions.
- Introduced new utility functions in collection.ts for adding, updating, and deleting data entries in an InfiniteData<TCollection>. These utility functions (`addData`, `updateData`, `deleteData`, `findPage`) are designed to be re-usable across different data types and collections.
- Adapted existing conversation utility functions in convos.ts to leverage these new generic utilities.

*  feat(shared-link): add functions to manipulate shared link cache list

implemented new utility functions to handle additions, updates, and deletions in the shared link cache list.

*  feat: Add mutations and queries for shared links

*  feat(shared-link): add `Share` button to conversation list

- Added a share button in each conversation in the conversation list.
- Implemented functionality where clicking the share button triggers a POST request to the API.
- The API checks if a share link was already created for the conversation today; if so, it returns the existing link.
- If no link was created for today, the API will create a new share link and return it.
- Each click on the share button results in a new API request, following the specification similar to ChatGPT's share link feature.

* ♻️ refactor(hooks): generalize useNavScrolling for broader use

- Modified `useNavScrolling` to accept a generic type parameter `TData`, allowing it to be used with different data structures besides `ConversationListResponse`.
- Updated instances in `Nav.tsx` and `ArchivedChatsTable.tsx` to explicitly specify `ConversationListResponse` as the type argument when invoking `useNavScrolling`.

*  feat(settings): add shared links listing table with delete functionality in settings

- Integrated a delete button for each shared link in the table, allowing users to remove links as needed.

* ♻️ refactor(components): separate `EndpointIcon` from `Icon` component for standalone use

* ♻️ refactor: update useGetSharedMessages to return TSharedLink

- Modified the useGetSharedMessages hook to return not only a list of TMessage but also the TSharedLink itself.
- This change was necessary to support displaying the title and date in the Shared Message UI, which requires data from TSharedLink.

*  feat(shared link): add UI for displaying shared conversations without authentication

- Implemented a new UI component to display shared conversations, designed to be accessible without requiring authentication.
- Reused components from the authenticated Messages module where possible. Copied and adapted components that could not be directly reused to fit the non-authenticated context.

* 🔧 chore: Add translations

Translate labels only. Messages remain in English as they are possibly subject to change.

* ♻️ refactor: add icon and tooltip props to EditMenuButton component

* moved icon and popover to arguments so that EditMenuButton can be reused.
* modified so that when a ShareButton is closed, the parent DropdownMenu is also closed.

* ♻️irefactor: added DropdownMenu for Export and Share

* ♻️ refactor: renamed component names more intuitive

* More accurate naming of the dropdown menu.
* When the export button is closed, the parent dropdown menu is also closed.

* 🌍 chore: updated translations

* 🐞 Fix: OpenID Profile Image Download (#2757)

* Add fetch requirement

Fixes - error: [openidStrategy] downloadImage: Error downloading image at URL "https://graph.microsoft.com/v1.0/me/photo/$value": TypeError: response.buffer is not a function

* Update openidStrategy.js

---------

Co-authored-by: Danny Avila <danacordially@gmail.com>

* 🚑 fix(export): Issue exporting Conversation with Assistants (#2769)

* 🚑 fix(export): use content as text if content is present in the message

If the endpoint is assistants, the text of the message goes into content, not message.text.

* refactor(ExportModel): TypeScript, remove unused code

---------

Co-authored-by: Yuichi Ohneda <ohneda@gmail.com>

* 📤style: export button icon (#2752)

* refactor(ShareDialog): logic and styling

* refactor(ExportAndShareMenu): imports order and icon update

* chore: imports

* chore: imports/render logic

* feat: message branching

* refactor: add optional config to useGetStartupConfig

* refactor: disable endpoints query

* chore: fix search view styling gradient in light mode

* style: ShareView gradient styling

* refactor(Share): use select queries

* style: shared link table buttons

* localization and dark text styling

* style: fix clipboard button layout shift app-wide and add localization for copy code

* support assistants message content in shared links, add useCopyToClipboard, add copy buttons to Search Messages and Shared Link Messages

* add localizations

* comparisons

---------

Co-authored-by: Yuichi Ohneda <ohneda@gmail.com>
Co-authored-by: bsu3338 <bsu3338@users.noreply.github.com>
Co-authored-by: Fuegovic <32828263+fuegovic@users.noreply.github.com>
2024-05-17 18:13:32 -04:00
Danny Avila
638ac5bba6
🚀 feat: gpt-4o (#2692)
* 🚀 feat: gpt-4o

* update readme.md

* feat: Add new test case for getMultiplier function

* feat: Refactor getMultiplier function to use valueKey variable
2024-05-13 14:25:02 -04:00
Danny Avila
6ba7f60eec
🪙 feat: Configure Max Context and Output Tokens (#2648)
* chore: make frequent 'error' log into 'debug' log

* feat: add maxContextTokens as a conversation field

* refactor(settings): increase popover height

* feat: add DynamicInputNumber and maxContextTokens to all endpoints that support it (frontend), fix schema

* feat: maxContextTokens handling (backend)

* style: revert popover height

* feat: max tokens

* fix: Ollama Vision firebase compatibility

* fix: Ollama Vision, use message_file_map to determine multimodal request

* refactor: bring back MobileNav and improve title styling
2024-05-09 13:27:13 -04:00
Yuichi Ohneda
89b1e33be0
🚀feat: Archive conversations (#2590)
* 🔧chore: add internationalization labels for archive feature

*  feat: Add function to useArchiveConversationMutation()

This commit adds a new mutation function `useArchiveConversationMutation()` for archiving conversations. This function takes the ID string of the conversation to be archived and returns a mutation result object. Upon successful archiving, it removes and refreshes the conversation from the query data cache.
While ChatGPT PATCHes the archived status by sending `{is_archived: true}` to the URL `/backend-api/conversation/$conversation_id`, this implementation uses the `dataService.updateConversation(payload)` with a POST method, aligning with the existing code conventions.

*  feat(api): add is_archived field to Conversation schema and update getConvosByPage method

This commit adds a new field `is_archived` with a default value of false to the Conversation schema. It also modifies the `getConvosByPage` method within the Conversation API to adjust the query to only target conversations where `is_archived` is set to false or where the `is_archived` field does not exist. The function `getConvosQueried`, which returns conversations for a specified Conversation ID, was determined not to require consideration of whether `is_archived` is true or false, and thus was not modified.

* ♻️ refactor: add className prop to DotsIcon component

To enhance the versatility of the DotsIcon component, this commit introduces the ability to specify a className prop, allowing for greater customization.

*  feat(ui): add Edit Button to group Title change and Conversation delete buttons

Added a new Edit Button to the conversations, similar to the ChatGPT UI, which groups options for editing the conversation title and deleting conversations. This grouping is accessible through a dialogue that appears when the three-dot icon is clicked.

* ♻️ refactor(ui): enhance Delete Button to accept className and label options

Enhanced the Delete Button component to accept a `className` for customization and an optional `appendLabel`. The DeleteButton component is used by both `Convo.tsx` and `Conversation.tsx`, but currently only `Convo.tsx` is active and `Conversation.tsx `is apparently not used; removing `Conversation.tsx` may eliminate the need for the `appendLabel` property in the future.

* ♻️ refactor(ui): enhance RenameButton to accept label options

Added the ability to optionally display labels; the Rename Button component is used by both `Convo.tsx` and `Conversation.tsx`, but currently only `Convo.tsx` is active and `Conversation.tsx `is apparently not used; removing `Conversation.tsx` may eliminate the need for the `appendLabel` property in the future.

* 🔧 chors: additional localization labels

* ♻️  refactor: change is_archived property of conversation to camelCase

* Refactor the is_archived property of conversation to camelCase (isArchived) to adhere to the existing code conventions
* Modify the function that retrieves conversations to accept the isArchived parameter

* ♻️ refactor: add archiveConversation mutation

I thought I could divert dataService.updateConversation, but added a new archiveConversation because the request types are different. It might be better to make them common, but to avoid side effects, I added a new function this time.
Added process to deleteConversationMutation to delete archived conversations

*  feat: Add the function to hide a cancel button in DialogTemplate component

The Cancel button is not needed when displaying the archive list, so I made the Cancel button optional.

* ♻️ refactor: Add support for filtering archived conversations in Nav component

This commit modifies the Nav component to add the ability to filter out archived conversations when fetching data. This is done by adding `isArchived: false` to the query parameters for both the `useConversationsInfiniteQuery()` and `useSearchInfiniteQuery()` hooks, effectively excluding any archived conversations from the results returned.

* ♻️ refactor: add Tooltip to DeleteButton

* Add Tooltip to DeleteButton component
* Display Tooltip when DeleteButton only shows an Icon without text

*  feat(ui): add ArchiveButton component for archiving conversations

To be compatible with the ChatGPT UI, no confirmation dialog is displayed when ArchiveButton is clicked. The basic behavior conforms to DeleteButton and RenameButton.

*  feat(ui): add Archive button to list of conversations

Modify the Nav of the conversation list to include a dropdown that contains the Rename and Delete options, similar to the ChatGPT UI. Additionally, an Archive button has been added adjacent to the dropdown menu.

*  feat: Add ArchivedChatsTable component

Adds the `ArchivedChatsTable` component, which displays a table of archived chats. It has been implemented to be as compatible with the ChatGPT UI as possible.

* 🚑 fix(tooltip): increase z-index to ensure visibility over Dialog

Resolve an issue where tooltips were not visible when displayed over a Dialog. The z-index of `DialogPrimitive.Portal` in `Dialog.tsx` is set to 999. Since the rationale for this value is unclear, the z-index of the tooltip has been increased to 1000 to guarantee its visibility above the Dialog component.

* 🔧 chors: add internationalization labels
2024-05-06 23:07:00 -04:00
Danny Avila
25fceb78b7
🌿 feat: Fork Messages/Conversations (#2617)
* typedef for ImportBatchBuilder

* feat: first pass, fork conversations

* feat: fork - getMessagesUpToTargetLevel

* fix: additional tests and fix getAllMessagesUpToParent

* chore: arrow function return

* refactor: fork 3 options

* chore: remove unused genbuttons

* chore: remove unused hover buttons code

* feat: fork first pass

* wip: fork remember setting

* style: user icon

* chore: move clear chats to data tab

* WIP: fork UI options

* feat: data-provider fork types/services/vars and use generic MutationOptions

* refactor: use single param for fork option, use enum, fix mongo errors, use Date.now(), add records flag for testing, use endpoint from original convo and messages, pass originalConvo to finishConversation

* feat: add fork mutation hook and consolidate type imports

* refactor: use enum

* feat: first pass, fork mutation

* chore: add enum for target level fork option

* chore: add enum for target level fork option

* show toast when checking remember selection

* feat: splitAtTarget

* feat: split at target option

* feat: navigate to new fork, show toasts, set result query data

* feat: hover info for all fork options

* refactor: add Messages settings tab

* fix(Fork): remember text info

* ci: test for single message and is target edge case

* feat: additional tests for getAllMessagesUpToParent

* ci: additional tests and cycle detection for getMessagesUpToTargetLevel

* feat: circular dependency checks for getAllMessagesUpToParent

* fix: getMessagesUpToTargetLevel circular dep. check

* ci: more tests for getMessagesForConversation

* style: hover text for checkbox fork items

* refactor: add statefulness to conversation import
2024-05-05 11:48:20 -04:00
Denis Palnitsky
ab6fbe48f1
📥 feat: Import Conversations from LibreChat, ChatGPT, Chatbot UI (#2355)
* Basic implementation of ChatGPT conversation import

* remove debug code

* Handle citations

* Fix updatedAt in import

* update default model

* Use job scheduler to handle import requests

* import job status endpoint

* Add wrapper around Agenda

* Rate limits for import endpoint

* rename import api path

* Batch save import to mongo

* Improve naming

* Add documenting comments

* Test for importers

* Change button for importing conversations

* Frontend changes

* Import job status endpoint

* Import endpoint response

* Add translations to new phrases

* Fix conversations refreshing

* cleanup unused functions

* set timeout for import job status polling

* Add documentation

* get extra spaces back

* Improve error message

* Fix translation files after merge

* fix translation files 2

* Add zh translation for import functionality

* Sync mailisearch index after import

* chore: add dummy uri for jest tests, as MONGO_URI should only be real for E2E tests

* docs: fix links

* docs: fix conversationsImport section

* fix: user role issue for librechat imports

* refactor: import conversations from json
- organize imports
- add additional jsdocs
- use multer with diskStorage to avoid loading file into memory outside of job
- use filepath instead of loading data string for imports
- replace console logs and some logger.info() with logger.debug
- only use multer for import route

* fix: undefined metadata edge case and replace ChatGtp -> ChatGpt

* Refactor importChatGptConvo function to handle undefined metadata edge case and replace ChatGtp with ChatGpt

* fix: chatgpt importer

* feat: maintain tree relationship for librechat messages

* chore: use enum

* refactor: saveMessage to use single object arg, replace console logs, add userId to log message

* chore: additional comment

* chore: multer edge case

* feat: first pass, maintain tree relationship

* chore: organize

* chore: remove log

* ci: add heirarchy test for chatgpt

* ci: test maintaining of heirarchy for librechat

* wip: allow non-text content type messages

* refactor: import content part object json string

* refactor: more content types to format

* chore: consolidate messageText formatting

* docs: update on changes, bump data-provider/config versions, update readme

* refactor(indexSync): singleton pattern for MeiliSearchClient

* refactor: debug log after batch is done

* chore: add back indexSync error handling

---------

Co-authored-by: jakubmieszczak <jakub.mieszczak@zendesk.com>
Co-authored-by: Danny Avila <danny@librechat.ai>
2024-05-02 02:48:26 -04:00
Danny Avila
0e50c07e3f
🤖 feat: Model Specs & Save Tools per Convo/Preset (#2578)
* WIP: first pass ModelSpecs

* refactor(onSelectEndpoint): use `getConvoSwitchLogic`

* feat: introduce iconURL, greeting, frontend fields for conversations/presets/messages

* feat: conversation.iconURL & greeting in Landing

* feat: conversation.iconURL & greeting in New Chat button

* feat: message.iconURL

* refactor: ConversationIcon -> ConvoIconURL

* WIP: add spec as a conversation field

* refactor: useAppStartup, set spec on initial load for new chat, allow undefined spec, add localStorage keys enum, additional type fields for spec

* feat: handle `showIconInMenu`, `showIconInHeader`, undefined `iconURL` and no specs on initial load

* chore: handle undefined or empty modelSpecs

* WIP: first pass, modelSpec schema for custom config

* refactor: move default filtered tools definition to ToolService

* feat: pass modelSpecs from backend via startupConfig

* refactor: modelSpecs config, return and define list

* fix: react error and include iconURL in responseMessage

* refactor: add iconURL to responseMessage only

* refactor: getIconEndpoint

* refactor: pass TSpecsConfig

* fix(assistants): differentiate compactAssistantSchema, correctly resets shared conversation state with other endpoints

* refactor: assistant id prefix localStorage key

* refactor: add more LocalStorageKeys and replace hardcoded values

* feat: prioritize spec on new chat behavior: last selected modelSpec behavior (localStorage)

* feat: first pass, interface config

* chore: WIP, todo: add warnings based on config.modelSpecs settings.

* feat: enforce modelSpecs if configured

* feat: show config file yaml errors

* chore: delete unused legacy Plugins component

* refactor: set tools to localStorage from recoil store

* chore: add stable recoil setter to useEffect deps

* refactor: save tools to conversation documents

* style(MultiSelectPop): dynamic height, remove unused import

* refactor(react-query): use localstorage keys and pass config to useAvailablePluginsQuery

* feat(utils): add mapPlugins

* refactor(Convo): use conversation.tools if defined, lastSelectedTools if not

* refactor: remove unused legacy code using `useSetOptions`, remove conditional flag `isMultiChat` for using legacy settings

* refactor(PluginStoreDialog): add exhaustive-deps which are stable react state setters

* fix(HeaderOptions): pass `popover` as true

* refactor(useSetStorage): use project enums

* refactor: use LocalStorageKeys enum

* fix: prevent setConversation from setting falsy values in lastSelectedTools

* refactor: use map for availableTools state and available Plugins query

* refactor(updateLastSelectedModel): organize logic better and add note on purpose

* fix(setAgentOption): prevent reseting last model to secondary model for gptPlugins

* refactor(buildDefaultConvo): use enum

* refactor: remove `useSetStorage` and consolidate areas where conversation state is saved to localStorage

* fix: conversations retain tools on refresh

* fix(gptPlugins): prevent nullish tools from being saved

* chore: delete useServerStream

* refactor: move initial plugins logic to useAppStartup

* refactor(MultiSelectDropDown): add more pass-in className props

* feat: use tools in presets

* chore: delete unused usePresetOptions

* refactor: new agentOptions default handling

* chore: note

* feat: add label and custom instructions to agents

* chore: remove 'disabled with tools' message

* style: move plugins to 2nd column in parameters

* fix: TPreset type for agentOptions

* fix: interface controls

* refactor: add interfaceConfig, use Separator within Switcher

* refactor: hide Assistants panel if interface.parameters are disabled

* fix(Header): only modelSpecs if list is greater than 0

* refactor: separate MessageIcon logic from useMessageHelpers for better react rule-following

* fix(AppService): don't use reserved keyword 'interface'

* feat: set existing Icon for custom endpoints through iconURL

* fix(ci): tests passing for App Service

* docs: refactor custom_config.md for readability and better organization, also include missing values

* docs: interface section and re-organize docs

* docs: update modelSpecs info

* chore: remove unused files

* chore: remove unused files

* chore: move useSetIndexOptions

* chore: remove unused file

* chore: move useConversation(s)

* chore: move useDefaultConvo

* chore: move useNavigateToConvo

* refactor: use plugin install hook so it can be used elsewhere

* chore: import order

* update docs

* refactor(OpenAI/Plugins): allow modelLabel as an initial value for chatGptLabel

* chore: remove unused EndpointOptionsPopover and hide 'Save as Preset' button if preset UI visibility disabled

* feat(loadDefaultInterface): issue warnings based on values

* feat: changelog for custom config file

* docs: add additional changelog note

* fix: prevent unavailable tool selection from preset and update availableTools on Plugin installations

* feat: add `filteredTools` option in custom config

* chore: changelog

* fix(MessageIcon): always overwrite conversation.iconURL in messageSettings

* fix(ModelSpecsMenu): icon edge cases

* fix(NewChat): dynamic icon

* fix(PluginsClient): always include endpoint in responseMessage

* fix: always include endpoint and iconURL in responseMessage across different response methods

* feat: interchangeable keys for modelSpec enforcing
2024-04-30 22:11:48 -04:00
Danny Avila
099aa9dead
feat: Stop Sequences for Conversations & Presets (#2536)
* feat: `stop` conversation parameter

* feat: Tag primitive

* feat: dynamic tags

* refactor: update tag styling

* feat: add stop sequences to OpenAI settings

* fix(Presentation): prevent `SidePanel` re-renders that flicker side panel

* refactor: use stop placeholder

* feat: type and schema update for `stop` and `TPreset` in generation param related types

* refactor: pass conversation to dynamic settings

* refactor(OpenAIClient): remove default handling for `modelOptions.stop`

* docs: fix Google AI Setup formatting

* feat: current_model

* docs: WIP update

* fix(ChatRoute): prevent default preset override before `hasSetConversation.current` becomes true by including latest conversation state as template

* docs: update docs with more info on `stop`

* chore: bump config_version

* refactor: CURRENT_MODEL handling
2024-04-25 11:40:17 -04:00
Danny Avila
3df4fac118
v0.7.1 (#2502)
* chore: make openai package definition explicit

*  v0.7.1

* chore: gpt-4-vision correct context length

* add `llava` to vision models list
2024-04-23 08:57:20 -04:00
Danny Avila
8c22bb1d3d
🛠️ fix(Azure/Assistants): Handle Long Domain Names & Other Minor chores (#2475)
* chore: replace violation cache accessors with enum

* chore: fix test

* chore(fileSchema): index timestamps

* fix(ActionService): use encoding/caching strategy for handling assistant function character length limit

* refactor(actions): async `domainParser` also resolve retrieved model (which is deployment name) to user-defined model

* style(AssistantAction): add `whitespace-nowrap` for ellipsis

* refactor(ActionService): if domain is less than or equal to encoded domain fixed length, return domain with replacement of separator

* refactor(actions): use sessions/transactions for updating Assistant Action database records

* chore: remove TTL from ENCODED_DOMAINS cache

* refactor(domainParser): minor optimization and add tests

* fix(spendTokens): use txData.user for token usage logging

* refactor(actions): add helper function `withSession` for database operations with sessions/transactions

* fix(PluginsClient): logger debug `message` field edge case
2024-04-20 15:02:56 -04:00
Danny Avila
9d854dac07
🤖 feat: Gemini 1.5 Support (+Vertex AI) (#2383)
* WIP: gemini-1.5 support

* feat: extended vertex ai support

* fix: handle possibly undefined modelName

* fix: gpt-4-turbo-preview invalid vision model

* feat: specify `fileConfig.imageOutputType` and make PNG default image conversion type

* feat: better truncation for errors including base64 strings

* fix: gemini inlineData formatting

* feat: RAG augmented prompt for gemini-1.5

* feat: gemini-1.5 rates and token window

* chore: adjust tokens, update docs, update vision Models

* chore: add back `ChatGoogleVertexAI` for chat models via vertex ai

* refactor: ask/edit controllers to not use `unfinished` field for google endpoint

* chore: remove comment

* chore(ci): fix AppService test

* chore: remove comment

* refactor(GoogleSearch): use `GOOGLE_SEARCH_API_KEY` instead, issue warning for old variable

* chore: bump data-provider to 0.5.4

* chore: update docs

* fix: condition for gemini-1.5 using generative ai lib

* chore: update docs

* ci: add additional AppService test for `imageOutputType`

* refactor: optimize new config value `imageOutputType`

* chore: bump CONFIG_VERSION

* fix(assistants): avatar upload
2024-04-16 08:32:40 -04:00