feat: Vision Support + New UI (#1203)
* feat: add timer duration to showToast, show toast for preset selection
* refactor: replace old /chat/ route with /c/. e2e tests will fail here
* refactor: move typedefs to root of /api/ and add a few to assistant types in TS
* refactor: reorganize data-provider imports, fix dependency cycle, strategize new plan to separate react dependent packages
* feat: add dataService for uploading images
* feat(data-provider): add mutation keys
* feat: file resizing and upload
* WIP: initial API image handling
* fix: catch JSON.parse of localStorage tools
* chore: experimental: use module-alias for absolute imports
* refactor: change temp_file_id strategy
* fix: updating files state by using Map and defining react query callbacks in a way that keeps them during component unmount, initial delete handling
* feat: properly handle file deletion
* refactor: unexpose complete filepath and resize from server for higher fidelity
* fix: make sure resized height, width is saved, catch bad requests
* refactor: use absolute imports
* fix: prevent setOptions from being called more than once for OpenAIClient, made note to fix for PluginsClient
* refactor: import supportsFiles and models vars from schemas
* fix: correctly replace temp file id
* refactor(BaseClient): use absolute imports, pass message 'opts' to buildMessages method, count tokens for nested objects/arrays
* feat: add validateVisionModel to determine if model has vision capabilities
* chore(checkBalance): update jsdoc
* feat: formatVisionMessage: change message content format dependent on role and image_urls passed
* refactor: add usage to File schema, make create and updateFile, correctly set and remove TTL
* feat: working vision support
TODO: file size, type, amount validations, making sure they are styled right, and making sure you can add images from the clipboard/dragging
* feat: clipboard support for uploading images
* feat: handle files on drop to screen, refactor top level view code to Presentation component so the useDragHelpers hook has ChatContext
* fix(Images): replace uploaded images in place
* feat: add filepath validation to protect sensitive files
* fix: ensure correct file_ids are push and not the Map key values
* fix(ToastContext): type issue
* feat: add basic file validation
* fix(useDragHelpers): correct context issue with `files` dependency
* refactor: consolidate setErrors logic to setError
* feat: add dialog Image overlay on image click
* fix: close endpoints menu on click
* chore: set detail to auto, make note for configuration
* fix: react warning (button desc. of button)
* refactor: optimize filepath handling, pass file_ids to images for easier re-use
* refactor: optimize image file handling, allow re-using files in regen, pass more file metadata in messages
* feat: lazy loading images including use of upload preview
* fix: SetKeyDialog closing, stopPropagation on Dialog content click
* style(EndpointMenuItem): tighten up the style, fix dark theme showing in lightmode, make menu more ux friendly
* style: change maxheight of all settings textareas to 138px from 300px
* style: better styling for textarea and enclosing buttons
* refactor(PresetItems): swap back edit and delete icons
* feat: make textarea placeholder dynamic to endpoint
* style: show user hover buttons only on hover when message is streaming
* fix: ordered list not going past 9, fix css
* feat: add User/AI labels; style: hide loading spinner
* feat: add back custom footer, change original footer text
* feat: dynamic landing icons based on endpoint
* chore: comment out assistants route
* fix: autoScroll to newest on /c/ view
* fix: Export Conversation on new UI
* style: match message style of official more closely
* ci: fix api jest unit tests, comment out e2e tests for now as they will fail until addressed
* feat: more file validation and use blob in preview field, not filepath, to fix temp deletion
* feat: filefilter for multer
* feat: better AI labels based on custom name, model, and endpoint instead of `ChatGPT`
2023-11-21 20:12:48 -05:00
|
|
|
const fs = require('fs');
|
|
|
|
const path = require('path');
|
|
|
|
const crypto = require('crypto');
|
|
|
|
const multer = require('multer');
|
🧠 feat: User Memories for Conversational Context (#7760)
* 🧠 feat: User Memories for Conversational Context
chore: mcp typing, use `t`
WIP: first pass, Memories UI
- Added MemoryViewer component for displaying, editing, and deleting user memories.
- Integrated data provider hooks for fetching, updating, and deleting memories.
- Implemented pagination and loading states for better user experience.
- Created unit tests for MemoryViewer to ensure functionality and interaction with data provider.
- Updated translation files to include new UI strings related to memories.
chore: move mcp-related files to own directory
chore: rename librechat-mcp to librechat-api
WIP: first pass, memory processing and data schemas
chore: linting in fileSearch.js query description
chore: rename librechat-api to @librechat/api across the project
WIP: first pass, functional memory agent
feat: add MemoryEditDialog and MemoryViewer components for managing user memories
- Introduced MemoryEditDialog for editing memory entries with validation and toast notifications.
- Updated MemoryViewer to support editing and deleting memories, including pagination and loading states.
- Enhanced data provider to handle memory updates with optional original key for better management.
- Added new localization strings for memory-related UI elements.
feat: add memory permissions management
- Implemented memory permissions in the backend, allowing roles to have specific permissions for using, creating, updating, and reading memories.
- Added new API endpoints for updating memory permissions associated with roles.
- Created a new AdminSettings component for managing memory permissions in the frontend.
- Integrated memory permissions into the existing roles and permissions schemas.
- Updated the interface to include memory settings and permissions.
- Enhanced the MemoryViewer component to conditionally render admin settings based on user roles.
- Added localization support for memory permissions in the translation files.
feat: move AdminSettings component to a new position in MemoryViewer for better visibility
refactor: clean up commented code in MemoryViewer component
feat: enhance MemoryViewer with search functionality and improve MemoryEditDialog integration
- Added a search input to filter memories in the MemoryViewer component.
- Refactored MemoryEditDialog to accept children for better customization.
- Updated MemoryViewer to utilize the new EditMemoryButton and DeleteMemoryButton components for editing and deleting memories.
- Improved localization support by adding new strings for memory filtering and deletion confirmation.
refactor: optimize memory filtering in MemoryViewer using match-sorter
- Replaced manual filtering logic with match-sorter for improved search functionality.
- Enhanced performance and readability of the filteredMemories computation.
feat: enhance MemoryEditDialog with triggerRef and improve updateMemory mutation handling
feat: implement access control for MemoryEditDialog and MemoryViewer components
refactor: remove commented out code and create runMemory method
refactor: rename role based files
feat: implement access control for memory usage in AgentClient
refactor: simplify checkVisionRequest method in AgentClient by removing commented-out code
refactor: make `agents` dir in api package
refactor: migrate Azure utilities to TypeScript and consolidate imports
refactor: move sanitizeFilename function to a new file and update imports, add related tests
refactor: update LLM configuration types and consolidate Azure options in the API package
chore: linting
chore: import order
refactor: replace getLLMConfig with getOpenAIConfig and remove unused LLM configuration file
chore: update winston-daily-rotate-file to version 5.0.0 and add object-hash dependency in package-lock.json
refactor: move primeResources and optionalChainWithEmptyCheck functions to resources.ts and update imports
refactor: move createRun function to a new run.ts file and update related imports
fix: ensure safeAttachments is correctly typed as an array of TFile
chore: add node-fetch dependency and refactor fetch-related functions into packages/api/utils, removing the old generators file
refactor: enhance TEndpointOption type by using Pick to streamline endpoint fields and add new properties for model parameters and client options
feat: implement initializeOpenAIOptions function and update OpenAI types for enhanced configuration handling
fix: update types due to new TEndpointOption typing
fix: ensure safe access to group parameters in initializeOpenAIOptions function
fix: remove redundant API key validation comment in initializeOpenAIOptions function
refactor: rename initializeOpenAIOptions to initializeOpenAI for consistency and update related documentation
refactor: decouple req.body fields and tool loading from initializeAgentOptions
chore: linting
refactor: adjust column widths in MemoryViewer for improved layout
refactor: simplify agent initialization by creating loadAgent function and removing unused code
feat: add memory configuration loading and validation functions
WIP: first pass, memory processing with config
feat: implement memory callback and artifact handling
feat: implement memory artifacts display and processing updates
feat: add memory configuration options and schema validation for validKeys
fix: update MemoryEditDialog and MemoryViewer to handle memory state and display improvements
refactor: remove padding from BookmarkTable and MemoryViewer headers for consistent styling
WIP: initial tokenLimit config and move Tokenizer to @librechat/api
refactor: update mongoMeili plugin methods to use callback for better error handling
feat: enhance memory management with token tracking and usage metrics
- Added token counting for memory entries to enforce limits and provide usage statistics.
- Updated memory retrieval and update routes to include total token usage and limit.
- Enhanced MemoryEditDialog and MemoryViewer components to display memory usage and token information.
- Refactored memory processing functions to handle token limits and provide feedback on memory capacity.
feat: implement memory artifact handling in attachment handler
- Enhanced useAttachmentHandler to process memory artifacts when receiving updates.
- Introduced handleMemoryArtifact utility to manage memory updates and deletions.
- Updated query client to reflect changes in memory state based on incoming data.
refactor: restructure web search key extraction logic
- Moved the logic for extracting API keys from the webSearchAuth configuration into a dedicated function, getWebSearchKeys.
- Updated webSearchKeys to utilize the new function for improved clarity and maintainability.
- Prevents build time errors
feat: add personalization settings and memory preferences management
- Introduced a new Personalization tab in settings to manage user memory preferences.
- Implemented API endpoints and client-side logic for updating memory preferences.
- Enhanced user interface components to reflect personalization options and memory usage.
- Updated permissions to allow users to opt out of memory features.
- Added localization support for new settings and messages related to personalization.
style: personalization switch class
feat: add PersonalizationIcon and align Side Panel UI
feat: implement memory creation functionality
- Added a new API endpoint for creating memory entries, including validation for key and value.
- Introduced MemoryCreateDialog component for user interface to facilitate memory creation.
- Integrated token limit checks to prevent exceeding user memory capacity.
- Updated MemoryViewer to include a button for opening the memory creation dialog.
- Enhanced localization support for new messages related to memory creation.
feat: enhance message processing with configurable window size
- Updated AgentClient to use a configurable message window size for processing messages.
- Introduced messageWindowSize option in memory configuration schema with a default value of 5.
- Improved logic for selecting messages to process based on the configured window size.
chore: update librechat-data-provider version to 0.7.87 in package.json and package-lock.json
chore: remove OpenAPIPlugin and its associated tests
chore: remove MIGRATION_README.md as migration tasks are completed
ci: fix backend tests
chore: remove unused translation keys from localization file
chore: remove problematic test file and unused var in AgentClient
chore: remove unused import and import directly for JSDoc
* feat: add api package build stage in Dockerfile for improved modularity
* docs: reorder build steps in contributing guide for clarity
2025-06-07 18:52:22 -04:00
|
|
|
const { sanitizeFilename } = require('@librechat/api');
|
✨ feat: Assistants API, General File Support, Side Panel, File Explorer (#1696)
* feat: assistant name/icon in Landing & Header
* feat: assistname in textarea placeholder, and use `Assistant` as default name
* feat: display non-image files in user messages
* fix: only render files if files.length is > 0
* refactor(config -> file-config): move file related configuration values to separate module, add excel types
* chore: spreadsheet file rendering
* fix(Landing): dark mode style for Assistant Name
* refactor: move progress incrementing to own hook, start smaller, cap near limit \(1\)
* refactor(useContentHandler): add empty Text part if last part was completed tool or image
* chore: add accordion trigger border styling for dark mode
* feat: Assistant Builder model selection
* chore: use Spinner when Assistant is mutating
* fix(get/assistants): return correct response object `AssistantListResponse`
* refactor(Spinner): pass size as prop
* refactor: make assistant crud mutations optimistic, add types for options
* chore: remove assistants route and view
* chore: move assistant builder components to separate directory
* feat(ContextButton): delete Assistant via context button/dialog, add localization
* refactor: conditionally show use and context menu buttons, add localization for create assistant
* feat: save side panel states to localStorage
* style(SidePanel): improve avatar menu and assistant select styling for dark mode
* refactor: make NavToggle reusable for either side (left or right), add SidePanel Toggle with ability to close it completely
* fix: resize handle and navToggle behavior
* fix(/avatar/:assistant_id): await `deleteFile` and assign unique name to uploaded image
* WIP: file UI components from PR #576
* refactor(OpenAIMinimalIcon): pass className
* feat: formatDate helper fn
* feat: DataTableColumnHeader
* feat: add row selection, formatted row values, number of rows selected
* WIP: add files to Side panel temporarily
* feat: `LB_QueueAsyncCall`: Leaky Bucket queue for external APIs, use in `processDeleteRequest`
* fix(TFile): correct `source` type with `FileSources`
* fix(useFileHandling): use `continue` instead of return when iterating multiple files, add file type to extendedFile
* chore: add generic setter type
* refactor(processDeleteRequest): settle promises to prevent rejections from processing deletions, log errors
* feat: `useFileDeletion` to reuse file deletion logic
* refactor(useFileDeletion): make `setFiles` an optional param and use object as param
* feat: useDeleteFilesFromTable
* feat: use real `files` data and add deletion action to data table
* fix(Table): make headers sticky
* feat: add dynamic filtering for columns; only show to user Host or OpenAI storage type
* style(DropdownMenu): replace `slate` with `gray`
* style(DataTable): apply dark mode themes and other misc styling
* style(Columns): add color to OpenAI Storage option
* refactor(FileContainer): make file preview reusable
* refactor(Images): make image preview reusable
* refactor(FilePreview): make file prop optional for FileIcon and FilePreview, fix relative style
* feat(Columns): add file/image previews, set a minimum size to show for file size in bytes
* WIP: File Panel with real files and formatted
* feat: open files dialog from panel
* style: file data table mobile and general column styling fixes
* refactor(api/files): return files sorted by the most recently updated
* refactor: provide fileMap through context to prevent re-selecting files to map in different areas; remove unused imports commented out in PanelColumns
* refactor(ExtendFile): make File type optional, add `attached` to prevent attached files from being deleted on remove, make Message.files a partial TFile type
* feat: attach files through file panel
* refactor(useFileHandling): move files to the start of cache list when uploaded
* refactor(useDeleteFilesMutation): delete files from cache when successfully deleted from server
* fix(FileRow): handle possible edge case of duplication due to attaching recently uploaded file
* style(SidePanel): make resize grip border transparent, remove unnecessary styling on close sidepanel button
* feat: action utilities and tests
* refactor(actions): add `ValidationResult` type and change wording for no server URL found
* refactor(actions): check for empty server URL
* fix(data-provider): revert tsconfig to fix type issue resolution
* feat(client): first pass of actions input for assistants
* refactor(FunctionSignature): change method to output object instead of string
* refactor(models/Assistant): add actions field to schema, use searchParams object for methods, and add `getAssistant`
* feat: post actions input first pass
- create new Action document
- add actions to Assistant DB document
- create /action/:assistant_id POST route
- pass more props down from PanelSwitcher, derive assistant_id from switcher
- move privacy policy to ActionInput
- reset data on input change/validation
- add `useUpdateAction`
- conform FunctionSignature type to FunctionTool
- add action, assistant doc, update hook related types
* refactor: optimize assistant/actions relationship
- past domain in metadata as hostname and not a URL
- include domain in tool name
- add `getActions` for actions retrieval by user
- add `getAssistants` for assistant docs retrieval by user
- add `assistant_id` to Action schema
- move actions to own module as a subroute to `api/assistants`
- add `useGetActionsQuery` and `useGetAssistantDocsQuery` hooks
- fix Action type def
* feat: show assistant actions in assistant builder
* feat: switch to actions on action click, editing action styling
* fix: add Assistant state for builder panel to allow immediate selection of newly created assistants as well as retaining the current assistant when switching to a different panel within the builder
* refactor(SidePanel/NavToggle): offset less from right when SidePanel is completely collapsed
* chore: rename `processActions` -> `processRequiredActions`
* chore: rename Assistant API Action to RequiredAction
* refactor(actions): avoid nesting actual API params under generic `requestBody` to optimize LLM token usage
* fix(handleTools): avoid calling `validTool` if not defined, add optional param to skip the loading of specs, which throws an error in the context of assistants
* WIP: working first pass of toolCalls generated from openapi specs
* WIP: first pass ToolCall styling
* feat: programmatic iv encryption/decryption helpers
* fix: correct ActionAuth types/enums, and define type for AuthForm
* feat: encryption/decryption helpers for Action AuthMetadata
* refactor(getActions): remove sensitive fields from query response
* refactor(POST/actions): encrypt and remove sensitive fields from mutation response
* fix(ActionService): change ESM import to CJS
* feat: frontend auth handling for actions + optimistic update on action update/creation
* refactor(actions): use the correct variables and types for setAuth method
* refactor: POST /:assistant_id action can now handle updating an existing action, add `saved_auth_fields` to determine when user explicitly saves new auth creds. only send auth metadata if user explicitly saved fields
* refactor(createActionTool): catch errors and send back meaningful error message, add flag to `getActions` to determine whether to retrieve sensitive values or not
* refactor(ToolService): add `action` property to ToolCall PartMetadata to determine if the tool call was an action, fix parsing function name issue with actionDelimiter
* fix(ActionRequest): use URL class to correctly join endpoint parts for `execute` call
* feat: delete assistant actions
* refactor: conditionally show Available actions
* refactor: show `retrieval` and `code_interpreter` as Capabilities, swap `Switch` for `Checkbox`
* chore: remove shadow-stroke from messages
* WIP: first pass of Assistants Knowledge attachments
* refactor: remove AssistantsProvider in favor of FormProvider, fix selectedAssistant re-render bug, map Assistant file_ids to files via fileMap, initialize Knowledge component with mapped files if any exist
* fix: prevent deleting files on assistant file upload
* chore: remove console.log
* refactor(useUploadFileMutation): update files and assistants cache on upload
* chore: disable oauth option as not supported yet
* feat: cancel assistant runs
* refactor: initialize OpenAI client with helper function, resolve all related circular dependencies
* fix(DALL-E): initialization
* fix(process): openai client initialization
* fix: select an existing Assistant when the active one is deleted
* chore: allow attaching files for assistant endpoint, send back relevant OpenAI error message when uploading, deconstruct openAI initialization correctly, add `message_file` to formData when a file is attached to the message but not the assistant
* fix: add assistant_id on newConvo
* fix(initializeClient): import fix
* chore: swap setAssistant for setOption in useEffect
* fix(DALL-E): add processFileURL to loadTools call
* chore: add customConfig to debug logs
* feat: delete threads on convo delete
* chore: replace Assistants icon
* chore: remove console.dir() in `abortRun`
* feat(AssistantService): accumulate text values from run in openai.responseText
* feat: titling for assistants endpoint
* chore: move panel file components to appropriate directory, add file checks for attaching files, change icon for Attach Files
* refactor: add localizations to tools, plugins, add condition for adding/remove user plugins so tool selections don't affect this value
* chore: disable `import from url` action for now
* chore: remove textMimeTypes from default fileConfig for now
* fix: catch tool errors and send as outputs with error messages
* fix: React warning about button as descendant of button
* style: retrieval and cancelled icon
* WIP: pass isSubmitting to Parts, use InProgressCall to display cancelled tool calls correctly, show domain/function name
* fix(meilisearch): fix `postSaveHook` issue where indexing expects a mongo document, and join all text content parts for meili indexing
* ci: fix dall-e tests
* ci: fix client tests
* fix: button types in actions panel
* fix: plugin auth form persisting across tool selections
* fix(ci): update AppService spec with `loadAndFormatTools`
* fix(clearConvos): add id check earlier on
* refactor(AssistantAvatar): set previewURL dynamically when emtadata.avatar changes
* feat(assistants): addTitle cache setting
* fix(useSSE): resolve rebase conflicts
* fix: delete mutation
* style(SidePanel): make grip visible on active and hover, invisible otherwise
* ci: add data-provider tests to workflow, also update eslint/tsconfig to recognize specs, and add `text/csv` to fileConfig
* fix: handle edge case where auth object is undefined, and log errors
* refactor(actions): resolve schemas, add tests for resolving refs, import specs from separate file for tests
* chore: remove comment
* fix(ActionsInput): re-render bug when initializing states with action fields
* fix(patch/assistant): filter undefined tools
* chore: add logging for errors in assistants routes
* fix(updateAssistant): map actions to functions to avoid overwriting
* fix(actions): properly handle GET paths
* fix(convos): unhandled delete thread exception
* refactor(AssistantService): pass both thread_id and conversationId when sending intermediate assistant messages, remove `mapMessagesToSteps` from AssistantService
* refactor(useSSE): replace all messages with runMessages and pass latestMessageId to abortRun; fix(checkMessageGaps): include tool calls when syncing messages
* refactor(assistants/chat): invoke `createOnTextProgress` after thread creation
* chore: add typing
* style: sidepanel styling
* style: action tool call domain styling
* feat(assistants): default models, limit retrieval to certain models, add env variables to to env.example
* feat: assistants api key in EndpointService
* refactor: set assistant model to conversation on assistant switch
* refactor: set assistant model to conversation on assistant select from panel
* fix(retrieveAndProcessFile): catch attempt to download file with `assistant` purpose which is not allowed; add logging
* feat: retrieval styling, handling, and logging
* chore: rename ASSISTANTS_REVERSE_PROXY to ASSISTANTS_BASE_URL
* feat: FileContext for file metadata
* feat: context file mgmt and filtering
* style(Select): hover/rounded changes
* refactor: explicit conversation switch, endpoint dependent, through `useSelectAssistant`, which does not create new chat if current endpoint is assistant endpoint
* fix(AssistantAvatar): make empty previewURL if no avatar present
* refactor: side panel mobile styling
* style: merge tool and action section, optimize mobile styling for action/tool buttons
* fix: localStorage issues
* fix(useSelectAssistant): invoke react query hook directly in select hook as Map was not being updated in time
* style: light mode fixes
* fix: prevent sidepanel nav styling from shifting layout up
* refactor: change default layout (collapsed by default)
* style: mobile optimization of DataTable
* style: datatable
* feat: client-side hide right-side panel
* chore(useNewConvo): add partial typing for preset
* fix(useSelectAssistant): pass correct model name by using template as preset
* WIP: assistant presets
* refactor(ToolService): add native solution for `TavilySearchResults` and log tool output errors
* refactor: organize imports and use native TavilySearchResults
* fix(TavilySearchResults): stringify result
* fix(ToolCall): show tool call outputs when not an action
* chore: rename Prompt Prefix to custom instructions (in user facing text only)
* refactor(EditPresetDialog): Optimize setting title by debouncing, reset preset on dialog close to avoid state mixture
* feat: add `presetOverride` to overwrite active conversation settings when saving a Preset (relevant for client side updates only)
* feat: Assistant preset settings (client-side)
* fix(Switcher): only set assistant_id and model if current endpoint is Assistants
* feat: use `useDebouncedInput` for updating conversation settings, starting with EditPresetDialog title setting and Assistant instructions setting
* feat(Assistants): add instructions field to settings
* feat(chat/assistants): pass conversation settings to run body
* wip: begin localization and only allow actions if the assistant is created
* refactor(AssistantsPanel): knowledge localization, allow tools on creation
* feat: experimental: allow 'priming' values before assistant is created, that would normally require an assistant_id to be defined
* chore: trim console logs and make more meaningful
* chore: toast messages
* fix(ci): date test
* feat: create file when uploading Assistant Avatar
* feat: file upload rate limiting from custom config with dynamic file route initialization
* refactor: use file upload limiters on post routes only
* refactor(fileConfig): add endpoints field for endpoint specific fileconfigs, add mergeConfig function, add tests
* refactor: fileConfig route, dynamic multer instances used on all '/' and '/images' POST routes, data service and query hook
* feat: supportedMimeTypesSchema, test for array of regex
* feat: configurable file config limits
* chore: clarify assistants file knowledge prereq.
* chore(useTextarea): default to localized 'Assistant' if assistant name is empty
* feat: configurable file limits and toggle file upload per endpoint
* fix(useUploadFileMutation): prevent updating assistant.files cache if file upload is a message_file attachment
* fix(AssistantSelect): set last selected assistant only when timeout successfully runs
* refactor(queries): disable assistant queries if assistants endpoint is not enabled
* chore(Switcher): add localization
* chore: pluralize `assistant` for `EModelEndpoint key and value
* feat: show/hide assistant UI components based on endpoint availability; librechat.yaml config for disabling builder section and setting polling/timeout intervals
* fix(compactEndpointSchemas): use EModelEndpoint for schema access
* feat(runAssistant): use configured values from `librechat.yaml` for `pollIntervalMs` and `timeout`
* fix: naming issue
* wip: revert landing
* 🎉 happy birthday LibreChat (#1768)
* happy birthday LibreChat
* Refactor endpoint condition in Landing component
* Update birthday message in Eng.tsx
* fix(/config): avoid nesting ternaries
* refactor(/config): check birthday
---------
Co-authored-by: Danny Avila <messagedaniel@protonmail.com>
* fix: landing
* fix: landing
* fix(useMessageHelpers): hardcoded check to use EModelEndpoint instead
* fix(ci): convo test revert to main
* fix(assistants/chat): fix issue where assistant_id was being saved as model for convo
* chore: added logging, promises racing to prevent longer timeouts, explicit setting of maxRetries and timeouts, robust catching of invalid abortRun params
* refactor: use recoil state for `showStopButton` and only show for assistants endpoint after syncing conversation data
* refactor: optimize abortRun strategy using localStorage, refactor `abortConversation` to use async/await and await the result, refactor how the abortKey cache is set for runs
* fix(checkMessageGaps): assign `assistant_id` to synced messages if defined; prevents UI from showing blank assistant for cancelled messages
* refactor: re-order sequence of chat route, only allow aborting messages after run is created, cancel abortRun if there was a cancelling error (likely due already cancelled in chat route), and add extra logging
* chore(typedefs): add httpAgent type to OpenAIClient
* refactor: use custom implementation of retrieving run with axios to allow for timing out run query
* fix(waitForRun): handle timed out run retrieval query
* refactor: update preset conditions:
- presets will retain settings when a different endpoint is selected; for existing convos, either when modular or is assistant switch
- no longer use `navigateToConvo` on preset select
* fix: temporary calculator hack as expects string input when invoked
* fix: cancel abortRun only when cancelling error is a result of the run already being cancelled
* chore: remove use of `fileMaxSizeMB` and total counterpart (redundant)
* docs: custom config documentation update
* docs: assistants api setup and dotenv, new custom config fields
* refactor(Switcher): make Assistant switcher sticky in SidePanel
* chore(useSSE): remove console log of data and message index
* refactor(AssistantPanel): button styling and add secondary select button to bottom of panel
* refactor(OpenAIClient): allow passing conversationId to RunManager through titleConvo and initializeLLM to properly record title context tokens used in cases where conversationId was not defined by the client
* feat(assistants): token tracking for assistant runs
* chore(spendTokens): improve logging
* feat: support/exclude specific assistant Ids
* chore: add update `librechat.example.yaml`, optimize `AppService` handling, new tests for `AppService`, optimize missing/outdate config logging
* chore: mount docker logs to root of project
* chore: condense axios errors
* chore: bump vite
* chore: vite hot reload fix using latest version
* chore(getOpenAIModels): sort instruct models to the end of models list
* fix(assistants): user provided key
* fix(assistants): user provided key, invalidate more queries on revoke
---------
Co-authored-by: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
2024-02-13 20:42:27 -05:00
|
|
|
const { fileConfig: defaultFileConfig, mergeFileConfig } = require('librechat-data-provider');
|
2024-11-04 12:59:04 -05:00
|
|
|
const { getCustomConfig } = require('~/server/services/Config');
|
feat: Vision Support + New UI (#1203)
* feat: add timer duration to showToast, show toast for preset selection
* refactor: replace old /chat/ route with /c/. e2e tests will fail here
* refactor: move typedefs to root of /api/ and add a few to assistant types in TS
* refactor: reorganize data-provider imports, fix dependency cycle, strategize new plan to separate react dependent packages
* feat: add dataService for uploading images
* feat(data-provider): add mutation keys
* feat: file resizing and upload
* WIP: initial API image handling
* fix: catch JSON.parse of localStorage tools
* chore: experimental: use module-alias for absolute imports
* refactor: change temp_file_id strategy
* fix: updating files state by using Map and defining react query callbacks in a way that keeps them during component unmount, initial delete handling
* feat: properly handle file deletion
* refactor: unexpose complete filepath and resize from server for higher fidelity
* fix: make sure resized height, width is saved, catch bad requests
* refactor: use absolute imports
* fix: prevent setOptions from being called more than once for OpenAIClient, made note to fix for PluginsClient
* refactor: import supportsFiles and models vars from schemas
* fix: correctly replace temp file id
* refactor(BaseClient): use absolute imports, pass message 'opts' to buildMessages method, count tokens for nested objects/arrays
* feat: add validateVisionModel to determine if model has vision capabilities
* chore(checkBalance): update jsdoc
* feat: formatVisionMessage: change message content format dependent on role and image_urls passed
* refactor: add usage to File schema, make create and updateFile, correctly set and remove TTL
* feat: working vision support
TODO: file size, type, amount validations, making sure they are styled right, and making sure you can add images from the clipboard/dragging
* feat: clipboard support for uploading images
* feat: handle files on drop to screen, refactor top level view code to Presentation component so the useDragHelpers hook has ChatContext
* fix(Images): replace uploaded images in place
* feat: add filepath validation to protect sensitive files
* fix: ensure correct file_ids are push and not the Map key values
* fix(ToastContext): type issue
* feat: add basic file validation
* fix(useDragHelpers): correct context issue with `files` dependency
* refactor: consolidate setErrors logic to setError
* feat: add dialog Image overlay on image click
* fix: close endpoints menu on click
* chore: set detail to auto, make note for configuration
* fix: react warning (button desc. of button)
* refactor: optimize filepath handling, pass file_ids to images for easier re-use
* refactor: optimize image file handling, allow re-using files in regen, pass more file metadata in messages
* feat: lazy loading images including use of upload preview
* fix: SetKeyDialog closing, stopPropagation on Dialog content click
* style(EndpointMenuItem): tighten up the style, fix dark theme showing in lightmode, make menu more ux friendly
* style: change maxheight of all settings textareas to 138px from 300px
* style: better styling for textarea and enclosing buttons
* refactor(PresetItems): swap back edit and delete icons
* feat: make textarea placeholder dynamic to endpoint
* style: show user hover buttons only on hover when message is streaming
* fix: ordered list not going past 9, fix css
* feat: add User/AI labels; style: hide loading spinner
* feat: add back custom footer, change original footer text
* feat: dynamic landing icons based on endpoint
* chore: comment out assistants route
* fix: autoScroll to newest on /c/ view
* fix: Export Conversation on new UI
* style: match message style of official more closely
* ci: fix api jest unit tests, comment out e2e tests for now as they will fail until addressed
* feat: more file validation and use blob in preview field, not filepath, to fix temp deletion
* feat: filefilter for multer
* feat: better AI labels based on custom name, model, and endpoint instead of `ChatGPT`
2023-11-21 20:12:48 -05:00
|
|
|
|
|
|
|
const storage = multer.diskStorage({
|
|
|
|
destination: function (req, file, cb) {
|
✨ feat: Assistants API, General File Support, Side Panel, File Explorer (#1696)
* feat: assistant name/icon in Landing & Header
* feat: assistname in textarea placeholder, and use `Assistant` as default name
* feat: display non-image files in user messages
* fix: only render files if files.length is > 0
* refactor(config -> file-config): move file related configuration values to separate module, add excel types
* chore: spreadsheet file rendering
* fix(Landing): dark mode style for Assistant Name
* refactor: move progress incrementing to own hook, start smaller, cap near limit \(1\)
* refactor(useContentHandler): add empty Text part if last part was completed tool or image
* chore: add accordion trigger border styling for dark mode
* feat: Assistant Builder model selection
* chore: use Spinner when Assistant is mutating
* fix(get/assistants): return correct response object `AssistantListResponse`
* refactor(Spinner): pass size as prop
* refactor: make assistant crud mutations optimistic, add types for options
* chore: remove assistants route and view
* chore: move assistant builder components to separate directory
* feat(ContextButton): delete Assistant via context button/dialog, add localization
* refactor: conditionally show use and context menu buttons, add localization for create assistant
* feat: save side panel states to localStorage
* style(SidePanel): improve avatar menu and assistant select styling for dark mode
* refactor: make NavToggle reusable for either side (left or right), add SidePanel Toggle with ability to close it completely
* fix: resize handle and navToggle behavior
* fix(/avatar/:assistant_id): await `deleteFile` and assign unique name to uploaded image
* WIP: file UI components from PR #576
* refactor(OpenAIMinimalIcon): pass className
* feat: formatDate helper fn
* feat: DataTableColumnHeader
* feat: add row selection, formatted row values, number of rows selected
* WIP: add files to Side panel temporarily
* feat: `LB_QueueAsyncCall`: Leaky Bucket queue for external APIs, use in `processDeleteRequest`
* fix(TFile): correct `source` type with `FileSources`
* fix(useFileHandling): use `continue` instead of return when iterating multiple files, add file type to extendedFile
* chore: add generic setter type
* refactor(processDeleteRequest): settle promises to prevent rejections from processing deletions, log errors
* feat: `useFileDeletion` to reuse file deletion logic
* refactor(useFileDeletion): make `setFiles` an optional param and use object as param
* feat: useDeleteFilesFromTable
* feat: use real `files` data and add deletion action to data table
* fix(Table): make headers sticky
* feat: add dynamic filtering for columns; only show to user Host or OpenAI storage type
* style(DropdownMenu): replace `slate` with `gray`
* style(DataTable): apply dark mode themes and other misc styling
* style(Columns): add color to OpenAI Storage option
* refactor(FileContainer): make file preview reusable
* refactor(Images): make image preview reusable
* refactor(FilePreview): make file prop optional for FileIcon and FilePreview, fix relative style
* feat(Columns): add file/image previews, set a minimum size to show for file size in bytes
* WIP: File Panel with real files and formatted
* feat: open files dialog from panel
* style: file data table mobile and general column styling fixes
* refactor(api/files): return files sorted by the most recently updated
* refactor: provide fileMap through context to prevent re-selecting files to map in different areas; remove unused imports commented out in PanelColumns
* refactor(ExtendFile): make File type optional, add `attached` to prevent attached files from being deleted on remove, make Message.files a partial TFile type
* feat: attach files through file panel
* refactor(useFileHandling): move files to the start of cache list when uploaded
* refactor(useDeleteFilesMutation): delete files from cache when successfully deleted from server
* fix(FileRow): handle possible edge case of duplication due to attaching recently uploaded file
* style(SidePanel): make resize grip border transparent, remove unnecessary styling on close sidepanel button
* feat: action utilities and tests
* refactor(actions): add `ValidationResult` type and change wording for no server URL found
* refactor(actions): check for empty server URL
* fix(data-provider): revert tsconfig to fix type issue resolution
* feat(client): first pass of actions input for assistants
* refactor(FunctionSignature): change method to output object instead of string
* refactor(models/Assistant): add actions field to schema, use searchParams object for methods, and add `getAssistant`
* feat: post actions input first pass
- create new Action document
- add actions to Assistant DB document
- create /action/:assistant_id POST route
- pass more props down from PanelSwitcher, derive assistant_id from switcher
- move privacy policy to ActionInput
- reset data on input change/validation
- add `useUpdateAction`
- conform FunctionSignature type to FunctionTool
- add action, assistant doc, update hook related types
* refactor: optimize assistant/actions relationship
- past domain in metadata as hostname and not a URL
- include domain in tool name
- add `getActions` for actions retrieval by user
- add `getAssistants` for assistant docs retrieval by user
- add `assistant_id` to Action schema
- move actions to own module as a subroute to `api/assistants`
- add `useGetActionsQuery` and `useGetAssistantDocsQuery` hooks
- fix Action type def
* feat: show assistant actions in assistant builder
* feat: switch to actions on action click, editing action styling
* fix: add Assistant state for builder panel to allow immediate selection of newly created assistants as well as retaining the current assistant when switching to a different panel within the builder
* refactor(SidePanel/NavToggle): offset less from right when SidePanel is completely collapsed
* chore: rename `processActions` -> `processRequiredActions`
* chore: rename Assistant API Action to RequiredAction
* refactor(actions): avoid nesting actual API params under generic `requestBody` to optimize LLM token usage
* fix(handleTools): avoid calling `validTool` if not defined, add optional param to skip the loading of specs, which throws an error in the context of assistants
* WIP: working first pass of toolCalls generated from openapi specs
* WIP: first pass ToolCall styling
* feat: programmatic iv encryption/decryption helpers
* fix: correct ActionAuth types/enums, and define type for AuthForm
* feat: encryption/decryption helpers for Action AuthMetadata
* refactor(getActions): remove sensitive fields from query response
* refactor(POST/actions): encrypt and remove sensitive fields from mutation response
* fix(ActionService): change ESM import to CJS
* feat: frontend auth handling for actions + optimistic update on action update/creation
* refactor(actions): use the correct variables and types for setAuth method
* refactor: POST /:assistant_id action can now handle updating an existing action, add `saved_auth_fields` to determine when user explicitly saves new auth creds. only send auth metadata if user explicitly saved fields
* refactor(createActionTool): catch errors and send back meaningful error message, add flag to `getActions` to determine whether to retrieve sensitive values or not
* refactor(ToolService): add `action` property to ToolCall PartMetadata to determine if the tool call was an action, fix parsing function name issue with actionDelimiter
* fix(ActionRequest): use URL class to correctly join endpoint parts for `execute` call
* feat: delete assistant actions
* refactor: conditionally show Available actions
* refactor: show `retrieval` and `code_interpreter` as Capabilities, swap `Switch` for `Checkbox`
* chore: remove shadow-stroke from messages
* WIP: first pass of Assistants Knowledge attachments
* refactor: remove AssistantsProvider in favor of FormProvider, fix selectedAssistant re-render bug, map Assistant file_ids to files via fileMap, initialize Knowledge component with mapped files if any exist
* fix: prevent deleting files on assistant file upload
* chore: remove console.log
* refactor(useUploadFileMutation): update files and assistants cache on upload
* chore: disable oauth option as not supported yet
* feat: cancel assistant runs
* refactor: initialize OpenAI client with helper function, resolve all related circular dependencies
* fix(DALL-E): initialization
* fix(process): openai client initialization
* fix: select an existing Assistant when the active one is deleted
* chore: allow attaching files for assistant endpoint, send back relevant OpenAI error message when uploading, deconstruct openAI initialization correctly, add `message_file` to formData when a file is attached to the message but not the assistant
* fix: add assistant_id on newConvo
* fix(initializeClient): import fix
* chore: swap setAssistant for setOption in useEffect
* fix(DALL-E): add processFileURL to loadTools call
* chore: add customConfig to debug logs
* feat: delete threads on convo delete
* chore: replace Assistants icon
* chore: remove console.dir() in `abortRun`
* feat(AssistantService): accumulate text values from run in openai.responseText
* feat: titling for assistants endpoint
* chore: move panel file components to appropriate directory, add file checks for attaching files, change icon for Attach Files
* refactor: add localizations to tools, plugins, add condition for adding/remove user plugins so tool selections don't affect this value
* chore: disable `import from url` action for now
* chore: remove textMimeTypes from default fileConfig for now
* fix: catch tool errors and send as outputs with error messages
* fix: React warning about button as descendant of button
* style: retrieval and cancelled icon
* WIP: pass isSubmitting to Parts, use InProgressCall to display cancelled tool calls correctly, show domain/function name
* fix(meilisearch): fix `postSaveHook` issue where indexing expects a mongo document, and join all text content parts for meili indexing
* ci: fix dall-e tests
* ci: fix client tests
* fix: button types in actions panel
* fix: plugin auth form persisting across tool selections
* fix(ci): update AppService spec with `loadAndFormatTools`
* fix(clearConvos): add id check earlier on
* refactor(AssistantAvatar): set previewURL dynamically when emtadata.avatar changes
* feat(assistants): addTitle cache setting
* fix(useSSE): resolve rebase conflicts
* fix: delete mutation
* style(SidePanel): make grip visible on active and hover, invisible otherwise
* ci: add data-provider tests to workflow, also update eslint/tsconfig to recognize specs, and add `text/csv` to fileConfig
* fix: handle edge case where auth object is undefined, and log errors
* refactor(actions): resolve schemas, add tests for resolving refs, import specs from separate file for tests
* chore: remove comment
* fix(ActionsInput): re-render bug when initializing states with action fields
* fix(patch/assistant): filter undefined tools
* chore: add logging for errors in assistants routes
* fix(updateAssistant): map actions to functions to avoid overwriting
* fix(actions): properly handle GET paths
* fix(convos): unhandled delete thread exception
* refactor(AssistantService): pass both thread_id and conversationId when sending intermediate assistant messages, remove `mapMessagesToSteps` from AssistantService
* refactor(useSSE): replace all messages with runMessages and pass latestMessageId to abortRun; fix(checkMessageGaps): include tool calls when syncing messages
* refactor(assistants/chat): invoke `createOnTextProgress` after thread creation
* chore: add typing
* style: sidepanel styling
* style: action tool call domain styling
* feat(assistants): default models, limit retrieval to certain models, add env variables to to env.example
* feat: assistants api key in EndpointService
* refactor: set assistant model to conversation on assistant switch
* refactor: set assistant model to conversation on assistant select from panel
* fix(retrieveAndProcessFile): catch attempt to download file with `assistant` purpose which is not allowed; add logging
* feat: retrieval styling, handling, and logging
* chore: rename ASSISTANTS_REVERSE_PROXY to ASSISTANTS_BASE_URL
* feat: FileContext for file metadata
* feat: context file mgmt and filtering
* style(Select): hover/rounded changes
* refactor: explicit conversation switch, endpoint dependent, through `useSelectAssistant`, which does not create new chat if current endpoint is assistant endpoint
* fix(AssistantAvatar): make empty previewURL if no avatar present
* refactor: side panel mobile styling
* style: merge tool and action section, optimize mobile styling for action/tool buttons
* fix: localStorage issues
* fix(useSelectAssistant): invoke react query hook directly in select hook as Map was not being updated in time
* style: light mode fixes
* fix: prevent sidepanel nav styling from shifting layout up
* refactor: change default layout (collapsed by default)
* style: mobile optimization of DataTable
* style: datatable
* feat: client-side hide right-side panel
* chore(useNewConvo): add partial typing for preset
* fix(useSelectAssistant): pass correct model name by using template as preset
* WIP: assistant presets
* refactor(ToolService): add native solution for `TavilySearchResults` and log tool output errors
* refactor: organize imports and use native TavilySearchResults
* fix(TavilySearchResults): stringify result
* fix(ToolCall): show tool call outputs when not an action
* chore: rename Prompt Prefix to custom instructions (in user facing text only)
* refactor(EditPresetDialog): Optimize setting title by debouncing, reset preset on dialog close to avoid state mixture
* feat: add `presetOverride` to overwrite active conversation settings when saving a Preset (relevant for client side updates only)
* feat: Assistant preset settings (client-side)
* fix(Switcher): only set assistant_id and model if current endpoint is Assistants
* feat: use `useDebouncedInput` for updating conversation settings, starting with EditPresetDialog title setting and Assistant instructions setting
* feat(Assistants): add instructions field to settings
* feat(chat/assistants): pass conversation settings to run body
* wip: begin localization and only allow actions if the assistant is created
* refactor(AssistantsPanel): knowledge localization, allow tools on creation
* feat: experimental: allow 'priming' values before assistant is created, that would normally require an assistant_id to be defined
* chore: trim console logs and make more meaningful
* chore: toast messages
* fix(ci): date test
* feat: create file when uploading Assistant Avatar
* feat: file upload rate limiting from custom config with dynamic file route initialization
* refactor: use file upload limiters on post routes only
* refactor(fileConfig): add endpoints field for endpoint specific fileconfigs, add mergeConfig function, add tests
* refactor: fileConfig route, dynamic multer instances used on all '/' and '/images' POST routes, data service and query hook
* feat: supportedMimeTypesSchema, test for array of regex
* feat: configurable file config limits
* chore: clarify assistants file knowledge prereq.
* chore(useTextarea): default to localized 'Assistant' if assistant name is empty
* feat: configurable file limits and toggle file upload per endpoint
* fix(useUploadFileMutation): prevent updating assistant.files cache if file upload is a message_file attachment
* fix(AssistantSelect): set last selected assistant only when timeout successfully runs
* refactor(queries): disable assistant queries if assistants endpoint is not enabled
* chore(Switcher): add localization
* chore: pluralize `assistant` for `EModelEndpoint key and value
* feat: show/hide assistant UI components based on endpoint availability; librechat.yaml config for disabling builder section and setting polling/timeout intervals
* fix(compactEndpointSchemas): use EModelEndpoint for schema access
* feat(runAssistant): use configured values from `librechat.yaml` for `pollIntervalMs` and `timeout`
* fix: naming issue
* wip: revert landing
* 🎉 happy birthday LibreChat (#1768)
* happy birthday LibreChat
* Refactor endpoint condition in Landing component
* Update birthday message in Eng.tsx
* fix(/config): avoid nesting ternaries
* refactor(/config): check birthday
---------
Co-authored-by: Danny Avila <messagedaniel@protonmail.com>
* fix: landing
* fix: landing
* fix(useMessageHelpers): hardcoded check to use EModelEndpoint instead
* fix(ci): convo test revert to main
* fix(assistants/chat): fix issue where assistant_id was being saved as model for convo
* chore: added logging, promises racing to prevent longer timeouts, explicit setting of maxRetries and timeouts, robust catching of invalid abortRun params
* refactor: use recoil state for `showStopButton` and only show for assistants endpoint after syncing conversation data
* refactor: optimize abortRun strategy using localStorage, refactor `abortConversation` to use async/await and await the result, refactor how the abortKey cache is set for runs
* fix(checkMessageGaps): assign `assistant_id` to synced messages if defined; prevents UI from showing blank assistant for cancelled messages
* refactor: re-order sequence of chat route, only allow aborting messages after run is created, cancel abortRun if there was a cancelling error (likely due already cancelled in chat route), and add extra logging
* chore(typedefs): add httpAgent type to OpenAIClient
* refactor: use custom implementation of retrieving run with axios to allow for timing out run query
* fix(waitForRun): handle timed out run retrieval query
* refactor: update preset conditions:
- presets will retain settings when a different endpoint is selected; for existing convos, either when modular or is assistant switch
- no longer use `navigateToConvo` on preset select
* fix: temporary calculator hack as expects string input when invoked
* fix: cancel abortRun only when cancelling error is a result of the run already being cancelled
* chore: remove use of `fileMaxSizeMB` and total counterpart (redundant)
* docs: custom config documentation update
* docs: assistants api setup and dotenv, new custom config fields
* refactor(Switcher): make Assistant switcher sticky in SidePanel
* chore(useSSE): remove console log of data and message index
* refactor(AssistantPanel): button styling and add secondary select button to bottom of panel
* refactor(OpenAIClient): allow passing conversationId to RunManager through titleConvo and initializeLLM to properly record title context tokens used in cases where conversationId was not defined by the client
* feat(assistants): token tracking for assistant runs
* chore(spendTokens): improve logging
* feat: support/exclude specific assistant Ids
* chore: add update `librechat.example.yaml`, optimize `AppService` handling, new tests for `AppService`, optimize missing/outdate config logging
* chore: mount docker logs to root of project
* chore: condense axios errors
* chore: bump vite
* chore: vite hot reload fix using latest version
* chore(getOpenAIModels): sort instruct models to the end of models list
* fix(assistants): user provided key
* fix(assistants): user provided key, invalidate more queries on revoke
---------
Co-authored-by: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
2024-02-13 20:42:27 -05:00
|
|
|
const outputPath = path.join(req.app.locals.paths.uploads, 'temp', req.user.id);
|
feat: Vision Support + New UI (#1203)
* feat: add timer duration to showToast, show toast for preset selection
* refactor: replace old /chat/ route with /c/. e2e tests will fail here
* refactor: move typedefs to root of /api/ and add a few to assistant types in TS
* refactor: reorganize data-provider imports, fix dependency cycle, strategize new plan to separate react dependent packages
* feat: add dataService for uploading images
* feat(data-provider): add mutation keys
* feat: file resizing and upload
* WIP: initial API image handling
* fix: catch JSON.parse of localStorage tools
* chore: experimental: use module-alias for absolute imports
* refactor: change temp_file_id strategy
* fix: updating files state by using Map and defining react query callbacks in a way that keeps them during component unmount, initial delete handling
* feat: properly handle file deletion
* refactor: unexpose complete filepath and resize from server for higher fidelity
* fix: make sure resized height, width is saved, catch bad requests
* refactor: use absolute imports
* fix: prevent setOptions from being called more than once for OpenAIClient, made note to fix for PluginsClient
* refactor: import supportsFiles and models vars from schemas
* fix: correctly replace temp file id
* refactor(BaseClient): use absolute imports, pass message 'opts' to buildMessages method, count tokens for nested objects/arrays
* feat: add validateVisionModel to determine if model has vision capabilities
* chore(checkBalance): update jsdoc
* feat: formatVisionMessage: change message content format dependent on role and image_urls passed
* refactor: add usage to File schema, make create and updateFile, correctly set and remove TTL
* feat: working vision support
TODO: file size, type, amount validations, making sure they are styled right, and making sure you can add images from the clipboard/dragging
* feat: clipboard support for uploading images
* feat: handle files on drop to screen, refactor top level view code to Presentation component so the useDragHelpers hook has ChatContext
* fix(Images): replace uploaded images in place
* feat: add filepath validation to protect sensitive files
* fix: ensure correct file_ids are push and not the Map key values
* fix(ToastContext): type issue
* feat: add basic file validation
* fix(useDragHelpers): correct context issue with `files` dependency
* refactor: consolidate setErrors logic to setError
* feat: add dialog Image overlay on image click
* fix: close endpoints menu on click
* chore: set detail to auto, make note for configuration
* fix: react warning (button desc. of button)
* refactor: optimize filepath handling, pass file_ids to images for easier re-use
* refactor: optimize image file handling, allow re-using files in regen, pass more file metadata in messages
* feat: lazy loading images including use of upload preview
* fix: SetKeyDialog closing, stopPropagation on Dialog content click
* style(EndpointMenuItem): tighten up the style, fix dark theme showing in lightmode, make menu more ux friendly
* style: change maxheight of all settings textareas to 138px from 300px
* style: better styling for textarea and enclosing buttons
* refactor(PresetItems): swap back edit and delete icons
* feat: make textarea placeholder dynamic to endpoint
* style: show user hover buttons only on hover when message is streaming
* fix: ordered list not going past 9, fix css
* feat: add User/AI labels; style: hide loading spinner
* feat: add back custom footer, change original footer text
* feat: dynamic landing icons based on endpoint
* chore: comment out assistants route
* fix: autoScroll to newest on /c/ view
* fix: Export Conversation on new UI
* style: match message style of official more closely
* ci: fix api jest unit tests, comment out e2e tests for now as they will fail until addressed
* feat: more file validation and use blob in preview field, not filepath, to fix temp deletion
* feat: filefilter for multer
* feat: better AI labels based on custom name, model, and endpoint instead of `ChatGPT`
2023-11-21 20:12:48 -05:00
|
|
|
if (!fs.existsSync(outputPath)) {
|
|
|
|
fs.mkdirSync(outputPath, { recursive: true });
|
|
|
|
}
|
|
|
|
cb(null, outputPath);
|
|
|
|
},
|
|
|
|
filename: function (req, file, cb) {
|
|
|
|
req.file_id = crypto.randomUUID();
|
2024-04-01 20:25:36 +08:00
|
|
|
file.originalname = decodeURIComponent(file.originalname);
|
2024-11-12 16:41:04 -05:00
|
|
|
const sanitizedFilename = sanitizeFilename(file.originalname);
|
|
|
|
cb(null, sanitizedFilename);
|
feat: Vision Support + New UI (#1203)
* feat: add timer duration to showToast, show toast for preset selection
* refactor: replace old /chat/ route with /c/. e2e tests will fail here
* refactor: move typedefs to root of /api/ and add a few to assistant types in TS
* refactor: reorganize data-provider imports, fix dependency cycle, strategize new plan to separate react dependent packages
* feat: add dataService for uploading images
* feat(data-provider): add mutation keys
* feat: file resizing and upload
* WIP: initial API image handling
* fix: catch JSON.parse of localStorage tools
* chore: experimental: use module-alias for absolute imports
* refactor: change temp_file_id strategy
* fix: updating files state by using Map and defining react query callbacks in a way that keeps them during component unmount, initial delete handling
* feat: properly handle file deletion
* refactor: unexpose complete filepath and resize from server for higher fidelity
* fix: make sure resized height, width is saved, catch bad requests
* refactor: use absolute imports
* fix: prevent setOptions from being called more than once for OpenAIClient, made note to fix for PluginsClient
* refactor: import supportsFiles and models vars from schemas
* fix: correctly replace temp file id
* refactor(BaseClient): use absolute imports, pass message 'opts' to buildMessages method, count tokens for nested objects/arrays
* feat: add validateVisionModel to determine if model has vision capabilities
* chore(checkBalance): update jsdoc
* feat: formatVisionMessage: change message content format dependent on role and image_urls passed
* refactor: add usage to File schema, make create and updateFile, correctly set and remove TTL
* feat: working vision support
TODO: file size, type, amount validations, making sure they are styled right, and making sure you can add images from the clipboard/dragging
* feat: clipboard support for uploading images
* feat: handle files on drop to screen, refactor top level view code to Presentation component so the useDragHelpers hook has ChatContext
* fix(Images): replace uploaded images in place
* feat: add filepath validation to protect sensitive files
* fix: ensure correct file_ids are push and not the Map key values
* fix(ToastContext): type issue
* feat: add basic file validation
* fix(useDragHelpers): correct context issue with `files` dependency
* refactor: consolidate setErrors logic to setError
* feat: add dialog Image overlay on image click
* fix: close endpoints menu on click
* chore: set detail to auto, make note for configuration
* fix: react warning (button desc. of button)
* refactor: optimize filepath handling, pass file_ids to images for easier re-use
* refactor: optimize image file handling, allow re-using files in regen, pass more file metadata in messages
* feat: lazy loading images including use of upload preview
* fix: SetKeyDialog closing, stopPropagation on Dialog content click
* style(EndpointMenuItem): tighten up the style, fix dark theme showing in lightmode, make menu more ux friendly
* style: change maxheight of all settings textareas to 138px from 300px
* style: better styling for textarea and enclosing buttons
* refactor(PresetItems): swap back edit and delete icons
* feat: make textarea placeholder dynamic to endpoint
* style: show user hover buttons only on hover when message is streaming
* fix: ordered list not going past 9, fix css
* feat: add User/AI labels; style: hide loading spinner
* feat: add back custom footer, change original footer text
* feat: dynamic landing icons based on endpoint
* chore: comment out assistants route
* fix: autoScroll to newest on /c/ view
* fix: Export Conversation on new UI
* style: match message style of official more closely
* ci: fix api jest unit tests, comment out e2e tests for now as they will fail until addressed
* feat: more file validation and use blob in preview field, not filepath, to fix temp deletion
* feat: filefilter for multer
* feat: better AI labels based on custom name, model, and endpoint instead of `ChatGPT`
2023-11-21 20:12:48 -05:00
|
|
|
},
|
|
|
|
});
|
|
|
|
|
2024-05-02 08:48:26 +02:00
|
|
|
const importFileFilter = (req, file, cb) => {
|
|
|
|
if (file.mimetype === 'application/json') {
|
|
|
|
cb(null, true);
|
|
|
|
} else if (path.extname(file.originalname).toLowerCase() === '.json') {
|
|
|
|
cb(null, true);
|
|
|
|
} else {
|
|
|
|
cb(new Error('Only JSON files are allowed'), false);
|
|
|
|
}
|
|
|
|
};
|
|
|
|
|
2024-11-07 11:11:20 -05:00
|
|
|
/**
|
|
|
|
*
|
|
|
|
* @param {import('librechat-data-provider').FileConfig | undefined} customFileConfig
|
|
|
|
*/
|
|
|
|
const createFileFilter = (customFileConfig) => {
|
|
|
|
/**
|
|
|
|
* @param {ServerRequest} req
|
|
|
|
* @param {Express.Multer.File}
|
|
|
|
* @param {import('multer').FileFilterCallback} cb
|
|
|
|
*/
|
|
|
|
const fileFilter = (req, file, cb) => {
|
|
|
|
if (!file) {
|
|
|
|
return cb(new Error('No file provided'), false);
|
|
|
|
}
|
✨ feat: Assistants API, General File Support, Side Panel, File Explorer (#1696)
* feat: assistant name/icon in Landing & Header
* feat: assistname in textarea placeholder, and use `Assistant` as default name
* feat: display non-image files in user messages
* fix: only render files if files.length is > 0
* refactor(config -> file-config): move file related configuration values to separate module, add excel types
* chore: spreadsheet file rendering
* fix(Landing): dark mode style for Assistant Name
* refactor: move progress incrementing to own hook, start smaller, cap near limit \(1\)
* refactor(useContentHandler): add empty Text part if last part was completed tool or image
* chore: add accordion trigger border styling for dark mode
* feat: Assistant Builder model selection
* chore: use Spinner when Assistant is mutating
* fix(get/assistants): return correct response object `AssistantListResponse`
* refactor(Spinner): pass size as prop
* refactor: make assistant crud mutations optimistic, add types for options
* chore: remove assistants route and view
* chore: move assistant builder components to separate directory
* feat(ContextButton): delete Assistant via context button/dialog, add localization
* refactor: conditionally show use and context menu buttons, add localization for create assistant
* feat: save side panel states to localStorage
* style(SidePanel): improve avatar menu and assistant select styling for dark mode
* refactor: make NavToggle reusable for either side (left or right), add SidePanel Toggle with ability to close it completely
* fix: resize handle and navToggle behavior
* fix(/avatar/:assistant_id): await `deleteFile` and assign unique name to uploaded image
* WIP: file UI components from PR #576
* refactor(OpenAIMinimalIcon): pass className
* feat: formatDate helper fn
* feat: DataTableColumnHeader
* feat: add row selection, formatted row values, number of rows selected
* WIP: add files to Side panel temporarily
* feat: `LB_QueueAsyncCall`: Leaky Bucket queue for external APIs, use in `processDeleteRequest`
* fix(TFile): correct `source` type with `FileSources`
* fix(useFileHandling): use `continue` instead of return when iterating multiple files, add file type to extendedFile
* chore: add generic setter type
* refactor(processDeleteRequest): settle promises to prevent rejections from processing deletions, log errors
* feat: `useFileDeletion` to reuse file deletion logic
* refactor(useFileDeletion): make `setFiles` an optional param and use object as param
* feat: useDeleteFilesFromTable
* feat: use real `files` data and add deletion action to data table
* fix(Table): make headers sticky
* feat: add dynamic filtering for columns; only show to user Host or OpenAI storage type
* style(DropdownMenu): replace `slate` with `gray`
* style(DataTable): apply dark mode themes and other misc styling
* style(Columns): add color to OpenAI Storage option
* refactor(FileContainer): make file preview reusable
* refactor(Images): make image preview reusable
* refactor(FilePreview): make file prop optional for FileIcon and FilePreview, fix relative style
* feat(Columns): add file/image previews, set a minimum size to show for file size in bytes
* WIP: File Panel with real files and formatted
* feat: open files dialog from panel
* style: file data table mobile and general column styling fixes
* refactor(api/files): return files sorted by the most recently updated
* refactor: provide fileMap through context to prevent re-selecting files to map in different areas; remove unused imports commented out in PanelColumns
* refactor(ExtendFile): make File type optional, add `attached` to prevent attached files from being deleted on remove, make Message.files a partial TFile type
* feat: attach files through file panel
* refactor(useFileHandling): move files to the start of cache list when uploaded
* refactor(useDeleteFilesMutation): delete files from cache when successfully deleted from server
* fix(FileRow): handle possible edge case of duplication due to attaching recently uploaded file
* style(SidePanel): make resize grip border transparent, remove unnecessary styling on close sidepanel button
* feat: action utilities and tests
* refactor(actions): add `ValidationResult` type and change wording for no server URL found
* refactor(actions): check for empty server URL
* fix(data-provider): revert tsconfig to fix type issue resolution
* feat(client): first pass of actions input for assistants
* refactor(FunctionSignature): change method to output object instead of string
* refactor(models/Assistant): add actions field to schema, use searchParams object for methods, and add `getAssistant`
* feat: post actions input first pass
- create new Action document
- add actions to Assistant DB document
- create /action/:assistant_id POST route
- pass more props down from PanelSwitcher, derive assistant_id from switcher
- move privacy policy to ActionInput
- reset data on input change/validation
- add `useUpdateAction`
- conform FunctionSignature type to FunctionTool
- add action, assistant doc, update hook related types
* refactor: optimize assistant/actions relationship
- past domain in metadata as hostname and not a URL
- include domain in tool name
- add `getActions` for actions retrieval by user
- add `getAssistants` for assistant docs retrieval by user
- add `assistant_id` to Action schema
- move actions to own module as a subroute to `api/assistants`
- add `useGetActionsQuery` and `useGetAssistantDocsQuery` hooks
- fix Action type def
* feat: show assistant actions in assistant builder
* feat: switch to actions on action click, editing action styling
* fix: add Assistant state for builder panel to allow immediate selection of newly created assistants as well as retaining the current assistant when switching to a different panel within the builder
* refactor(SidePanel/NavToggle): offset less from right when SidePanel is completely collapsed
* chore: rename `processActions` -> `processRequiredActions`
* chore: rename Assistant API Action to RequiredAction
* refactor(actions): avoid nesting actual API params under generic `requestBody` to optimize LLM token usage
* fix(handleTools): avoid calling `validTool` if not defined, add optional param to skip the loading of specs, which throws an error in the context of assistants
* WIP: working first pass of toolCalls generated from openapi specs
* WIP: first pass ToolCall styling
* feat: programmatic iv encryption/decryption helpers
* fix: correct ActionAuth types/enums, and define type for AuthForm
* feat: encryption/decryption helpers for Action AuthMetadata
* refactor(getActions): remove sensitive fields from query response
* refactor(POST/actions): encrypt and remove sensitive fields from mutation response
* fix(ActionService): change ESM import to CJS
* feat: frontend auth handling for actions + optimistic update on action update/creation
* refactor(actions): use the correct variables and types for setAuth method
* refactor: POST /:assistant_id action can now handle updating an existing action, add `saved_auth_fields` to determine when user explicitly saves new auth creds. only send auth metadata if user explicitly saved fields
* refactor(createActionTool): catch errors and send back meaningful error message, add flag to `getActions` to determine whether to retrieve sensitive values or not
* refactor(ToolService): add `action` property to ToolCall PartMetadata to determine if the tool call was an action, fix parsing function name issue with actionDelimiter
* fix(ActionRequest): use URL class to correctly join endpoint parts for `execute` call
* feat: delete assistant actions
* refactor: conditionally show Available actions
* refactor: show `retrieval` and `code_interpreter` as Capabilities, swap `Switch` for `Checkbox`
* chore: remove shadow-stroke from messages
* WIP: first pass of Assistants Knowledge attachments
* refactor: remove AssistantsProvider in favor of FormProvider, fix selectedAssistant re-render bug, map Assistant file_ids to files via fileMap, initialize Knowledge component with mapped files if any exist
* fix: prevent deleting files on assistant file upload
* chore: remove console.log
* refactor(useUploadFileMutation): update files and assistants cache on upload
* chore: disable oauth option as not supported yet
* feat: cancel assistant runs
* refactor: initialize OpenAI client with helper function, resolve all related circular dependencies
* fix(DALL-E): initialization
* fix(process): openai client initialization
* fix: select an existing Assistant when the active one is deleted
* chore: allow attaching files for assistant endpoint, send back relevant OpenAI error message when uploading, deconstruct openAI initialization correctly, add `message_file` to formData when a file is attached to the message but not the assistant
* fix: add assistant_id on newConvo
* fix(initializeClient): import fix
* chore: swap setAssistant for setOption in useEffect
* fix(DALL-E): add processFileURL to loadTools call
* chore: add customConfig to debug logs
* feat: delete threads on convo delete
* chore: replace Assistants icon
* chore: remove console.dir() in `abortRun`
* feat(AssistantService): accumulate text values from run in openai.responseText
* feat: titling for assistants endpoint
* chore: move panel file components to appropriate directory, add file checks for attaching files, change icon for Attach Files
* refactor: add localizations to tools, plugins, add condition for adding/remove user plugins so tool selections don't affect this value
* chore: disable `import from url` action for now
* chore: remove textMimeTypes from default fileConfig for now
* fix: catch tool errors and send as outputs with error messages
* fix: React warning about button as descendant of button
* style: retrieval and cancelled icon
* WIP: pass isSubmitting to Parts, use InProgressCall to display cancelled tool calls correctly, show domain/function name
* fix(meilisearch): fix `postSaveHook` issue where indexing expects a mongo document, and join all text content parts for meili indexing
* ci: fix dall-e tests
* ci: fix client tests
* fix: button types in actions panel
* fix: plugin auth form persisting across tool selections
* fix(ci): update AppService spec with `loadAndFormatTools`
* fix(clearConvos): add id check earlier on
* refactor(AssistantAvatar): set previewURL dynamically when emtadata.avatar changes
* feat(assistants): addTitle cache setting
* fix(useSSE): resolve rebase conflicts
* fix: delete mutation
* style(SidePanel): make grip visible on active and hover, invisible otherwise
* ci: add data-provider tests to workflow, also update eslint/tsconfig to recognize specs, and add `text/csv` to fileConfig
* fix: handle edge case where auth object is undefined, and log errors
* refactor(actions): resolve schemas, add tests for resolving refs, import specs from separate file for tests
* chore: remove comment
* fix(ActionsInput): re-render bug when initializing states with action fields
* fix(patch/assistant): filter undefined tools
* chore: add logging for errors in assistants routes
* fix(updateAssistant): map actions to functions to avoid overwriting
* fix(actions): properly handle GET paths
* fix(convos): unhandled delete thread exception
* refactor(AssistantService): pass both thread_id and conversationId when sending intermediate assistant messages, remove `mapMessagesToSteps` from AssistantService
* refactor(useSSE): replace all messages with runMessages and pass latestMessageId to abortRun; fix(checkMessageGaps): include tool calls when syncing messages
* refactor(assistants/chat): invoke `createOnTextProgress` after thread creation
* chore: add typing
* style: sidepanel styling
* style: action tool call domain styling
* feat(assistants): default models, limit retrieval to certain models, add env variables to to env.example
* feat: assistants api key in EndpointService
* refactor: set assistant model to conversation on assistant switch
* refactor: set assistant model to conversation on assistant select from panel
* fix(retrieveAndProcessFile): catch attempt to download file with `assistant` purpose which is not allowed; add logging
* feat: retrieval styling, handling, and logging
* chore: rename ASSISTANTS_REVERSE_PROXY to ASSISTANTS_BASE_URL
* feat: FileContext for file metadata
* feat: context file mgmt and filtering
* style(Select): hover/rounded changes
* refactor: explicit conversation switch, endpoint dependent, through `useSelectAssistant`, which does not create new chat if current endpoint is assistant endpoint
* fix(AssistantAvatar): make empty previewURL if no avatar present
* refactor: side panel mobile styling
* style: merge tool and action section, optimize mobile styling for action/tool buttons
* fix: localStorage issues
* fix(useSelectAssistant): invoke react query hook directly in select hook as Map was not being updated in time
* style: light mode fixes
* fix: prevent sidepanel nav styling from shifting layout up
* refactor: change default layout (collapsed by default)
* style: mobile optimization of DataTable
* style: datatable
* feat: client-side hide right-side panel
* chore(useNewConvo): add partial typing for preset
* fix(useSelectAssistant): pass correct model name by using template as preset
* WIP: assistant presets
* refactor(ToolService): add native solution for `TavilySearchResults` and log tool output errors
* refactor: organize imports and use native TavilySearchResults
* fix(TavilySearchResults): stringify result
* fix(ToolCall): show tool call outputs when not an action
* chore: rename Prompt Prefix to custom instructions (in user facing text only)
* refactor(EditPresetDialog): Optimize setting title by debouncing, reset preset on dialog close to avoid state mixture
* feat: add `presetOverride` to overwrite active conversation settings when saving a Preset (relevant for client side updates only)
* feat: Assistant preset settings (client-side)
* fix(Switcher): only set assistant_id and model if current endpoint is Assistants
* feat: use `useDebouncedInput` for updating conversation settings, starting with EditPresetDialog title setting and Assistant instructions setting
* feat(Assistants): add instructions field to settings
* feat(chat/assistants): pass conversation settings to run body
* wip: begin localization and only allow actions if the assistant is created
* refactor(AssistantsPanel): knowledge localization, allow tools on creation
* feat: experimental: allow 'priming' values before assistant is created, that would normally require an assistant_id to be defined
* chore: trim console logs and make more meaningful
* chore: toast messages
* fix(ci): date test
* feat: create file when uploading Assistant Avatar
* feat: file upload rate limiting from custom config with dynamic file route initialization
* refactor: use file upload limiters on post routes only
* refactor(fileConfig): add endpoints field for endpoint specific fileconfigs, add mergeConfig function, add tests
* refactor: fileConfig route, dynamic multer instances used on all '/' and '/images' POST routes, data service and query hook
* feat: supportedMimeTypesSchema, test for array of regex
* feat: configurable file config limits
* chore: clarify assistants file knowledge prereq.
* chore(useTextarea): default to localized 'Assistant' if assistant name is empty
* feat: configurable file limits and toggle file upload per endpoint
* fix(useUploadFileMutation): prevent updating assistant.files cache if file upload is a message_file attachment
* fix(AssistantSelect): set last selected assistant only when timeout successfully runs
* refactor(queries): disable assistant queries if assistants endpoint is not enabled
* chore(Switcher): add localization
* chore: pluralize `assistant` for `EModelEndpoint key and value
* feat: show/hide assistant UI components based on endpoint availability; librechat.yaml config for disabling builder section and setting polling/timeout intervals
* fix(compactEndpointSchemas): use EModelEndpoint for schema access
* feat(runAssistant): use configured values from `librechat.yaml` for `pollIntervalMs` and `timeout`
* fix: naming issue
* wip: revert landing
* 🎉 happy birthday LibreChat (#1768)
* happy birthday LibreChat
* Refactor endpoint condition in Landing component
* Update birthday message in Eng.tsx
* fix(/config): avoid nesting ternaries
* refactor(/config): check birthday
---------
Co-authored-by: Danny Avila <messagedaniel@protonmail.com>
* fix: landing
* fix: landing
* fix(useMessageHelpers): hardcoded check to use EModelEndpoint instead
* fix(ci): convo test revert to main
* fix(assistants/chat): fix issue where assistant_id was being saved as model for convo
* chore: added logging, promises racing to prevent longer timeouts, explicit setting of maxRetries and timeouts, robust catching of invalid abortRun params
* refactor: use recoil state for `showStopButton` and only show for assistants endpoint after syncing conversation data
* refactor: optimize abortRun strategy using localStorage, refactor `abortConversation` to use async/await and await the result, refactor how the abortKey cache is set for runs
* fix(checkMessageGaps): assign `assistant_id` to synced messages if defined; prevents UI from showing blank assistant for cancelled messages
* refactor: re-order sequence of chat route, only allow aborting messages after run is created, cancel abortRun if there was a cancelling error (likely due already cancelled in chat route), and add extra logging
* chore(typedefs): add httpAgent type to OpenAIClient
* refactor: use custom implementation of retrieving run with axios to allow for timing out run query
* fix(waitForRun): handle timed out run retrieval query
* refactor: update preset conditions:
- presets will retain settings when a different endpoint is selected; for existing convos, either when modular or is assistant switch
- no longer use `navigateToConvo` on preset select
* fix: temporary calculator hack as expects string input when invoked
* fix: cancel abortRun only when cancelling error is a result of the run already being cancelled
* chore: remove use of `fileMaxSizeMB` and total counterpart (redundant)
* docs: custom config documentation update
* docs: assistants api setup and dotenv, new custom config fields
* refactor(Switcher): make Assistant switcher sticky in SidePanel
* chore(useSSE): remove console log of data and message index
* refactor(AssistantPanel): button styling and add secondary select button to bottom of panel
* refactor(OpenAIClient): allow passing conversationId to RunManager through titleConvo and initializeLLM to properly record title context tokens used in cases where conversationId was not defined by the client
* feat(assistants): token tracking for assistant runs
* chore(spendTokens): improve logging
* feat: support/exclude specific assistant Ids
* chore: add update `librechat.example.yaml`, optimize `AppService` handling, new tests for `AppService`, optimize missing/outdate config logging
* chore: mount docker logs to root of project
* chore: condense axios errors
* chore: bump vite
* chore: vite hot reload fix using latest version
* chore(getOpenAIModels): sort instruct models to the end of models list
* fix(assistants): user provided key
* fix(assistants): user provided key, invalidate more queries on revoke
---------
Co-authored-by: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
2024-02-13 20:42:27 -05:00
|
|
|
|
2024-11-12 16:41:04 -05:00
|
|
|
if (req.originalUrl.endsWith('/speech/stt') && file.mimetype.startsWith('audio/')) {
|
|
|
|
return cb(null, true);
|
|
|
|
}
|
|
|
|
|
2024-11-07 11:11:20 -05:00
|
|
|
const endpoint = req.body.endpoint;
|
|
|
|
const supportedTypes =
|
|
|
|
customFileConfig?.endpoints?.[endpoint]?.supportedMimeTypes ??
|
|
|
|
customFileConfig?.endpoints?.default.supportedMimeTypes ??
|
|
|
|
defaultFileConfig?.endpoints?.[endpoint]?.supportedMimeTypes;
|
|
|
|
|
|
|
|
if (!defaultFileConfig.checkType(file.mimetype, supportedTypes)) {
|
|
|
|
return cb(new Error('Unsupported file type: ' + file.mimetype), false);
|
|
|
|
}
|
|
|
|
|
|
|
|
cb(null, true);
|
|
|
|
};
|
feat: Vision Support + New UI (#1203)
* feat: add timer duration to showToast, show toast for preset selection
* refactor: replace old /chat/ route with /c/. e2e tests will fail here
* refactor: move typedefs to root of /api/ and add a few to assistant types in TS
* refactor: reorganize data-provider imports, fix dependency cycle, strategize new plan to separate react dependent packages
* feat: add dataService for uploading images
* feat(data-provider): add mutation keys
* feat: file resizing and upload
* WIP: initial API image handling
* fix: catch JSON.parse of localStorage tools
* chore: experimental: use module-alias for absolute imports
* refactor: change temp_file_id strategy
* fix: updating files state by using Map and defining react query callbacks in a way that keeps them during component unmount, initial delete handling
* feat: properly handle file deletion
* refactor: unexpose complete filepath and resize from server for higher fidelity
* fix: make sure resized height, width is saved, catch bad requests
* refactor: use absolute imports
* fix: prevent setOptions from being called more than once for OpenAIClient, made note to fix for PluginsClient
* refactor: import supportsFiles and models vars from schemas
* fix: correctly replace temp file id
* refactor(BaseClient): use absolute imports, pass message 'opts' to buildMessages method, count tokens for nested objects/arrays
* feat: add validateVisionModel to determine if model has vision capabilities
* chore(checkBalance): update jsdoc
* feat: formatVisionMessage: change message content format dependent on role and image_urls passed
* refactor: add usage to File schema, make create and updateFile, correctly set and remove TTL
* feat: working vision support
TODO: file size, type, amount validations, making sure they are styled right, and making sure you can add images from the clipboard/dragging
* feat: clipboard support for uploading images
* feat: handle files on drop to screen, refactor top level view code to Presentation component so the useDragHelpers hook has ChatContext
* fix(Images): replace uploaded images in place
* feat: add filepath validation to protect sensitive files
* fix: ensure correct file_ids are push and not the Map key values
* fix(ToastContext): type issue
* feat: add basic file validation
* fix(useDragHelpers): correct context issue with `files` dependency
* refactor: consolidate setErrors logic to setError
* feat: add dialog Image overlay on image click
* fix: close endpoints menu on click
* chore: set detail to auto, make note for configuration
* fix: react warning (button desc. of button)
* refactor: optimize filepath handling, pass file_ids to images for easier re-use
* refactor: optimize image file handling, allow re-using files in regen, pass more file metadata in messages
* feat: lazy loading images including use of upload preview
* fix: SetKeyDialog closing, stopPropagation on Dialog content click
* style(EndpointMenuItem): tighten up the style, fix dark theme showing in lightmode, make menu more ux friendly
* style: change maxheight of all settings textareas to 138px from 300px
* style: better styling for textarea and enclosing buttons
* refactor(PresetItems): swap back edit and delete icons
* feat: make textarea placeholder dynamic to endpoint
* style: show user hover buttons only on hover when message is streaming
* fix: ordered list not going past 9, fix css
* feat: add User/AI labels; style: hide loading spinner
* feat: add back custom footer, change original footer text
* feat: dynamic landing icons based on endpoint
* chore: comment out assistants route
* fix: autoScroll to newest on /c/ view
* fix: Export Conversation on new UI
* style: match message style of official more closely
* ci: fix api jest unit tests, comment out e2e tests for now as they will fail until addressed
* feat: more file validation and use blob in preview field, not filepath, to fix temp deletion
* feat: filefilter for multer
* feat: better AI labels based on custom name, model, and endpoint instead of `ChatGPT`
2023-11-21 20:12:48 -05:00
|
|
|
|
2024-11-07 11:11:20 -05:00
|
|
|
return fileFilter;
|
feat: Vision Support + New UI (#1203)
* feat: add timer duration to showToast, show toast for preset selection
* refactor: replace old /chat/ route with /c/. e2e tests will fail here
* refactor: move typedefs to root of /api/ and add a few to assistant types in TS
* refactor: reorganize data-provider imports, fix dependency cycle, strategize new plan to separate react dependent packages
* feat: add dataService for uploading images
* feat(data-provider): add mutation keys
* feat: file resizing and upload
* WIP: initial API image handling
* fix: catch JSON.parse of localStorage tools
* chore: experimental: use module-alias for absolute imports
* refactor: change temp_file_id strategy
* fix: updating files state by using Map and defining react query callbacks in a way that keeps them during component unmount, initial delete handling
* feat: properly handle file deletion
* refactor: unexpose complete filepath and resize from server for higher fidelity
* fix: make sure resized height, width is saved, catch bad requests
* refactor: use absolute imports
* fix: prevent setOptions from being called more than once for OpenAIClient, made note to fix for PluginsClient
* refactor: import supportsFiles and models vars from schemas
* fix: correctly replace temp file id
* refactor(BaseClient): use absolute imports, pass message 'opts' to buildMessages method, count tokens for nested objects/arrays
* feat: add validateVisionModel to determine if model has vision capabilities
* chore(checkBalance): update jsdoc
* feat: formatVisionMessage: change message content format dependent on role and image_urls passed
* refactor: add usage to File schema, make create and updateFile, correctly set and remove TTL
* feat: working vision support
TODO: file size, type, amount validations, making sure they are styled right, and making sure you can add images from the clipboard/dragging
* feat: clipboard support for uploading images
* feat: handle files on drop to screen, refactor top level view code to Presentation component so the useDragHelpers hook has ChatContext
* fix(Images): replace uploaded images in place
* feat: add filepath validation to protect sensitive files
* fix: ensure correct file_ids are push and not the Map key values
* fix(ToastContext): type issue
* feat: add basic file validation
* fix(useDragHelpers): correct context issue with `files` dependency
* refactor: consolidate setErrors logic to setError
* feat: add dialog Image overlay on image click
* fix: close endpoints menu on click
* chore: set detail to auto, make note for configuration
* fix: react warning (button desc. of button)
* refactor: optimize filepath handling, pass file_ids to images for easier re-use
* refactor: optimize image file handling, allow re-using files in regen, pass more file metadata in messages
* feat: lazy loading images including use of upload preview
* fix: SetKeyDialog closing, stopPropagation on Dialog content click
* style(EndpointMenuItem): tighten up the style, fix dark theme showing in lightmode, make menu more ux friendly
* style: change maxheight of all settings textareas to 138px from 300px
* style: better styling for textarea and enclosing buttons
* refactor(PresetItems): swap back edit and delete icons
* feat: make textarea placeholder dynamic to endpoint
* style: show user hover buttons only on hover when message is streaming
* fix: ordered list not going past 9, fix css
* feat: add User/AI labels; style: hide loading spinner
* feat: add back custom footer, change original footer text
* feat: dynamic landing icons based on endpoint
* chore: comment out assistants route
* fix: autoScroll to newest on /c/ view
* fix: Export Conversation on new UI
* style: match message style of official more closely
* ci: fix api jest unit tests, comment out e2e tests for now as they will fail until addressed
* feat: more file validation and use blob in preview field, not filepath, to fix temp deletion
* feat: filefilter for multer
* feat: better AI labels based on custom name, model, and endpoint instead of `ChatGPT`
2023-11-21 20:12:48 -05:00
|
|
|
};
|
|
|
|
|
✨ feat: Assistants API, General File Support, Side Panel, File Explorer (#1696)
* feat: assistant name/icon in Landing & Header
* feat: assistname in textarea placeholder, and use `Assistant` as default name
* feat: display non-image files in user messages
* fix: only render files if files.length is > 0
* refactor(config -> file-config): move file related configuration values to separate module, add excel types
* chore: spreadsheet file rendering
* fix(Landing): dark mode style for Assistant Name
* refactor: move progress incrementing to own hook, start smaller, cap near limit \(1\)
* refactor(useContentHandler): add empty Text part if last part was completed tool or image
* chore: add accordion trigger border styling for dark mode
* feat: Assistant Builder model selection
* chore: use Spinner when Assistant is mutating
* fix(get/assistants): return correct response object `AssistantListResponse`
* refactor(Spinner): pass size as prop
* refactor: make assistant crud mutations optimistic, add types for options
* chore: remove assistants route and view
* chore: move assistant builder components to separate directory
* feat(ContextButton): delete Assistant via context button/dialog, add localization
* refactor: conditionally show use and context menu buttons, add localization for create assistant
* feat: save side panel states to localStorage
* style(SidePanel): improve avatar menu and assistant select styling for dark mode
* refactor: make NavToggle reusable for either side (left or right), add SidePanel Toggle with ability to close it completely
* fix: resize handle and navToggle behavior
* fix(/avatar/:assistant_id): await `deleteFile` and assign unique name to uploaded image
* WIP: file UI components from PR #576
* refactor(OpenAIMinimalIcon): pass className
* feat: formatDate helper fn
* feat: DataTableColumnHeader
* feat: add row selection, formatted row values, number of rows selected
* WIP: add files to Side panel temporarily
* feat: `LB_QueueAsyncCall`: Leaky Bucket queue for external APIs, use in `processDeleteRequest`
* fix(TFile): correct `source` type with `FileSources`
* fix(useFileHandling): use `continue` instead of return when iterating multiple files, add file type to extendedFile
* chore: add generic setter type
* refactor(processDeleteRequest): settle promises to prevent rejections from processing deletions, log errors
* feat: `useFileDeletion` to reuse file deletion logic
* refactor(useFileDeletion): make `setFiles` an optional param and use object as param
* feat: useDeleteFilesFromTable
* feat: use real `files` data and add deletion action to data table
* fix(Table): make headers sticky
* feat: add dynamic filtering for columns; only show to user Host or OpenAI storage type
* style(DropdownMenu): replace `slate` with `gray`
* style(DataTable): apply dark mode themes and other misc styling
* style(Columns): add color to OpenAI Storage option
* refactor(FileContainer): make file preview reusable
* refactor(Images): make image preview reusable
* refactor(FilePreview): make file prop optional for FileIcon and FilePreview, fix relative style
* feat(Columns): add file/image previews, set a minimum size to show for file size in bytes
* WIP: File Panel with real files and formatted
* feat: open files dialog from panel
* style: file data table mobile and general column styling fixes
* refactor(api/files): return files sorted by the most recently updated
* refactor: provide fileMap through context to prevent re-selecting files to map in different areas; remove unused imports commented out in PanelColumns
* refactor(ExtendFile): make File type optional, add `attached` to prevent attached files from being deleted on remove, make Message.files a partial TFile type
* feat: attach files through file panel
* refactor(useFileHandling): move files to the start of cache list when uploaded
* refactor(useDeleteFilesMutation): delete files from cache when successfully deleted from server
* fix(FileRow): handle possible edge case of duplication due to attaching recently uploaded file
* style(SidePanel): make resize grip border transparent, remove unnecessary styling on close sidepanel button
* feat: action utilities and tests
* refactor(actions): add `ValidationResult` type and change wording for no server URL found
* refactor(actions): check for empty server URL
* fix(data-provider): revert tsconfig to fix type issue resolution
* feat(client): first pass of actions input for assistants
* refactor(FunctionSignature): change method to output object instead of string
* refactor(models/Assistant): add actions field to schema, use searchParams object for methods, and add `getAssistant`
* feat: post actions input first pass
- create new Action document
- add actions to Assistant DB document
- create /action/:assistant_id POST route
- pass more props down from PanelSwitcher, derive assistant_id from switcher
- move privacy policy to ActionInput
- reset data on input change/validation
- add `useUpdateAction`
- conform FunctionSignature type to FunctionTool
- add action, assistant doc, update hook related types
* refactor: optimize assistant/actions relationship
- past domain in metadata as hostname and not a URL
- include domain in tool name
- add `getActions` for actions retrieval by user
- add `getAssistants` for assistant docs retrieval by user
- add `assistant_id` to Action schema
- move actions to own module as a subroute to `api/assistants`
- add `useGetActionsQuery` and `useGetAssistantDocsQuery` hooks
- fix Action type def
* feat: show assistant actions in assistant builder
* feat: switch to actions on action click, editing action styling
* fix: add Assistant state for builder panel to allow immediate selection of newly created assistants as well as retaining the current assistant when switching to a different panel within the builder
* refactor(SidePanel/NavToggle): offset less from right when SidePanel is completely collapsed
* chore: rename `processActions` -> `processRequiredActions`
* chore: rename Assistant API Action to RequiredAction
* refactor(actions): avoid nesting actual API params under generic `requestBody` to optimize LLM token usage
* fix(handleTools): avoid calling `validTool` if not defined, add optional param to skip the loading of specs, which throws an error in the context of assistants
* WIP: working first pass of toolCalls generated from openapi specs
* WIP: first pass ToolCall styling
* feat: programmatic iv encryption/decryption helpers
* fix: correct ActionAuth types/enums, and define type for AuthForm
* feat: encryption/decryption helpers for Action AuthMetadata
* refactor(getActions): remove sensitive fields from query response
* refactor(POST/actions): encrypt and remove sensitive fields from mutation response
* fix(ActionService): change ESM import to CJS
* feat: frontend auth handling for actions + optimistic update on action update/creation
* refactor(actions): use the correct variables and types for setAuth method
* refactor: POST /:assistant_id action can now handle updating an existing action, add `saved_auth_fields` to determine when user explicitly saves new auth creds. only send auth metadata if user explicitly saved fields
* refactor(createActionTool): catch errors and send back meaningful error message, add flag to `getActions` to determine whether to retrieve sensitive values or not
* refactor(ToolService): add `action` property to ToolCall PartMetadata to determine if the tool call was an action, fix parsing function name issue with actionDelimiter
* fix(ActionRequest): use URL class to correctly join endpoint parts for `execute` call
* feat: delete assistant actions
* refactor: conditionally show Available actions
* refactor: show `retrieval` and `code_interpreter` as Capabilities, swap `Switch` for `Checkbox`
* chore: remove shadow-stroke from messages
* WIP: first pass of Assistants Knowledge attachments
* refactor: remove AssistantsProvider in favor of FormProvider, fix selectedAssistant re-render bug, map Assistant file_ids to files via fileMap, initialize Knowledge component with mapped files if any exist
* fix: prevent deleting files on assistant file upload
* chore: remove console.log
* refactor(useUploadFileMutation): update files and assistants cache on upload
* chore: disable oauth option as not supported yet
* feat: cancel assistant runs
* refactor: initialize OpenAI client with helper function, resolve all related circular dependencies
* fix(DALL-E): initialization
* fix(process): openai client initialization
* fix: select an existing Assistant when the active one is deleted
* chore: allow attaching files for assistant endpoint, send back relevant OpenAI error message when uploading, deconstruct openAI initialization correctly, add `message_file` to formData when a file is attached to the message but not the assistant
* fix: add assistant_id on newConvo
* fix(initializeClient): import fix
* chore: swap setAssistant for setOption in useEffect
* fix(DALL-E): add processFileURL to loadTools call
* chore: add customConfig to debug logs
* feat: delete threads on convo delete
* chore: replace Assistants icon
* chore: remove console.dir() in `abortRun`
* feat(AssistantService): accumulate text values from run in openai.responseText
* feat: titling for assistants endpoint
* chore: move panel file components to appropriate directory, add file checks for attaching files, change icon for Attach Files
* refactor: add localizations to tools, plugins, add condition for adding/remove user plugins so tool selections don't affect this value
* chore: disable `import from url` action for now
* chore: remove textMimeTypes from default fileConfig for now
* fix: catch tool errors and send as outputs with error messages
* fix: React warning about button as descendant of button
* style: retrieval and cancelled icon
* WIP: pass isSubmitting to Parts, use InProgressCall to display cancelled tool calls correctly, show domain/function name
* fix(meilisearch): fix `postSaveHook` issue where indexing expects a mongo document, and join all text content parts for meili indexing
* ci: fix dall-e tests
* ci: fix client tests
* fix: button types in actions panel
* fix: plugin auth form persisting across tool selections
* fix(ci): update AppService spec with `loadAndFormatTools`
* fix(clearConvos): add id check earlier on
* refactor(AssistantAvatar): set previewURL dynamically when emtadata.avatar changes
* feat(assistants): addTitle cache setting
* fix(useSSE): resolve rebase conflicts
* fix: delete mutation
* style(SidePanel): make grip visible on active and hover, invisible otherwise
* ci: add data-provider tests to workflow, also update eslint/tsconfig to recognize specs, and add `text/csv` to fileConfig
* fix: handle edge case where auth object is undefined, and log errors
* refactor(actions): resolve schemas, add tests for resolving refs, import specs from separate file for tests
* chore: remove comment
* fix(ActionsInput): re-render bug when initializing states with action fields
* fix(patch/assistant): filter undefined tools
* chore: add logging for errors in assistants routes
* fix(updateAssistant): map actions to functions to avoid overwriting
* fix(actions): properly handle GET paths
* fix(convos): unhandled delete thread exception
* refactor(AssistantService): pass both thread_id and conversationId when sending intermediate assistant messages, remove `mapMessagesToSteps` from AssistantService
* refactor(useSSE): replace all messages with runMessages and pass latestMessageId to abortRun; fix(checkMessageGaps): include tool calls when syncing messages
* refactor(assistants/chat): invoke `createOnTextProgress` after thread creation
* chore: add typing
* style: sidepanel styling
* style: action tool call domain styling
* feat(assistants): default models, limit retrieval to certain models, add env variables to to env.example
* feat: assistants api key in EndpointService
* refactor: set assistant model to conversation on assistant switch
* refactor: set assistant model to conversation on assistant select from panel
* fix(retrieveAndProcessFile): catch attempt to download file with `assistant` purpose which is not allowed; add logging
* feat: retrieval styling, handling, and logging
* chore: rename ASSISTANTS_REVERSE_PROXY to ASSISTANTS_BASE_URL
* feat: FileContext for file metadata
* feat: context file mgmt and filtering
* style(Select): hover/rounded changes
* refactor: explicit conversation switch, endpoint dependent, through `useSelectAssistant`, which does not create new chat if current endpoint is assistant endpoint
* fix(AssistantAvatar): make empty previewURL if no avatar present
* refactor: side panel mobile styling
* style: merge tool and action section, optimize mobile styling for action/tool buttons
* fix: localStorage issues
* fix(useSelectAssistant): invoke react query hook directly in select hook as Map was not being updated in time
* style: light mode fixes
* fix: prevent sidepanel nav styling from shifting layout up
* refactor: change default layout (collapsed by default)
* style: mobile optimization of DataTable
* style: datatable
* feat: client-side hide right-side panel
* chore(useNewConvo): add partial typing for preset
* fix(useSelectAssistant): pass correct model name by using template as preset
* WIP: assistant presets
* refactor(ToolService): add native solution for `TavilySearchResults` and log tool output errors
* refactor: organize imports and use native TavilySearchResults
* fix(TavilySearchResults): stringify result
* fix(ToolCall): show tool call outputs when not an action
* chore: rename Prompt Prefix to custom instructions (in user facing text only)
* refactor(EditPresetDialog): Optimize setting title by debouncing, reset preset on dialog close to avoid state mixture
* feat: add `presetOverride` to overwrite active conversation settings when saving a Preset (relevant for client side updates only)
* feat: Assistant preset settings (client-side)
* fix(Switcher): only set assistant_id and model if current endpoint is Assistants
* feat: use `useDebouncedInput` for updating conversation settings, starting with EditPresetDialog title setting and Assistant instructions setting
* feat(Assistants): add instructions field to settings
* feat(chat/assistants): pass conversation settings to run body
* wip: begin localization and only allow actions if the assistant is created
* refactor(AssistantsPanel): knowledge localization, allow tools on creation
* feat: experimental: allow 'priming' values before assistant is created, that would normally require an assistant_id to be defined
* chore: trim console logs and make more meaningful
* chore: toast messages
* fix(ci): date test
* feat: create file when uploading Assistant Avatar
* feat: file upload rate limiting from custom config with dynamic file route initialization
* refactor: use file upload limiters on post routes only
* refactor(fileConfig): add endpoints field for endpoint specific fileconfigs, add mergeConfig function, add tests
* refactor: fileConfig route, dynamic multer instances used on all '/' and '/images' POST routes, data service and query hook
* feat: supportedMimeTypesSchema, test for array of regex
* feat: configurable file config limits
* chore: clarify assistants file knowledge prereq.
* chore(useTextarea): default to localized 'Assistant' if assistant name is empty
* feat: configurable file limits and toggle file upload per endpoint
* fix(useUploadFileMutation): prevent updating assistant.files cache if file upload is a message_file attachment
* fix(AssistantSelect): set last selected assistant only when timeout successfully runs
* refactor(queries): disable assistant queries if assistants endpoint is not enabled
* chore(Switcher): add localization
* chore: pluralize `assistant` for `EModelEndpoint key and value
* feat: show/hide assistant UI components based on endpoint availability; librechat.yaml config for disabling builder section and setting polling/timeout intervals
* fix(compactEndpointSchemas): use EModelEndpoint for schema access
* feat(runAssistant): use configured values from `librechat.yaml` for `pollIntervalMs` and `timeout`
* fix: naming issue
* wip: revert landing
* 🎉 happy birthday LibreChat (#1768)
* happy birthday LibreChat
* Refactor endpoint condition in Landing component
* Update birthday message in Eng.tsx
* fix(/config): avoid nesting ternaries
* refactor(/config): check birthday
---------
Co-authored-by: Danny Avila <messagedaniel@protonmail.com>
* fix: landing
* fix: landing
* fix(useMessageHelpers): hardcoded check to use EModelEndpoint instead
* fix(ci): convo test revert to main
* fix(assistants/chat): fix issue where assistant_id was being saved as model for convo
* chore: added logging, promises racing to prevent longer timeouts, explicit setting of maxRetries and timeouts, robust catching of invalid abortRun params
* refactor: use recoil state for `showStopButton` and only show for assistants endpoint after syncing conversation data
* refactor: optimize abortRun strategy using localStorage, refactor `abortConversation` to use async/await and await the result, refactor how the abortKey cache is set for runs
* fix(checkMessageGaps): assign `assistant_id` to synced messages if defined; prevents UI from showing blank assistant for cancelled messages
* refactor: re-order sequence of chat route, only allow aborting messages after run is created, cancel abortRun if there was a cancelling error (likely due already cancelled in chat route), and add extra logging
* chore(typedefs): add httpAgent type to OpenAIClient
* refactor: use custom implementation of retrieving run with axios to allow for timing out run query
* fix(waitForRun): handle timed out run retrieval query
* refactor: update preset conditions:
- presets will retain settings when a different endpoint is selected; for existing convos, either when modular or is assistant switch
- no longer use `navigateToConvo` on preset select
* fix: temporary calculator hack as expects string input when invoked
* fix: cancel abortRun only when cancelling error is a result of the run already being cancelled
* chore: remove use of `fileMaxSizeMB` and total counterpart (redundant)
* docs: custom config documentation update
* docs: assistants api setup and dotenv, new custom config fields
* refactor(Switcher): make Assistant switcher sticky in SidePanel
* chore(useSSE): remove console log of data and message index
* refactor(AssistantPanel): button styling and add secondary select button to bottom of panel
* refactor(OpenAIClient): allow passing conversationId to RunManager through titleConvo and initializeLLM to properly record title context tokens used in cases where conversationId was not defined by the client
* feat(assistants): token tracking for assistant runs
* chore(spendTokens): improve logging
* feat: support/exclude specific assistant Ids
* chore: add update `librechat.example.yaml`, optimize `AppService` handling, new tests for `AppService`, optimize missing/outdate config logging
* chore: mount docker logs to root of project
* chore: condense axios errors
* chore: bump vite
* chore: vite hot reload fix using latest version
* chore(getOpenAIModels): sort instruct models to the end of models list
* fix(assistants): user provided key
* fix(assistants): user provided key, invalidate more queries on revoke
---------
Co-authored-by: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
2024-02-13 20:42:27 -05:00
|
|
|
const createMulterInstance = async () => {
|
|
|
|
const customConfig = await getCustomConfig();
|
|
|
|
const fileConfig = mergeFileConfig(customConfig?.fileConfig);
|
2024-11-07 11:11:20 -05:00
|
|
|
const fileFilter = createFileFilter(fileConfig);
|
✨ feat: Assistants API, General File Support, Side Panel, File Explorer (#1696)
* feat: assistant name/icon in Landing & Header
* feat: assistname in textarea placeholder, and use `Assistant` as default name
* feat: display non-image files in user messages
* fix: only render files if files.length is > 0
* refactor(config -> file-config): move file related configuration values to separate module, add excel types
* chore: spreadsheet file rendering
* fix(Landing): dark mode style for Assistant Name
* refactor: move progress incrementing to own hook, start smaller, cap near limit \(1\)
* refactor(useContentHandler): add empty Text part if last part was completed tool or image
* chore: add accordion trigger border styling for dark mode
* feat: Assistant Builder model selection
* chore: use Spinner when Assistant is mutating
* fix(get/assistants): return correct response object `AssistantListResponse`
* refactor(Spinner): pass size as prop
* refactor: make assistant crud mutations optimistic, add types for options
* chore: remove assistants route and view
* chore: move assistant builder components to separate directory
* feat(ContextButton): delete Assistant via context button/dialog, add localization
* refactor: conditionally show use and context menu buttons, add localization for create assistant
* feat: save side panel states to localStorage
* style(SidePanel): improve avatar menu and assistant select styling for dark mode
* refactor: make NavToggle reusable for either side (left or right), add SidePanel Toggle with ability to close it completely
* fix: resize handle and navToggle behavior
* fix(/avatar/:assistant_id): await `deleteFile` and assign unique name to uploaded image
* WIP: file UI components from PR #576
* refactor(OpenAIMinimalIcon): pass className
* feat: formatDate helper fn
* feat: DataTableColumnHeader
* feat: add row selection, formatted row values, number of rows selected
* WIP: add files to Side panel temporarily
* feat: `LB_QueueAsyncCall`: Leaky Bucket queue for external APIs, use in `processDeleteRequest`
* fix(TFile): correct `source` type with `FileSources`
* fix(useFileHandling): use `continue` instead of return when iterating multiple files, add file type to extendedFile
* chore: add generic setter type
* refactor(processDeleteRequest): settle promises to prevent rejections from processing deletions, log errors
* feat: `useFileDeletion` to reuse file deletion logic
* refactor(useFileDeletion): make `setFiles` an optional param and use object as param
* feat: useDeleteFilesFromTable
* feat: use real `files` data and add deletion action to data table
* fix(Table): make headers sticky
* feat: add dynamic filtering for columns; only show to user Host or OpenAI storage type
* style(DropdownMenu): replace `slate` with `gray`
* style(DataTable): apply dark mode themes and other misc styling
* style(Columns): add color to OpenAI Storage option
* refactor(FileContainer): make file preview reusable
* refactor(Images): make image preview reusable
* refactor(FilePreview): make file prop optional for FileIcon and FilePreview, fix relative style
* feat(Columns): add file/image previews, set a minimum size to show for file size in bytes
* WIP: File Panel with real files and formatted
* feat: open files dialog from panel
* style: file data table mobile and general column styling fixes
* refactor(api/files): return files sorted by the most recently updated
* refactor: provide fileMap through context to prevent re-selecting files to map in different areas; remove unused imports commented out in PanelColumns
* refactor(ExtendFile): make File type optional, add `attached` to prevent attached files from being deleted on remove, make Message.files a partial TFile type
* feat: attach files through file panel
* refactor(useFileHandling): move files to the start of cache list when uploaded
* refactor(useDeleteFilesMutation): delete files from cache when successfully deleted from server
* fix(FileRow): handle possible edge case of duplication due to attaching recently uploaded file
* style(SidePanel): make resize grip border transparent, remove unnecessary styling on close sidepanel button
* feat: action utilities and tests
* refactor(actions): add `ValidationResult` type and change wording for no server URL found
* refactor(actions): check for empty server URL
* fix(data-provider): revert tsconfig to fix type issue resolution
* feat(client): first pass of actions input for assistants
* refactor(FunctionSignature): change method to output object instead of string
* refactor(models/Assistant): add actions field to schema, use searchParams object for methods, and add `getAssistant`
* feat: post actions input first pass
- create new Action document
- add actions to Assistant DB document
- create /action/:assistant_id POST route
- pass more props down from PanelSwitcher, derive assistant_id from switcher
- move privacy policy to ActionInput
- reset data on input change/validation
- add `useUpdateAction`
- conform FunctionSignature type to FunctionTool
- add action, assistant doc, update hook related types
* refactor: optimize assistant/actions relationship
- past domain in metadata as hostname and not a URL
- include domain in tool name
- add `getActions` for actions retrieval by user
- add `getAssistants` for assistant docs retrieval by user
- add `assistant_id` to Action schema
- move actions to own module as a subroute to `api/assistants`
- add `useGetActionsQuery` and `useGetAssistantDocsQuery` hooks
- fix Action type def
* feat: show assistant actions in assistant builder
* feat: switch to actions on action click, editing action styling
* fix: add Assistant state for builder panel to allow immediate selection of newly created assistants as well as retaining the current assistant when switching to a different panel within the builder
* refactor(SidePanel/NavToggle): offset less from right when SidePanel is completely collapsed
* chore: rename `processActions` -> `processRequiredActions`
* chore: rename Assistant API Action to RequiredAction
* refactor(actions): avoid nesting actual API params under generic `requestBody` to optimize LLM token usage
* fix(handleTools): avoid calling `validTool` if not defined, add optional param to skip the loading of specs, which throws an error in the context of assistants
* WIP: working first pass of toolCalls generated from openapi specs
* WIP: first pass ToolCall styling
* feat: programmatic iv encryption/decryption helpers
* fix: correct ActionAuth types/enums, and define type for AuthForm
* feat: encryption/decryption helpers for Action AuthMetadata
* refactor(getActions): remove sensitive fields from query response
* refactor(POST/actions): encrypt and remove sensitive fields from mutation response
* fix(ActionService): change ESM import to CJS
* feat: frontend auth handling for actions + optimistic update on action update/creation
* refactor(actions): use the correct variables and types for setAuth method
* refactor: POST /:assistant_id action can now handle updating an existing action, add `saved_auth_fields` to determine when user explicitly saves new auth creds. only send auth metadata if user explicitly saved fields
* refactor(createActionTool): catch errors and send back meaningful error message, add flag to `getActions` to determine whether to retrieve sensitive values or not
* refactor(ToolService): add `action` property to ToolCall PartMetadata to determine if the tool call was an action, fix parsing function name issue with actionDelimiter
* fix(ActionRequest): use URL class to correctly join endpoint parts for `execute` call
* feat: delete assistant actions
* refactor: conditionally show Available actions
* refactor: show `retrieval` and `code_interpreter` as Capabilities, swap `Switch` for `Checkbox`
* chore: remove shadow-stroke from messages
* WIP: first pass of Assistants Knowledge attachments
* refactor: remove AssistantsProvider in favor of FormProvider, fix selectedAssistant re-render bug, map Assistant file_ids to files via fileMap, initialize Knowledge component with mapped files if any exist
* fix: prevent deleting files on assistant file upload
* chore: remove console.log
* refactor(useUploadFileMutation): update files and assistants cache on upload
* chore: disable oauth option as not supported yet
* feat: cancel assistant runs
* refactor: initialize OpenAI client with helper function, resolve all related circular dependencies
* fix(DALL-E): initialization
* fix(process): openai client initialization
* fix: select an existing Assistant when the active one is deleted
* chore: allow attaching files for assistant endpoint, send back relevant OpenAI error message when uploading, deconstruct openAI initialization correctly, add `message_file` to formData when a file is attached to the message but not the assistant
* fix: add assistant_id on newConvo
* fix(initializeClient): import fix
* chore: swap setAssistant for setOption in useEffect
* fix(DALL-E): add processFileURL to loadTools call
* chore: add customConfig to debug logs
* feat: delete threads on convo delete
* chore: replace Assistants icon
* chore: remove console.dir() in `abortRun`
* feat(AssistantService): accumulate text values from run in openai.responseText
* feat: titling for assistants endpoint
* chore: move panel file components to appropriate directory, add file checks for attaching files, change icon for Attach Files
* refactor: add localizations to tools, plugins, add condition for adding/remove user plugins so tool selections don't affect this value
* chore: disable `import from url` action for now
* chore: remove textMimeTypes from default fileConfig for now
* fix: catch tool errors and send as outputs with error messages
* fix: React warning about button as descendant of button
* style: retrieval and cancelled icon
* WIP: pass isSubmitting to Parts, use InProgressCall to display cancelled tool calls correctly, show domain/function name
* fix(meilisearch): fix `postSaveHook` issue where indexing expects a mongo document, and join all text content parts for meili indexing
* ci: fix dall-e tests
* ci: fix client tests
* fix: button types in actions panel
* fix: plugin auth form persisting across tool selections
* fix(ci): update AppService spec with `loadAndFormatTools`
* fix(clearConvos): add id check earlier on
* refactor(AssistantAvatar): set previewURL dynamically when emtadata.avatar changes
* feat(assistants): addTitle cache setting
* fix(useSSE): resolve rebase conflicts
* fix: delete mutation
* style(SidePanel): make grip visible on active and hover, invisible otherwise
* ci: add data-provider tests to workflow, also update eslint/tsconfig to recognize specs, and add `text/csv` to fileConfig
* fix: handle edge case where auth object is undefined, and log errors
* refactor(actions): resolve schemas, add tests for resolving refs, import specs from separate file for tests
* chore: remove comment
* fix(ActionsInput): re-render bug when initializing states with action fields
* fix(patch/assistant): filter undefined tools
* chore: add logging for errors in assistants routes
* fix(updateAssistant): map actions to functions to avoid overwriting
* fix(actions): properly handle GET paths
* fix(convos): unhandled delete thread exception
* refactor(AssistantService): pass both thread_id and conversationId when sending intermediate assistant messages, remove `mapMessagesToSteps` from AssistantService
* refactor(useSSE): replace all messages with runMessages and pass latestMessageId to abortRun; fix(checkMessageGaps): include tool calls when syncing messages
* refactor(assistants/chat): invoke `createOnTextProgress` after thread creation
* chore: add typing
* style: sidepanel styling
* style: action tool call domain styling
* feat(assistants): default models, limit retrieval to certain models, add env variables to to env.example
* feat: assistants api key in EndpointService
* refactor: set assistant model to conversation on assistant switch
* refactor: set assistant model to conversation on assistant select from panel
* fix(retrieveAndProcessFile): catch attempt to download file with `assistant` purpose which is not allowed; add logging
* feat: retrieval styling, handling, and logging
* chore: rename ASSISTANTS_REVERSE_PROXY to ASSISTANTS_BASE_URL
* feat: FileContext for file metadata
* feat: context file mgmt and filtering
* style(Select): hover/rounded changes
* refactor: explicit conversation switch, endpoint dependent, through `useSelectAssistant`, which does not create new chat if current endpoint is assistant endpoint
* fix(AssistantAvatar): make empty previewURL if no avatar present
* refactor: side panel mobile styling
* style: merge tool and action section, optimize mobile styling for action/tool buttons
* fix: localStorage issues
* fix(useSelectAssistant): invoke react query hook directly in select hook as Map was not being updated in time
* style: light mode fixes
* fix: prevent sidepanel nav styling from shifting layout up
* refactor: change default layout (collapsed by default)
* style: mobile optimization of DataTable
* style: datatable
* feat: client-side hide right-side panel
* chore(useNewConvo): add partial typing for preset
* fix(useSelectAssistant): pass correct model name by using template as preset
* WIP: assistant presets
* refactor(ToolService): add native solution for `TavilySearchResults` and log tool output errors
* refactor: organize imports and use native TavilySearchResults
* fix(TavilySearchResults): stringify result
* fix(ToolCall): show tool call outputs when not an action
* chore: rename Prompt Prefix to custom instructions (in user facing text only)
* refactor(EditPresetDialog): Optimize setting title by debouncing, reset preset on dialog close to avoid state mixture
* feat: add `presetOverride` to overwrite active conversation settings when saving a Preset (relevant for client side updates only)
* feat: Assistant preset settings (client-side)
* fix(Switcher): only set assistant_id and model if current endpoint is Assistants
* feat: use `useDebouncedInput` for updating conversation settings, starting with EditPresetDialog title setting and Assistant instructions setting
* feat(Assistants): add instructions field to settings
* feat(chat/assistants): pass conversation settings to run body
* wip: begin localization and only allow actions if the assistant is created
* refactor(AssistantsPanel): knowledge localization, allow tools on creation
* feat: experimental: allow 'priming' values before assistant is created, that would normally require an assistant_id to be defined
* chore: trim console logs and make more meaningful
* chore: toast messages
* fix(ci): date test
* feat: create file when uploading Assistant Avatar
* feat: file upload rate limiting from custom config with dynamic file route initialization
* refactor: use file upload limiters on post routes only
* refactor(fileConfig): add endpoints field for endpoint specific fileconfigs, add mergeConfig function, add tests
* refactor: fileConfig route, dynamic multer instances used on all '/' and '/images' POST routes, data service and query hook
* feat: supportedMimeTypesSchema, test for array of regex
* feat: configurable file config limits
* chore: clarify assistants file knowledge prereq.
* chore(useTextarea): default to localized 'Assistant' if assistant name is empty
* feat: configurable file limits and toggle file upload per endpoint
* fix(useUploadFileMutation): prevent updating assistant.files cache if file upload is a message_file attachment
* fix(AssistantSelect): set last selected assistant only when timeout successfully runs
* refactor(queries): disable assistant queries if assistants endpoint is not enabled
* chore(Switcher): add localization
* chore: pluralize `assistant` for `EModelEndpoint key and value
* feat: show/hide assistant UI components based on endpoint availability; librechat.yaml config for disabling builder section and setting polling/timeout intervals
* fix(compactEndpointSchemas): use EModelEndpoint for schema access
* feat(runAssistant): use configured values from `librechat.yaml` for `pollIntervalMs` and `timeout`
* fix: naming issue
* wip: revert landing
* 🎉 happy birthday LibreChat (#1768)
* happy birthday LibreChat
* Refactor endpoint condition in Landing component
* Update birthday message in Eng.tsx
* fix(/config): avoid nesting ternaries
* refactor(/config): check birthday
---------
Co-authored-by: Danny Avila <messagedaniel@protonmail.com>
* fix: landing
* fix: landing
* fix(useMessageHelpers): hardcoded check to use EModelEndpoint instead
* fix(ci): convo test revert to main
* fix(assistants/chat): fix issue where assistant_id was being saved as model for convo
* chore: added logging, promises racing to prevent longer timeouts, explicit setting of maxRetries and timeouts, robust catching of invalid abortRun params
* refactor: use recoil state for `showStopButton` and only show for assistants endpoint after syncing conversation data
* refactor: optimize abortRun strategy using localStorage, refactor `abortConversation` to use async/await and await the result, refactor how the abortKey cache is set for runs
* fix(checkMessageGaps): assign `assistant_id` to synced messages if defined; prevents UI from showing blank assistant for cancelled messages
* refactor: re-order sequence of chat route, only allow aborting messages after run is created, cancel abortRun if there was a cancelling error (likely due already cancelled in chat route), and add extra logging
* chore(typedefs): add httpAgent type to OpenAIClient
* refactor: use custom implementation of retrieving run with axios to allow for timing out run query
* fix(waitForRun): handle timed out run retrieval query
* refactor: update preset conditions:
- presets will retain settings when a different endpoint is selected; for existing convos, either when modular or is assistant switch
- no longer use `navigateToConvo` on preset select
* fix: temporary calculator hack as expects string input when invoked
* fix: cancel abortRun only when cancelling error is a result of the run already being cancelled
* chore: remove use of `fileMaxSizeMB` and total counterpart (redundant)
* docs: custom config documentation update
* docs: assistants api setup and dotenv, new custom config fields
* refactor(Switcher): make Assistant switcher sticky in SidePanel
* chore(useSSE): remove console log of data and message index
* refactor(AssistantPanel): button styling and add secondary select button to bottom of panel
* refactor(OpenAIClient): allow passing conversationId to RunManager through titleConvo and initializeLLM to properly record title context tokens used in cases where conversationId was not defined by the client
* feat(assistants): token tracking for assistant runs
* chore(spendTokens): improve logging
* feat: support/exclude specific assistant Ids
* chore: add update `librechat.example.yaml`, optimize `AppService` handling, new tests for `AppService`, optimize missing/outdate config logging
* chore: mount docker logs to root of project
* chore: condense axios errors
* chore: bump vite
* chore: vite hot reload fix using latest version
* chore(getOpenAIModels): sort instruct models to the end of models list
* fix(assistants): user provided key
* fix(assistants): user provided key, invalidate more queries on revoke
---------
Co-authored-by: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
2024-02-13 20:42:27 -05:00
|
|
|
return multer({
|
|
|
|
storage,
|
|
|
|
fileFilter,
|
|
|
|
limits: { fileSize: fileConfig.serverFileSizeLimit },
|
|
|
|
});
|
|
|
|
};
|
feat: Vision Support + New UI (#1203)
* feat: add timer duration to showToast, show toast for preset selection
* refactor: replace old /chat/ route with /c/. e2e tests will fail here
* refactor: move typedefs to root of /api/ and add a few to assistant types in TS
* refactor: reorganize data-provider imports, fix dependency cycle, strategize new plan to separate react dependent packages
* feat: add dataService for uploading images
* feat(data-provider): add mutation keys
* feat: file resizing and upload
* WIP: initial API image handling
* fix: catch JSON.parse of localStorage tools
* chore: experimental: use module-alias for absolute imports
* refactor: change temp_file_id strategy
* fix: updating files state by using Map and defining react query callbacks in a way that keeps them during component unmount, initial delete handling
* feat: properly handle file deletion
* refactor: unexpose complete filepath and resize from server for higher fidelity
* fix: make sure resized height, width is saved, catch bad requests
* refactor: use absolute imports
* fix: prevent setOptions from being called more than once for OpenAIClient, made note to fix for PluginsClient
* refactor: import supportsFiles and models vars from schemas
* fix: correctly replace temp file id
* refactor(BaseClient): use absolute imports, pass message 'opts' to buildMessages method, count tokens for nested objects/arrays
* feat: add validateVisionModel to determine if model has vision capabilities
* chore(checkBalance): update jsdoc
* feat: formatVisionMessage: change message content format dependent on role and image_urls passed
* refactor: add usage to File schema, make create and updateFile, correctly set and remove TTL
* feat: working vision support
TODO: file size, type, amount validations, making sure they are styled right, and making sure you can add images from the clipboard/dragging
* feat: clipboard support for uploading images
* feat: handle files on drop to screen, refactor top level view code to Presentation component so the useDragHelpers hook has ChatContext
* fix(Images): replace uploaded images in place
* feat: add filepath validation to protect sensitive files
* fix: ensure correct file_ids are push and not the Map key values
* fix(ToastContext): type issue
* feat: add basic file validation
* fix(useDragHelpers): correct context issue with `files` dependency
* refactor: consolidate setErrors logic to setError
* feat: add dialog Image overlay on image click
* fix: close endpoints menu on click
* chore: set detail to auto, make note for configuration
* fix: react warning (button desc. of button)
* refactor: optimize filepath handling, pass file_ids to images for easier re-use
* refactor: optimize image file handling, allow re-using files in regen, pass more file metadata in messages
* feat: lazy loading images including use of upload preview
* fix: SetKeyDialog closing, stopPropagation on Dialog content click
* style(EndpointMenuItem): tighten up the style, fix dark theme showing in lightmode, make menu more ux friendly
* style: change maxheight of all settings textareas to 138px from 300px
* style: better styling for textarea and enclosing buttons
* refactor(PresetItems): swap back edit and delete icons
* feat: make textarea placeholder dynamic to endpoint
* style: show user hover buttons only on hover when message is streaming
* fix: ordered list not going past 9, fix css
* feat: add User/AI labels; style: hide loading spinner
* feat: add back custom footer, change original footer text
* feat: dynamic landing icons based on endpoint
* chore: comment out assistants route
* fix: autoScroll to newest on /c/ view
* fix: Export Conversation on new UI
* style: match message style of official more closely
* ci: fix api jest unit tests, comment out e2e tests for now as they will fail until addressed
* feat: more file validation and use blob in preview field, not filepath, to fix temp deletion
* feat: filefilter for multer
* feat: better AI labels based on custom name, model, and endpoint instead of `ChatGPT`
2023-11-21 20:12:48 -05:00
|
|
|
|
2024-05-02 08:48:26 +02:00
|
|
|
module.exports = { createMulterInstance, storage, importFileFilter };
|