mirror of
https://github.com/danny-avila/LibreChat.git
synced 2025-12-21 02:40:14 +01:00
✨ feat: Assistants API, General File Support, Side Panel, File Explorer (#1696)
* feat: assistant name/icon in Landing & Header * feat: assistname in textarea placeholder, and use `Assistant` as default name * feat: display non-image files in user messages * fix: only render files if files.length is > 0 * refactor(config -> file-config): move file related configuration values to separate module, add excel types * chore: spreadsheet file rendering * fix(Landing): dark mode style for Assistant Name * refactor: move progress incrementing to own hook, start smaller, cap near limit \(1\) * refactor(useContentHandler): add empty Text part if last part was completed tool or image * chore: add accordion trigger border styling for dark mode * feat: Assistant Builder model selection * chore: use Spinner when Assistant is mutating * fix(get/assistants): return correct response object `AssistantListResponse` * refactor(Spinner): pass size as prop * refactor: make assistant crud mutations optimistic, add types for options * chore: remove assistants route and view * chore: move assistant builder components to separate directory * feat(ContextButton): delete Assistant via context button/dialog, add localization * refactor: conditionally show use and context menu buttons, add localization for create assistant * feat: save side panel states to localStorage * style(SidePanel): improve avatar menu and assistant select styling for dark mode * refactor: make NavToggle reusable for either side (left or right), add SidePanel Toggle with ability to close it completely * fix: resize handle and navToggle behavior * fix(/avatar/:assistant_id): await `deleteFile` and assign unique name to uploaded image * WIP: file UI components from PR #576 * refactor(OpenAIMinimalIcon): pass className * feat: formatDate helper fn * feat: DataTableColumnHeader * feat: add row selection, formatted row values, number of rows selected * WIP: add files to Side panel temporarily * feat: `LB_QueueAsyncCall`: Leaky Bucket queue for external APIs, use in `processDeleteRequest` * fix(TFile): correct `source` type with `FileSources` * fix(useFileHandling): use `continue` instead of return when iterating multiple files, add file type to extendedFile * chore: add generic setter type * refactor(processDeleteRequest): settle promises to prevent rejections from processing deletions, log errors * feat: `useFileDeletion` to reuse file deletion logic * refactor(useFileDeletion): make `setFiles` an optional param and use object as param * feat: useDeleteFilesFromTable * feat: use real `files` data and add deletion action to data table * fix(Table): make headers sticky * feat: add dynamic filtering for columns; only show to user Host or OpenAI storage type * style(DropdownMenu): replace `slate` with `gray` * style(DataTable): apply dark mode themes and other misc styling * style(Columns): add color to OpenAI Storage option * refactor(FileContainer): make file preview reusable * refactor(Images): make image preview reusable * refactor(FilePreview): make file prop optional for FileIcon and FilePreview, fix relative style * feat(Columns): add file/image previews, set a minimum size to show for file size in bytes * WIP: File Panel with real files and formatted * feat: open files dialog from panel * style: file data table mobile and general column styling fixes * refactor(api/files): return files sorted by the most recently updated * refactor: provide fileMap through context to prevent re-selecting files to map in different areas; remove unused imports commented out in PanelColumns * refactor(ExtendFile): make File type optional, add `attached` to prevent attached files from being deleted on remove, make Message.files a partial TFile type * feat: attach files through file panel * refactor(useFileHandling): move files to the start of cache list when uploaded * refactor(useDeleteFilesMutation): delete files from cache when successfully deleted from server * fix(FileRow): handle possible edge case of duplication due to attaching recently uploaded file * style(SidePanel): make resize grip border transparent, remove unnecessary styling on close sidepanel button * feat: action utilities and tests * refactor(actions): add `ValidationResult` type and change wording for no server URL found * refactor(actions): check for empty server URL * fix(data-provider): revert tsconfig to fix type issue resolution * feat(client): first pass of actions input for assistants * refactor(FunctionSignature): change method to output object instead of string * refactor(models/Assistant): add actions field to schema, use searchParams object for methods, and add `getAssistant` * feat: post actions input first pass - create new Action document - add actions to Assistant DB document - create /action/:assistant_id POST route - pass more props down from PanelSwitcher, derive assistant_id from switcher - move privacy policy to ActionInput - reset data on input change/validation - add `useUpdateAction` - conform FunctionSignature type to FunctionTool - add action, assistant doc, update hook related types * refactor: optimize assistant/actions relationship - past domain in metadata as hostname and not a URL - include domain in tool name - add `getActions` for actions retrieval by user - add `getAssistants` for assistant docs retrieval by user - add `assistant_id` to Action schema - move actions to own module as a subroute to `api/assistants` - add `useGetActionsQuery` and `useGetAssistantDocsQuery` hooks - fix Action type def * feat: show assistant actions in assistant builder * feat: switch to actions on action click, editing action styling * fix: add Assistant state for builder panel to allow immediate selection of newly created assistants as well as retaining the current assistant when switching to a different panel within the builder * refactor(SidePanel/NavToggle): offset less from right when SidePanel is completely collapsed * chore: rename `processActions` -> `processRequiredActions` * chore: rename Assistant API Action to RequiredAction * refactor(actions): avoid nesting actual API params under generic `requestBody` to optimize LLM token usage * fix(handleTools): avoid calling `validTool` if not defined, add optional param to skip the loading of specs, which throws an error in the context of assistants * WIP: working first pass of toolCalls generated from openapi specs * WIP: first pass ToolCall styling * feat: programmatic iv encryption/decryption helpers * fix: correct ActionAuth types/enums, and define type for AuthForm * feat: encryption/decryption helpers for Action AuthMetadata * refactor(getActions): remove sensitive fields from query response * refactor(POST/actions): encrypt and remove sensitive fields from mutation response * fix(ActionService): change ESM import to CJS * feat: frontend auth handling for actions + optimistic update on action update/creation * refactor(actions): use the correct variables and types for setAuth method * refactor: POST /:assistant_id action can now handle updating an existing action, add `saved_auth_fields` to determine when user explicitly saves new auth creds. only send auth metadata if user explicitly saved fields * refactor(createActionTool): catch errors and send back meaningful error message, add flag to `getActions` to determine whether to retrieve sensitive values or not * refactor(ToolService): add `action` property to ToolCall PartMetadata to determine if the tool call was an action, fix parsing function name issue with actionDelimiter * fix(ActionRequest): use URL class to correctly join endpoint parts for `execute` call * feat: delete assistant actions * refactor: conditionally show Available actions * refactor: show `retrieval` and `code_interpreter` as Capabilities, swap `Switch` for `Checkbox` * chore: remove shadow-stroke from messages * WIP: first pass of Assistants Knowledge attachments * refactor: remove AssistantsProvider in favor of FormProvider, fix selectedAssistant re-render bug, map Assistant file_ids to files via fileMap, initialize Knowledge component with mapped files if any exist * fix: prevent deleting files on assistant file upload * chore: remove console.log * refactor(useUploadFileMutation): update files and assistants cache on upload * chore: disable oauth option as not supported yet * feat: cancel assistant runs * refactor: initialize OpenAI client with helper function, resolve all related circular dependencies * fix(DALL-E): initialization * fix(process): openai client initialization * fix: select an existing Assistant when the active one is deleted * chore: allow attaching files for assistant endpoint, send back relevant OpenAI error message when uploading, deconstruct openAI initialization correctly, add `message_file` to formData when a file is attached to the message but not the assistant * fix: add assistant_id on newConvo * fix(initializeClient): import fix * chore: swap setAssistant for setOption in useEffect * fix(DALL-E): add processFileURL to loadTools call * chore: add customConfig to debug logs * feat: delete threads on convo delete * chore: replace Assistants icon * chore: remove console.dir() in `abortRun` * feat(AssistantService): accumulate text values from run in openai.responseText * feat: titling for assistants endpoint * chore: move panel file components to appropriate directory, add file checks for attaching files, change icon for Attach Files * refactor: add localizations to tools, plugins, add condition for adding/remove user plugins so tool selections don't affect this value * chore: disable `import from url` action for now * chore: remove textMimeTypes from default fileConfig for now * fix: catch tool errors and send as outputs with error messages * fix: React warning about button as descendant of button * style: retrieval and cancelled icon * WIP: pass isSubmitting to Parts, use InProgressCall to display cancelled tool calls correctly, show domain/function name * fix(meilisearch): fix `postSaveHook` issue where indexing expects a mongo document, and join all text content parts for meili indexing * ci: fix dall-e tests * ci: fix client tests * fix: button types in actions panel * fix: plugin auth form persisting across tool selections * fix(ci): update AppService spec with `loadAndFormatTools` * fix(clearConvos): add id check earlier on * refactor(AssistantAvatar): set previewURL dynamically when emtadata.avatar changes * feat(assistants): addTitle cache setting * fix(useSSE): resolve rebase conflicts * fix: delete mutation * style(SidePanel): make grip visible on active and hover, invisible otherwise * ci: add data-provider tests to workflow, also update eslint/tsconfig to recognize specs, and add `text/csv` to fileConfig * fix: handle edge case where auth object is undefined, and log errors * refactor(actions): resolve schemas, add tests for resolving refs, import specs from separate file for tests * chore: remove comment * fix(ActionsInput): re-render bug when initializing states with action fields * fix(patch/assistant): filter undefined tools * chore: add logging for errors in assistants routes * fix(updateAssistant): map actions to functions to avoid overwriting * fix(actions): properly handle GET paths * fix(convos): unhandled delete thread exception * refactor(AssistantService): pass both thread_id and conversationId when sending intermediate assistant messages, remove `mapMessagesToSteps` from AssistantService * refactor(useSSE): replace all messages with runMessages and pass latestMessageId to abortRun; fix(checkMessageGaps): include tool calls when syncing messages * refactor(assistants/chat): invoke `createOnTextProgress` after thread creation * chore: add typing * style: sidepanel styling * style: action tool call domain styling * feat(assistants): default models, limit retrieval to certain models, add env variables to to env.example * feat: assistants api key in EndpointService * refactor: set assistant model to conversation on assistant switch * refactor: set assistant model to conversation on assistant select from panel * fix(retrieveAndProcessFile): catch attempt to download file with `assistant` purpose which is not allowed; add logging * feat: retrieval styling, handling, and logging * chore: rename ASSISTANTS_REVERSE_PROXY to ASSISTANTS_BASE_URL * feat: FileContext for file metadata * feat: context file mgmt and filtering * style(Select): hover/rounded changes * refactor: explicit conversation switch, endpoint dependent, through `useSelectAssistant`, which does not create new chat if current endpoint is assistant endpoint * fix(AssistantAvatar): make empty previewURL if no avatar present * refactor: side panel mobile styling * style: merge tool and action section, optimize mobile styling for action/tool buttons * fix: localStorage issues * fix(useSelectAssistant): invoke react query hook directly in select hook as Map was not being updated in time * style: light mode fixes * fix: prevent sidepanel nav styling from shifting layout up * refactor: change default layout (collapsed by default) * style: mobile optimization of DataTable * style: datatable * feat: client-side hide right-side panel * chore(useNewConvo): add partial typing for preset * fix(useSelectAssistant): pass correct model name by using template as preset * WIP: assistant presets * refactor(ToolService): add native solution for `TavilySearchResults` and log tool output errors * refactor: organize imports and use native TavilySearchResults * fix(TavilySearchResults): stringify result * fix(ToolCall): show tool call outputs when not an action * chore: rename Prompt Prefix to custom instructions (in user facing text only) * refactor(EditPresetDialog): Optimize setting title by debouncing, reset preset on dialog close to avoid state mixture * feat: add `presetOverride` to overwrite active conversation settings when saving a Preset (relevant for client side updates only) * feat: Assistant preset settings (client-side) * fix(Switcher): only set assistant_id and model if current endpoint is Assistants * feat: use `useDebouncedInput` for updating conversation settings, starting with EditPresetDialog title setting and Assistant instructions setting * feat(Assistants): add instructions field to settings * feat(chat/assistants): pass conversation settings to run body * wip: begin localization and only allow actions if the assistant is created * refactor(AssistantsPanel): knowledge localization, allow tools on creation * feat: experimental: allow 'priming' values before assistant is created, that would normally require an assistant_id to be defined * chore: trim console logs and make more meaningful * chore: toast messages * fix(ci): date test * feat: create file when uploading Assistant Avatar * feat: file upload rate limiting from custom config with dynamic file route initialization * refactor: use file upload limiters on post routes only * refactor(fileConfig): add endpoints field for endpoint specific fileconfigs, add mergeConfig function, add tests * refactor: fileConfig route, dynamic multer instances used on all '/' and '/images' POST routes, data service and query hook * feat: supportedMimeTypesSchema, test for array of regex * feat: configurable file config limits * chore: clarify assistants file knowledge prereq. * chore(useTextarea): default to localized 'Assistant' if assistant name is empty * feat: configurable file limits and toggle file upload per endpoint * fix(useUploadFileMutation): prevent updating assistant.files cache if file upload is a message_file attachment * fix(AssistantSelect): set last selected assistant only when timeout successfully runs * refactor(queries): disable assistant queries if assistants endpoint is not enabled * chore(Switcher): add localization * chore: pluralize `assistant` for `EModelEndpoint key and value * feat: show/hide assistant UI components based on endpoint availability; librechat.yaml config for disabling builder section and setting polling/timeout intervals * fix(compactEndpointSchemas): use EModelEndpoint for schema access * feat(runAssistant): use configured values from `librechat.yaml` for `pollIntervalMs` and `timeout` * fix: naming issue * wip: revert landing * 🎉 happy birthday LibreChat (#1768) * happy birthday LibreChat * Refactor endpoint condition in Landing component * Update birthday message in Eng.tsx * fix(/config): avoid nesting ternaries * refactor(/config): check birthday --------- Co-authored-by: Danny Avila <messagedaniel@protonmail.com> * fix: landing * fix: landing * fix(useMessageHelpers): hardcoded check to use EModelEndpoint instead * fix(ci): convo test revert to main * fix(assistants/chat): fix issue where assistant_id was being saved as model for convo * chore: added logging, promises racing to prevent longer timeouts, explicit setting of maxRetries and timeouts, robust catching of invalid abortRun params * refactor: use recoil state for `showStopButton` and only show for assistants endpoint after syncing conversation data * refactor: optimize abortRun strategy using localStorage, refactor `abortConversation` to use async/await and await the result, refactor how the abortKey cache is set for runs * fix(checkMessageGaps): assign `assistant_id` to synced messages if defined; prevents UI from showing blank assistant for cancelled messages * refactor: re-order sequence of chat route, only allow aborting messages after run is created, cancel abortRun if there was a cancelling error (likely due already cancelled in chat route), and add extra logging * chore(typedefs): add httpAgent type to OpenAIClient * refactor: use custom implementation of retrieving run with axios to allow for timing out run query * fix(waitForRun): handle timed out run retrieval query * refactor: update preset conditions: - presets will retain settings when a different endpoint is selected; for existing convos, either when modular or is assistant switch - no longer use `navigateToConvo` on preset select * fix: temporary calculator hack as expects string input when invoked * fix: cancel abortRun only when cancelling error is a result of the run already being cancelled * chore: remove use of `fileMaxSizeMB` and total counterpart (redundant) * docs: custom config documentation update * docs: assistants api setup and dotenv, new custom config fields * refactor(Switcher): make Assistant switcher sticky in SidePanel * chore(useSSE): remove console log of data and message index * refactor(AssistantPanel): button styling and add secondary select button to bottom of panel * refactor(OpenAIClient): allow passing conversationId to RunManager through titleConvo and initializeLLM to properly record title context tokens used in cases where conversationId was not defined by the client * feat(assistants): token tracking for assistant runs * chore(spendTokens): improve logging * feat: support/exclude specific assistant Ids * chore: add update `librechat.example.yaml`, optimize `AppService` handling, new tests for `AppService`, optimize missing/outdate config logging * chore: mount docker logs to root of project * chore: condense axios errors * chore: bump vite * chore: vite hot reload fix using latest version * chore(getOpenAIModels): sort instruct models to the end of models list * fix(assistants): user provided key * fix(assistants): user provided key, invalidate more queries on revoke --------- Co-authored-by: Marco Beretta <81851188+Berry-13@users.noreply.github.com>
This commit is contained in:
Danny Avila
GitHub
parent
cd2786441a
commit
ecd63eb9f1
316 changed files with 21873 additions and 6315 deletions
|
|
@ -1,5 +1,6 @@
|
|||
const fetch = require('node-fetch');
|
||||
const { ref, uploadBytes, getDownloadURL, deleteObject } = require('firebase/storage');
|
||||
const { getBufferMetadata } = require('~/server/utils');
|
||||
const { getFirebaseStorage } = require('./initialize');
|
||||
|
||||
/**
|
||||
|
|
@ -41,9 +42,8 @@ async function deleteFile(basePath, fileName) {
|
|||
* @param {string} [params.basePath='images'] - Optional. The base basePath in Firebase Storage where the file will
|
||||
* be stored. Defaults to 'images' if not specified.
|
||||
*
|
||||
* @returns {Promise<string|null>}
|
||||
* A promise that resolves to the file name if the file is successfully uploaded, or null if there
|
||||
* is an error in initialization or upload.
|
||||
* @returns {Promise<{ bytes: number, type: string, dimensions: Record<string, number>} | null>}
|
||||
* A promise that resolves to the file metadata if the file is successfully saved, or null if there is an error.
|
||||
*/
|
||||
async function saveURLToFirebase({ userId, URL, fileName, basePath = 'images' }) {
|
||||
const storage = getFirebaseStorage();
|
||||
|
|
@ -53,10 +53,12 @@ async function saveURLToFirebase({ userId, URL, fileName, basePath = 'images' })
|
|||
}
|
||||
|
||||
const storageRef = ref(storage, `${basePath}/${userId.toString()}/${fileName}`);
|
||||
const response = await fetch(URL);
|
||||
const buffer = await response.buffer();
|
||||
|
||||
try {
|
||||
await uploadBytes(storageRef, await fetch(URL).then((response) => response.buffer()));
|
||||
return fileName;
|
||||
await uploadBytes(storageRef, buffer);
|
||||
return await getBufferMetadata(buffer);
|
||||
} catch (error) {
|
||||
console.error('Error uploading file to Firebase Storage:', error.message);
|
||||
return null;
|
||||
|
|
|
|||
|
|
@ -1,7 +1,8 @@
|
|||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const sharp = require('sharp');
|
||||
const { resizeImage } = require('../images/resize');
|
||||
const { resizeImageBuffer } = require('../images/resize');
|
||||
const { updateUser } = require('~/models/userMethods');
|
||||
const { saveBufferToFirebase } = require('./crud');
|
||||
const { updateFile } = require('~/models/File');
|
||||
const { logger } = require('~/config');
|
||||
|
|
@ -11,7 +12,7 @@ const { logger } = require('~/config');
|
|||
* resolution.
|
||||
*
|
||||
*
|
||||
* @param {Object} req - The request object from Express. It should have a `user` property with an `id`
|
||||
* @param {Express.Request} req - The request object from Express. It should have a `user` property with an `id`
|
||||
* representing the user, and an `app.locals.paths` object with an `imageOutput` path.
|
||||
* @param {Express.Multer.File} file - The file object, which is part of the request. The file object should
|
||||
* have a `path` property that points to the location of the uploaded file.
|
||||
|
|
@ -26,7 +27,8 @@ const { logger } = require('~/config');
|
|||
*/
|
||||
async function uploadImageToFirebase(req, file, resolution = 'high') {
|
||||
const inputFilePath = file.path;
|
||||
const { buffer: resizedBuffer, width, height } = await resizeImage(inputFilePath, resolution);
|
||||
const inputBuffer = await fs.promises.readFile(inputFilePath);
|
||||
const { buffer: resizedBuffer, width, height } = await resizeImageBuffer(inputBuffer, resolution);
|
||||
const extension = path.extname(inputFilePath);
|
||||
const userId = req.user.id;
|
||||
|
||||
|
|
@ -73,15 +75,15 @@ async function prepareImageURL(req, file) {
|
|||
*
|
||||
* @param {object} params - The parameters object.
|
||||
* @param {Buffer} params.buffer - The Buffer containing the avatar image in WebP format.
|
||||
* @param {object} params.User - The User document (mongoose); TODO: remove direct use of Model, `User`
|
||||
* @param {string} params.userId - The user ID.
|
||||
* @param {string} params.manual - A string flag indicating whether the update is manual ('true' or 'false').
|
||||
* @returns {Promise<string>} - A promise that resolves with the URL of the uploaded avatar.
|
||||
* @throws {Error} - Throws an error if Firebase is not initialized or if there is an error in uploading.
|
||||
*/
|
||||
async function processFirebaseAvatar({ buffer, User, manual }) {
|
||||
async function processFirebaseAvatar({ buffer, userId, manual }) {
|
||||
try {
|
||||
const downloadURL = await saveBufferToFirebase({
|
||||
userId: User._id.toString(),
|
||||
userId,
|
||||
buffer,
|
||||
fileName: 'avatar.png',
|
||||
});
|
||||
|
|
@ -91,8 +93,7 @@ async function processFirebaseAvatar({ buffer, User, manual }) {
|
|||
const url = `${downloadURL}?manual=${isManual}`;
|
||||
|
||||
if (isManual) {
|
||||
User.avatar = url;
|
||||
await User.save();
|
||||
await updateUser(userId, { avatar: url });
|
||||
}
|
||||
|
||||
return url;
|
||||
|
|
|
|||
|
|
@ -1,8 +1,9 @@
|
|||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const axios = require('axios');
|
||||
const { logger } = require('~/config');
|
||||
const { getBufferMetadata } = require('~/server/utils');
|
||||
const paths = require('~/config/paths');
|
||||
const { logger } = require('~/config');
|
||||
|
||||
/**
|
||||
* Saves a file to a specified output path with a new filename.
|
||||
|
|
@ -13,7 +14,7 @@ const paths = require('~/config/paths');
|
|||
* @returns {Promise<string>} The full path of the saved file.
|
||||
* @throws Will throw an error if the file saving process fails.
|
||||
*/
|
||||
async function saveFile(file, outputPath, outputFilename) {
|
||||
async function saveLocalFile(file, outputPath, outputFilename) {
|
||||
try {
|
||||
if (!fs.existsSync(outputPath)) {
|
||||
fs.mkdirSync(outputPath, { recursive: true });
|
||||
|
|
@ -44,9 +45,41 @@ async function saveFile(file, outputPath, outputFilename) {
|
|||
const saveLocalImage = async (req, file, filename) => {
|
||||
const imagePath = req.app.locals.paths.imageOutput;
|
||||
const outputPath = path.join(imagePath, req.user.id ?? '');
|
||||
await saveFile(file, outputPath, filename);
|
||||
await saveLocalFile(file, outputPath, filename);
|
||||
};
|
||||
|
||||
/**
|
||||
* Saves a buffer to a specified directory on the local file system.
|
||||
*
|
||||
* @param {Object} params - The parameters object.
|
||||
* @param {string} params.userId - The user's unique identifier. This is used to create a user-specific directory.
|
||||
* @param {Buffer} params.buffer - The buffer to be saved.
|
||||
* @param {string} params.fileName - The name of the file to be saved.
|
||||
* @param {string} [params.basePath='images'] - Optional. The base path where the file will be stored.
|
||||
* Defaults to 'images' if not specified.
|
||||
* @returns {Promise<string>} - A promise that resolves to the path of the saved file.
|
||||
*/
|
||||
async function saveLocalBuffer({ userId, buffer, fileName, basePath = 'images' }) {
|
||||
try {
|
||||
const { publicPath, uploads } = paths;
|
||||
|
||||
const directoryPath = path.join(basePath === 'images' ? publicPath : uploads, basePath, userId);
|
||||
|
||||
if (!fs.existsSync(directoryPath)) {
|
||||
fs.mkdirSync(directoryPath, { recursive: true });
|
||||
}
|
||||
|
||||
fs.writeFileSync(path.join(directoryPath, fileName), buffer);
|
||||
|
||||
const filePath = path.posix.join('/', basePath, userId, fileName);
|
||||
|
||||
return filePath;
|
||||
} catch (error) {
|
||||
logger.error('[saveLocalBuffer] Error while saving the buffer:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Saves a file from a given URL to a local directory. The function fetches the file using the provided URL,
|
||||
* determines the content type, and saves it to a specified local directory with the correct file extension.
|
||||
|
|
@ -62,20 +95,18 @@ const saveLocalImage = async (req, file, filename) => {
|
|||
* @param {string} [params.basePath='images'] - Optional. The base directory where the file will be saved.
|
||||
* Defaults to 'images' if not specified.
|
||||
*
|
||||
* @returns {Promise<string|null>}
|
||||
* A promise that resolves to the file name if the file is successfully saved, or null if there is an error.
|
||||
* @returns {Promise<{ bytes: number, type: string, dimensions: Record<string, number>} | null>}
|
||||
* A promise that resolves to the file metadata if the file is successfully saved, or null if there is an error.
|
||||
*/
|
||||
async function saveFileFromURL({ userId, URL, fileName, basePath = 'images' }) {
|
||||
try {
|
||||
// Fetch the file from the URL
|
||||
const response = await axios({
|
||||
url: URL,
|
||||
responseType: 'stream',
|
||||
responseType: 'arraybuffer',
|
||||
});
|
||||
|
||||
// Get the content type from the response headers
|
||||
const contentType = response.headers['content-type'];
|
||||
let extension = contentType.split('/').pop();
|
||||
const buffer = Buffer.from(response.data, 'binary');
|
||||
const { bytes, type, dimensions, extension } = await getBufferMetadata(buffer);
|
||||
|
||||
// Construct the outputPath based on the basePath and userId
|
||||
const outputPath = path.join(paths.publicPath, basePath, userId.toString());
|
||||
|
|
@ -92,17 +123,15 @@ async function saveFileFromURL({ userId, URL, fileName, basePath = 'images' }) {
|
|||
fileName += `.${extension}`;
|
||||
}
|
||||
|
||||
// Create a writable stream for the output path
|
||||
const outputFilePath = path.join(outputPath, path.basename(fileName));
|
||||
const writer = fs.createWriteStream(outputFilePath);
|
||||
// Save the file to the output path
|
||||
const outputFilePath = path.join(outputPath, fileName);
|
||||
fs.writeFileSync(outputFilePath, buffer);
|
||||
|
||||
// Pipe the response data to the output file
|
||||
response.data.pipe(writer);
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
writer.on('finish', () => resolve(fileName));
|
||||
writer.on('error', reject);
|
||||
});
|
||||
return {
|
||||
bytes,
|
||||
type,
|
||||
dimensions,
|
||||
};
|
||||
} catch (error) {
|
||||
logger.error('[saveFileFromURL] Error while saving the file:', error);
|
||||
return null;
|
||||
|
|
@ -171,4 +200,11 @@ const deleteLocalFile = async (req, file) => {
|
|||
await fs.promises.unlink(filepath);
|
||||
};
|
||||
|
||||
module.exports = { saveFile, saveLocalImage, saveFileFromURL, getLocalFileURL, deleteLocalFile };
|
||||
module.exports = {
|
||||
saveLocalFile,
|
||||
saveLocalImage,
|
||||
saveLocalBuffer,
|
||||
saveFileFromURL,
|
||||
getLocalFileURL,
|
||||
deleteLocalFile,
|
||||
};
|
||||
|
|
|
|||
|
|
@ -1,7 +1,8 @@
|
|||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const sharp = require('sharp');
|
||||
const { resizeImage } = require('../images/resize');
|
||||
const { resizeImageBuffer } = require('../images/resize');
|
||||
const { updateUser } = require('~/models/userMethods');
|
||||
const { updateFile } = require('~/models/File');
|
||||
|
||||
/**
|
||||
|
|
@ -28,7 +29,8 @@ const { updateFile } = require('~/models/File');
|
|||
*/
|
||||
async function uploadLocalImage(req, file, resolution = 'high') {
|
||||
const inputFilePath = file.path;
|
||||
const { buffer: resizedBuffer, width, height } = await resizeImage(inputFilePath, resolution);
|
||||
const inputBuffer = await fs.promises.readFile(inputFilePath);
|
||||
const { buffer: resizedBuffer, width, height } = await resizeImageBuffer(inputBuffer, resolution);
|
||||
const extension = path.extname(inputFilePath);
|
||||
|
||||
const { imageOutput } = req.app.locals.paths;
|
||||
|
|
@ -96,17 +98,17 @@ async function prepareImagesLocal(req, file) {
|
|||
}
|
||||
|
||||
/**
|
||||
* Uploads a user's avatar to Firebase Storage and returns the URL.
|
||||
* Uploads a user's avatar to local server storage and returns the URL.
|
||||
* If the 'manual' flag is set to 'true', it also updates the user's avatar URL in the database.
|
||||
*
|
||||
* @param {object} params - The parameters object.
|
||||
* @param {Buffer} params.buffer - The Buffer containing the avatar image in WebP format.
|
||||
* @param {object} params.User - The User document (mongoose); TODO: remove direct use of Model, `User`
|
||||
* @param {string} params.userId - The user ID.
|
||||
* @param {string} params.manual - A string flag indicating whether the update is manual ('true' or 'false').
|
||||
* @returns {Promise<string>} - A promise that resolves with the URL of the uploaded avatar.
|
||||
* @throws {Error} - Throws an error if Firebase is not initialized or if there is an error in uploading.
|
||||
*/
|
||||
async function processLocalAvatar({ buffer, User, manual }) {
|
||||
async function processLocalAvatar({ buffer, userId, manual }) {
|
||||
const userDir = path.resolve(
|
||||
__dirname,
|
||||
'..',
|
||||
|
|
@ -117,10 +119,11 @@ async function processLocalAvatar({ buffer, User, manual }) {
|
|||
'client',
|
||||
'public',
|
||||
'images',
|
||||
User._id.toString(),
|
||||
userId,
|
||||
);
|
||||
|
||||
const fileName = `avatar-${new Date().getTime()}.png`;
|
||||
const urlRoute = `/images/${User._id.toString()}/${fileName}`;
|
||||
const urlRoute = `/images/${userId}/${fileName}`;
|
||||
const avatarPath = path.join(userDir, fileName);
|
||||
|
||||
await fs.promises.mkdir(userDir, { recursive: true });
|
||||
|
|
@ -130,8 +133,7 @@ async function processLocalAvatar({ buffer, User, manual }) {
|
|||
let url = `${urlRoute}?manual=${isManual}`;
|
||||
|
||||
if (isManual) {
|
||||
User.avatar = url;
|
||||
await User.save();
|
||||
await updateUser(userId, { avatar: url });
|
||||
}
|
||||
|
||||
return url;
|
||||
|
|
|
|||
49
api/server/services/Files/OpenAI/crud.js
Normal file
49
api/server/services/Files/OpenAI/crud.js
Normal file
|
|
@ -0,0 +1,49 @@
|
|||
const fs = require('fs');
|
||||
|
||||
/**
|
||||
* Uploads a file that can be used across various OpenAI services.
|
||||
*
|
||||
* @param {Express.Request} req - The request object from Express. It should have a `user` property with an `id`
|
||||
* representing the user, and an `app.locals.paths` object with an `imageOutput` path.
|
||||
* @param {Express.Multer.File} file - The file uploaded to the server via multer.
|
||||
* @param {OpenAI} openai - The initialized OpenAI client.
|
||||
* @returns {Promise<OpenAIFile>}
|
||||
*/
|
||||
async function uploadOpenAIFile(req, file, openai) {
|
||||
try {
|
||||
const uploadedFile = await openai.files.create({
|
||||
file: fs.createReadStream(file.path),
|
||||
purpose: 'assistants',
|
||||
});
|
||||
|
||||
console.log('File uploaded successfully to OpenAI');
|
||||
|
||||
return uploadedFile;
|
||||
} catch (error) {
|
||||
console.error('Error uploading file to OpenAI:', error.message);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Deletes a file previously uploaded to OpenAI.
|
||||
*
|
||||
* @param {Express.Request} req - The request object from Express.
|
||||
* @param {MongoFile} file - The database representation of the uploaded file.
|
||||
* @param {OpenAI} openai - The initialized OpenAI client.
|
||||
* @returns {Promise<void>}
|
||||
*/
|
||||
async function deleteOpenAIFile(req, file, openai) {
|
||||
try {
|
||||
const res = await openai.files.del(file.file_id);
|
||||
if (!res.deleted) {
|
||||
throw new Error('OpenAI returned `false` for deleted status');
|
||||
}
|
||||
console.log('File deleted successfully from OpenAI');
|
||||
} catch (error) {
|
||||
console.error('Error deleting file from OpenAI:', error.message);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { uploadOpenAIFile, deleteOpenAIFile };
|
||||
5
api/server/services/Files/OpenAI/index.js
Normal file
5
api/server/services/Files/OpenAI/index.js
Normal file
|
|
@ -0,0 +1,5 @@
|
|||
const crud = require('./crud');
|
||||
|
||||
module.exports = {
|
||||
...crud,
|
||||
};
|
||||
|
|
@ -1,42 +1,29 @@
|
|||
const sharp = require('sharp');
|
||||
const fs = require('fs').promises;
|
||||
const fetch = require('node-fetch');
|
||||
const User = require('~/models/User');
|
||||
const { getStrategyFunctions } = require('~/server/services/Files/strategies');
|
||||
const { resizeAndConvert } = require('./resize');
|
||||
const { logger } = require('~/config');
|
||||
|
||||
async function convertToWebP(inputBuffer) {
|
||||
return sharp(inputBuffer).resize({ width: 150 }).toFormat('webp').toBuffer();
|
||||
}
|
||||
|
||||
/**
|
||||
* Uploads an avatar image for a user. This function can handle various types of input (URL, Buffer, or File object),
|
||||
* processes the image to a square format, converts it to WebP format, and then uses a specified file strategy for
|
||||
* further processing. It performs validation on the user ID and the input type. The function can throw errors for
|
||||
* invalid input types, fetching issues, or other processing errors.
|
||||
* processes the image to a square format, converts it to WebP format, and returns the resized buffer.
|
||||
*
|
||||
* @param {Object} params - The parameters object.
|
||||
* @param {string} params.userId - The unique identifier of the user for whom the avatar is being uploaded.
|
||||
* @param {FileSources} params.fileStrategy - The file handling strategy to use, determining how the avatar is processed.
|
||||
* @param {(string|Buffer|File)} params.input - The input representing the avatar image. Can be a URL (string),
|
||||
* a Buffer, or a File object.
|
||||
* @param {string} params.manual - A string flag indicating whether the upload process is manual.
|
||||
*
|
||||
* @returns {Promise<any>}
|
||||
* A promise that resolves to the result of the `processAvatar` function, specific to the chosen file
|
||||
* strategy. Throws an error if any step in the process fails.
|
||||
* A promise that resolves to a resized buffer.
|
||||
*
|
||||
* @throws {Error} Throws an error if the user ID is undefined, the input type is invalid, the image fetching fails,
|
||||
* or any other error occurs during the processing.
|
||||
*/
|
||||
async function uploadAvatar({ userId, fileStrategy, input, manual }) {
|
||||
async function resizeAvatar({ userId, input }) {
|
||||
try {
|
||||
if (userId === undefined) {
|
||||
throw new Error('User ID is undefined');
|
||||
}
|
||||
const _id = userId;
|
||||
// TODO: remove direct use of Model, `User`
|
||||
const oldUser = await User.findOne({ _id });
|
||||
|
||||
let imageBuffer;
|
||||
if (typeof input === 'string') {
|
||||
|
|
@ -66,13 +53,12 @@ async function uploadAvatar({ userId, fileStrategy, input, manual }) {
|
|||
})
|
||||
.toBuffer();
|
||||
|
||||
const webPBuffer = await convertToWebP(squaredBuffer);
|
||||
const { processAvatar } = getStrategyFunctions(fileStrategy);
|
||||
return await processAvatar({ buffer: webPBuffer, User: oldUser, manual });
|
||||
const { buffer } = await resizeAndConvert(squaredBuffer);
|
||||
return buffer;
|
||||
} catch (error) {
|
||||
logger.error('Error uploading the avatar:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = uploadAvatar;
|
||||
module.exports = { resizeAvatar };
|
||||
|
|
|
|||
69
api/server/services/Files/images/convert.js
Normal file
69
api/server/services/Files/images/convert.js
Normal file
|
|
@ -0,0 +1,69 @@
|
|||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const sharp = require('sharp');
|
||||
const { resizeImageBuffer } = require('./resize');
|
||||
const { getStrategyFunctions } = require('../strategies');
|
||||
|
||||
/**
|
||||
* Converts an image file or buffer to WebP format with specified resolution.
|
||||
*
|
||||
* @param {Express.Request} req - The request object, containing user and app configuration data.
|
||||
* @param {Buffer | Express.Multer.File} file - The file object, containing either a path or a buffer.
|
||||
* @param {'low' | 'high'} [resolution='high'] - The desired resolution for the output image.
|
||||
* @param {string} [basename=''] - The basename of the input file, if it is a buffer.
|
||||
* @returns {Promise<{filepath: string, bytes: number, width: number, height: number}>} An object containing the path, size, and dimensions of the converted image.
|
||||
* @throws Throws an error if there is an issue during the conversion process.
|
||||
*/
|
||||
async function convertToWebP(req, file, resolution = 'high', basename = '') {
|
||||
try {
|
||||
let inputBuffer;
|
||||
let outputBuffer;
|
||||
let extension = path.extname(file.path ?? basename).toLowerCase();
|
||||
|
||||
// Check if the input is a buffer or a file path
|
||||
if (Buffer.isBuffer(file)) {
|
||||
inputBuffer = file;
|
||||
} else if (file && file.path) {
|
||||
const inputFilePath = file.path;
|
||||
inputBuffer = await fs.promises.readFile(inputFilePath);
|
||||
} else {
|
||||
throw new Error('Invalid input: file must be a buffer or contain a valid path.');
|
||||
}
|
||||
|
||||
// Resize the image buffer
|
||||
const {
|
||||
buffer: resizedBuffer,
|
||||
width,
|
||||
height,
|
||||
} = await resizeImageBuffer(inputBuffer, resolution);
|
||||
|
||||
// Check if the file is already in WebP format
|
||||
// If it isn't, convert it:
|
||||
if (extension === '.webp') {
|
||||
outputBuffer = resizedBuffer;
|
||||
} else {
|
||||
outputBuffer = await sharp(resizedBuffer).toFormat('webp').toBuffer();
|
||||
extension = '.webp';
|
||||
}
|
||||
|
||||
// Generate a new filename for the output file
|
||||
const newFileName =
|
||||
path.basename(file.path ?? basename, path.extname(file.path ?? basename)) + extension;
|
||||
|
||||
const { saveBuffer } = getStrategyFunctions(req.app.locals.fileStrategy);
|
||||
|
||||
const savedFilePath = await saveBuffer({
|
||||
userId: req.user.id,
|
||||
buffer: outputBuffer,
|
||||
fileName: newFileName,
|
||||
});
|
||||
|
||||
const bytes = Buffer.byteLength(outputBuffer);
|
||||
return { filepath: savedFilePath, bytes, width, height };
|
||||
} catch (err) {
|
||||
console.error(err);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { convertToWebP };
|
||||
|
|
@ -1,13 +1,13 @@
|
|||
const avatar = require('./avatar');
|
||||
const convert = require('./convert');
|
||||
const encode = require('./encode');
|
||||
const parse = require('./parse');
|
||||
const resize = require('./resize');
|
||||
const validate = require('./validate');
|
||||
|
||||
module.exports = {
|
||||
...convert,
|
||||
...encode,
|
||||
...parse,
|
||||
...resize,
|
||||
...validate,
|
||||
avatar,
|
||||
};
|
||||
|
|
|
|||
|
|
@ -1,6 +1,16 @@
|
|||
const sharp = require('sharp');
|
||||
|
||||
async function resizeImage(inputFilePath, resolution) {
|
||||
/**
|
||||
* Resizes an image from a given buffer based on the specified resolution.
|
||||
*
|
||||
* @param {Buffer} inputBuffer - The buffer of the image to be resized.
|
||||
* @param {'low' | 'high'} resolution - The resolution to resize the image to.
|
||||
* 'low' for a maximum of 512x512 resolution,
|
||||
* 'high' for a maximum of 768x2000 resolution.
|
||||
* @returns {Promise<{buffer: Buffer, width: number, height: number}>} An object containing the resized image buffer and its dimensions.
|
||||
* @throws Will throw an error if the resolution parameter is invalid.
|
||||
*/
|
||||
async function resizeImageBuffer(inputBuffer, resolution) {
|
||||
const maxLowRes = 512;
|
||||
const maxShortSideHighRes = 768;
|
||||
const maxLongSideHighRes = 2000;
|
||||
|
|
@ -12,7 +22,7 @@ async function resizeImage(inputFilePath, resolution) {
|
|||
resizeOptions.width = maxLowRes;
|
||||
resizeOptions.height = maxLowRes;
|
||||
} else if (resolution === 'high') {
|
||||
const metadata = await sharp(inputFilePath).metadata();
|
||||
const metadata = await sharp(inputBuffer).metadata();
|
||||
const isWidthShorter = metadata.width < metadata.height;
|
||||
|
||||
if (isWidthShorter) {
|
||||
|
|
@ -43,10 +53,28 @@ async function resizeImage(inputFilePath, resolution) {
|
|||
throw new Error('Invalid resolution parameter');
|
||||
}
|
||||
|
||||
const resizedBuffer = await sharp(inputFilePath).rotate().resize(resizeOptions).toBuffer();
|
||||
const resizedBuffer = await sharp(inputBuffer).rotate().resize(resizeOptions).toBuffer();
|
||||
|
||||
const resizedMetadata = await sharp(resizedBuffer).metadata();
|
||||
return { buffer: resizedBuffer, width: resizedMetadata.width, height: resizedMetadata.height };
|
||||
}
|
||||
|
||||
module.exports = { resizeImage };
|
||||
/**
|
||||
* Resizes an image buffer to webp format as well as reduces 150 px width.
|
||||
*
|
||||
* @param {Buffer} inputBuffer - The buffer of the image to be resized.
|
||||
* @returns {Promise<{ buffer: Buffer, width: number, height: number, bytes: number }>} An object containing the resized image buffer, its size and dimensions.
|
||||
* @throws Will throw an error if the resolution parameter is invalid.
|
||||
*/
|
||||
async function resizeAndConvert(inputBuffer) {
|
||||
const resizedBuffer = await sharp(inputBuffer).resize({ width: 150 }).toFormat('webp').toBuffer();
|
||||
const resizedMetadata = await sharp(resizedBuffer).metadata();
|
||||
return {
|
||||
buffer: resizedBuffer,
|
||||
width: resizedMetadata.width,
|
||||
height: resizedMetadata.height,
|
||||
bytes: Buffer.byteLength(resizedBuffer),
|
||||
};
|
||||
}
|
||||
|
||||
module.exports = { resizeImageBuffer, resizeAndConvert };
|
||||
|
|
|
|||
|
|
@ -1,13 +0,0 @@
|
|||
const { visionModels } = require('librechat-data-provider');
|
||||
|
||||
function validateVisionModel(model) {
|
||||
if (!model) {
|
||||
return false;
|
||||
}
|
||||
|
||||
return visionModels.some((visionModel) => model.includes(visionModel));
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
validateVisionModel,
|
||||
};
|
||||
|
|
@ -1,7 +1,25 @@
|
|||
const { updateFileUsage, createFile } = require('~/models');
|
||||
const path = require('path');
|
||||
const { v4 } = require('uuid');
|
||||
const mime = require('mime/lite');
|
||||
const {
|
||||
isUUID,
|
||||
megabyte,
|
||||
FileContext,
|
||||
FileSources,
|
||||
imageExtRegex,
|
||||
EModelEndpoint,
|
||||
mergeFileConfig,
|
||||
} = require('librechat-data-provider');
|
||||
const { convertToWebP, resizeAndConvert } = require('~/server/services/Files/images');
|
||||
const { initializeClient } = require('~/server/services/Endpoints/assistant');
|
||||
const { createFile, updateFileUsage, deleteFiles } = require('~/models/File');
|
||||
const { isEnabled, determineFileType } = require('~/server/utils');
|
||||
const { LB_QueueAsyncCall } = require('~/server/utils/queue');
|
||||
const { getStrategyFunctions } = require('./strategies');
|
||||
const { logger } = require('~/config');
|
||||
|
||||
const { GPTS_DOWNLOAD_IMAGES = 'true' } = process.env;
|
||||
|
||||
const processFiles = async (files) => {
|
||||
const promises = [];
|
||||
for (let file of files) {
|
||||
|
|
@ -13,6 +31,99 @@ const processFiles = async (files) => {
|
|||
return await Promise.all(promises);
|
||||
};
|
||||
|
||||
/**
|
||||
* Enqueues the delete operation to the leaky bucket queue if necessary, or adds it directly to promises.
|
||||
*
|
||||
* @param {Express.Request} req - The express request object.
|
||||
* @param {MongoFile} file - The file object to delete.
|
||||
* @param {Function} deleteFile - The delete file function.
|
||||
* @param {Promise[]} promises - The array of promises to await.
|
||||
* @param {OpenAI | undefined} [openai] - If an OpenAI file, the initialized OpenAI client.
|
||||
*/
|
||||
function enqueueDeleteOperation(req, file, deleteFile, promises, openai) {
|
||||
if (file.source === FileSources.openai) {
|
||||
// Enqueue to leaky bucket
|
||||
promises.push(
|
||||
new Promise((resolve, reject) => {
|
||||
LB_QueueAsyncCall(
|
||||
() => deleteFile(req, file, openai),
|
||||
[],
|
||||
(err, result) => {
|
||||
if (err) {
|
||||
logger.error('Error deleting file from OpenAI source', err);
|
||||
reject(err);
|
||||
} else {
|
||||
resolve(result);
|
||||
}
|
||||
},
|
||||
);
|
||||
}),
|
||||
);
|
||||
} else {
|
||||
// Add directly to promises
|
||||
promises.push(
|
||||
deleteFile(req, file).catch((err) => {
|
||||
logger.error('Error deleting file', err);
|
||||
return Promise.reject(err);
|
||||
}),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// TODO: refactor as currently only image files can be deleted this way
|
||||
// as other filetypes will not reside in public path
|
||||
/**
|
||||
* Deletes a list of files from the server filesystem and the database.
|
||||
*
|
||||
* @param {Object} params - The params object.
|
||||
* @param {MongoFile[]} params.files - The file objects to delete.
|
||||
* @param {Express.Request} params.req - The express request object.
|
||||
* @param {DeleteFilesBody} params.req.body - The request body.
|
||||
* @param {string} [params.req.body.assistant_id] - The assistant ID if file uploaded is associated to an assistant.
|
||||
*
|
||||
* @returns {Promise<void>}
|
||||
*/
|
||||
const processDeleteRequest = async ({ req, files }) => {
|
||||
const file_ids = files.map((file) => file.file_id);
|
||||
|
||||
const deletionMethods = {};
|
||||
const promises = [];
|
||||
promises.push(deleteFiles(file_ids));
|
||||
|
||||
/** @type {OpenAI | undefined} */
|
||||
let openai;
|
||||
if (req.body.assistant_id) {
|
||||
({ openai } = await initializeClient({ req }));
|
||||
}
|
||||
|
||||
for (const file of files) {
|
||||
const source = file.source ?? FileSources.local;
|
||||
|
||||
if (source === FileSources.openai && !openai) {
|
||||
({ openai } = await initializeClient({ req }));
|
||||
}
|
||||
|
||||
if (req.body.assistant_id) {
|
||||
promises.push(openai.beta.assistants.files.del(req.body.assistant_id, file.file_id));
|
||||
}
|
||||
|
||||
if (deletionMethods[source]) {
|
||||
enqueueDeleteOperation(req, file, deletionMethods[source], promises, openai);
|
||||
continue;
|
||||
}
|
||||
|
||||
const { deleteFile } = getStrategyFunctions(source);
|
||||
if (!deleteFile) {
|
||||
throw new Error(`Delete function not implemented for ${source}`);
|
||||
}
|
||||
|
||||
deletionMethods[source] = deleteFile;
|
||||
enqueueDeleteOperation(req, file, deleteFile, promises, openai);
|
||||
}
|
||||
|
||||
await Promise.allSettled(promises);
|
||||
};
|
||||
|
||||
/**
|
||||
* Processes a file URL using a specified file handling strategy. This function accepts a strategy name,
|
||||
* fetches the corresponding file processing functions (for saving and retrieving file URLs), and then
|
||||
|
|
@ -21,25 +132,38 @@ const processFiles = async (files) => {
|
|||
* exception with an appropriate message.
|
||||
*
|
||||
* @param {Object} params - The parameters object.
|
||||
* @param {FileSources} params.fileStrategy - The file handling strategy to use. Must be a value from the
|
||||
* `FileSources` enum, which defines different file handling
|
||||
* strategies (like saving to Firebase, local storage, etc.).
|
||||
* @param {FileSources} params.fileStrategy - The file handling strategy to use.
|
||||
* Must be a value from the `FileSources` enum, which defines different file
|
||||
* handling strategies (like saving to Firebase, local storage, etc.).
|
||||
* @param {string} params.userId - The user's unique identifier. Used for creating user-specific paths or
|
||||
* references in the file handling process.
|
||||
* references in the file handling process.
|
||||
* @param {string} params.URL - The URL of the file to be processed.
|
||||
* @param {string} params.fileName - The name that will be used to save the file. This should include the
|
||||
* file extension.
|
||||
* @param {string} params.fileName - The name that will be used to save the file (including extension)
|
||||
* @param {string} params.basePath - The base path or directory where the file will be saved or retrieved from.
|
||||
*
|
||||
* @returns {Promise<string>}
|
||||
* A promise that resolves to the URL of the processed file. It throws an error if the file processing
|
||||
* fails at any stage.
|
||||
* @param {FileContext} params.context - The context of the file (e.g., 'avatar', 'image_generation', etc.)
|
||||
* @returns {Promise<MongoFile>} A promise that resolves to the DB representation (MongoFile)
|
||||
* of the processed file. It throws an error if the file processing fails at any stage.
|
||||
*/
|
||||
const processFileURL = async ({ fileStrategy, userId, URL, fileName, basePath }) => {
|
||||
const processFileURL = async ({ fileStrategy, userId, URL, fileName, basePath, context }) => {
|
||||
const { saveURL, getFileURL } = getStrategyFunctions(fileStrategy);
|
||||
try {
|
||||
await saveURL({ userId, URL, fileName, basePath });
|
||||
return await getFileURL({ fileName: `${userId}/${fileName}`, basePath });
|
||||
const { bytes, type, dimensions } = await saveURL({ userId, URL, fileName, basePath });
|
||||
const filepath = await getFileURL({ fileName: `${userId}/${fileName}`, basePath });
|
||||
return await createFile(
|
||||
{
|
||||
user: userId,
|
||||
file_id: v4(),
|
||||
bytes,
|
||||
filepath,
|
||||
filename: fileName,
|
||||
source: fileStrategy,
|
||||
type,
|
||||
context,
|
||||
width: dimensions.width,
|
||||
height: dimensions.height,
|
||||
},
|
||||
true,
|
||||
);
|
||||
} catch (error) {
|
||||
logger.error(`Error while processing the image with ${fileStrategy}:`, error);
|
||||
throw new Error(`Failed to process the image with ${fileStrategy}. ${error.message}`);
|
||||
|
|
@ -49,7 +173,6 @@ const processFileURL = async ({ fileStrategy, userId, URL, fileName, basePath })
|
|||
/**
|
||||
* Applies the current strategy for image uploads.
|
||||
* Saves file metadata to the database with an expiry TTL.
|
||||
* Files must be deleted from the server filesystem manually.
|
||||
*
|
||||
* @param {Object} params - The parameters object.
|
||||
* @param {Express.Request} params.req - The Express request object.
|
||||
|
|
@ -58,7 +181,7 @@ const processFileURL = async ({ fileStrategy, userId, URL, fileName, basePath })
|
|||
* @param {ImageMetadata} params.metadata - Additional metadata for the file.
|
||||
* @returns {Promise<void>}
|
||||
*/
|
||||
const processImageUpload = async ({ req, res, file, metadata }) => {
|
||||
const processImageFile = async ({ req, res, file, metadata }) => {
|
||||
const source = req.app.locals.fileStrategy;
|
||||
const { handleImageUpload } = getStrategyFunctions(source);
|
||||
const { file_id, temp_file_id } = metadata;
|
||||
|
|
@ -71,6 +194,7 @@ const processImageUpload = async ({ req, res, file, metadata }) => {
|
|||
bytes,
|
||||
filepath,
|
||||
filename: file.originalname,
|
||||
context: FileContext.message_attachment,
|
||||
source,
|
||||
type: 'image/webp',
|
||||
width,
|
||||
|
|
@ -81,8 +205,271 @@ const processImageUpload = async ({ req, res, file, metadata }) => {
|
|||
res.status(200).json({ message: 'File uploaded and processed successfully', ...result });
|
||||
};
|
||||
|
||||
/**
|
||||
* Applies the current strategy for image uploads and
|
||||
* returns minimal file metadata, without saving to the database.
|
||||
*
|
||||
* @param {Object} params - The parameters object.
|
||||
* @param {Express.Request} params.req - The Express request object.
|
||||
* @param {FileContext} params.context - The context of the file (e.g., 'avatar', 'image_generation', etc.)
|
||||
* @returns {Promise<{ filepath: string, filename: string, source: string, type: 'image/webp'}>}
|
||||
*/
|
||||
const uploadImageBuffer = async ({ req, context }) => {
|
||||
const source = req.app.locals.fileStrategy;
|
||||
const { saveBuffer } = getStrategyFunctions(source);
|
||||
const { buffer, width, height, bytes } = await resizeAndConvert(req.file.buffer);
|
||||
const file_id = v4();
|
||||
const fileName = `img-${file_id}.webp`;
|
||||
|
||||
const filepath = await saveBuffer({ userId: req.user.id, fileName, buffer });
|
||||
return await createFile(
|
||||
{
|
||||
user: req.user.id,
|
||||
file_id,
|
||||
bytes,
|
||||
filepath,
|
||||
filename: req.file.originalname,
|
||||
context,
|
||||
source,
|
||||
type: 'image/webp',
|
||||
width,
|
||||
height,
|
||||
},
|
||||
true,
|
||||
);
|
||||
};
|
||||
|
||||
/**
|
||||
* Applies the current strategy for file uploads.
|
||||
* Saves file metadata to the database with an expiry TTL.
|
||||
* Files must be deleted from the server filesystem manually.
|
||||
*
|
||||
* @param {Object} params - The parameters object.
|
||||
* @param {Express.Request} params.req - The Express request object.
|
||||
* @param {Express.Response} params.res - The Express response object.
|
||||
* @param {Express.Multer.File} params.file - The uploaded file.
|
||||
* @param {FileMetadata} params.metadata - Additional metadata for the file.
|
||||
* @returns {Promise<void>}
|
||||
*/
|
||||
const processFileUpload = async ({ req, res, file, metadata }) => {
|
||||
const isAssistantUpload = metadata.endpoint === EModelEndpoint.assistants;
|
||||
const source = isAssistantUpload ? FileSources.openai : req.app.locals.fileStrategy;
|
||||
const { handleFileUpload } = getStrategyFunctions(source);
|
||||
const { file_id, temp_file_id } = metadata;
|
||||
|
||||
/** @type {OpenAI | undefined} */
|
||||
let openai;
|
||||
if (source === FileSources.openai) {
|
||||
({ openai } = await initializeClient({ req }));
|
||||
}
|
||||
|
||||
const { id, bytes, filename, filepath } = await handleFileUpload(req, file, openai);
|
||||
|
||||
if (isAssistantUpload && !metadata.message_file) {
|
||||
await openai.beta.assistants.files.create(metadata.assistant_id, {
|
||||
file_id: id,
|
||||
});
|
||||
}
|
||||
|
||||
const result = await createFile(
|
||||
{
|
||||
user: req.user.id,
|
||||
file_id: id ?? file_id,
|
||||
temp_file_id,
|
||||
bytes,
|
||||
filepath: isAssistantUpload ? `https://api.openai.com/v1/files/${id}` : filepath,
|
||||
filename: filename ?? file.originalname,
|
||||
context: isAssistantUpload ? FileContext.assistants : FileContext.message_attachment,
|
||||
source,
|
||||
type: file.mimetype,
|
||||
},
|
||||
true,
|
||||
);
|
||||
res.status(200).json({ message: 'File uploaded and processed successfully', ...result });
|
||||
};
|
||||
|
||||
/**
|
||||
* Retrieves and processes an OpenAI file based on its type.
|
||||
*
|
||||
* @param {Object} params - The params passed to the function.
|
||||
* @param {OpenAIClient} params.openai - The params passed to the function.
|
||||
* @param {string} params.file_id - The ID of the file to retrieve.
|
||||
* @param {string} params.basename - The basename of the file (if image); e.g., 'image.jpg'.
|
||||
* @param {boolean} [params.unknownType] - Whether the file type is unknown.
|
||||
* @returns {Promise<{file_id: string, filepath: string, source: string, bytes?: number, width?: number, height?: number} | null>}
|
||||
* - Returns null if `file_id` is not defined; else, the file metadata if successfully retrieved and processed.
|
||||
*/
|
||||
async function retrieveAndProcessFile({ openai, file_id, basename: _basename, unknownType }) {
|
||||
if (!file_id) {
|
||||
return null;
|
||||
}
|
||||
|
||||
if (openai.attachedFileIds?.has(file_id)) {
|
||||
return {
|
||||
file_id,
|
||||
// filepath: TODO: local source filepath?,
|
||||
source: FileSources.openai,
|
||||
};
|
||||
}
|
||||
|
||||
let basename = _basename;
|
||||
const downloadImages = isEnabled(GPTS_DOWNLOAD_IMAGES);
|
||||
|
||||
/**
|
||||
* @param {string} file_id - The ID of the file to retrieve.
|
||||
* @param {boolean} [save] - Whether to save the file metadata to the database.
|
||||
*/
|
||||
const retrieveFile = async (file_id, save = false) => {
|
||||
const _file = await openai.files.retrieve(file_id);
|
||||
const filepath = `/api/files/download/${file_id}`;
|
||||
const file = {
|
||||
..._file,
|
||||
type: mime.getType(_file.filename),
|
||||
filepath,
|
||||
usage: 1,
|
||||
file_id,
|
||||
context: _file.purpose ?? FileContext.message_attachment,
|
||||
source: FileSources.openai,
|
||||
};
|
||||
|
||||
if (save) {
|
||||
await createFile(file, true);
|
||||
} else {
|
||||
try {
|
||||
await updateFileUsage({ file_id });
|
||||
} catch (error) {
|
||||
logger.error('Error updating file usage', error);
|
||||
}
|
||||
}
|
||||
|
||||
return file;
|
||||
};
|
||||
|
||||
// If image downloads are not enabled or no basename provided, return only the file metadata
|
||||
if (!downloadImages || (!basename && !downloadImages)) {
|
||||
return await retrieveFile(file_id, true);
|
||||
}
|
||||
|
||||
let data;
|
||||
try {
|
||||
const response = await openai.files.content(file_id);
|
||||
data = await response.arrayBuffer();
|
||||
} catch (error) {
|
||||
logger.error('Error downloading file from OpenAI:', error);
|
||||
return await retrieveFile(file_id);
|
||||
}
|
||||
|
||||
if (!data) {
|
||||
return await retrieveFile(file_id);
|
||||
}
|
||||
const dataBuffer = Buffer.from(data);
|
||||
|
||||
/**
|
||||
* @param {Buffer} dataBuffer
|
||||
* @param {string} fileExt
|
||||
*/
|
||||
const processAsImage = async (dataBuffer, fileExt) => {
|
||||
// Logic to process image files, convert to webp, etc.
|
||||
const _file = await convertToWebP(openai.req, dataBuffer, 'high', `${file_id}${fileExt}`);
|
||||
const file = {
|
||||
..._file,
|
||||
type: 'image/webp',
|
||||
usage: 1,
|
||||
file_id,
|
||||
source: FileSources.openai,
|
||||
};
|
||||
createFile(file, true);
|
||||
return file;
|
||||
};
|
||||
|
||||
/** @param {Buffer} dataBuffer */
|
||||
const processOtherFileTypes = async (dataBuffer) => {
|
||||
// Logic to handle other file types
|
||||
logger.debug('[retrieveAndProcessFile] Non-image file type detected');
|
||||
return { filepath: `/api/files/download/${file_id}`, bytes: dataBuffer.length };
|
||||
};
|
||||
|
||||
// If the filetype is unknown, inspect the file
|
||||
if (unknownType || !path.extname(basename)) {
|
||||
const detectedExt = await determineFileType(dataBuffer);
|
||||
if (detectedExt && imageExtRegex.test('.' + detectedExt)) {
|
||||
return await processAsImage(dataBuffer, detectedExt);
|
||||
} else {
|
||||
return await processOtherFileTypes(dataBuffer);
|
||||
}
|
||||
}
|
||||
|
||||
// Existing logic for processing known image types
|
||||
if (downloadImages && basename && path.extname(basename) && imageExtRegex.test(basename)) {
|
||||
return await processAsImage(dataBuffer, path.extname(basename));
|
||||
} else {
|
||||
logger.debug('[retrieveAndProcessFile] Not an image or invalid extension: ', basename);
|
||||
return await processOtherFileTypes(dataBuffer);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Filters a file based on its size and the endpoint origin.
|
||||
*
|
||||
* @param {Object} params - The parameters for the function.
|
||||
* @param {Express.Request} params.req - The request object from Express.
|
||||
* @param {Express.Multer.File} params.file - The file uploaded to the server via multer.
|
||||
* @param {boolean} [params.image] - Whether the file expected is an image.
|
||||
* @returns {void}
|
||||
*
|
||||
* @throws {Error} If a file exception is caught (invalid file size or type, lack of metadata).
|
||||
*/
|
||||
function filterFile({ req, file, image }) {
|
||||
const { endpoint, file_id, width, height } = req.body;
|
||||
|
||||
if (!file_id) {
|
||||
throw new Error('No file_id provided');
|
||||
}
|
||||
|
||||
/* parse to validate api call, throws error on fail */
|
||||
isUUID.parse(file_id);
|
||||
|
||||
if (!endpoint) {
|
||||
throw new Error('No endpoint provided');
|
||||
}
|
||||
|
||||
const fileConfig = mergeFileConfig(req.app.locals.fileConfig);
|
||||
|
||||
const { fileSizeLimit, supportedMimeTypes } =
|
||||
fileConfig.endpoints[endpoint] ?? fileConfig.endpoints.default;
|
||||
|
||||
if (file.size > fileSizeLimit) {
|
||||
throw new Error(
|
||||
`File size limit of ${fileSizeLimit / megabyte} MB exceeded for ${endpoint} endpoint`,
|
||||
);
|
||||
}
|
||||
|
||||
const isSupportedMimeType = fileConfig.checkType(file.mimetype, supportedMimeTypes);
|
||||
|
||||
if (!isSupportedMimeType) {
|
||||
throw new Error('Unsupported file type');
|
||||
}
|
||||
|
||||
if (!image) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (!width) {
|
||||
throw new Error('No width provided');
|
||||
}
|
||||
|
||||
if (!height) {
|
||||
throw new Error('No height provided');
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
processImageUpload,
|
||||
filterFile,
|
||||
processFiles,
|
||||
processFileURL,
|
||||
processImageFile,
|
||||
uploadImageBuffer,
|
||||
processFileUpload,
|
||||
processDeleteRequest,
|
||||
retrieveAndProcessFile,
|
||||
};
|
||||
|
|
|
|||
|
|
@ -4,46 +4,82 @@ const {
|
|||
prepareImageURL,
|
||||
saveURLToFirebase,
|
||||
deleteFirebaseFile,
|
||||
saveBufferToFirebase,
|
||||
uploadImageToFirebase,
|
||||
processFirebaseAvatar,
|
||||
} = require('./Firebase');
|
||||
const {
|
||||
// saveLocalFile,
|
||||
getLocalFileURL,
|
||||
saveFileFromURL,
|
||||
saveLocalBuffer,
|
||||
deleteLocalFile,
|
||||
uploadLocalImage,
|
||||
prepareImagesLocal,
|
||||
processLocalAvatar,
|
||||
} = require('./Local');
|
||||
const { uploadOpenAIFile, deleteOpenAIFile } = require('./OpenAI');
|
||||
|
||||
// Firebase Strategy Functions
|
||||
/**
|
||||
* Firebase Storage Strategy Functions
|
||||
*
|
||||
* */
|
||||
const firebaseStrategy = () => ({
|
||||
// saveFile:
|
||||
saveURL: saveURLToFirebase,
|
||||
getFileURL: getFirebaseURL,
|
||||
deleteFile: deleteFirebaseFile,
|
||||
saveBuffer: saveBufferToFirebase,
|
||||
prepareImagePayload: prepareImageURL,
|
||||
processAvatar: processFirebaseAvatar,
|
||||
handleImageUpload: uploadImageToFirebase,
|
||||
});
|
||||
|
||||
// Local Strategy Functions
|
||||
/**
|
||||
* Local Server Storage Strategy Functions
|
||||
*
|
||||
* */
|
||||
const localStrategy = () => ({
|
||||
// saveFile: ,
|
||||
// saveFile: saveLocalFile,
|
||||
saveURL: saveFileFromURL,
|
||||
getFileURL: getLocalFileURL,
|
||||
saveBuffer: saveLocalBuffer,
|
||||
deleteFile: deleteLocalFile,
|
||||
processAvatar: processLocalAvatar,
|
||||
handleImageUpload: uploadLocalImage,
|
||||
prepareImagePayload: prepareImagesLocal,
|
||||
});
|
||||
|
||||
/**
|
||||
* OpenAI Strategy Functions
|
||||
*
|
||||
* Note: null values mean that the strategy is not supported.
|
||||
* */
|
||||
const openAIStrategy = () => ({
|
||||
/** @type {typeof saveFileFromURL | null} */
|
||||
saveURL: null,
|
||||
/** @type {typeof getLocalFileURL | null} */
|
||||
getFileURL: null,
|
||||
/** @type {typeof saveLocalBuffer | null} */
|
||||
saveBuffer: null,
|
||||
/** @type {typeof processLocalAvatar | null} */
|
||||
processAvatar: null,
|
||||
/** @type {typeof uploadLocalImage | null} */
|
||||
handleImageUpload: null,
|
||||
/** @type {typeof prepareImagesLocal | null} */
|
||||
prepareImagePayload: null,
|
||||
deleteFile: deleteOpenAIFile,
|
||||
handleFileUpload: uploadOpenAIFile,
|
||||
});
|
||||
|
||||
// Strategy Selector
|
||||
const getStrategyFunctions = (fileSource) => {
|
||||
if (fileSource === FileSources.firebase) {
|
||||
return firebaseStrategy();
|
||||
} else if (fileSource === FileSources.local) {
|
||||
return localStrategy();
|
||||
} else if (fileSource === FileSources.openai) {
|
||||
return openAIStrategy();
|
||||
} else {
|
||||
throw new Error('Invalid file source');
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue