LibreChat/api/server/services
Danny Avila 48f6f8f2f8
📎 feat: Upload as Text Support for Plaintext, STT, RAG, and Token Limits (#8868)
* 🪶 feat: Add Support for Uploading Plaintext Files

feat: delineate between OCR and text handling in fileConfig field of config file

- also adds support for passing in mimetypes as just plain file extensions

feat: add showLabel bool to support future synthetic component DynamicDropdownInput

feat: add new combination dropdown-input component in params panel to support file type token limits

refactor: move hovercard to side to align with other hovercards

chore: clean up autogenerated comments

feat: add delineation to file upload path between text and ocr configured filetypes

feat: add token limit checks during file upload

refactor: move textParsing out of ocrEnabled logic

refactor: clean up types for filetype config

refactor: finish decoupling DynamicDropdownInput from fileTokenLimits

fix: move image token cost function into file to fix circular dependency causing unittest to fail and remove unused var for linter

chore: remove out of scope code following review

refactor: make fileTokenLimit conform to existing styles

chore: remove unused localization string

chore: undo changes to DynamicInput and other strays

feat: add fileTokenLimit to all provider config panels

fix: move textParsing back into ocr tool_resource block for now so that it doesn't interfere with other upload types

* 📤 feat: Add RAG API Endpoint Support for Text Parsing (#8849)

* feat: implement RAG API integration for text parsing with fallback to native parsing

* chore: remove TODO now that placeholder and fllback are implemented

* ✈️ refactor: Migrate Text Parsing to TS (#8892)

* refactor: move generateShortLivedToken to packages/api

* refactor: move textParsing logic into packages/api

* refactor: reduce nesting and dry code with createTextFile

* fix: add proper source handling

* fix: mock new parseText and parseTextNative functions in jest file

* ci: add test coverage for textParser

* 💬 feat: Add Audio File Support to Upload as Text (#8893)

* feat: add STT support for Upload as Text

* refactor: move processAudioFile to packages/api

* refactor: move textParsing from utils to files

* fix: remove audio/mp3 from unsupported mimetypes test since it is now supported

* ✂️ feat: Configurable File Token Limits and Truncation (#8911)

* feat: add configurable fileTokenLimit default value

* fix: add stt to fileConfig merge logic

* fix: add fileTokenLimit to mergeFileConfig logic so configurable value is actually respected from yaml

* feat: add token limiting to parsed text files

* fix: add extraction logic and update tests so fileTokenLimit isnt sent to LLM providers

* fix: address comments

* refactor: rename textTokenLimiter.ts to text.ts

* chore: update form-data package to address CVE-2025-7783 and update package-lock

* feat: use default supported mime types for ocr on frontend file validation

* fix: should be using logger.debug not console.debug

* fix: mock existsSync in text.spec.ts

* fix: mock logger rather than every one of its function calls

* fix: reorganize imports and streamline file upload processing logic

* refactor: update createTextFile function to use destructured parameters and improve readability

* chore: update file validation to use EToolResources for improved type safety

* chore: update import path for types in audio processing module

* fix: update file configuration access and replace console.debug with logger.debug for improved logging

---------

Co-authored-by: Dustin Healy <dustinhealy1@gmail.com>
Co-authored-by: Dustin Healy <54083382+dustinhealy@users.noreply.github.com>
2025-08-27 03:44:39 -04:00
..
Artifacts 🧪 fix: Editor Styling, Incomplete Artifact Editing, Optimize Artifact Context (#8953) 2025-08-08 15:49:58 -04:00
Config 🛜 refactor: Streamline App Config Usage (#9234) 2025-08-26 12:10:18 -04:00
Endpoints 📎 feat: Upload as Text Support for Plaintext, STT, RAG, and Token Limits (#8868) 2025-08-27 03:44:39 -04:00
Files 📎 feat: Upload as Text Support for Plaintext, STT, RAG, and Token Limits (#8868) 2025-08-27 03:44:39 -04:00
Runs 🛜 refactor: Streamline App Config Usage (#9234) 2025-08-26 12:10:18 -04:00
start 🛜 refactor: Streamline App Config Usage (#9234) 2025-08-26 12:10:18 -04:00
Threads 🔧 fix: Assistants API SDK calls to match Updated Arguments (#8818) 2025-08-02 12:19:58 -04:00
Tools 🛜 refactor: Streamline App Config Usage (#9234) 2025-08-26 12:10:18 -04:00
ActionService.js 🪐 feat: MCP OAuth 2.0 Discovery Support (#7924) 2025-06-17 13:50:33 -04:00
ActionService.spec.js 🛜 refactor: Streamline App Config Usage (#9234) 2025-08-26 12:10:18 -04:00
AppService.interface.spec.js 🛜 refactor: Streamline App Config Usage (#9234) 2025-08-26 12:10:18 -04:00
AppService.js 🛜 refactor: Streamline App Config Usage (#9234) 2025-08-26 12:10:18 -04:00
AppService.spec.js 🛜 refactor: Streamline App Config Usage (#9234) 2025-08-26 12:10:18 -04:00
AssistantService.js 🛜 refactor: Streamline App Config Usage (#9234) 2025-08-26 12:10:18 -04:00
AuthService.js 📎 feat: Upload as Text Support for Plaintext, STT, RAG, and Token Limits (#8868) 2025-08-27 03:44:39 -04:00
cleanup.js 🧹 fix: Resolve Unarchive Conversation Bug, Archive Pagination (#4189) 2024-09-22 17:21:50 -04:00
createRunBody.js feat: Add Current Datetime to Assistants (v1/v2) (#4952) 2024-12-11 15:26:18 -05:00
domains.js 🛜 refactor: Streamline App Config Usage (#9234) 2025-08-26 12:10:18 -04:00
domains.spec.js 🛜 refactor: Streamline App Config Usage (#9234) 2025-08-26 12:10:18 -04:00
GraphApiService.js 🔐 feat: Granular Role-based Permissions + Entra ID Group Discovery (#7804) 2025-08-13 16:24:17 -04:00
GraphApiService.spec.js 🔐 feat: Granular Role-based Permissions + Entra ID Group Discovery (#7804) 2025-08-13 16:24:17 -04:00
GraphTokenService.js 📁 feat: Integrate SharePoint File Picker and Download Workflow (#8651) 2025-08-13 16:24:16 -04:00
initializeMCPs.js 🛜 refactor: Streamline App Config Usage (#9234) 2025-08-26 12:10:18 -04:00
MCP.js 🛜 refactor: Streamline App Config Usage (#9234) 2025-08-26 12:10:18 -04:00
MCP.spec.js 🛜 refactor: Streamline App Config Usage (#9234) 2025-08-26 12:10:18 -04:00
ModelService.js 🛣️ feat: directEndpoint Fetch Override for Custom Endpoints (#9179) 2025-08-20 15:55:32 -04:00
ModelService.spec.js 👁️ feat: Azure Mistral OCR Strategy (#7888) 2025-06-13 15:14:57 -04:00
PermissionService.js 🛂 feat: Role as Permission Principal Type 2025-08-13 16:24:23 -04:00
PermissionService.spec.js 🛂 feat: Role as Permission Principal Type 2025-08-13 16:24:23 -04:00
PluginService.js 🔌 feat: MCP Reinitialization and OAuth in UI (#8598) 2025-07-22 22:52:45 -04:00
ToolService.js 🛜 refactor: Streamline App Config Usage (#9234) 2025-08-26 12:10:18 -04:00
twoFactorService.js 🪐 feat: MCP OAuth 2.0 Discovery Support (#7924) 2025-06-17 13:50:33 -04:00
UserService.js 🪐 feat: MCP OAuth 2.0 Discovery Support (#7924) 2025-06-17 13:50:33 -04:00