mirror of
https://github.com/danny-avila/LibreChat.git
synced 2025-12-16 08:20:14 +01:00
8bdc808074
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Waiting to run
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
* refactor: Token Limit Processing with Enhanced Efficiency - Added a new test suite for `processTextWithTokenLimit`, ensuring comprehensive coverage of various scenarios including under, at, and exceeding token limits. - Refactored the `processTextWithTokenLimit` function to utilize a ratio-based estimation method, significantly reducing the number of token counting function calls compared to the previous binary search approach. - Improved handling of edge cases and variable token density, ensuring accurate truncation and performance across diverse text inputs. - Included direct comparisons with the old implementation to validate correctness and efficiency improvements. * refactor: Remove Tokenizer Route and Related References - Deleted the tokenizer route from the server and removed its references from the routes index and server files, streamlining the API structure. - This change simplifies the routing configuration by eliminating unused endpoints. * refactor: Migrate countTokens Utility to API Module - Removed the local countTokens utility and integrated it into the @librechat/api module for centralized access. - Updated various files to reference the new countTokens import from the API module, ensuring consistent usage across the application. - Cleaned up unused references and imports related to the previous countTokens implementation. * refactor: Centralize escapeRegExp Utility in API Module - Moved the escapeRegExp function from local utility files to the @librechat/api module for consistent usage across the application. - Updated imports in various files to reference the new centralized escapeRegExp function, ensuring cleaner code and reducing redundancy. - Removed duplicate implementations of escapeRegExp from multiple files, streamlining the codebase. * refactor: Enhance Token Counting Flexibility in Text Processing - Updated the `processTextWithTokenLimit` function to accept both synchronous and asynchronous token counting functions, improving its versatility. - Introduced a new `TokenCountFn` type to define the token counting function signature. - Added comprehensive tests to validate the behavior of `processTextWithTokenLimit` with both sync and async token counting functions, ensuring consistent results. - Implemented a wrapper to track call counts for the `countTokens` function, optimizing performance and reducing unnecessary calls. - Enhanced existing tests to compare the performance of the new implementation against the old one, demonstrating significant improvements in efficiency. * chore: documentation for Truncation Safety Buffer in Token Processing - Added a safety buffer multiplier to the character position estimates during text truncation to prevent overshooting token limits. - Updated the `processTextWithTokenLimit` function to utilize the new `TRUNCATION_SAFETY_BUFFER` constant, enhancing the accuracy of token limit processing. - Improved documentation to clarify the rationale behind the buffer and its impact on performance and efficiency in token counting.
59 lines
1.3 KiB
JavaScript
59 lines
1.3 KiB
JavaScript
const accessPermissions = require('./accessPermissions');
|
|
const assistants = require('./assistants');
|
|
const categories = require('./categories');
|
|
const endpoints = require('./endpoints');
|
|
const staticRoute = require('./static');
|
|
const messages = require('./messages');
|
|
const memories = require('./memories');
|
|
const presets = require('./presets');
|
|
const prompts = require('./prompts');
|
|
const balance = require('./balance');
|
|
const plugins = require('./plugins');
|
|
const actions = require('./actions');
|
|
const banner = require('./banner');
|
|
const search = require('./search');
|
|
const models = require('./models');
|
|
const convos = require('./convos');
|
|
const config = require('./config');
|
|
const agents = require('./agents');
|
|
const roles = require('./roles');
|
|
const oauth = require('./oauth');
|
|
const files = require('./files');
|
|
const share = require('./share');
|
|
const tags = require('./tags');
|
|
const auth = require('./auth');
|
|
const edit = require('./edit');
|
|
const keys = require('./keys');
|
|
const user = require('./user');
|
|
const mcp = require('./mcp');
|
|
|
|
module.exports = {
|
|
mcp,
|
|
edit,
|
|
auth,
|
|
keys,
|
|
user,
|
|
tags,
|
|
roles,
|
|
oauth,
|
|
files,
|
|
share,
|
|
banner,
|
|
agents,
|
|
convos,
|
|
search,
|
|
config,
|
|
models,
|
|
prompts,
|
|
plugins,
|
|
actions,
|
|
presets,
|
|
balance,
|
|
messages,
|
|
memories,
|
|
endpoints,
|
|
assistants,
|
|
categories,
|
|
staticRoute,
|
|
accessPermissions,
|
|
};
|