9 commits

Author	SHA1	Message	Date
Danny Avila	e1e204d6cf	🧮 refactor: Bulk Transactions & Balance Updates for Token Spending (#11996) ... * refactor: transaction handling by integrating pricing and bulk write operations - Updated `recordCollectedUsage` to accept pricing functions and bulk write operations, improving transaction management. - Refactored `AgentClient` and related controllers to utilize the new transaction handling capabilities, ensuring better performance and accuracy in token spending. - Added tests to validate the new functionality, ensuring correct behavior for both standard and bulk transaction paths. - Introduced a new `transactions.ts` file to encapsulate transaction-related logic and types, enhancing code organization and maintainability. * chore: reorganize imports in agents client controller - Moved `getMultiplier` and `getCacheMultiplier` imports to maintain consistency and clarity in the import structure. - Removed duplicate import of `updateBalance` and `bulkInsertTransactions`, streamlining the code for better readability. * refactor: add TransactionData type and CANCEL_RATE constant to data-schemas Establishes a single source of truth for the transaction document shape and the incomplete-context billing rate constant, both consumed by packages/api and api/. * refactor: use proper types in data-schemas transaction methods - Replace `as unknown as { tokenCredits }` with `lean<IBalance>()` - Use `TransactionData[]` instead of `Record<string, unknown>[]` for bulkInsertTransactions parameter - Add JSDoc noting insertMany bypasses document middleware - Remove orphan section comment in methods/index.ts * refactor: use shared types in transactions.ts, fix bulk write logic - Import CANCEL_RATE from data-schemas instead of local duplicate - Import TransactionData from data-schemas for PreparedEntry/BulkWriteDeps - Use tilde alias for EndpointTokenConfig import - Pass valueKey through to getMultiplier - Only sum tokenValue for balance-enabled docs in bulkWriteTransactions - Consolidate two loops into single-pass map * refactor: remove duplicate updateBalance from Transaction.js Import updateBalance from ~/models (sourced from data-schemas) instead of maintaining a second copy. Also import CANCEL_RATE from data-schemas and remove the Balance model import (no longer needed directly). * fix: test real spendCollectedUsage instead of IIFE replica Export spendCollectedUsage from abortMiddleware.js and rewrite the test file to import and test the actual function. Previously the tests ran against a hand-written replica that could silently diverge from the real implementation. * test: add transactions.spec.ts and restore regression comments Add 22 direct unit tests for transactions.ts financial logic covering prepareTokenSpend, prepareStructuredTokenSpend, bulkWriteTransactions, CANCEL_RATE paths, NaN guards, disabled transactions, zero tokens, cache multipliers, and balance-enabled filtering. Restore critical regression documentation comments in recordCollectedUsage.spec.js explaining which production bugs the tests guard against. * fix: widen setValues type to include lastRefill The UpdateBalanceParams.setValues type was Partial<Pick<IBalance, 'tokenCredits'>> which excluded lastRefill — used by createAutoRefillTransaction. Widen to also pick 'lastRefill'. * test: use real MongoDB for bulkWriteTransactions tests Replace mock-based bulkWriteTransactions tests with real DB tests using MongoMemoryServer. Pure function tests (prepareTokenSpend, prepareStructuredTokenSpend) remain mock-based since they don't touch DB. Add end-to-end integration tests that verify the full prepare → bulk write → DB state pipeline with real Transaction and Balance models. * chore: update @librechat/agents dependency to version 3.1.54 in package-lock.json and related package.json files * test: add bulk path parity tests proving identical DB outcomes Three test suites proving the bulk path (prepareTokenSpend/ prepareStructuredTokenSpend + bulkWriteTransactions) produces numerically identical results to the legacy path for all scenarios: - usage.bulk-parity.spec.ts: mirrors all legacy recordCollectedUsage tests; asserts same return values and verifies metadata fields on the insertMany docs match what spendTokens args would carry - transactions.bulk-parity.spec.ts: real-DB tests using actual getMultiplier/getCacheMultiplier pricing functions; asserts exact tokenValue, rate, rawAmount and balance deductions for standard tokens, structured/cache tokens, CANCEL_RATE, premium pricing, multi-entry batches, and edge cases (NaN, zero, disabled) - Transaction.spec.js: adds describe('Bulk path parity') that mirrors 7 key legacy tests via recordCollectedUsage + bulk deps against real MongoDB, asserting same balance deductions and doc counts * refactor: update llmConfig structure to use modelKwargs for reasoning effort Refactor the llmConfig in getOpenAILLMConfig to store reasoning effort within modelKwargs instead of directly on llmConfig. This change ensures consistency in the configuration structure and improves clarity in the handling of reasoning properties in the tests. * test: update performance checks in processAssistantMessage tests Revise the performance assertions in the processAssistantMessage tests to ensure that each message processing time remains under 100ms, addressing potential ReDoS vulnerabilities. This change enhances the reliability of the tests by focusing on maximum processing time rather than relative ratios. * test: fill parity test gaps — model fallback, abort context, structured edge cases - usage.bulk-parity: add undefined model fallback test - transactions.bulk-parity: add abort context test (txns inserted, balance unchanged when balance not passed), fix readTokens type cast - Transaction.spec: add 3 missing mirrors — balance disabled with transactions enabled, structured transactions disabled, structured balance disabled * fix: deduct balance before inserting transactions to prevent orphaned docs Swap the order in bulkWriteTransactions: updateBalance runs before insertMany. If updateBalance fails (after exhausting retries), no transaction documents are written — avoiding the inconsistent state where transactions exist in MongoDB with no corresponding balance deduction. * chore: import order * test: update config.spec.ts for OpenRouter reasoning in modelKwargs Same fix as llm.spec.ts — OpenRouter reasoning is now passed via modelKwargs instead of llmConfig.reasoning directly.	2026-03-01 12:26:36 -05:00
Danny Avila	7a1d2969b8	🤖 feat: Gemini 3.1 Pricing and Context Window (#11884) ... - Added support for the new Gemini 3.1 models, including 'gemini-3.1-pro-preview' and 'gemini-3.1-pro-preview-customtools'. - Updated pricing logic to apply standard and premium rates based on token usage thresholds for the new models. - Enhanced tests to validate pricing behavior for both standard and premium scenarios. - Modified configuration files to include Gemini 3.1 models in the default model lists and token value mappings. - Updated environment example file to reflect the new model options.	2026-02-20 16:21:32 -05:00
Danny Avila	41e2348d47	🤖 feat: Claude Opus 4.6 - 1M Context, Premium Pricing, Adaptive Thinking (#11670) ... * feat: Implement new features for Claude Opus 4.6 model - Added support for tiered pricing based on input token count for the Claude Opus 4.6 model. - Updated token value calculations to include inputTokenCount for accurate pricing. - Enhanced transaction handling to apply premium rates when input tokens exceed defined thresholds. - Introduced comprehensive tests to validate pricing logic for both standard and premium rates across various scenarios. - Updated related utility functions and models to accommodate new pricing structure. This change improves the flexibility and accuracy of token pricing for the Claude Opus 4.6 model, ensuring users are charged appropriately based on their usage. * feat: Add effort field to conversation and preset schemas - Introduced a new optional `effort` field of type `String` in both the `IPreset` and `IConversation` interfaces. - Updated the `conversationPreset` schema to include the `effort` field, enhancing the data structure for better context management. * chore: Clean up unused variable and comments in initialize function * chore: update dependencies and SDK versions - Updated @anthropic-ai/sdk to version 0.73.0 in package.json and overrides. - Updated @anthropic-ai/vertex-sdk to version 0.14.3 in packages/api/package.json. - Updated @librechat/agents to version 3.1.34 in packages/api/package.json. - Refactored imports in packages/api/src/endpoints/anthropic/vertex.ts for consistency. * chore: remove postcss-loader from dependencies * feat: Bedrock model support for adaptive thinking configuration - Updated .env.example to include new Bedrock model IDs for Claude Opus 4.6. - Refactored bedrockInputParser to support adaptive thinking for Opus models, allowing for dynamic thinking configurations. - Introduced a new function to check model compatibility with adaptive thinking. - Added an optional `effort` field to the input schemas and updated related configurations. - Enhanced tests to validate the new adaptive thinking logic and model configurations. * feat: Add tests for Opus 4.6 adaptive thinking configuration * feat: Update model references for Opus 4.6 by removing version suffix * feat: Update @librechat/agents to version 3.1.35 in package.json and package-lock.json * chore: @librechat/agents to version 3.1.36 in package.json and package-lock.json * feat: Normalize inputTokenCount for spendTokens and enhance transaction handling - Introduced normalization for promptTokens to ensure inputTokenCount does not go negative. - Updated transaction logic to reflect normalized inputTokenCount in pricing calculations. - Added comprehensive tests to validate the new normalization logic and its impact on transaction rates for both standard and premium models. - Refactored related functions to improve clarity and maintainability of token value calculations. * chore: Simplify adaptive thinking configuration in helpers.ts - Removed unnecessary type casting for the thinking property in updatedOptions. - Ensured that adaptive thinking is directly assigned when conditions are met, improving code clarity. * refactor: Replace hard-coded token values with dynamic retrieval from maxTokensMap in model tests * fix: Ensure non-negative token values in spendTokens calculations - Updated token value retrieval to use Math.max for prompt and completion tokens, preventing negative values. - Enhanced clarity in token calculations for both prompt and completion transactions. * test: Add test for normalization of negative structured token values in spendStructuredTokens - Implemented a test to ensure that negative structured token values are normalized to zero during token spending. - Verified that the transaction rates remain consistent with the expected standard values after normalization. * refactor: Bedrock model support for adaptive thinking and context handling - Added tests for various alternate naming conventions of Claude models to validate adaptive thinking and context support. - Refactored `supportsAdaptiveThinking` and `supportsContext1m` functions to utilize new parsing methods for model version extraction. - Updated `bedrockInputParser` to handle effort configurations more effectively and strip unnecessary fields for non-adaptive models. - Improved handling of anthropic model configurations in the input parser. * fix: Improve token value retrieval in getMultiplier function - Updated the token value retrieval logic to use optional chaining for better safety against undefined values. - Added a test case to ensure that the function returns the default rate when the provided valueKey does not exist in tokenValues.	2026-02-06 18:35:36 -05:00
Sebastien Bruel	e95e0052da	🗄️ feat: Allow Skipping Transactions When Balance is Disabled (#9419) ... * Disable transaction creation when balance is disabled * Add configuration to disable transactions creation * chore: remove comments --------- Co-authored-by: Danny Avila <danacordially@gmail.com>	2025-09-05 11:21:02 -04:00
Danny Avila	9a210971f5	🛜 refactor: Streamline App Config Usage (#9234) ... * WIP: app.locals refactoring WIP: appConfig fix: update memory configuration retrieval to use getAppConfig based on user role fix: update comment for AppConfig interface to clarify purpose 🏷️ refactor: Update tests to use getAppConfig for endpoint configurations ci: Update AppService tests to initialize app config instead of app.locals ci: Integrate getAppConfig into remaining tests refactor: Update multer storage destination to use promise-based getAppConfig and improve error handling in tests refactor: Rename initializeAppConfig to setAppConfig and update related tests ci: Mock getAppConfig in various tests to provide default configurations refactor: Update convertMCPToolsToPlugins to use mcpManager for server configuration and adjust related tests chore: rename `Config/getAppConfig` -> `Config/app` fix: streamline OpenAI image tools configuration by removing direct appConfig dependency and using function parameters chore: correct parameter documentation for imageOutputType in ToolService.js refactor: remove `getCustomConfig` dependency in config route refactor: update domain validation to use appConfig for allowed domains refactor: use appConfig registration property chore: remove app parameter from AppService invocation refactor: update AppConfig interface to correct registration and turnstile configurations refactor: remove getCustomConfig dependency and use getAppConfig in PluginController, multer, and MCP services refactor: replace getCustomConfig with getAppConfig in STTService, TTSService, and related files refactor: replace getCustomConfig with getAppConfig in Conversation and Message models, update tempChatRetention functions to use AppConfig type refactor: update getAppConfig calls in Conversation and Message models to include user role for temporary chat expiration ci: update related tests refactor: update getAppConfig call in getCustomConfigSpeech to include user role fix: update appConfig usage to access allowedDomains from actions instead of registration refactor: enhance AppConfig to include fileStrategies and update related file strategy logic refactor: update imports to use normalizeEndpointName from @librechat/api and remove redundant definitions chore: remove deprecated unused RunManager refactor: get balance config primarily from appConfig refactor: remove customConfig dependency for appConfig and streamline loadConfigModels logic refactor: remove getCustomConfig usage and use app config in file citations refactor: consolidate endpoint loading logic into loadEndpoints function refactor: update appConfig access to use endpoints structure across various services refactor: implement custom endpoints configuration and streamline endpoint loading logic refactor: update getAppConfig call to include user role parameter refactor: streamline endpoint configuration and enhance appConfig usage across services refactor: replace getMCPAuthMap with getUserMCPAuthMap and remove unused getCustomConfig file refactor: add type annotation for loadedEndpoints in loadEndpoints function refactor: move /services/Files/images/parse to TS API chore: add missing FILE_CITATIONS permission to IRole interface refactor: restructure toolkits to TS API refactor: separate manifest logic into its own module refactor: consolidate tool loading logic into a new tools module for startup logic refactor: move interface config logic to TS API refactor: migrate checkEmailConfig to TypeScript and update imports refactor: add FunctionTool interface and availableTools to AppConfig refactor: decouple caching and DB operations from AppService, make part of consolidated `getAppConfig` WIP: fix tests * fix: rebase conflicts * refactor: remove app.locals references * refactor: replace getBalanceConfig with getAppConfig in various strategies and middleware * refactor: replace appConfig?.balance with getBalanceConfig in various controllers and clients * test: add balance configuration to titleConvo method in AgentClient tests * chore: remove unused `openai-chat-tokens` package * chore: remove unused imports in initializeMCPs.js * refactor: update balance configuration to use getAppConfig instead of getBalanceConfig * refactor: integrate configMiddleware for centralized configuration handling * refactor: optimize email domain validation by removing unnecessary async calls * refactor: simplify multer storage configuration by removing async calls * refactor: reorder imports for better readability in user.js * refactor: replace getAppConfig calls with req.config for improved performance * chore: replace getAppConfig calls with req.config in tests for centralized configuration handling * chore: remove unused override config * refactor: add configMiddleware to endpoint route and replace getAppConfig with req.config * chore: remove customConfig parameter from TTSService constructor * refactor: pass appConfig from request to processFileCitations for improved configuration handling * refactor: remove configMiddleware from endpoint route and retrieve appConfig directly in getEndpointsConfig if not in `req.config` * test: add mockAppConfig to processFileCitations tests for improved configuration handling * fix: pass req.config to hasCustomUserVars and call without await after synchronous refactor * fix: type safety in useExportConversation * refactor: retrieve appConfig using getAppConfig in PluginController and remove configMiddleware from plugins route, to avoid always retrieving when plugins are cached * chore: change `MongoUser` typedef to `IUser` * fix: Add `user` and `config` fields to ServerRequest and update JSDoc type annotations from Express.Request to ServerRequest * fix: remove unused setAppConfig mock from Server configuration tests	2025-08-26 12:10:18 -04:00
Danny Avila	a2fc7d312a	🏗️ refactor: Extract DB layers to data-schemas for shared use (#7650) ... * refactor: move model definitions and database-related methods to packages/data-schemas * ci: update tests due to new DB structure fix: disable mocking `librechat-data-provider` feat: Add schema exports to data-schemas package - Introduced a new schema module that exports various schemas including action, agent, and user schemas. - Updated index.ts to include the new schema exports for better modularity and organization. ci: fix appleStrategy tests fix: Agent.spec.js ci: refactor handleTools tests to use MongoMemoryServer for in-memory database fix: getLogStores imports ci: update banViolation tests to use MongoMemoryServer and improve session mocking test: refactor samlStrategy tests to improve mock configurations and user handling ci: fix crypto mock in handleText tests for improved accuracy ci: refactor spendTokens tests to improve model imports and setup ci: refactor Message model tests to use MongoMemoryServer and improve database interactions * refactor: streamline IMessage interface and move feedback properties to types/message.ts * refactor: use exported initializeRoles from `data-schemas`, remove api workspace version (this serves as an example of future migrations that still need to happen) * refactor: update model imports to use destructuring from `~/db/models` for consistency and clarity * refactor: remove unused mongoose imports from model files for cleaner code * refactor: remove unused mongoose imports from Share, Prompt, and Transaction model files for cleaner code * refactor: remove unused import in Transaction model for cleaner code * ci: update deploy workflow to reference new Docker Dev Branch Images Build and add new workflow for building Docker images on dev branch * chore: cleanup imports	2025-05-30 22:18:13 -04:00
Ruben Talstra	3a62a2633d	💵 feat: Add Automatic Balance Refill (#6452) ... * 🚀 feat: Add automatic refill settings to balance schema * 🚀 feat: Refactor balance feature to use global interface configuration * 🚀 feat: Implement auto-refill functionality for balance management * 🚀 feat: Enhance auto-refill logic and configuration for balance management * 🚀 chore: Bump version to 0.7.74 in package.json and package-lock.json * 🚀 chore: Bump version to 0.0.5 in package.json and package-lock.json * 🚀 docs: Update comment for balance settings in librechat.example.yaml * chore: space in `.env.example` * 🚀 feat: Implement balance configuration loading and refactor related components * 🚀 test: Refactor tests to use custom config for balance feature * 🚀 fix: Update balance response handling in Transaction.js to use Balance model * 🚀 test: Update AppService tests to include balance configuration in mock setup * 🚀 test: Enhance AppService tests with complete balance configuration scenarios * 🚀 refactor: Rename balanceConfig to balance and update related tests for clarity * 🚀 refactor: Remove loadDefaultBalance and update balance handling in AppService * 🚀 test: Update AppService tests to reflect new balance structure and defaults * 🚀 test: Mock getCustomConfig in BaseClient tests to control balance configuration * 🚀 test: Add get method to mockCache in OpenAIClient tests for improved cache handling * 🚀 test: Mock getCustomConfig in OpenAIClient tests to control balance configuration * 🚀 test: Remove mock for getCustomConfig in OpenAIClient tests to streamline configuration handling * 🚀 fix: Update balance configuration reference in config.js for consistency * refactor: Add getBalanceConfig function to retrieve balance configuration * chore: Comment out example balance settings in librechat.example.yaml * refactor: Replace getCustomConfig with getBalanceConfig for balance handling * fix: tests * refactor: Replace getBalanceConfig call with balance from request locals * refactor: Update balance handling to use environment variables for configuration * refactor: Replace getBalanceConfig calls with balance from request locals * refactor: Simplify balance configuration logic in getBalanceConfig --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2025-03-21 17:48:11 -04:00
Danny Avila	24cad6bbd4	🤖 feat: Support Google Agents, fix Various Provider Configurations (#5126) ... * feat: Refactor ModelEndHandler to collect usage metadata only if it exists * feat: google tool end handling, custom anthropic class for better token ux * refactor: differentiate between client <> request options * feat: initial support for google agents * feat: only cache messages with non-empty text * feat: Cache non-empty messages in chatV2 controller * fix: anthropic llm client options llmConfig * refactor: streamline client options handling in LLM configuration * fix: VertexAI Agent Auth & Tool Handling * fix: additional fields for llmConfig, however customHeaders are not supported by langchain, requires PR * feat: set default location for vertexai LLM configuration * fix: outdated OpenAI Client options for getLLMConfig * chore: agent provider options typing * chore: add note about currently unsupported customHeaders in langchain GenAI client * fix: skip transaction creation when rawAmount is NaN	2024-12-28 17:15:03 -05:00
Danny Avila	d54458b3a6	🧮 feat: Improve structured token spending and testing; fix: Anthropic Cache Spend (#3766) ... * chore: update jest and mongodb-memory-server dependencies * fix: Anthropic edge case of increasing balance * refactor: Update token calculations in Transaction model/spec * refactor: `spendTokens` always record transactions, add Tx related tests * feat: Add error handling for CHECK_BALANCE environment variable * feat: Update import path for Balance model in Balance controller * chore: remove logging * refactor: Improve structured token spend logging in spendTokens.js * ci: add unit tests for spend token * ci: Improve structured token spend unit testing * chore: improve logging phrase for no tx spent from balance	2024-08-24 04:36:08 -04:00