Andreas/LibreChat
1
0
Fork 0
mirror of https://github.com/danny-avila/LibreChat.git synced 2026-01-23 10:46:12 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

🪙 refactor: Collected Usage & Anthropic Prompt Caching (#11319)
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Details
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
Details
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Waiting to run
Details
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Waiting to run
Details
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Details
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
Details

Browse source 
* 🔧 refactor: Improve token calculation in AgentClient.recordCollectedUsage

- Updated the token calculation logic to sum output tokens directly from all entries, addressing issues with negative values in parallel execution scenarios.
- Added comments for clarity on the usage of input tokens and output tokens.
- Introduced a new test file for comprehensive testing of the recordCollectedUsage function, covering various execution scenarios including sequential and parallel processing, cache token handling, and model fallback logic.

* 🔧 refactor: Anthropic `promptCache` handling in LLM configuration

* 🔧 test: Add comprehensive test for cache token handling in recordCollectedUsage

- Introduced a new test case to validate the handling of cache tokens across multiple tool calls in the recordCollectedUsage function.
- Ensured correct calculations for input and output tokens, including scenarios with cache creation and reading.
- Verified the expected interactions with token spending methods to enhance the robustness of the token management logic.
This commit is contained in:
Danny Avila 2026-01-12 23:02:08 -05:00 • committed by GitHub
parent 1329e16d3a
commit 2a50c372ef
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 828 additions and 40 deletions
Download patch file Download diff file Expand all files Collapse all files

2
api/package.json
View file

@ -46,7 +46,7 @@
"@googleapis/youtube": "^20.0.0",
"@keyv/redis": "^4.3.3",
"@langchain/core": "^0.3.80",
"@librechat/agents": "^3.0.66",
"@librechat/agents": "^3.0.77",
"@librechat/api": "*",
"@librechat/data-schemas": "*",
"@microsoft/microsoft-graph-client": "^3.0.7",