🪙 refactor: Collected Usage & Anthropic Prompt Caching (#11319)
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Waiting to run
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions

* 🔧 refactor: Improve token calculation in AgentClient.recordCollectedUsage

- Updated the token calculation logic to sum output tokens directly from all entries, addressing issues with negative values in parallel execution scenarios.
- Added comments for clarity on the usage of input tokens and output tokens.
- Introduced a new test file for comprehensive testing of the recordCollectedUsage function, covering various execution scenarios including sequential and parallel processing, cache token handling, and model fallback logic.

* 🔧 refactor: Anthropic `promptCache` handling in LLM configuration

* 🔧 test: Add comprehensive test for cache token handling in recordCollectedUsage

- Introduced a new test case to validate the handling of cache tokens across multiple tool calls in the recordCollectedUsage function.
- Ensured correct calculations for input and output tokens, including scenarios with cache creation and reading.
- Verified the expected interactions with token spending methods to enhance the robustness of the token management logic.
This commit is contained in:
Danny Avila 2026-01-12 23:02:08 -05:00 committed by GitHub
parent 1329e16d3a
commit 2a50c372ef
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 828 additions and 40 deletions

17
package-lock.json generated
View file

@ -60,7 +60,7 @@
"@googleapis/youtube": "^20.0.0",
"@keyv/redis": "^4.3.3",
"@langchain/core": "^0.3.80",
"@librechat/agents": "^3.0.66",
"@librechat/agents": "^3.0.77",
"@librechat/api": "*",
"@librechat/data-schemas": "*",
"@microsoft/microsoft-graph-client": "^3.0.7",
@ -12660,9 +12660,9 @@
}
},
"node_modules/@librechat/agents": {
"version": "3.0.66",
"resolved": "https://registry.npmjs.org/@librechat/agents/-/agents-3.0.66.tgz",
"integrity": "sha512-JpQo7w+/yLM3dJ46lyGrm4gPTjiHERwcpojw7drvpYWqOU4e2jmjK0JbNxQ0jP00q+nDhPG+mqJ2qQU7TVraOQ==",
"version": "3.0.77",
"resolved": "https://registry.npmjs.org/@librechat/agents/-/agents-3.0.77.tgz",
"integrity": "sha512-Wr9d8bjJAQSl03nEgnAPG6jBQT1fL3sNV3TFDN1FvFQt6WGfdok838Cbcn+/tSGXSPJcICTxNkMT7VN8P6bCPw==",
"license": "MIT",
"dependencies": {
"@langchain/anthropic": "^0.3.26",
@ -12686,6 +12686,7 @@
"https-proxy-agent": "^7.0.6",
"mathjs": "^15.1.0",
"nanoid": "^3.3.7",
"okapibm25": "^1.4.1",
"openai": "5.8.2"
},
"engines": {
@ -34310,6 +34311,12 @@
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/okapibm25": {
"version": "1.4.1",
"resolved": "https://registry.npmjs.org/okapibm25/-/okapibm25-1.4.1.tgz",
"integrity": "sha512-UHmeH4MAtZXGFVncwbY7pfFvDVNxpsyM3W66aGPU0SHj1+ld59ty+9lJ0ifcrcnPUl1XdYoDgb06ObyCnpTs3g==",
"license": "MIT"
},
"node_modules/ollama": {
"version": "0.5.18",
"resolved": "https://registry.npmjs.org/ollama/-/ollama-0.5.18.tgz",
@ -43169,7 +43176,7 @@
"@google/genai": "^1.19.0",
"@keyv/redis": "^4.3.3",
"@langchain/core": "^0.3.80",
"@librechat/agents": "^3.0.66",
"@librechat/agents": "^3.0.77",
"@librechat/data-schemas": "*",
"@modelcontextprotocol/sdk": "^1.25.2",
"@smithy/node-http-handler": "^4.4.5",