Commit graph

3 commits

Author SHA1 Message Date
Marco Beretta
a06e999dd6
cleanup: code formatting and improve readability across multiple components 2025-05-31 20:48:23 +02:00
Danny Avila
8ca1e4f94f
🐛 fix: Anthropic Prompt Caching Edge Case (#3690) 2024-08-18 19:03:03 -04:00
Danny Avila
a45b384bbc
💾 feat: Anthropic Prompt Caching (#3670)
* wip: initial cache control implementation, add typing for transactions handling

* feat: first pass of Anthropic Prompt Caching

* feat: standardize stream usage as pass in when calculating token counts

* feat: Add getCacheMultiplier function to calculate cache multiplier for different valueKeys and cacheTypes

* chore: imports order

* refactor: token usage recording in AnthropicClient, no need to "correct" as we have the correct amount

* feat: more accurate token counting using stream usage data

* feat: Improve token counting accuracy with stream usage data

* refactor: ensure more accurate than not token estimations if custom instructions or files are not being resent with every request

* refactor: cleanup updateUserMessageTokenCount to allow transactions to be as accurate as possible even if we shouldn't update user message token counts

* ci: fix tests
2024-08-17 03:24:09 -04:00