mirror of
https://github.com/danny-avila/LibreChat.git
synced 2025-12-21 19:00:13 +01:00
a45b384bbc
* wip: initial cache control implementation, add typing for transactions handling * feat: first pass of Anthropic Prompt Caching * feat: standardize stream usage as pass in when calculating token counts * feat: Add getCacheMultiplier function to calculate cache multiplier for different valueKeys and cacheTypes * chore: imports order * refactor: token usage recording in AnthropicClient, no need to "correct" as we have the correct amount * feat: more accurate token counting using stream usage data * feat: Improve token counting accuracy with stream usage data * refactor: ensure more accurate than not token estimations if custom instructions or files are not being resent with every request * refactor: cleanup updateUserMessageTokenCount to allow transactions to be as accurate as possible even if we shouldn't update user message token counts * ci: fix tests |
||
|---|---|---|
| .. | ||
| assistants | ||
| limiters | ||
| roles | ||
| spec | ||
| validate | ||
| abortControllers.js | ||
| abortMiddleware.js | ||
| abortRun.js | ||
| buildEndpointOption.js | ||
| canDeleteAccount.js | ||
| checkBan.js | ||
| checkDomainAllowed.js | ||
| concurrentLimiter.js | ||
| denyRequest.js | ||
| enforceModelSpec.js | ||
| enforceModelSpec.spec.js | ||
| index.js | ||
| moderateText.js | ||
| noIndex.js | ||
| requireJwtAuth.js | ||
| requireLdapAuth.js | ||
| requireLocalAuth.js | ||
| setHeaders.js | ||
| uaParser.js | ||
| validateEndpoint.js | ||
| validateImageRequest.js | ||
| validateMessageReq.js | ||
| validateModel.js | ||
| validatePasswordReset.js | ||
| validateRegistration.js | ||