mirror of
https://github.com/danny-avila/LibreChat.git
synced 2025-12-19 01:40:15 +01:00
🪨 feat: AWS Bedrock support (#3935)
* feat: Add BedrockIcon component to SVG library * feat: EModelEndpoint.bedrock * feat: first pass, bedrock chat. note: AgentClient is returning `agents` as conversation.endpoint * fix: declare endpoint in initialization step * chore: Update @librechat/agents dependency to version 1.4.5 * feat: backend content aggregation for agents/bedrock * feat: abort agent requests * feat: AWS Bedrock icons * WIP: agent provider schema parsing * chore: Update EditIcon props type * refactor(useGenerationsByLatest): make agents and bedrock editable * refactor: non-assistant message content, parts * fix: Bedrock response `sender` * fix: use endpointOption.model_parameters not endpointOption.modelOptions * fix: types for step handler * refactor: Update Agents.ToolCallDelta type * refactor: Remove unnecessary assignment of parentMessageId in AskController * refactor: remove unnecessary assignment of parentMessageId (agent request handler) * fix(bedrock/agents): message regeneration * refactor: dynamic form elements using react-hook-form Controllers * fix: agent icons/labels for messages * fix: agent actions * fix: use of new dynamic tags causing application crash * refactor: dynamic settings touch-ups * refactor: update Slider component to allow custom track class name * refactor: update DynamicSlider component styles * refactor: use Constants value for GLOBAL_PROJECT_NAME (enum) * feat: agent share global methods/controllers * fix: agents query * fix: `getResponseModel` * fix: share prompt a11y issue * refactor: update SharePrompt dialog theme styles * refactor: explicit typing for SharePrompt * feat: add agent roles/permissions * chore: update @librechat/agents dependency to version 1.4.7 for tool_call_ids edge case * fix(Anthropic): messages.X.content.Y.tool_use.input: Input should be a valid dictionary * fix: handle text parts with tool_call_ids and empty text * fix: role initialization * refactor: don't make instructions required * refactor: improve typing of Text part * fix: setShowStopButton for agents route * chore: remove params for now * fix: add streamBuffer and streamRate to help prevent 'Overloaded' errors from Anthropic API * refactor: remove console.log statement in ContentRender component * chore: typing, rename Context to Delete Button * chore(DeleteButton): logging * refactor(Action): make accessible * style(Action): improve a11y again * refactor: remove use/mention of mongoose sessions * feat: first pass, sharing agents * feat: visual indicator for global agent, remove author when serving to non-author * wip: params * chore: fix typing issues * fix(schemas): typing * refactor: improve accessibility of ListCard component and fix console React warning * wip: reset templates for non-legacy new convos * Revert "wip: params" This reverts commitf8067e91d4. * Revert "refactor: dynamic form elements using react-hook-form Controllers" This reverts commit2150c4815d. * fix(Parameters): types and parameter effect update to only update local state to parameters * refactor: optimize useDebouncedInput hook for better performance * feat: first pass, anthropic bedrock params * chore: paramEndpoints check for endpointType too * fix: maxTokens to use coerceNumber.optional(), * feat: extra chat model params * chore: reduce code repetition * refactor: improve preset title handling in SaveAsPresetDialog component * refactor: improve preset handling in HeaderOptions component * chore: improve typing, replace legacy dialog for SaveAsPresetDialog * feat: save as preset from parameters panel * fix: multi-search in select dropdown when using Option type * refactor: update default showDefault value to false in Dynamic components * feat: Bedrock presets settings * chore: config, fix agents schema, update config version * refactor: update AWS region variable name in bedrock options endpoint to BEDROCK_AWS_DEFAULT_REGION * refactor: update baseEndpointSchema in config.ts to include baseURL property * refactor: update createRun function to include req parameter and set streamRate based on provider * feat: availableRegions via config * refactor: remove unused demo agent controller file * WIP: title * Update @librechat/agents to version 1.5.0 * chore: addTitle.js to handle empty responseText * feat: support images and titles * feat: context token updates * Refactor BaseClient test to use expect.objectContaining * refactor: add model select, remove header options params, move side panel params below prompts * chore: update models list, catch title error * feat: model service for bedrock models (env) * chore: Remove verbose debug log in AgentClient class following stream * feat(bedrock): track token spend; fix: token rates, value key mapping for AWS models * refactor: handle streamRate in `handleLLMNewToken` callback * chore: AWS Bedrock example config in `.env.example` * refactor: Rename bedrockMeta to bedrockGeneral in settings.ts and use for AI21 and Amazon Bedrock providers * refactor: Update `.env.example` with AWS Bedrock model IDs URL and additional notes * feat: titleModel support for bedrock * refactor: Update `.env.example` with additional notes for AWS Bedrock model IDs
This commit is contained in:
Danny Avila
GitHub
parent
8c14360263
commit
d59b62174f
134 changed files with 3684 additions and 1213 deletions
|
|
@ -142,10 +142,19 @@ export const defaultAssistantsVersion = {
|
|||
|
||||
export const baseEndpointSchema = z.object({
|
||||
streamRate: z.number().optional(),
|
||||
baseURL: z.string().optional(),
|
||||
titlePrompt: z.string().optional(),
|
||||
titleModel: z.string().optional(),
|
||||
});
|
||||
|
||||
export type TBaseEndpoint = z.infer<typeof baseEndpointSchema>;
|
||||
|
||||
export const bedrockEndpointSchema = baseEndpointSchema.merge(
|
||||
z.object({
|
||||
availableRegions: z.array(z.string()).optional(),
|
||||
}),
|
||||
);
|
||||
|
||||
export const assistantEndpointSchema = baseEndpointSchema.merge(
|
||||
z.object({
|
||||
/* assistants specific */
|
||||
|
|
@ -169,7 +178,6 @@ export const assistantEndpointSchema = baseEndpointSchema.merge(
|
|||
]),
|
||||
/* general */
|
||||
apiKey: z.string().optional(),
|
||||
baseURL: z.string().optional(),
|
||||
models: z
|
||||
.object({
|
||||
default: z.array(z.string()).min(1),
|
||||
|
|
@ -179,7 +187,6 @@ export const assistantEndpointSchema = baseEndpointSchema.merge(
|
|||
.optional(),
|
||||
titleConvo: z.boolean().optional(),
|
||||
titleMethod: z.union([z.literal('completion'), z.literal('functions')]).optional(),
|
||||
titleModel: z.string().optional(),
|
||||
headers: z.record(z.any()).optional(),
|
||||
}),
|
||||
);
|
||||
|
|
@ -187,43 +194,39 @@ export const assistantEndpointSchema = baseEndpointSchema.merge(
|
|||
export type TAssistantEndpoint = z.infer<typeof assistantEndpointSchema>;
|
||||
|
||||
export const agentsEndpointSChema = baseEndpointSchema.merge(
|
||||
baseEndpointSchema.merge(
|
||||
z.object({
|
||||
/* assistants specific */
|
||||
disableBuilder: z.boolean().optional(),
|
||||
pollIntervalMs: z.number().optional(),
|
||||
timeoutMs: z.number().optional(),
|
||||
version: z.union([z.string(), z.number()]).default(2),
|
||||
supportedIds: z.array(z.string()).min(1).optional(),
|
||||
excludedIds: z.array(z.string()).min(1).optional(),
|
||||
privateAssistants: z.boolean().optional(),
|
||||
retrievalModels: z.array(z.string()).min(1).optional().default(defaultRetrievalModels),
|
||||
capabilities: z
|
||||
.array(z.nativeEnum(Capabilities))
|
||||
.optional()
|
||||
.default([
|
||||
Capabilities.code_interpreter,
|
||||
Capabilities.image_vision,
|
||||
Capabilities.retrieval,
|
||||
Capabilities.actions,
|
||||
Capabilities.tools,
|
||||
]),
|
||||
/* general */
|
||||
apiKey: z.string().optional(),
|
||||
baseURL: z.string().optional(),
|
||||
models: z
|
||||
.object({
|
||||
default: z.array(z.string()).min(1),
|
||||
fetch: z.boolean().optional(),
|
||||
userIdQuery: z.boolean().optional(),
|
||||
})
|
||||
.optional(),
|
||||
titleConvo: z.boolean().optional(),
|
||||
titleMethod: z.union([z.literal('completion'), z.literal('functions')]).optional(),
|
||||
titleModel: z.string().optional(),
|
||||
headers: z.record(z.any()).optional(),
|
||||
}),
|
||||
),
|
||||
z.object({
|
||||
/* assistants specific */
|
||||
disableBuilder: z.boolean().optional(),
|
||||
pollIntervalMs: z.number().optional(),
|
||||
timeoutMs: z.number().optional(),
|
||||
version: z.union([z.string(), z.number()]).default(2),
|
||||
supportedIds: z.array(z.string()).min(1).optional(),
|
||||
excludedIds: z.array(z.string()).min(1).optional(),
|
||||
privateAssistants: z.boolean().optional(),
|
||||
retrievalModels: z.array(z.string()).min(1).optional().default(defaultRetrievalModels),
|
||||
capabilities: z
|
||||
.array(z.nativeEnum(Capabilities))
|
||||
.optional()
|
||||
.default([
|
||||
Capabilities.code_interpreter,
|
||||
Capabilities.image_vision,
|
||||
Capabilities.retrieval,
|
||||
Capabilities.actions,
|
||||
Capabilities.tools,
|
||||
]),
|
||||
/* general */
|
||||
apiKey: z.string().optional(),
|
||||
models: z
|
||||
.object({
|
||||
default: z.array(z.string()).min(1),
|
||||
fetch: z.boolean().optional(),
|
||||
userIdQuery: z.boolean().optional(),
|
||||
})
|
||||
.optional(),
|
||||
titleConvo: z.boolean().optional(),
|
||||
titleMethod: z.union([z.literal('completion'), z.literal('functions')]).optional(),
|
||||
headers: z.record(z.any()).optional(),
|
||||
}),
|
||||
);
|
||||
|
||||
export type TAgentsEndpoint = z.infer<typeof agentsEndpointSChema>;
|
||||
|
|
@ -244,7 +247,6 @@ export const endpointSchema = baseEndpointSchema.merge(
|
|||
}),
|
||||
titleConvo: z.boolean().optional(),
|
||||
titleMethod: z.union([z.literal('completion'), z.literal('functions')]).optional(),
|
||||
titleModel: z.string().optional(),
|
||||
summarize: z.boolean().optional(),
|
||||
summaryModel: z.string().optional(),
|
||||
forcePrompt: z.boolean().optional(),
|
||||
|
|
@ -500,6 +502,7 @@ export const configSchema = z.object({
|
|||
[EModelEndpoint.assistants]: assistantEndpointSchema.optional(),
|
||||
[EModelEndpoint.agents]: agentsEndpointSChema.optional(),
|
||||
[EModelEndpoint.custom]: z.array(endpointSchema.partial()).optional(),
|
||||
[EModelEndpoint.bedrock]: baseEndpointSchema.optional(),
|
||||
})
|
||||
.strict()
|
||||
.refine((data) => Object.keys(data).length > 0, {
|
||||
|
|
@ -552,6 +555,7 @@ export const defaultEndpoints: EModelEndpoint[] = [
|
|||
EModelEndpoint.google,
|
||||
EModelEndpoint.anthropic,
|
||||
EModelEndpoint.custom,
|
||||
EModelEndpoint.bedrock,
|
||||
];
|
||||
|
||||
export const alternateName = {
|
||||
|
|
@ -566,6 +570,7 @@ export const alternateName = {
|
|||
[EModelEndpoint.google]: 'Google',
|
||||
[EModelEndpoint.anthropic]: 'Anthropic',
|
||||
[EModelEndpoint.custom]: 'Custom',
|
||||
[EModelEndpoint.bedrock]: 'AWS Bedrock',
|
||||
};
|
||||
|
||||
const sharedOpenAIModels = [
|
||||
|
|
@ -588,6 +593,52 @@ const sharedOpenAIModels = [
|
|||
'gpt-3.5-turbo-0613',
|
||||
];
|
||||
|
||||
const sharedAnthropicModels = [
|
||||
'claude-3-5-sonnet-20240620',
|
||||
'claude-3-opus-20240229',
|
||||
'claude-3-sonnet-20240229',
|
||||
'claude-3-haiku-20240307',
|
||||
'claude-2.1',
|
||||
'claude-2',
|
||||
'claude-1.2',
|
||||
'claude-1',
|
||||
'claude-1-100k',
|
||||
'claude-instant-1',
|
||||
'claude-instant-1-100k',
|
||||
];
|
||||
|
||||
export const bedrockModels = [
|
||||
'anthropic.claude-3-5-sonnet-20240620-v1:0',
|
||||
'anthropic.claude-3-haiku-20240307-v1:0',
|
||||
'anthropic.claude-3-opus-20240229-v1:0',
|
||||
'anthropic.claude-3-sonnet-20240229-v1:0',
|
||||
'anthropic.claude-v2',
|
||||
'anthropic.claude-v2:1',
|
||||
'anthropic.claude-instant-v1',
|
||||
'cohere.command-text-v14',
|
||||
'cohere.command-light-text-v14',
|
||||
'cohere.command-r-v1:0',
|
||||
'cohere.command-r-plus-v1:0',
|
||||
'meta.llama2-13b-chat-v1',
|
||||
'meta.llama2-70b-chat-v1',
|
||||
'meta.llama3-8b-instruct-v1:0',
|
||||
'meta.llama3-70b-instruct-v1:0',
|
||||
'meta.llama3-1-8b-instruct-v1:0',
|
||||
'meta.llama3-1-70b-instruct-v1:0',
|
||||
'meta.llama3-1-405b-instruct-v1:0',
|
||||
'mistral.mistral-7b-instruct-v0:2',
|
||||
'mistral.mixtral-8x7b-instruct-v0:1',
|
||||
'mistral.mistral-large-2402-v1:0',
|
||||
'mistral.mistral-large-2407-v1:0',
|
||||
'mistral.mistral-small-2402-v1:0',
|
||||
'ai21.jamba-instruct-v1:0',
|
||||
// 'ai21.j2-mid-v1', // no streaming
|
||||
// 'ai21.j2-ultra-v1', no conversation history
|
||||
'amazon.titan-text-lite-v1',
|
||||
'amazon.titan-text-express-v1',
|
||||
'amazon.titan-text-premier-v1:0',
|
||||
];
|
||||
|
||||
export const defaultModels = {
|
||||
[EModelEndpoint.azureAssistants]: sharedOpenAIModels,
|
||||
[EModelEndpoint.assistants]: ['chatgpt-4o-latest', ...sharedOpenAIModels],
|
||||
|
|
@ -606,19 +657,7 @@ export const defaultModels = {
|
|||
'code-bison',
|
||||
'code-bison-32k',
|
||||
],
|
||||
[EModelEndpoint.anthropic]: [
|
||||
'claude-3-5-sonnet-20240620',
|
||||
'claude-3-opus-20240229',
|
||||
'claude-3-sonnet-20240229',
|
||||
'claude-3-haiku-20240307',
|
||||
'claude-2.1',
|
||||
'claude-2',
|
||||
'claude-1.2',
|
||||
'claude-1',
|
||||
'claude-1-100k',
|
||||
'claude-instant-1',
|
||||
'claude-instant-1-100k',
|
||||
],
|
||||
[EModelEndpoint.anthropic]: sharedAnthropicModels,
|
||||
[EModelEndpoint.openAI]: [
|
||||
'chatgpt-4o-latest',
|
||||
...sharedOpenAIModels,
|
||||
|
|
@ -626,6 +665,7 @@ export const defaultModels = {
|
|||
'gpt-3.5-turbo-instruct-0914',
|
||||
'gpt-3.5-turbo-instruct',
|
||||
],
|
||||
[EModelEndpoint.bedrock]: bedrockModels,
|
||||
};
|
||||
|
||||
const fitlerAssistantModels = (str: string) => {
|
||||
|
|
@ -645,6 +685,7 @@ export const initialModelsConfig: TModelsConfig = {
|
|||
[EModelEndpoint.chatGPTBrowser]: ['text-davinci-002-render-sha'],
|
||||
[EModelEndpoint.google]: defaultModels[EModelEndpoint.google],
|
||||
[EModelEndpoint.anthropic]: defaultModels[EModelEndpoint.anthropic],
|
||||
[EModelEndpoint.bedrock]: defaultModels[EModelEndpoint.bedrock],
|
||||
};
|
||||
|
||||
export const EndpointURLs: { [key in EModelEndpoint]: string } = {
|
||||
|
|
@ -658,7 +699,8 @@ export const EndpointURLs: { [key in EModelEndpoint]: string } = {
|
|||
[EModelEndpoint.chatGPTBrowser]: `/api/ask/${EModelEndpoint.chatGPTBrowser}`,
|
||||
[EModelEndpoint.azureAssistants]: '/api/assistants/v1/chat',
|
||||
[EModelEndpoint.assistants]: '/api/assistants/v2/chat',
|
||||
[EModelEndpoint.agents]: '/api/agents/chat',
|
||||
[EModelEndpoint.agents]: `/api/${EModelEndpoint.agents}/chat`,
|
||||
[EModelEndpoint.bedrock]: `/api/${EModelEndpoint.bedrock}/chat`,
|
||||
};
|
||||
|
||||
export const modularEndpoints = new Set<EModelEndpoint | string>([
|
||||
|
|
@ -668,6 +710,13 @@ export const modularEndpoints = new Set<EModelEndpoint | string>([
|
|||
EModelEndpoint.openAI,
|
||||
EModelEndpoint.azureOpenAI,
|
||||
EModelEndpoint.custom,
|
||||
EModelEndpoint.agents,
|
||||
EModelEndpoint.bedrock,
|
||||
]);
|
||||
|
||||
export const paramEndpoints = new Set<EModelEndpoint | string>([
|
||||
EModelEndpoint.agents,
|
||||
EModelEndpoint.bedrock,
|
||||
]);
|
||||
|
||||
export const supportsBalanceCheck = {
|
||||
|
|
@ -679,6 +728,7 @@ export const supportsBalanceCheck = {
|
|||
[EModelEndpoint.agents]: true,
|
||||
[EModelEndpoint.azureAssistants]: true,
|
||||
[EModelEndpoint.azureOpenAI]: true,
|
||||
[EModelEndpoint.bedrock]: true,
|
||||
};
|
||||
|
||||
export const visionModels = [
|
||||
|
|
@ -1007,7 +1057,7 @@ export enum Constants {
|
|||
/** Key for the app's version. */
|
||||
VERSION = 'v0.7.5-rc1',
|
||||
/** Key for the Custom Config's version (librechat.yaml). */
|
||||
CONFIG_VERSION = '1.1.6',
|
||||
CONFIG_VERSION = '1.1.7',
|
||||
/** Standard value for the first message's `parentMessageId` value, to indicate no parent exists. */
|
||||
NO_PARENT = '00000000-0000-0000-0000-000000000000',
|
||||
/** Standard value for the initial conversationId before a request is sent */
|
||||
|
|
@ -1026,6 +1076,8 @@ export enum Constants {
|
|||
SAVED_TAG = 'Saved',
|
||||
/** Max number of Conversation starters for Agents/Assistants */
|
||||
MAX_CONVO_STARTERS = 4,
|
||||
/** Global/instance Project Name */
|
||||
GLOBAL_PROJECT_NAME = 'instance',
|
||||
}
|
||||
|
||||
export enum LocalStorageKeys {
|
||||
|
|
@ -1105,6 +1157,7 @@ export enum SystemCategories {
|
|||
|
||||
export const providerEndpointMap = {
|
||||
[EModelEndpoint.openAI]: EModelEndpoint.openAI,
|
||||
[EModelEndpoint.bedrock]: EModelEndpoint.bedrock,
|
||||
[EModelEndpoint.azureOpenAI]: EModelEndpoint.openAI,
|
||||
[EModelEndpoint.anthropic]: EModelEndpoint.anthropic,
|
||||
};
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue