Andreas/LibreChat
1
0
Fork 0
mirror of https://github.com/danny-avila/LibreChat.git synced 2026-01-19 16:56:12 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

🚧 chore: merge latest dev build to main repo (#3844)

Browse source 
* agents - phase 1 (#30)

* chore: copy assistant files

* feat: frontend and data-provider

* feat: backend get endpoint test

* fix(MessageEndpointIcon): switched to AgentName and AgentAvatar

* fix: small fixes

* fix: agent endpoint config

* fix: show Agent Builder

* chore: install agentus

* chore: initial scaffolding for agents

* fix: updated Assistant logic to Agent Logic for some Agent components

* WIP first pass, demo of agent package

* WIP: initial backend infra for agents

* fix: agent list error

* wip: agents routing

* chore: Refactor useSSE hook to handle different data events

* wip: correctly emit events

* chore: Update @librechat/agentus npm dependency to version 1.0.9

* remove comment

* first pass: streaming agent text

* chore: Remove @librechat/agentus root-level workspace npm dependency

* feat: Agent Schema and Model

* fix: content handling fixes

* fix: content message save

* WIP: new content data

* fix: run step issue with tool calls

* chore: Update @librechat/agentus npm dependency to version 1.1.5

* feat: update controller and agent routes

* wip: initial backend tool and tool error handling support

* wip: tool chunks

* chore: Update @librechat/agentus npm dependency to version 1.1.7

* chore: update tool_call typing, add test conditions and logs

* fix: create agent

* fix: create agent

* first pass: render completed content parts

* fix: remove logging, fix step handler typing

* chore: Update @librechat/agentus npm dependency to version 1.1.9

* refactor: cleanup maps on unmount

* chore: Update BaseClient.js to safely count tokens for string, number, and boolean values

* fix: support subsequent messages with tool_calls

* chore: export order

* fix: select agent

* fix: tool call types and handling

* chore: switch to anthropic for testing

* fix: AgentSelect

* refactor: experimental: OpenAIClient to use array for intermediateReply

* fix(useSSE): revert old condition for streaming legacy client tokens

* fix: lint

* revert `agent_id` to `id`

* chore: update localization keys for agent-related components

* feat: zod schema handling for actions

* refactor(actions): if no params, no zodSchema

* chore: Update @librechat/agentus npm dependency to version 1.2.1

* feat: first pass, actions

* refactor: empty schema for actions without params

* feat: Update createRun function to accept additional options

* fix: message payload formatting; feat: add more client options

* fix: ToolCall component rendering when action has no args but has output

* refactor(ToolCall): allow non-stringy args

* WIP: first pass, correctly formatted tool_calls between providers

* refactor: Remove duplicate import of 'roles' module

* refactor: Exclude 'vite.config.ts' from TypeScript compilation

* refactor: fix agent related types
> - no need to use endpoint/model fields for identifying agent metadata
> - add `provider` distinction for agent-configured 'endpoint'
- no need for agent-endpoint map
- reduce complexity of tools as functions into tools as string[]
- fix types related to above changes
- reduce unnecessary variables for queries/mutations and corresponding react-query keys

* refactor: Add tools and tool_kwargs fields to agent schema

* refactor: Remove unused code and update dependencies

* refactor: Update updateAgentHandler to use req.body directly

* refactor: Update AgentSelect component to use localized hooks

* refactor: Update agent schema to include tools and provider fields

* refactor(AgentPanel): add scrollbar gutter, add provider field to form, fix agent schema required values

* refactor: Update AgentSwitcher component to use selectedAgentId instead of selectedAgent

* refactor: Update AgentPanel component to include alternateName import and defaultAgentFormValues

* refactor(SelectDropDown): allow setting value as option while still supporting legacy usage (string values only)

* refactor: SelectDropdown changes - Only necessary when the available values are objects with label/value fields and the selected value is expected to be a string.

* refactor: TypeError issues and handle provider as option

* feat: Add placeholder for provider selection in AgentPanel component

* refactor: Update agent schema to include author and provider fields

* fix: show expected 'create agent' placeholder when creating agent

* chore: fix localization strings, hide capabilities form for now

* chore: typing

* refactor: import order and use compact agents schema for now

* chore: typing

* refactor: Update AgentForm type to use AgentCapabilities

* fix agent form agent selection issues

* feat: responsive agent selection

* fix: Handle cancelled fetch in useSelectAgent hook

* fix: reset agent form on accordion close/open

* feat: Add agent_id to default conversation for agents endpoint

* feat: agents endpoint request handling

* refactor: reset conversation model on agent select

* refactor: add `additional_instructions` to conversation schema, organize other fields

* chore: casing

* chore: types

* refactor(loadAgentTools): explicitly pass agent_id, do not pass `model` to loadAgentTools for now, load action sets by agent_id

* WIP: initial draft of real agent client initialization

* WIP: first pass, anthropic agent requests

* feat: remember last selected agent

* feat: openai and azure connected

* fix: prioritize agent model for runs unless an explicit override model is passed from client

* feat: Agent Actions

* fix: save agent id to convo

* feat: model panel (#29)

* feat: model panel

* bring back comments

* fix: method still null

* fix: AgentPanel FormContext

* feat: add more parameters

* fix: style issues; refactor: Agent Controller

* fix: cherry-pick

* fix: Update AgentAvatar component to use AssistantIcon instead of BrainCircuit

* feat: OGDialog for delete agent; feat(assistant): update Agent types, introduced `model_parameters`

* feat: icon and general `model_parameters` update

* feat: use react-hook-form better

* fix: agent builder form reset issue when switching panels

* refactor: modularize agent builder form

---------

Co-authored-by: Danny Avila <danny@librechat.ai>

* fix: AgentPanel and ModelPanel type issues and use `useFormContext` and `watch` instead of `methods` directly and `useWatch`.

* fix: tool call issues due to invalid input (anthropic) of empty string

* fix: handle empty text in Part component

---------

Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>

* refactor: remove form ModelPanel and fixed nested ternary expressions in AgentConfig

* fix: Model Parameters not saved correctly

* refactor: remove console log

* feat: avatar upload and get for Agents (#36)

Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>

* chore: update to public package

* fix: typing, optional chaining

* fix: cursor not showing for content parts

* chore: conditionally enable agents

* ci: fix azure test

* ci: fix frontend tests, fix eslint api

* refactor: Remove unused errorContentPart variable

* continue of the agent message PR (#40)

* last fixes

* fix: agentMap

* pr merge test  (#41)

* fix: model icon not fetching correctly

* remove console logs

* feat: agent name

* refactor: pass documentsMap as a prop to allow re-render of assistant form

* refactor: pass documentsMap as a prop to allow re-render of assistant form

* chore: Bump version to 0.7.419

* fix: TypeError: Cannot read properties of undefined (reading 'id')

* refactor: update AgentSwitcher component to use ControlCombobox instead of Combobox

---------

Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
This commit is contained in:
Danny Avila 2024-08-31 16:33:51 -04:00 • committed by GitHub
parent 618be4bf2b
commit a0291ed155
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
141 changed files with 14473 additions and 5714 deletions
Download patch file Download diff file Expand all files Collapse all files

36
client/src/components/Chat/Input/CircleRender.tsx Normal file
View file

@ -0,0 +1,36 @@
import React from 'react';
import { CircleIcon, CircleDotsIcon } from '~/components/svg';
import { ECallState } from 'librechat-data-provider';
const CircleRender = ({ rmsLevel, isCameraOn, state }) => {
const getIconComponent = (state) => {
switch (state) {
case ECallState.Thinking:
return <CircleDotsIcon />;
default:
return (
<div className="smooth-transition" style={{ transform: `scale(${transformScale})` }}>
<CircleIcon state={state} size="256" />
</div>
);
}
};
const baseScale = isCameraOn ? 0.5 : 1;
const scaleMultiplier =
rmsLevel > 0.08
? 1.8
: rmsLevel > 0.07
? 1.6
: rmsLevel > 0.05
? 1.4
: rmsLevel > 0.01
? 1.2
: 1;
const transformScale = baseScale * scaleMultiplier;
return getIconComponent(state);
};
export default CircleRender;

3
client/src/components/Chat/Input/Files/FileRow.tsx
View file

@ -11,6 +11,8 @@ export default function FileRow({
setFiles,
setFilesLoading,
assistant_id,
// TODO: Agent file handling
agent_id,
tool_resource,
fileFilter,
isRTL,
@ -21,6 +23,7 @@ export default function FileRow({
setFilesLoading: React.Dispatch<React.SetStateAction<boolean>>;
fileFilter?: (file: ExtendedFile) => boolean;
assistant_id?: string;
agent_id?: string;
tool_resource?: EToolResources;
isRTL?: boolean;
Wrapper?: React.FC<{ children: React.ReactNode }>;

22
client/src/components/Chat/Menus/Endpoints/Icons.tsx
View file

@ -1,5 +1,6 @@
import { EModelEndpoint } from 'librechat-data-provider';
import type { IconMapProps } from '~/common';
import type { IconMapProps, AgentIconMapProps } from '~/common';
import { BrainCircuit } from 'lucide-react';
import {
MinimalPlugin,
GPTIcon,
@ -33,6 +34,24 @@ const AssistantAvatar = ({ className = '', assistantName, avatar, size }: IconMa
return <Sparkles className={cn(assistantName === '' ? 'icon-2xl' : '', className)} />;
};
const AgentAvatar = ({ className = '', agentName, avatar, size }: AgentIconMapProps) => {
if (agentName && avatar) {
return (
<img
src={avatar}
className="bg-token-surface-secondary dark:bg-token-surface-tertiary h-full w-full rounded-full object-cover"
alt={agentName}
width="80"
height="80"
/>
);
} else if (agentName) {
return <AssistantIcon className={cn('text-token-secondary', className)} size={size} />;
}
return <BrainCircuit className={cn(agentName === '' ? 'icon-2xl' : '', className)} />;
};
export const icons = {
[EModelEndpoint.azureOpenAI]: AzureMinimalIcon,
[EModelEndpoint.openAI]: GPTIcon,
@ -44,5 +63,6 @@ export const icons = {
[EModelEndpoint.custom]: CustomMinimalIcon,
[EModelEndpoint.assistants]: AssistantAvatar,
[EModelEndpoint.azureAssistants]: AssistantAvatar,
[EModelEndpoint.agents]: AgentAvatar,
unknown: UnknownIcon,
};

2
client/src/components/Chat/Messages/Content/ContentParts.tsx
View file

@ -28,7 +28,7 @@ any) => {
return (
<Part
key={`display-${messageId}-${idx}`}
showCursor={showCursor && isSubmitting}
showCursor={showCursor === true && isSubmitting}
isSubmitting={isSubmitting}
part={part}
{...props}

142
client/src/components/Chat/Messages/Content/Part.tsx
View file

@ -15,10 +15,8 @@ import Container from './Container';
import ToolCall from './ToolCall';
import Markdown from './Markdown';
import ImageGen from './ImageGen';
import Image from './Image';
import { cn } from '~/utils';
// import EditMessage from './EditMessage';
import Image from './Image';
// Display Message Component
const DisplayMessage = ({ text, isCreatedByUser = false, message, showCursor }: TDisplayProps) => {
@ -31,6 +29,10 @@ const DisplayMessage = ({ text, isCreatedByUser = false, message, showCursor }:
() => message.messageId === latestMessage?.messageId,
[message.messageId, latestMessage?.messageId],
);
// Note: for testing purposes
// isSubmitting && isLatestMessage && logger.log('message_stream', { text, isCreatedByUser, isSubmitting, showCursorState });
return (
<div
className={cn(
@ -55,7 +57,7 @@ export default function Part({
isSubmitting,
message,
}: {
part: TMessageContentParts;
part: TMessageContentParts | undefined;
isSubmitting: boolean;
showCursor: boolean;
message: TMessage;
@ -67,78 +69,92 @@ export default function Part({
if (part.type === ContentTypes.ERROR) {
return <ErrorMessage message={message} text={part[ContentTypes.TEXT].value} className="my-2" />;
} else if (part.type === ContentTypes.TEXT) {
// Access the value property
const text = typeof part.text === 'string' ? part.text : part.text.value;
if (typeof text !== 'string') {
return null;
}
return (
<Container message={message}>
<DisplayMessage
text={part[ContentTypes.TEXT].value}
text={text}
isCreatedByUser={message.isCreatedByUser}
message={message}
showCursor={showCursor}
/>
</Container>
);
} else if (
part.type === ContentTypes.TOOL_CALL &&
part[ContentTypes.TOOL_CALL].type === ToolCallTypes.CODE_INTERPRETER
) {
} else if (part.type === ContentTypes.TOOL_CALL) {
const toolCall = part[ContentTypes.TOOL_CALL];
const code_interpreter = toolCall[ToolCallTypes.CODE_INTERPRETER];
return (
<CodeAnalyze
initialProgress={toolCall.progress ?? 0.1}
code={code_interpreter.input}
outputs={code_interpreter.outputs ?? []}
isSubmitting={isSubmitting}
/>
);
} else if (
part.type === ContentTypes.TOOL_CALL &&
(part[ContentTypes.TOOL_CALL].type === ToolCallTypes.RETRIEVAL ||
part[ContentTypes.TOOL_CALL].type === ToolCallTypes.FILE_SEARCH)
) {
const toolCall = part[ContentTypes.TOOL_CALL];
return <RetrievalCall initialProgress={toolCall.progress ?? 0.1} isSubmitting={isSubmitting} />;
} else if (
part.type === ContentTypes.TOOL_CALL &&
part[ContentTypes.TOOL_CALL].type === ToolCallTypes.FUNCTION &&
imageGenTools.has(part[ContentTypes.TOOL_CALL].function.name)
) {
const toolCall = part[ContentTypes.TOOL_CALL];
return (
<ImageGen initialProgress={toolCall.progress ?? 0.1} args={toolCall.function.arguments} />
);
} else if (
part.type === ContentTypes.TOOL_CALL &&
part[ContentTypes.TOOL_CALL].type === ToolCallTypes.FUNCTION
) {
const toolCall = part[ContentTypes.TOOL_CALL];
if (isImageVisionTool(toolCall)) {
if (isSubmitting && showCursor) {
return (
<Container message={message}>
<DisplayMessage
text={''}
isCreatedByUser={message.isCreatedByUser}
message={message}
showCursor={showCursor}
/>
</Container>
);
}
if (!toolCall) {
return null;
}
return (
<ToolCall
initialProgress={toolCall.progress ?? 0.1}
isSubmitting={isSubmitting}
args={toolCall.function.arguments}
name={toolCall.function.name}
output={toolCall.function.output}
/>
);
if ('args' in toolCall && (!toolCall.type || toolCall.type === ToolCallTypes.TOOL_CALL)) {
return (
<ToolCall
args={toolCall.args}
name={toolCall.name ?? ''}
output={toolCall.output ?? ''}
initialProgress={toolCall.progress ?? 0.1}
isSubmitting={isSubmitting}
/>
);
} else if (toolCall.type === ToolCallTypes.CODE_INTERPRETER) {
const code_interpreter = toolCall[ToolCallTypes.CODE_INTERPRETER];
return (
<CodeAnalyze
initialProgress={toolCall.progress ?? 0.1}
code={code_interpreter.input}
outputs={code_interpreter.outputs ?? []}
isSubmitting={isSubmitting}
/>
);
} else if (
toolCall.type === ToolCallTypes.RETRIEVAL ||
toolCall.type === ToolCallTypes.FILE_SEARCH
) {
return (
<RetrievalCall initialProgress={toolCall.progress ?? 0.1} isSubmitting={isSubmitting} />
);
} else if (
toolCall.type === ToolCallTypes.FUNCTION &&
ToolCallTypes.FUNCTION in toolCall &&
imageGenTools.has(toolCall.function.name)
) {
return (
<ImageGen
initialProgress={toolCall.progress ?? 0.1}
args={toolCall.function.arguments as string}
/>
);
} else if (toolCall.type === ToolCallTypes.FUNCTION && ToolCallTypes.FUNCTION in toolCall) {
if (isImageVisionTool(toolCall)) {
if (isSubmitting && showCursor) {
return (
<Container message={message}>
<DisplayMessage
text={''}
isCreatedByUser={message.isCreatedByUser}
message={message}
showCursor={showCursor}
/>
</Container>
);
}
return null;
}
return (
<ToolCall
initialProgress={toolCall.progress ?? 0.1}
isSubmitting={isSubmitting}
args={toolCall.function.arguments as string}
name={toolCall.function.name}
output={toolCall.function.output}
/>
);
}
} else if (part.type === ContentTypes.IMAGE_FILE) {
const imageFile = part[ContentTypes.IMAGE_FILE];
const height = imageFile.height ?? 1920;
@ -153,8 +169,6 @@ export default function Part({
height: height + 'px',
width: width + 'px',
}}
// n={imageFiles.length}
// i={i}
/>
);
}

32
client/src/components/Chat/Messages/Content/ToolCall.tsx
View file

@ -1,4 +1,4 @@
// import { useState, useEffect } from 'react';
import { useMemo } from 'react';
import { actionDelimiter, actionDomainSeparator, Constants } from 'librechat-data-provider';
import * as Popover from '@radix-ui/react-popover';
import useLocalize from '~/hooks/useLocalize';
@ -11,18 +11,19 @@ import ToolPopover from './ToolPopover';
// import ActionIcon from './ActionIcon';
import WrenchIcon from './WrenchIcon';
import { useProgress } from '~/hooks';
import { logger } from '~/utils';
export default function ToolCall({
initialProgress = 0.1,
isSubmitting,
name,
args = '',
args: _args = '',
output,
}: {
initialProgress: number;
isSubmitting: boolean;
name: string;
args: string;
args: string | Record<string, unknown>;
output?: string | null;
}) {
const localize = useLocalize();
@ -35,6 +36,27 @@ export default function ToolCall({
const domain = _domain?.replaceAll(actionDomainSeparator, '.') ?? null;
const error = output?.toLowerCase()?.includes('error processing tool');
const args = useMemo(() => {
if (typeof _args === 'string') {
return _args;
}
try {
return JSON.stringify(_args, null, 2);
} catch (e) {
logger.error(
'client/src/components/Chat/Messages/Content/ToolCall.tsx - Failed to stringify args',
e,
);
return '';
}
}, [_args]);
const hasInfo = useMemo(
() => (args?.length || 0) > 0 || (output?.length || 0) > 0,
[args, output],
);
return (
<Popover.Root>
<div className="my-2.5 flex items-center gap-2.5">
@ -67,10 +89,10 @@ export default function ToolCall({
? localize('com_assistants_completed_action', domain)
: localize('com_assistants_completed_function', function_name)
}
hasInput={!!args?.length}
hasInput={hasInfo}
popover={true}
/>
{!!args?.length && (
{hasInfo && (
<ToolPopover input={args} output={output} domain={domain} function_name={function_name} />
)}
</div>

20
client/src/components/Chat/Messages/MessageIcon.tsx
View file

@ -1,6 +1,6 @@
import React, { useMemo, memo } from 'react';
import { useGetEndpointsQuery } from 'librechat-data-provider/react-query';
import type { TMessage, TPreset, Assistant } from 'librechat-data-provider';
import type { TMessage, TPreset, Assistant, Agent } from 'librechat-data-provider';
import type { TMessageProps } from '~/common';
import ConvoIconURL from '~/components/Endpoints/ConvoIconURL';
import { getEndpointField, getIconEndpoint } from '~/utils';
@ -10,15 +10,26 @@ const MessageIcon = memo(
(
props: Pick<TMessageProps, 'message' | 'conversation'> & {
assistant?: Assistant;
agent?: Agent;
},
) => {
const { data: endpointsConfig } = useGetEndpointsQuery();
const { message, conversation, assistant } = props;
const { message, conversation, assistant, agent } = props;
const assistantName = useMemo(() => assistant?.name ?? '', [assistant]);
const assistantAvatar = useMemo(() => assistant?.metadata?.avatar ?? '', [assistant]);
const agentName = useMemo(() => props.agent?.name ?? '', [props.agent]);
const agentAvatar = useMemo(() => props.agent?.avatar?.filepath ?? '', [props.agent]);
const isCreatedByUser = useMemo(() => message?.isCreatedByUser ?? false, [message]);
let avatarURL = '';
if (assistant) {
avatarURL = assistantAvatar;
} else if (agent) {
avatarURL = agentAvatar;
}
const messageSettings = useMemo(
() => ({
...(conversation ?? {}),
@ -47,8 +58,10 @@ const MessageIcon = memo(
preset={messageSettings as typeof messageSettings & TPreset}
context="message"
assistantAvatar={assistantAvatar}
agentAvatar={agentAvatar}
endpointIconURL={endpointIconURL}
assistantName={assistantName}
agentName={agentName}
/>
);
}
@ -57,9 +70,10 @@ const MessageIcon = memo(
<Icon
isCreatedByUser={isCreatedByUser}
endpoint={endpoint}
iconURL={!assistant ? endpointIconURL : assistantAvatar}
iconURL={avatarURL || endpointIconURL}
model={message?.model ?? conversation?.model}
assistantName={assistantName}
agentName={agentName}
size={28.8}
/>
);

24
client/src/components/Chat/Messages/MessageParts.tsx
View file

@ -20,6 +20,7 @@ export default function Message(props: TMessageProps) {
ask,
edit,
index,
agent,
isLast,
enterEdit,
assistant,
@ -38,6 +39,16 @@ export default function Message(props: TMessageProps) {
return null;
}
let name = '';
if (isCreatedByUser === true) {
name = localize('com_user_message');
} else if (assistant) {
name = assistant.name ?? localize('com_ui_assistant');
} else if (agent) {
name = agent.name ?? localize('com_ui_agent');
}
return (
<>
<div
@ -51,7 +62,12 @@ export default function Message(props: TMessageProps) {
<div>
<div className="pt-0.5">
<div className="shadow-stroke flex h-6 w-6 items-center justify-center overflow-hidden rounded-full">
<Icon message={message} conversation={conversation} assistant={assistant} />
<Icon
message={message}
conversation={conversation}
assistant={assistant}
agent={agent}
/>
</div>
</div>
</div>
@ -62,11 +78,7 @@ export default function Message(props: TMessageProps) {
isCreatedByUser === true ? '' : 'agent-turn',
)}
>
<div className={cn('select-none font-semibold', fontSize)}>
{isCreatedByUser === true
? localize('com_user_message')
: (assistant && assistant.name) ?? localize('com_ui_assistant')}
</div>
<div className={cn('select-none font-semibold', fontSize)}>{name}</div>
<div className="flex-col gap-1 md:gap-3">
<div className="flex max-w-full flex-grow flex-col gap-0">
<ContentParts