Commit graph

412 commits

Author SHA1 Message Date
Hongkai Ye
7d5be68747
🌊 feat: add streaming support for o1 models (#4760) 2024-11-20 15:53:23 -05:00
Danny Avila
95201908e9
📦 fix: npm warnings; chore: bump deprecated packages (#4707)
* chore: bump langchain deps to address vulnerability warnings

* chore: bump community package and install textsplitters package

* fix: update expected result in tokenSplit tests for accuracy

* chore: remove CodeSherpa tools

* chore: remove E2B tools and loadToolSuite

* chore: remove CodeBrew tool and update related references

* chore: remove HumanTool and ChatTool, update tool references

* chore: remove Zapier tool from manifest.json and update SerpAPI

* chore: remove basic tools

* chore: update import path for RecursiveCharacterTextSplitter

* chore: update import path for DynamicStructuredTool

* chore: remove extractionChain.js and update tool filtering logic

* chore: npm audit fix

* chore: bump google packages

* chore: update DALL-E tool to DALL-E-3 and adjust authentication logic

* ci: update message classes

* chore: elliptic npm audit fix

* chore: update CallbackManager import and remove deprecated tool handling logic

* chore: imports order

* chore: remove unused code

---------

Co-authored-by: Max Sanna <max@maxsanna.com>
2024-11-12 18:51:32 -05:00
Danny Avila
766643ea1c
🎯 feat: Enhance Title Parameter Parsing with new Anthropic Format 2024-11-06 13:40:49 -05:00
Danny Avila
0c2a583df8
🔧 refactor: Optimize Agent Tool Loading and Fix Bedrock Tool Handling (#4641)
* fix: bedrock tool name regex

* fix: pass args as single input, attempt json first.

* refactor: remove toolMap from agent tool load as is not used

* fix: update formatAgentMessages test to use strictEqual for args comparison, testing new behavior
2024-11-05 11:24:26 -05:00
Danny Avila
3428c3c647
feat: Known Endpoint, xAI (#4632)
* feat: Known Endpoint, xAI

* chore: update librechat-data-provider version to 0.7.53

* ci: name property removal

* feat: add XAI_API_KEY to example environment variables
2024-11-04 16:27:54 -05:00
Danny Avila
fc41032923
🤖 feat: Claude 3.5 Haiku (#4629) 2024-11-04 15:10:24 -05:00
Danny Avila
2e519f9b57
🤖 feat: Custom Endpoint Agents (experimental) (#4627)
* wip: first pass, custom endpoint agents

* chore: imports

* chore: consolidate exports

* fix: imports

* feat: convert message.content array to strings for legacy format handling (deepseek/groq)

* refactor: normalize ollama endpoint name

* refactor: update mocking in isDomainAllowed.spec.js

* refactor: update deepseekModels in tokens.js and tokens.spec.js
2024-11-04 12:59:04 -05:00
Danny Avila
95011ce349
🚧 WIP: Merge Dev Build (#4611)
* refactor: Agent CodeFiles, abortUpload WIP

* feat: code environment file upload

* refactor: useLazyEffect

* refactor:
- Add `watch` from `useFormContext` to check if code execution is enabled
- Disable file upload button if `agent_id` is not selected or code execution is disabled

* WIP: primeCodeFiles; refactor: rename sessionId to session_id for uniformity

* Refactor: Rename session_id to sessionId for uniformity in AuthService.js

* chore: bump @librechat/agents to version 1.7.1

* WIP: prime code files

* refactor: Update code env file upload method to use read stream

* feat: reupload code env file if no longer active

* refactor: isAssistantTool -> isEntityTool + address type issues

* feat: execute code tool hook

* refactor: Rename isPluginAuthenticated to checkPluginAuth in PluginController.js

* refactor: Update PluginController.js to use AuthType constant for comparison

* feat: verify tool authentication (execute_code)

* feat: enter librechat_code_api_key

* refactor: Remove unused imports in BookmarkForm.tsx

* feat: authenticate code tool

* refactor: Update Action.tsx to conditionally render the key and revoke key buttons

* refactor(Code/Action): prevent uncheck-able 'Run Code' capability when key is revoked

* refactor(Code/Action): Update Action.tsx to conditionally render the key and revoke key buttons

* fix: agent file upload edge cases

* chore: bump @librechat/agents

* fix: custom endpoint providerValue icon

* feat: ollama meta modal token values + context

* feat: ollama agents

* refactor: Update token models for Ollama models

* chore: Comment out CodeForm

* refactor: Update token models for Ollama and Meta models
2024-11-01 18:36:39 -04:00
Danny Avila
b939e24f67
🔄 feat: Add Configurable Cache Headers for Index.html (#4565)
* refactor: move o1 model check, after vision request check

* feat(server): add configurable cache headers for index.html

• Add environment variables to control index.html cache headers
• Default to no-cache configuration for consistent app updates
• Document cache control options in .env.example
2024-10-28 11:01:31 -04:00
Danny Avila
4da35b9cf5
🤖 feat: Add support for claude-3-5-sonnet-20241022 (#4510) 2024-10-22 16:45:26 -04:00
Danny Avila
b85c6206ab
🤖 fix: Minor Assistants Issues (#4436)
* refactor(OpenAIClient): titleChatCompletion try/catch

* fix: remove duplicate concatenation as seems to be handled by client SDK now

* fix: assistants image upload

* chore: imports order
2024-10-16 15:04:10 -04:00
Danny Avila
c54a57019e
🕒 feat: Add 5-second timeout for Fetching Model Lists (#4423)
* refactor: add 5 second timeout for fetching AI provider model lists

* ci: fix test due to recent changes
2024-10-15 19:37:41 -04:00
Marco Beretta
ef118009f6
feat: Add GOOGLE_LOC environment variable (#4395) 2024-10-15 18:10:48 -04:00
Abhijith E A
2846779603
🔨 fix(AzureOpenAI): o1 model, stream parameter check (#4381) 2024-10-10 11:10:15 -04:00
Danny Avila
873e0473ec
🧠 feat: Implement O1 Model Support for Max Tokens Handling (#4376) 2024-10-10 02:36:36 -04:00
Danny Avila
2ca257dfb9
⚙️ fix: minor issues related to agents (#4297)
* chore: deprecate `web-browser` tool

* fix: edit agent permission
2024-10-01 11:11:15 -04:00
Danny Avila
ad74350036
🚧 chore: merge latest dev build (#4288)
* fix: agent initialization, add `collectedUsage` handling

* style: improve side panel styling

* refactor(loadAgent): Optimize order agent project ID retrieval

* feat: code execution

* fix: typing issues

* feat: ExecuteCode content part

* refactor: use local state for default collapsed state of analysis content parts

* fix: code parsing in ExecuteCode component

* chore: bump agents package, export loadAuthValues

* refactor: Update handleTools.js to use EnvVar for code execution tool authentication

* WIP

* feat: download code outputs

* fix(useEventHandlers): type issues

* feat: backend handling for code outputs

* Refactor: Remove console.log statement in Part.tsx

* refactor: add attachments to TMessage/messageSchema

* WIP: prelim handling for code outputs

* feat: attachments rendering

* refactor: improve attachments rendering

* fix: attachments, nullish edge case, handle attachments from event stream, bump agents package

* fix filename download

* fix: tool assignment for 'run code' on agent creation

* fix: image handling by adding attachments

* refactor: prevent agent creation without provider/model

* refactor: remove unnecessary space in agent creation success message

* refactor: select first model if selecting provider from empty on form

* fix: Agent avatar bug

* fix: `defaultAgentFormValues` causing boolean typing issue and typeerror

* fix: capabilities counting as tools, causing duplication of them

* fix: formatted messages edge case where consecutive content text type parts with the latter having tool_call_ids would cause consecutive AI messages to be created. furthermore, content could not be an array for tool_use messages (anthropic limitation)

* chore: bump @librechat/agents dependency to version 1.6.9

* feat: bedrock agents

* feat: new Agents icon

* feat: agent titling

* feat: agent landing

* refactor: allow sharing agent globally only if user is admin or author

* feat: initial AgentPanelSkeleton

* feat: AgentPanelSkeleton

* feat: collaborative agents

* chore: add potential authorName as part of schema

* chore: Remove unnecessary console.log statement

* WIP: agent model parameters

* chore: ToolsDialog typing and tool related localization chnages

* refactor: update tool instance type (latest langchain class), and rename google tool to 'google' proper

* chore: add back tools

* feat: Agent knowledge files upload

* refactor: better verbiage for disabled knowledge

* chore: debug logs for file deletions

* chore: debug logs for file deletions

* feat: upload/delete agent knowledge/file-search files

* feat: file search UI for agents

* feat: first pass, file search tool

* chore: update default agent capabilities and info
2024-09-30 17:17:57 -04:00
Danny Avila
6f498eee0f
🏷️ fix: Ensure modelLabel Field Usage for ModelSpecs/GPTPlugins (#4224) 2024-09-24 07:27:11 -04:00
Danny Avila
17e59349ff
📎 feat: Attachment Handling for v1/completions (#4205)
* refactor: add handling of attachments in v1/completions method

* ci: update OpenAIClient.test.js
2024-09-23 11:03:28 -04:00
Danny Avila
3ea2d908e0
🛠️ fix: getStreamUsage Method in OpenAIClient (#4133) 2024-09-19 18:07:49 -04:00
Danny Avila
5f28682314
🔧 fix: OpenAIClient Response Handling for Legacy /v1/completions (#4128) 2024-09-19 13:20:29 -04:00
Danny Avila
8dc5b320bc
📊 refactor: use Parameters from Side Panel for OpenAI, Anthropic, and Custom endpoints (#4092)
* feat: openai parameters

* refactor: anthropic/bedrock params, add preset params for openai, and add azure params

* refactor: use 'compact' schemas for anthropic/openai

* refactor: ensure custom endpoints are properly recognized as valid param endpoints

* refactor: update paramEndpoints check in BaseClient.js

* chore: optimize logging by omitting modelsConfig

* refactor: update label casing in baseDefinitions combobox items

* fix: remove 'stop' model options when using o1 series models

* refactor(AnthropicClient): remove default `stop` value

* refactor: reset params on parameters change

* refactor: remove unused default parameter value map introduced in prior commit

* fix: 'min' typo for 'max' value

* refactor: preset settings

* refactor: replace dropdown for image detail with slider; remove `preventDelayedUpdate` condition from DynamicSlider

* fix: localizations for freq./pres. penalty

* Refactor maxOutputTokens to use coerceNumber in tConversationSchema

* refactor(AnthropicClient): use `getModelMaxOutputTokens`
2024-09-17 22:25:54 -04:00
Danny Avila
f1ae267850
🪙 fix: usage check for reasoning_tokens 2024-09-13 09:51:09 -04:00
Danny Avila
e293ff63f9
🪨 feat: AWS Bedrock Default Credentials Chain (#4038)
* feat: use AWS cascading default providers if credentials are omitted

Environment variables exposed via process.env
SSO credentials from token cache
Web identity token credentials
Shared credentials and config ini files
The EC2/ECS Instance Metadata Service

The default credential provider will invoke one provider at a time and only continue to the next if no credentials have been located. For example, if the process finds values defined via the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables, the files at ~/.aws/credentials and ~/.aws/config will not be read, nor will any messages be sent to the Instance Metadata Service.

* fix: usage check in OpenAIClient

* refactor: Improve usage check in OpenAIClient
2024-09-13 08:53:50 -04:00
Danny Avila
45b42830a5
🚀 feat: o1 (#4019)
* feat: o1 default response sender string

* feat: add o1 models to default openai models list, add `no_system_messages` error type; refactor: use error type as localization key

* refactor(MessageEndpointIcon): differentiate openAI icon model color for o1 models

* refactor(AnthropicClient): use new input/output tokens keys; add prompt caching for claude-3-opus

* refactor(BaseClient): to use new input/output tokens keys; update typedefs

* feat: initial o1 model handling, including token cost complexity

* EXPERIMENTAL: special handling for o1 model with custom instructions
2024-09-12 18:15:43 -04:00
Danny Avila
9a393be012
🪨 fix: Formatting Edge Case Handling for Bedrock Messages (#4016)
* refactor: Remove console.log statement in SelectDropDown component

* fix(bedrock): edge case - message.content as string creating message formatting issue
2024-09-12 06:06:22 -04:00
Sebastian Diez
07e5531b5b
⚙️ fix: Ensure Azure AI Search TOP is a number (#3891)
allows configuration through AZURE_AI_SEARCH_SEARCH_OPTION_TOP enviroment variable
2024-09-11 09:21:33 -04:00
Danny Avila
341e086d70
🛠️ fix: Completion Edge Cases & Improve Error Handling UX (#3968)
* fix: edge cases concerning completion response as an array

* refactor: improve invalid request error UX
2024-09-09 20:58:15 -04:00
Danny Avila
d59b62174f
🪨 feat: AWS Bedrock support (#3935)
* feat: Add BedrockIcon component to SVG library

* feat: EModelEndpoint.bedrock

* feat: first pass, bedrock chat. note: AgentClient is returning `agents` as conversation.endpoint

* fix: declare endpoint in initialization step

* chore: Update @librechat/agents dependency to version 1.4.5

* feat: backend content aggregation for agents/bedrock

* feat: abort agent requests

* feat: AWS Bedrock icons

* WIP: agent provider schema parsing

* chore: Update EditIcon props type

* refactor(useGenerationsByLatest): make agents and bedrock editable

* refactor: non-assistant message content, parts

* fix: Bedrock response `sender`

* fix: use endpointOption.model_parameters not endpointOption.modelOptions

* fix: types for step handler

* refactor: Update Agents.ToolCallDelta type

* refactor: Remove unnecessary assignment of parentMessageId in AskController

* refactor: remove unnecessary assignment of parentMessageId (agent request handler)

* fix(bedrock/agents): message regeneration

* refactor: dynamic form elements using react-hook-form Controllers

* fix: agent icons/labels for messages

* fix: agent actions

* fix: use of new dynamic tags causing application crash

* refactor: dynamic settings touch-ups

* refactor: update Slider component to allow custom track class name

* refactor: update DynamicSlider component styles

* refactor: use Constants value for GLOBAL_PROJECT_NAME (enum)

* feat: agent share global methods/controllers

* fix: agents query

* fix: `getResponseModel`

* fix: share prompt a11y issue

* refactor: update SharePrompt dialog theme styles

* refactor: explicit typing for SharePrompt

* feat: add agent roles/permissions

* chore: update @librechat/agents dependency to version 1.4.7 for tool_call_ids edge case

* fix(Anthropic): messages.X.content.Y.tool_use.input: Input should be a valid dictionary

* fix: handle text parts with tool_call_ids and empty text

* fix: role initialization

* refactor: don't make instructions required

* refactor: improve typing of Text part

* fix: setShowStopButton for agents route

* chore: remove params for now

* fix: add streamBuffer and streamRate to help prevent 'Overloaded' errors from Anthropic API

* refactor: remove console.log statement in ContentRender component

* chore: typing, rename Context to Delete Button

* chore(DeleteButton): logging

* refactor(Action): make accessible

* style(Action): improve a11y again

* refactor: remove use/mention of mongoose sessions

* feat: first pass, sharing agents

* feat: visual indicator for global agent, remove author when serving to non-author

* wip: params

* chore: fix typing issues

* fix(schemas): typing

* refactor: improve accessibility of ListCard component and fix console React warning

* wip: reset templates for non-legacy new convos

* Revert "wip: params"

This reverts commit f8067e91d4.

* Revert "refactor: dynamic form elements using react-hook-form Controllers"

This reverts commit 2150c4815d.

* fix(Parameters): types and parameter effect update to only update local state to parameters

* refactor: optimize useDebouncedInput hook for better performance

* feat: first pass, anthropic bedrock params

* chore: paramEndpoints check for endpointType too

* fix: maxTokens to use coerceNumber.optional(),

* feat: extra chat model params

* chore: reduce code repetition

* refactor: improve preset title handling in SaveAsPresetDialog component

* refactor: improve preset handling in HeaderOptions component

* chore: improve typing, replace legacy dialog for SaveAsPresetDialog

* feat: save as preset from parameters panel

* fix: multi-search in select dropdown when using Option type

* refactor: update default showDefault value to false in Dynamic components

* feat: Bedrock presets settings

* chore: config, fix agents schema, update config version

* refactor: update AWS region variable name in bedrock options endpoint to BEDROCK_AWS_DEFAULT_REGION

* refactor: update baseEndpointSchema in config.ts to include baseURL property

* refactor: update createRun function to include req parameter and set streamRate based on provider

* feat: availableRegions via config

* refactor: remove unused demo agent controller file

* WIP: title

* Update @librechat/agents to version 1.5.0

* chore: addTitle.js to handle empty responseText

* feat: support images and titles

* feat: context token updates

* Refactor BaseClient test to use expect.objectContaining

* refactor: add model select, remove header options params, move side panel params below prompts

* chore: update models list, catch title error

* feat: model service for bedrock models (env)

* chore: Remove verbose debug log in AgentClient class following stream

* feat(bedrock): track token spend; fix: token rates, value key mapping for AWS models

* refactor: handle streamRate in `handleLLMNewToken` callback

* chore: AWS Bedrock example config in `.env.example`

* refactor: Rename bedrockMeta to bedrockGeneral in settings.ts and use for AI21 and Amazon Bedrock providers

* refactor: Update `.env.example` with AWS Bedrock model IDs URL and additional notes

* feat: titleModel support for bedrock

* refactor: Update `.env.example` with additional notes for AWS Bedrock model IDs
2024-09-09 12:06:59 -04:00
Danny Avila
a0291ed155
🚧 chore: merge latest dev build to main repo (#3844)
* agents - phase 1 (#30)

* chore: copy assistant files

* feat: frontend and data-provider

* feat: backend get endpoint test

* fix(MessageEndpointIcon): switched to AgentName and AgentAvatar

* fix: small fixes

* fix: agent endpoint config

* fix: show Agent Builder

* chore: install agentus

* chore: initial scaffolding for agents

* fix: updated Assistant logic to Agent Logic for some Agent components

* WIP first pass, demo of agent package

* WIP: initial backend infra for agents

* fix: agent list error

* wip: agents routing

* chore: Refactor useSSE hook to handle different data events

* wip: correctly emit events

* chore: Update @librechat/agentus npm dependency to version 1.0.9

* remove comment

* first pass: streaming agent text

* chore: Remove @librechat/agentus root-level workspace npm dependency

* feat: Agent Schema and Model

* fix: content handling fixes

* fix: content message save

* WIP: new content data

* fix: run step issue with tool calls

* chore: Update @librechat/agentus npm dependency to version 1.1.5

* feat: update controller and agent routes

* wip: initial backend tool and tool error handling support

* wip: tool chunks

* chore: Update @librechat/agentus npm dependency to version 1.1.7

* chore: update tool_call typing, add test conditions and logs

* fix: create agent

* fix: create agent

* first pass: render completed content parts

* fix: remove logging, fix step handler typing

* chore: Update @librechat/agentus npm dependency to version 1.1.9

* refactor: cleanup maps on unmount

* chore: Update BaseClient.js to safely count tokens for string, number, and boolean values

* fix: support subsequent messages with tool_calls

* chore: export order

* fix: select agent

* fix: tool call types and handling

* chore: switch to anthropic for testing

* fix: AgentSelect

* refactor: experimental: OpenAIClient to use array for intermediateReply

* fix(useSSE): revert old condition for streaming legacy client tokens

* fix: lint

* revert `agent_id` to `id`

* chore: update localization keys for agent-related components

* feat: zod schema handling for actions

* refactor(actions): if no params, no zodSchema

* chore: Update @librechat/agentus npm dependency to version 1.2.1

* feat: first pass, actions

* refactor: empty schema for actions without params

* feat: Update createRun function to accept additional options

* fix: message payload formatting; feat: add more client options

* fix: ToolCall component rendering when action has no args but has output

* refactor(ToolCall): allow non-stringy args

* WIP: first pass, correctly formatted tool_calls between providers

* refactor: Remove duplicate import of 'roles' module

* refactor: Exclude 'vite.config.ts' from TypeScript compilation

* refactor: fix agent related types
> - no need to use endpoint/model fields for identifying agent metadata
> - add `provider` distinction for agent-configured 'endpoint'
- no need for agent-endpoint map
- reduce complexity of tools as functions into tools as string[]
- fix types related to above changes
- reduce unnecessary variables for queries/mutations and corresponding react-query keys

* refactor: Add tools and tool_kwargs fields to agent schema

* refactor: Remove unused code and update dependencies

* refactor: Update updateAgentHandler to use req.body directly

* refactor: Update AgentSelect component to use localized hooks

* refactor: Update agent schema to include tools and provider fields

* refactor(AgentPanel): add scrollbar gutter, add provider field to form, fix agent schema required values

* refactor: Update AgentSwitcher component to use selectedAgentId instead of selectedAgent

* refactor: Update AgentPanel component to include alternateName import and defaultAgentFormValues

* refactor(SelectDropDown): allow setting value as option while still supporting legacy usage (string values only)

* refactor: SelectDropdown changes - Only necessary when the available values are objects with label/value fields and the selected value is expected to be a string.

* refactor: TypeError issues and handle provider as option

* feat: Add placeholder for provider selection in AgentPanel component

* refactor: Update agent schema to include author and provider fields

* fix: show expected 'create agent' placeholder when creating agent

* chore: fix localization strings, hide capabilities form for now

* chore: typing

* refactor: import order and use compact agents schema for now

* chore: typing

* refactor: Update AgentForm type to use AgentCapabilities

* fix agent form agent selection issues

* feat: responsive agent selection

* fix: Handle cancelled fetch in useSelectAgent hook

* fix: reset agent form on accordion close/open

* feat: Add agent_id to default conversation for agents endpoint

* feat: agents endpoint request handling

* refactor: reset conversation model on agent select

* refactor: add `additional_instructions` to conversation schema, organize other fields

* chore: casing

* chore: types

* refactor(loadAgentTools): explicitly pass agent_id, do not pass `model` to loadAgentTools for now, load action sets by agent_id

* WIP: initial draft of real agent client initialization

* WIP: first pass, anthropic agent requests

* feat: remember last selected agent

* feat: openai and azure connected

* fix: prioritize agent model for runs unless an explicit override model is passed from client

* feat: Agent Actions

* fix: save agent id to convo

* feat: model panel (#29)

* feat: model panel

* bring back comments

* fix: method still null

* fix: AgentPanel FormContext

* feat: add more parameters

* fix: style issues; refactor: Agent Controller

* fix: cherry-pick

* fix: Update AgentAvatar component to use AssistantIcon instead of BrainCircuit

* feat: OGDialog for delete agent; feat(assistant): update Agent types, introduced `model_parameters`

* feat: icon and general `model_parameters` update

* feat: use react-hook-form better

* fix: agent builder form reset issue when switching panels

* refactor: modularize agent builder form

---------

Co-authored-by: Danny Avila <danny@librechat.ai>

* fix: AgentPanel and ModelPanel type issues and use `useFormContext` and `watch` instead of `methods` directly and `useWatch`.

* fix: tool call issues due to invalid input (anthropic) of empty string

* fix: handle empty text in Part component

---------

Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>

* refactor: remove form ModelPanel and fixed nested ternary expressions in AgentConfig

* fix: Model Parameters not saved correctly

* refactor: remove console log

* feat: avatar upload and get for Agents (#36)

Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>

* chore: update to public package

* fix: typing, optional chaining

* fix: cursor not showing for content parts

* chore: conditionally enable agents

* ci: fix azure test

* ci: fix frontend tests, fix eslint api

* refactor: Remove unused errorContentPart variable

* continue of the agent message PR (#40)

* last fixes

* fix: agentMap

* pr merge test  (#41)

* fix: model icon not fetching correctly

* remove console logs

* feat: agent name

* refactor: pass documentsMap as a prop to allow re-render of assistant form

* refactor: pass documentsMap as a prop to allow re-render of assistant form

* chore: Bump version to 0.7.419

* fix: TypeError: Cannot read properties of undefined (reading 'id')

* refactor: update AgentSwitcher component to use ControlCombobox instead of Combobox

---------

Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
2024-08-31 16:33:51 -04:00
Danny Avila
7536e649d4
🚨 feat: Implement INPUT_LENGTH Error Type (#3866)
* feat: CONTEXT_LENGTH error type

* chore: rename error type

* chore: import order
2024-08-30 15:01:29 -04:00
Danny Avila
7c1ee242eb
🪄 feat: Code Artifacts (#3798)
* feat: Add CodeArtifacts component to Beta settings tab

* chore: Update npm dependency to @codesandbox/sandpack-react@2.18.2

* WIP: artifacts first pass

* WIP first pass remark-directive

* chore: revert markdown to original component + new artifacts rendering

* refactor: first pass rewrite

* refactor: add throttling

* first pass styling

* style: Add Radix Tabs, more styling changes

* feat: second pass

* style: code styling

* fix: package markdown fixes

* feat: Add useEffect hook to Artifacts component for visibility control, slide in animation

* fix: only set artifact if there is content

* refactor: typing and make latest artifact active if the number of artifacts changed

* feat: artifacts + shadcnui

* feat: Add Copy Code button to Artifacts component

* feat: first pass streaming updates

* refactor: optimize ordering of artifacts in Artifacts component

* refactor: optimize ordering of artifacts and add latest artifact activation in Artifacts component

* refactor: add order prop to Artifact

* feat: update to latest, use update time for ordering

* refactor: optimize ordering of artifacts and activate latest artifact in Artifacts component

* wip: remove thinking text and artifact formatting if empty

* refactor: optimize Markdown rendering and add support for code artifacts

* feat: global state for current artifact Id and set on artifact preview click

* refactor: Rename CodePreview component to ArtifactButton

* refactor: apply growth to artifact frame so artifact preview can take full space

* refactor: remove artifactIdsState

* refactor: nullify artifact state and reset on empty conversation

* feat: reset artifact state on conversation change

* feat: artifacts system prompt in backend

* refactor: update UI artifact toggle label to match localization key

* style: remove ArtifactButton inline-block styling

* feat: memoize ArtifactPreview, add html support

* refactor: abstract out components

* chore: bump react-resizable-panel

* refactor: resizable panel order props

* fix: side panel resizing crashes

* style: temporarily remove scrolling, add better styling

* chore: remove thinking for now

* chore: preprocess artifacts for now

* feat: Add auto scrolling to CodeMarkdown (artifacts)

* feat: autoswitch to preview

* feat: auto switch to code, adjust prompt, remove unused code

* feat: refresh button

* feat: open/close artifacts

* wip: mermaid

* refactor: w-fit Artifact button

* chore: organize code

* feat: first pass mermaid

* refactor: improve panning logic in MermaidDiagram component

* feat: center/zoom on first render

* refactor: add centering with reset button

* style: mermaid styling

* refactor: add back MermaidDiagram

* fix: static/html template

* fix: mermaid

* add examples to artifacts prompt

* refactor: fix CodeBar plugin prop logic

* refactor: remove unnecessary mention of artifacts when not requested

* fix: remove preprocessCodeArtifacts function and fix imports

* feat: improve artifacts guidelines and remove unnecessary mentions

* refactor: improve artifacts guidelines and remove unnecessary mentions

* chore: uninstall unused packages

* chore: bump vite

* chore: update three dependency to version 0.167.1

* refactor: move beta settings, add additional artifacts toggles

* feat: artifacts mode toggles

* refactor: adjust prompt

* feat: shadcnui instructions

* feat: code artifacts custom prompt mode

* chore: Update artifacts UI labels and instructions localizations

* refactor: Remove unused code in Markdown component
2024-08-27 17:03:16 -04:00
Danny Avila
5694ad4e55
🧠 feat: Prompt caching switch, prompt query params; refactor: static cache, prompt/markdown styling, trim copied code, switch new chat to convo URL (#3784)
* refactor: Update staticCache to use oneDayInSeconds for sMaxAge and maxAge

* refactor: role updates

* style: first pass cursor

* style: Update nested list styles in style.css

* feat: setIsSubmitting to true in message handler to prevent edge case where submitting turns false during message stream

* feat: Add logic to redirect to conversation page after creating a new conversation

* refactor: Trim code string before copying in CodeBlock component

* feat: configSchema bookmarks and presets defaults

* feat: Update loadDefaultInterface to handle undefined config

* refactor: use  for compression check

* feat: first pass, query params

* fix: styling issues for prompt cards

* feat: anthropic prompt caching UI switch

* chore: Update static file cache control defaults/comments in .env.example

* ci: fix tests

* ci: fix tests

* chore:  use "submitting" class server error connection suspense fallback
2024-08-26 15:34:46 -04:00
Danny Avila
98b437edd5
🖱️ fix: Message Scrolling UX; refactor: Frontend UX/DX Optimizations (#3733)
* refactor(DropdownPopup): set MenuButton `as` prop to `div` to prevent React warning: validateDOMNesting(...): <button> cannot appear as a descendant of <button>

* refactor: memoize ChatGroupItem and ControlCombobox components

* refactor(OpenAIClient): await stream process finish before finalCompletion event handling

* refactor: update useSSE.ts typing to handle null and undefined values in data properties

* refactor: set abort scroll to false on SSE connection open

* refactor: improve logger functionality with filter support

* refactor: update handleScroll typing in MessageContainer component

* refactor: update logger.dir call in useChatFunctions to log 'message_stream' tag format instead of the entire submission object as first arg

* refactor: fix null check for message object in Message component

* refactor: throttle handleScroll to help prevent auto-scrolling issues on new message requests; fix type issues within useMessageProcess

* refactor: add abortScrollByIndex logging effect

* refactor: update MessageIcon and Icon components to use React.memo for performance optimization

* refactor: memoize ConvoIconURL component for performance optimization

* chore: type issues

* chore: update package version to 0.7.414
2024-08-21 18:18:45 -04:00
Danny Avila
14ddfec9b7
🧹 chore: address minor issues (#3710)
* refactor(OpenAIClient): improve error handling in chat completion

Improve error handling in the chat completion process to handle scenarios where the response has no choices or the message is undefined. This ensures that the intermediate reply is returned in such cases. Also, add logging statements for better debugging.

* refactor(Markdown): only show blinking cursor for latest message

* refactor: add submitting selector to prevent blinking cursor for empty, non-streaming, non-latest messages
2024-08-20 07:00:43 -04:00
Danny Avila
8ca1e4f94f
🐛 fix: Anthropic Prompt Caching Edge Case (#3690) 2024-08-18 19:03:03 -04:00
Danny Avila
683702d555
🤖 refactor: Remove Default Model Params for All Endpoints (#3682)
* refactor: use parseCompactConvo in buildOptions, and generate no default values for the API to avoid weird model behavior with defaults

* refactor: OTHER - always show cursor when markdown component is empty (preferable to not)

* refactor(OpenAISettings): use config object for setting defaults app-wide

* refactor: Use removeNullishValues in buildOptions for ALL  endpoints

* fix: add missing conversationId to title methods for transactions; refactor(GoogleClient): model options, set no default, add todo note for recording token usage

* fix: at minimum set a model default, as is required by API (edge case)
2024-08-18 06:00:03 -04:00
Danny Avila
a45b384bbc
💾 feat: Anthropic Prompt Caching (#3670)
* wip: initial cache control implementation, add typing for transactions handling

* feat: first pass of Anthropic Prompt Caching

* feat: standardize stream usage as pass in when calculating token counts

* feat: Add getCacheMultiplier function to calculate cache multiplier for different valueKeys and cacheTypes

* chore: imports order

* refactor: token usage recording in AnthropicClient, no need to "correct" as we have the correct amount

* feat: more accurate token counting using stream usage data

* feat: Improve token counting accuracy with stream usage data

* refactor: ensure more accurate than not token estimations if custom instructions or files are not being resent with every request

* refactor: cleanup updateUserMessageTokenCount to allow transactions to be as accurate as possible even if we shouldn't update user message token counts

* ci: fix tests
2024-08-17 03:24:09 -04:00
Danny Avila
f8a5dad265
🦙 fix: Update Title Message Role for Ollama if None Provided (#3663) 2024-08-16 04:54:37 -04:00
Dongwoo Jeong
6879de0bf1
🔧 fix: API Key Handling for GoogleSearch and TavilySearch Tools (#3541)
* fix: get user-provided API key using environment variable names as keys

* feat: Add error handling for missing API key and search engine ID

* feat: Add GoogleSearch and TavilySearchResults specs for environment variables handling

---------

Co-authored-by: Dongwoo Jeong <dongwoo.jeong@lge.com>
2024-08-05 19:29:16 -04:00
Danny Avila
458dc9c88e
🛡️ feat: Optimize and Improve Anonymity of SharedLinks (#3543)
* ci: add spec for validateImageRequest

* chore: install nanoid and run npm audit fix

* refactor: optimize and further anonymize shared links data

* ci: add SharedLink specs

* feat: anonymize asst_id's

* ci: tests actually failing, to revisit later

* fix: do not anonymize authenticated shared links query

* refactor: Update ShareView component styling

* refactor: remove nested ternary

* refactor: no nested ternaries

* fix(ShareView): eslint warnings

* fix(eslint): remove nested terns
2024-08-05 03:34:00 -04:00
Danny Avila
5baa95bd9a
🖼️ fix: correct image extraction (#3538)
* fix: regex issue extracting text with images in markdown

* fix: update addImages function to include only the first observed image path in the response message

* ci: tests for addImages function: correct image extraction

* fix(GoogleClient): linting

---------

Co-authored-by: Dongwoo Jeong <dongwoo.jeong@lge.com>
Co-authored-by: Dongwoo Jeong <dongwoo@duck.com>
2024-08-04 20:53:11 -04:00
Oliver Faust
433d8f832a
🛠️fix: apply safety settings to Gemini API (#3533)
- Introduce getSafetySettings method for centralized safety settings
- Apply safety settings uniformly in sendCompletion for all models, including Gemini
- Ensure consistent safety settings application in getCompletion
- Update .env.example to clarify safety settings usage for both APIs
2024-08-04 14:08:57 -04:00
Danny Avila
b8a6efa649
👷‍♂️ chore: pass Tavily Response error and revert test commit on userSchema.js (#3496)
* chore: revert test commit

* chore: make sure tavily error gets passed
2024-07-31 12:06:27 -04:00
Danny Avila
422d1a2c91
🤖 feat: new Anthropic Default Settings / Increased Output Tokens for 3.5-Sonnet (#3407)
* chore: bump data-provider

* feat: Add anthropicSettings to endpointSettings

The commit adds the `anthropicSettings` object to the `endpointSettings` in the `schemas.ts` file. This allows for the configuration of settings specific to the `anthropic` model endpoint.

* chore: adjust maxoutputtokens localization

* feat: Update AnthropicClient to use anthropicSettings for default model options and increased output beta header

* ci: new anthropic tests
2024-07-20 08:53:16 -04:00
Danny Avila
2ad097647c
fix: Wait for Initial Message Save & Correct Latest Message (#3399)
* chore: assistants, unsupported assistant, better logging

* chore: remove unnecessary logger in validateAssistant middleware

* fix: resolve initial conversation save/promise before saving response

* chore: Import and organize dependencies in Speech component

* fix: conversation statefulness
- Latest Message (at index 0) should not be reset if existing convo
- add debugging context for clearAllLatestMessages
- Added logging concerning latest Message updates (dev mode only)
- update latest message Set logic, also checks for change in conversation Id
- consolidated latest message helpers to client/src/utils/messages.ts
2024-07-20 01:51:59 -04:00
Danny Avila
f6125ccd59
🌿 fix: Message Route Improvements pt. 2 (#3379)
* fix: edge case with debugTraverse function in parsers.js

* chore: Update error message in audio stream processing

* refactor: Add ONE_MINUTE and THIRTY_SECONDS options to Time enum

* fix: logging
2024-07-18 13:56:19 -04:00
Danny Avila
5d40d0a37a
⚙️ feat: Adjust Rate of Stream Progress (#3244)
* chore: bump data-provider and add MESSAGES CacheKey

* refactor: avoid saving messages while streaming, save partial text to cache instead

* fix(ci): processChunks

* chore: logging aborted request to debug

* feat: set stream rate for token processing

* chore: specify default stream rate

* fix(ci): Update AppService.js to use optional chaining for endpointLocals assignment

* refactor: abstract the error handler

* feat: streamRate for assistants; refactor: update default rate for token

* refactor: update error handling in assistants/errors.js

* refactor: update error handling in assistants/errors.js
2024-07-17 10:47:17 -04:00
Danny Avila
d5d188eebf
🔐 fix: Enhance Message & Image Access Security (#3363)
* chore: slight refactor

* fix: prevent message updates unless explicitly owned

* refactor: rethrow errors, update deleteMessagesSince (not used), add basic tests

* fix: Add path normalization and validation to image request middleware

* fix: image validation path security
2024-07-17 09:51:03 -04:00
Marco Beretta
1aad315de6
🎤 feat: add custom speech config, browser TTS/STT features, and dynamic speech tab settings (#2921)
* feat: update useTextToSpeech and useSpeechToText hooks to support external audio endpoints

This commit updates the useTextToSpeech and useSpeechToText hooks in the Input directory to support external audio endpoints. It introduces the useGetExternalTextToSpeech and useGetExternalSpeechToText hooks, which determine whether the audio endpoints should be set to 'browser' or 'external' based on the value of the endpointTTS and endpointSTT Recoil states. The useTextToSpeech and useSpeechToText hooks now use these new hooks to determine whether to use external audio endpoints

* feat: add userSelect style to ConversationModeSwitch label

* fix: remove unused updateTokenWebsocket function and import

The updateTokenWebsocket function and its import are no longer used in the OpenAIClient module. This commit removes the function and import to clean up the codebase

* feat: support external audio endpoints in useTextToSpeech and useSpeechToText hooks

This commit updates the useTextToSpeech and useSpeechToText hooks in the Input directory to support external audio endpoints. It introduces the useGetExternalTextToSpeech and useGetExternalSpeechToText hooks, which determine whether the audio endpoints should be set to 'browser' or 'external' based on the value of the endpointTTS and endpointSTT Recoil states. The useTextToSpeech and useSpeechToText hooks now use these new hooks to determine whether to use external audio endpoints

* feat: update AutomaticPlayback component to AutomaticPlaybackSwitch; tests: added AutomaticPlaybackSwitch.spec
>
> This commit renames the AutomaticPlayback component to AutomaticPlaybackSwitch in the Speech directory. The new name better reflects the purpose of the component and aligns with the naming convention used in the codebase.

* feat: update useSpeechToText hook to include interimTranscript

This commit updates the useSpeechToText hook in the client/src/components/Chat/Input/AudioRecorder.tsx file to include the interimTranscript state. This allows for real-time display of the speech-to-text transcription while the user is still speaking. The interimTranscript is now used to update the text area value during recording.

* feat: Add customConfigSpeech API endpoint for retrieving custom speech configuration

This commit adds a new API endpoint  in the  file under the  directory. This endpoint is responsible for retrieving the custom speech configuration using the  function from the  module

* feat: update store var  and ; fix: getCustomConfigSpeech

* fix: client tests, removed unused import

* feat: Update useCustomConfigSpeechQuery to return an array of custom speech configurations

This commit modifies the useCustomConfigSpeechQuery function in the client/src/data-provider/queries.ts file to return an array of custom speech configurations instead of a single object. This change allows for better handling and manipulation of the data in the application

* feat: Update useCustomConfigSpeechQuery to return an array of custom speech configurations

* refactor: Update variable name in speechTab schema

* refactor: removed unused and nested code

* fix: using recoilState

* refactor: Update Speech component to use useCallback for setting settings

* fix: test

* fix: tests

* feature: ensure that the settings don't change after modifying then through the UI

* remove comment

* fix: Handle error gracefully in getCustomConfigSpeech and getVoices endpoints

* fix: Handle error

* fix: backend tests

* fix: invalid custom config logging

* chore: add back custom config info logging

* chore: revert loadCustomConfig spec

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2024-07-05 10:13:34 -04:00