Andreas/LibreChat
1
0
Fork 0
mirror of https://github.com/danny-avila/LibreChat.git synced 2025-12-23 11:50:14 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

🎨 feat: OpenAI Image Tools (GPT-Image-1) (#7079)

Browse source 
* wip: OpenAI Image Generation Tool with customizable options

* WIP: First pass OpenAI Image Generation Tool and integrate into existing tools

* 🔀 fix: Comment out unused validation for image generation tool parameters

* 🔀 refactor: Update primeResources function parameters for better destructuring

* feat: Add image_edit resource to EToolResources and update AgentToolResources interface

* feat: Enhance file retrieval with tool resource filtering for image editing

* refactor: add OpenAI Image Tools for generation and editing, refactor related components, pass current request image attachments as tool resources for editing

* refactor: Remove commented-out code and clean up API key retrieval in createOpenAIImageTools function

* fix: show message attachments in shared links

* fix: Correct parent message retrieval logic for regenerated messages in useChatFunctions

* fix: Update primeResources to utilize requestFileSet for image file processing

* refactor: Improve description for image generation tool and clarify usage conditions, only provide edit tool if there are images available to edit

* chore: Update OpenAI Image Tools icon to use local asset

* refactor: Update image generation tool description and logic to prioritize editing tool when files are uploaded

* refactor: Enhance image tool descriptions to clarify usage conditions and note potential unavailability of uploaded images

* refactor: Update useAttachmentHandler to accept queryClient to update query cache with newly created file

* refactor: Add customizable descriptions and prompts for OpenAI image generation and editing tools

* chore: Update comments to use JSDoc style for better clarity and consistency

* refactor: Rename config variable to clientConfig for clarity and update signal handling in image generation

* refactor: Update axios request configuration to include derived signal and baseURL for improved request handling

* refactor: Update baseURL environment variable for OpenAI image generation tool configuration

* refactor: Enhance axios request configuration with conditional headers and improved clientConfig setup

* chore: Update comments for clarity and remove unnecessary lines in OpenAI image tools

* refactor: Update description for image generation without files to clarify user instructions

* refactor: Simplify target parent message logic for regeneration and resubmission cases

* chore: Remove backticks from error messages in image generation and editing functions

* refactor: Rename toolResources to toolResourceSet for clarity in file retrieval functions

* chore: Remove redundant comments and clean up TODOs in OpenAI image tools

* refactor: Rename fileStrategy to appFileStrategy for clarity and improve error handling in image processing

* chore: Update react-resizable-panels to version 2.1.8 in package.json and package-lock.json

* chore: Ensure required validation for logs and Code of Conduct agreement in bug report template

* fix: Update ArtifactPreview to use startupConfig and currentCode from memoized props to prevent unnecessary re-renders

* fix: improve robustness of `save & submit` when used from a user-message with existing attachments

* fix: add null check for artifact index in CodeEditor to prevent errors, trigger re-render on artifact ID change

* fix: standardize default values for artifact properties in Artifact component, avoiding prematurely setting an "empty/default" artifact

* fix: reset current artifact ID before setting a new one in ArtifactButton to ensure correct state management

* chore: rename `setArtifactId` variable to `setCurrentArtifactId`  for consistency

* chore: update type annotations in File and S3 CRUD functions for consistency

* refactor: improve image handling in OpenAI tools by using image_id references and enhance tool context for image editing

* fix: update image_ids schema in image_edit_oai to enforce presence and provide clear guidelines for usage

* fix: enhance file fetching logic to ensure user-specific and dimension-validated results

* chore: add details on image generation and editing capabilities with various models
This commit is contained in:
Danny Avila 2025-04-26 04:30:58 -04:00 • committed by GitHub
parent 0ee1dcc479
commit c0ebb434a6
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
30 changed files with 841 additions and 104 deletions
Download patch file Download diff file Expand all files Collapse all files

1
client/src/components/Chat/Messages/Content/EditMessage.tsx
View file

@ -61,6 +61,7 @@ const EditMessage = ({
},
{
isResubmission: true,
overrideFiles: message.files,
},
);

19
client/src/components/Chat/Messages/Content/SearchContent.tsx
View file

@ -1,16 +1,23 @@
import { Suspense } from 'react';
import { Suspense, useMemo } from 'react';
import { useRecoilValue } from 'recoil';
import type { TMessage, TMessageContentParts } from 'librechat-data-provider';
import { ContentTypes } from 'librechat-data-provider';
import type { Agents, TMessage, TMessageContentParts } from 'librechat-data-provider';
import { UnfinishedMessage } from './MessageContent';
import { DelayedRender } from '~/components/ui';
import MarkdownLite from './MarkdownLite';
import { cn } from '~/utils';
import { cn, mapAttachments } from '~/utils';
import store from '~/store';
import Part from './Part';
const SearchContent = ({ message }: { message: TMessage }) => {
const enableUserMsgMarkdown = useRecoilValue(store.enableUserMsgMarkdown);
const { messageId } = message;
const messageAttachmentsMap = useRecoilValue(store.messageAttachmentsMap);
const attachmentMap = useMemo(
() => mapAttachments(message?.attachments ?? messageAttachmentsMap[messageId] ?? []),
[message?.attachments, messageAttachmentsMap, messageId],
);
if (Array.isArray(message.content) && message.content.length > 0) {
return (
<>
@ -20,13 +27,17 @@ const SearchContent = ({ message }: { message: TMessage }) => {
if (!part) {
return null;
}
const toolCallId =
(part?.[ContentTypes.TOOL_CALL] as Agents.ToolCall | undefined)?.id ?? '';
const attachments = attachmentMap[toolCallId];
return (
<Part
key={`display-${messageId}-${idx}`}
showCursor={false}
isSubmitting={false}
isCreatedByUser={message.isCreatedByUser}
messageId={message.messageId}
attachments={attachments}
part={part}
/>
);