🎨 feat: OpenAI Image Tools (GPT-Image-1) (#7079)

* wip: OpenAI Image Generation Tool with customizable options

* WIP: First pass OpenAI Image Generation Tool and integrate into existing tools

* 🔀 fix: Comment out unused validation for image generation tool parameters

* 🔀 refactor: Update primeResources function parameters for better destructuring

* feat: Add image_edit resource to EToolResources and update AgentToolResources interface

* feat: Enhance file retrieval with tool resource filtering for image editing

* refactor: add OpenAI Image Tools for generation and editing, refactor related components, pass current request image attachments as tool resources for editing

* refactor: Remove commented-out code and clean up API key retrieval in createOpenAIImageTools function

* fix: show message attachments in shared links

* fix: Correct parent message retrieval logic for regenerated messages in useChatFunctions

* fix: Update primeResources to utilize requestFileSet for image file processing

* refactor: Improve description for image generation tool and clarify usage conditions, only provide edit tool if there are images available to edit

* chore: Update OpenAI Image Tools icon to use local asset

* refactor: Update image generation tool description and logic to prioritize editing tool when files are uploaded

* refactor: Enhance image tool descriptions to clarify usage conditions and note potential unavailability of uploaded images

* refactor: Update useAttachmentHandler to accept queryClient to update query cache with newly created file

* refactor: Add customizable descriptions and prompts for OpenAI image generation and editing tools

* chore: Update comments to use JSDoc style for better clarity and consistency

* refactor: Rename config variable to clientConfig for clarity and update signal handling in image generation

* refactor: Update axios request configuration to include derived signal and baseURL for improved request handling

* refactor: Update baseURL environment variable for OpenAI image generation tool configuration

* refactor: Enhance axios request configuration with conditional headers and improved clientConfig setup

* chore: Update comments for clarity and remove unnecessary lines in OpenAI image tools

* refactor: Update description for image generation without files to clarify user instructions

* refactor: Simplify target parent message logic for regeneration and resubmission cases

* chore: Remove backticks from error messages in image generation and editing functions

* refactor: Rename toolResources to toolResourceSet for clarity in file retrieval functions

* chore: Remove redundant comments and clean up TODOs in OpenAI image tools

* refactor: Rename fileStrategy to appFileStrategy for clarity and improve error handling in image processing

* chore: Update react-resizable-panels to version 2.1.8 in package.json and package-lock.json

* chore: Ensure required validation for logs and Code of Conduct agreement in bug report template

* fix: Update ArtifactPreview to use startupConfig and currentCode from memoized props to prevent unnecessary re-renders

* fix: improve robustness of `save & submit` when used from a user-message with existing attachments

* fix: add null check for artifact index in CodeEditor to prevent errors, trigger re-render on artifact ID change

* fix: standardize default values for artifact properties in Artifact component, avoiding prematurely setting an "empty/default" artifact

* fix: reset current artifact ID before setting a new one in ArtifactButton to ensure correct state management

* chore: rename `setArtifactId` variable to `setCurrentArtifactId`  for consistency

* chore: update type annotations in File and S3 CRUD functions for consistency

* refactor: improve image handling in OpenAI tools by using image_id references and enhance tool context for image editing

* fix: update image_ids schema in image_edit_oai to enforce presence and provide clear guidelines for usage

* fix: enhance file fetching logic to ensure user-specific and dimension-validated results

* chore: add details on image generation and editing capabilities with various models
This commit is contained in:
Danny Avila 2025-04-26 04:30:58 -04:00 committed by GitHub
parent 0ee1dcc479
commit c0ebb434a6
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
30 changed files with 841 additions and 104 deletions

21
package-lock.json generated
View file

@ -1222,7 +1222,7 @@
"react-i18next": "^15.4.0",
"react-lazy-load-image-component": "^1.6.0",
"react-markdown": "^9.0.1",
"react-resizable-panels": "^2.1.7",
"react-resizable-panels": "^2.1.8",
"react-router-dom": "^6.11.2",
"react-speech-recognition": "^3.10.0",
"react-textarea-autosize": "^8.4.0",
@ -3266,15 +3266,6 @@
"node": ">=0.10.0"
}
},
"client/node_modules/react-resizable-panels": {
"version": "2.1.7",
"resolved": "https://registry.npmjs.org/react-resizable-panels/-/react-resizable-panels-2.1.7.tgz",
"integrity": "sha512-JtT6gI+nURzhMYQYsx8DKkx6bSoOGFp7A3CwMrOb8y5jFHFyqwo9m68UhmXRw57fRVJksFn1TSlm3ywEQ9vMgA==",
"peerDependencies": {
"react": "^16.14.0 || ^17.0.0 || ^18.0.0 || ^19.0.0 || ^19.0.0-rc",
"react-dom": "^16.14.0 || ^17.0.0 || ^18.0.0 || ^19.0.0 || ^19.0.0-rc"
}
},
"client/node_modules/ts-jest": {
"version": "29.2.5",
"resolved": "https://registry.npmjs.org/ts-jest/-/ts-jest-29.2.5.tgz",
@ -38092,6 +38083,16 @@
}
}
},
"node_modules/react-resizable-panels": {
"version": "2.1.8",
"resolved": "https://registry.npmjs.org/react-resizable-panels/-/react-resizable-panels-2.1.8.tgz",
"integrity": "sha512-oDvD0sw34Ecx00cQFLiRJpAE2fCgNLBr8DMrBzkrsaUiLpAycIQoY3eAWfMblDql3pTIMZ60wJ/P89RO1htM2w==",
"license": "MIT",
"peerDependencies": {
"react": "^16.14.0 || ^17.0.0 || ^18.0.0 || ^19.0.0 || ^19.0.0-rc",
"react-dom": "^16.14.0 || ^17.0.0 || ^18.0.0 || ^19.0.0 || ^19.0.0-rc"
}
},
"node_modules/react-router": {
"version": "6.22.0",
"resolved": "https://registry.npmjs.org/react-router/-/react-router-6.22.0.tgz",