mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-03-23 16:16:34 +01:00
🖼️ fix: Correct ToolMessage Response Format for Agent-Mode Image Tools (#12310)
* fix: Set response format for agent tools in DALLE3, FluxAPI, and StableDiffusion classes - Added logic to set `responseFormat` to 'content_and_artifact' when `isAgent` is true in DALLE3.js, FluxAPI.js, and StableDiffusion.js. * test: Add regression tests for image tool agent mode in imageTools-agent.spec.js - Introduced a new test suite for DALLE3, FluxAPI, and StableDiffusion classes to verify that the invoke() method returns a ToolMessage with base64 in artifact.content, ensuring it is not serialized into content. - Validated that responseFormat is set to 'content_and_artifact' when isAgent is true, and confirmed the correct handling of base64 data in the response. * fix: handle agent error paths and generateFinetunedImage in image tools - StableDiffusion._call() was returning a raw string on API error, bypassing returnValue() and breaking the content_and_artifact contract when isAgent is true - FluxAPI.generateFinetunedImage() had no isAgent branch; it would call processFileURL (unset in agent context) instead of fetching and returning the base64 image as an artifact tuple - Add JSDoc to all three responseFormat assignments clarifying why LangChain requires this property for correct ToolMessage construction * test: expand image tool agent mode regression suite - Add env var save/restore in beforeEach/afterEach to prevent test pollution - Add error path tests for all three tools verifying ToolMessage content and artifact are correctly populated when the upstream API fails - Add generate_finetuned action test for FluxAPI covering the new agent branch in generateFinetunedImage * chore: fix lint errors in FluxAPI and imageTools-agent spec * chore: fix import ordering in imageTools-agent spec
This commit is contained in:
parent
93952f06b4
commit
a88bfae4dd
4 changed files with 338 additions and 8 deletions
|
|
@ -43,6 +43,10 @@ class StableDiffusionAPI extends Tool {
|
|||
this.returnMetadata = fields.returnMetadata ?? false;
|
||||
/** @type {boolean} */
|
||||
this.isAgent = fields.isAgent;
|
||||
if (this.isAgent) {
|
||||
/** Ensures LangChain maps [content, artifact] tuple to ToolMessage fields instead of serializing it into content. */
|
||||
this.responseFormat = 'content_and_artifact';
|
||||
}
|
||||
if (fields.uploadImageBuffer) {
|
||||
/** @type {uploadImageBuffer} Necessary for output to contain all image metadata. */
|
||||
this.uploadImageBuffer = fields.uploadImageBuffer.bind(this);
|
||||
|
|
@ -115,7 +119,7 @@ class StableDiffusionAPI extends Tool {
|
|||
generationResponse = await axios.post(`${url}/sdapi/v1/txt2img`, payload);
|
||||
} catch (error) {
|
||||
logger.error('[StableDiffusion] Error while generating image:', error);
|
||||
return 'Error making API request.';
|
||||
return this.returnValue('Error making API request.');
|
||||
}
|
||||
const image = generationResponse.data.images[0];
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue