refactor: Enhance GenerationJobManager with In-Memory Implementations

- Introduced InMemoryJobStore, InMemoryEventTransport, and InMemoryContentState for improved job management and event handling.
- Updated GenerationJobManager to utilize these new implementations, allowing for better separation of concerns and easier maintenance.
- Enhanced job metadata handling to support user messages and response IDs for resumable functionality.
- Improved cleanup and state management processes to prevent memory leaks and ensure efficient resource usage.
This commit is contained in:
Danny Avila 2025-12-12 02:16:24 -05:00
parent 5ff66f2d77
commit ff86f96416
No known key found for this signature in database
GPG key ID: BF31EEB2C5CA0956
14 changed files with 892 additions and 321 deletions

View file

@ -177,10 +177,16 @@ const ResumableAgentController = async (req, res, next, initializeClient, addTit
const onStart = (userMsg, respMsgId, _isNewConvo) => {
userMessage = userMsg;
// Store the response messageId upfront so partial saves use the same ID
if (respMsgId) {
GenerationJobManager.updateMetadata(streamId, { responseMessageId: respMsgId });
}
// Store userMessage and responseMessageId upfront for resume capability
GenerationJobManager.updateMetadata(streamId, {
responseMessageId: respMsgId,
userMessage: {
messageId: userMsg.messageId,
parentMessageId: userMsg.parentMessageId,
conversationId: userMsg.conversationId,
text: userMsg.text,
},
});
GenerationJobManager.emitChunk(streamId, {
created: true,