feat: Implement Resumable Generation Jobs with SSE Support

- Introduced GenerationJobManager to handle resumable LLM generation jobs independently of HTTP connections.
- Added support for subscribing to ongoing generation jobs via SSE, allowing clients to reconnect and receive updates without losing progress.
- Enhanced existing agent controllers and routes to integrate resumable functionality, including job creation, completion, and error handling.
- Updated client-side hooks to manage adaptive SSE streams, switching between standard and resumable modes based on user settings.
- Added UI components and settings for enabling/disabling resumable streams, improving user experience during unstable connections.
This commit is contained in:
Danny Avila 2025-12-03 21:48:04 -05:00
parent 3213f574c6
commit 6bb2fac0ec
No known key found for this signature in database
GPG key ID: BF31EEB2C5CA0956
17 changed files with 1212 additions and 37 deletions

View file

@ -65,18 +65,21 @@ const initializeClient = async ({ req, res, signal, endpointOption }) => {
}
const appConfig = req.config;
// TODO: use endpointOption to determine options/modelOptions
/** @type {string | null} */
const streamId = req._resumableStreamId || null;
/** @type {Array<UsageMetadata>} */
const collectedUsage = [];
/** @type {ArtifactPromises} */
const artifactPromises = [];
const { contentParts, aggregateContent } = createContentAggregator();
const toolEndCallback = createToolEndCallback({ req, res, artifactPromises });
const toolEndCallback = createToolEndCallback({ req, res, artifactPromises, streamId });
const eventHandlers = getDefaultHandlers({
res,
aggregateContent,
toolEndCallback,
collectedUsage,
streamId,
});
if (!endpointOption.agent) {