feat: Implement Resumable Generation Jobs with SSE Support

- Introduced GenerationJobManager to handle resumable LLM generation jobs independently of HTTP connections.
- Added support for subscribing to ongoing generation jobs via SSE, allowing clients to reconnect and receive updates without losing progress.
- Enhanced existing agent controllers and routes to integrate resumable functionality, including job creation, completion, and error handling.
- Updated client-side hooks to manage adaptive SSE streams, switching between standard and resumable modes based on user settings.
- Added UI components and settings for enabling/disabling resumable streams, improving user experience during unstable connections.
This commit is contained in:
Danny Avila 2025-12-03 21:48:04 -05:00
parent 3213f574c6
commit 6bb2fac0ec
No known key found for this signature in database
GPG key ID: BF31EEB2C5CA0956
17 changed files with 1212 additions and 37 deletions

View file

@ -16,6 +16,7 @@ const {
performStartupChecks,
handleJsonParseError,
initializeFileStorage,
GenerationJobManager,
} = require('@librechat/api');
const { connectDb, indexSync } = require('~/db');
const initializeOAuthReconnectManager = require('./services/initializeOAuthReconnectManager');
@ -192,6 +193,7 @@ const startServer = async () => {
await initializeMCPs();
await initializeOAuthReconnectManager();
await checkMigrations();
GenerationJobManager.initialize();
});
};