mirror of
https://github.com/danny-avila/LibreChat.git
synced 2025-12-17 17:00:15 +01:00
refactor: Update GenerationJobManager documentation and structure
- Enhanced the documentation for GenerationJobManager to clarify the architecture and pluggable service design. - Updated comments to reflect the potential for Redis integration and the need for async refactoring. - Improved the structure of the GenerationJob facade to emphasize the unified API while allowing for implementation swapping without affecting consumer code.
This commit is contained in:
parent
75a01f222d
commit
3d28289b52
1 changed files with 26 additions and 7 deletions
|
|
@ -33,14 +33,30 @@ interface RuntimeJobState {
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Manages generation jobs for resumable LLM streams.
|
* Manages generation jobs for resumable LLM streams.
|
||||||
* Composes three implementations for clean separation of concerns:
|
*
|
||||||
* - InMemoryJobStore: Serializable job metadata (swappable for Redis)
|
* Architecture: Composes three pluggable services for clean separation:
|
||||||
* - InMemoryEventTransport: Pub/sub events (swappable for Redis Pub/Sub)
|
* - jobStore: Serializable job metadata (InMemory → Redis/KV for horizontal scaling)
|
||||||
* - InMemoryContentState: Volatile content refs with WeakRef (always in-memory)
|
* - eventTransport: Pub/sub events (InMemory → Redis Pub/Sub for horizontal scaling)
|
||||||
|
* - contentState: Volatile content refs with WeakRef (always in-memory, not shared)
|
||||||
|
*
|
||||||
|
* Current implementation uses sync methods for performance. When adding Redis support,
|
||||||
|
* the manager methods will need to become async, or use a sync-capable Redis client.
|
||||||
|
*
|
||||||
|
* @example Future Redis injection (requires async refactor):
|
||||||
|
* ```ts
|
||||||
|
* const manager = new GenerationJobManagerClass({
|
||||||
|
* jobStore: new RedisJobStore(redisClient),
|
||||||
|
* eventTransport: new RedisPubSubTransport(redisClient),
|
||||||
|
* contentState: new InMemoryContentState(), // Always local
|
||||||
|
* });
|
||||||
|
* ```
|
||||||
*/
|
*/
|
||||||
class GenerationJobManagerClass {
|
class GenerationJobManagerClass {
|
||||||
|
/** Job metadata storage - swappable for Redis, KV store, etc. */
|
||||||
private jobStore: InMemoryJobStore;
|
private jobStore: InMemoryJobStore;
|
||||||
|
/** Event pub/sub transport - swappable for Redis Pub/Sub, etc. */
|
||||||
private eventTransport: InMemoryEventTransport;
|
private eventTransport: InMemoryEventTransport;
|
||||||
|
/** Volatile content state with WeakRef - always in-memory per instance */
|
||||||
private contentState: InMemoryContentState;
|
private contentState: InMemoryContentState;
|
||||||
|
|
||||||
/** Runtime state - always in-memory, not serializable */
|
/** Runtime state - always in-memory, not serializable */
|
||||||
|
|
@ -146,9 +162,12 @@ class GenerationJobManagerClass {
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Build a GenerationJob facade from job data and runtime state.
|
* Build a GenerationJob facade from composed services.
|
||||||
* This maintains backwards compatibility with existing code that expects
|
*
|
||||||
* job.emitter, job.abortController, etc.
|
* This facade provides a unified API (job.emitter, job.abortController, etc.)
|
||||||
|
* while internally delegating to the injected services (jobStore, eventTransport,
|
||||||
|
* contentState). This allows swapping implementations (e.g., Redis) without
|
||||||
|
* changing consumer code.
|
||||||
*
|
*
|
||||||
* IMPORTANT: The emitterProxy.on('allSubscribersLeft') handler registration
|
* IMPORTANT: The emitterProxy.on('allSubscribersLeft') handler registration
|
||||||
* does NOT use eventTransport.subscribe(). This is intentional:
|
* does NOT use eventTransport.subscribe(). This is intentional:
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue