mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-01-22 18:26:12 +01:00
🪐 feat: Cross-replica support in GenerationJobManager for Redis mode (#11233)
* feat: Implement cross-replica support in GenerationJobManager for Redis mode - Enhanced GenerationJobManager to support cross-replica scenarios by lazily creating runtime states from Redis when jobs exist but are not present in local memory. - Added functionality to persist `syncSent` and `finalEvent` states to Redis for consistency across replicas. - Implemented abort signal handling to allow replicas to receive and respond to abort requests from other instances, ensuring proper job termination. - Updated tests to validate cross-replica behavior, including job retrieval, subscription, and abort signal propagation. This update improves the robustness and reliability of job management in distributed environments. * fix: Enhance error handling and implement abort signal for cross-replica jobs in GenerationJobManager - Added error handling for Redis job updates in GenerationJobManager to log failures when persisting `syncSent` and `finalEvent` states. - Implemented a listener for cross-replica abort signals, ensuring that lazily-initialized jobs can respond to abort requests from other replicas. - Introduced a new integration test to validate the handling of abort signals for lazily-initialized jobs across replicas. These changes improve the reliability and robustness of job management in distributed environments.
This commit is contained in:
parent
b5aa38ff33
commit
a7645f4705
5 changed files with 826 additions and 7 deletions
|
|
@ -236,6 +236,21 @@ export interface IEventTransport {
|
|||
/** Publish an error event */
|
||||
emitError(streamId: string, error: string): void;
|
||||
|
||||
/**
|
||||
* Publish an abort signal to all replicas (Redis mode).
|
||||
* Enables cross-replica abort: user aborts on Replica B,
|
||||
* generating Replica A receives signal and stops.
|
||||
* Optional - only implemented in Redis transport.
|
||||
*/
|
||||
emitAbort?(streamId: string): void;
|
||||
|
||||
/**
|
||||
* Register callback for abort signals from any replica (Redis mode).
|
||||
* Called when abort is triggered from any replica.
|
||||
* Optional - only implemented in Redis transport.
|
||||
*/
|
||||
onAbort?(streamId: string, callback: () => void): void;
|
||||
|
||||
/** Get subscriber count for a stream */
|
||||
getSubscriberCount(streamId: string): number;
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue