🪐 feat: Cross-replica support in GenerationJobManager for Redis mode (#11233)
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run

* feat: Implement cross-replica support in GenerationJobManager for Redis mode

- Enhanced GenerationJobManager to support cross-replica scenarios by lazily creating runtime states from Redis when jobs exist but are not present in local memory.
- Added functionality to persist `syncSent` and `finalEvent` states to Redis for consistency across replicas.
- Implemented abort signal handling to allow replicas to receive and respond to abort requests from other instances, ensuring proper job termination.
- Updated tests to validate cross-replica behavior, including job retrieval, subscription, and abort signal propagation.

This update improves the robustness and reliability of job management in distributed environments.

* fix: Enhance error handling and implement abort signal for cross-replica jobs in GenerationJobManager

- Added error handling for Redis job updates in GenerationJobManager to log failures when persisting `syncSent` and `finalEvent` states.
- Implemented a listener for cross-replica abort signals, ensuring that lazily-initialized jobs can respond to abort requests from other replicas.
- Introduced a new integration test to validate the handling of abort signals for lazily-initialized jobs across replicas.

These changes improve the reliability and robustness of job management in distributed environments.
This commit is contained in:
Danny Avila 2026-01-06 11:39:24 -05:00 committed by GitHub
parent b5aa38ff33
commit a7645f4705
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 826 additions and 7 deletions

View file

@ -236,6 +236,21 @@ export interface IEventTransport {
/** Publish an error event */
emitError(streamId: string, error: string): void;
/**
* Publish an abort signal to all replicas (Redis mode).
* Enables cross-replica abort: user aborts on Replica B,
* generating Replica A receives signal and stops.
* Optional - only implemented in Redis transport.
*/
emitAbort?(streamId: string): void;
/**
* Register callback for abort signals from any replica (Redis mode).
* Called when abort is triggered from any replica.
* Optional - only implemented in Redis transport.
*/
onAbort?(streamId: string, callback: () => void): void;
/** Get subscriber count for a stream */
getSubscriberCount(streamId: string): number;