🔄 refactor: Sequential Event Ordering in Redis Streaming Mode (#11650)

* chore: linting image context file

* refactor: Event Emission with Async Handling for Redis Ordering

- Updated emitEvent and related functions to be async, ensuring proper event ordering in Redis mode.
- Refactored multiple handlers to await emitEvent calls, improving reliability for streaming deltas.
- Enhanced GenerationJobManager to await chunk emissions, critical for maintaining sequential event delivery.
- Added tests to verify that events are delivered in strict order when using Redis, addressing previous issues with out-of-order messages.

* refactor: Clear Pending Buffers and Timeouts in RedisEventTransport

- Enhanced the cleanup process in RedisEventTransport by ensuring that pending messages and flush timeouts are cleared when the last subscriber unsubscribes.
- Updated the destroy method to also clear pending messages and flush timeouts for all streams, improving resource management and preventing memory leaks.

* refactor: Update Event Emission to Async for Improved Ordering

- Refactored GenerationJobManager and RedisEventTransport to make emitDone and emitError methods async, ensuring proper event ordering in Redis mode.
- Updated all relevant calls to await these methods, enhancing reliability in event delivery.
- Adjusted tests to verify that events are processed in the correct sequence, addressing previous issues with out-of-order messages.

* refactor: Adjust RedisEventTransport for 0-Indexed Sequence Handling

- Updated sequence handling in RedisEventTransport to be 0-indexed, ensuring consistency across event emissions and buffer management.
- Modified integration tests to reflect the new sequence logic, improving the accuracy of event processing and delivery order.
- Enhanced comments for clarity on sequence management and terminal event handling.

* chore: Add Redis dump file to .gitignore

- Included dump.rdb in .gitignore to prevent accidental commits of Redis database dumps, enhancing repository cleanliness and security.

* test: Increase wait times in RedisEventTransport integration tests for CI stability

- Adjusted wait times for subscription establishment and event propagation from 100ms and 200ms to 500ms to improve reliability in CI environments.
- Enhanced code readability by formatting promise resolution lines for better clarity.

api/server/services/ToolService.js

@@ -526,8 +526,8 @@ async function loadToolDefinitionsWrapper({ req, res, agent, streamId = null, to
       const runStepDeltaEvent = { event: GraphEvents.ON_RUN_STEP_DELTA, data: runStepDeltaData };
 
       if (streamId) {
-        GenerationJobManager.emitChunk(streamId, runStepEvent);
-        GenerationJobManager.emitChunk(streamId, runStepDeltaEvent);
+        await GenerationJobManager.emitChunk(streamId, runStepEvent);
+        await GenerationJobManager.emitChunk(streamId, runStepDeltaEvent);
       } else if (res && !res.writableEnded) {
         sendEvent(res, runStepEvent);
         sendEvent(res, runStepDeltaEvent);