mORMot Open Source

ec

Member

Registered: 2023-08-24

Posts: 37

Proper Backpressure handling in TWebSocketAsyncServer for broadcast

Hi Arnaud,

I am currently leveraging the excellent TWebSocketAsyncServer in mORMot 2 for a high-throughput application involving massive broadcasting from server to clients.

While implementing the asynchronous transmission, I noticed that under heavy network congestion or with slow clients, the outbound data can accumulate significantly. To ensure server stability and avoid excessive memory consumption (or potential contention during WaitLock), I am looking for the "mORMot-way" of detecting the optimal moment to push more data into the websocket.

Specifically:

Is monitoring TPollAsyncConnection.fWr.Len the recommended approach to implement a backpressure mechanism, or is there a higher-level event/callback in the TWebSocketAsyncProcess that I should use to detect when the outgoing buffer has reached a safe threshold?

In the context of THttpAsyncServer, would you recommend manually dropping frames when the outbound buffer exceeds a certain limit, or is there a built-in gathering strategy (like SendDelay) that I should tune to handle thousands of concurrent massive sends more gracefully?

I want to avoid any "freezing" effects caused by RAM saturation or lock contention when the kernel's TCP buffers are full.

Thank you for your tireless work on this amazing framework.

Best regards

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 15,447

Website

Re: Proper Backpressure handling in TWebSocketAsyncServer for broadcast

Dropping frames could be an option, but it depends on your actual use-case.

For instance, compressed video content flow may accept or not accept frame drops, depending on the algorithm.
Or you could use a specific "flush" message when RAM is saturated.
But in this case, it could be handy to flush the data pending in the mormot async server buffers, because it is likely to be deprecated. I am not sure how to flush kernel's TCP buffers anyway...
Of course, for a websockets server, flushing data should perhaps be done at the websockets frames layer, not in the TCP layer, to avoid any broken frame content...

ec

Member

Registered: 2023-08-24

Posts: 37

Re: Proper Backpressure handling in TWebSocketAsyncServer for broadcast

Hi Arnaud,

Thank you for the insights.

Actually, dropping frames or flushing the buffers is exactly what I would like to avoid. My goal is to ensure data integrity without losing any frames, even when the RAM reaches a critical point.

Instead of discarding data, I am looking for a way to implement proper backpressure within the mORMot 2 async server architecture. My intention is to have the server 'wait' or slow down the injection of new frames until the current buffers are successfully transmitted and acknowledged at the TCP/Websocket layer.

Is there a recommended way in mORMot 2 to monitor the state of the async output buffers or the underlying socket readiness, so I can pause the data flow instead of saturating the RAM or being forced to flush pending data?

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 15,447

Website

Re: Proper Backpressure handling in TWebSocketAsyncServer for broadcast

But how do you pause the data flow, if you can't send any data?

Anyway, perhaps retrieving the number pending bytes in the connection output buffer may help you:
https://github.com/synopse/mORMot2/commit/45162a1fc

ec

Member

Registered: 2023-08-24

Posts: 37

Re: Proper Backpressure handling in TWebSocketAsyncServer for broadcast

Hi Arnaud,

Thank you very much for the prompt update! The PendingWrite property is exactly what I needed to implement a smarter flow control.

Regarding your question on how to pause the flow: my plan is to use a threshold-based approach.

Since I am streaming frames (like video or large DICOM datasets), I can now check PendingWrite before injecting the next chunk into the async server. If PendingWrite exceeds a predefined safety limit (e.g., 2MB), I will pause the frame producer (the source) and wait for the buffer to drain below a certain 'low-water mark' before resuming.

This way, I can leverage mORMot's high-performance async architecture without overwhelming the server's RAM when the network throughput is slower than the data production rate.

I really appreciate the support and the commitment to mORMot 2!