mORMot Open Source

keinn

Member

Registered: 2014-10-20

Posts: 103

seems that the websocket recv latency is too high

client and server at the same machine,
server use TWebSocketAsyncServerRest,
client use THttpClientWebSockets,

mark timestamp both before sending

and at recving (in TOnWebSocketProtocolChatIncomingFrame callback)

(no other time consumeing process),

it has atleast 30~50ms delay ,even set Settings^.LoopDelay to 1 or 0.

i think in localhost it would not have such big latency

any chance to improve this ? it is very important in a latency sensitive system

Last edited by keinn (2023-08-02 09:07:51)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: seems that the websocket recv latency is too high

The WebSocket server and client have a small latency by design for non-blocking methods:
- output frames are gathered together before sending, if several calls are made
- it scales better to send all pending notifications from a dedicated "write" thread
The LoopDelay parameter does not change this design.

What a "non-blocking method" is a method with not result nor output, i.e. a procedure with no "out" nor "var" parameter.
It is typically an event.
My guess is that if you add a "result" to the method as a function, you won't see the latency any more.

keinn

Member

Registered: 2014-10-20

Posts: 103

Re: seems that the websocket recv latency is too high

my finding is that latency happened in recv,
because i use other websocket server lib like “delphi-cross-socket”, client still use our mormot THttpClientWebSockets,
the latency is less then 1ms.
test is client send to server.

about "non-blocking-method",what and where should i change? i test this within mormot pattern,in chat protocol,
TOnWebSocketProtocolChatIncomingFrame,  it always get triggered with 30+ms delay.
so it may not about non-blocking-sending

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: seems that the websocket recv latency is too high

You are right, the latency is in THttpClientWebSockets, but onlly when it is in idle mode.

If there is another request very soon, there is no latency: as we can see from the regression tests, the WS performance for requests in a loop show no such latency:

  - TCP sockets: 9,890 assertions passed  587.34ms
     1=10234/s  2=19372/s  5=16467/s  10=15707/s  30=18753/s  50=16140/s  
  - Unix domain sockets: 8,427 assertions passed  585.59ms
     1=13620/s  2=25040/s  5=66673/s  10=37573/s  30=23549/s  50=21036/s  
  - Websockets: 8,409 assertions passed  485.65ms
     1=6110/s  2=12997/s  5=13619/s  10=13909/s  30=12276/s  50=16570/s  
 

We can reach 16K requests per second, so there is no latency any more I guess.

This is how the THttpClientWebSockets thread is implemented in TWebSocketProcess.ProcessLoop.
And in fact, LoopDelay = 1 should reduce this latency somewhat, but due to Windows limitation and its 16ms sleep() timer resolution, there is a bigger latency than expected.

Honestly, I do not see the problem of a small client-side latency in such case, because the benefit of being able to gather outgoing frames is higher than the downside of waiting a few ms.

keinn

Member

Registered: 2014-10-20

Posts: 103

Re: seems that the websocket recv latency is too high

well ,it's not only in the client side .

as i tested, i replace server side with other library , client side still use THttpClientWebSockets, the problem is gone .

latency is happened in recv ,both client and server side , TWebSocketProcess.ProcessLoop effect both.

normally ,it is not a problem . but when it comes to time sensitive case , it is a big problem. It is not about only about QPS.

like :send a trading signal to other side, other side should react as soon as posible ,otherwise , money will lose.

and i thought mormot2 use system event driven mode ,so why need hand made dely?

or is posible workaround this ?

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: seems that the websocket recv latency is too high

TWebSocketProcess.ProcessLoop does not affect the Async WS server IIRC, because it is not called by it.

There may be a small latency on an idle Async server for sure, due to how its main thread work at idle process.
But on a non-idle server, there is no such latency. Its thread usage helps scaling with high numbers, e.g. 7M req/s for TFB benchmarks.

IMHO your concern is very theoretical.
If you have a lot of concurrent clients, there is no latency with the Async server.
If you have a few clients, don't use the Async server, and you won't have any latency. Or use the Async server, and since you have a few clients, you are likely to have no latency requirements anyway. Or even better, on the localhost, do not use any by-process communication, but a single process.

keinn

Member

Registered: 2014-10-20

Posts: 103

Re: seems that the websocket recv latency is too high

localhost is only for test.
clients are few, just have to send realtime signal over Internet.
as you say, i should not use AsyncServer，i will try socket based server for this.
thanks a lot.