mORMot Open Source

Gigo

Member

From: Split, Croatia

Registered: 2012-01-27

Posts: 16

High CPU load on TWebSocketAsyncServerRest

Using OpenSSL, Server created with TRestHttpServer.Create with aUse=useBidirAsync param.
When a client disconnects without upgrading to WebSockets, server starts to put load on CPU (threads W root, R0,R1,R2).

On destroy, I get 3 Shutdown unfinished lines :

mormot.net.ws.async.TWebSocketAsyncConnections(02d60700) Shutdown unfinished={"TAsyncConnectionsThread(028c7600)":{Process:"atpReadPending",Index:11,Name:"R11 root"}}
mormot.net.ws.async.TWebSocketAsyncConnections(02d60700) Shutdown unfinished={"TAsyncConnectionsThread(028c6fc0)":{Process:"atpReadPending",Index:1,Name:"R1 root"}}
mormot.net.ws.async.TWebSocketAsyncConnections(02d60700) Shutdown unfinished={"TAsyncConnectionsThread(028c81e0)":{Process:"atpReadPending",Index:30,Name:"R30 root"}}

If client upgrades connection to WebSockets, everything is normal.

Same behaviour on Win32 and Win64 target.
Delphi 10.3.1, using build 2.0.5007

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: High CPU load on TWebSocketAsyncServerRest

I am not able to reproduce the issue here.

Do the mormot2tests regression tests pass on your computer?

And without OpenSSL?

Could you try to reproduce within the debugger, and see where the lock/CPU burn happen in TAsyncConnectionsThread.Execute ?

Please also check if something changes with https://github.com/synopse/mORMot2/commit/20b5f3c1

Gigo

Member

From: Split, Croatia

Registered: 2012-01-27

Posts: 16

Re: High CPU load on TWebSocketAsyncServerRest

- mormot2tests regression tests passes without any regressions.
- Without OpenSSL program works as expected, without heavy CPU load.
- I've tried both OpenSSL 1.1 and OpenSSL3 with same result.
- Same thing happens with new commit.

In TAsyncConnectionsThread.Execute program loops at

  ...
      case fProcess of
      ...
        atpReadPending:
          // secondary threads wait, then read and process pending events
          begin
            fWaitForReadPending := true;
            fEvent.WaitForEver;                              //// Executes and returns
            if Terminated then
              break;
            fWakeUpFromSlowProcess := false;
            while GetNextRead(notif) do
              fOwner.fClients.ProcessRead(self, notif);     //// Executes and returns
            fThreadPollingLastWakeUpTix := 0; // will now need to wakeup
            fOwner.fThreadReadPoll.ReleaseEvent; // atpReadPoll lock above
          end;
      ...

To me, looks like fEvent.Handle should be reset, but it's not, so it loops continuously, banging on CPU.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: High CPU load on TWebSocketAsyncServerRest

Try to add:

        atpReadPending:
          // secondary threads wait, then read and process pending events
          begin
            fWaitForReadPending := true;
            fEvent.ResetEvent;     /////////// ADD THIS
            fEvent.WaitForEver;

Gigo

Member

From: Split, Croatia

Registered: 2012-01-27

Posts: 16

Re: High CPU load on TWebSocketAsyncServerRest

            fEvent.ResetEvent;     /////////// ADD THIS

Solved. Please commit.

Btw, I was able to reproduce the issue on another computer with Delphi 10.4.1

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: High CPU load on TWebSocketAsyncServerRest

Should be available with https://github.com/synopse/mORMot2/commit/24ca48ab

Thanks a lot for the feedback and investigation!