Help - mORMot2 Raw TCP Async Client & Server

Coldzer0 · 2021-10-25 12:14:47

Hello guys

I was trying to create an Async raw TCP client & server.

I Created A Server and it works very well It can handle thousands of concurrent connections.

I tested it with a tool for stress testing and I got (about 13,000+ (limited by the client machine) connections On Virtual RDP Server : CPU=5xIntel(R)Xeon(R)CPUE5-24200@1.90GHz(x64) 5 Cores - RAM=8GB ).

Echo Server Code:
https://gist.github.com/Coldzer0/d00c04 … 72765506c5

But when I tried to Create Client with the Code of use of mORMot2 TAsyncClient Everything works except for the OnRead, when I try to send any data to the client the OnRead never called.

Client Code:
https://gist.github.com/Coldzer0/8c98a0 … 89791f55e4

And I've another question about Async send, How can I make sure that the server received the sent data or not ?

And please let me know If I'm doing anything wrong, And If there's any good demo of showing how to deal with RAW TCP Client & Server with mORMot2.

ab · 2021-10-25 17:22:09

Thanks for the feedback, especially the good stability of the async server.
Do you have any number to share?
From what I have measured, the async server is slower for a few connections (one or two e.g.) in respect to the thread-based server, but much more stable and using very little resource when the number of connections grows.

I didn't test the async clients yet....
because we didn't have any use of it at the moment.

So I don't know exactly what is going on. Your code seems fine and it is very likely than the TAsyncClient code is the culprit.
Could you make a pull request and include your code to the regression tests, e.g. in TNetworkProtocols from test.net.proto.pas ?
It would help me debugging the problem.

About Async send() the sending is asynchronous by definition, so you can't be notified that the server actually received the data. But it is also the case for plain TCP IIRC.
If you need an acknolewdge, you may need to code an explicit answer.

Coldzer0 · 2021-10-25 19:40:32

Thanks ab for reply.

First of all thanks so much for the amazing work in mORMot2.

Do you have any number to share?

Sure

The Sever RDP
The Machine I uses for Server is "CPU= 5 x Intel(R)Xeon(R)CPUE5-24200@1.90GHz(x64) - RAM= 8GB"

This is a x64 Debug Build with Delphi Memory Manager.

Built with Delphi 11 Alexandria : Debug Mode
Memory Usage: About "50MB" for "11,166" concurrent Connections With 16 Async Threads (TAsyncServer).
CPU Usage   : Between 17% & 18% never higher.

Even when it reach the 13,000+ It was still very small memory usage about "52-54 MB"

Here's a screenshot from when I was testing.
Screenshot from server while testing

The Client Side: (I didn't use mORMot2 I used a tool called EchoServerTest to do the testing)
The Stress testing Machine is "CPU=16 x Intel(R)Core(TM)i9-9900KCPU@3.60GHz(x64) RAM= 32GB"

I was aiming for 40000 concurrent connections But windows has limits, I think there's some registry options I can edit to extend the numbers of connections.

Anyway I'll do the test again but with Linux x64 (Will test Release Build Later with mORMot2 Memory Manager).

Could you make a pull request and include your code to the regression tests

Yes no problem I will try my best.

so you can't be notified that the server actually received the data. But it is also the case for plain TCP IIRC.

In normal blocking mode you got the sent length so you can make sure how much of your buffer sent to the server.

But I'll implement custom acknowledgment send by the server to be confirmed by the client as you advised.

I've a question about the internal buffer of the sockets, How can I read a specific size of data keeping in mind that the read and write in Async.
What I mean is how can I make sure that what I receive is the full packet if the size if like 1MB or Bigger keeping in mind that the maximum buffer described in code is "8KB".

Coldzer0 · 2021-10-26 10:37:50

Hello ab

So basically I took a good look at the TAsyncServer and the TAsyncClient and the main difference is the Server has a mechanism to notify the ReadPoll to handle new subscriptions, But the Client didn't have.

So What I did is adding the following code in the "ConnectionCreate" CallBack.

// assign the new connection to the internal reading poll.
If Clients.Start(aconnection) Then
Begin
    // release atpReadPoll lock to handle new subscription ASAP
    ThreadPollingWakeup(fClients.PollRead.PollForPendingEvents(0));

    // Send 2MB As Test - we should receive it
    WriteString(aConnection, DupeString('A', 1024*1024*2));
    Result := True;
End

Here's the new Client Code
https://gist.github.com/Coldzer0/226e0d … 4485594e28

It Works well for Reading and Writing (Still didn't test it fully).

I didn't add my code in a PR because I didn't know how it suppose to be handled in the test ( I'm new to mORMot internals ).

Please Check it and let me know if it's Ok to use it like this or if there's a better way.

Thanks.

ab · 2021-10-26 11:54:06

Note that with a lot of concurrent connections (more than a few hunderths) I would recommend switching to a Linux server instance.
It would scale much better, thanks to the epoll() API use, and also a better TCP layer.

I am not convinced that calling ThreadPollingWakeup() here is the solution.
This is done in TAsyncConnectionsThread.Execute for the atpReadPoll main thread.

TAsyncClient should do that IIRC.

Coldzer0 · 2021-10-26 12:05:35

Thanks for reply.

Yes I'll use Linux But I was testing windows for other project I've in mind.

And About the TAsyncClient there's no release for the "atpReadPoll" that's why I called ThreadPollingWakeup.

I think TAsyncClient needs to be updated.

One thing I notice is when I use TAsyncConnections with 1 thread count the reading is working fine once the ThreadPoolCount is bigger than 1 there's no OnRead call.

Coldzer0 · 2021-10-27 16:13:02

Hello ab,

I Notice today that there's an update on the TAsyncServer But not the TAsyncClient

https://github.com/synopse/mORMot2/comm … 2538a2f0c4

Is there any update coming for the TAsyncClient OnRead Problem ?

Thanks.

ab · 2021-10-29 12:19:41

Please try https://github.com/synopse/mORMot2/comm … abee81c464

Coldzer0 · 2021-10-29 18:33:41

Hello ab,

Please check my last comment https://github.com/synopse/mORMot2/issu … -954962300

I add a git patch propose some changes to let the TAsyncClient work as expected.

mORMot Open Source

#1 2021-10-25 12:14:47

Help - mORMot2 Raw TCP Async Client & Server

#2 2021-10-25 17:22:09

Re: Help - mORMot2 Raw TCP Async Client & Server

#3 2021-10-25 19:40:32

Re: Help - mORMot2 Raw TCP Async Client & Server

#4 2021-10-26 10:37:50

Re: Help - mORMot2 Raw TCP Async Client & Server

#5 2021-10-26 11:54:06

Re: Help - mORMot2 Raw TCP Async Client & Server

#6 2021-10-26 12:05:35

Re: Help - mORMot2 Raw TCP Async Client & Server

#7 2021-10-27 16:13:02

Re: Help - mORMot2 Raw TCP Async Client & Server

#8 2021-10-29 12:19:41

Re: Help - mORMot2 Raw TCP Async Client & Server

#9 2021-10-29 18:33:41

Re: Help - mORMot2 Raw TCP Async Client & Server

Board footer