mORMot Open Source

I would recommend making sure that the one you are using is ecc256r1 and not ec25519, Because mORMot2 only support the ecc256r1.

In my projects, I'm using OnBeforeBody. It's a good point to check for API tokens or header values.

For example, I've had this in one of my projects.

function TMainServer.BeforeBody(var aUrl, aMethod, aInHeaders, aInContentType, aRemoteIP, aBearerToken: RawUtf8; aContentLength: Int64; aFlags: THttpServerRequestFlags): cardinal;
var
  AuthData : String;
Begin
  // Always return HTTP_UNAUTHORIZED if there's any problem ^_^
  Result := HTTP_UNAUTHORIZED;

  // If user asks for time/ping we allow it
  if (LowerCase(aUrl) = '/api/v1/time') or (LowerCase(aUrl) = '/api/v1/ping') Then
  begin
    Exit(HTTP_SUCCESS);
  end;

  if aBearerToken <> '' then
  begin
    AuthData := Base64ToBin(aBearerToken);
    // Check Telegram webapp initData
    if IsValidTelegramUser(AuthData) then
    begin
      if IsAllowedUser(GetUserIDFromToken(AuthData)) then
        Result := HTTP_SUCCESS;
    end;
  end;
End;

Hello,

I recently started using IDocDict and IDocList, and everything works well.
But there's a small thing I can't figure out.

Is there a way to read a value from a JSON path, for example, this data

{"@type":"updateNewMessage","message":{"@type":"message","id":123456789,"sender_id":{"@type":"messageSenderUser","user_id":123456789},"chat_id":123456789,"is_outgoing":false,"is_pinned":false,"can_be_edited":false,"can_be_forwarded":true,"can_be_replied_in_another_chat":true,"can_be_saved":true,"can_be_deleted_only_for_self":true,"can_be_deleted_for_all_users":true,"can_get_added_reactions":false,"can_get_statistics":false,"can_get_message_thread":false,"can_get_read_date":false,"can_get_viewers":false,"can_get_media_timestamp_links":false,"can_report_reactions":false,"has_timestamped_media":true,"is_channel_post":false,"is_topic_message":false,"contains_unread_mention":false,"date":1711194238,"edit_date":0,"unread_reactions":[],"message_thread_id":0,"self_destruct_in":0.000000,"auto_delete_in":0.000000,"via_bot_user_id":0,"author_signature":"","media_album_id":"0","restriction_reason":"","content":{"@type":"messageText","text":{"@type":"formattedText","text":"/start","entities":[{"@type":"textEntity","offset":0,"length":6,"type":{"@type":"textEntityTypeBotCommand"}}]}}}}

Can I do a reading using the full path

function ReadMsg(JSONStr : String) : String;
var
    JSON: IDocDict;
begin
    result := '';
    if IsValidJson(JSONStr) then
    begin
      JSON := DocDict(JSONStr); 
      result := JSON.S['message.content.text.text']; // Can I do that using mormot?
    end;    
end;

And for the Exists method

if JSON.Exists('message.content.text') then

Thanks.

I found a bug in the static unit and sent a fix in the same pull request.

For me, everything still works very fine ( the server has been running for 5 days 16 hours - from the last update I did to the server code not from the last WebSocket fix )
So everything works very well.

What kind of Async server you are using? WebSocket ? or just HTTPAsync Server?

Because the problem with the WebSocket one has nothing to do with the HTTPAsync Server.

I have a question regarding exceptions at functions like

function TWebProcessInFrame.GetHeader: boolean;

Do we need exceptions here?

Like

EWebSockets.CreateUtf8('%.GetFrame: truncated mask', [process]);

Can't we return false on errors because exceptions like these hang the Debugger while testing?

So after running for a while, the high CPU thing happens, and here's some call Stackframe for a couple of threads, and all of the high CPU ones stuck either at

Procedure TRWLock.WriteLock;
or
procedure TRWLock.ReadOnlyLockSpin;

and all of them come from
TAsyncConnections.ConnectionFind
TAsyncConnections.ConnectionFindAndLock

Here's screenshots

and

I'll try to check with fewer threads to see if I can catch what is happening.

The last test I'm doing right now has the hsoBan40xIP and hsoLogVerbose options in the server

And I'm doing a LOG_VERBOSE for the TSynLog.Family.Level

Right now the server has been running for 13+ hours and the server status seems stable

If the server reacts the same as before I'll send you the full server logs in a private message.

Right Now I added the ConsoleWrite you mentioned

The server has been running for "12 hours +" and waiting to see what will happen

BTW I'm using "TWebSocketAsyncServer" and all the connections sending data all the time like ping and other small text chat data

For now, this is the network status.

And for the performance for now it uses only 88MB of RAM and less than 1% of CPU which is amazing.

I'll try to enable full logging after this test and share it with you.

This is what happens after 14 hours, plus the console is not updating anymore.

I got the same error on logs after keeping the server running for a couple of hours.

I'll try to enable trace logging and see if I can get it to have the same issue again.

Hello ab,

Please check my last comment https://github.com/synopse/mORMot2/issu … -954962300

I add a git patch propose some changes to let the TAsyncClient work as expected.

Thanks for reply.

Yes I'll use Linux But I was testing windows for other project I've in mind.

And About the TAsyncClient there's no release for the "atpReadPoll" that's why I called ThreadPollingWakeup.

I think TAsyncClient needs to be updated.

One thing I notice is when I use TAsyncConnections with 1 thread count the reading is working fine once the ThreadPoolCount is bigger than 1 there's no OnRead call.

Thanks ab for reply.

First of all thanks so much for the amazing work in mORMot2.

Sure

The Sever RDP
The Machine I uses for Server is "CPU= 5 x Intel(R)Xeon(R)CPUE5-24200@1.90GHz(x64) - RAM= 8GB"

This is a x64 Debug Build with Delphi Memory Manager.

Built with Delphi 11 Alexandria : Debug Mode
Memory Usage: About "50MB" for "11,166" concurrent Connections With 16 Async Threads (TAsyncServer).
CPU Usage   : Between 17% & 18% never higher.

Even when it reach the 13,000+ It was still very small memory usage about "52-54 MB"

Here's a screenshot from when I was testing.

The Client Side: (I didn't use mORMot2 I used a tool called EchoServerTest to do the testing)
The Stress testing Machine is "CPU=16 x Intel(R)Core(TM)i9-9900KCPU@3.60GHz(x64) RAM= 32GB"

I was aiming for 40000 concurrent connections But windows has limits, I think there's some registry options I can edit to extend the numbers of connections.

Anyway I'll do the test again but with Linux x64 (Will test Release Build Later with mORMot2 Memory Manager).

Yes no problem I will try my best.

In normal blocking mode you got the sent length so you can make sure how much of your buffer sent to the server.

But I'll implement custom acknowledgment send by the server to be confirmed by the client as you advised.

I've a question about the internal buffer of the sockets, How can I read a specific size of data keeping in mind that the read and write in Async.
What I mean is how can I make sure that what I receive is the full packet if the size if like 1MB or Bigger keeping in mind that the maximum buffer described in code is "8KB".

this's the smallest size i get so far 6.8MB

with this config

../configure --enable-release --disable-ion --enable-ctypes --disable-jemalloc --enable-nspr-build --disable-debug --disable-debug-symbols --enable-optimize=-O3 --without-intl-api --enable-strip --disable-ion --without-system-icu