Today i decided to upgrade to last build, now my code is not working

koraycayiroglu · 2020-06-11 14:21:03

Hi everyone,

Once again thanks for a great framework. I am using mormot over a year now and it was working fine. Today i decided to upgrade it to nightlybuild. After doing some coding for changes within MVCApplicaton , i tried my apps and none of my services start.

First this code wait forever:

  StaticMongoDBRegisterAll(DB, MongoCli.Open(CollectionName));

I don't know why but i did some debug and found out I have a TSQLRecord within my TSQLModel which i never used. Found problem here:

procedure TSQLRestServer.InitializeTables(Options: TSQLInitializeTableOptions);
var t: integer;
begin
  if (Self<>nil) and (Model<>nil) then
    for t := 0 to Model.TablesMax do
      if not TableHasRows(Model.Tables[t]) then
        Model.Tables[t].InitializeTable(self,'',Options);   // <<--------- HERE IT WAITS FOREVER
end;

I decided to remove this TSQLRecord from my TSQLModel, now my services are online. But most of my TMVCApplication's is not working probably due to some strange behavior with i don't have any knowledge to fix.

For example i have been retrieving records from rest like:

   Rec:= TSQLRecord.Create(RestModel, 'Field1 = ? And Field2 = ? And Field3 = ?', [val1, val2, va3]);

This was working fine before and it has to be, it's very simple record retrieving but after i upgrade, this code waits forever as it did with StaticMongoDBRegisterAll. No errors, nothing. It hanging at:

procedure TMongoConnection.GetCursor(Request: TMongoRequest; var Result: TMongoReplyCursor);
var reply: TMongoReply;
begin
  GetReply(Request,reply);  // <<--------- HERE IT WAITS FOREVER
  Result.Init(reply);
  if (Client.LogReplyEvent<>sllNone) and (Client.Log<>nil) and
     (Client.LogReplyEvent in Client.Log.Family.Level) then
    Client.Log.Log(Client.LogReplyEvent,
      Result.ToJSON(modMongoShell,True,Client.LogReplyEventMaxSize),Request);
  if mrfQueryFailure in Result.ResponseFlags then
    raise EMongoRequestException.Create('Query failure',self,Request,Result);
end;


procedure TMongoConnection.GetReply(Request: TMongoRequest; out result: TMongoReply);
var Header: TMongoReplyHeader;
    HeaderLen, DataLen: integer;
begin
  if self=nil then
    raise EMongoRequestException.Create('Connection=nil',self,Request);
  FillCharFast(Header,sizeof(Header),0);
  HeaderLen := SizeOf(Header);
  try
    Lock;
    if Send(Request) then
      while true do
      if fSocket.TrySockRecv(@Header,HeaderLen) then begin // <<--------- WAITING FOR THIS LINE AT SECOND RUNNING
        if (Header.MessageLength<SizeOf(Header)) or
           (Header.MessageLength>MONGODB_MAXMESSAGESIZE) then
          raise EMongoRequestException.CreateUTF8('%.GetReply: MessageLength=%',
            [self,Header.MessageLength],self,Request);
        SetLength(result,Header.MessageLength);
        PMongoReplyHeader(result)^ := Header;
        DataLen := Header.MessageLength-sizeof(Header);
        if fSocket.TrySockRecv(@PByteArray(result)[sizeof(Header)],DataLen) then
          if Header.ResponseTo=Request.MongoRequestID then // success
            exit else
          if Header.OpCode=ord(opMsg) then begin
            if Client.Log<>nil then
              Client.Log.Log(sllWarning,'Msg from MongoDB: %',
                [BSONToJSON(@PByteArray(result)[sizeof(Header)],betDoc,DataLen,modMongoShell)],Request);
          end else
            raise EMongoRequestException.CreateUTF8(
              '%.GetReply: ResponseTo=% Expected:% in current blocking mode',
              [self,Header.ResponseTo,Request.MongoRequestID],self,Request);
      end else
        try
          Close;
        finally
          raise EMongoRequestException.Create('Server did reset the connection: '+
            'probably due to a bad formatted BSON request -> close socket',self,Request);
        end;
    // if we reached here, this is due to a socket error
    raise EMongoRequestOSException.Create('GetReply',self,Request);
  finally
    UnLock;
  end;
end;

funny enough, i am using same code all over the place and it just didn't work for this TSQLRecord. Anyone have any idea what's happening here?

Best regards

Last edited by koraycayiroglu (2020-06-11 15:30:55)

ab · 2020-06-11 19:26:48

There is indeed a SynMongoDB regression.
Already known in this forum IIRC.
I will investigate tomorrow.

My initial guess is that it is due to a small regression in SynCrtSock.pas in low-level socket connection delay/retry pattern.
If you know the data size (e.g. with HTTP), no problem.
But with MongoDB you don't know the result-set size in advance, so I guess that SynCrtSock is waiting for more data, whereas there is no more to wait for.

Did you observe this on Windows? Or Linux? With Delphi? With FPC?

koraycayiroglu · 2020-06-12 07:29:56

Windows and Delphi 10.3. Quick response as always. thanks.

ab · 2020-06-12 07:38:43

I guess the problem is not a SynCrtSock.pas regression, but a modification in the MongoDB wire protocol.
I guess the previous version of SynCrtSock was working by change, due to a bug we fixed...

There was a new https://docs.mongodb.com/manual/referen … ol/#op-msg introduced.
It has a code 2013 and replaces the former/deprecated OP_MSG opcode which had a code 1000.
I will try to properly implement it.

ab · 2020-06-12 09:16:22

I just fixed the protocol problem by https://synopse.info/fossil/info/0aba463a6d

But I still have issues with the authentication.
I guess something was also changed with MongoDB 4.x about default authentication parameters.
Investigating...

Edit: see also https://synopse.info/fossil/info/fd93a93e5e

koraycayiroglu · 2020-06-12 10:17:26

Both problems are solved

Thanks ab

danielkuettner · 2020-06-12 10:45:09

Wow, please send me your account number to make a donation

Greg0r · 2020-06-12 11:43:18

danielkuettner wrote:

Wow, please send me your account number to make a donation

I believe it's in the top menu of website: https://synopse.info/fossil/wiki?name=HelpDonate

danielkuettner · 2020-06-12 11:47:07

I've checked your hint, but it's PayPal only. I need a real IBAN/SWIFT...

Vitaly · 2020-06-12 12:10:40

PayPal allows binding a bank card

ab · 2020-06-13 10:14:16

@danielkuettner
I thought it was a funny joke.

danielkuettner · 2020-06-13 17:26:47

No, it wasn’t a joke.
We are using MongoDB but atm without auth. But in future Atlas would be an option and it was impressive how fast you are finding the solution and I wanted to donate to honor your work. I don’t know if your account number has changed and PayPal isn’t optimal for me. If you want you can send it per email...

ab · 2020-06-13 20:13:14

mORMot Open Source

#1 2020-06-11 14:21:03

Today i decided to upgrade to last build, now my code is not working

#2 2020-06-11 19:26:48

Re: Today i decided to upgrade to last build, now my code is not working

#3 2020-06-12 07:29:56

Re: Today i decided to upgrade to last build, now my code is not working

#4 2020-06-12 07:38:43

Re: Today i decided to upgrade to last build, now my code is not working

#5 2020-06-12 09:16:22

Re: Today i decided to upgrade to last build, now my code is not working

#6 2020-06-12 10:17:26

Re: Today i decided to upgrade to last build, now my code is not working

#7 2020-06-12 10:45:09

Re: Today i decided to upgrade to last build, now my code is not working

#8 2020-06-12 11:43:18

Re: Today i decided to upgrade to last build, now my code is not working

#9 2020-06-12 11:47:07

Re: Today i decided to upgrade to last build, now my code is not working

#10 2020-06-12 12:10:40

Re: Today i decided to upgrade to last build, now my code is not working

#11 2020-06-13 10:14:16

Re: Today i decided to upgrade to last build, now my code is not working

#12 2020-06-13 17:26:47

Re: Today i decided to upgrade to last build, now my code is not working

#13 2020-06-13 20:13:14

Re: Today i decided to upgrade to last build, now my code is not working

Board footer