mORMot Open Source

Thomas, since I discovered the outstanding mORMot framework almost three years ago, I realized I should start using it deeply in my projects, but it requires some education to begin playing with it. For the project in question, and as I am still learning mORMot, I created a testing environment application where I also learn mORMot fundamentals, and then next I use what is mature in the real projects. I was never happy with the fact that when serializing/unserializing JSON I was not using the right counterpart function, as I had tested the right pair or functions (like 'DynArrayLoadJson / DynArraySaveJson', 'RecordLoadJson / RecordSaveJson') and it never worked as I was expecting, probably due something I was doing wrong. I never felt well when I was mixing those previous functions to read/write JSON. The main JSON I have to deal with I defined like this:

TMyArray = array of packed record
                           ...
                           end;

TMyEntity = packed record
                     MyArray : TMyArray;
                   end; 

The "combination" that worked, in this case, was something like this:

When reading:

var TempRec : TMyEntity;

RecordLoadJson(TempRec, my_json TypeInfo(TMyEntity));

When writing:

var tmp : RawJson;

tmp := DynArraySaveJson(MyEntity.MyArray, TypeInfo(TMyArray), false);

I never make that work if I had used the pair "DynArrayLoadJson / DynArraySaveJson". This has been working since March/2022. No "access violation" until some weeks ago, then AV for the writing part, and after the last commit it came back to normal (ALL that without any change in my code), but now I got an AV in a unit that I don't use at all.

You said that you don't know how to load an array of records with 'RecordLoadJson'. The way I did above allow me to do it, although that would sound weird because 'RecordLoadJson' is used to load a 'TMyEntity' (that is a record of an array of record), I guess the whole bunch of JSON records are treated as a big JSON, but I am not sure. I would like to simply use 'DynArrayLoadJson / DynArraySaveJson' for that, but I never knew how to do it.

I hope everybody has a Happy New Year!

Let's do a simple chronology of what has happened so far, using Delphi 11.2:

1) My test environment has been working well since March/2022, in this case, using only three mORMot objects/functions: 'TSimpleHttpClient', 'RecordLoadJson', and 'DynArraySaveJson';

2) After the commit right before my first message (or the previous one) on this thread, without changing anything in my code, I began to get an "access violation" error according to what was described above;

3) After the latest commit I got today, the "access violation" is gone. I thought something at 'mormot.core.json' could be the cause of the access violation, but I checked all the latest changes on that unit and found nothing relevant that could lead to that error;

4) Now when I close my test application I get another "access violation" error, now at 'mormot.core.base', precisely on this procedure, line 7086:

procedure ObjArrayClear(var aObjArray);
var
  a: TObjectDynArray absolute aObjArray;
begin
  if a = nil then
    exit;
  // release all owned TObject instances
  RawObjectsClear(pointer(aObjArray), PDALen(PAnsiChar(a) - _DALEN)^ + _DAOFF);      <--- The "access violation" is here
  // release the dynamic array itself
  a := nil;
end;

5) I debugged all my 'finalization' sections (I don't have any 'OnClose/OnDestroy' events in my test application) and all then are clear, no error at all;

6) I personally don't use any function/procedure of 'mormot.core.base'. It is listed in my uses clause just because I use 'RawUtf8' variables and that type is defined there;

7) It looks like some side effect is taking place in my case, but I don't think it is easily reproducible in a 'test case'. As I said, I only use those three 'object/functions' from mORMot in this test application. I really don't know why that function is been called at the end of the application, but it has anything to do with my code.

8) For sure there is a hidden "bug" somewhere in mORMot, and if I have a clue where it is I'll post it here.

It is like

MyTypeArray = array of packed record
                                               ...
                                              end;

I read a huge JSON that I receive from a REST service and load it using 'RecordLoadJson' without trouble. At some point in my test environment, I need to save the final JSON and it is when I use 'tmp := DynArraySaveJson(MyData, TypeInfo(MyTypeArray), false);'. It was working great until some days ago. Now it raises that "access violation error".

Fixed!

Thanks!

One more great contribution!

Thanks, Thomas!

Email sent

Awesome!

While Ab fixes that properly, you can do this to fix that compile error:

---
FormatString('%.%', [efn, ALGO_EXT[a]], String(fn));
---

That's it!! A 'RawUtf8' did the thing!!

I knew that the record as it is defined here is well dealt with by 'mORMot' because it worked before for other situations like that, but I didn't realize about 'RawUtf8'!

Thank you, Ab!

   Thanks, Thomas, ALL worked like a charm!

I am still learning the HUGE JSON framework of mORMot, but there are situations where I still don't know how to figure them out, like the one below:

1) I need to read JSON values that are like the one in the file 'JSON_Test.TXT' (link below). The values there are not properly an array (looking at the JSON file there is no '[',']' pair), but look like one, and neither is a sole record. Is there a mORMot function that can read it and fill an 'array of packed record' able to be read by a call like 'DynArrayLoadJson' for that situation?

2) Another question: Is there a way to read the values that precede the initial record ('Record01') of the set of records for each entity? For the specific file the values 'ABC' and 'DEF'?

https://github.com/PrometeusRec/JSON_Test

Thanks for any help!

So that test had really passed!

Input headers were empty

Based on the test done by 'aProtocol.SetSubprotocol(prot)', I realized that the problem was when creating an instance of 'TWebSocketProtocolChat', where I was giving a name when calling 'Create'. It was not clear to me, based on the comments in the source code, what 'aName' meant exactly, and I thought it was a name the programmer was giving to that specific use. When I left it empty it passed the test done at line 451 (above) and worked as expected.

--> What is the proper method to send a text back to the server using the bidirectional open connection? Thanks!

I am still doing some tests with mORMot WebSockets but for some reason, I am getting 'Invalid HTTP Upgrade Header' after upgrading to WebSocket. I tried with five different WebSocket servers that I found on Internet, but for only one of them, the result was ok, although they are all supposed to respond the same way, with JSON or plain text.

I did a simple test program here https://github.com/PrometeusRec/mormot_ … t_Test.dpr where you can see the five websites above, and uncomment one website at a time and test it. Currently, only the last one is giving an answer different from 'Invalid HTTP Upgrade Header' after calling 'WebSocketsUpgrade'. I tested all those websites on 'https://www.piesocket.com/websocket-tester#' and all worked as expected. For two of those, you can enter the JSON  in the comments if you want to see them respond, as all the others will give an automatic response right after the connection.

What is the proper method to send a text back to the server using the bi-direction open connection? Thanks!

Indeed, I was mixing those pieces of information. Thanks!

  Ok, thanks!

The applications use the FB database, and now it will be accessed by a mobile application as well. I am really impressed with SQLite 3 performance, and I thought about converting the FB database to it, but I am concerned about SQLite issues with simultaneous writing. I don't know SQLite well, but I am aware of its limitation when dealing with concurrency writings, so I will stick with an external database for while.

I already followed Mormot documentation about SSL, and I am trying to test it using Chrome or Firefox. Despite Chrome not liking my "Dev Certification Authority" and showing it as "Not Secure" (For Firefox it is ok), I am doing simple tests using a browser, and after that build the client-side. After the SLL all set, I am trying accessing the REST API with SSL enabled (the source above was updated):

'https://localhost:8043/root/test/1'

and I am receiving:

{
"errorCode":403,
"errorText":"Authentication Failed: Invalid signature (0)"
}

Googling a little, I found that this message is related to a problem in authenticating. However, I am setting 'aHandleUserAuthentication' to 'false', at first. 'AUTHGROUP' and 'AUTHUSER' tables were created by Mormot when I was testing authenticating without SSL (I guess that they are only used if 'aHandleUserAuthentication' is set to 'true', right?), but I want to have the REST API accessing working with SSL without authentication first, then with it.

After fixing this issue (accessing the REST API through a browser or AJAX client) can the right above input using authentication like this?

'https://localhost:8043/root/test/1?user=User&password=synopse'

Thank you for the suggestion. I already got my application test working as expected and using the 'standard' way with TSQLModel. I have included in my tests Mormot secure authentication and it worked too, and now I am trying to have it working with SSL. Is there a real advantage for using interface services, like more speed in the JSON response, for instance? I still didn't study this part of Mormot documentation, but anyway I am already glad about the results that I am getting. Mormot framework is the most impressive piece of code for Delphi in this area that I have seen, and I wonder why I didn't hear about it before.