Seriallize Error mORMot2

Alek · 2022-05-13 08:20:02

Good morning.

I wanted to do a trick, to persist data of record(structure) throught a variant published field. I use pre unicode version Delphi.

Well. I run my code with mormot1(switch the VER2 directive) and it work very well(all fields are serialized), but with mormot2, I get an error.

Here is my code ready for compile and json file "SaveAll.json"

https://gist.github.com/delphiapp/5e1e4 … 796d9b9a8a

Last edited by Alek (2022-05-13 08:50:24)

ab · 2022-05-13 08:37:44

Please follow the forum rules and don't publish such code directly in the forum thread. We have limited storage abilities.

Use e.g. a gist instead.

Alek · 2022-05-13 08:52:00

ok, sorry

ab · 2022-05-13 08:59:52

You need to add jpoHandleCustomVariants for such input containing custom variants.
If you use [jpoIgnoreUnknownProperty, jpoHandleCustomVariants] then it is read as expected.

A good idea in such case may be to just use the JSONPARSER_TOLERANTOPTIONS constant (JSONTOOBJECT_TOLERANTOPTIONS in mORMot 1).

So in fact, mORMOt 1 was incorrect/too lazy.

Alek · 2022-05-24 12:39:43

Good afternoon. Please, tell me about how I can save a variant as a json object, but not as a json Array? By default mORMot1 save the variant as an object, and mORMot2 as an array.

https://gist.github.com/delphiapp/d9e89 … f3ea1080f0

Last edited by Alek (2022-05-24 12:45:04)

igors233 · 2022-05-24 16:46:15

Interesting, GetVariantRecIEC is called and it returns Variant as DVD object (uses TDocVariantData internally) which is then converted to array with this line from core.rtti
RVD.NeedsClear := true; // we allocated a RVD for the getter result

That clears dvObject from internal DVD and it becomes dvArray;

I don't know if that's intended behaviour or bug, it's something for ab to look at.

For your issue, you can declare published property differently, for example:
property RecIEC: TRecIEC read fRecIEC write fRecIEC;

or

property JRecIEC: RawJson read GetRecJSON write SetRecJson;

GetRecJSON/SetRecJson will serialize fRecIEC as text so there won't be a problem.

Last edited by igors233 (2022-05-24 16:48:45)

Alek · 2022-08-26 08:22:59

Hello @ab. I got an error when reloading serialized data in Mormot2. When reloading data, an error appears when clearing T*ObjArray. File mormot.core.base, line 6996.

In version 1(switch the define) it's all right. You can reproduce the error through the source code.

https://gist.github.com/delphiapp/6d017 … 7c4402e0fe

Last edited by Alek (2022-08-26 08:39:00)

tbo · 2022-08-26 14:16:47

Alek wrote:

You can reproduce the error through the source code.
https://gist.github.com/delphiapp/6d017 … 7c4402e0fe

Your example is not comprehensible, because you forgot to specify the destructors, the one for the object TModelListUstr. It is not possible to see who is deleting the data from the list here. For an array of objects, the Clear function must be called. The help says:

Note that TDynArray is just a wrapper around an existing dynamic array: methods can modify the content of the associated variable but the TDynArray doesn't contain any data by itself.

It is also better to use a count variable:

fModelListDA.Init(TypeInfo(TModelUstrObjArray), fModelList, @fModelListCount);

With best regards
Thomas

ab · 2022-08-26 14:26:59

I guess that TModelListUstr should inherit from TSynAutoCreateFields otherwise it would leak memory.

Alek · 2022-08-26 18:52:24

Thanks a lot for your advices, but none of them helped. The error still exists.

tbo · 2022-08-26 21:44:38

Alek wrote:

The error still exists.

Please update your example with all the changes you have made in the meantime. With your information no help is possible.

With best regards
Thomas

Alek · 2022-08-29 07:19:45

Good morning. I have updated the example. See: https://gist.github.com/delphiapp/b2558 … 2cab5a6cc1

Since the class inherits from TSynAutoCreateFields, as I understand it, there is no need to prescribe a constructor and a destructor.

Thank you very much, Thomas, for wanting to help

tbo · 2022-08-29 20:01:41

Alek wrote:

I have updated the example. See: https://gist.github.com/delphiapp/b2558 … 2cab5a6cc1

You can't load twice in a row. The function JsonToObject reads the object property values from JSON and if it is necessary, it also instantiates. You overwrite the previous one and these objects are never destroyed. So you leak memory. If you need a function Clear, you have to write it and call it between the two load calls.

  ModelApp.LoadConfiguration(ExtractFilePath(Application.ExeName) + 'SaveAll.json');
  ...
  ModelApp.Clear;
  ModelApp.LoadConfiguration(ExtractFilePath(Application.ExeName) + 'SaveAll.json');

You can write the LoadConfiguration function like this:

function TModelApp.LoadConfiguration(AFileName: string): Boolean;
begin
  result := JsonFileToObject(...);

With best regards
Thomas

Last edited by tbo (2022-08-29 20:03:42)

Alek · 2022-08-30 05:53:47

You're right. In mormot 1, any previous instances of objects were forcibly released when json was loaded, but in mormot 2, it is now necessary to set the [jpoClearValues] parameter in the JsonFileToObject function. Thanks a lot.

Last edited by Alek (2022-08-30 05:56:49)

Alek · 2022-08-30 07:55:23

Please bear my foolishness. Why then is the T*ObjArray type property with the [jpoClearValues] parameter not released

See: https://gist.github.com/delphiapp/0e05d … e0f2697c17

Alek · 2022-08-31 06:17:57

Good morning.

The probable cause of the error is an incorrect value of the first parameter of the RawObjectsClear function in the _ObjArrayClear procedure (line 5743, mormot.core.rtti)

there is -

RawObjectsClear(pointer(V), PDALen(PAnsiChar(V^) - _DALEN)^ + _DAOFF);

must be-

RawObjectsClear(V^, PDALen(PAnsiChar(V^) - _DALEN)^ + _DAOFF);

Last edited by Alek (2022-08-31 06:20:08)

Alek · 2022-08-31 07:23:22

One interesting feature turned out. When using the [jpoClearValues] parameter, the FinalizeAndClearPublishedProperties function is called and if the property is of type T*Objarray, the _ObjArrayClear function is called. This function not only frees the memory of array elements, but also deletes the array itself via _DynArrayClear.

But when we use the AutoDestroyFields function, if the property is of type T*ObjArray, then only the RawObjectsClear function is called. The array itself is not deleted

Last edited by Alek (2022-08-31 07:32:30)

mORMot Open Source

#1 2022-05-13 08:20:02

Seriallize Error mORMot2

#2 2022-05-13 08:37:44

Re: Seriallize Error mORMot2

#3 2022-05-13 08:52:00

Re: Seriallize Error mORMot2

#4 2022-05-13 08:59:52

Re: Seriallize Error mORMot2

#5 2022-05-24 12:39:43

Re: Seriallize Error mORMot2

#6 2022-05-24 16:46:15

Re: Seriallize Error mORMot2

#7 2022-08-26 08:22:59

Re: Seriallize Error mORMot2

#8 2022-08-26 14:16:47

Re: Seriallize Error mORMot2

#9 2022-08-26 14:26:59

Re: Seriallize Error mORMot2

#10 2022-08-26 18:52:24

Re: Seriallize Error mORMot2

#11 2022-08-26 21:44:38

Re: Seriallize Error mORMot2

#12 2022-08-29 07:19:45

Re: Seriallize Error mORMot2

#13 2022-08-29 20:01:41

Re: Seriallize Error mORMot2

#14 2022-08-30 05:53:47

Re: Seriallize Error mORMot2

#15 2022-08-30 07:55:23

Re: Seriallize Error mORMot2

#16 2022-08-31 06:17:57

Re: Seriallize Error mORMot2

#17 2022-08-31 07:23:22

Re: Seriallize Error mORMot2

Board footer