TBCD Serialization

CycleSoft · 2024-12-07 10:33:01

Hi!
I'm migrating a mORMOt1 project to mORMot2.

The server fails with an AV when trying to serialize a TBCD property of a TCollectionItem descendent .
With mORMot1 that propertry is serialized like this:
"MyProp":
{
"Precision":1,
"SignSpecialPlaces":0,
"Fraction":"5000000000000000000000000000000000000000000000000000000000000000"
}

I have to 'enable' or configure something or is this just a mORMOt2 bug?

Thanks in advance!

ab · 2024-12-07 13:01:52

There was no specific support of TBCD in mORMot 1.
Sounds like if it was serialized using the extended RTTI of the published record, available since Delphi XE5.
It should work the same with mORMot 2.

Which compiler and which compiler version are you using?
Which serialization method did you use?
Could you write a small program to reproduce the issue? - and put it on a gist or a link, not directly in the forum.

CycleSoft · 2024-12-07 13:53:02

Thank you for the quick reply!

I'm using Delphi 12 ( Version 29.0.51511.6924 )
I simply had defined a TMyItem class descendant of TCollectionItem with a published TBCD property, defined a TMyCollection class descendant of TInterfacedCollection that 'use' items of the TMyItem class, and declared a method with a 'out' parameter of type 'TMyCollection'.
I let mORMot handle all the serialization process, without any effort by my side (and mORMot rocks in this!)

If these details are not enough to spread a light in your head, let me know, and I will prepare an example as you suggest!

ab · 2024-12-07 14:27:06

Was the unserialization really working previously with mORmot 1?
I have a doubt about the Fraction part - it is a fixed array, and should not be serialized as such, but as base-64.

Where is the access violation happening in mORMot 2?

CycleSoft · 2024-12-07 15:19:36

> Was the unserialization really working previously with mORmot 1?
yes, absolutely. The example I posted in the first post is taken directly from the browser's network console (I have a Web client) and just added a bit of formatting/indentation to make it more readable

> Where is the access violation happening in mORMot 2?
I have pulled the latest version as for this morning.
The AV accurs when processing the TBCD property of the first TMyItem instance, in this line ( line nr 5893 file mormot.core.json )

_JS_OneProp(c, p, Data);

if I step-in into this procedure the IDE freezes. If I let the server run w/o breakpoints, an AV is raised. As far as I can tell, all the 3 parameters of the procedure are assigned when entering.

Last edited by CycleSoft (2024-12-07 15:20:12)

ab · 2024-12-07 20:24:50

Remove the "inline" of _JS_OneProp() to debug in it.
I suspect c.Info.JsonSave is nil.

Anyway, in the next days, I will make true and proper serialization of TBcd in mormot.db.rad.pas.

ab · 2024-12-09 08:06:41

Please try
https://github.com/synopse/mORMot2/commit/bf4e89f6

Now TBcd should be serialized as a proper number, between " " for consistency.

I also identified several issues with TBcd process (affecting moRMot 1 too), and added some regression tests.

CycleSoft · 2024-12-09 09:37:21

Hi,
made just a quick try, and the AV is disappeared, but the value of the property is serialized as 'null'

I will prepare a sample project as you suggested, so you can easily verify and hopefully solve my problem.

Thanks!

ab · 2024-12-09 10:10:04

The basic RTTI returned by Delphi is not enough: it returns no type info for the last member of TBcd.
mORMot 1 was making a wild guess about what was missing, but it was unsafe, and working by chance.

You need to add the mormot.db.rad.pas unit to your project somewhere, to fix the serialization.

CycleSoft · 2024-12-09 20:40:29

It works!
once I added the mormot.db.rad unit, the property now is serialized as string.
I'm impressed by your effectiveness!

Now I noticed another, maybe related, problem: The BCD values stored in a SQLite3 DB by the old server, are not read back correctly. I guess the same serialization/deserialization scheme is used also for the DB side, so I only need to 'convert' the old values.
I'm too tired now to further dig into... I will update you as soon as I can.
Many thanks and have a good night!

ab · 2024-12-10 08:09:33

In TJSONCustomParserRTTI.CreateFromRTTI, mORMot 1 detect that the "Fraction" static array has no RTTI, then serializes it as hexadecimal, guessing its size.
This is what mORMot 2 does not. When a field has no RTTI, it does not do anything yet.

I will try to fix it.

ab · 2024-12-10 10:01:34

It should now recognize the mORMot serialization format of TBcd, and read it.
https://github.com/synopse/mORMot2/commit/482adab0

Still need to fix the global RTTI discrepancy.

Edit: this discrepancy should be now fixed globally with https://github.com/synopse/mORMot2/commit/e5f55fcd

CycleSoft · 2024-12-12 11:58:18

Hello,
sorry for the late reply, I was busy on another project :-(

I can confirm that:
- before your last commit, also on the ORM side the TBCD value was stored on the DB as a simple srtring, ie as "12.345"
- Just updated to the latest and greatest mORMot2, and my 'old' format stored values (ie a string representing a JSON object with 3 properties, like {"Precision":1, "SignSpecialPlaces":0, "Fraction":"...."} ) are correctly read from the ORM, and serialized to the client with the 'new' format (simple string ie "12.345")

So if I understand correctly, this will be the 'out-of-the-box' behavior of mORMOT2, 'officially' supported?

And just for my curiosity, it means that mORMot2 is capable of reading TBCD values on the ORM side in both formats ?

By the way, with this last update my issue is solved, thanks again for your invaluable support!

ab · 2024-12-12 12:08:10

Yes to all your questions.

mORMot Open Source

#1 2024-12-07 10:33:01

TBCD Serialization

#2 2024-12-07 13:01:52

Re: TBCD Serialization

#3 2024-12-07 13:53:02

Re: TBCD Serialization

#4 2024-12-07 14:27:06

Re: TBCD Serialization

#5 2024-12-07 15:19:36

Re: TBCD Serialization

#6 2024-12-07 20:24:50

Re: TBCD Serialization

#7 2024-12-09 08:06:41

Re: TBCD Serialization

#8 2024-12-09 09:37:21

Re: TBCD Serialization

#9 2024-12-09 10:10:04

Re: TBCD Serialization

#10 2024-12-09 20:40:29

Re: TBCD Serialization

#11 2024-12-10 08:09:33

Re: TBCD Serialization

#12 2024-12-10 10:01:34

Re: TBCD Serialization

#13 2024-12-12 11:58:18

Re: TBCD Serialization

#14 2024-12-12 12:08:10

Re: TBCD Serialization

Board footer