JSON de-serialization does not fail but record is not usable either

ertank · 2025-07-23 13:31:36

Hello,

I am using Delphi 12.3 and the latest mORMot2 for JSON operations from 20th July (SHA-1: 072fe3a3412a75b05522c7a5a673615ec7c97dc8 * tests: enhanced TFormatUtf8 coverage).

I have a situation that I cannot understand the reason.
Code example: https://codefile.io/f/wozJ7ViV7J

When I de-serialize a JSON it completes OK. But, I cannot use the record and its data.

Here is a screen from debug window after de-serialization completed
https://ibb.co/hFqM6tb2
There should be an arrow > displayed next to the LProduct record, but there is none.

I got a problem report that some products are not processed and I could find problematic JSON in the code example above. It is normally part of an array.
When the loop arrives to this specific element in array it is not causing any exceptions or anything but is not being processed either. Array after de-serialized is displayed as following in Delphi debug screen
https://ibb.co/fV851Lxb
Item number 16 is the sample JSON in above code snippet.

If I remove "ccValidationResults" from the record definition, there is no problem and de-serialization seems to complete OK. I can use record OK. However "price" value which is after ccValidationResults in JSON is not zero and some random number. I don't know if later is a problem or a feature.

Any help is appreciated.

Thanks & Regards,
Ertan

ab · 2025-07-23 14:38:34

1) Why is there a trailing , at the end of your JSON?

2) Try adding a "price": item after the variant object in the JSON.

I guess since your JSON is incorrect, there is an error, and price is left with its default random value from the stack.
It is as expected: you need to fill the record with zeros (e.g. LProduct := default;) before filling it.

ertank · 2025-07-23 16:33:46

Hello,

It is my mistake that I missed the trailing comma when I copy-paste it from the JSON array. Sorry about that.
If you check the second picture, 16th item in the OP, it does not have ">" mark in debug window. The problem I am facing is some where in the middle of a JSON array not at the end or the beginning. Array item before and after are fine not the 16th item though.
Taking it as a stand alone JSON and I still can not see ">" mark in debug window.

Here is the updated project with trailing comma removed.
https://codefile.io/f/ZaODJjXb1M

Here is LProducts debug window view using the updated sample code.
https://ibb.co/Kx19YBdV

If I remove ccValidationResults from the record definition my debug window displays like below and I can expand it reading all record values separately
https://ibb.co/FqsN1530

I think there is a problem some where that I do not know the reason why. I can reproduce the same behavior on 32bit and 64bit. My problem emerged with the 64bit originally.

It feels like mORMot2 is not processing the JSON to the end and stopping at some point. I think this is might also explain random value on LProduct.price.

ab · 2025-07-23 17:27:52

I don't know what to say about the > mark in the DElphi IDE.
Sounds like a Delphi IDE bug.

Random values in LProduct.price would appear even if there is no field. You need to explicitly set it to 0 before unserialization.

I tried to reproduce the problem with some specific data structures:
https://github.com/synopse/mORMot2/commit/3e7beb9f7
But no problem with this kind of content.

Can you debug a little more and see where it stops during deserialization?

tbo · 2025-07-23 20:38:41

ertank wrote:

Here is the updated project with trailing comma removed.
https://codefile.io/f/ZaODJjXb1M

I have no problem with the example. It worked in all scenarios.
Delphi 12.3 32/64Bit, Debug/Release, mORMot2 2.3.11026/2.3.11647.

With best regards
Thomas

ertank · 2025-07-23 23:19:25

ab wrote:

Can you debug a little more and see where it stops during deserialization?

I couldn't easily trace this in mormot codes.
I tried to use Lazarus (FPC v3.2.3, Lazarus v4.1 64bit). Project was targeting 64bit, too. This time I get "failed" message displayed.

Lazarus codes as follows

uses
  mormot.core.json,
  mormot.core.base;

const
  cJson = '{"ccValidationResults":{"Zorunlu ürün özellikleri":{"attribute.Paket_Icerigi":["Zorunlu özellik değeri Paket İçeriği boş olamaz"],"attribute.000002C":["Zorunlu özellik değeri Kumaş Tipi boş olamaz"]}}}';

type
  THepsiburadaProduct = packed record
    ccValidationResults: Variant;
  end;

procedure TForm1.FormCreate(Sender: TObject);
var
  LProduct: THepsiburadaProduct;
begin
  FillChar(LProduct, SizeOf(LProduct), 0);
  if not RecordLoadJson(LProduct, RawUtf8(cJson), TypeInfo(THepsiburadaProduct)) then
    ShowMessage('failed');
  // Lazarus display failed message on Windows.
end;

I see function TRttiJson.ValueLoadJson returns nil and this is an error status. Code execution goes to "try if binary serialization was used" part in this function
I couldn't identify exact part of the JSON which stops the parsing.

I also tried several online JSON validator tools and simplified JSON passed as valid in all of them.

ab · 2025-07-24 06:39:51

In FPC 3.2.3 there is no extended RTTI available for the record fields, so you need to call TRttiJson.RegisterFromText() with the definition (or use FPC trunk).

In Delphi, put a breakpoint in _JL_Variant() from mormot.core.json and see what happens when you reach the variant.
Ensure the encoding is really UTF-8 with it.

ab · 2025-07-24 07:14:21

Edit:
I have tried your code, and it works well on my side.
https://gist.github.com/synopse/b8ad0ca … 5e16bd9511

The only difference is that I use an UTF-8 file instead of a constant + RawUtF8() which seems a problem in Delphi.
It sounded like an encoding problem, due to your test. The deserialization was fine, but the content in memory was not valid UTF-8 so it did not display as expected.

With the proper encoding at input, it works fine in the IDE:

ertank · 2025-07-24 07:38:45

ab wrote:

I don't know what to say about the > mark in the DElphi IDE.
Sounds like a Delphi IDE bug.

This seems like it. If I add the variable in Watch List it is just fine.
Sorry about all this trouble.

Thank you.

mORMot Open Source

#1 2025-07-23 13:31:36

JSON de-serialization does not fail but record is not usable either

#2 2025-07-23 14:38:34

Re: JSON de-serialization does not fail but record is not usable either

#3 2025-07-23 16:33:46

Re: JSON de-serialization does not fail but record is not usable either

#4 2025-07-23 17:27:52

Re: JSON de-serialization does not fail but record is not usable either

#5 2025-07-23 20:38:41

Re: JSON de-serialization does not fail but record is not usable either

#6 2025-07-23 23:19:25

Re: JSON de-serialization does not fail but record is not usable either

#7 2025-07-24 06:39:51

Re: JSON de-serialization does not fail but record is not usable either

#8 2025-07-24 07:14:21

Re: JSON de-serialization does not fail but record is not usable either

#9 2025-07-24 07:38:45

Re: JSON de-serialization does not fail but record is not usable either

Board footer