DynArrayLoadJson can not process the JSON with "a", "A" names data?

wqmeng · 2023-10-21 09:47:22

Hello ab, nice again.

I read a Json from a web service which return with a JSON array such as

[
   {
      "c": "28403.81000000",
      "a": "28420.61000000",
      "A": "0.00351000",
   },
   {
      "c": "0.13690000",
      "a": "0.13930000",
      "A": "408.00000000",
   }
]

When try DynArrayLoadJson to parse the Json text, it will not process the "a" and "A" name of data. Report a error

First chance exception at $769390E2. Exception class ERttiException with message 'Duplicated A property name'.

So that in this case there is not possible to use DynArrayLoadJson function? Will you consider to handle this?

Tough the data titile "a" and "A" is Duplicated consider by pascal language, but our purpose is to parse the data to a pre-defined record, even the name is Duplicated in the Json text, that will not matter.

Thank you.

tbo · 2023-10-21 12:07:25

wqmeng wrote:

Tough the data titile "a" and "A" is Duplicated consider by pascal language, but our purpose is to parse the data to a pre-defined record, even the name is Duplicated in the Json text, that will not matter.

Try it this way:

type
  TTestRec = packed record
    c: Double;
    a1: Double;
    a2: Double;
  end;

  TTestRecArray = array of TTestRec;

const
  JSON = '[{"c": "28403.81000000","a": "28420.61000000","A": "0.00351000"},{"c": "0.13690000","a": "0.13930000","A": "408.00000000"}]';
var
  recs: TTestRecArray;
begin
  if DynArrayLoadJson(recs, JSON, TypeInfo(TTestRecArray), Nil, True) then
    ShowMessage(Format('a: %f, A: %f', [recs[1].a1, recs[1].a2]));

initialization
  Rtti.ByTypeInfo[TypeInfo(TTestRec)].Props.NameChanges(['c', 'a1', 'a2'], ['c', 'a', 'A']);

With best regards
Thomas

wqmeng · 2023-10-21 15:55:47

Hello Thomas,

Thank you very much.

larand54 · 2023-10-22 08:15:03

There is an error in the json structure
{
"c": "28403.81000000",
"a": "28420.61000000",
"A": "0.00351000", <--- THIS COMMA is not valid here.
},

You can use this link: https://jsonlint.com/
Best regards
//LG

wqmeng · 2023-10-22 08:59:59

@larand54

Sorry, I cut a small part from a long list of fields and forget to delete the COMMA at the end.

@Thomas

And do you know if it is possible to match a nested record such as by the DynArrayLoadJson in this case?

type
PTR1 = ^TR1;
TR1 = packed record
  a1: double;
  a2: double;
end;

TTestRec = packed record
  c: double;
  R1: PTR1;
end;

OR 

TTestRec = record
  c: double;
  R1: TR1;
end;

TTestRecArray = array of TTestRec;

const
  JSON = '[{"c": "28403.81000000","a": "28420.61000000","A": "0.00351000"},{"c": "0.13690000","a": "0.13930000","A": "408.00000000"}]';
var
  recs: TTestRecArray;

As you see that we split the JSON text to be a nested record, and would like to DynArrayLoadJson the JSON directly to the recs array.

Thank you

tbo · 2023-10-22 15:25:50

wqmeng wrote:

As you see that we split the JSON text to be a nested record, and would like to DynArrayLoadJson the JSON directly to the recs array.

This does not work automatically. Your definition must correspond to the JSON data format. But you can write and register your own serializer:

type
  TTestRec = packed record
    c: Double;     // -> c
    R1: record
      a1: Double;  // -> a
      a2: Double;  // -> A
    end;
  end;
  PTestRec = ^TTestRec;

  TTestRecArray = array of TTestRec;

  TTestRecArrayFiler = class(TObject)
  public
    class procedure CustomReader(var pmvContext: TJsonParserContext; pmData: Pointer);
  end;

class procedure TTestRecArrayFiler.CustomReader(var pmvContext: TJsonParserContext; pmData: Pointer);
var
  rec: PTestRec absolute pmData;
  recValues: array[0..2] of TValuePUtf8Char;
begin
  if pmvContext.ParseObject(['c', 'a', 'A'], @recValues) then
  begin
    rec.c := GetExtended(recValues[0].Text);
    rec.R1.a1 := GetExtended(recValues[1].Text);
    rec.R1.a2 := GetExtended(recValues[2].Text);
  end;
end;

const
  JSON = '[{"c":"28403.81000000","a":"28420.61000000","A":"0.00351000"},{"c":"0.13690000","a":"0.13930000","A":"408.00000000"}]';
var
  recs: TTestRecArray;
begin
  if DynArrayLoadJson(recs, JSON, TypeInfo(TTestRecArray), Nil, True) then
  begin
    for var i: Integer := 0 to High(recs) do
      ShowMessage(Format('c: %.5f, a: %.5f, A: %.5f', [recs[i].c, recs[i].R1.a1, recs[i].R1.a2]));
  end;
  
initialization
  TRttiJson.RegisterCustomSerializer(TypeInfo(TTestRec), TTestRecArrayFiler.CustomReader, Nil);

With best regards
Thomas

ab · 2023-10-23 09:42:00

It was obvious that a TValuePUtf8Char.ToDouble method was missing.

With https://github.com/synopse/mORMot2/commit/9c246512
we could write:

  if pmvContext.ParseObject(['c', 'a', 'A'], @recValues) then
  begin
    rec.c := recValues[0].GetDouble;
    rec.R1.a1 :=recValues[1].GetDouble;
    rec.R1.a2 := recValues[2].GetDouble;
  end;

which seems more direct.

tbo · 2023-10-23 19:13:19

ab wrote:

It was obvious that a TValuePUtf8Char.ToDouble method was missing.

I would lean towards the following:

type
  TValuePUtf8Char = record
    ...
    function ToFloat: TSynExtended; overload;
    function ToFloat(const pmcDefaultValue: TSynExtended): TSynExtended; overload;

function TValuePUtf8Char.ToFloat: TSynExtended;
begin
  Result := GetExtended(Text);
end;

function TValuePUtf8Char.ToFloat(const pmcDefaultValue: TSynExtended): TSynExtended;
var
  err: Integer;
begin
  Result := GetExtended(Text, err);
  if err <> 0 then
    Result := pmcDefaultValue;
end;

Then it can be written like this:

if pmvContext.ParseObject(['c', 'a', 'A'], @recValues) then
begin
  rec.c := recValues[0].ToFloat;
  rec.R1.a1 := recValues[1].ToFloat(-1);
  rec.R1.a2 := recValues[2].ToFloat(NaN);

With best regards
Thomas

wqmeng · 2023-10-26 04:52:14

Hello Thomas and ab,

Thank you for all your great works.

Since we added ToDouble function, why not also add a ToCurrency function too? They are almost the same.

And another matter which is not a must be feature, we have found that when we would like to skip some field, in this case if we only want 'A' and skip 'a', and set recValues = array[0..1] of TValuePUtf8Char;

pmvContext.ParseObject(['c', 'A'], @recValues)

We will get the 'c', 'a' values, not 'c', 'A'.

I also tried to change the props name in the rtti with

Rtti.ByTypeInfo[TypeInfo(TTestRec)].Props.NameChanges(['a2'], ['A']);

It seems that the above changes will not work.
I mean the parser will not care the Props Name's Letter case. If the Name is case sensitive, then the problems above should all get resolved.

Thank you very much.

Last edited by wqmeng (2023-10-26 04:55:14)

mORMot Open Source

#1 2023-10-21 09:47:22

DynArrayLoadJson can not process the JSON with "a", "A" names data?

#2 2023-10-21 12:07:25

Re: DynArrayLoadJson can not process the JSON with "a", "A" names data?

#3 2023-10-21 15:55:47

Re: DynArrayLoadJson can not process the JSON with "a", "A" names data?

#4 2023-10-22 08:15:03

Re: DynArrayLoadJson can not process the JSON with "a", "A" names data?

#5 2023-10-22 08:59:59

Re: DynArrayLoadJson can not process the JSON with "a", "A" names data?

#6 2023-10-22 15:25:50

Re: DynArrayLoadJson can not process the JSON with "a", "A" names data?

#7 2023-10-23 09:42:00

Re: DynArrayLoadJson can not process the JSON with "a", "A" names data?

#8 2023-10-23 19:13:19

Re: DynArrayLoadJson can not process the JSON with "a", "A" names data?

#9 2023-10-26 04:52:14

Re: DynArrayLoadJson can not process the JSON with "a", "A" names data?

Board footer