Custom Json Serializer for Simple Types

Hafedh TRIMECHE · 2020-12-08 11:25:36

Hello,

Would a custom Json serializer be used for simple types Char, TDateTime, TTime, ...?

Best regards.

ab · 2021-01-02 10:54:19

With mORMot 1.18 you can't.
With mORMot 2 you could, but it is very unsafe, because it is easy to break some existing code.

You should better define your own sub-type, and set its custom serialization.

Hafedh TRIMECHE · 2021-01-02 12:01:20

Thank you for response.

Would you please provide an example related to sub-type custom serialization?

Best regards.

ab · 2021-01-02 14:55:27

Something like

type
  TMyDateTime = type TDateTime;

...
  TTextWriter.RegisterCustomJSONSerializer(TypeInfo(TMyDateTime), ...);

Hafedh TRIMECHE · 2021-01-02 17:07:18

Using mORMot 2 (mormot.core.json), an error is generated.

[dcc32 Error] Main.pas(215): E2003 Undeclared identifier: 'RegisterCustomJSONSerializer'

ab · 2021-01-02 20:15:20

This was for mORMot 1.18.

For mORMot 2, just use TRttiJson.RegisterCustomSerializer().

Hafedh TRIMECHE · 2021-01-03 12:36:17

Hello,

Please how to cancel writing an empty string or integer value set to 0 (Zero) so the generated Json file would be as compact as possible?

class procedure TCustomJSONSerializer.StringWriter(W:TTextWriter;Data:pointer;Options:TTextWriterWriteObjectOptions);
var
  Value : string;
begin
  Value := String(Data^)+#0;
  if Value>#0 then .....
              else W.AddTextW(PWord(Value));
end;

Best regards.

ab · 2021-01-04 09:04:23

This is done globally using twoIgnoreDefaultInRecord option.

Hafedh TRIMECHE · 2021-01-04 10:33:07

Demo project: https://mega.nz/file/Dl4wmBhY#-r34M8GP5 … KayXu38Ejo
Please note that:
1- Write Option is set to [twoIgnoreDefaultInRecord,twoEnumSetsAsTextInRecord] but the output Json still including empty fields (Empty record checked)

  Serializer = record
  const
    WriteOptions = [twoIgnoreDefaultInRecord,twoEnumSetsAsTextInRecord];
  public
    class function Marshal<T>(const Data:T):string;overload;inline;static;
    class function Marshal<T>(const Data:T;const Service:Byte):TBytes;overload;inline;static;
    class function Unmarshal<T>(Text:string;out Data:T):Boolean;overload;inline;static;
    class function Unmarshal<T>(const TextBytes:TBytes;out Data:T):Boolean;overload;inline;static;
  end;

class function Serializer.Marshal<T>(const Data:T):string;
var
  rResult : RawUTF8;
begin
  if @Data=nil then Result := '' else
  begin
    try
      mormot.core.json.SaveJson(Data,TypeInfo(T),WriteOptions,rResult);
    except
      rResult := '';
    end;
    Result := string(rResult);
    if Result='' then raise Exception.Create('Serializer.Marshal:');
  end;
end;

2- When embedding an Object into the record, a memory leak is generated:

A memory block has been leaked. The size is: 20

This block was allocated by thread 0xFEC, and the stack trace (return addresses) at the time was:
0064A753 [uMemory.pas][uMemory][NewAllocMem][73]
00407152 [System.pas][System][AllocMem][4773]
00668CC1 [mormot.core.rtti.pas][mormot.core.rtti][DynArrayNew][4934]
0069C836 [mormot.core.json.pas][mormot.core.json][_JL_DynArray][7395]
0069B6C3 [mormot.core.json.pas][mormot.core.json][_JL_Integer][7059]
0069CBA8 [mormot.core.json.pas][mormot.core.json][JsonLoadProp][7473]
0069D29E [mormot.core.json.pas][mormot.core.json][_JL_RttiCustom][7578]
00418CB2 [FastMM4][CalculateHeaderCheckSum]
00419B20 [FastMM4][DebugGetMem]
0064A828 [uMemory.pas][uMemory][NewAllocMem][83]
0069C87A [mormot.core.json.pas][mormot.core.json][_JL_DynArray][7407]

The block is currently used for an object of class: Unknown

The allocation number is: 3202

3- Custom Serialization of TBytes can't be achieved because the Serializer wont distinguish between TBytes and other dynamic array types.
Uncommenting for testing.

//    RegisterCustomSerializer(TypeInfo(TBytes),BytesReader,BytesWriter);

Best regards.

ab · 2021-01-04 11:59:49

1. Perhaps there are some limitation. Empty values are handled, but I am not sure for not empty record.

2. Class fields are not handled properly in records, for sure.
This is not a mORMot problem, but a pascal language limitation. You need to call Free manually. Use nested classes for proper class lifetime support.

3. You need to sub-class the types for custom serialization.

I don't know what you expect and want to do.
Trying to push the serializer into its border cases is probably not the right way to do what you expect, unless you understand better how it works.
What I am sure is that your root goal (change the default serialization of plain Char/TBytes/DateTime/Time/Date) is weird and not a good idea.
Existing code will be broken.

Possible better ideas:
a) Use a custom serializer for a whole record, not simple types.
b) Use an in-between record type for serialization of complex content - to be handled like a DTO.

Hafedh TRIMECHE · 2021-06-30 00:53:02

Hello,

Please how to implement a custom serialization for TBytes?

  TCustomJSONSerializer=class
  private
    class procedure CustomReader(var Context:TJsonParserContext;Data:pointer);
    class procedure CustomWriter(W:TTextWriter;Data:pointer;Options:TTextWriterWriteObjectOptions);
  end;


class procedure TCustomJSONSerializer.CustomReader(var Context:TJsonParserContext;Data:pointer);
begin
  ...
end;

class procedure TCustomJSONSerializer.CustomWriter(W:TTextWriter;Data:pointer;Options:TTextWriterWriteObjectOptions);
begin
  ...
end;

initialization

  TRttiJson.RegisterCustomSerializer(TypeInfo(TBytes),TCustomJSONSerializer.CustomReader,TCustomJSONSerializer.CustomWriter);

ab · 2021-07-01 10:12:03

Take a look at TCollTstDynArray.FVReader/FVWriter and TCollTstDynArray.FVReader2/FVWriter2 in test.core.data.pas

https://github.com/synopse/mORMot2/blob … a.pas#L718

Hafedh TRIMECHE · 2021-07-01 16:02:25

The solution I implemented to handle TBytes:

class procedure TCustomJSONSerializer.CustomWriter(W:TTextWriter;Data:pointer;Options:TTextWriterWriteObjectOptions);
var
  Bytes   : TBytes;
  EndChar : AnsiChar;
begin
  Bytes   := TBytes(Data);
  EndChar := W.LastChar;
  W.CancelLastChar;
  if EndChar='[' then
  begin
    W.Add('"');
    if Assigned(Bytes) then W.WrBase64(@Bytes[0],Length(Bytes),False);
    W.Add('"');
  end;
end;

but generates a wrong Json with ] character not removed:

{"I1":0,"I2":0,"S":"String","B":"AQIDBA=="],"D":"2021-07-01T16:56:40","T":44378.7060185417}

Record used:
  TData2=
  record
    I1 : Integer;
    I2 : Integer;
    S  : string;
    B  : TBytes;
    D  : TDateTime;
    T  : TTime;
  end;

Is there a way to replace the hole TBytes content by the Base64 encoded not iterating into the array elements?

ab · 2021-07-01 17:53:24

The easiest and safest is to customize TData2 and not TBytes.

Overwriting TBytes is not safe because it could break the default serialization.

IIRC if you define a RawByteString instead of a TBytes, you will have a base-64 encoded string by default.

mORMot Open Source

#1 2020-12-08 11:25:36

Custom Json Serializer for Simple Types

#2 2021-01-02 10:54:19

Re: Custom Json Serializer for Simple Types

#3 2021-01-02 12:01:20

Re: Custom Json Serializer for Simple Types

#4 2021-01-02 14:55:27

Re: Custom Json Serializer for Simple Types

#5 2021-01-02 17:07:18

Re: Custom Json Serializer for Simple Types

#6 2021-01-02 20:15:20

Re: Custom Json Serializer for Simple Types

#7 2021-01-03 12:36:17

Re: Custom Json Serializer for Simple Types

#8 2021-01-04 09:04:23

Re: Custom Json Serializer for Simple Types

#9 2021-01-04 10:33:07

Re: Custom Json Serializer for Simple Types

#10 2021-01-04 11:59:49

Re: Custom Json Serializer for Simple Types

#11 2021-06-30 00:53:02

Re: Custom Json Serializer for Simple Types

#12 2021-07-01 10:12:03

Re: Custom Json Serializer for Simple Types

#13 2021-07-01 16:02:25

Re: Custom Json Serializer for Simple Types

#14 2021-07-01 17:53:24

Re: Custom Json Serializer for Simple Types

Board footer