mORMot Open Source

mik

Member

Registered: 2015-10-14

Posts: 8

[REQUEST] tolerant JSON parsing for database backward compatibility

suppose we have :

  TMyItem = class(TCollectionItem)
  private
    fBar: String;
    fFoo: String;
  ...
  end;

  TMyCollection= class(TCollection)
   ...
  public
    function Add: TMyItem;
    ...
  end;


  TMyClass = class(TMySQLRecord)
  public
    fMC: TMyCollection;
    .....
  end;

in the database fMC field will be saved as JSON, for example:

[{"Bar":"b1", "Foo":"f1"},{"Bar":"b2", "Foo":"f2"}]

We release application to clients with this database as version 1.

Later on, we decide to extend TMyItem and add a new field:

  TMyItem = class(TCollectionItem)
  private
    fBar: String;
    fFoo: String;
    fNew: String;
  ...
  end;

fNew field is handled by the new code (version 2). Now in the database the field will be saved as:

[{"Bar":"b1", "Foo":"f1", "New":"n1"},{"Bar":"b2", "Foo":"f2", "New":"n2"}]

Now I want to distribute this database to all clients, some of them are using application version 1, and some application version 2 (and are not willing to upgrade). Unfortunately, this will not work for "version 1". Current TJSONToObject.Parse will discover the "unknown" property "New" - this will stop the parser, and the actual content of the read record will contain only the first collection item - all other will be ignored.

The requested fix is rather simple, and safe (IMHO):

in the mormot.pas unit:

procedure TSQLPropInfoRTTIObject.SetValue(Instance: TObject; Value: PUTF8Char;
  wasString: boolean);
var valid: boolean;
    tmp: TSynTempBuffer;
begin
  tmp.Init(Value); // private copy since the buffer will be modified
  try
    PropInfo^.ClassFromJSON(Instance,tmp.buf,valid, JSONTOOBJECT_TOLERANTOPTIONS);
  finally
    tmp.Done;
  end;
end;

I've added JSONTOOBJECT_TOLERANTOPTIONS parameter to the call of ClassFromJSON()

Can this fix be propagated to production code?

Maybe a better way to handle it is to find a way to define the "Options: TJSONToObjectOptions" in such a way that user can control it and pass its user-controlled value to TSQLPropInfoRTTIObject.SetValue() (possibly to some other methods) - but maybe this is an overkill...

Please let me know your thoughts.

Last edited by mik (2019-09-19 02:06:53)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,666

Website

Re: [REQUEST] tolerant JSON parsing for database backward compatibility

Good idea.
I don't think it would break any existing code, just allow more relaxed behavior, as you expect.

Please check https://synopse.info/fossil/info/ed58b97a61