Serializing an array of records How To?

larand54 · 2023-02-22 08:52:22

I have a class of TSynPersistent containing an array of records that I want to populate from a DB-table.
For the moment while testing, I use SQLite3.

I worked out a TOrm descendant used for reading the data.
Everything reads correctly except for the array which is nil.

I thought I didn't need to register but I now realized that I was wrong.

But I don't know how to do it. Following instructions from the documentation I tried this:

The record:

  TWeek = packed record
    rYear: TYear;
    rIndex: TWeekBufferIndex;
    rWeekNo: TWeekNo;
    rAvailableAmt: TAvailableAmt;
    rRequiredAmt: TRequiredAmt;
    function getAvailableOnWeek: TAvailableOnWeek;
    property availableOnWeek: TAvailableOnWeek read getAvailableOnWeek;
  end;
  TWeekArray                 = array of TWeek;

The class:

  TWeeks = class(TSynpersistent)
    private
      fWeekArray  : TWeekArray;
      fActualWeek : TWeekNo;
    public
      function getWeek(index: integer): TWeek;
      procedure SetWeek(const index: integer; aValue: TWeek);
    published
      property weekArray: TWeekArray read fWeekArray write fWeekArray;
  end;

The registration:

initialization
var T: Pointer;
  T := TypeInfo(TWeekArray);
  TJSONSerializer.RegisterObjArrayForJSON(T,TWeekArray);

The TOrm descendant:

  TOrmWeeks = class(TOrm)
    private
      fWeeks      : TWeekArray;
      fActualWeek : TWeekNo;
    protected
    public
    constructor create;
    published
      property ActualWeek: TWeekNo read fActualWeek write fActualWeek;
      property Weeks: TWeekArray read fWeeks write fWeeks;
  end;

The problem is that the compiler doesn't like this: " TJSONSerializer.RegisterObjArrayForJSON(T,TWeekArray);"
It say's: [dcc32 Error] dddLOBTypes.pas(294): E2029 '(' expected but ')' found
and next error: [dcc32 Error] dddLOBTypes.pas(295): E2250 There is no overloaded version of 'RegisterObjArrayForJson' that can be called with these arguments

If I replace TWeekArray with a class the compiler will be satisfied.
So what do I missing here?

lfyey121 · 2023-02-22 09:35:41

it should be: TJsonSerializer.registerObjArrayForJson(T,Tweek) or Rtti.registerObjArray(T,Tweek)

Last edited by lfyey121 (2023-02-22 09:38:37)

ttomas · 2023-02-22 09:47:08

If TWeek = class, not record, then name array TWeekObjArray = array of TWeek. Then you can use
Rtti.RegisterObjArray(T,Tweek);
In your case TWeek is packed record, just use:
Rtti.RegisterType(TypeInfo(TWeek));
Rtti.RegisterType(TypeInfo(TWeekArray));
If you need compatibility with older Delphi versions and Fpc then you can use
Rtti.RegisterFromText(TypeInfo(TWeek), __TWeek);
where __TWeek is const string with definition of record, read documentation for details.

Edited :-), Sorry typo TRtti->Rtti

Last edited by ttomas (2023-02-22 13:58:00)

larand54 · 2023-02-22 09:53:53

Where can I find TRtti? I use System.rtti but that won't help.
Ok, I found it in mormot.core.rtti.
But I couldn't use TRtti but Rtti.

Last edited by larand54 (2023-02-22 09:59:39)

larand54 · 2023-02-22 10:04:57

Ok, the compiler is now satisfied but I still missing the data for the WeekArray.

The field "Week" in the db looks like : '{"currentYear": 2023,"ActualWeek": 8,"weeks": [{"ryear": 2023,"rindex": 1,"rweekno": 8,"rAvailableAmt": 186,"rRequiredAmt": 99}]}'.
It's a valid json string but I'm not sure if it is valid in this case?

ttomas · 2023-02-22 14:10:01

This json is not array! You need record

TMyData = packed record
  currentYear: integer;
  ActualWeek: integer;
  weeks: TWeekArray;
end;

Then you can use RecordLoadJson/RecordSaveJson from mormot.core.json.pas

larand54 · 2023-02-22 17:05:27

Sorry, the JSON I wrote is not a real JSON array I only typed it in by hand into the database record to test my program.
I'm not sure about how a json string should look to correspond to my class so I'm not surprised that it's not correct.

A Week is a packed record with the name TWeek and then I declared the array TWeekArray from TWeek.
This Array is contained in a class: TWeeks and this class is the one I want to persist in a sqlite3-table. In fact, later I want to have it in an IN-Memory table but in the beginning, I will have it in a database during the test.

So the question is, how can I best persist this table, TWeeks, you have it at the beginning of this thread.
Forget about the JSON string.

tbo · 2023-02-22 19:52:03

larand54 wrote:

So the question is, how can I best persist this table, TWeeks, you have it at the beginning of this thread.

type
  TWeeks = class(TOrm)
  private
    fWeekArray: TWeekArray;
  public
    function AddWeek(pmYear: TYear; ...): Integer;
  published
    property weekArray: TWeekArray
      read fWeekArray write fWeekArray;
  end;

FRestServer := TRestServerDB.CreateWithOwnModel([TWeeks], MakePath([Executable.ProgramFilePath, 'data.dat']), False);
FRestServer.DB.Synchronous := smNormal;
FRestServer.DB.LockingMode := lmExclusive;
FRestServer.Server.CreateMissingTables(0, [itoNoAutoCreateGroups, itoNoAutoCreateUsers]);

ormWeeks := TWeeks.Create;
try
  ormWeeks.AddWeek(2022, ...);
  ormWeeks.AddWeek(2023, ...);
  FRestServer.Server.Add(ormWeeks, True);
finally
  ormWeeks.Free;
end;

With best regards
Thomas

larand54 · 2023-02-23 09:01:35

Ok, thanks a lot for the almost complete code.
But this one store the week array as a blob, how about having it stored as a JSON string? Wouldn't I need to register the array for JSON? And what more?
with best regards
Lars

dcoun · 2023-02-23 09:21:35

larand54 wrote:

Ok, thanks a lot for the almost complete code.
But this one store the week array as a blob, how about having it stored as a JSON string? Wouldn't I need to register the array for JSON? And what more?
with best regards
Lars

Arrays are by default stored as blob. Me too, I need them as JSON. The way I am using is:

Declare the field as RawJSON in Torm definition with getter/setter and use DynArraySaveJson/DynArrayLoadJson inside them.

cweekArray:TArray<TWeekArray>;
property weekArray: RawJSON read getWeekArray write setWeekArray; 
....

function TORMmyorm.getWeekArray: rawjson;
begin result:=DynArraySaveJson(cweekArray,typeinfo(TArray<TWeekArray>)); end;

procedure TORMmyorm.setWeekArray(const Value: rawjson);
begin if (value='') or (not DynArrayLoadJson(cweekArray,value,typeinfo(TArray<TWeekArray>))) then weekArray:=nil; end;

RawJSON is automatically created as TEXT

Last edited by dcoun (2023-02-23 09:25:12)

larand54 · 2023-02-23 11:04:33

Ok, that worked well, so many thanks.
I think there are a lot of different ways of doing this and it's hard to find out which method to use.
The project I'm working on now is a replacement of an existing function to make it much faster so the speed is the main purpose of this project.
The reason I want the data of the array stored as JSON is mainly to have better control when testing, perhaps when the application is tested and works well I might consider going back to blob-storage.
But it is still nice to have the possibility to check out the contents in the database.
Any ideas or suggestions are very welcome.

best regards
Lars

ttomas · 2023-02-23 11:23:54

If you use SQLite and store as RawJSON you can use JSON functions and operators.
https://www.sqlite.org/json1.html

tbo · 2023-02-23 16:02:11

If you want to look at the JSON functionality mentioned by ttomas, you can find the example ORM-DocVariant here.

With best regards
Thomas

Last edited by tbo (2023-02-23 19:20:13)

mORMot Open Source

#1 2023-02-22 08:52:22

Serializing an array of records How To?

#2 2023-02-22 09:35:41

Re: Serializing an array of records How To?

#3 2023-02-22 09:47:08

Re: Serializing an array of records How To?

#4 2023-02-22 09:53:53

Re: Serializing an array of records How To?

#5 2023-02-22 10:04:57

Re: Serializing an array of records How To?

#6 2023-02-22 14:10:01

Re: Serializing an array of records How To?

#7 2023-02-22 17:05:27

Re: Serializing an array of records How To?

#8 2023-02-22 19:52:03

Re: Serializing an array of records How To?

#9 2023-02-23 09:01:35

Re: Serializing an array of records How To?

#10 2023-02-23 09:21:35

Re: Serializing an array of records How To?

#11 2023-02-23 11:04:33

Re: Serializing an array of records How To?

#12 2023-02-23 11:23:54

Re: Serializing an array of records How To?

#13 2023-02-23 16:02:11

Re: Serializing an array of records How To?

Board footer