Client-server ORM, static tables and caching

dcoun · 2021-11-24 08:35:40

Probably stupid questions, I apologize, but I am not sure if the following is supported and I am a bit confused:

I am having a number of tables in an external database mapped to TormXXX classes in Mormot2 that can seldom be changed.
The client does not modify them, an other external process can modify them.

1. When using RetrieveIlist or OneFieldValues or any other content request, is this data stored in a possible client's cache?
2. Is it possible by the framework to retrieve a local memory/disk copy using Torm functions to do queries locally and update them when needed?
3. Should I have to create different models?
4. Which is the most efficient way to have a local memory DB with ORMs retrieved by a server running mormot?

Plus: To have an offline work in ORMs and upload them after being online is really fantastic also, but in such a case how does mormot2 deal with new Torms instances that are related to other Torms?

Thank you in advance

Last edited by dcoun (2021-11-24 10:59:56)

ab · 2021-11-24 17:40:38

The mORMot 1 documentation is accurate about mORMot 2 caching policy.
Please ensure you read https://synopse.info/files/html/Synopse … l#MNDX_706
and all entries about "Cache" in the Keyword index list https://synopse.info/files/html/Synopse … X_KEYWORDS

I guess you are using the ORM also at client level.
By default, there is no cache unless you setup the ORM cache at client level.

Some ideas:

a) Instead of using the client/server ORM directly, you may define an API which would work as requested, with a local storage at client side, e.g. using a full in-memory list of TOrm instances.

or

b) Take a look at master/slave replication https://synopse.info/files/html/Synopse … l#TITL_147 which may be a working solution for your problem.
See the usecases possible in https://synopse.info/files/html/Synopse … #TITLE_123

dcoun · 2021-11-25 21:45:11

Thank you a lot Arnaud
An other problem I am not sure is the cross-platform way to retrieve data from an ORM table and put them inside a local table:
I looked at
var list:ilist<tormtable>;
Trestclient.orm.RetrieveIList(tormtable,list,'',[],'*')
but the data in the ilist should be copied to a TRestStorageInMemory for example.
ilist owns the data and I should copy them to insert them in a TRestStorageInMemory
How to stop the ilist<tormtable> that it is returned by RetrieveIList to free the items?
Do you propose an other way?

ab · 2021-11-25 22:29:34

I would use JSON as an efficient way of copying data between instances.

dcoun · 2021-11-26 08:22:43

Thank you a lot Arnaud
I noticed that I could also run the following:
jsonstring:=h.Orm.RetrieveListJson(tormtest,'',[],'*');
Which is the best and cross-platform way to transform this json string to a Tlist<tormtest> and to an array of records, keeping the ownership of the instances to my code?
Will it be efficient for 5-10 records as with 5000 records?

Also please notice the following:
Using Mormot2, I tried to write the following:
mydata:=TRestStorageInMemory.Create(TOrmtest,nil);

After that I am feed it with data, every ok
I am going now to retrieve data and I am starting something easy
mydata.RetrieveIList(tormtest,tt,'',[],'*')

I get a access violation due to line 2991 in mormot.orm.storage
ForceAjax := ForceAjax or
not Owner.Owner.NoAjaxJson;

Owner here is null and this is the cause for the violation
Changing it to the following, no error is created:
ForceAjax := ForceAjax or
( assigned(owner) and (not Owner.Owner.NoAjaxJson));

Last edited by dcoun (2021-11-26 08:24:04)

ab · 2021-11-26 08:31:52

The fastest is to use TOrm.CreateAndFillPrepare(json) then FillNext in a loop when you want to access the tormtest values.
But 5000 records is very low, so creating a TList<> won't slow down your process - note that the ORM generates JSON from the DB anyway.

You can't use TRestStorageInMemory stand-alone.
Please use TRestOrmServerFullMemory.

dcoun · 2021-11-26 09:13:10

I wrote the following and it works

var o:TOrmtest; t:RawJson;
begin
testlist:=tlist<tormtest>.create;
t:=mydata.EngineList('SELECT ID,nam FROM table',false);
o:=TOrmtest.CreateAndFillPrepare(t);
while o.FillOne do
testlist.Add(tormtest(o.CreateCopy));
o.FillClose;
o.Free;
end;

Thank you again Arnaud

Last edited by dcoun (2021-11-26 10:16:54)

ab · 2021-11-26 14:19:24

Note that o.FillClose will be done by o.Free anyway.

mORMot Open Source

#1 2021-11-24 08:35:40

Client-server ORM, static tables and caching

#2 2021-11-24 17:40:38

Re: Client-server ORM, static tables and caching

#3 2021-11-25 21:45:11

Re: Client-server ORM, static tables and caching

#4 2021-11-25 22:29:34

Re: Client-server ORM, static tables and caching

#5 2021-11-26 08:22:43

Re: Client-server ORM, static tables and caching

#6 2021-11-26 08:31:52

Re: Client-server ORM, static tables and caching

#7 2021-11-26 09:13:10

Re: Client-server ORM, static tables and caching

#8 2021-11-26 14:19:24

Re: Client-server ORM, static tables and caching

Board footer