mORMot Open Source

@tbo  Thank you for the heads up.

I am looking at mormot.net.client.HttpGet function, which does not provide a parameter for setting connection time out. Internally, HttpGet ->OpenHttpGet->OpenHttp->TCrtSocket.Open->THttpClientSocket.Create(0)

I understand I can change HTTP_DEFAULT_RECEIVETIMEOUT to a different value, but I really don't want to change that default value.

Is it an intentional design that mormot.net.client.HttpGet does not allow setting a non-default timeout?  @ab  Do you think it make sense to add an extra timeout parameter, so we can use a non-default value?

That fixed it.

Thank you.

Thank you.

I downloaded latest commit as of 8:11am, Jan 6, 2022 (Eastern Time).  This seems to fix the "hanging problem" of the unit test case (as reported above), and that test case was able to pass.

But then, another test case fails, at mormot.core.json, line 3196-3236, function GotoEndJsonItemString(P: PUtf8Char): PUtf8Char;

it hangs inside that function, and was never able to return.

The following is the call stack:

mormot.core.json.GotoEndJsonItemString
mormot.orm.core.TOrmTableJson.ParseAndConvert
mormot.orm.core.TOrmTableJson.CreateFromTables
mormot.orm.storage.TRestStorageInMemory.LoadFromJson
mormot.orm.storage.TRestStorageInMemory.ReloadFromFile
mormot.orm.storage.TRestStorageInMemory.Create
...

Any thoughts?

@ab - you are right!  After making the constructor override, everything works!

Thank you @ab again for the advice.  I installed Lazarus on Windows and on Kali Linux,  with the docked layout enabled.   It looks quite similar to Delphi, and very neat.

I am surprised and impressed that Lazarus/FPC has progressed this good.  It is great to be able to directly build applications inside Linux.

Thanks again!  Go mORMot~

Thank you @ab

Forgive my ignorance. Is FPC based on LLVM (or gcc), or it is by itself?  Since it is your favorite, I guess performance wise, it would generate better optimized binary code than Delphi?

Yes it is fixed.

How did you make it happen that mormot.core.base.finalization is guaranteed to be called last?

It is a time-series in-memory (mORMot SQLite direct in-mem) DB - the data are from roadway network sensors that report data every 1 minutes - so the data are WRITTEN to the DB every minute

There are 6000 sensors.

Every 1 minute, I need to calculate a statistical measure using the latest 20 minutes data — this query is for for the purpose of retrieving (READ) the lastest 20 minutes data. It is like a 20-minutes rolling window moving forward at 1 minute step in real time.

The database is not a read only DB - it keeps receiving new data every 1 minutes from those 6000 sensors. So I guess “replication” is really not meaningful here?

Yes it works now

hi @ab

mormot.ab.rad.pas,  line84

type
  TBcd = FMTBcd.TBCD;

gives the following compiling error for Delphi 11 (at least):
[dcc64 Error] mormot.db.rad.pas(84): E2003 Undeclared identifier: 'FMTBcd'
[dcc64 Error] mormot.db.rad.pas(84): E2029 '=' expected but ';' found

Thank you very much for the advice.  I'll look into it.

The time series data is generated from sensors deployed at city streets (6~7K streets), at an interval of every 1 minute - so the ID burst is really happening every 1 minute, thus we need 6K new IDs to be generated every 1 minute.  Ah - I mis-stated - the daily data increment is not 100 million but 10 million records/day (3650 million/yearly).

I tried and it doesn’t work with postgresql

The partition key must be part of the primary key and any unique constraint.

I have an ever-increasing time series database - at a rate of 100 million records per day - so I need to partition it for query performance. I need to keep at least 1 year data (365000 million records)

That is what I thought, but I just was just asking in case there might be more elegant way from mORMot2 framework side.

In the OnTableCreate handler,  what is the best way to determine the Table Creation result (True / False)?

TRestStorageExternal has a method ExecuteDirect which would return a boolean, but it is not accessible from TSqlDBConnectionProperties.

function TMormotOrmModel.OnTableCreate(ASender: TSqlDBConnectionProperties; const ATableName: RawUtf8; const
  AColumns: TSqlDBColumnCreateDynArray; const ASQL: RawUtf8): Boolean;
begin
   ASender.ExecuteNoResult(ASQL, []); // <<--- how to determine if this table-creation Query is successful or not?   
end;

Edit:

Is it possible to changed the callback parameter const ATableName: RawUtf8 to const ATable: TOrmClass?

Congratulations!  @ab

Really, really neat, amazing, and fantastic work!  It rocks!!!!

Wow wow wow! 
Yes Yes Yes! - this time, it is indeed that the Direct SQLite File/InMem is much faster than External SQLite File/InMem!!!!!!
Super, super, super fasttttttttttttttttttttttttt!

Thank you!!!!!!!!!!!!!!!!!!!!!!!!

Many thanks @ab!

Not sure if it makes sense — Would it be possible to add another callback for TResrServerDB.CreateMissingTables?  The callback is expected to include the table class, and generated SQL passed by reference -- this way, "PARTITION BY RANGE" can be appended,  and extra SQL code can be added for creating child partitions.  The table class is needed - thus by the table class type, the callback can put different/customized partition logic according to the table class.

This seems to be better and organized than directly creating tables before calling CreateMissingTables.

Thank you again.  If you don't think it makes senses, that is also fine. Just let us know your thoughts.