#1 Re: mORMot 2 » Linux or Windows » 2023-02-04 01:50:41

Never used Lazarus before, and trying to pick it up.

When installing mormot2ui into Lazarus, I got this error - should i worry about it?  I am using the latest clone (as of Feb 3, 2023).

Suspicious include path

The package mormot2ui 2.0.1 adds the path "$(PkgIncPath)" to the include path of the IDE. This si probably a misconfiguration of the package.

#2 mORMot 2 » Linux or Windows » 2023-01-16 15:29:58

xiwuping
Replies: 2

ab, congratulations on mORMot2!

Previously I've been using mORMot with Delphi.  But recently I've decided to embrace FPC + mORMot, seriously.

Question - in terms of performance,  will mORMot on Linux give better performance than on Windows, or similar? 

Thanks!

#3 Re: mORMot 1 » Can I use mORMot as a Rest Client Library? » 2021-01-01 16:27:51

Vitaly wrote:

I would recommend taking a look at SynCrtSock unit, there are several client classes. I'm using mostly TWinHTTP-based classes for many third-party REST services with different rather complex schemas. But probably the fastest and easiest way might be TSimpleHttpClient.
JSON parsing to object: https://synopse.info/files/html/api-1.1 … ONTOOBJECT

Thank you @Vitaly.  Appreciate your suggestion.  Following your suggestion, I looked into SynCrtSock.pas, and end up using HttpGet directly, and _JsonFast.

#4 mORMot 1 » Can I use mORMot as a Rest Client Library? » 2020-12-31 17:13:25

xiwuping
Replies: 2

I need to access a third-part Rest server (that I have no control) - it will return JSON data.

Is it possible to use mORMot as a Rest Client Library? For example, I'd like to use mORMot to access the JSON data, transform it to Delphi Object, then persist the data to another RDBMS.

Is mORMot a good candidate to accomplish this? Is there any document or sample that I can follow?

Thank you

#5 Re: mORMot 1 » Does Client of mORMot RPC server have to be in Delphi/FPC? » 2019-11-17 21:19:38

Thank you.

But is it already supporting automatically generated OpenAPI (also kow as 'Swagger') service definition? @edwinsn @ab

#6 mORMot 1 » Can Interfaced Service return another interfaced service? » 2019-11-14 20:55:06

xiwuping
Replies: 1

Is it "legal" to have an interfaced service to return another interfaced service?

INetwork = interface(IInvokable)
.....
end;

ISimulation = interface(IInvokable)
   function GetNetwork: INetwork;           <----------------------- is this allowed in interfaced service?
end;

#7 mORMot 1 » Does Client of mORMot RPC server have to be in Delphi/FPC? » 2019-11-14 17:23:57

xiwuping
Replies: 4

I have a mORMot interfaced service RPC server that exposes several service methods.  I understand how to make a client in Delphi to call those remote methods.

Is it possible to use another language to make the remote method calls too?  For example,  C++?

#8 Re: mORMot 1 » How to do with "rotated" tables using mORMot? » 2019-06-02 11:04:09

edwinsn wrote:

Really interesting question.

What do you mean by not having control of the server? Is the server also based on mORMot?

A thought:
- Pause the client, then perform the table rotating (renaming) on the db level (not the ORMot leve), empty the cache (both the client and server side), and resuming using the clients.

The database server is just a standard MySQL database server. The tricky part is that - there is a separate, independent service not under my control which will constantly check the database server, and if the relevant tables have not been rotated (for example, at the beginning midnight of a new rotation time such as start of a new month), that service will CREATE new rotated table automatically.  This process is not controlled or managed within mORMot framework.

So, from mORMot side,  if the target table has already been rotated, it will do nothing and just connect to that table.  If the table SHOULD have been rotated, but has NOT been,  mORMot will create a new rotated table.  Is this doable?  Any advice?  Or there is some fundamental problem with the overall scheme?

#9 Re: mORMot 1 » How to do with "rotated" tables using mORMot? » 2019-06-01 17:08:29

TSQLRecord.SQLTableName

Can I override this class function, so as to access the correct rotated table names in an existing database server?  Inside the overridden SQLTablename function,  I can apply the same table name rotation rules as the database server (which I don't have control),  so the ORM can connect to the correct table with names changing over time.

If I do so,  any forseeable side-effects?

#10 Re: mORMot 1 » How to do with "rotated" tables using mORMot? » 2019-06-01 16:42:06

ab wrote:

See e.g. how TSQLRestStorageShard (and TSQLRestStorageShardDB) works.

Thank you for the heads up.  I am looking at the documentation.  For helping dummy users like me to jump start - would it be possible to have a demo project to illustrate the usage?

#11 mORMot 1 » How to do with "rotated" tables using mORMot? » 2019-06-01 15:15:16

xiwuping
Replies: 6

Need advice from mORMot experts.

I have a database server that archives real time simulation data in multiple tables.   Each table has incoming data of 10 million records per day.  Because the data are ever-increasing with time,  I have to rotate the table names by year-month.  That means,  every month,  a new table will be created, with the name as original_table_name_yyyymm. For example,  original_table_name_201901 representing the data archived for January, 2019.

So I have a situation that the table names are changing with time.

How can I cope with this situation with mORMot framework?  Can it automatically send/get data from tables that have changing names?

Thank you.

#12 Re: mORMot 1 » Still confused TSQLRestClientDB versus TSQLRestServerDB » 2019-05-27 12:51:10

ab wrote:

TSQLRestClientDB is when your code requires explicitly a TSQLRestClient descendant.
This is the case for some samples, but not very realistic.
But it is not mandatory, and using TSQLRestServer and a TSQLRest variable should be enough for most of the cases.

Usually, I use as variables/fields (using the constructor of the actual class):
- TSQLRest if possible (more abstract definition)
- TSQLRestClient for client side (and the proper connection class)
- TSQLRestServer or TSQLRestServerDB when direct access to the DB or the services is required
- almost never TSQLRestClientDB, which is just an encapsulation of TSQLRestServerDB

I would like to have both the Server, and Client in the same process, thus in the following code, which client class had I best use?

var
  client: TSQLRest;
  server: TSQLRestServerDB;
  model: TSQLModel;
begin
  model := TSQLModel.Create([TSQLAuthUser, TSQLAuthGroup]);
  server := TSQLRestServerDB.Create(model, 'test.db3', False);
  client := ?.Create()    // Which proper client class should I use?
  ... ...
end;

#13 mORMot 1 » Still confused TSQLRestClientDB versus TSQLRestServerDB » 2019-05-27 12:21:01

xiwuping
Replies: 4

Across the various mORMot official samples,  Erick Engelke book,  and some online blogs (such as this very good one: https://medium.com/@step.bester/the-orm … 11c8ac0c35),  I see sometimes people use TSQLRestServerDB directly to do  the ORM actions,  sometimes they use TSQLRestClientDB, and sometimes TSQLRestClient

Forgive my dumb question - what is the use case for each of these?

When should I -
- Use TSQLRestServerDB?
- Use TSQLRestClientDB?
- Use TSQLRestClient?

Any advice is appreciated.  Still on the learning curve.

#14 Re: mORMot 1 » TSQLDBUniDACConnectionProperties NOT used in an optimal way by mORMot? » 2019-05-26 22:52:35

ab wrote:

Please don't post huge source code in the forum, as stated by https://synopse.info/forum/misc.php?action=rules

Thank you for the heads up.  I edited the post and removed the source code.   The source code is now posted at gist.

#15 mORMot 1 » TSQLDBUniDACConnectionProperties NOT used in an optimal way by mORMot? » 2019-05-26 15:29:56

xiwuping
Replies: 2

I run the Samples 15-External DB performance, and I got the following results:
Insertion speed
   
                        Direct        Batch        Trans         Batch Trans
FireDAC SQlite3    22468       42605         53038       168953
UniDAC  SQlite3    401          8744           33784       45844

I am very upset with the poor performance of UniDAC, hence I contacted Devart. Here is their response:

"Note that unlike FireDAC, our components use by default the same settings as the client library sqlite3.dll when working with SQLite. Therefore, it’s necessary that you set the same values for the Locking_Mode and Synchronous parameters as for FireDAC:

  UniConnection.ExecSQL('PRAGMA locking_mode=EXCLUSIVE');
  UniConnection.ExecSQL('PRAGMA synchronous=0');

We are not able to identify the settings and function invocation order in the mORMot framework."

They also sent me their own sample code comparing UniDAC versus FireDAC, and the results are TOTALLY different:

UniDAC : 2.672 sec, INSERT count = 100000
FireDAC: 2.735 sec, INSERT count = 100000

UniDAC : 2.562 sec,  BATCH INSERT count = 1000000
FireDAC: 21.297 sec, BATCH INSERT count = 1000000

My feelings are that mORMot probably does not use UniDAC in an optimal way, which might not be fair for UniDAC users that wishes to use mORMot.  Any thoughts?  The the sample code that Devart sent back to me can be downlaode from this gist address: https://gist.github.com/wuxixigit/f32d2 … 8812e337a8

Thanks.

#16 Re: mORMot 1 » TSQLRestServerFullMemory Batch mode? » 2019-05-22 14:02:14

Update:  previous testing results are 64bit.  I switched to 32 bit, and mORMot virtual table seems to have better performance.

To insert 100000 records

FDMemTable Rio 10.3 UP1   -  187 ms
UniDAC Virtual Table v7.4.12 - 195 ms
mORMot TSQLRestServerFullMemory - 178 ms
DevExpress MemDataTable - 155 ms

So not bad.  I am a bit surprise that DevExpress TMemData is so fast.

#17 Re: mORMot 1 » TSQLRestServerFullMemory Batch mode? » 2019-05-21 14:37:49

I set smOff and lmExclusive,  it made no difference as I saw.

Re: "Your mORMot testing code uses the ORM, while the UniDAC testing code access the db directly, it's not a fair comparison. Use ISQLDBStatement (from SynDb.pas) instead."

- I understand what you meant. But all I want is to test the performance of mORMot's in-memory virtual table,  as compared to UniDAC's virtual table.  To that end,  the ORM element is quite thin here, and I don't think it is an unfair comparison.  I just want to  compare the virtual table performance of both.

I also tested by accessing TSQLRestServerFullMemory directly, still not very happy.  I am not sure if my mORMot code is the right way to use mORMot virtual table.  Any one can give me some headsup/advice?

I guess I was somewhat disappointed since everybody is talking about mORMot and everybody is saying it is highly optimized for performance, and it happens that I need a high-performance in-memory virtual table, hence give mORMot a try.

#18 Re: mORMot 1 » TSQLRestServerFullMemory Batch mode? » 2019-05-21 00:44:43

I tried to compare the performance of mORMot In Memory server versus UniDAC TVirtualTable. 

Here is the code:

**** For UniDAC **********
FUniDacVirtualTable.Clear;
  FUniDacVirtualTable.Open;

  BeginTest(Sender);
 
  FUniDacVirtualTable.DisableControls;
  try
    Field1 := FUniDacVirtualTable.FieldByName('ID');
    Field2 := FUniDacVirtualTable.FieldByName('Status');
    Field3 := FUniDacVirtualTable.FieldByName('Created');
    Field4 := FUniDacVirtualTable.FieldByName('Volume');

    for I := 1 to 100000 do
    begin
        FUniDacVirtualTable.Insert;
        Field1.AsInteger := I;
        Field2.AsString := 'Code' + IntToStr(I);
        Field3.AsDateTime := Date();
        Field4.AsFloat := Random(10000);
        FUniDacVirtualTable.Post;
    end;
  finally
    FUniDacVirtualTable.EnableControls;
    EndTest(Sender);
  end;
end;

******* For mORMot ***********

procedure TForm1.btnmORMotTestClick(Sender: TObject);
var
  I: Integer;
  recData: TmORMotTestRecord;
  IDs: TIDDynArray;
  restClient: TSQLRestClientDB;
  model: TSQLModel;
begin
  model := TSQLModel.Create([TmORMotTestRecord]);
  model.VirtualTableRegister(TmORMotTestRecord, TSQLVirtualTableBinary);
  restClient := TSQLRestClientDB.Create(model, nil, 'mORMotTes.data', TSQLRestServer. False, '');
  restClient.Server.CreateMissingTables;
  recData := TmORMotTestRecord.Create;

  BeginTest(Sender);
  try
    restClient.BatchStart(TmORMotTable);
    for I := 1 to 100000 do
    begin
      with recData do
      begin
        SeqID := I;
        Status := 'Code' + IntToStr(I);
        CreatedDate := Now;
        Volume := Random(10000);
      end;

      restClient.BatchAdd(recData, True);
    end;
    restClient.BatchSend(IDs);
  finally
    EndTest(Sender);
    recData.Free;
  end;
end;

------------------
The result is
UniDac: 208 ms
mORMoT: 265 ms

What am I missing here?  I expect mORMot to be faster!  Why it is slower than UniDAC's virtual table?

Also - how can I Save the data to File?  restClient.Server.StaticVirtualTable[TmORMotTestRecord].UpdateToFile doesn't exist.
       - Again, how can I load the data from a file?  I am asking because I want to bench marking the data loading time versus UniDAC's virtual table.
       -  How to save to a stream? Load from a stream?

#19 mORMot 1 » TSQLRestServerFullMemory Batch mode? » 2019-05-20 14:37:17

xiwuping
Replies: 10

Does TSQLRestServerFullMemory support BatchAdd/Update/Delete?

#20 mORMot 1 » MainDemo cannot be compiled in Delphi » 2018-12-23 18:54:01

xiwuping
Replies: 2

The MainDemo seems can NOT be compiled in Delphi (RIO 10.3).  Any heads up?

Thank you.

#21 Re: mORMot 1 » What is best practice for multi-threaded in-memory database? » 2018-08-25 15:36:33

Thank you ab, for this extensive and insightful advice.

I looked into TSynQueue, seems it still relies on some "locking" among threads?  Does mORMot provide some "lock-free" circular buffer implementation already?  Or you are sure TSynQueue will be good to implement similar pattern as LMAX?

Thank you again.

#22 Re: mORMot 1 » What is best practice for multi-threaded in-memory database? » 2018-08-25 15:32:49

edwinsn wrote:

3 thoughts:
- Try ScaleMM and benchmark to see if there are any improvements.
- Try TSQLRestServerDB with in-memory sqlite db (use ":memory:" as the db name).
-  Use TSQLDataBase directly with in-memory sqlite db.

Out of curiosity, what kind of program is it?

Hi edwinsn - it is a simulation application for academic use.

#23 mORMot 1 » What is best practice for multi-threaded in-memory database? » 2018-08-24 23:10:19

xiwuping
Replies: 7

Hi

I have a scientific multi-threaded application, using TSQLRestServerFullMemory.  This application has high-frequency read-write (> 100) to the in-memory database every millisecond. Each read-write just involves a couple of records, but up to 8 threads may do the read-write at the same time.

It appears to me - TSQLRestServerFullMemory does NOT handle this situation well.  The scientific computation becomes very slow and degrades to the same performance as a single-threaded application.

Is there any tricks in using TSQLRestServerFullMemory in a multi-threaded context?  Or,  should I use TSQLRestServerFullMemory, at all?

Thank you.
Wux

#24 mORMot 1 » TSQLRestServerDB empty a table » 2018-07-12 21:44:06

xiwuping
Replies: 1

this may be a dumb newbie quesion. 

What is the best way to Empty all tables in the DB (TSQLRestServerDB)?

#25 mORMot 1 » W1032 Exported package threadvar 'mORMot.ServiceContext' cannot be use » 2018-07-12 19:31:40

xiwuping
Replies: 1

When rebuild mORMot as a bpl,  I had this warning:
[dcc64 Warning]  W1032 Exported package threadvar 'mORMot.ServiceContext' cannot be used outside of this package

Should ignore this?

#26 Re: mORMot 1 » TSQLRestStorageInMemory, and RestServer » 2018-07-11 12:43:10

i did, and got lost by the document. It is not beginner friendly.

the PROJECT01 used it standalone, is it the right way to have a fast in memory single table db? What you said made me more confused

#27 Re: mORMot 1 » TSQLRestStorageInMemory, and RestServer » 2018-07-11 01:39:01

but project01 in the samples uses it standalone? the above line of code was taken from the sample code directly

What is the correct way of using it?

#28 mORMot 1 » TSQLRestStorageInMemory, and RestServer » 2018-07-10 13:20:01

xiwuping
Replies: 5

TSQLRestStorageInMemory.Create(
                                                  TSQLSampleRecord,
                                                  nil,
                                                 ChangeFileExt(ExeVersion.ProgramFileName,'.db')
                                             )

The second parameter of the constructor is "nil" -  could someone help me understand - in what cases it should NOT be nil,  and in what cases it should be nil?

Thanks

#29 Re: mORMot 1 » TSynLog for a DLL » 2018-02-11 21:34:26

Thank you.  I appended the mab file to the DLL using copy /b.  But the generated new log doesn't seem to have the line number?   Any heads up?

#30 Re: mORMot 1 » TSynLog for a DLL » 2018-02-11 18:24:14

Thank you again. How can I automatically generate the mab file (as a post build event) and merge the exe/DLL with the mab file?

#31 Re: mORMot 1 » TSynLog for a DLL » 2018-02-11 16:26:01

Thank you AB.  By "append it to the .dll",  you mean putting the ".mab" file in the same folder as the DLL?  Or else?

#32 mORMot 1 » TSynLog for a DLL » 2018-02-11 15:14:08

xiwuping
Replies: 6

I am developing a DLL for a third-party Host Executable (that I have no control and no access the its C++ source code).

QUESTION 1:

I used TSynLog to generate logging for my DLL.  I also generated MAP file for the DLL.  How can I use the MAP file correctly for my DLL, so the log can show line numbers?

QUESTION 2:

The hosting executable use LoadLibrary to load my DLL.

Inside my DLL, I use LOG_VERBOSE,  which include Exception and ExceptionOS.  I also put the line ExceptionIgnore.Add(EExternalException);

However,  the hosting executable always crashes when closing with LOG_VERBOSE.  The hosting executable works fine if I remove sllException and sllExceptionOS.  One exception I caught from host executable was   
DBG_PRINTEXCEPTION_C (0x40010006)

So,  any advice,  on the CORRECT way to use TSynlog, in a DLL?

Thank you.
xiwu

Board footer

Powered by FluxBB