#1 Re: mORMot 2 » AFK for a Week » 2023-06-25 02:20:00

beautiful scenery & exquisite room  :-)    wish you a good journey

#2 Re: mORMot 2 » Apply CDS updates to rest server » 2023-06-13 01:38:58

maybe this function helps

in-cds, your clientdataset  contains modified data
in-rest  clientrest,  funciton will loop all cds delta (insert update delete),  then invoke rest-batch uri call
in-oc tableOrmclass

function YGGCDSApplyREST(cds: TClientDataSet; const rest: TRestClientURI; oc: TOrmClass):Boolean;
  prev: record
    BM: TBookMark;
    Active: Boolean;
    ReadOnly: Boolean;
    LogChanges: Boolean;
    AfterScroll: TDataSetNotifyEvent;

  change: TClientDataset;
  o: TOrm;
  m,n :integer;
  del: TID;
  IDs: TIDDynArray;
  insertBatchIdx: Array of Integer;

  insertBM: Array of TBookMark;
  batch : TRestBatch;
  UpdateCSV: RawUTF8;

  function oprops(isUpdate: Boolean): RawUTF8;
    f: integer;
    fn: String;
    v: Variant;
    u: RawUTF8;
    Result:= '';  //csv, updated fields.  eg   FP_Name,FP_Code
    for f := 0 to cds.Fields.Count-1 do
      fn:= ( cds.Fields[f].FieldName );
      u:= LowerCase( StringToUTf8(fn) );
      if SameText(UpperCase(fn), 'ID') then continue;         // skip id

      if isUpdate and (
        ) then Continue;                   //skip field if not modify

      v:= cds.Fields[f].Value;
      if (1=1) and
         (o.OrmProps.Fields.ByRawUtf8Name(u).OrmFieldType = oftUtf8Text) and
         ( (cds.Fields[f].DataType= ftString) or
           (cds.Fields[f].DataType= ftWideString) ) and
         Varisnull(v)  then

      o.SetFieldVariant(fn, v);
      AddToCsv(u, Result);
  Result:= False;
  if cds.ChangeCount =0 then Exit;

  Prev.BM:= cds.Bookmark;
  prev.AfterScroll:= cds.AfterScroll;

  change := TClientDataset.Create(nil);
  batch := TRestBatch.Create(rest.Orm, oc);
    SetLength(insertBatchIdx, 0);


    while not cds.Eof do
      case cds.UpdateStatus of
        usUnmodified: begin

        usModified: begin
            o:= oc.Create;
              o.IDValue:= cds.FieldByName('ID').AsInteger;
              UpdateCSV:= oprops(True);
              batch.Update(o, UpdateCSV);
        usInserted: begin
            o:= oc.Create;

            SetLength(insertBatchIdx , Length(insertBatchIdx)+1);
              batch.Add(o, True);

            SetLength(insertBM , Length(insertBM)+1);
            insertBM[Length(insertBM)-1]:= cds.GetBookmark;
        usDeleted: begin


    Change.Data:= cds.Delta;
    while not Change.Eof do
      case Change.UpdateStatus of
        usDeleted: begin
            del:= Change.FieldByName('ID').OldValue;

    Result:= rest.Orm.BatchSend(batch, IDs) = HTTP_SUCCESS;
    if not Result then

    if Length(insertBatchIdx) >0  then begin          // fill REST-Returned Inserted IDs
      Change.Data:= cds.Data;
      Change.LogChanges:= False;

      for m := Low(IDS) to High(IDs) do       //  [200, 200, 15,16,17, 404]
        for n:= Low(insertBatchIdx) to High(insertBatchIdx) do        // [2,3,4]
          if m= insertBatchIdx[n] then
            Change.Bookmark:= insertBM[m];

            Change.FieldByName('ID').AsInteger:= IDS[m];
      cds.Data:= Change.Data;

    if Assigned(Prev.BM) then
      cds.Bookmark:= Prev.BM;

    Result:= True;
    cds.AfterScroll:= prev.AfterScroll;



#3 Re: mORMot 2 » Away From Keyboard for 2 Weeks » 2023-05-28 14:33:25

wish you a pleasant holiday  :-)

#4 Re: mORMot 2 » Utf8ToString does not support Chinese » 2023-05-08 09:43:06

problem exists with mormot2, tested below:

D7 + mormot


D7 + mormot2


#5 Re: mORMot 2 » mORMot 2 Release Candidate » 2023-03-07 04:38:40

Congratulations!!!  thank you ab!

#8 Re: mORMot 1 » ICU extension for SQLite » 2022-07-25 13:20:41

just verify the possiblity,  my sample-exe was build with mormot static SQlite3 Engine,  place the FTS-tokenizer-extension-dll to exe folder

the sample-exe comes from : mormot2/ex/extdb-bench/PerfTestCase

  sqlite3.db_config(Client.DB.DB, SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION, 1);
  Load:= Client.DB.SQLite3Library.load_extension(Client.DB.DB, 'simple.dll', 0, Msg);
  if not (Load= SQLITE_OK) then
    raise Exception.Create('Error Message');

then some where you can create a table with new tokenizer

  Client.DB.Execute('CREATE VIRTUAL TABLE t33 USING fts5(text, tokenize = ''simple''); ');

#9 Re: mORMot 1 » ICU extension for SQLite » 2022-07-19 12:06:01

In mormot2,  add this line before Client.DB.SQLite3Library.load_extension() will work as expected.

  sqlite3.db_config(Client.DB.DB, SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION, 1);

according to sqlite3.h, the root cause is :

** ^Extension loading must be enabled using
** [sqlite3_enable_load_extension()] or
** prior to calling this API,
** otherwise an error will be returned.

and thanks to AB  for this amazing framework and lasting enthusiasm  tongue

#10 Re: mORMot 1 » ICU extension for SQLite » 2022-07-19 09:40:46

in fact I wants to load a FTS5 tokenizer extension,  which mainly deals with Chinese characters.  As Chinese differs a lot with alphabetic language (English French ...) , tokenizer is very complicated ,  so some professional tokenizer extension are indispensable.

#11 Re: mORMot 1 » ICU extension for SQLite » 2022-07-19 08:37:10

tried with mormot1 also ,  modify sources of SynDBExplorer(mormot 1 sample 12 )to load an external SQLite3.dll (3.39.0 , this dll can load extension, verified by SQLiteExpert tool ),

then excute SQL:  select load_extension('./simple.dll');   

got Exception:  "not authorized " too.

#12 Re: mORMot 1 » ICU extension for SQLite » 2022-07-19 08:10:55

ab wrote:

You need to call the sqlite3_load_extension() function.

So perhaps it is easier to define in sqlite3mc.c and recompile it:


A custom LIKE function calling the ICU comparison available in SynFPCLinux.pas may be a better alternative.

Hi ab, just tried to load a tokenizer extension,  in mormot2  \mORMot2\ex\extdb-bench\Perftestcases , add this line:

  if not Client.DB.SQLite3Library.load_extension(Client.DB.DB, 'simple', '', Msg)= SQLITE_OK then
    raise Exception.Create('Error Message');

the load_extension returns SQLITE_OK,  but Msg contains 'not authorized' , am I missed something ?

#14 Re: mORMot 1 » TSQLRestServerFullMemory Batch mode? » 2019-05-21 05:50:43

restClient.Server.DB.Synchronous := smOff;
      restClient.Server.DB.LockingMode := lmExclusive;

guess maybe helpful?

#15 Re: mORMot 1 » [suggestion] Allow specifiying a WHERE clause in TSQLRestBatch.Update » 2019-04-20 11:53:32

Hi, ab, Let's consider this situation :

Client A,Client B both wants to update the same record (eg: TSQLtask.workername ), maybe at same time.
0:00 ClientA REST-retrive table-task, id is 27
0:01 ClientB REST-retrive table-task, id is 27
0:05 ClientA REST-update table-task 27, set workername=A
0:06 CilentB REST-update table-task 27, set workername=B
now , ClientA & ClientB they all feels like gets the task, but ClientA is failed actually.

So, edwinsn's proposal is reasonable, if there is some method like :

TSQLRESTClient.Update(Value: TSQLRecord; wherefields: array of const;  wherevalues: array of const; out rowsaffected:Integer ).  

We call call this new method, then eventually mormot.pas will send SQL like: update task set workername=A where id =27 and workername=''  , and we'll check if rowsaffected =1, then everything will work as expected.

It seems now, mormot level does't care about syndb's ISQLDBStatement.UpdateCount, every REST Request (Create&Upate) the server received will turned as Insert&Update SQL , then server return HTTPOK if there is no exception, regardless how many rows affected. so I add a rowsaffected parameter above.

And IMHO almost every ORM framework takes care of the actual rows affected, it's import to developers. Syndb is so powerful, and REST C/S ORM is a unique &flexible feature, but this little function insufficient may requires developer to write a SOA method to fulfil the data integrity job. Hope this feature worth a consideration.

#16 Re: mORMot 1 » Fix SynDBOracle for work under linux » 2019-03-16 09:02:19

Hi mpv,

  Thank you for your help with the linux problems I've posted. And I've met some thread sync problems with SynDBOracle@Linux too, maybe some transaction-deadlock,  still trying to find out the root cause.

  and Did you tried with the synDBODBC , will this one be more stable?

#17 Re: mORMot 1 » potential thread-sync bug with SOA interface service » 2019-03-12 06:52:32

Thanks mpv,  your code works fine , it helps gracefully quit app.  And fpPause will just sleep mainthread but not respond to later sub-thread synchorize ,  and CheckSynchronize(10) will do the work.

#18 Re: mORMot 1 » potential thread-sync bug with SOA interface service » 2019-03-08 09:34:03

uh , my low level bug,  thank you ab. 

used Syncommons.ConsoleWaitForEnterKey , which will give mainthread chance to work on windows,  so works well on windows; but this function just Readln on linux, so mainthread is starved to end.
And is there some util-function can do this work ?

#19 mORMot 1 » potential thread-sync bug with SOA interface service » 2019-03-08 03:03:58

Replies: 6

Hi ab,   I'm trying to add a new restserver which exposes interface service running in Mainthread.  But if client call the new service, server will hang on this request till timeout.

Server side compiled with FPC(tried with: 3.2.0-beta-r41498 & 3.1.1-r39235 ) + mormot (latest code on github) running on Linux x64.

server code like this:

      aServer.ServiceRegister(TServiceRemoteSQL,[TypeInfo(IRemoteSQL)], sicPerThread).SetOptions([], []);
      bServer.ServiceRegister(TServiceRemoteTS,[TypeInfo(IRemoteTS)], sicClientDriven).SetOptions([], [optExecInMainThread,optFreeInMainThread]);
      aHTTPServer := TSQLHttpServer.Create(PORT_NAME,[aServer, bServer]);

Request URL: … 595fe9b2ca   //this request fine … 59a13f2640  //this request will hang

server log :

20190308 02331860  # srvr  	mORMot.TSQLRestServerFullMemory(00007F9CF08E4D30) User POST root/RemoteSQL.ExeBin Interface=200 out=397 B in 3.95ms
20190308 02331860  #  -    00.004.030
20190308 02331901  #  +    mORMot.TSQLRestServerFullMemory(00007F9CF08E6DF0).URI POST boot/RemoteTS._instance_?session_signature=6a48c0cf007f71595fe9b2ca in=2 B
20190308 02331901  # auth  	mORMot.TSQLRestRoutingREST(00007F9CEE8F5D30) User/1783152847
20190308 02331901  # debug 	mORMot.TSQLRestServerFullMemory(00007F9CF08E6DF0) TServiceFactoryServer.InternalInstanceRetrieve: Adding RemoteTS(00007F9CF07B2600) instance (id=1) count=1
20190308 02331901  # srvr  	mORMot.TSQLRestServerFullMemory(00007F9CF08E6DF0) User POST boot/RemoteTS._instance_ Interface=200 out=12 B in 124us
20190308 02331901  # ret   	mORMot.TSQLRestServerFullMemory(00007F9CF08E6DF0) {"result":1}
20190308 02331901  #  -    00.000.234
20190308 02331903  #  +    mORMot.TSQLRestServerFullMemory(00007F9CF08E6DF0).URI POST boot/RemoteTS.TransactionStart/1?session_signature=6a48c0cf007f7159a13f2640 in=2 B
20190308 02331903  # auth  	mORMot.TSQLRestRoutingREST(00007F9CEE8F5D30) User/1783152847
20190308 02334961  $ info  SetThreadName 00007F9CEFB17700=TSQLHttpServer 8888/root boot TSynThreadPoolSubThread
20190308 02334961  $ trace mORMot.TSQLRestServerFullMemory(00007F9CF08E4D30) BeginCurrentThread(TSynThreadPoolSubThread) root=root ThreadID=00007F9CEFB17700 ThreadCount=3
20190308 02334961  $ trace mORMot.TSQLRestServerFullMemory(00007F9CF08E6DF0) BeginCurrentThread(TSynThreadPoolSubThread) root=boot ThreadID=00007F9CEFB17700 ThreadCount=3
20190308 02334961  $  +    mORMot.TSQLRestServerFullMemory(00007F9CF08E6DF0).URI POST boot/RemoteTS._free_/1?session_signature=6a48c0cf007f71d255b6dea8 in=2 B
20190308 02334961  $ auth  	mORMot.TSQLRestRoutingREST(00007F9CED18C970) User/1783152847
20190308 02342161  % info  SetThreadName 00007F9CEFA96700=TSQLHttpServer 8888/root boot TSynThreadPoolSubThread
20190308 02342161  % trace mORMot.TSQLRestServerFullMemory(00007F9CF08E4D30) BeginCurrentThread(TSynThreadPoolSubThread) root=root ThreadID=00007F9CEFA96700 ThreadCount=4
20190308 02342161  % trace mORMot.TSQLRestServerFullMemory(00007F9CF08E6DF0) BeginCurrentThread(TSynThreadPoolSubThread) root=boot ThreadID=00007F9CEFA96700 ThreadCount=4
20190308 02342161  %  +    mORMot.TSQLRestServerFullMemory(00007F9CF08E4D30).URI GET root/Auth?UserName=User&Session=520179731&session_signature=1f015013007f724f984fdeb5 in=0 B
20190308 02342161  % auth  	mORMot.TSQLRestRoutingREST(00007F9CED124BF0) User/520179731

20190308 02405509  %  +    mORMot.TSQLRestServerFullMemory(00007F9CF08E4D30).URI GET root/timestamp in=0 B
20190308 02405509  % srvr  	mORMot.TSQLRestServerFullMemory(00007F9CF08E4D30) GET root/timestamp Method=200 out=12 B in 39us
20190308 02405509  % ret   	mORMot.TSQLRestServerFullMemory(00007F9CF08E4D30) 135502113335
20190308 02405509  %  -    00.000.133

#20 Re: mORMot 1 » Need help, how can i connect to mormot server via .net webapp with c#? » 2018-12-05 13:04:53

As @Chaa stated , your server code users Restful auth, but C# send HTTP Basic auth, so you get 403

see docs https://synopse.info/files/html/Synopse … #TITLE_535

#21 Re: mORMot 1 » SynDBOracle@FPC_Linux, Varchar2 column content is larger then expected » 2018-12-05 01:16:03


my server fetch data by ISQLDBRows.FetchAllToBinary(a Response:TRawByteStringStream as a parameter) and return Response , 
then client will create Tclientdataset uses SynDBMidasVCL.ToClientDataSet() funtion.

the difference comes from FetchAllToBinary, below is the partition about column definition in the byteString:

    .. 41 4D 45 06 0B 0A
    .. 41 4D 45 06 00 0A

the same three byte (41 4D 45) represents Fields name,  then type & field size.  we can see ZEOS lost 0B, which means length 11.

#22 Re: mORMot 1 » SynDBOracle@FPC_Linux, Varchar2 column content is larger then expected » 2018-12-04 16:06:00


for Oracle- mycolumn:  varchar2(10)

SyndbOracle will produce dataset contains
  <FIELD fieldtype="string.uni" attrname="mycolumn" WIDTH="22"/>

while SynDBZeos will produce dataset like:
  <FIELD attrname="mycolumn" fieldtype="bin.hex" SUBTYPE="Text"/>

#23 Re: mORMot 1 » SynDBOracle@FPC_Linux, Varchar2 column content is larger then expected » 2018-12-04 15:37:36

Thank you very much , ab, for so quick response. 

as you stated, modify function TSynAnsiConvert.AnsiBufferToUnicode,
  original code   result := Dest+SourceChars;
  fixed code  result := Dest+length(tmp);     

Works fine now ,  got exactly same data with D7@win32. 
And I'm curios is it depends on OS?  My ubuntu server installed Chinese locales, if it's not, will the Ansi->unicode conversion OK?

#24 Re: mORMot 1 » SynDBOracle@FPC_Linux, Varchar2 column content is larger then expected » 2018-12-04 13:34:15

No it's a very old database, all string column is varchar2. And my server code fetch data uses:  ISQLDBRows.FetchAllToBinary,  return binary to client.
the situation looks like :

oracle-column:  varchar2(10)

Delphi7@win32 got value(hex):
  E7 94 B7,  these three bytes are one character in Chinese means "man", and it's correct.

FPC@linux got value(hex):
  E7 94 B7 E7 BA A7,  the last three bytes are not expected.

thank you mpv, seems some FPC ansiconvert works failed, but not got a clue.  Any hint is helpful.

#25 mORMot 1 » SynDBOracle@FPC_Linux, Varchar2 column content is larger then expected » 2018-12-04 07:02:00

Replies: 11

SynDBOracle.pas when compiled with FPC target linux64,  some varchar2 column value are larger ( 2 widechar or 5-6 bytes ) then expected,  and the content encoding is correct (oracle-db is codepage 936, the client-got utf8 correctly) ,  but something more weired is some column-value are just as exptected. 

The server-db connection info:

20181204 05594456  # DB            SynDBOracle.TSQLDBOracleStatement(00007FD647A5C830) 1 row(s) in 290us
20181204 05594456  # info          SynDBOracle.TSQLDBOracleConnection(00007FD647AEB220) Connected to as haha with libclntsh.so rev., codepage 936 (852/ZHS16GBK)

some other tests:
1. Same code & same sql  compiled with Delphi7 running on windows is OK
2. Same code( just repacle syndboracle with zdbc) & same sql compiled with FPC running on linux 64  column-length is OK
3.  btw , the zdbc resultset seems lost field length info, so it's not an alternative to me .

#26 Re: mORMot 1 » FetchAllToBinary - Too Many Columns » 2018-09-13 08:48:59

Meet same problem here,  Syndb support fetch unlimted columns, but this FetchAllToBinary break the completeness. 
And FetchAllToJSON() will loose ColumnValueDBSize (original DBMS declared fieldsize ) information , FetchAllToBinary is the only way in my enviorment.  hope it will be corrected.

#27 Re: mORMot 1 » sicClientDriven mode Client re-connect problem » 2018-09-13 02:07:43

thanks for reply pvn0,  tried but failed. Add FClient.Services.Resolve on OnAuthentificationFailed  event, then framework will post _free_  _instances_ , but both will get 403 Forbidden. Guess may because after server restart, client will need auth agian.

#1   Get 200 OK, then server restarted.
#2   Get 403 Forbidden
#3  FClient.Services.Resolve , but Get 403 Forbidden
#4   FClient.Services.Resolve , but Get 403 Forbidden

Currently, I'm force FService:=nil and FreeAndNil(RestClient), then wait my app recreate Restclient&doInit on next-request.

#28 mORMot 1 » sicClientDriven mode Client re-connect problem » 2018-09-12 09:34:31

Replies: 2

Backend server maybe restarted without notifcation to clients.  so just after server restart,  the next-client-request contains the old-ClientDrivenID will get an resonse-error: 403 Forbidden.
My first try was on restclient.OnAuthentificationFailed event,  add some code:

   Result:= False;

but meets some curious problem, about the ClientDrivenID, seems later request totally ignore-ClientDrivenID.  And finally I lost in TServiceFactoryClient.InternalInvoke:

  if (status=HTTP_UNAUTHORIZED) and (clientDrivenID<>'') and
     (fInstanceCreation=sicClientDriven) and (aClientDrivenID<>nil) then begin
    {$ifdef WITHLOG}
    log.Log(sllClient,'% -> try to recreate ClientDrivenID',[resp],self);
    aClientDrivenID^ := 0;
    uri := baseuri;
    status := aClient.URI(uri,'POST',@resp,@head,@sent).Lo;

Here , can we have an optional recreate judgement? or some callbacks?

  if aServiceCustomAnswer=nil then begin
  end else begin
    // custom answer returned in TServiceCustomAnswer
    fRest.InternalLog('TServiceCustomAnswer(%) returned status=% len=%',
    aServiceCustomAnswer^.Status := status;
    aServiceCustomAnswer^.Header := head;
    aServiceCustomAnswer^.Content := resp;
    if aClientDrivenID<>nil then
      aClientDrivenID^ := 0;

And here,  why just let aClientDrivenID^ := 0 ?  My backend-service function returned TServiceCustomAnswer. it seems this line occurs the problem.

#29 mORMot 1 » TSynTableStatement support order by multipy fields with desc/asc. » 2018-09-12 09:10:28

Replies: 0

Original TSynTableStatement has a property: OrderByDesc:Boolean, it will remember whether last word=desc.

But think if a SQL like this: select name,price,year from cata where 1=1 order by price desc, year, name .  Then TSynTableStatement will work incorrectly, it will ignore , year,name . So I have made some changes , add a property:  OrderByDescField:TSQLFieldIndexDynArray, it will remember every SQL-Field marked with "DESC" .

Test passed with TestSQL3 on FPC&Delphi7 on windows.

I have created a P/R on github,  https://github.com/synopse/mORMot/pull/123, Please review it, thanks ab.

#30 Re: mORMot 1 » HTTP remote - WinHTTP.dll error 12002 (The operation timed out) » 2018-07-20 01:43:40

Maybe you can use fiddler (webdev proxy software ) record your client http packet, then replay the packet from fiddler,  see if it also takes 20 sec , may help you get clue (whether network problem or winhttplib or RESTClient code flaw)

#31 Re: mORMot 1 » Problem migrating THttpApiServer -> THttpServer » 2018-07-06 09:22:27

thanks a lot mpv, just have made nginx_proxy_mormot works fine on linux, and performance seems great! 
Tested with Jmeter (60 thread target 6 service-url ,and test 3 round), backend mormot is SOA-service, mainly issue sql query aginst oracle-db then return JSON to Jmeter.

client-Jmeter keepalive, Post to mormot, result is: 466KB/sec, 7QPS;
client-Jmeter keepalive, Post to nginx_proxy_mormot, result is: 13MB/sec, 190QPS;

client-Jmeter shortconnect , Post to mormot, result is: 15MB/sec, 225QPS;
client-Jmeter shortconnect , Post to nginx_proxy_mormot, result is: 14MB/sec, 207QPS;

when service on windows HTTP.sys:
client-Jmeter keepalive, Post to mormot(HTTP.sys), result is: 4MB/sec, 70QPS;

good news for nginx&mormot@linux

#32 Re: mORMot 1 » Problem migrating THttpApiServer -> THttpServer » 2018-06-19 07:43:04

@mpv Could you pls paste your nginx configuration ?  Same keep-alive performance prob occurs, and not familiar with nginx, That will be very helpful to me !

#33 Re: mORMot 1 » Interface Based Services - Called From Javascript (AJAX) » 2018-06-06 09:26:10

In order to stick SOA-method thread enviorment within TSynThreadPool   , trying modify THREADPOOL_MAXWORKTHREADS = 0,

then test with Jmeter (100 client-thread concurrent ) , totaly 1000 http-request, result in about 30% broken connection ( Non HTTP response message: Software caused connection abort: recv failed ).

maybe the problem occurs at  function TSynThreadPool.Push(aContext: pointer): boolean;
  {$ifdef USE_WINIOCP}
  result := PostQueuedCompletionStatus(fRequestQueue,0,0,aContext);

and now the syncrtsock-IOCP is not fullfeature,   https://synopse.info/forum/viewtopic.php?id=4049.

It seems, HTTPAPI @windows + SOA sicPerthread is more proper.

#34 Re: mORMot 1 » Interface Based Services - Called From Javascript (AJAX) » 2018-06-06 07:57:12

HTTPAPI mode works fine,  32 thread serves all 100-Jmeter clients, DB connection is reuesed by server thread.

The prev problem seems occurs with THttpServer (socket based ), the thread pool only serve reqest, but further calc&rest logic is dispatched with dedicate THttpServerResp thread everytime.
So when THttpServer working with sicPerThread mode SOA-service,  the SOA-interface instance lifetime is so short.

  /// a simple Thread Pool, used for fast handling HTTP requests of a THttpServer
  // - will handle multi-connection with less overhead than creating a thread
  // for each incoming request
  // - will create a THttpServerResp response thread, if the incoming request is
  // identified as HTTP/1.1 keep alive, or HTTP body length is bigger than 1 MB
  TSynThreadPoolTHttpServer = class(TSynThreadPool)

    // here aContext is a pointer(TSocket=THandle) value
    procedure Task(aCaller: TSynThread; aContext: Pointer); override;
      if (fServer.ServerKeepAliveTimeOut>0) and
         (fServer.fInternalHttpServerRespList.Count<THREADPOOL_MAXWORKTHREADS) and
         (ServerSock.KeepAliveClient or
          (ServerSock.ContentLength>THREADPOOL_BIGBODYSIZE)) then begin
        // HTTP/1.1 Keep Alive (including WebSockets) or posted data > 1 MB
        // -> process in dedicated background thread

and my question is why THttpServer internally desinged like this ?

#35 Re: mORMot 1 » Interface Based Services - Called From Javascript (AJAX) » 2018-05-24 04:22:16

Hi ab, Some sicPerThread problems

code based on Sample-16, I made little modification on sicPerThread, the result is not expected.

{ TServiceRemoteSQL }
procedure TServiceRemoteSQL.Connect(aEngine: TRemoteSQLEngine;
  const aServerName, aDatabaseName, aUserID, aPassWord: RawUTF8);
  if fProps<>nil then
    raise Exception.Create('Connect called more than once');
  if TYPES[aEngine]=nil then
    raise Exception.CreateFmt('aEngine=%s is not supported',
  fProps := TYPES[aEngine].Create(aServerName,aDatabaseName,aUserID,aPassWord);

function TServiceRemoteSQL.Execute(const aSQL: RawUTF8; aExpectResults, aExpanded: Boolean): RawJSON;
var res: ISQLDBRows;
  if fProps=nil then
    Connect(rseOracle, '', '', 'ydjg508_mid', 'pde');
  res := fProps.ExecuteInlined(aSQL,aExpectResults);
  if res=nil then
    result := '' else
    result := res.FetchAllAsJSON(aExpanded);

destructor TServiceRemoteSQL.Destroy;

    aServer := TSQLRestServerFullMemory.Create(aModel,false);
      aHTTPServer := TSQLHttpServer.Create(PORT_NAME,[aServer],'+',useHttpSocket); 

The server log displayed by Synlogview image:


The server log :

20180522 09343026 "O trace mORMot.TSQLRestServerFullMemory(020F29A0) BeginCurrentThread(THttpServerResp) root=root ThreadID=00001D40 ThreadCount=126
20180522 09343908 "O trace mORMot.TSQLRestServerFullMemory(020F29A0) EndCurrentThread(THttpServerResp) ThreadID=00001D40 ThreadCount=102

20180522 09345313 !$ trace mORMot.TSQLRestServerFullMemory(020F29A0) BeginCurrentThread(THttpServerResp) root=root ThreadID=00001D40 ThreadCount=111
20180522 09345650 !$ trace mORMot.TSQLRestServerFullMemory(020F29A0) EndCurrentThread(THttpServerResp) ThreadID=00001D40 ThreadCount=111

Client is apache JMeter, with 100 thread simultaneously, each thread will send 5 HTTP request body (JSON encoded SQL expected by the RemoteSQL.Execute URL).

It seems like RESTServer randomly pick one thread  then create TServiceRemoteSQL and process DB logics then release it.
With my expectation, the second picture #68 thread with threadID=00001D40 should run TServiceRemoteSQL.Execute and the fProps should be assigned .
But it seems the TServiceRemoteSQL instance is destroyed by previous #175 thread.

#36 Re: mORMot 1 » Is there some convenient funcs to deal with Transfer-Encoding: chunked » 2018-04-28 02:49:33

Do some research again, there are some misleading in my answer.  ab you are right, The framework has no strip chunked logic.  But the winhttp has. 

https://stackoverflow.com/questions/251 … tpreaddata
https://msdn.microsoft.com/en-us/librar … 2147217396
Starting in Windows Vista and Windows Server 2008, WinHttp enables applications to perform chunked transfer encoding on data sent to the server. When the Transfer-Encoding header is present on the WinHttp response, WinHttpReadData strips the chunking information before giving the data to the application.

API-WinHttpReadData() auto strip the chunked-response, then send the data to application. Only when we wants to send chunked request, some manually code is needed before we call WinHttpSendRequest().

So my last correct output is a coincidence, winhttp does it. then CompressGZip() can decompress correctly.

#37 Re: mORMot 1 » Is there some convenient funcs to deal with Transfer-Encoding: chunked » 2018-04-27 02:58:33

Got it , SynCommons.IsContentCompressed   SynZip.CompressGZip can do it.

#38 mORMot 1 » Is there some convenient funcs to deal with Transfer-Encoding: chunked » 2018-04-26 15:16:15

Replies: 3

use THttpClientSocket / TWinhttp to grab some html-content, but response is chunked.
searched in syncommons.pas & syncrtsock.pas without any clue, any hint? thank you !

#39 Re: mORMot 1 » Default Winhttp proxy modification request » 2017-08-17 00:43:15

thank you ab, for so quick response and continuous passion on mormot !

#41 mORMot 1 » Default Winhttp proxy modification request » 2017-08-16 03:34:46

Replies: 6

Fiddler is a wonderful debug tools which could catch every winhttp/winitnet call , and when it startsup , it could set winhttp proxy automatically.  so it is very convenient on momort c/s communication debugging.

but now in the SynCrtSock.pas , it will bypass the system proxy

procedure TWinHTTP.InternalConnect(ConnectionTimeOut,SendTimeout,ReceiveTimeout: DWORD);
  if fProxyName='' then
    // add https://msdn.microsoft.com/en-us/library/windows/desktop/aa384122 ?
  fSession := WinHttpOpen(pointer(Ansi7ToUnicode(fUserAgent)), OpenType,
    pointer(Ansi7ToUnicode(fProxyName)), pointer(Ansi7ToUnicode(fProxyByPass)), 0);

and MSDN ( https://msdn.microsoft.com/en-us/librar … s.85).aspx ) told us :  WINHTTP_ACCESS_TYPE_DEFAULT_PROXY -- this option is deprecated on Windows 8.1 and newer. Use WINHTTP_ACCESS_TYPE_AUTOMATIC_PROXY instead.

so I wonder if it is possible to change the implemention here to : if client >8.1 then use WINHTTP_ACCESS_TYPE_AUTOMATIC_PROXY .  it could help coding&debuging a lot.

#43 Re: mORMot 1 » Clear separation of components » 2017-07-05 14:15:51

mORMot framework is definitely productive framework and highly optimized  ,to build an enterprise software with compiling programming tools (C C++ Pascal Java .net ) , maybe mORMot is the best. But somehow, it is not friendly to newbie (lacks of production-demo; SAD is comprehensive but feels like a well-builded churh, grandness but scaffold is disassembled, we can't figure out how it was builded).
eg: the mormot.pas is so complicatd , there are cornerstone class(TSQLRest, TSQLRecord),and also lots of aided classes(TSQLRestServerMonitor, TSQLRecordServiceLog).  from my personal perspective, make some separation between the core and the assistant maybe seems like more reader-friendly and more maintainable.

#44 Re: mORMot 1 » Clear separation of components » 2017-07-05 02:06:40

Good idea !  Like apple reinvent phone, mORMot is a great framework, we can say ab reinvent Delphi :-)

As ab said, The deep integration of the mORMot framework makes it unusual and it seems unbelievable. The Java Spring project divied into sring MVC, spring Cloud, spring Data, spring Security ... and each of them can be used alone. Well refactoring will make mORMot more developer friendly and more persuasively.

maybe such subprojects could be introduced:
mORMot RTL - rawutf8 dynarray docvariant ... from SynCommons.pas
mORMot JSON-  fast Json enc-dec from SynCommons.pas
mORMot REST- core C/S ORM, TSQLRest client server storage .. from mormot.pas
mORMot SOA- interface based service from mormot.pas
mORMot SQL- remote db ... from syndb.pas
mORMot noSQL- mongo db 
mORMot NET- core http-communication, http.sys, iocp epoll ...
mORMot security- AES HMAC SHA JWT... from SynCrypto.pas
mORMot C- sqlite3 openssl ecc .. some C project, like FPC said write once compile anywhere, C&Pascal are best friend.
mORMot Report - pdf based report generation
mORMot MVC- mustache based MVC

#45 Re: mORMot 1 » SOA service-contract on linux differs with on windows » 2017-06-28 12:23:53

fContract JSON content on windows:


fContract JSON content on linux:


Original interface definition :

    /// returns the thread ID running the method on server side
    function GetCurrentThreadID: TThreadID;

which on linux64 produce : {"argument":"Result","direction":"out","type":"int64"}
but on win32 produce: {"argument":"Result","direction":"out","type":"cardinal"}
also on win64 produce : {"argument":"Result","direction":"out","type":"int64"}

So, strictly speaking, it's not a bug ,just an implemention flaw. 
Thank you, ab, for your so quick replying .

#46 Re: mORMot 1 » SOA service-contract on linux differs with on windows » 2017-06-28 12:19:46

Tested with :

FPC svn 52403 on Windows
  target windows ,return A9C202E36B8F93AB
  crosscompile to linux64 then running on ubuntu, return 0063F5DAFC95F854

FPC svn 55405 on Ubuntu
  retrun  0063F5DAFC95F854

#47 mORMot 1 » SOA service-contract on linux differs with on windows » 2017-06-28 09:38:02

Replies: 4

Same service code, both compiled with FPC, but service-contract differs . the client expected contract is the version on windows.

Linux-retruns {"result":["0063F5DAFC95F854"]}

windows-retruns {"result":["A9C202E36B8F93AB"]}

#49 mORMot 1 » Question about TSQLHttpClient.ServicePublishOwnInterfaces » 2017-06-26 14:05:04

Replies: 2

I'm tring to merge TTestServiceOrientedArchitecture.TestOverHttp implemention into Sample21-Multithread stress test tool,  but stucked on a weired statement:

in the TTestServiceOrientedArchitecture.TestOverHTTP  procedure :

Everything is OK with original TestSQL3, but if I delete this line, then the later running HttpClient.ServiceRetrieveAssociated will get nothing.

Here, send fClient.Server (a TSQLRestServer instance) to httpclient,  can't figure out why it was designed like this. In most situation, client-side code will know nothing about server-implemention.  My version of Sample21, client code can't get fClient.Server instance, so I have to delete the line,  then stucked.  Am I missed something?

#50 Re: mORMot 1 » Clone database using mORMot » 2017-04-20 09:34:04

Maybe there are too much index on your sqlite table ?  7GB table may counts 2~3 billion rows , update is not easy on any RDBMS. I'm curious whether your MSSQL faster now ?  @turrican

Board footer

Powered by FluxBB