#1 Re: mORMot 2 » How To Make Special Query faster » 2026-01-02 19:03:33

I testet with mysql, here i add 33000 ID in one insert Statement and it does it in 0.139 seconds. (Create Table, Insert IDs, Create Index) (Batchsize set to 100000)

#2 Re: mORMot 2 » How To Make Special Query faster » 2026-01-01 12:08:48

I post my little code here, may be you see something  smile
Some Testcode is commented out

procedure TDFSynDBStringBuilder.CreateTempTable(const AIDs: TIDDynArray; var ATablename: String);
begin
  TSynLog.Enter('Create Temp Table for % IDs', [System.Length(AIDs)], Self);
  if ATablename.IsEmpty then
    ATablename := 'tmp_ids_' + StringReplace(TGUID.NewGuid.ToString, '{', '', []).Replace('}', '').Replace('-', '');

  var LConnection := TSqlDBUniDACConnection(FConnectionProperties.MainConnection);
  var LInTransaction := LConnection.InTransaction;
  if not LInTransaction then begin
    if not LConnection.Connected then
      LConnection.Connect;
    LConnection.StartTransaction;
  end;
  try
    var LScript := TUniScript.Create(nil);
    var LSQLStream := TStringStream.Create;
    LScript.Connection := LConnection.Database;
    try
      var LBatchSize := 1000;
      case DBMS of
        dMySQL : begin
          LBatchSize := 10000;  // Teste
          SimpleQuery('create temporary table '+ATableName+' (TempID bigint primary key)'); // .ExecuteInlined(false);
          LSQLStream.WriteString(ToString);
        end;
        dMSSQL : begin
          ATablename := '#'+ATablename;
          LBatchSize := 1000;
          SimpleQuery('create table '+ATableName+' (TempID bigint primary key)'); // .ExecuteInlined(false);
          LSQLStream.WriteString(ToString);
        end;
      end;

    // Benutze UniDac Batch Insert Methode ! (Leider auch maximal 1000 / Execute !)
  //    var Q := TUniQuery.Create(nil);
  //    try
  //      Q.Connection := LConnection.Database;
  //      Q.SQL.Text := 'INSERT INTO '+ATablename+' (TempID) VALUES (:id)';
  //      Q.ParamByName('id').DataType := ftLargeInt;
  //      Q.Params.ValueCount := System.Length(AIDs);
  //
  //      for var i := 0 to High(AIDs) do
  //        Q.Params[0][i].AsLargeInt := AIDs[i];
  //
  //      Q.Execute(System.Length(AIDs));
  //    finally
  //      Q.Free;
  //    end;
  // Maximal 1000 Werte pro Zeile !
      var i := 0;
      while i < System.Length(AIDs) do begin
        SimpleQuery('insert into '+ATableName+' (TempID) values ');
        for var j := 0 to LBatchSize - 1 do begin
          if i >= System.Length(AIDs) then
            Break;
          Append('(').Append(AIDs[i]).Append('),');
          Inc(i);
        end;
        // letztes Komma entfernen
        Length := Length - 1;
        LSQLStream.WriteString(ToString);
//        ExecuteInlined(false);
       end;
  //    case DBMS of
  //      dMySQL : SimpleQuery('CREATE UNIQUE CLUSTERED INDEX idx_tempid ON '+ATableName+' (TempID)').ExecuteInlined(false);
  //      dMSSQL : SimpleQuery('CREATE UNIQUE INDEX idx_tempid ON '+ATableName+' (TempID)').ExecuteInlined(false);
  //    end;

      LScript.ExecuteStream(LSQLStream);
      if not LInTransaction then
        LConnection.Commit;
    finally
      FreeAndNil(LSQLStream);
      FreeAndNil(LScript);
    end;
  except
    if not LInTransaction then
      LConnection.Rollback;
    raise;
  end;
end;

#3 Re: mORMot 2 » How To Make Special Query faster » 2026-01-01 12:02:14

The Version with Index after Inserts i tried. but no real difference

Array Binding with UniDac i also tried, but it creates 50 Insert Statements with 1000 IDs as i did - also no difference.

#4 mORMot 2 » How To Make Special Query faster » 2025-12-31 08:54:12

itSDS
Replies: 6

Hi Arnaud, using ConnectionProperties with UniDac i have the following Problem:

I Have to Create a Temp Table on my SQL Server (MS-SQL or MySQL)
The Table has only 1 Field TempID BIGINT (für int64 ID)

Now i have to insert 1 to 50K or 100K ID depends on the case into to Table. (It is used in other Queries to replace IN ... Queries with an INNER JOIN)

In Principle i have to execute a SQL Script

Begin Transaction
create table #tmp_ids_1F2575D95F4141A3B67B5051843D65F6 (TempID bigint primary key)
insert into #tmp_ids_1F2575D95F4141A3B67B5051843D65F6 (TempID) values (412953),(456388),(494710),(503522),(545873), ...
insert into #tmp_ids_1F2575D95F4141A3B67B5051843D65F6 (TempID) values (546308),(548805),(555293),(555830),(598162) ...
Commit

I tried now different ways but all take too long (About 1second for 50K ID's)

1. i tried to ExecuteInlined 50 Lines of code
2. i tried TUniScript Component

i found out: If i use Uniscript and repeat the same Insert Script 2 or more Times the Execution time goes down from 1000ms to 100-200 ms

ChatGPT Means that it could be done in under 50ms

what do you think ? (Or may be any other has a good idea)

#5 Re: mORMot 2 » mormot throwing Exception ESChannel {Message:"recv: Handshake aborted" » 2025-12-13 11:22:11

Ok Arnaud, i found the Error on my side - my Client made a test connection which results in the displayed error ! i removed it.

#6 Re: mORMot 2 » mormot throwing Exception ESChannel {Message:"recv: Handshake aborted" » 2025-12-12 08:59:14

Ok back again, i testet now Win64 Server with OPENSSL and libcrypt 3.1
There i get this Error:

20251212 09523554  " EXC   EOpenSslNetTls {LastError:1,OpenSsl:"30100030",Message:"TOpenSslNetTls.AfterAccept accept: OpenSSL 30100030 error 1 [SSL_ERROR_SSL (error:0A000126:SSL routines::unexpected eof while reading)]"} [R0:IMSYS ThumbnailCache BinaryD] at 0112fd97 mormot.lib.openssl11.pas EOpenSsl.CheckFailed (2879)  {8 3.51 3.70 1 20GB/63.9GB 23f00f01}

I testet both client (Win64) with OpenSSL and without OpenSSl, Error is only on Server, and Client works as expected.

#7 Re: mORMot 2 » mormot throwing Exception ESChannel {Message:"recv: Handshake aborted" » 2025-12-12 08:43:23

SRY i have to test again, i have to define FORCE_OPENSSL in my dproj

#8 Re: mORMot 2 » mormot throwing Exception ESChannel {Message:"recv: Handshake aborted" » 2025-12-12 08:37:12

Just to say, the Server is running and full responsive, im only wondering if this error is an error or something like a warning ?!

#9 Re: mORMot 2 » mormot throwing Exception ESChannel {Message:"recv: Handshake aborted" » 2025-12-12 08:36:02

Ok with 64 Bit Windows the Error is a little bit other displayed, but from the content the same

20251212 09343114  % EXC   ESChannel {Message:"recv: Handshake aborted"} [R3:IMSYS ThumbnailCache BinaryD] at 01091990 mormot.net.sock.windows.inc TSChannelNetTls.FreeAndCheckSocket (2057)  {8 3.47 3.68 1 19.4GB/63.9GB 23f00f01}

#10 Re: mORMot 2 » mormot throwing Exception ESChannel {Message:"recv: Handshake aborted" » 2025-12-12 08:33:07

ab wrote:

Do you compile as a Win64 executable?

It may be due to PRISM itself, especially to the fact that this emulator does not like multiple threads, so it becomes unstable on WinArm.
With PRISM, we had to reduce the number of threads to let mormot2tests pass.
It is as known and documented.

No it was a 32Bit Debug executable. I try with 64 Bit

#11 Re: mORMot 2 » mormot throwing Exception ESChannel {Message:"recv: Handshake aborted" » 2025-12-12 08:32:03

I testet it with OpenSSL (USE_OPENSSL defined) same error, is there anything more i have to do to activate openssl ?

#13 mORMot 2 » mormot throwing Exception ESChannel {Message:"recv: Handshake aborted" » 2025-12-11 19:09:54

itSDS
Replies: 11

Hello Arnaud, we are also using m2 with self-signed certificates in our test environment.

In mormot.net.sock.windows.inc there is a place where you raise: ESChannel {Message: "recv: Handshake aborted"}
when res = 0
in FreeAndCheckSocket.

Is it necessary for this very frequent exception to appear as an error in the log?
It’s a bit confusing. Or is it actually an error and we need to do something differently here?

Also like to ask if there is an option to shorten the Exception:

20251211 19560724  # EXC   ESChannel {Message:"recv: Handshake aborted"} [R1:IMSYS ThumbnailCache BinaryD] at 0116540e LiteStaticUni.pas sqlite3win64_xe5.sqlite3BtreeOpen (1439)  System.Classes.pas .{System.Generics.Collections}TList<System.Classes.TPersistentClass> (3555) System.Classes.pas .{System.Generics.Collections}TList<System.Classes.TPersistentClass> (3555) System.pas ReductionPi2n (10160) LiteStaticUni.pas sqlite3win64_xe5.sqlite3BtreeOpen (1439) LiteStaticUni.pas sqlite3win64_xe5.sqlite3PagerClose (1439) LiteStaticUni.pas sqlite3win64_xe5.sqlite3PagerSetPagesize (1439) LiteStaticUni.pas sqlite3win64_xe5.sqlite3Select (1439) LiteStaticUni.pas sqlite3win64_xe5.sqlite3CreateIndex (1439) LiteStaticUni.pas sqlite3win64_xe5.sqlite3CreateIndex (1439) LiteStaticUni.pas sqlite3win64_xe5.sqlite3CreateForeignKey (1439) .{System.Generics.Collections}TList<DFDatenbankenStruktur.TDFESIndex> System.Generics.Collections.pas TArray.BinarySearch<System.Pointer> (1389) System.pas Ln (11082)  {8 3.36 2.74 18.5GB/63.9GB 23f00f01}

without the call Stack ?

#15 mORMot 2 » SQLite Version - Question » 2025-11-04 09:35:41

itSDS
Replies: 1

Hi Arnaud, today i looked at SQLite.org and saw that there is 3.50.4
Normally mormot was in sync with latest SQLite Releases.
Is there a reason, why mormot stays at 3.46.1 ?

(its just a question)

#16 mORMot 2 » Feature Request Logging » 2025-09-26 18:46:13

itSDS
Replies: 3

Hi Arnaud, i have a question / Feature for Logging.

Is it possible to Disable Logging for 1 Thread after calling function like DisableLog
Reason: I have recurring Function called by a Timer which do nothing if Query gives no result, but flod the log with "useless" stuff if they find something by Query i will Enable Log.

here a sample

function DoOnTimer;
begin
  TSynLog.DisableLog;
  if ....Orm.RetrieveListObjArray(list, TOrmQueue, 'Status=?', [ord(TOrmStatus.New)], '*') then begin
    TSynLog.EnableLog; // Enables Log
// Do something with logging enabled
    TSynLog.Enter('Data Found');
   
  end;
   
// Calls EnableLog on exit (Like Enter/Leave) if Log enabled does nothing :)
end;

#18 Re: mORMot 2 » New Feature (BUG) with Rad Studio 13 » 2025-09-19 08:59:09

there are only 2 Places where i got the Error

mormot.core.variants.pas - 9400
and the above unit

#19 mORMot 2 » New Feature (BUG) with Rad Studio 13 » 2025-09-19 08:57:14

itSDS
Replies: 4

Hi Arnaud,

compiling with new 32Bit Compiler for Release Mode, we get the following Error:

[dcc32 Fataler Fehler] mormot.core.interfaces.pas(8256): F2084 Interner Fehler: C2802

The Reason is:

    TDocVariant.NewFast(result);

With this WorkAround it works:

    var LResult : variant;
    TDocVariant.NewFast(LResult);
    result := LResult;

#20 Re: mORMot 2 » Regression with RootRedirectToURI » 2025-09-02 07:29:52

In my case i have 6 Server registered in HttpServer and using BoilerplateServer derived from resthttpserver
May be the error was in default options of boilerplate redirecting to index.html...
i changed this 2 things.

#21 Re: mORMot 2 » Regression with RootRedirectToURI » 2025-09-02 07:05:58

Thank you I found the solution: I had to to set AServer.RootRedirectGet which wasn’t required before.

#22 mORMot 2 » Regression with RootRedirectToURI » 2025-09-01 13:14:10

itSDS
Replies: 5

Hi Arnaud,

did you change something with this feature ? atm our base url is not redirected
We use RootRedirectToURI as before ?

best regards

#23 Re: mORMot 2 » Question concerning JSON generation DateTime -> Null » 2025-08-29 07:37:03

One Question where can i set this Option globally ?

#24 Re: mORMot 2 » Question concerning JSON generation DateTime -> Null » 2025-08-29 07:21:30

Looks good, TYVM

one thing: in mormot.ui.pdf the unit   mormot.crypt.other, is missing !

#25 mORMot 2 » Question concerning JSON generation DateTime -> Null » 2025-08-28 12:16:16

itSDS
Replies: 4

Hi Arnaud, since 3Month or so JSON Output write null for TDateTime if Date is 0
Before it wrotes a empty String "" in JSON Output.

Is it possible to enable old Feature OnDemand ?
Because we have old SynCrossplatform Clients throwing exception implicit converting Variant Null to String

For new Clients i added this to SynCrossplatformJSON:

function Iso8601ToDateTime(const Value: Variant): TDateTime; overload;
function Iso8601ToDateTime(const Value: string): TDateTime; overload;

#26 Re: mORMot 2 » Serve multiple hostnames on one IP with separate mORMot binaries » 2025-08-21 11:44:02

there is a solution for iis we use, Install URL Rewrite 2.0 or higher and ARR (Application Request Routing Cache) Work perfekt !

#27 Re: mORMot 2 » Question getting Blob Data RawBlob » 2025-08-14 14:31:05

atm i us a SOA call which gives blob for guid to solve problem

#28 Re: mORMot 2 » Question getting Blob Data RawBlob » 2025-08-14 13:59:59

Yes i can:
Object looks like this:

  TBild = class(TDFDomBase)
  private
    fBildID : TBildBildID;
    fGUID : TBildGUID;
    fBinaerdaten : RawBlob;
  public
    procedure SetBinaerdaten(const ABinaerdaten : RawBlob);
  published
    property BildID: TBildBildID read fBildID;
    property GUID: TBildGUID read fGUID;
    property Binaerdaten : RawBlob read fBinaerdaten;
    property Datenformat : RawUtf8 read fDatenformat;
  end;

SOA Function like this:

function FillBildStreamFromBDS(var ABild : TBild) : TresultRec; 
begin
  var LBinaerdaten : RawBlob;
  Result := GetBildBinaerdaten(ABild.GUID, LBinaerdaten);
  if Result.Erfolgreich then
    ABild.SetBinaerdaten(LBinaerdaten);
end;

In Log Binaerdaten is null but was filled with 4MB PNG before

#29 mORMot 2 » Question getting Blob Data RawBlob » 2025-08-14 09:11:37

itSDS
Replies: 4

Hi Arnaud, i like to get an Object containing a RawBlob Field by SOA Call.
Blob is filled on Server side, but not sent to client, in JSON Blob:null is postet.

Is there an Option to send the Blob Data here ?

#30 Re: mORMot 2 » Cookie not found in new TMvcSessionWithRestServer » 2025-08-14 06:24:21

Thanks, the session cookies are working again now.

#31 mORMot 2 » Cookie not found in new TMvcSessionWithRestServer » 2025-08-07 09:34:48

itSDS
Replies: 2

Hi Arnaud,

after your mvc rework, the Cookies do not work anymore.

In Former Version you set OutCookie and InCookie to the Cookie.
In the New Version only outcookie is set, but InCookieSearch does not find it.

We derived out Application from new TMVCApplicationRest

may be you can solve it.

#32 mORMot 2 » Managed Records not initialized by "CallMethod" » 2025-08-06 09:09:05

itSDS
Replies: 1

Hi Arnaud, im using a managed Record as Result of a SOA function.

result = packed record
  
public
    class operator Initialize(out Dest : result);
end;

// SOA Function:

function Test : result;

if Test is called directly result.Initialize is called
if it is called remote (by mormot.core.interface.CallMethod) result.Initialize is not called and record uninitialized

is it possible to call it ?

#33 mORMot 2 » GetCurrenThreadID in mormot.core.base undefined » 2025-07-30 10:14:47

itSDS
Replies: 1

Hi Arnaud,

added uses mormot.core.os to mormot.core.base to prevent compiler Error

#34 Re: mORMot 2 » RS12.3 64Bit ComputeRoutes takes 99s in DEBUG Mode » 2025-07-21 10:31:07

itSDS wrote:

Hi Arnaud, via ChatGPT i found this and it seems to help:

In Registry:
HKEY_CURRENT_USER\Software\Embarcadero\BDS\22.0\Debugging

Add EnableWaitChainDetection as DWORD with value 0

it disables the Thread Wait Chain Feature from Delphi.

Sry today long wait times, did not fix the problem - I investigate further

#35 Re: mORMot 2 » RS12.3 64Bit ComputeRoutes takes 99s in DEBUG Mode » 2025-07-18 09:37:49

Hi Arnaud, via ChatGPT i found this and it seems to help:

In Registry:
HKEY_CURRENT_USER\Software\Embarcadero\BDS\22.0\Debugging

Add EnableWaitChainDetection as DWORD with value 0

it disables the Thread Wait Chain Feature from Delphi.

#36 Re: mORMot 2 » RS12.3 64Bit ComputeRoutes takes 99s in DEBUG Mode » 2025-07-17 17:23:18

Without it runs as the 32Bit Version
Yes there must something the Debugger does what the exe dont like, but i dont know what

#37 Re: mORMot 2 » RS12.3 64Bit ComputeRoutes takes 99s in DEBUG Mode » 2025-07-17 10:44:27

After a long time i had to debug my m2 service with 64Bit again today.

and there is a delay of 55s at the moment. Starting Service in RS 12.3 64Bit Debugger

https://imgtr.ee/image/image.SE1M

Using the 32Bit exe with RS 12.3 Debugger it only takes 1s at the same function

Int the Picture there is a gap in the log at 12:14:12.528 -> 12.15.02.944 of 50s I don't know what the programm does at this point ???

#38 Re: mORMot 2 » regression in logging » 2025-07-14 09:28:38

Ty arnaud, i'll try it
i made this serious observation running a lengthy prozess:

https://imgtr.ee/image/image.SDgW

at 23:00 many small log files are created

    TSynLog.Family.RotateFileDailyAtHour := ServiceManager.EinstellungenManager.ReadInteger(cRotateFileDailyAtHour, 23);

may be it depends ot the rotatefiledailyathour, but i think that the log shoult not create that many files, because older logs are overwritten (lost) then.

#39 Re: mORMot 2 » Problem senden Enum - Value » 2025-07-08 08:23:42

Hi i found the Error

My Enum is like this:
TEinheitEnum = (
first,
second,
third,
fourth
);

in class constructor i set member fEinheit to "third" (Because i like it initialized with third value) The Enum is kind of sorted smile
Now in client its set to "first"

if its set to first, the value is missing in JSON - Data send to server (as you said its optimized)
Because of this the expected Value is not sent to server.

As Solution i put my desired default value in top of enum
enum is now : (third, first, second, fourth)
It does not look good, but it works now.

#40 Re: mORMot 2 » Problem senden Enum - Value » 2025-07-07 19:47:40

Hm what previous Value do you mean. I only change 1 Value in Object then put whole Objekt to soa call ?
All members are transfered except the enum one.

On Server Site a wrong value is in that enum field.

#41 Re: mORMot 2 » regression in logging » 2025-07-07 14:26:59

Hi Arnaud,

i have the same problem in
TRestServerUriContext.InternalExecuteSoaByInterface

Here you call DoLog and DoLog cuts the the received string, is it possible to add something like LogResponseMaxBytes here ?
Else i have to look at client Log what's send and this is not possible in many times.

#42 Re: mORMot 2 » Problem senden Enum - Value » 2025-07-07 14:12:37

it is not a generell problem, sending other Object containing enum value works as expected...

#43 mORMot 2 » Problem senden Enum - Value » 2025-07-07 14:08:02

itSDS
Replies: 6

Hi Arnaud, im sending objects with soa call to server.
This Object contains a enum

published
property Einheit : TEinheitEnum read fEinheit

the Value Einheit is not contained in the JSON send to m2 Server, Einheit on Server is uninitialized
i checked ctxt on client an not Entry for Einheit.

What can i do ?

#44 Re: mORMot 2 » mORMot 2 support Delphi Mobile compiler ? » 2025-06-27 09:12:51

ab wrote:
itSDS wrote:

i think no one will make a server on mobile platform.

Our "peer cache" feature relies on local UDP/TCP servers, and could benefit on client side too, but I don't know how it is possible on the mobile, especially about firewall port opening.
smile
I would think about a "pure client" peercache feature, without any UDP/TCP server.

I used http server on mobile a time ago and did not have to open any port but may this changed meanwhile

#45 Re: mORMot 2 » mORMot 2 support Delphi Mobile compiler ? » 2025-06-25 07:17:38

I too asked this question long time ago wink and actually using SynCrossPlatform Client with some Patches. It works generally great.

The Problem is that you have to generate special Client.pas Unit from Server. And it would be great if this step can be removed.
for the iOS/Android it is not neccessary to port the whole m2. only the client stuff and rtti because i think no one will make a server on mobile platform.

#46 Re: mORMot 2 » regression in logging » 2025-06-24 19:11:23

Ty Arnaud it solves my problem

#47 Re: mORMot 2 » regression in logging » 2025-06-10 11:50:40

Hi Arnaud, ty for your patch.

Today we wanted to see the Result of a SOA Call in Log. In former Versions there was the complete JSON Array.
In the new m2 there is only 1 line saying Response with 1.7MB

Is it possible to add the complete JSON to Log again. We need it to debug problems. Log Size and Performance does not matter in our case ! More important is to log the Information.
A switch to enable / Disable Big Data would be ok too.

#48 Re: mORMot 2 » regression in logging » 2025-05-16 08:41:46

As Workaround i changed:

procedure TRest.InternalLogResponse(const aContent: RawByteString;
  const aContext: shortstring; Level: TSynLogLevel);
begin // caller checked that self<>nil and sllServiceReturn in fLogLevel
//  fLogFamily.Add.LogEscape(Level, '%', [aContext], pointer(aContent), length(aContent), self);
  fLogFamily.Add.Log(Level, aContent, self, MAX_SIZE_RESPONSE_LOG);
end;

#49 mORMot 2 » regression in logging » 2025-05-16 08:27:39

itSDS
Replies: 9

Hi Arnaud,

today i saw that some SOA Parameters i formerly could read are shortened now.
There was a const MAX_SIZE_RESPONSE_LOG which i changed to a larger Value

now this const is not used any more and at the places it was formerly used InternalLogResponse is called!
InterlogResponse uses LogEscape which uses a ShortString now an cuts all after 200 Char.

This is not very useful.

#50 Re: mORMot 2 » Simple Question for new DoLog/Enter » 2025-05-15 09:19:17

TY Daniel, this makes sense if you like to add something else to the Log.
My Question was concerning the Use of the Cast to PUtf8Char

Board footer

Powered by FluxBB