mORMot Open Source

WladiD

Member

From: Germany

Registered: 2010-10-27

Posts: 40

Performance of Mormot Collections vs. Spring v2 Collections

Hi ab,

we have done some benchmarkings here and I was surprised, that the collections of Spring v2 was at almost every aspect faster than the collections of mormot v2. So I want to share the benchmark code here, so you can make your mormot even faster. We used Delphi 11.3.

Here the code:

program SpringVsMormotCollections;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils,
  System.Diagnostics,

  mormot.core.collections,

  Spring,
  Spring.Collections;

{ ----------------------------------------------------------------------- }

type

  TBenchmarks = class
   private
    function CreateTestListString(ACount: Integer; const APrefix: String): TArray<String>;
    procedure RunBenchmark(const AProc: TProc; ABenchmarkRounds: Integer);
    procedure ListBenchmark;
    procedure DictBenchmark;
   public
    procedure Run;
  end;

{ ======================================================================= }
{ TBenchmarks                                                             }
{ ======================================================================= }

function TBenchmarks.CreateTestListString(ACount: Integer; const APrefix: String): TArray<String>;
begin
  SetLength(Result, ACount);
  for var Loop:=0 to ACount-1
    do Result[Loop]:=APrefix+' #'+IntToStr(Loop+1);
end;

{ ----------------------------------------------------------------------- }

procedure TBenchmarks.DictBenchmark;
const
  BenchRounds = 100;
begin
  var TestKeys:=CreateTestListString(100000,'Key');
  var TestValues:=CreateTestListString(100000,'Value');

  var MormotDict:=Collections.NewKeyValue<String,String>;
  var SpringDict:=TCollections.CreateDictionary<String,String>;

  Writeln(Format('Populate MormotDict with %d items...',[Length(TestKeys)]));
  RunBenchmark(
    procedure
    begin
      MormotDict.Clear;
      for var Loop:=0 to Length(TestKeys)-1
        do MormotDict.Add(TestKeys[Loop],TestValues[Loop]);
    end,BenchRounds);

  Writeln(Format('Populate SpringDict with %d items...',[Length(TestKeys)]));
  RunBenchmark(
    procedure
    begin
      SpringDict.Clear;
      for var Loop:=0 to Length(TestKeys)-1
        do SpringDict.Add(TestKeys[Loop],TestValues[Loop]);
    end,BenchRounds);

  Writeln('Compare MormotDict with SpringDict...');
  RunBenchmark(
    procedure
    var
      CurKey        : String;
      CurMormotValue: String;
      CurSpringValue: String;
    begin
      for var Loop:=0 to Length(TestKeys)-1 do
      begin
        CurKey:=TestKeys[Loop];
        if not
          (
            MormotDict.TryGetValue(CurKey,CurMormotValue) and
            SpringDict.TryGetValue(CurKey,CurSpringValue) and
            (CurMormotValue=CurSpringValue)
          )
          then raise Exception.Create('Different content!');
      end;
    end,BenchRounds);

  Writeln('Iterate MormotDict by key (TryGetValue)...');
  RunBenchmark(
    procedure
    var
      CurKey  : String;
      CurValue: String;
    begin
      for var Loop:=0 to Length(TestKeys)-1 do
      begin
        CurKey:=TestKeys[Loop];
        if not MormotDict.TryGetValue(CurKey,CurValue)
          then raise Exception.Create('Something is wrong!');
      end;
    end,BenchRounds);

  Writeln('Iterate SpringDict by key (TryGetValue)...');
  RunBenchmark(
    procedure
    var
      CurKey  : String;
      CurValue: String;
    begin
      for var Loop:=0 to Length(TestKeys)-1 do
      begin
        CurKey:=TestKeys[Loop];
        if not SpringDict.TryGetValue(CurKey,CurValue)
          then raise Exception.Create('Something is wrong!');
      end;
    end,BenchRounds);

  Writeln('Iterate (for..in) MormotDict ...');
  RunBenchmark(
    procedure
    var
      CurKey  : String;
      CurValue: String;
    begin
      for var Entry in MormotDict do
      begin
        CurKey:=Entry.Key;
        CurValue:=Entry.Value;
      end;
    end,BenchRounds);

  Writeln('Iterate (for..in) SpringDict ...');
  RunBenchmark(
    procedure
    var
      CurKey  : String;
      CurValue: String;
    begin
      for var Entry in SpringDict do
      begin
        CurKey:=Entry.Key;
        CurValue:=Entry.Value;
      end;
    end,BenchRounds);
end;

{ ----------------------------------------------------------------------- }

procedure TBenchmarks.ListBenchmark;
const
  BenchRounds = 100;
begin
  var TestData:=CreateTestListString(100000,'Value');
  var MormotList:=Collections.NewList<String>;
  var SpringList:=TCollections.CreateList<String>;

  Writeln(Format('Populate MormotList with %d items...',[Length(TestData)]));
  RunBenchmark(
    procedure
    begin
      MormotList.Clear;
      for var Entry in TestData do
      begin
        MormotList.Add(Entry);
      end;
    end,BenchRounds);

  Writeln(Format('Populate SpringList with %d items...',[Length(TestData)]));
  RunBenchmark(
    procedure
    begin
      SpringList.Clear;
      for var Entry in TestData do
      begin
        SpringList.Add(Entry);
      end;
    end,BenchRounds);

  Writeln('Compare MormotList with SpringList...');
  RunBenchmark(
    procedure
    begin
      for var Loop:=0 to SpringList.Count-1 do
      begin
        if MormotList[Loop]<>SpringList[Loop]
          then raise Exception.Create('Different content!');
      end;
    end,BenchRounds);

  Writeln('Iterate (for..in) through MormotList...');
  RunBenchmark(
    procedure
    begin
      var LastEntry: String;
      for var Entry in MormotList do
      begin
        LastEntry:=Entry;
      end;
    end,BenchRounds);

  Writeln('Iterate (for..in) through SpringList...');
  RunBenchmark(
    procedure
    begin
      var LastEntry: String;
      for var Entry in SpringList do
      begin
        LastEntry:=Entry;
      end;
    end,BenchRounds);

  Writeln('Iterate (for..to) through MormotList...');
  RunBenchmark(
    procedure
    begin
      var LastEntry: String;
      for var Loop:=0 to MormotList.Count-1 do
      begin
        LastEntry:=MormotList[Loop];
      end;
    end,BenchRounds);

  Writeln('Iterate (for..to) through SpringList...');
  RunBenchmark(
    procedure
    begin
      var LastEntry: String;
      for var Loop:=0 to SpringList.Count-1 do
      begin
        LastEntry:=SpringList[Loop];
      end;
    end,BenchRounds);
end;

{ ----------------------------------------------------------------------- }

procedure TBenchmarks.RunBenchmark(const AProc: TProc; ABenchmarkRounds: Integer);
begin
  var Stopper:=TStopwatch.StartNew;
  for var Loop:=1 to ABenchmarkRounds do
  begin
    AProc;
  end;
  Stopper.Stop;
  Writeln(Format('- (%d runs in %d msec.)',[ABenchmarkRounds,Stopper.ElapsedMilliseconds]));
  Writeln;
end;

{ ----------------------------------------------------------------------- }

procedure TBenchmarks.Run;
begin
  Writeln('---ListBenchmark---');
  ListBenchmark;

  Writeln('---DictBenchmark---');
  DictBenchmark;
end;

{ ======================================================================= }

begin
  try
    var Benchmarks:=TBenchmarks.Create;
    try
      Benchmarks.Run;
    finally
      Benchmarks.Free;
    end;
  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;

  Writeln('Done. Press Enter for exit.');
  Readln;
end.

And here some results:

---ListBenchmark---
Populate MormotList with 100000 items...
- (100 runs in 260 msec.)

Populate SpringList with 100000 items...
- (100 runs in 228 msec.)

Compare MormotList with SpringList...
- (100 runs in 212 msec.)

Iterate (for..in) through MormotList...
- (100 runs in 353 msec.)

Iterate (for..in) through SpringList...
- (100 runs in 192 msec.)

Iterate (for..to) through MormotList...
- (100 runs in 100 msec.)

Iterate (for..to) through SpringList...
- (100 runs in 102 msec.)

---DictBenchmark---
Populate MormotDict with 100000 items...
- (100 runs in 1198 msec.)

Populate SpringDict with 100000 items...
- (100 runs in 926 msec.)

Compare MormotDict with SpringDict...
- (100 runs in 846 msec.)

Iterate MormotDict by key (TryGetValue)...
- (100 runs in 557 msec.)

Iterate SpringDict by key (TryGetValue)...
- (100 runs in 286 msec.)

Iterate (for..in) MormotDict ...
- (100 runs in 641 msec.)

Iterate (for..in) SpringDict ...
- (100 runs in 372 msec.)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Performance of Mormot Collections vs. Spring v2 Collections

Yes, we know that Spring4D enumerators are faster because they do work at a lower level.
About the key/value search, there is also an overhead coming from the optional thread-safety of mORMot store, IIRC.
Note that iterating on a dictionary is not what a dictionary was meant to, so it seems a bit as a weird test.

The idea was not to be the fastest collection library, but the most integrated with mORMot other parts, and to be compatible with both FPC and Delphi.
In fact, it is a bit unfair to compare them, because they do not have same feature set. Since TDynArray and TSynDictionary, as used in mORMot collections, they have additional features like binary or JSON serialization, or timeouts and thread-safety for IKeyValue.
You need also to compare the memory consumption and the executable size, to be fair. You may also try on other sort of values. For instance, mORMot IndexOf() over integers is likely to be faster than other libraries...

So due to the way our collections are designed internally, I don't think we could make them much faster.
We think they are already fast enough for our purpose.
If anyone has some ideas, they are of course welcome!

WladiD

Member

From: Germany

Registered: 2010-10-27

Posts: 40

Re: Performance of Mormot Collections vs. Spring v2 Collections

Hi ab,

I had no intention of attacking you! I just wanted to share the results.

Note that iterating on a dictionary is not what a dictionary was meant to, so it seems a bit as a weird test.

Iterating over all keys in a dictionary to test the speed of TryGetValue is not a weird test for me. Here it is obviously, that TryGetValue of Spring dictionaries is over 90% faster (286 msec. vs 557 msec.) in this case.

You need also to compare the memory consumption and the executable size, to be fair.

I know, that the dcus are much smaller, when we use generic lists/dicts with mormot collections. This is a big plus! In this benchmarks, we want to see other metrics, this has nothing to do with "fair", it's simply a comparison.

wqmeng

Member

Registered: 2017-09-29

Posts: 18

Re: Performance of Mormot Collections vs. Spring v2 Collections

Hello,

Interesting.

I added another generic collections lib which on

https://github.com/d-mozulyov/Rapid.Generics

program SpringVsMormotCollections;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils,
  System.Diagnostics,
  mormot.core.collections,
  Spring,
  Spring.Collections
  , Rapid.Generics
  ;

{ ----------------------------------------------------------------------- }

type

  TBenchmarks = class
   private
    function CreateTestListString(ACount: Integer; const APrefix: String): TArray<String>;
    procedure RunBenchmark(const AProc: TProc; ABenchmarkRounds: Integer);
    procedure ListBenchmark;
    procedure DictBenchmark;
   public
    procedure Run;
  end;

{ ======================================================================= }
{ TBenchmarks                                                             }
{ ======================================================================= }

function TBenchmarks.CreateTestListString(ACount: Integer; const APrefix: String): TArray<String>;
begin
  SetLength(Result, ACount);
  for var Loop:=0 to ACount-1
    do Result[Loop]:=APrefix+' #'+IntToStr(Loop+1);
end;

{ ----------------------------------------------------------------------- }

procedure TBenchmarks.DictBenchmark;
const
  BenchRounds = 100;
begin
  var TestKeys:=CreateTestListString(100000,'Key');
  var TestValues:=CreateTestListString(100000,'Value');

  var MormotDict:=Collections.NewKeyValue<String,String>;
  var SpringDict:=TCollections.CreateDictionary<String,String>;
  var RapidDict:=Rapid.Generics.TRapidDictionary<String, String>.Create;

  Writeln(Format('Populate MormotDict with %d items...',[Length(TestKeys)]));
  RunBenchmark(
    procedure
    begin
      MormotDict.Clear;
      for var Loop:=0 to Length(TestKeys)-1
        do MormotDict.Add(TestKeys[Loop],TestValues[Loop]);
    end,BenchRounds);

  Writeln(Format('Populate SpringDict with %d items...',[Length(TestKeys)]));
  RunBenchmark(
    procedure
    begin
      SpringDict.Clear;
      for var Loop:=0 to Length(TestKeys)-1
        do SpringDict.Add(TestKeys[Loop],TestValues[Loop]);
    end,BenchRounds);

  Writeln(Format('Populate RapidDict with %d items...',[Length(TestKeys)]));
  RunBenchmark(
    procedure
    begin
      RapidDict.Clear;
      for var Loop:=0 to Length(TestKeys)-1
        do RapidDict.Add(TestKeys[Loop],TestValues[Loop]);
    end,BenchRounds);

  Writeln(Format('---------------------------------... %d',[Length(TestKeys)]));

  Writeln('Compare MormotDict with SpringDict...');
  RunBenchmark(
    procedure
    var
      CurKey        : String;
      CurMormotValue: String;
      CurSpringValue: String;
    begin
      for var Loop:=0 to Length(TestKeys)-1 do
      begin
        CurKey:=TestKeys[Loop];
        if not
          (
            MormotDict.TryGetValue(CurKey,CurMormotValue) and
            SpringDict.TryGetValue(CurKey,CurSpringValue) and
            (CurMormotValue=CurSpringValue)
          )
          then raise Exception.Create('Different content!');
      end;
    end,BenchRounds);

  Writeln('Compare MormotDict with RapidDict...');
  RunBenchmark(
    procedure
    var
      CurKey        : String;
      CurMormotValue: String;
      CurSpringValue: String;
    begin
      for var Loop:=0 to Length(TestKeys)-1 do
      begin
        CurKey:=TestKeys[Loop];
        if not
          (
            MormotDict.TryGetValue(CurKey,CurMormotValue) and
            RapidDict.TryGetValue(CurKey,CurSpringValue) and
            (CurMormotValue=CurSpringValue)
          )
          then raise Exception.Create('Different content!');
      end;
    end,BenchRounds);

  Writeln('Compare SpringDict with RapidDict...');
  RunBenchmark(
    procedure
    var
      CurKey        : String;
      CurMormotValue: String;
      CurSpringValue: String;
    begin
      for var Loop:=0 to Length(TestKeys)-1 do
      begin
        CurKey:=TestKeys[Loop];
        if not
          (
            RapidDict.TryGetValue(CurKey,CurSpringValue) and
            SpringDict.TryGetValue(CurKey,CurMormotValue) and
            (CurMormotValue=CurSpringValue)
          )
          then raise Exception.Create('Different content!');
      end;
    end,BenchRounds);

  Writeln(Format('---------------------------------... %d',[Length(TestKeys)]));

  Writeln('Iterate MormotDict by key (TryGetValue)...');
  RunBenchmark(
    procedure
    var
      CurKey  : String;
      CurValue: String;
    begin
      for var Loop:=0 to Length(TestKeys)-1 do
      begin
        CurKey:=TestKeys[Loop];
        if not MormotDict.TryGetValue(CurKey,CurValue)
          then raise Exception.Create('Something is wrong!');
      end;
    end,BenchRounds);

  Writeln('Iterate SpringDict by key (TryGetValue)...');
  RunBenchmark(
    procedure
    var
      CurKey  : String;
      CurValue: String;
    begin
      for var Loop:=0 to Length(TestKeys)-1 do
      begin
        CurKey:=TestKeys[Loop];
        if not SpringDict.TryGetValue(CurKey,CurValue)
          then raise Exception.Create('Something is wrong!');
      end;
    end,BenchRounds);

  Writeln('Iterate RapidDict by key (TryGetValue)...');
  RunBenchmark(
    procedure
    var
      CurKey  : String;
      CurValue: String;
    begin
      for var Loop:=0 to Length(TestKeys)-1 do
      begin
        CurKey:=TestKeys[Loop];
        if not RapidDict.TryGetValue(CurKey,CurValue)
          then raise Exception.Create('Something is wrong!');
      end;
    end,BenchRounds);

  Writeln(Format('---------------------------------... %d',[Length(TestKeys)]));

  Writeln('Iterate (for..in) MormotDict ...');
  RunBenchmark(
    procedure
    var
      CurKey  : String;
      CurValue: String;
    begin
      for var Entry in MormotDict do
      begin
        CurKey:=Entry.Key;
        CurValue:=Entry.Value;
      end;
    end,BenchRounds);

  Writeln('Iterate (for..in) SpringDict ...');
  RunBenchmark(
    procedure
    var
      CurKey  : String;
      CurValue: String;
    begin
      for var Entry in SpringDict do
      begin
        CurKey:=Entry.Key;
        CurValue:=Entry.Value;
      end;
    end,BenchRounds);

  Writeln('Iterate (for..in) RapidDict ...');
  RunBenchmark(
    procedure
    var
      CurKey  : String;
      CurValue: String;
    begin
      for var Entry in RapidDict do
      begin
        CurKey:=Entry.Key;
        CurValue:=Entry.Value;
      end;
    end,BenchRounds);

  Writeln(Format('---------------------------------... %d',[Length(TestKeys)]));
end;

{ ----------------------------------------------------------------------- }

procedure TBenchmarks.ListBenchmark;
const
  BenchRounds = 100;
begin
  var TestData:=CreateTestListString(100000,'Value');
  var MormotList:=Collections.NewList<String>;
  var SpringList:=TCollections.CreateList<String>;
  var RapidList:=Rapid.Generics.TList<string>.create;

  Writeln(Format('Populate MormotList with %d items...',[Length(TestData)]));
  RunBenchmark(
    procedure
    begin
      MormotList.Clear;
      for var Entry in TestData do
      begin
        MormotList.Add(Entry);
      end;
    end,BenchRounds);

  Writeln(Format('Populate SpringList with %d items...',[Length(TestData)]));
  RunBenchmark(
    procedure
    begin
      SpringList.Clear;
      for var Entry in TestData do
      begin
        SpringList.Add(Entry);
      end;
    end,BenchRounds);

  Writeln(Format('Populate RapidList with %d items...',[Length(TestData)]));
  RunBenchmark(
    procedure
    begin
      RapidList.Clear;
      for var Entry in TestData do
      begin
        RapidList.Add(Entry);
      end;
    end,BenchRounds);

  Writeln(Format('---------------------------------... %d',[Length(TestData)]));

  Writeln('Compare MormotList with SpringList...');
  RunBenchmark(
    procedure
    begin
      for var Loop:=0 to SpringList.Count-1 do
      begin
        if MormotList[Loop]<>SpringList[Loop]
          then raise Exception.Create('Different content!');
      end;
    end,BenchRounds);

  Writeln('Compare MormotList with RapidList...');
  RunBenchmark(
    procedure
    begin
      for var Loop:=0 to MormotList.Count-1 do
      begin
        if MormotList[Loop]<>RapidList[Loop]
          then raise Exception.Create('Different content!');
      end;
    end,BenchRounds);

  Writeln('Compare SpringList with RapidList...');
  RunBenchmark(
    procedure
    begin
      for var Loop:=0 to SpringList.Count-1 do
      begin
        if SpringList[Loop]<>RapidList[Loop]
          then raise Exception.Create('Different content!');
      end;
    end,BenchRounds);

  Writeln(Format('---------------------------------... %d',[Length(TestData)]));

  Writeln('Iterate (for..in) through MormotList...');
  RunBenchmark(
    procedure
    begin
      var LastEntry: String;
      for var Entry in MormotList do
      begin
        LastEntry:=Entry;
      end;
    end,BenchRounds);

  Writeln('Iterate (for..in) through SpringList...');
  RunBenchmark(
    procedure
    begin
      var LastEntry: String;
      for var Entry in SpringList do
      begin
        LastEntry:=Entry;
      end;
    end,BenchRounds);

  Writeln('Iterate (for..in) through RapidList...');
  RunBenchmark(
    procedure
    begin
      var LastEntry: String;
      for var Entry in RapidList do
      begin
        LastEntry:=Entry;
      end;
    end,BenchRounds);

  Writeln(Format('---------------------------------... %d',[Length(TestData)]));

  Writeln('Iterate (for..to) through MormotList...');
  RunBenchmark(
    procedure
    begin
      var LastEntry: String;
      for var Loop:=0 to MormotList.Count-1 do
      begin
        LastEntry:=MormotList[Loop];
      end;
    end,BenchRounds);

  Writeln('Iterate (for..to) through SpringList...');
  RunBenchmark(
    procedure
    begin
      var LastEntry: String;
      for var Loop:=0 to SpringList.Count-1 do
      begin
        LastEntry:=SpringList[Loop];
      end;
    end,BenchRounds);

  Writeln('Iterate (for..to) through RapidList...');
  RunBenchmark(
    procedure
    begin
      var LastEntry: String;
      for var Loop:=0 to RapidList.Count-1 do
      begin
        LastEntry:=RapidList[Loop];
      end;
    end,BenchRounds);
end;

{ ----------------------------------------------------------------------- }

procedure TBenchmarks.RunBenchmark(const AProc: TProc; ABenchmarkRounds: Integer);
begin
  var Stopper:=TStopwatch.StartNew;
  for var Loop:=1 to ABenchmarkRounds do
  begin
    AProc;
  end;
  Stopper.Stop;
  Writeln(Format('- (%d runs in %d msec.)',[ABenchmarkRounds,Stopper.ElapsedMilliseconds]));
  Writeln;
end;

{ ----------------------------------------------------------------------- }

procedure TBenchmarks.Run;
begin
  Writeln('---ListBenchmark---');
  ListBenchmark;

  Writeln('---DictBenchmark---');
  Writeln('---DictBenchmark---');
  Writeln('---DictBenchmark---');
  DictBenchmark;
end;

{ ======================================================================= }

begin
  try
    var Benchmarks:=TBenchmarks.Create;
    try
      Benchmarks.Run;
    finally
      Benchmarks.Free;
    end;
  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;

  Writeln('Done. Press Enter for exit.');
  Readln;
end.

And here is a result.

---ListBenchmark---
Populate MormotList with 100000 items...
- (100 runs in 275 msec.)

Populate SpringList with 100000 items...
- (100 runs in 245 msec.)

Populate RapidList with 100000 items...
- (100 runs in 241 msec.)

---------------------------------... 100000
Compare MormotList with SpringList...
- (100 runs in 214 msec.)

Compare MormotList with RapidList...
- (100 runs in 287 msec.)

Compare SpringList with RapidList...
- (100 runs in 285 msec.)

---------------------------------... 100000
Iterate (for..in) through MormotList...
- (100 runs in 353 msec.)

Iterate (for..in) through SpringList...
- (100 runs in 203 msec.)

Iterate (for..in) through RapidList...
- (100 runs in 285 msec.)

---------------------------------... 100000
Iterate (for..to) through MormotList...
- (100 runs in 102 msec.)

Iterate (for..to) through SpringList...
- (100 runs in 109 msec.)

Iterate (for..to) through RapidList...
- (100 runs in 236 msec.)

---DictBenchmark---
---DictBenchmark---
---DictBenchmark---
Populate MormotDict with 100000 items...
- (100 runs in 1216 msec.)

Populate SpringDict with 100000 items...
- (100 runs in 962 msec.)

Populate RapidDict with 100000 items...
- (100 runs in 573 msec.)

---------------------------------... 100000
Compare MormotDict with SpringDict...
- (100 runs in 848 msec.)

Compare MormotDict with RapidDict...
- (100 runs in 826 msec.)

Compare SpringDict with RapidDict...
- (100 runs in 483 msec.)

---------------------------------... 100000
Iterate MormotDict by key (TryGetValue)...
- (100 runs in 578 msec.)

Iterate SpringDict by key (TryGetValue)...
- (100 runs in 286 msec.)

Iterate RapidDict by key (TryGetValue)...
- (100 runs in 297 msec.)

---------------------------------... 100000
Iterate (for..in) MormotDict ...
- (100 runs in 662 msec.)

Iterate (for..in) SpringDict ...
- (100 runs in 402 msec.)

Iterate (for..in) RapidDict ...
- (100 runs in 611 msec.)

---------------------------------... 100000
Done. Press Enter for exit.

dcoun

Member

From: Crete, Greece

Registered: 2020-02-18

Posts: 430

Re: Performance of Mormot Collections vs. Spring v2 Collections

Can you provide performance results for Freepascal too?

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Performance of Mormot Collections vs. Spring v2 Collections

To my understanding, mORMot collection performance is fine.
Spring v2 has been deeply rewritten for performance, so mORMot collections being in the same order of magnitude (up to only 2 times slower) seems fair enough.
On some other tests (e.g. search an integer by value, not by key), mORMot would be much faster than Spring v2.

We would also need to benchmark some other types, like complex records, TGuid, integer, double...
And check about item deletion speed.
Checking memory consumption could also be interesting. My guess is that the hash table algorithms of mORMot could make pretty good use of the memory, especially for a few items. Also the data is likely to be more locally situated in the CPU cache, because the hashes are not stored with the data. Note that it is possible with mORMot (using direct TDynArray and TDynArrayHashed access) to maintain several indexes on the same data. Try this with other libraries.

About FPC, our unit seems faster than the version in the RTL. It seems that FPC calls AnsiCompareStr() on string values, which is slow for no reason.
We also noticed that latest iterations of the standard Delphi generics collections are also faster than mORMot, for such microbenchmakrs (adding + getting + enumerating).
But the Delphi collections library has a huge complexity, and also a huge size in the executable.
Note that search of a value (IndexOf or ContainsValue) is much faster with mORMot, because it uses optimized TDynArray search methods.

According to my experiment, such microbenchmarks are not so meaningful.
As I wrote, no real process is making a TryGetValue() over all items one by one. Or use "for ... in" over a dictionary. You just use an array or a list for this. You don't need the overhead of the dictionary hash table. If you use a dictionary to iterate over all its values, your code is just wrong, and should be fixed if you need additional performance. Use of a dynamic array is the way to go, then.
For instance, if you need a string/string dictionary, in mORMot you won't use collections, but TRawUtf8List.

With its additional optional featuresets (e.g.  thread safety, lower executable size, binary and JSON serialization, builtin string case-sensitivity, hash index for IList items, timeout of key/value pairs, custom hasher, key values interning, possibility to have several indexes on the same data, and - the most important to me - cross-compiler compatible), mORMot seems fast enough to me.
To respond to the initial point of this forum thread, the mormot.core.collections.pas unit has already been profiled and already tuned as much as possible. Of course, if you find some bottlenecks, you are welcome to make some pull requests!
Perhaps we could make it faster, but it would be at the expense of more complex code, perhaps more bugs, and we won't see any difference on real apps.
Our goal is to be fast on real apps, not on microbenchmarks.
Please refer to our blog article for the purpose of this unit: https://blog.synopse.info/?post/2021/12 … ollections
And I have seen so many "internal compiler errors" compiling generics, that I would only use them for small code snippet of the most obvious code patterns.

danielkuettner

Member

From: Germany

Registered: 2014-08-06

Posts: 357

Re: Performance of Mormot Collections vs. Spring v2 Collections

I'm aware I'm not speaking for the whole mOMRot community, but in our project we can get without generics. But without mORMot using Delphi/Pascal were impossible.

Thats all what I can say to this topic from side.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Performance of Mormot Collections vs. Spring v2 Collections

Thanks Daniel for the feedback.

About performance, I found out that IKeyValue<>.TryGetValue() was less optimized than IKeyValue<>.GetItem()
You may try https://github.com/synopse/mORMot2/commit/76ace06b for a slightly better performance.

Perhaps also using "string" for type is not the best optimized for mORMot yet.
Server code using mORMot is expected to use RawUtf8, not string.
And string comparison of UnicodeString with mORMot TDynArray/TSynDictionary on Delphi is using StrCompW() which is not the most optimized.
So benchmarking with "string" is a worse case scenario. You may try with Utf8Unicode and see better numbers.