mORMot Open Source

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Faster realloc on SynScaleMM

We received, from EMB forum, an interesting feedback:

I have one test case where ScaleMM is slower (a lot) than FastMM4, in multi or single threaded applications: If you deal with ClientDataSets and use Andreas Hausladen MidasSpeedFix (http://andy.jgknet.de/blog/2009/01/mida … -fix-unit/). MidasLib always calls ReallocMem when appending data to the ClientDataSet. Debugging ScaleMM I could see that MidasLib's ReallocMem (when appending data) calls is doing what Andreas MidasSpeedFix was trying to avoid: a call to GetMem + Move. When dealing with a lot of ReallocMem calls I guess that FastMM4 is faster?

The current implementation of ScaleMM try to use the immediately bigger buffer size, in case of a growing ReallocMem.

Since the buffer granularity is some kind of small, a ReallocMem() which increase a buffer length for some bytes at a time will make it move the data through all buffer size, that is 32, 64, 96, 128, 160, 192, 224, 256,512,768,1024,1280,1536,1792, 2048, 4096...16384 bytes.

In this particular scenario, it could make sense to use some bigger buffers for the reallocation...

Here is the patch:
http://synopse.info/fossil/fdiff?v1=087 … d0d469a17c

I think it will be much faster in this particular case, because it will grow up the buffer into three intermediate size only: 224, 1792 and 16384, i.e. the highest available item of every block series.

cstuffer

Member

Registered: 2010-07-21

Posts: 11

Re: Faster realloc on SynScaleMM

Hello,

Recently i begun to use ScaleMM and have noticed some great speed enchancements on my project.

My question is:
Is SynScaleMM is always fully updated with the last revisions of ScaleMM ??

My second is:
What are the advantage of using SynScaleMM ?  (and disavantages if any)

My third is:
I also tried your enhanced RTL... in their DCU forms.
I am using Delphi 7 all updated and i just copied your enhanced DCU's to my Delphi Lib directory.
I also use FastMM as first  item in the uses.

I am not sure (this is difficult to evaluate in a real project) but i think i got a little degradation in speed.
Is this possible or just a bad feeling ?

Thank you for your reply
Carl

Last edited by cstuffer (2011-02-27 11:20:35)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: Faster realloc on SynScaleMM

You'll find here some answers:
http://synopse.info/forum/viewtopic.php?pid=1100#p1100

In short:
- SynScaleMM = ScaleMM1 + some enhancements (see link above about the latest reallocmem patch);
- ScaleMM2 = more powerful (new design) but yet slower than SynScaleMM/ScaleMM1, because first draft from its author (and I didn't spent time trying to implement some part in optimized asm, whereas I've done this for SynScaleMM).

When did the speed degradation be introduced?
With SynScaleMM or with the Enhanced RTL?
It's possible with the first, not possible with the second IMHO.

And when you deal with speed, don't trust your feeling, but your clock and profiling.

About SynScaleMM, be sure to have taken the latest version from the link above.

Thanks for your interest.

cstuffer

Member

Registered: 2010-07-21

Posts: 11

Re: Faster realloc on SynScaleMM

Hello ab,

Thank you for the details

I agree with you, it's better to use clock than feeling for evaluation.

In a real life project (not a test speed test), it's quite hard to note difference... even with clock.

By using the enhanced RTL...
My project is using a skin engine (VCLSkin),
and when the program is starting, i had the feeling of a blink between states "start and fully skinned".
As you may know, in such case, even with a clock result it's hard to catch.
But again... it's feeling. Haha

I will try some more specific tests

Carl

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: Faster realloc on SynScaleMM

I suspect you used the enhanced RTL without using FastMM4 or SynScaleMM.

Our enhanced RTL has its own very thin memory manager, which is calling the Windows slow memory manager.
It was on purpose, because the Borland MM was faster than Windows's one, but much slower than FastMM4 and SynScaleMM.

If you want a smaller exe, you'll use our enhanced RTL default MM, which will save you some KB.
If you want a fast exe, you'll have to add a FastMM4 and/or SynScaleMM call in the first line of your .dpr file, like this:

program Test;

uses
  FastMM4,
  Windows,
....

or

program Test;

uses
  FastMM4,
  SynScaleMM, 
  Windows,
....

or (should be fast in practice, slower for one thread application, but faster for multi-threaded application):

program Test;

uses
  SynScaleMM, 
  Windows,
....

So I suspect you didn't specify to use FastMM4 nor SynScaleMM, so you used Windows MM... and your application is slower than previously!

If you want your code to compile with all versions of Delphi, including those how have FastMM4 as default MM, you can use this:

program Test;

uses
  {$I SynDprUses.inc}
  Windows,
....

Here is the content of SynDprUses.inc, as available in http://synopse.info/fossil :

{$ifdef ENHANCEDRTL}
  FastMM4, // FastMM4 is not integrated by default in our Enhanced Run Time Library
{$else}
  {$IFDEF CONDITIONALEXPRESSIONS}
    {$if CompilerVersion <= 17} // Delphi 2006 (and up) have FastMM4 integrated
      FastMM4,
    {$ifend}
  {$ELSE}
    FastMM4, // no conditional expressions -> versions older than Delphi 6
  {$ENDIF}
{$endif}

cstuffer

Member

Registered: 2010-07-21

Posts: 11

Re: Faster realloc on SynScaleMM

Hello,

Thank you for your reply.

No... do not worry, the uses are there

With Delphi 7...

***************
uses
  FastMM4,
  SynScaleMM,
  Forms,
  ....
***************

and i am using your enhanced RTL's too.

I am also using QStrings.pas which is all ASM, really fast for POS and REPLACE functions( and others like functions)

I am always searching for things to speed up my applications

Carl

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: Faster realloc on SynScaleMM

I think the Pos() function in our Enhanced RTL is already very fast.

I didn't optimize StringReplace() function much, only the UTF-8 version in our framework.

Thanks for your interest!