How to write fast multi-thread Delphi applications

ab · 2011-05-20 19:05:56

How to make your software run fast, especially in a multi-threaded architecture?

We tried to remove the Memory Manager scaling problems in our SynScaleMM. It worked as expected in a multi-threaded server environment. Scaling is much better than FastMM4, for some critical tests. But it's not ready for production yet...

To be honest, the Memory Manager is perhaps not the bigger bottleneck in Multi-Threaded applications.

Here are some (not dogmatic, just from experiment and knowledge of low-level Delphi RTL) advice if you want to write FAST multi-threaded application in Delphi.

- Always use const for string or dynamic array parameters like in MyFunc(const aString: String) to avoid allocating a temporary string per each call;
- Avoid using string concatenation (s := s+'Blabla'+IntToStr(i)) , but rely on a buffered writing such as TStringBuilder available in latest versions of Delphi;
- TStringBuilder is not perfect either: for instance, it will create a lot of temporary strings for appending some numerical data, and will use the awfully slow SysUtils.IntToStr() function when you add some integer value - I had to rewrite a lot of low-level functions to avoid most string allocation in our TTextWriter class as defined in SynCommons.pas unit;
- Don't abuse on critical sections, let them be as small as possible, but rely on some atomic modifiers if you need some concurrent access - see e.g. InterlockedIncrement / InterlockedExchangeAdd;
- Don't share data between threads, but rather make your own private copy or rely on some read-only buffers (the RCU pattern is the better for scaling);
- Don't use indexed access to string characters, but rely on some optimized functions like PosEx() for instance;
- Don't mix AnsiString/UnicodeString kind of variables/functions, and check the generated asm code via Alt-F2 to track any hidden unwanted conversion (e.g. call UStrFromPCharLen);
- Rather use var parameters in a procedure instead of a function returning a string (a function returning a string will add an UStrAsg/LStrAsg call which has a LOCK which will flush all CPU cores);
- If you can, for your data or text parsing, use pointers and some static stack-allocated buffers instead of temporary strings or dynamic arrays;
- Don't create a TMemoryStream each time you need one, but rely on a private instance in your class, already sized in enough memory, in which you will write data using Position to retrieve the end of data and not changing its Size property (which will be the memory block allocated by the MM);
- Limit the number of class instances you create: try to reuse the same instance, and if you can, use some record/object pointers on already allocated memory buffers, mapping the data without copying it into temporary memory;
- Always use test-driven development, with dedicated multi-threaded test, trying to reach the worse-case limit (increase number of threads, data content, add some incoherent data, pause at random, try to stress network or disk access, benchmark with timing on real data...);
- Never trust your instinct, we are no computer but men... use accurate timing on real data and process.

I tried to follow those rules in our Open Source framework, and if you take a look at our code, you'll find out a lot of real-world sample code...

ab · 2011-06-05 08:50:49

Some nice advice from real SW masters:

- Make it right before you make it fast. Make it clear before you make it faster. Keep it right when you make it faster. — Kernighan and Plauger, Elements of Programming Style.
- Premature optimization is the root of all evil. — Donald Knuth, quoting C. A. R. Hoare
- The key to performance is elegance, not battalions of special cases. The terrible temptation to tweak should be resisted. — Jon Bentley and Doug McIlroy
- The rules boil down to: "1. Don't optimize early. 2. Don't optimize until you know that it's needed. 3. Even then, don't optimize until you know what's needed, and where." — Herb Sutter

See also http://drdobbs.com/tools/229900148

Starkis · 2011-06-09 15:31:30

what about TStringBuilder for Delphi 7? or just use TTextWriter?

ab · 2011-06-09 16:39:42

TTextWriter is a TStringBuilder on steroids...

TPrami · 2011-06-10 04:30:52

Hope that TTextWriter is not a Female then

(Sorry could not resist)

Starkis · 2011-06-13 19:37:04

meaning MyStuff( const myArray : array of integer )?

- Always use const for string or dynamic array parameters like in MyFunc(const aString: String) to avoid allocating a temporary string per each call;

good advice for recent Delphi, which tries to catch .NET/Java world, too bad for the Delphi 7 without ala s := _stringBuilder( s, 'Blabla', IntToStr( i ) ) or TTextWriter on steroids

- Avoid using string concatenation (s := s+'Blabla'+IntToStr(i)) , but rely on a buffered writing such as TStringBuilder available in latest versions of Delphi;

TPrami · 2011-06-14 05:16:54

Hello,

I have been thinking of two terms that should be used.

ThreadSafe - For code that works in multithreaded environment, don't crash and works as expected. (what it means )

ThreadSmart - That runs very fast and is (maybe) best solution

Basically some guidelines, as mentioned above are very very important. To help programmer to look for own code to make it better.

I just like to make challenge to someone(s) to start keeping up (a blog) document where would be up to date list of these. And I would like to have some kind of example codes sniplets.

Code Before and After from ThreadSafe to ThreadSmart Because this way it would be much more clear. We all have not very good English, me included, but we all speak Pascal. So it would be the universal language for all of us. This would help all very very much (And when I am talking of Someone I am not talking of that it necessarily should be Arnaud Bouchez, he has done so much already)

-Tee-

ab · 2011-06-14 05:35:35

And I'm hoping that ThreadSmart is also ThreadSafe: I've already seen applications running very fast... until they crash...

Code snippets are a very good idea.
I'll look into it. There are a lot of code sample of those techniques in the framework, so a simple copy & paste should be good enough...

TPrami · 2011-06-14 09:41:32

ab wrote:

And I'm hoping that ThreadSmart is also ThreadSafe: I've already seen applications running very fast... until they crash...
Code snippets are a very good idea.
I'll look into it. There are a lot of code sample of those techniques in the framework, so a simple copy & paste should be good enough...

Yes, if you don't mind, that would be very good.

And My idea of ThreadSmart would be also the ThreadSafe, but smarter

.-TP-.

visli · 2011-08-09 13:12:05

When to use TTextWriter.Flush method?

ab · 2011-08-09 13:59:52

visli wrote:

When to use TTextWriter.Flush method?

When you want to use the internal Stream directly.
Its purpose is to write the internal memory buffer to the Stream.
If you don't call Flush, some pending characters may be not yet copied to the Stream.

For example, TTextWriter.Text calls Flush.

I've updated the documentation to reflect this.

TPrami · 2011-09-22 11:03:18

Hello hello,

I was just thinking that because in many Cases the using the String will cause problems in Multithreaded Delphi program.

And sometimes you only pass the string around and not actually do anything with it. Like some kind of metadata with string information.

And that causes lot of Locked operations on low level, kid of invisible to programmer. Just having the strings around, not actually using them most of the time.

Then I started to think that maybe we should have some container classes to store those kind of variables/data.

Like some simple buffer/storage class for String. It would not use string internally, just an piece of allocated memory. And it should/could have some kind of non locked copy method. etc.

This way the having the String data around, and maybe passing it along would not cause so much Locks. But it should have some kind of wrapper class (maybe for Dynamic arrays also). To make it safer and less error prone.

This is just an idea to get rid of the String related locks...

Even better ideas are more than welcome

-TP-

ab · 2011-09-22 14:31:34

Strings and dynamic arrays are reference counted, so passing them around within a class won't change much, I guess.

The only benefit would be of making a private copy for each thread.
But low-level RTL is not handling this yet...

TPrami · 2011-09-22 17:52:15

Hello,

I think my English was not good enough to describe my idea.

My idea was to get rid of reference count by making simple class that will hold the string (or array) in plain memory area,

Pointer and GetMem kind of solution.

To reason for the Wrapper class is that using pointer and getmem is error prone to use everywhere.

So the lightweight wrapper class to hold reference counted variables and you could then pass it around and copy other container, without reference counting.

This will not solve all the problems, but would help in some cases...

-TP-

ab · 2011-09-22 19:17:45

This is a nice idea.

Such a structure may be able to handle cross-thread, RCU feature in Delphi.

With operator overloading and a record, it could make sense, especially on newer Delphi versions.

Even if the "+" operator is not so easy to define for multi-concatenation, if I remember well.

TPrami · 2011-09-23 05:10:43

ab wrote:

This is a nice idea.
Such a structure may be able to handle cross-thread, RCU feature in Delphi.
With operator overloading and a record, it could make sense, especially on newer Delphi versions.
Even if the "+" operator is not so easy to define for multi-concatenation, if I remember well.

But in some tight spots just having the some kind of generic (not <generic> ) class would help a lot.

other thing I've been thinking of is the "multithread data" container.

Think of web server cache. You got the data cached in streams and you pass the data in higher level codee (like Indy) you can't have multiple readers at same time.

Streams are handy but you can't read them in multiple threads.

It would be cool to have some data storage that you could have "Stream adapters to" which would handle the passing into the higher level code (and custom code of course could use the lower level container directly).

I think that plain reading the memory is thread safe (simultaneusly) if you make sure that the memory is there and don't change.

I was thinking that is the Data updates, once it is ready it would be available to new readers, and old data would sit there until all readers are finished reading it.

This is just rambling this and possibly to get some kind of comments and verification. I've been thinking of some server code, which would have some more than less static data, so some way like this would be efficient way to cache it. So I would not need to Lock on every copy from the Cache.

Maybe cache like this would be usable to others also

-Tee-

ab · 2011-09-23 08:23:58

Starting with Windows Vista, there is the a new interresting feature to implement such behavior, and avoid copying the content among threads.

Slim reader/writer (SRW) locks enable the threads of a single process to access shared resources; they are optimized for speed and occupy very little memory.

Its main disadvantage is that SRW locks cannot be acquired recursively. So it's not a direct replacement of the critical sections.

Before Vista, we may use the TMultiReadExclusiveWriteSynchronizer class supplied by Delphi, which seems slower, but available on all platforms.

See http://msdn.microsoft.com/en-us/library … S.85).aspx

Starkis · 2013-09-09 20:00:50

how to pass records as params without copying? decorate with the var or const?

strings and open arrays with the const, while static and dynamic arrays with the var

ab · 2013-09-10 06:27:31

Yes, "const" is your friend, here.

"var" allows writing to the content, which should be only explicit when working in multi-thread.

louis_riviera · 2013-10-24 12:40:12

The problem is that "var" also adds this "call @UStrAsg"

with this procedure Test(const A, B, C: Integer; var Return: String);

So what is real example on how to solve this?

ab · 2013-10-24 13:47:01

From my tests, it compiles into:

mORMotSelfTests.pas.227: Test(1,2,3,s);
005515E7 8D45FC           lea eax,[ebp-$04]
005515EA 50               push eax
005515EB B903000000       mov ecx,$00000003
005515F0 BA02000000       mov edx,$00000002
005515F5 B801000000       mov eax,$00000001
005515FA E8C5FFFFFF       call Test

So the "var s: string" parameter is passed with no reference count addition.

The UStrAsg/LStrAsg is only generated if you assign some value to s within the Test procedure - but this is by design.

louis_riviera · 2013-10-24 13:47:58

Oh I forgot to mention this was compiled with Delphi XE5. And analyzed with Dissasembly tool via Delphi Debugger.
I almost get the same result with function. So this solution doesn't really do anything..

I think this happens only when you use old old compilers such as D7. But not new.

function Test(const A, B, C: Integer):String;
begin
  Result := 'Test';
end;

Turns to:

TForm2.Test:
005B4910 55               push ebp
005B4911 8BEC             mov ebp,esp
005B4913 83C4F4           add esp,-$0c
005B4916 894DF4           mov [ebp-$0c],ecx
005B4919 8955F8           mov [ebp-$08],edx
005B491C 8945FC           mov [ebp-$04],eax
005B491F 8B4508           mov eax,[ebp+$08]
005B4922 BA40495B00       mov edx,$005b4940
005B4927 E8604DE5FF       call @UStrAsg

procedure Test(const A, B, C: Integer; var Return:String);
begin
  Return := 'Test';
end;

Turns to:

TForm2.Test:
005B4918 55                 push ebp
005B4919 8BEC              mov ebp,esp
005B491B 83C4F4           add esp,-$0c
005B491E 894DF4           mov [ebp-$0c],ecx
005B4921 8955F8           mov [ebp-$08],edx
005B4924 8945FC           mov [ebp-$04],eax
005B4927 8B4508           mov eax,[ebp+$08]
005B492A BA48495B00    mov edx,$005b4948
005B492F E8584DE5FF    call @UStrAsg

The more important question not raised here is.. What to use instead of "string" ??

Last edited by louis_riviera (2013-10-24 14:40:13)

ab · 2013-10-24 15:14:43

For all compilers, a "string" function result is implemented as an additional "var" parameter added as latest of the parameter list.
See http://docwiki.embarcadero.com/RADStudi … am_Control

You can use a shortstring or a fixed-sized array of char...
Any reference-counted types (like string or dynamic array) will need reference-count management so a call to UStrAsg or DynArrAsg...

louis_riviera · 2013-10-24 15:43:09

I think this needs to be fixed on compiler side. Fiddling with code is waste of time and productivity. If you fix the string then you still other components that use it. So it is pretty much pointless. Must be fixed by Embercadero..

ab · 2013-10-24 17:26:20

AFAIK this is not a bug, but a feature: without this global lock, the reference-count mechanism won't work in multi-thread.

My proposal of http://blog.synopse.info/post/2010/07/3 … adlocalvar to define 'thread-local' variables could make sense.

louis_riviera · 2013-10-24 20:11:11

C++ Builder has these problems aswell?

ab · 2013-10-25 06:38:11

I guess yes, since both Delphi and C++Builder share the same RTL code for common types like string.

louis_riviera · 2013-10-25 07:07:04

So only solution is to modify System.pas? But i guess how to recompile? They add these stupid locks everywhere.

With this lock isn't called.. LStrAsg is called and LOCK is skipped.

procedure Test(const A,B:Integer; var Return:RAWUTF8);
begin
Return := 'A';
end;

Correct way of using?

Last edited by louis_riviera (2013-10-25 12:42:47)

ab · 2013-10-25 13:25:17

AFAIK UStrAsg and LStrAsg will always uses a LOCKed opcode....
Using RawUTF8 won't change anything here!

louis_riviera · 2013-10-25 14:36:21

So TTextWriter can be used as a String replacement?

ab · 2013-10-25 14:51:55

Yes, as a 'string builder' class with almost no lock.

louis_riviera · 2013-10-25 15:02:40

What about a lock free dynamic array ? Many thanks!

Last edited by louis_riviera (2013-10-25 15:02:55)

louis_riviera · 2013-10-26 14:50:42

Why not patch these functions in runtime? LStrAsg etc. ?

ab · 2013-10-26 15:10:20

Because it will break expected multi thread behavior....
So it would be unsafe.

louis_riviera · 2013-10-26 15:12:57

Does SynCommons contain a fast dynamic array which dont use LOCK instructions?

Also is it possible to use sqlite3 with SynSQLite3 without dll? In 64 bit?

Last edited by louis_riviera (2013-10-26 15:13:59)

ab · 2013-10-26 15:22:20

louis_riviera wrote:

Does SynCommons contain a fast dynamic array which dont use LOCK instructions?

No, but I suspect speed benefit will be negligeable in respect to allocation and filling time of the array itself.
Premature optimization is the root of all evil!

louis_riviera wrote:

Also is it possible to use sqlite3 with SynSQLite3 without dll? In 64 bit?

It is possible in 32 bit mode (we supplied optimized .obj), but not in 64 bit mode.

louis_riviera · 2013-10-26 15:27:53

Ah yes so we need to compile 64 bit our self to get .obj? I really don't like dll mess

ab · 2013-10-26 15:37:08

Or you can use your own dll safely.

louis_riviera · 2013-10-26 16:05:45

Yes but it would be nice if you could provide a 64 bit .obj file.

DISQLite3 supports 64 bit since ages!

EDIT: We decided to use UNIDAC.

Last edited by louis_riviera (2013-10-26 22:45:28)

Starkis · 2014-07-11 12:44:29

so var and const for records prevent param copying, what about static and dynamic arrays - const and var also prevent dublication? or only var (const only tu ensure read access)?

Starkis wrote:

how to pass records as params without copying? decorate with the var or const?
strings and open arrays with the const, while static and dynamic arrays with the var

ab · 2014-07-11 15:16:57

Dynamic arrays use a light COW (copy on write) pattern, so even without const they are passed by reference, and a reference count is quickly incremented.
But if you change it, it will in fact change it everywhere.
It is not a true COW implementation, as it is for strings.

My advice is to use var for dynamic arrays, and explicitly copy them before local modification.

Dynamic arrays content is one of the hardest think to manage in Delphi.
See http://delphihaven.wordpress.com/2009/1 … alf-baked/
and http://www.delphitools.info/2011/06/15/ … alue-type/

leeonsoft · 2014-09-07 02:50:39

Don't create a TMemoryStream each time you need one, but rely on a private instance in your class, already sized in enough memory, in which you will write data using Position to retrieve the end of data and not changing its Size property (which will be the memory block allocated by the MM);

can you share some sample code ? tks

ab · 2014-09-07 07:23:39

type
  TMyClass = class
  protected
    fMS: TMemoryStream;
  ...

constructor TMyClass.Create;
begin
  fMS := TMemoryStream.Create;
  fMS.Size := 1024*1024; // enough size
end;

procedure TMyClass.OneMethod;
var i: integer;
     datasize: integer;
     FS: TFileSTream;
begin
  fMS.Position := 0;
  for i := 1 to 1000 do
    fMS.Write(i,sizeof(i));
  datasize := fMS.Position;
  fMS.Position := 0; // rewind
  FS := TFileStream.Create('toto.data',fmCreate);
  FS.CopyFrom(fMS,datasize);
  FS.Free;
end;

Of course, when targeting a TFileSTream, it won't change much.
But if you target a RawByteString or a string, it would change a lot.

See for instance how our TTextWriter class works.

mORMot Open Source

#1 2011-05-20 19:05:56

How to write fast multi-thread Delphi applications

#2 2011-06-05 08:50:49

Re: How to write fast multi-thread Delphi applications

#3 2011-06-09 15:31:30

Re: How to write fast multi-thread Delphi applications

#4 2011-06-09 16:39:42

Re: How to write fast multi-thread Delphi applications

#5 2011-06-10 04:30:52

Re: How to write fast multi-thread Delphi applications

#6 2011-06-13 19:37:04

Re: How to write fast multi-thread Delphi applications

#7 2011-06-14 05:16:54

Re: How to write fast multi-thread Delphi applications

#8 2011-06-14 05:35:35

Re: How to write fast multi-thread Delphi applications

#9 2011-06-14 09:41:32

Re: How to write fast multi-thread Delphi applications

#10 2011-08-09 13:12:05

Re: How to write fast multi-thread Delphi applications

#11 2011-08-09 13:59:52

Re: How to write fast multi-thread Delphi applications

#12 2011-09-22 11:03:18

Re: How to write fast multi-thread Delphi applications

#13 2011-09-22 14:31:34

Re: How to write fast multi-thread Delphi applications

#14 2011-09-22 17:52:15

Re: How to write fast multi-thread Delphi applications

#15 2011-09-22 19:17:45

Re: How to write fast multi-thread Delphi applications

#16 2011-09-23 05:10:43

Re: How to write fast multi-thread Delphi applications

#17 2011-09-23 08:23:58

Re: How to write fast multi-thread Delphi applications

#18 2013-09-09 20:00:50

Re: How to write fast multi-thread Delphi applications

#19 2013-09-10 06:27:31

Re: How to write fast multi-thread Delphi applications

#20 2013-10-24 12:40:12

Re: How to write fast multi-thread Delphi applications

#21 2013-10-24 13:47:01

Re: How to write fast multi-thread Delphi applications

#22 2013-10-24 13:47:58

Re: How to write fast multi-thread Delphi applications

#23 2013-10-24 15:14:43

Re: How to write fast multi-thread Delphi applications

#24 2013-10-24 15:43:09

Re: How to write fast multi-thread Delphi applications

#25 2013-10-24 17:26:20

Re: How to write fast multi-thread Delphi applications

#26 2013-10-24 20:11:11

Re: How to write fast multi-thread Delphi applications

#27 2013-10-25 06:38:11

Re: How to write fast multi-thread Delphi applications

#28 2013-10-25 07:07:04

Re: How to write fast multi-thread Delphi applications

#29 2013-10-25 13:25:17

Re: How to write fast multi-thread Delphi applications

#30 2013-10-25 14:36:21

Re: How to write fast multi-thread Delphi applications

#31 2013-10-25 14:51:55

Re: How to write fast multi-thread Delphi applications

#32 2013-10-25 15:02:40

Re: How to write fast multi-thread Delphi applications

#33 2013-10-26 14:50:42

Re: How to write fast multi-thread Delphi applications

#34 2013-10-26 15:10:20

Re: How to write fast multi-thread Delphi applications

#35 2013-10-26 15:12:57

Re: How to write fast multi-thread Delphi applications

#36 2013-10-26 15:22:20

Re: How to write fast multi-thread Delphi applications

#37 2013-10-26 15:27:53

Re: How to write fast multi-thread Delphi applications

#38 2013-10-26 15:37:08

Re: How to write fast multi-thread Delphi applications

#39 2013-10-26 16:05:45

Re: How to write fast multi-thread Delphi applications

#40 2014-07-11 12:44:29