question about thread-safety in services

danielkuettner · 2014-10-27 09:55:31

Hi Arnaud,

I have a mORMOt server defined like this:

Props := TSQLDBZEOSConnectionProperties.Create(
    TSQLDBZEOSConnectionProperties.URI(dFirebird,'192.168.1.249:3050', 'fbclient.dll',false),
    cDBFILE, '...', '...');
Props.ConnectionTimeOutMinutes:= 1;
RegisterExternalDB;

DB:= TSQLPrcashRestServerDB.Create(Model,cSQLiteFile,false); //no user-auth
DB.DB.LockingMode:= lmExclusive;
DB.DB.Synchronous:= smOff;
DB.CreateMissingTables(0);

//Interfaces
DB.ServiceRegister(THBCI,[TypeInfo(IHBCI)],sicShared).SetOptions(['DoEU'], [optExecinPerInterfaceThread]); <-- with this option no errors

Server := TSQLHttpServer.Create(cHTTP_PORT,[DB],'+',useHttpApiRegisteringURI);
Server.AccessControlAllowOrigin := '*'; // allow cross-site AJAX queries

And the Interface base service is:

procedure THBCI.DoEU(const ID: Integer; const GUID: RawUTF8; const Zweischritt: Boolean; out res: RawUTF8);
var
..
  jtZV, jtDDBAC: TSQlTableJSON;
..
  DDAC_EUL: TDDAC_EUL;
begin
  ..
  jtZV:= DB.MultiFieldValues(TSQLZV, 'Status,AKontoID,EKontoNr,EBLZ,EName,VWZ,Betrag', 'ID=?', [id]); <-- here comes exception when SetOptions is not set like above

..

After reading page 293/294 of your docu I've thought, using of DB is making sql-select and this is thread-safe as using of Props.Execute. These are db-requests and not my own variables.
On page 384 is a hint, to use TRTLCriticalSection mutex to make my code thread-safe.

I want maximum performance and no useless locks.

Is there a way to lock the DB.MultiFieldValues (e.g. TMonitor.Enter or DB.DB.Lock('here comes the select, but how do I get the result'))?

Daniel

ab · 2014-10-27 11:25:29

All ORM methods have their own thread-safety.
Even if optExecinPerInterfaceThread is set, the ORM methods may be run in their own thread.
See "Advanced threading settings" in the SAD 1.18 pdf - and TSQLRest.AcquireExecutionLockedTimeOut[] property.

AFAIK FireBird is not very thread friendly - so perhaps amBackgroundThread is a good idea.
Perhaps it changed in newer Firebird version...

optExecinPerInterfaceThread is not mandatory, if your existing business code is re-entrant (i.e. not using any "shared variables" among calls).
Then take a look at our TAutoLocker class (in SynCommons.pas) if you want easy to use lock mechanism. See also TLockedDocVariant if you want to store some shared data.
The smaller code extend is locked, the better.

danielkuettner · 2014-10-27 12:53:18

First thanks for your tipps and answer.

ab wrote:

Even if optExecinPerInterfaceThread is set, the ORM methods may be run in their own thread.
...
optExecinPerInterfaceThread is not mandatory, if your existing business code is re-entrant (i.e. not using any "shared variables" among calls).

As I wrote in last post I've tested a service with only one call of

jtZV:= DB.MultiFieldValues(TSQLZV, 'Status,AKontoID,EKontoNr,EBLZ,EName,VWZ,Betrag', 'ID=?', [id]); <-- here comes exception when SetOptions is not set like above

Theres no business-code of my own.
Here I get exceptions in ZDbcInterbase6 if [optExecinPerInterfaceThread] is missing. With [optExecinPerInterfaceThread] set there are absolute no exceptions.

amBackgroundThread is not a good idea. This is the slowest solution. All client threads have to wait for the one BackgroundThread of the Firebird-connection.

Please don't understand me wrong: I can live with [optExecinPerInterfaceThread]. In my test (call of service 3 times in a loop of 1-100) it was stable and fast. But your statements doesn't match to that.

Last edited by danielkuettner (2014-10-27 13:03:33)

ab · 2014-10-27 14:13:52

If you set optExecinPerInterfaceThread for the service, you have to leave the default value for AcquireExecutionLockedTimeOut[], i.e. amUnlocked, for the ORM.
Note that by default, AcquireExecutionLockedTimeOut[execORMWrite] will be amLocked.

optExecinPerInterfaceThread + amBackgroundThread has no benefit at all.
Default service option + amBackgroundThread may make sense.

Note that one connection will be done per thread, in SynDBZeos.
Consider this when running your service.

optExecinPerInterfaceThread with default ORM settings sounds indeed like a good option.

BTW all this may sound a bit complicated at first, but is needed.
AFAIK the latest "service" feature of TMS XData does not have any threading settings, as far as the documentation states. Big issue for any serious work, IMHO.

danielkuettner · 2014-10-27 14:51:08

Thanks for your advise.

I think mORMOt is really without competition. To hide this complexity a wizard for creating a new project could be a profitable solution.

You have such a feature in the samples, AFAIR.

E.g. the combination of optExecinPerInterfaceThread with default ORM was for me one line of code but the solution at all. But to get this I've to reed the doc for hours and then making tests (No problem for me, I appreciate the time savings with mORMot at all).

The code of an mORMot project consists of view lines of code. This could be easily done by an wizard.

But what I'm not understand is:

ab wrote:

Note that one connection will be done per thread, in SynDBZeos.
Consider this when running your service.

Does this mean that my result can be changed from another thread? In such I case I would make an DB.DB.Lock() -> my code that works with the result from DB.MultiFieldValues -->DB.DB.Unlock;

ab · 2014-10-27 15:32:02

danielkuettner wrote:

Does this mean that my result can be changed from another thread? In such I case I would make an DB.DB.Lock() -> my code that works with the result from DB.MultiFieldValues -->DB.DB.Unlock;

You are in a multi-thread application. Other threads may access the same data.

The ways to be sure that your data is not currently modified by another thread are several:
- Use a transaction - but would block all other write process;
- Use a lock local to your interface: if your data is mainly used and modified by this interface (following the interface segregation principle), you can protect data for a given "bounded context" (if you follow DDD naming);
- Get your data, then use a TSQLRestBatch to write all modifications at once, with its own transaction - this would be not breaking;
- Use an in-memory working copy of your data, as objects or record, local to your application-layer interface, then use small locks around it, and use ORM methods just for persistence - this is what DDD propose, i.e. put your business logic as class/record in your DDD domain layer, using an aggregate class containing all data of a given bounded context, then call this business logic from your interface, which would be the DDD application layer.

In practice, you should never use DB.DB in your business code, only at application startup if you want to set some SQLite3 specific options (like exclusive mode, or timeout).
Transactions at DB level are a nightmare, in a cross-database environment like mORMot - so they are not handled at DB level, or with a naive - but safe - global lock at TSQLRestServer level if you use BeginTransaction/Commit/RollBack. Even Microsoft and Java did have to deploy huge and complex system to let their ORM be ACID, with RDBMS-like transaction on transverse systems.
mORMot's idea is to stay simple, and be ACID at the DDD's Aggregate boundaries. This is where TSQLRestBach comes in action, with great success about speed and ease of use, IMHO.

ab · 2014-10-27 15:47:06

For the Wizard thing, it is a good idea, but I do not like them very much.
First step is easy, you get your code, but then maintenance is difficult, since you did not get the concepts, but let the wizard do its wizardry for you.

Once you are used to it, defining your data structure as TSQLRecord and your services as interface is IMHO much better, and more productive than a UI modeling system.
Writing an UI for generating complex code is a huge task, far above our own available time.
But some simple tools may be written, e.g. to write TSQLRecord from an existing database, or generate interfaces from BDD scenarios.
I discovered how Mustache is just great to generate code!

A whole set of samples, covering most use case, is perhaps better.
Perhaps the forum may help to find out a list of missing sample code.
I do not speak about huge samples, but small samples only showing one thing at a time, like threading abilities.

Or perhaps one somewhat big sample, showing most techniques and best practices of the framework, with a lot of comments, may be a good idea.
I like how I commented the "30 - MVC Server" sample in the latest version of the SAD pdf.
For BeDelphi in November, I would like to show some real sample in real time. I may write a new "main sample", for a real use case, to show how ORM/SOA (and MVC) is implemented with mORMot.
Reference code is of great interest, IMHO.

moctes · 2014-10-27 15:56:13

For BeDelphi in November, I would like to show some real sample in real time. I may write a new "main sample", for a real use case, to show how ORM/SOA (and MVC) is implemented with mORMot.

Great! I'd like to see that sample, please keep us informed when it happen

danielkuettner · 2014-10-27 16:35:34

I know you don't like wizards, and this word in special (this comes from a Datasnap allergy?). But the success of mORMot depends on a large number of users. And the wizard would take the first step.
"The first cut is the deepest", a song from Sheryl Crow.

I want a mORMOt server with http, no auth, external db (Firebird, Databasename, port ...).
Now after this "two" lines of code are created the server startet as console-app oder VCL-Form with a link to the Browser that displays a JSON of a Table. The user is happy and looks at the samples...

Samples are also good, but the testing of them takes time. Then the sample has the wrong db and the new user is away. Many users in this forum have had a problem to come in.

The Wizard could also download the actual files and - also very important - the sqlite3.obj files. These are all problems for users at first time.

But please belief me, it's only a suggestion. I don't need this, but my way to mORMot would have been easier with that. The best on Datasnap is his wizard.

And belief me one thing: The last thing a new user needs is a big sample.

Thanks for the tip with DB.DB.Lock. Such informations are very useful for me.

mORMot Open Source

#1 2014-10-27 09:55:31

question about thread-safety in services

#2 2014-10-27 11:25:29

Re: question about thread-safety in services

#3 2014-10-27 12:53:18

Re: question about thread-safety in services

#4 2014-10-27 14:13:52

Re: question about thread-safety in services

#5 2014-10-27 14:51:08

Re: question about thread-safety in services

#6 2014-10-27 15:32:02

Re: question about thread-safety in services

#7 2014-10-27 15:47:06

Re: question about thread-safety in services

#8 2014-10-27 15:56:13

Re: question about thread-safety in services

#9 2014-10-27 16:35:34

Re: question about thread-safety in services

Board footer