Multithreaded data retreival...

Amir · 2014-04-08 12:43:21

Hi dear Arnaud and THANKS FOR SUCH A GREAT FRAMEWORK...

I decide to use mORMot in my recent project and I've started to do some excercises to get fluent with mORMots' language !!
Now I've encountered a problem with a very simple test-app; this app uses TSQLRestClient.Retreive method with the definite signature of TSQLRestClient_Instance.Retreive(aID, TSQLRecord_instance, FALSE); to retreive data within background threads simultanously.
When I adapt the code to run within the main thread, everything sounds well; but when I switch to spread the retreival-tasks within threads, sometimes (NOT ALWAYS), it fails. I'd like to know if there are any notes or advices that I should consider or even this approach is a working-solution or I have to implement my retreival-task as a service and consume it at client-side using consumer-threads instead ?

(I'm using the latest version of mORMot framework (1.8.x) + its SQLite kernel as data-store, and I've tested my sample app in HTTP Client-Server and also Windows-Messages architectures and the problem still persists.).

Thanks.

ab · 2014-04-08 12:58:00

What are your exact client and server classes?

MultiThread access is tested and validated by the integrated regression tests (TTestMultiThreadProcess) and by the "21 - HTTP Client-Server performance" sample programs.
See http://blog.synopse.info/post/2013/09/1 … -of-mORMot

Amir · 2014-04-09 11:32:56

"21 - HTTP Client-Server performance" Sample compiles with Delphi XE5 update 2; but when running, after clicking the "Start" button, an access violation halts the program execution.

The debugger points to "SynSelfTests.pas" at line #10207

{$ifdef LVCL}
if fDatabase.AquireWriteMode = amMainThread then ....
//..
//..
//..
{$endif

as LVCL is not defined in my test-case and well, the fDatabase has not been initialized; because the fDatabase is to be Created in the section

if fClientOnlyServerIP = '' then begin
  // ...
  fDatabase := TSQLRestServerDB.Create(.......);
  // the rest of operations...
end;

and the "ClientOnlyServerIP" property is initialized in the "OnClick event of Start button" to be localhost ip-address, so the above-mentioned condition never matches and so the fDatabase is not instanciated with a TSQLRestServerDB.

it seems the first if-block should be embraced by the second one or some extra checks should be considered or ....

Thanks.

Amir · 2014-04-09 11:58:37

To show my exact problem I have changed sample "04 - HTTP Client-Server" a little - ofcourse only the "Project04Client" part is modified and the server-application will stay unchanged - to help with reraising the exception that I'm suffering from.

here is the modified version of unit1.dfm

object Form1: TForm1
  Left = 604
  Top = 370
  BorderStyle = bsSingle
  ClientHeight = 416
  ClientWidth = 490
  Color = clBtnFace
  Font.Charset = DEFAULT_CHARSET
  Font.Color = clWindowText
  Font.Height = -13
  Font.Name = 'Tahoma'
  Font.Style = []
  OldCreateOrder = False
  OnCreate = FormCreate
  OnDestroy = FormDestroy
  PixelsPerInch = 96
  TextHeight = 16
  object Label1: TLabel
    Left = 40
    Top = 16
    Width = 67
    Height = 16
    Caption = 'Your name:'
  end
  object Label2: TLabel
    Left = 40
    Top = 72
    Width = 86
    Height = 16
    Caption = 'Your message:'
  end
  object Label3: TLabel
    Left = 232
    Top = 192
    Width = 37
    Height = 16
    Caption = 'Label3'
  end
  object Label4: TLabel
    Left = 48
    Top = 267
    Width = 393
    Height = 48
    AutoSize = False
    Caption = 
      'PLEASE run this application in debugging-mode within the IDE, To' +
      ' reveal and catch the OS ERROR #12019 in SynCrtSock unit which I' +
      #39'm suffering from.'
    Font.Charset = DEFAULT_CHARSET
    Font.Color = clRed
    Font.Height = -13
    Font.Name = 'Tahoma'
    Font.Style = []
    ParentFont = False
    ShowAccelChar = False
    WordWrap = True
  end
  object Label5: TLabel
    Left = 48
    Top = 326
    Width = 393
    Height = 54
    AutoSize = False
    Caption = 
      'I tried to show the exact system error number and its correspond' +
      'ing message, but its not simply possible;  as it may be overwrit' +
      'ten with other consequent errors or even reset.'
    Font.Charset = DEFAULT_CHARSET
    Font.Color = clWindowText
    Font.Height = -13
    Font.Name = 'Tahoma'
    Font.Style = []
    ParentFont = False
    ShowAccelChar = False
    WordWrap = True
  end
  object QuestionMemo: TMemo
    Left = 32
    Top = 88
    Width = 409
    Height = 121
    ScrollBars = ssVertical
    TabOrder = 0
  end
  object NameEdit: TEdit
    Left = 32
    Top = 32
    Width = 217
    Height = 24
    TabOrder = 1
  end
  object AddButton: TButton
    Left = 48
    Top = 232
    Width = 145
    Height = 25
    Caption = 'Add the message'
    TabOrder = 2
    OnClick = AddButtonClick
  end
  object QuitButton: TButton
    Left = 312
    Top = 232
    Width = 129
    Height = 25
    Caption = 'Quit'
    TabOrder = 3
    OnClick = QuitButtonClick
  end
  object FindButton: TButton
    Left = 256
    Top = 32
    Width = 185
    Height = 25
    Caption = 'Find a previous message'
    TabOrder = 4
    OnClick = FindButtonClick
  end
  object Button1: TButton
    Left = 48
    Top = 384
    Width = 393
    Height = 25
    Caption = 'Please add a message and check this'
    TabOrder = 5
    OnClick = Button1Click
  end
end

and this is the modified version of unit1.pas

unit Unit1;

interface

uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, StdCtrls,
  SynCommons, mORMot,
  SampleData, System.Contnrs;

type

  TTestThread = class(TThread)
  private
    P: TSQLSampleRecord;
    FOccured: Boolean;
  protected
    procedure Execute; override;
  public
    destructor Destroy; override;
  end;

  TForm1 = class(TForm)
    QuestionMemo: TMemo;
    NameEdit: TEdit;
    Label1: TLabel;
    Label2: TLabel;
    AddButton: TButton;
    QuitButton: TButton;
    FindButton: TButton;
    Button1: TButton;
    Label3: TLabel;
    Label4: TLabel;
    Label5: TLabel;
    procedure AddButtonClick(Sender: TObject);
    procedure FormDestroy(Sender: TObject);
    procedure QuitButtonClick(Sender: TObject);
    procedure FindButtonClick(Sender: TObject);
    procedure FormCreate(Sender: TObject);
    procedure Button1Click(Sender: TObject);
  private
    FList: TObjectList;
  public
    Database: TSQLRest;
    Model: TSQLModel;
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}

{ TTestThread }

destructor TTestThread.Destroy;
begin
  inherited Destroy;
  P.Free;
end;

procedure TTestThread.Execute;
var LError: Cardinal;
begin
  if P = nil then
    P := TSQLSampleRecord.Create;
  while not (Terminated or FOccured) do begin
    Form1.Database.Retrieve(1, P, False);
    LError := GetLastError;
    if LError <> ERROR_SUCCESS then begin
      FOccured := True;
      Synchronize(
        procedure
        begin
          Form1.QuestionMemo.Lines.Add(Format('Error Code: %d'#13#10'Error Message: %s'#13#10'=========',
              [LError, SysErrorMessage(LError)]));
        end
      );
    end;
  end;
end;

procedure TForm1.AddButtonClick(Sender: TObject);
var Rec: TSQLSampleRecord;
begin
  Rec := TSQLSampleRecord.Create;
  try
    // we use explicit StringToUTF8() for conversion below
    // a real application should use TLanguageFile.StringToUTF8() in mORMoti18n
    Rec.Name := StringToUTF8(NameEdit.Text);
    Rec.Question := StringToUTF8(QuestionMemo.Text);
    if Database.Add(Rec,true)=0 then
      ShowMessage('Error adding the data') else begin
      NameEdit.Text := '';
      QuestionMemo.Text := '';
      NameEdit.SetFocus;
    end;
  finally
    Rec.Free;
  end;
end;

procedure TForm1.FormDestroy(Sender: TObject);
begin
  FList.Free;
  Database.Free;
  Model.Free;
end;

procedure TForm1.QuitButtonClick(Sender: TObject);
begin
  Close;
end;

procedure TForm1.Button1Click(Sender: TObject);
var I: NativeInt;
begin
  for I := 0 to 9 do
    FList.Add(TTestThread.Create);
end;

procedure TForm1.FindButtonClick(Sender: TObject);
var Rec: TSQLSampleRecord;
begin
  Rec := TSQLSampleRecord.Create(Database,'Name=?',[StringToUTF8(NameEdit.Text)]);
  try
    if Rec.ID=0 then
      QuestionMemo.Text := 'Not found' else
      QuestionMemo.Text := UTF8ToString(Rec.Question);
  finally
    Rec.Free;
  end;
end;

procedure TForm1.FormCreate(Sender: TObject);
begin
  Model := CreateSampleModel; // from SampleData unit
  FList := TObjectList.Create(True);
end;

end.

Please run the client application with debugging enabled within the IDE to let the debugger reveal the OS error #12019 as mORMot's SynCrtSock.pas seems fault-tolerant enough to hide that.

Please let me know if something is wrong here;

Thanks in advance.

Last edited by Amir (2014-04-09 12:01:58)

ab · 2014-04-09 12:10:56

Are you sure the server is running the http.sys server?
The socket-based HTTP server is less robust, and won't scale as such.
Ensure you run the server with administrator rights (right click + "Run as Administrator"), so that the http.sys API will be used.

Amir · 2014-04-09 15:27:14

Yes, I had run the server with admin privileges several times, not only the server but also the client app
I'm using win 8.1, I doubt if something within this new windows version causes that, I'll do test on win 7 and XP and post the feedback ASAP.
But one more thing that has messed up my mind is that the same problem occurs when using windows message-mode !! I think this mode should be well thread-safe and scalable...
I may be doing something wrong, would be so thankful if you help me solve this.

Thanks.

ab · 2014-04-09 17:00:54

Perhaps there was some regression introduced by latest commits.

I'll check sample 21 and come back here.

Edit
There was indeed a small regression in sample 21.
See http://synopse.info/fossil/info/03691f2091

Amir · 2014-04-10 06:24:58

Hi ...
I tested the "04 - HTTP Client-Server" as modified like what you see in previous post in windows XP and 7 as well (and always the applications had been running with admin privileges), and they all behaved the same and the problem still persists.

Would you please help me find where I'm going wrong ?

ab · 2014-04-10 14:14:42

AFAIR the TSQLHttpClient class is NOT supposed to be thread safe.
So you are just expect to NOT share the same DataBase: TSQLRestClientURI instance among several threads.

You need either to protect its execution via mutexes (critical sections), either use one client per thread.
The 2nd solution (one client per thread) is what is done with sample 21.

If you want your client application to be responsive, you can run all HTTP requests on a background thread.
See http://blog.synopse.info/post/2013/07/0 … responsive

I'll modify the code to let TSQLHttpClient be as thread-safe as TSQLRestClientDB, TSQLRestClientURINamedPipe and TSQLRestClientURIMessage.
Stay tuned!

Edit:

All TSQLHttpClient* classes are now thread-safe (i.e. protected by a global mutex, as other TSQLRestClientURI implementations already did).
See http://synopse.info/fossil/info/7719d8274d
Now your updated "Sample 4" program works without any error.

As soon as there is a program to reproduce the problem, it is most of the time easy to fix it!
Thanks a lot for the feedback!

Amir · 2014-04-13 09:40:38

Thanks alot. well working.
But is there any difference performance-wise between the use of dedicated client per thread and this new threadsafe implementation in general ?
I think there should be no (significant) difference with use of SQLite kernel but for other engines like MS SQL Server or Oracle, using a seperate client per thread may provide better scaling; Is it right ?

Thanks.

Last edited by Amir (2014-04-13 09:44:06)

ab · 2014-04-13 10:05:00

What is slow here is the network latency over http, not the db back end.

One client per thread won't block so will be faster. But for ui clients it does not make much sense.

mORMot Open Source

#1 2014-04-08 12:43:21

Multithreaded data retreival...

#2 2014-04-08 12:58:00

Re: Multithreaded data retreival...

#3 2014-04-09 11:32:56

Re: Multithreaded data retreival...

#4 2014-04-09 11:58:37

Re: Multithreaded data retreival...

#5 2014-04-09 12:10:56

Re: Multithreaded data retreival...

#6 2014-04-09 15:27:14

Re: Multithreaded data retreival...

#7 2014-04-09 17:00:54

Re: Multithreaded data retreival...

#8 2014-04-10 06:24:58

Re: Multithreaded data retreival...

#9 2014-04-10 14:14:42

Re: Multithreaded data retreival...

#10 2014-04-13 09:40:38

Re: Multithreaded data retreival...

#11 2014-04-13 10:05:00

Re: Multithreaded data retreival...

Board footer