mORMot Open Source

Hi all
Could any one witness any compiled-code performance enhancements in comparison with previous versions (10-10.1.x) !!??
At least in Win32 compiler, as for Win64 the compilation process always fails as noticed..

And dear Arnaud,
It would be very thankful if you let us know about the quality of generated code with the very new LLVM-based Linux compiler, as I remembered, you mentioned somewhere the final generated code produced using Delphi mobile compilers are not so optimized; may be due to unqualified settings applied to the LLVM-backend...
I'd like to know if this new server-support-release also suffers from the same performance issue !!??

Thanks in advance.

Hi,

"Ping" uses an ICMP echo, so it is expected to work with any active connection regardless of being run within a network-tunnel (if we assume there would be NO blocking firewall-roles or route-maps). As the problem is basically the web-socket itself, we should send the traffic directly through the currently upgraded web-socket and get the result(s) back; in order to validate its functionality.

You have introduced a method to check the server availability as "TSQLRestClientURI.ServerTimeStampSynchronize()" which performs fast and simple pulling. After the web-socket's handshake - which is HTTP-based -, the rest of connection semantics uses web-socket's standards; so if the "upgrade" is satisfied, any other traffic should be transferred in web-socket's way. Therefore "ServerTimeStampSynchronize()" should fulfill the need.

What is your idea ??

As you may know https://www.websocket.org/echo.html implements an echo-server using HTML5 web-sockets. Here you can use a VPN (using different tunneling protocols) to reveal the problem, as the problem is also noticed on the page.

To be honest I think implementing HTML5 Web-sockets was a very nice try, but unfortunately it is not so accepted by many active network-devices' configurations, simply because it is very young and not widely used, today.
This great technology works nice on a local-host, but may not work as expected in a real world use-case.

For example; consider xDSL users which are connected to their ISPs using a tunnel (very common), they simply may have problems with web-socket connections and also offices/organizations which use proxies and different network tunneling services for different purposes (Security, Segmentation and Net. Traffic management) and ... They all may have unsuccessful experiment with their web-sockets-powered apps.

As I mentioned in my earlier post, secure web-sockets (wss://) using TLS may fail in very rare situations and seem to be operational in real world projects. so if we want to have a safe and working web-socket service with today's common network settings, we'd better use secure web-sockets i.e: over TLS.

But as you yourself noticed, IMHO the best and always working solution to implement a bidirectional, asynchronous, and REST-friendly publish-subscribe collaboration mechanism, for today's mORMot; is hiring a message queue system, as the great, light, cross-platform, open-source and of course very fast "0MQ" - which you also liked.

...and as always, thanks for you great work.

Best.

Thanks for your notice... But I was very excited about introducing another great feature that satisfies my current project's need in a very simple way. But I know as a convention, if a road-map listed feature or important-update is completed and well tested, most of time a dedicated blog post is born and "self-tests" group of projects involve new test-cases...

To be honest, I've been familiar with Synopse Tools and mORMot framework since 2010-2011, and I followed its progress periodically by checking the timeline and reading your great blog posts (introducing new features and/or challenging other tools/technologies AND sharing experiences and ideas) and also some forum discussions; And as many other things I decided to use mORMot multiple times and unfortunately this never achieved, but these days I'm feeling deeply integrated with your mORMots' life-cycle !!!, as I'm carefully studding mORMot's documentation (Very detail and intensive !!! Thanks.) as well as some most-visited forum topics in addition to reviewing/reading blog posts. (WOWWW !!! GREAT WORK... REALLY GREAT CROSS-PLATFORM OPEN SOURCE PROJECT IN PASCAL WORLD !!!).

THANKS FOR SUCH A GREAT WORK...

Hi ...
I tested the "04 - HTTP Client-Server" as modified like what you see in previous post in windows XP and 7 as well (and always the applications had been running with admin privileges), and they all behaved the same and the problem still persists.

Would you please help me find where I'm going wrong ?

To show my exact problem I have changed sample "04 - HTTP Client-Server" a little - ofcourse only the "Project04Client" part is modified and the server-application will stay unchanged - to help with reraising the exception that I'm suffering from.

here is the modified version of unit1.dfm

object Form1: TForm1
  Left = 604
  Top = 370
  BorderStyle = bsSingle
  ClientHeight = 416
  ClientWidth = 490
  Color = clBtnFace
  Font.Charset = DEFAULT_CHARSET
  Font.Color = clWindowText
  Font.Height = -13
  Font.Name = 'Tahoma'
  Font.Style = []
  OldCreateOrder = False
  OnCreate = FormCreate
  OnDestroy = FormDestroy
  PixelsPerInch = 96
  TextHeight = 16
  object Label1: TLabel
    Left = 40
    Top = 16
    Width = 67
    Height = 16
    Caption = 'Your name:'
  end
  object Label2: TLabel
    Left = 40
    Top = 72
    Width = 86
    Height = 16
    Caption = 'Your message:'
  end
  object Label3: TLabel
    Left = 232
    Top = 192
    Width = 37
    Height = 16
    Caption = 'Label3'
  end
  object Label4: TLabel
    Left = 48
    Top = 267
    Width = 393
    Height = 48
    AutoSize = False
    Caption = 
      'PLEASE run this application in debugging-mode within the IDE, To' +
      ' reveal and catch the OS ERROR #12019 in SynCrtSock unit which I' +
      #39'm suffering from.'
    Font.Charset = DEFAULT_CHARSET
    Font.Color = clRed
    Font.Height = -13
    Font.Name = 'Tahoma'
    Font.Style = []
    ParentFont = False
    ShowAccelChar = False
    WordWrap = True
  end
  object Label5: TLabel
    Left = 48
    Top = 326
    Width = 393
    Height = 54
    AutoSize = False
    Caption = 
      'I tried to show the exact system error number and its correspond' +
      'ing message, but its not simply possible;  as it may be overwrit' +
      'ten with other consequent errors or even reset.'
    Font.Charset = DEFAULT_CHARSET
    Font.Color = clWindowText
    Font.Height = -13
    Font.Name = 'Tahoma'
    Font.Style = []
    ParentFont = False
    ShowAccelChar = False
    WordWrap = True
  end
  object QuestionMemo: TMemo
    Left = 32
    Top = 88
    Width = 409
    Height = 121
    ScrollBars = ssVertical
    TabOrder = 0
  end
  object NameEdit: TEdit
    Left = 32
    Top = 32
    Width = 217
    Height = 24
    TabOrder = 1
  end
  object AddButton: TButton
    Left = 48
    Top = 232
    Width = 145
    Height = 25
    Caption = 'Add the message'
    TabOrder = 2
    OnClick = AddButtonClick
  end
  object QuitButton: TButton
    Left = 312
    Top = 232
    Width = 129
    Height = 25
    Caption = 'Quit'
    TabOrder = 3
    OnClick = QuitButtonClick
  end
  object FindButton: TButton
    Left = 256
    Top = 32
    Width = 185
    Height = 25
    Caption = 'Find a previous message'
    TabOrder = 4
    OnClick = FindButtonClick
  end
  object Button1: TButton
    Left = 48
    Top = 384
    Width = 393
    Height = 25
    Caption = 'Please add a message and check this'
    TabOrder = 5
    OnClick = Button1Click
  end
end

and this is the modified version of unit1.pas

unit Unit1;

interface

uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, StdCtrls,
  SynCommons, mORMot,
  SampleData, System.Contnrs;

type

  TTestThread = class(TThread)
  private
    P: TSQLSampleRecord;
    FOccured: Boolean;
  protected
    procedure Execute; override;
  public
    destructor Destroy; override;
  end;

  TForm1 = class(TForm)
    QuestionMemo: TMemo;
    NameEdit: TEdit;
    Label1: TLabel;
    Label2: TLabel;
    AddButton: TButton;
    QuitButton: TButton;
    FindButton: TButton;
    Button1: TButton;
    Label3: TLabel;
    Label4: TLabel;
    Label5: TLabel;
    procedure AddButtonClick(Sender: TObject);
    procedure FormDestroy(Sender: TObject);
    procedure QuitButtonClick(Sender: TObject);
    procedure FindButtonClick(Sender: TObject);
    procedure FormCreate(Sender: TObject);
    procedure Button1Click(Sender: TObject);
  private
    FList: TObjectList;
  public
    Database: TSQLRest;
    Model: TSQLModel;
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}

{ TTestThread }

destructor TTestThread.Destroy;
begin
  inherited Destroy;
  P.Free;
end;

procedure TTestThread.Execute;
var LError: Cardinal;
begin
  if P = nil then
    P := TSQLSampleRecord.Create;
  while not (Terminated or FOccured) do begin
    Form1.Database.Retrieve(1, P, False);
    LError := GetLastError;
    if LError <> ERROR_SUCCESS then begin
      FOccured := True;
      Synchronize(
        procedure
        begin
          Form1.QuestionMemo.Lines.Add(Format('Error Code: %d'#13#10'Error Message: %s'#13#10'=========',
              [LError, SysErrorMessage(LError)]));
        end
      );
    end;
  end;
end;

procedure TForm1.AddButtonClick(Sender: TObject);
var Rec: TSQLSampleRecord;
begin
  Rec := TSQLSampleRecord.Create;
  try
    // we use explicit StringToUTF8() for conversion below
    // a real application should use TLanguageFile.StringToUTF8() in mORMoti18n
    Rec.Name := StringToUTF8(NameEdit.Text);
    Rec.Question := StringToUTF8(QuestionMemo.Text);
    if Database.Add(Rec,true)=0 then
      ShowMessage('Error adding the data') else begin
      NameEdit.Text := '';
      QuestionMemo.Text := '';
      NameEdit.SetFocus;
    end;
  finally
    Rec.Free;
  end;
end;

procedure TForm1.FormDestroy(Sender: TObject);
begin
  FList.Free;
  Database.Free;
  Model.Free;
end;

procedure TForm1.QuitButtonClick(Sender: TObject);
begin
  Close;
end;

procedure TForm1.Button1Click(Sender: TObject);
var I: NativeInt;
begin
  for I := 0 to 9 do
    FList.Add(TTestThread.Create);
end;

procedure TForm1.FindButtonClick(Sender: TObject);
var Rec: TSQLSampleRecord;
begin
  Rec := TSQLSampleRecord.Create(Database,'Name=?',[StringToUTF8(NameEdit.Text)]);
  try
    if Rec.ID=0 then
      QuestionMemo.Text := 'Not found' else
      QuestionMemo.Text := UTF8ToString(Rec.Question);
  finally
    Rec.Free;
  end;
end;

procedure TForm1.FormCreate(Sender: TObject);
begin
  Model := CreateSampleModel; // from SampleData unit
  FList := TObjectList.Create(True);
end;

end.

Please run the client application with debugging enabled within the IDE to let the debugger reveal the OS error  #12019 as mORMot's SynCrtSock.pas seems fault-tolerant enough to hide that.

Please let me know if something is wrong here;

Thanks in advance.

Dear Arnaud,

As you know, Delphi visual forms are encoded as DFM files and included into the executables as resources (RT_RCDATA). This implementation causes that DFMs could be simply exposed to the public, and this resource leak might be harmful in some situations.

my main question, concerns about how we can prevent this... and I prefer to do it in a delphi way, not using third parties.

I searched the web and I found many utilities designed to provide secure executables and libraries, but all of them are going to protect and compress whole of executable (or libraries) that is earned with heavy overhead. then I asked this question to find a solution with the least overhead and speed penalty.

and I made a guss it could be possible if I use a compression library (something using LZO algorithms for online decompression with no memory overhead) like "SynLZO.pas" and compress only forms and data modules with password protection, and also decomress it while initializing the forms, this causes to have a smaller executable and also provides a good security.

And I agree with you about the previous topic. but I am a novice programmer and computer studet who encountered lots of problems and questions while challenging the programming languages and utilities, and I try to find the best answer to them. so I look up the answer in some related books in hand or search the web and also post my questions to good forums. and I am not producing a software so I need no protection mechanism, I just wanted to learn and practice.

Thanks.

Thanks...

I remember all your remarks. There are lots of things I should learn... I know nothing about pipelining & ..., beside documents you introduced, I've started to study "The Intel Microprocessors ... 7th Ed - Barry B. Brey". I decide to learn this completely... I love lowlevel

All thanks.

function NumToPersian(const str: string): string;
asm
      test  eax, eax
      xchg  eax, edx
      jz    System.@UStrClr
@@Continue:
      push  edx
      mov   edx, [edx - 4]
      push  edx
      push  eax
      push  edx
      call  System.@UStrClr
      pop   eax
      call  System.@NewUnicodeString
      pop   edx
      mov   [edx], eax
      mov   edx, eax
      pop   ecx
      pop   eax
      shr   ecx, 1
      push  ebx
      jz    @@One
      nop
@@Loop:
      dec   ecx
      mov   ebx, dword ptr [eax]
      lea   ebx, ebx + $06C006C0
      lea   eax, eax + 4
      mov   dword ptr [edx], ebx
      lea   edx, edx + 4
      jnz   @@Loop
@@One:
      mov   ebx, dword ptr [eax]
      test  bx, bx
      jz    @@Quit
      add   ebx, $6C0
      mov   dword ptr [edx], ebx
@@Quit:
      pop   ebx
end;

Hi Dear Arnaud

Here's new code, using UStrClr + NewUnicodeString, which has @@Loop at 4-byte aligned address, I tried to make an aligned loop without adding NOPs...

And about "size"... I was about aligning both address of the loop and size of the loop, this achieved by adding one extra NOP instraction before loop start-point.

It was because when studying "Intel Optimization... PDF" which you introduced, I misunderstood a subject, that after progressing in study I found it.

Sorry.

function NumToPersian(const str: string): string;
asm
      test  eax, eax
      xchg  eax, edx
      jnz   @@Continue
      call System.@UStrClr
      ret
@@Continue:
      push  edx
      mov   edx, [edx - 4]
      push  edx
      push  eax
      push  edx
      call  System.@UStrClr
      pop   eax
      call  System.@NewUnicodeString
      pop   edx
      mov   [edx], eax
      mov   edx, [edx]
      pop   ecx
      pop   eax
      test  ecx, 1
      jz   @@Prepare
      inc   ecx
@@Prepare:
      shr   ecx, 1
      push  ebx
@@Loop:
      dec   ecx
      mov   ebx, dword ptr [eax]
      lea   ebx, ebx + $06C006C0
      lea   eax, eax + 4
      mov   dword ptr [edx], ebx
      lea   edx, edx + 4
      jnz   @@Loop
      pop   ebx
end;

best.

Thanks Dear Arnaud.

I put a breakpoint at the start point of the @@Loop and using the delphi dissembler stopping at that point, I could simply calculate the size of Loop which was 11 bytes, and needed only one NOP instruction, to be DWORD aligned.

      .
      .
      . 
      nop
@@Loop:
      dec   ecx
      mov   ebx, dword ptr [eax]
      lea   ebx, ebx - $06C006C0
      lea   eax, eax + 4
      mov   dword ptr [edx], ebx
      lea   edx, edx + 4
      jnz   @@Loop
      .
      .
      .
      

I'll also replace the UStrSetLength with combination of UStrClr and NewUnicodeString as you advised to optimize the function.

All Thanks for ur help.
Cheers.

Hi ...

Where can I read about data alignment !!! I know nothing about what u say I really like to learn why it is so important.

Know only I can make a guss of what you said about alignment, that I'll change the code and post it again.

Please let me know a complete reference about it.

as you noticed the the dummy code removed from first start of the function:

function NumStrToFa(const str: string): string;
asm
      test  eax, eax
      jz    @@Null
      push  eax
      mov   eax, dword ptr [eax - 4]
      push  eax
      .
      .
      .

The function you mentioned is "UStrClr" which shoulf be applied at the end of the function.
that should manage the emty string explicitly.

   .
   .
   .

@@Null:
      mov   eax, edx               //  preparing the result parameter for "UStrClr" function
      jmp   System.@UStrClr
end;                                  // end of the function declaration

Realllyyyy Thankssssss again.