#1 mORMot 1 » SynCommon comment issue » 2018-05-10 06:24:14

DDGG
Replies: 1

I found some document issue in SynCommons.pas start from the line 14309:

/// same as VarIsEmpty(V) or VarIsEmpty(V), but faster
// - we also discovered some issues with FPC's Variants unit, so this function
// may be used even in end-user cross-compiler code
function VarIsEmptyOrNull(const V: Variant): Boolean;
  {$ifdef HASINLINE}inline;{$endif}

/// same as VarIsEmpty(PVariant(V)^) or VarIsEmpty(PVariant(V)^), but faster
// - we also discovered some issues with FPC's Variants unit, so this function
// may be used even in end-user cross-compiler code
function VarDataIsEmptyOrNull(VarData: pointer): Boolean;
  {$ifdef HASINLINE}inline;{$endif}

Best regards.

#2 Re: mORMot 1 » "Commands out of sync" error while 2nd call to a stored procedure » 2018-04-04 04:49:57

Thanks for your reply! @ab

We plan to gradually move from the stored procedure to using ORM, but currently we can not discard all legacy stored procedure.
And we want to use the mORMot framework, so this is the problem I have to face.

This is the step to reproduce the problem:

Create a new MySQL database named test, then create a new table named user:

CREATE TABLE `user` (
  `id` bigint(255) NOT NULL AUTO_INCREMENT,
  `name` varchar(50) NOT NULL,
  `age` int(11) NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Create a stored procedure named GetUserByLevel:

CREATE DEFINER=`root`@`%` PROCEDURE `GetUserByLevel`(IN `ALevel` int(4))
BEGIN
  IF ALevel = 0 THEN
    SELECT `name`, `age` FROM `user` WHERE `age` BETWEEN 0 AND 6 ORDER BY `age`;
  ELSEIF ALevel = 1 THEN
    SELECT `name`, `age` FROM `user` WHERE `age` BETWEEN 6 AND 18 ORDER BY `age`;
  ELSE
    SELECT `name`, `age` FROM `user` WHERE `age` > 18 ORDER BY `age`; 
  END IF;

END

Then run this simple program:

program Project11;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils, SynCommons, SynDB, SynDBODBC;

var
  Props: TODBCConnectionProperties;

procedure InsertRecords;
begin
  Props.Execute('TRUNCATE TABLE user', []);
  Props.Execute('INSERT INTO user (name, age) VALUES (?, ?)', ['John', 1]);
  Props.Execute('INSERT INTO user (name, age) VALUES (?, ?)', ['Mary', 10]);
  Props.Execute('INSERT INTO user (name, age) VALUES (?, ?)', ['Lucas', 20]);
end;

procedure CallStoredProcedure;
var
  Row: ISQLDBRows;
begin
  Row := Props.Execute('CALL GetUserByLevel(?)', [1]);
  while Row.Step do
    Writeln(FormatUTF8('name: %, age: %', [Row['name'], Row['age']]));
end;

const
  CONN_STRING = 'Driver=MySQL ODBC 5.3 UNICODE Driver;Database=test;Server=127.0.0.1;Port=3306;UID=test;Pwd=';
begin
  Props := TODBCConnectionProperties.Create('', CONN_STRING, '', '');
  try
    try
      InsertRecords;                 // If comment this line
      CallStoredProcedure;
      CallStoredProcedure;           // error occurs
    except
      on E: Exception do
        Writeln(E.ClassName, ': ', E.Message);
    end;
  finally
    Props.Free;
  end;
end.

After the first run (no error occurs), comment the line `InsertRecords`, run it again, then error occurs.

I need your help, thanks a lot!

#3 Re: mORMot 1 » "Commands out of sync" error while 2nd call to a stored procedure » 2018-04-02 11:21:38

Could someone please tell me how to release things which should be release in above situation, so that I can continue to use the same TODBCConnectionProperties?

Sometimes I can't use a new TODBCConnectionProperties because I have to make the call to a stored procedure in a single transaction. sad

#5 mORMot 1 » How to change property-names of a Variant efficiently? » 2018-03-13 16:09:55

DDGG
Replies: 2

I want to change some property-names of a Variant, but after a search I have not found such a function, so I decide to write one:

program Project7;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils, SynCommons;

function VariantChangeNames(var V: Variant; const FromNames, ToNames: TRawUTF8DynArray): Boolean;
var
  Name: RawUTF8;
  PData: PDocVariantData;
  Val: Variant;
  i, k: Integer;
begin
  Result := False;
  PData := DocVariantData(V);
  i := 0;
  for Name in FromNames do
  begin
    k := PData^.GetValueIndex(Name);
    if k <> -1 then
    begin
      Val := PData^.Value[k];
      PData^.Value[ToNames[i]] := Val;
      PData^.Delete(k); // If delete before setting value, Val will be incorrect.
      Result := True;
    end;
    Inc(i);
  end;
end;

var
  V: Variant;
begin
  V := TDocVariant.New();
  V.Name := 'John';
  V.Age := 20;
  VariantChangeNames(V, ['Name'], ['RealName']);
  Writeln(VariantToString(V)); // {"Age":20,"RealName":"John"}
end.

VariantChangeNames I wrote is not work efficiently since it needs to do both delete and add, and there is a side effect: the order of names is disrupted.
It seems that directly modify the VName member of the TDocVariantData is the most efficiently way, but it's a private member.

Any advice? Thanks in advance.

#6 Re: mORMot 1 » Does THttpApiServer is single-threaded? » 2018-03-13 02:17:12

edwinsn wrote:

No, I didn't mean you need to re-implement/duplicate the code logic from the stored procedure, you just call it from within the service class. What I mean is that, if I understand you correctly you don't have to manually do the following:

...

if you use TSQLRestServer instead.

Okay, I will continue learning, thank you!

#7 Re: mORMot 1 » Does THttpApiServer is single-threaded? » 2018-03-09 11:05:11

edwinsn wrote:

For approach #1, you can take a look at https://github.com/synopse/mORMot/blob/ … Server.dpr

See that? You don't even need the TSQLRecord class or the SQLITE engine.

Thanks for your detailed instructions, I have learned the sample you mentioned.

Do you mean I need to re-implement a GetUserLoginInfo service by Delphi code? No, I just want to re-use those legacy stored procedure in database.

So I'm writing a service class and some methods simply call the underlying stored procedure (e.g. GetUserLoginInfo).
I'm going to convert the stored procedure's result from variant to object for temporary storage and processing, and convert it to JSON for a HTTP response. (I know there is a FetchAllAsJSON but I do sure need an object)

#8 Re: mORMot 1 » Does THttpApiServer is single-threaded? » 2018-03-09 08:58:58

@edwinsn Although I don’t quite understand you said, I'm still grateful!

@ab I seem to understand: Because TSynLog.Family has been called in the main thread, so when TSynLog.Add called in TMyServer.Process with multi-threads it just simply execute result := f.SynLog; (it is a Singleton design pattern). is it?
Thanks.

I don't doubt the framework's multithreaded security design, just worry that I'm using it incorrectly...

#9 Re: mORMot 1 » Does THttpApiServer is single-threaded? » 2018-03-09 05:55:28

ab wrote:

Also note that your use of TSynLog is plain wrong.
Each TSynLog instance should be used in a single thread, locally.
...

I will use it in accordance with your instructions, thanks!

BTW: I saw the implementation of the TSynLog.Add, it seems not thread safe, is it?


class function TSynLog.Add: TSynLog;
{$ifdef HASINLINENOTX86}
var f: TSynLogFamily;
begin
  if Self<>nil then begin // inlined TSynLog.Family (Add is already inlined)
    f := PPointer(PtrInt(Self)+vmtAutoTable)^;
    if f=nil then
     result := FamilyCreate.SynLog else
     result := f.SynLog;
  end else
    result := nil;
end;
{$else}
asm
  call TSynLog.Family
  jmp TSynLogFamily.SynLog
end;

Thanks.

#10 Re: mORMot 1 » Does THttpApiServer is single-threaded? » 2018-03-09 04:02:21

Thank you very much!  fServer.Clone() works great!

I have considered TSQLHttpServer, but it takes more learning.
According to my current understanding, it seems that TSQLHttpServer needs to work with a TSQLRestServer together,
and the TSQLRestServer needs to work with TSQLRecord which needs to associate database tables (and record columns with published properties) exactly.

But my project have plenty of legacy stored procedures. e.g. a GetUserLoginInfo procedure accepts a username and password then do validation and logs to an authorization log table internally, this procedure returns an multi-table joined query result (not only the user table stuffs). So maybe I could not declare a TSQLRecord based TUser class exactly maps to existed User table, since the result of GetUserLoginInfo is mixed with multi-tables fields. I don't know how to fetch an multi-table joined query result into inter-related TSQLRecord objects till now.

Please forgive my poor understanding and description...

Your documents is a great, it talks about not only the use of framework classes, but also advanced software architecture ideology.
So it deserves repeated readings, but it takes a lot of time for me, I wish there will be more practical examples.
Thanks to all who work for the great mORMot framework!

#11 Re: mORMot 1 » Does THttpApiServer is single-threaded? » 2018-03-08 03:01:48

ab wrote:

No, it uses a thread pool, of 32 threads by default.

I guess the blocking is on the client side.


As the simple program above, I use a test-tool named ab (ApacheBench) to test it, the option -c5 means to use five requests to make at a time.

C:\Users\user>ab -n10 -c5 http://127.0.0.1:8080/root
This is ApacheBench, Version 2.3 <$Revision: 1663405 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient).....done


Server Software:        mORMot
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /root
Document Length:        2 bytes

Concurrency Level:      5
Time taken for tests:   30.011 seconds
Complete requests:      10
Failed requests:        0
Total transferred:      1840 bytes
HTML transferred:       20 bytes
Requests per second:    0.33 [#/sec] (mean)
Time per request:       15005.250 [ms] (mean)
Time per request:       3001.050 [ms] (mean, across all concurrent requests)
Transfer rate:          0.06 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        1    1   0.4      1       2
Processing:  3002 12000 4472.0  15001   15002
Waiting:     3002 12000 4472.0  15001   15002
Total:       3003 12002 4471.7  15002   15002

Percentage of the requests served within a certain time (ms)
  50%  15002
  66%  15002
  75%  15002
  80%  15002
  90%  15002
  95%  15002
  98%  15002
  99%  15002
 100%  15002 (longest request)

C:\Users\user>

And the program output as following:

20180308 02325142 info  : server start!
20180308 02325428 debug 2018-03-08 02:32:54 : GET /root
20180308 02325728 debug 2018-03-08 02:32:57 : GET /root => OK
20180308 02325728 debug 2018-03-08 02:32:57 : GET /root
20180308 02330028 debug 2018-03-08 02:33:00 : GET /root => OK
20180308 02330028 debug 2018-03-08 02:33:00 : GET /root
20180308 02330328 debug 2018-03-08 02:33:03 : GET /root => OK
20180308 02330328 debug 2018-03-08 02:33:03 : GET /root
20180308 02330628 debug 2018-03-08 02:33:06 : GET /root => OK
20180308 02330628 debug 2018-03-08 02:33:06 : GET /root
20180308 02330928 debug 2018-03-08 02:33:09 : GET /root => OK
20180308 02330928 debug 2018-03-08 02:33:09 : GET /root
20180308 02331228 debug 2018-03-08 02:33:12 : GET /root => OK
20180308 02331228 debug 2018-03-08 02:33:12 : GET /root
20180308 02331528 debug 2018-03-08 02:33:15 : GET /root => OK
20180308 02331528 debug 2018-03-08 02:33:15 : GET /root
20180308 02331828 debug 2018-03-08 02:33:18 : GET /root => OK
20180308 02331828 debug 2018-03-08 02:33:18 : GET /root
20180308 02332128 debug 2018-03-08 02:33:21 : GET /root => OK
20180308 02332128 debug 2018-03-08 02:33:21 : GET /root
20180308 02332428 debug 2018-03-08 02:33:24 : GET /root => OK

It's obvious that the requests were processed one-by-one.
What should I do? Thanks a lot!

#12 Re: mORMot 1 » Does THttpApiServer is single-threaded? » 2018-03-08 02:51:22

Thanks for your reply!

I wrote a simple console application to test it.

program TestHttpApiServer;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils, SynCommons, SynCrtSock, SynLog;

type
  TMyServer = class
    constructor Create;
    destructor Destroy; override;
  private
    fServer: THttpApiServer;
    function Process(Ctxt: THttpServerRequest): Cardinal;
  public
    procedure Start;
  end;

var
  ILog: ISynLog;
  Server: TMyServer;

{ TMyServer }

constructor TMyServer.Create;
begin
  fServer := THttpApiServer.Create(True);
  fServer.OnRequest := Process;
  fServer.AddUrl('root', '8080', False, '+', True);
end;

destructor TMyServer.Destroy;
begin
  fServer.Free;
  inherited;
end;

function TMyServer.Process(Ctxt: THttpServerRequest): Cardinal;
begin
  ILog.Log(sllDebug, '% : % %', [NowUTCToString, Ctxt.Method, Ctxt.URL]);

  Sleep(3000);

  ILog.Log(sllDebug, '% : % % => %', [NowUTCToString, Ctxt.Method, Ctxt.URL, 'OK']);
  Ctxt.OutContent := 'OK';
  Result := 200;
end;

procedure TMyServer.Start;
begin
  fServer.Start;
  ILog.Log(sllInfo, ': server start!');
end;

begin
  with TSynLog.Family do
  begin
    Level := LOG_VERBOSE;
    EchoToConsole := LOG_VERBOSE;
    PerThreadLog := ptOneFilePerThread;
  end;
  ILog := TSynLog.Add;

  Server := TMyServer.Create;
  Server.Start;

  Readln;
end.

Could you please give me some advice?

#13 mORMot 1 » Does THttpApiServer is single-threaded? » 2018-03-07 11:04:49

DDGG
Replies: 15

I'm fairly new to mORMot, so please bear with me if my question seems stupid.

I'm using THttpApiServer to build an API server, and I found out that if I make a response for a long time (e.g. Sleep(3000) in the OnProcess callback), the follow-up requests will blocking.
So does THttpApiServer is a single-threaded server? If it is, what should I do if I need to handle multiple requests in a moment?

I'm not good at English, sorry for that...
Thanks in advance!

#15 Re: mORMot 1 » "Commands out of sync" error while 2nd call to a stored procedure » 2018-02-22 10:56:16

@EgonHugeist

Thanks for your reply!
So it's ok to create a new TODBCConnectionProperties every time I need to query something?
Zeos is strange to me.

#16 Re: mORMot 1 » "Commands out of sync" error while 2nd call to a stored procedure » 2018-02-21 10:49:24

mpv wrote:

ODBC is not a good choice for mySql.  Try Zeos instead

I agree with you, but this is not an ODBC issue I think.

Any other advice?

#18 mORMot 1 » "Commands out of sync" error while 2nd call to a stored procedure » 2018-02-13 06:45:18

DDGG
Replies: 11

Hello everyone!

I am a newbie of mORMot, and I have a question about calling stored procedure by a TODBCConnectionProperties.

Here is my simple code:

const
  CONN_STRING = 'Driver=MySQL ODBC 5.3 UNICODE Driver;Database=Test;Server=127.0.0.1;Port=3306;UID=root;Pwd=';

var
  Props: TODBCConnectionProperties;

procedure TestCallStoreProcedure;
var
  Row: ISQLDBRows;
  S: RawUTF8;
begin
  Row := Props.Execute('CALL GetUserByID(?)', [1]);
  while Row.Step do
  begin
    S := ToUTF8(Row['name']);
    Writeln(S);
  end;
end;

begin
  Props := TODBCConnectionProperties.Create('', CONN_STRING, '', '');
  try
    try
      TestCallStoreProcedure;
      TestCallStoreProcedure;    // error occurs
    except
      on E: Exception do
        Writeln(E.ClassName, ': ', E.Message);
    end;
  finally
    Props.Free;
  end;
end.

After run, I get a "Commands out of sync" error:

EODBCException: TODBCStatement - TODBCLib error: [HY000] [MySQL][ODBC 5.3(w) Driver][mysqld-5.7.18-log]Commands out of sync; you can't run this command now (2014)

I searched for the "commands out of sync;" on the internet, it seems something needs to freed before the next call.

Row is an interface variant so need not to be freed manually I think, I tried to set Row to nil and surely take no effect.

Props means one of connections in the underlying database connection pool, it should be used continuously I think, but after I make the Props be a local variant (create and free it in TestCallStoreProcedure), the issue solved.

But is this the right approach? Please help to understand, thanks a lot!

Board footer

Powered by FluxBB