#1 2016-05-30 11:00:11

George
Member
Registered: 2016-04-05
Posts: 142

JSON RESTful web api from DataSnap to mORMot

Hello!

Currently i use Datasnap for REST implementation.
After i've seen performance comparison between datasnap and mORMot, i want try reimplement same (or almost same) REST api as i've done with DataSnap before.
My target goals is to make test case that will include all features that was used with datasnap.

I'm pretty satisfied how server class module looks in DataSnap (clean business code, many background things works automatically "from the box").
So, here is part of my server class from datasnap:

type

  lsscAuth = (sscAuthNOTRequired, sscAuthRequired);

  TRestNewUser = class(TRestNewUser_session);

  TRestClientInfo = class
    Name: string;
    PreferredLanguage: string;
    sscAPITargetVersion: Double;
    sscLogin: string;
    sscPassword: string;
  end;

  TRestResult<T> = class
    MethodResult: T;
    Code: integer;
    Description: string;
    Comment: string;
  public
    procedure InitResult();
    procedure SetResult(pResCode: integer; pComment: string = ''); overload;
    procedure SetResult(Result: rResult); overload;
    function Success(): boolean;
    function GetDescription(OtherResCode: integer = -1): string;
    function GetReport(): string;
    constructor Create(MethodResult: T; RestClientInfo: TRestClientInfo; MethodName: string; SessionContext: TServerSessionContext); overload;
    constructor Create(MethodResult: T; RestClientInfo: TRestClientInfo; MethodName: string; SessionContext: TServerSessionContext;
      var UserAuthData: rRestUserAuthData); overload;
    destructor Destroy(); override;
  end;

  TGetAPIInfoResult = class
    APIVersion: Double;
    APINode: string;
    MethodsInfo: array of string;
    constructor Create;
    destructor Destroy; override;
  end;

  TRestMethodAttribute = class(TCustomAttribute)
    APIMinSupportedVersion: Double;
    sscAuth: lsscAuth;
  public
    constructor Create(MinSupportedVersion: Double; sscAuth: lsscAuth);
  end;

{$METHODINFO ON}

  TNode1 = class(TDataModule)
  private
    Settings: TSSCSettings;
    LeaveSessionAlive: boolean;
    SessionID: string;
    SessionContext: TServerSessionContext;
    class function GetMethodInfo(MethodName: string): TRestMethodAttribute;
    class function GetMethodsMinSupportedAPIVersions(): TStringList;
  published
    constructor Create(Settings: TSSCSettings); reintroduce;
    destructor Destroy(); override;
  public

    sscApiLogin: string;

    [TRestMethodAttribute(1.1, sscAuthNOTRequired)]
    function CreateSession(RestClientInfo: TRestClientInfo): TRestResult<string>;

    [TRestMethodAttribute(1.1, sscAuthNOTRequired)]
    function DestroySession(RestClientInfo: TRestClientInfo): TRestResult<string>;

    [TRestMethodAttribute(1.1, sscAuthNOTRequired)]
    function GetSessionID(RestClientInfo: TRestClientInfo): TRestResult<string>;

    [TRestMethodAttribute(1.1, sscAuthNOTRequired)]
    function GetAPIInfo(RestClientInfo: TRestClientInfo): TRestResult<TGetAPIInfoResult>;

    [TRestMethodAttribute(1.1, sscAuthNOTRequired)]
    function EchoString(RestClientInfo: TRestClientInfo; Value: string): TRestResult<string>;

    [TRestMethodAttribute(1.1, sscAuthNOTRequired)]
    function CheckUserExists(RestClientInfo: TRestClientInfo; UserID: string): TRestResult<boolean>;

    [TRestMethodAttribute(1.1, sscAuthNOTRequired)]
    function GetUserAuthorizationToken(RestClientInfo: TRestClientInfo; UserID: string): TRestResult<string>;

    [TRestMethodAttribute(1.1, sscAuthNOTRequired)]
    function RegisterNewUser(RestClientInfo: TRestClientInfo; NewUserData: TRestNewUser): TRestResult<string>;

    [TRestMethodAttribute(1.2, sscAuthRequired)]
    function SetRequestStatus(RestClientInfo: TRestClientInfo; RequestNumber, RequestStatusID: string; HTMLMessage: string = ''): TRestResult<string>;

    [TRestMethodAttribute(1.2, sscAuthRequired)]
    function AddRequestMessage(RestClientInfo: TRestClientInfo; RequestNumber, HTMLMessage: string; HiddenFromUser: boolean): TRestResult<string>;

  end;

{$METHODINFO OFF}

No interfaces, only classes and objects.
Same result "TRestResult" (for each method) with dynamic field "MethodResult" that can be represented as simple type like string or complicated object.
TRestResult has no properties. Method attributes contain additional method properties.
Datasnap convert input JSON to corresponding object automatically, as well as result type (even with arrays).

For example, GetAPIInfo method code:

// Returns API info
function TNode1.GetAPIInfo(RestClientInfo: TRestClientInfo): TRestResult<TGetAPIInfoResult>;
const
  MethodName = 'GetAPIInfo';
var
  SL: TStringList;
  i: integer;
begin
  // Here i initialize result type as common TRestResult class with field "MethodResult" as TGetAPIInfoResult.
  Result := TRestResult<TGetAPIInfoResult>.Create(TGetAPIInfoResult.Create, RestClientInfo, MethodName, SessionContext);
  try
    if Result.Success() then
      begin
        // Method body
        Result.MethodResult.APIVersion := const_APIVersion;
        Result.MethodResult.APINode := Self.ClassName;
        SL := GetMethodsMinSupportedAPIVersions();
        SetLength(Result.MethodResult.MethodsInfo, SL.Count);
        for i := 0 to SL.Count - 1 do
          Result.MethodResult.MethodsInfo[i] := SL.Strings[i];
        SL.Free;
      end;
  finally
    // Save log
    SessionContext.restADCAddToExecutionHistory(MethodName, RestClientInfo.Name, sscApiLogin, RestClientInfo.sscLogin, RestClientInfo.sscAPITargetVersion,
      Result.Code);
  end;
end;

Below, JSON input structure:

{
   "_parameters":[
      {
         "type":"NodeOne.TRestClientInfo",
         "id":1,
         "fields":{
            "Name":"Имя моего приложения",
            "PreferredLanguage":"Russian",
            "sscAPITargetVersion":"1.2",
            "sscLogin":"",
            "sscPassword":""
         }
      }
   ]
}

Below, JSON output structure:

{
   "result":[
      {
         "type":"NodeOne.TRestResult<NodeOne.TGetAPIInfoResult>",
         "id":1,
         "fields":{
            "MethodResult":{
               "type":"NodeOne.TGetAPIInfoResult",
               "id":2,
               "fields":{
                  "APIVersion":1.2,
                  "APINode":"TNode1 ",
                  "MethodsInfo":[
                     "GetAPIInfo=1,1",
                     "EchoString=1,1"
                  ]
               }
            },
            "Code":0,
            "Description":"No errors.",
            "Comment":""
         }
      }
   ]
}

So, my goals for mORMot test project are:

  • [done]1. Multi threaded processing.

  • [done]2. Method call via URL.

  • [done]2.1 Send parameters via url.

  • [partially done]2.2 Return custom object as JSON string.

  • [done]2.3 Send parameters as JSON via body.

  • [?]2.4 Send multiple parameters as JSON via body.

  • [under investigation]3. Server side session that allow store data between calls.

  • [under investigation]4. Authentication to allow general access to call methods.

  • [under investigation]4.1 Authorization to execute method.

  • [under investigation]5. HTTPS support.

  • [under investigation]5.1 Custom cert installation.

Current version 1.00 of mORMot test project can be downloaded here.
I will update file time to time. I use JMeter as client application, so JMeterTestPlan.jmx included.

Questions will appear below.

Offline

#2 2016-05-30 14:05:53

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

How to create TSQLHttpServer with disabled listner and then activate or deactivate listner on the fly?

Related to 2.2: Is there a way to serialize objects more similar like in datasnap?
- without property declaration.
- with nested objects
- with arrays (string, integer, etc.)

Related to 2.3: JSONToObject(CustomResult, pointer(ObjectAsJSONstr), ObjectAsJSONstrValid); ObjectAsJSONstrValid = False.
Class:

  TCustomResult = class
  protected
    fResultCode: integer;
    fResultStr: string;
    fResultTimeStamp: TDateTime;
  published
    property ResultCode: integer read fResultCode write fResultCode;
    property ResultStr: string read fResultStr write fResultStr;
    property ResultTimeStamp: TDateTime read fResultTimeStamp write fResultTimeStamp;
  public
    constructor Create();
    destructor Destroy(); override;
  end;

  JSON:
 

{
	 "ClassName":"TCustomResult",
	 "ResultCode":999999,
	 "ResultStr":"Awesome!",
	 "ResultTimeStamp":"2016-05-30T10:40:00"
}

Related to 2.4: I know that i can access to body text via "Ctxt.Call.InBody", but how access to specified parameter when multiple json objects was passed via body data? Only manual parsing?
Something Like this one:
[
   {
      "param1":{
         "ClassName":"TCustomResult",
         "ResultCode":999999,
         "ResultStr":"Awesome!",
         "ResultTimeStamp":"2016-05-30T10:40:00"
      }
   },
   {
      "param2":{
         "ClassName":"TCustomResult",
         "ResultCode":90000,
         "ResultStr":"Wow!",
         "ResultTimeStamp":"2015-05-30T10:30:00"
      }
   }
]

Offline

#3 2016-05-30 14:45:48

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

Please do not post huge pieces of code in the forum.
It is bearly readable, and the FluxBB php forum tends to break the web server when you try to modify the content afterwards.

The best is to fork the project on github.
Or supply a link to a public .zip containing the modified files, or use something like http://pastebin.com/

This is clearly asked in the forum rules:

Rules wrote:

7. Please do not post any huge code or log content in the forum: this would load the server DB for nothing. Rather post your data in a remote storage location, like gist, paste.ee, DropBox or Google Drive.

Offline

#4 2016-05-30 15:00:31

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

Sorry, not suppose that listed code is huge(
I can try make first message smaller, but not sure that it will not drop web server again, so, i leave it as is.

Last edited by George (2016-05-30 15:29:18)

Offline

#5 2016-05-30 16:23:03

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

Related to 2.2:
- without property declaration no, properties are needed for classes, but you could use records
- with nested objects yes, use TSynAutoCreateFieldsclass
- with arrays (string, integer, etc.) yes, use TSynAutoCreateFields and dynamic arrays - for nested lists of classes, define T*ObjArray

Related to 2.3: I do not understand what you want/need

Related to 2.4: how access to specified parameter define several parameters, including arrays in the method - also take a look at record serialization
See the doc about input/output parameters and JSON encoding.
http://synopse.info/files/html/Synopse% … l#TITL_154

Offline

#6 2016-05-31 16:34:54

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

Ok, thanks!

Now i switched to interface based approach, it looks more like datasnap, include session management and authorization control.
Everything works great so far, i like mORMot REST smile
(test project updated, still not final build).

Interesting that DS allow serialize any custom class without custom serialization code...
But, i can live without this feature. Speed and security much more important.

Last edited by George (2016-05-31 16:36:35)

Offline

#7 2016-06-01 08:31:51

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

The idea with our SOA pattern is that you use dedicated DTO objects for your data export.
You should not export the classes of your business, but only dedicated structures which are needed in each service context.
This is a best practice from DDD ("do not leak your domain"), and all serious SOA/nTier implementation ("uncouple your services implementation from their publication").
So IMHO the ability to serialize any business class with attributes is something we do not like (nor want), since it would definitvely pollute the business code.
Check the mORMot documentation about this.

Offline

#8 2016-06-06 15:01:58

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

I've published GIT repository instead Google drive.

How to use TSQLRestServerAuthenticationURI scheme?
Is there any manual or demo related to this scheme?

Is there a simple way to disable automatic protocol downgrade or specify allowed protocols on server side?
I want prevent situations when server use websocket (binary + AES) while client use socket (or websocket JSON or websocket binary without AES).

Last edited by George (2016-06-08 07:54:48)

Offline

#9 2016-06-07 19:00:17

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

Would be cool demo project when everything will be implemented.

screenshot.png

Last edited by George (2016-06-07 19:00:45)

Offline

#10 2016-06-08 07:59:08

edwinsn
Member
Registered: 2010-07-02
Posts: 1,218

Re: JSON RESTful web api from DataSnap to mORMot

Cool! Just an idea - add some benchmarking wink


Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.

Offline

#11 2016-06-08 08:59:06

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

Well, would be cool. Right now i use jmeter as benchmark tool with heavy load simulation.
File "JMeterTestPlan_NoAuthentication.jmx" with test plan for jmeter available in repository.
(in this build, log output should be disabled for best server performance).

Offline

#12 2016-06-08 09:11:48

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

Thanks for the feedback!

About TSQLRestServerAuthenticationURI, this is mostly an abstract scheme: it is not usable directly, I guess.
Since an user name is needed to perform authorization, you need to use the inherited class, e.g. TSQLRestServerAuthenticationNone.
But you may inherit from TSQLRestServerAuthenticationURI to build your own scheme.

Offline

#13 2016-06-08 11:06:22

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

How to properly override fill TSQLAuthGroup and TSQLAuthUser?

procedure tRestServer.ApplyAuthorizationRules(ServiceFactoryServer: TServiceFactoryServer);
var
  User: TSQLAuthUser;
  Group: TSQLAuthGroup;
  SQLAccessRights: TSQLAccessRights;
  GroupID, UserID: TID;
begin

  { TSQLAuthGroup }
  // Clear default groups
  fRestServer.Delete(TSQLAuthGroup, '');
  // Prepare AccessRights
  SQLAccessRights.AllowRemoteExecute := [reService];
  // Create test group
  Group := TSQLAuthGroup.Create();
  Group.Ident := 'MyTestGroupAdmin';
  Group.SQLAccessRights := SQLAccessRights;
  Group.SessionTimeout := 10;
  // Save object to ORM
  GroupID := fRestServer.Add(Group, True);
  // Cleanup
  Group.Free;

  { TSQLAuthUser }
  // Clear default groups
  fRestServer.Delete(TSQLAuthUser, '');
  // Create test user
  User := TSQLAuthUser.Create();
  User.DisplayName := 'test';
  User.LogonName := 'test';
  User.PasswordPlain := 'test';
  User.GroupRights := TSQLAuthGroup.Create(fRestServer, GroupID);
  // Save object to ORM
  UserID := fRestServer.Add(User, True);
  // Cleanup
  User.GroupRights.Free;
  User.Free;

  // Test user data
  User := TSQLAuthUser.Create(fRestServer, UserID);
  if User.GroupRights.Ident = '' then
    ShowMessage('User group should not be empty! Something went wrong.');
  User.Free;

  // Apply Authorization Rules
  ServiceFactoryServer.AllowAll();
end;

I get server error: ESecurityException with message 'Invalid TAuthSession.Create(TSQLRestRoutingREST, TSQLAuthUser)'.
Seems, User.GroupRights (from TAuthSession.Create) is empty.

Last edited by George (2016-06-08 12:24:54)

Offline

#14 2016-06-08 11:09:13

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

What do you expect?
What do you call "override"?

Creating your own classes?
In this case, it is easy to do: inherit from TSQLAuthGroup and TSQLAuthUSer, and put your custom classes in the TSQLMOdel.
They would be recognized and used instead of plain TSQLAuthGroup/User by the server.

Offline

#15 2016-06-08 11:30:59

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

No, i want fill roles and users using default classes.

Line: "User.GroupRights.Free" is the source of the issue.
But, without this line, memory leak appear.

Last edited by George (2016-06-08 12:20:31)

Offline

#16 2016-06-08 12:33:42

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

Please read the doc about TSQLRecord properties.
They are not meant to contain true instances by default, but a fake pointer containing the ID.
Try

User.GroupRights := pointer(GroupID);

Offline

#17 2016-06-08 12:47:43

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

I will read about TSQLRecord properties, hope i will find solution there)

Yes, that works, thanks.

Last edited by George (2016-06-08 13:17:19)

Offline

#18 2016-06-08 15:36:27

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

This "fake pointer" stuff is really confusing, I know.
And it would be limited to 32-bit ID values for a 32-bit executable...
sad

In fact, the clean way to create relations may be to use a TID property, not a TSQLRecord property as reference.
But TSQLAuthUser was created in a time where TID feature did not exist yet..

Perhaps some refactoring is needed here...

Offline

#19 2016-06-08 17:45:55

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

Maybe, but anyway, i'm going furher now)

Authorization code almost done, but i can't understand why i get strange results that affect to ServiceFactoryServer.AllowByName method:

fRestServer.MainFieldIDs(TSQLAuthGroup, ['Administrators'], IDs); // IDs = [2]
fRestServer.MainFieldIDs(TSQLAuthGroup, ['Users'], IDs); // IDs = [1]
fRestServer.MainFieldIDs(TSQLAuthGroup, ['Administrators', 'Users'], IDs); // IDs = [], but why? ((((

I've double checked TSQLAuthGroup, there is only 3 groups each with different "Group.Ident" and ID ...

  // DEBUG {
  fRestServer.MainFieldIDs(TSQLAuthGroup, ['Administrators', 'Users'], IDs);
  if Length(IDs) = 0 then
    ShowMessage('why IDs = []? (((');
  // WHY IDs empty?
  Group := TSQLAuthGroup.CreateAndFillPrepare(fRestServer, '');
  try
    while Group.FillOne do
      ShowMessage(Group.Ident); 
      {'Users'; 'Administrators'; 'SomeoneElse'}
  finally
    Group.Free;
  end;
  // DEBUG }

Last edited by George (2016-06-08 18:01:27)

Offline

#20 2016-06-08 18:38:59

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

I've committed updates to repository.

With temporary solution:

// ServiceFactoryServer.AllowByName([MethodAuthorizationSettings.MethodName], MethodAuthorizationSettings.AllowedGroups); // not work for some reason :(
// ServiceFactoryServer.DenyByName([MethodAuthorizationSettings.MethodName], MethodAuthorizationSettings.DeniedGroups);
for j := 0 to Length(MethodAuthorizationSettings.AllowedGroups) - 1 do
  ServiceFactoryServer.AllowByName([MethodAuthorizationSettings.MethodName], MethodAuthorizationSettings.AllowedGroups[j]);
for j := 0 to Length(MethodAuthorizationSettings.DeniedGroups)  - 1 do
  ServiceFactoryServer.DenyByName([MethodAuthorizationSettings.MethodName], MethodAuthorizationSettings.DeniedGroups[j]);

Last edited by George (2016-06-08 18:40:39)

Offline

#21 2016-06-08 18:57:29

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

I've added an explicit test for TSQLRest.MainFieldIDs, and was not able to reproduce your problem...
See http://synopse.info/fossil/info/fd8aba6477

Offline

#22 2016-06-09 09:04:33

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

Well, here is super simple test case where issue persists.
Delphi XE6U1

(where is forum email notification settings? smile)

Last edited by George (2016-06-09 09:09:18)

Offline

#23 2016-06-09 09:26:22

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

Calling MainFieldIDs() before CreateMissingTables() in your code is incorrect I'm afraid.
You are requesting the TSQLAuthGroup content BEFORE it has been actually created..

Offline

#24 2016-06-09 09:57:55

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

Calling MainFieldIDs() before CreateMissingTables() in your code is incorrect I'm afraid.

Opps, you right.
Anyway results are same, even with this code (test case updated):

procedure TForm1.Button1Click(Sender: TObject);
var
  fModel: TSQLModel;
  fRestServer: TSQLRestServer;
  IDs: TIDDynArray;
  Group: TSQLAuthGroup;
begin
  fModel := TSQLModel.Create([]);
  fRestServer := TSQLRestServerFullMemory.Create(fModel, True);
  fRestServer.ServiceDefine(TIT, [IT], sicSingle);
  fRestServer.CreateMissingTables();

  // DEBUG
  fRestServer.MainFieldIDs(TSQLAuthGroup, ['Admin', 'User'], IDs);
  if Length(IDs) = 0 then
    ShowMessage('why IDs = []? (((');
  // WHY IDs empty?
  Group := TSQLAuthGroup.CreateAndFillPrepare(fRestServer, '');
  try
    while Group.FillOne do
      ShowMessage(UTF8ToString(Group.Ident));
  finally
    Group.Free;
  end;
  // DEBUG

  fRestServer.Free;
  fModel.Free;
end;

Last edited by George (2016-06-09 10:42:01)

Offline

#25 2016-06-09 11:07:52

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

TSQLRestServerFullMemory does not support all SQL statements.
It does not support the IN (...) statement created by MainFieldIDs.

With a regular SQlite3 engine, or an external MongoDB or SynDB database, it would work.

Offline

#26 2016-06-09 12:07:45

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

Well, that not obvious, especially from this place:
fRestServer.MainFieldIDs(

Maybe possible somehow solve incompatibility deeper to allow execute same MainFieldIDs method for all server classes.
Or at least raise error to show that method not supported with selected server class...
But anyway, thank you for information) I have acceptable workaround.

Last edited by George (2016-06-09 12:08:45)

Offline

#27 2016-06-09 14:32:35

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

Interesting results...
HTTP server via TWebSocketServerRest (even without WebSocketsEnable) works faster than via THttpServer.
Actually, speed almost same as when http.sys used.

Client side both times same - JMeter.
Want screenshots? Or it is known normal behavior?

Offline

#28 2016-06-09 15:13:20

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

Yes, but TWebSocketServerRest will not scale with multiple connection as good as with http.sys.
You can easily have  10,000 simultaneous connections with http.sys, whereas it would not scale as good with TWebSocketServerRest.

Offline

#29 2016-06-09 15:28:28

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

So, you recommend http.sys when available otherwise TWebSocketServerRest, right?
And THttpServer without http.sys should not be used.

Last edited by George (2016-06-09 15:28:58)

Offline

#30 2016-06-09 16:06:21

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

In practice, yes.
Only THttpServer without http.sys would have a thread-poll, so would scale better than TWebSocketServerRest, and is to be used under Linux, where http.sys is not avaible (by definition).

Offline

#31 2016-06-14 08:30:31

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

Hello!

With "Client-Server services via interfaces" architecture,
framework can't detect empty parameters.
Am i right?

My test project include JMeter test plan.
I have implemented server method "SendCustomRecord".
Server side not detect empty parameter even if entire body is empty.

I know, that datasnap check parameters and even types (not only simple types, but also record and class types).
Only solution with mORMot is manual access to body with additional parameter tests?

Offline

#32 2016-06-14 17:30:30

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

What do you call "detect empty parameters"?

It depends on how the client send parameters.
There are several ways of sending parameters in mORMot, depending on the routing scheme.
For instance, you may encode the parameters at URI level, or send a JSON object as POST body, or even send a JSON array as POST body (the later being the default between Delphi clients).

At URI level and with a JSON object as POST body, any missing parameter would be replaced with the default value.

Offline

#33 2016-06-14 18:35:04

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

I use most reliable way - array of JSON objects as POST body.
URI length may be limited to 2000 characters in some client realizations.

At URI level and with a JSON object as POST body, any missing parameter would be replaced with the default value.

Well, unfortunately it's not good sad
When developer implement server method with mandatory parameters, client must send all parameters while server should check that all required by developer parameters was sent, otherwise error message should be returned to client side.
Besides, that's how works datasnap (datasnap check even json object type).
And Delphi works in same way, mandatory parameters must be filled, it's intuitively expected behavior.

My test application targeted to Delphi client and http/s client (now represented as JMeter test plan, but http client may be implemented in any language that support http requests/responses).
So server side should not suppose that client send data as server expect.

That's my architectural vision, of course you may disagree wink

Last edited by George (2016-06-14 18:43:25)

Offline

#34 2016-06-14 19:10:36

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

George wrote:

I use most reliable way - array of JSON objects as POST body.

I guess, you mean 'a JSON object as POST body'.

In the framework, parameter match is done at contract level, using the _contract_ pseudo-method.
It is up to the client to validate the contract with the server.
Here we validate not only the parameter existence, but their type and direction.

Then, if the client want to omit a parameter, for ease of calling, it is allowed.

Edit: I've added optErrorOnMissingParam option for paranoid SOA method execution.
See http://synopse.info/fossil/info/135dcd0c7a

Offline

#35 2016-06-14 21:35:15

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

Thanks!

I guess, you mean 'a JSON object as POST body'.

Here is the sample:

function SendMultipleCustomRecords(const CustomResult: rCustomRecord; const CustomComplicatedRecord: rCustomComplicatedRecord): Boolean;
[  
   {  
      "ResultCode":200,
      "ResultStr":"Awesome",
      "ResultArray":[  
         "str_0",
         "str_1",
         "str_2"
      ],
      "ResultTimeStamp":"2016-06-01T19:42:14"
   },
   {
   	"SimpleString": "Simple string, Простая строка",
   	"SimpleInteger":100500,
   	"AnotherRecord": {
	      "ResultCode":200,
	      "ResultStr":"Awesome",
	      "ResultArray":[  
	         "str_0",
	         "str_1",
	         "str_2"
	      ],
	      "ResultTimeStamp":"2016-06-01T19:42:14"   		
   	}
   }
]

Last edited by George (2016-06-14 21:45:42)

Offline

#36 2016-06-14 21:46:21

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

I've tested ServiceFactoryServer.SetOptions([], [optErrorOnMissingParam]).
If body is empty, paranoid check not catch execution.
If body = "[]" my test function now returns cool result:

{
"errorCode":406,
"errorText":"(sicSingle) execution failed (probably due to bad input parameters) for RestMethods.SendMultipleCustomRecords"
}

Which is very nice!

Something like that should help:

...
// decode input parameters (if any) in f*[]
if (ArgsInLast - ArgsInFirst > 0) and (Par=nil) and (optErrorOnMissingParam in Options) then
  exit; // paranoid setting
if (Par<>nil) or (ParObjValues<>nil) then
...

Last edited by George (2016-06-14 22:09:15)

Offline

#37 2016-06-15 08:45:58

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

Offline

#38 2016-06-15 09:40:44

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

Yep, that works)

Offline

#39 2016-06-15 11:14:44

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

I found that under windows 10, URL unregistration not works, while result = 0 (no errors).
App run as admin, url auto registration works (useHttpApiRegisteringURI).

RemUrlResult := THttpApiServer(fHTTPServer.HttpServer).RemoveUrl(ROOT_NAME, fHTTPServer.Port, False, '+');
// THttpApiServer(fHTTPServer.HttpServer).RemoveUrl('service', '777', False, '+');

Cmd command works as expected:

netsh http delete urlacl http://+:777/service/

Last edited by George (2016-06-15 11:19:19)

Offline

#40 2016-06-15 12:51:46

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

What returns GetLastError at API level?

Offline

#41 2016-06-15 13:34:09

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

If you mean GetLastError() method from module "WindowsAPIs.inc", it returns 0.

  //todo: getlasterror
function GetLastError: Integer; stdcall;
  external kernel name 'GetLastError';

In this place: (method THttpApiServer.RemoveUrl)

result := Http.RemoveUrlFromUrlGroup(fUrlGroupID,pointer(uri),0)

fUrlGroupID = -648518319497805279 looks weird, but probably correct (i've discovered that value comes from Httpapi.dll).

Maybe i'm doing something wrong...
Can anyone else test URI unregistration?

Last edited by George (2016-06-15 19:08:53)

Offline

#42 2016-06-15 20:42:57

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

While i have no idea why unregistration not work (maybe there is a bug in winapi library?) i'm trying to use ssl.

AFAIK Https may be used only over http.sys, right?
i've created self signed certificates (ca and signed server cert), both installed on local machine.
Successfully registered cert on https port:

netsh http add sslcert ipport=0.0.0.0:777 certhash=9707a1065f2be25fc8d6f634f66196e151f49625 appid={AA4AC37D-B812-46A7-BEFB-A68167A05BA7}

In addition i've created bat file for simple cert installation.
Another bat file provide cleaning features (delere port, certs and so on).
(available on GitHub).

Successfully started server with secSSL security parameter:

TSQLHttpServer.Create(AnsiString(fServerSettings.Port), [fRestServer], '+', useHttpApiRegisteringURI, 32, TSQLHttpServerSecurity.secSSL)

Cant find what should be done on client side,
but something should be. Isn't it?

fClient := TSQLHttpClientWinHTTP.Create('127.0.0.1', '777', fModel);

Test client application show error message "EWinHTTP:Winhttp.dll error 12002 (timeout)" when i'm trying to connect.
While browser return proper result (https://127.0.0.1:777/service/RestMethods.HelloWorld):

{"result":["Hello world"]}

Last edited by George (2016-06-15 21:15:07)

Offline

#43 2016-06-16 12:11:45

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

Sounds like a client-side problem, here.

You did not set aHttps=true parameter to the constructor.

Offline

#44 2016-06-16 14:20:06

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

Thanks)
Don't know why i missed that overloaded constructor.
Now test project include all available protocols, authentication schemes and authorization rules.
Next i want add proxy usage and additional method interface with session usage.

(i found that under windows 7 certificate installation and port registration via my bat file not work as expected, so i will investigate it).
In general, indy provide much easier https deployment and support any windows, rather than httpapi... (just thoughts).

About URI unregistration, on your machine it works as expected?

Last edited by George (2016-06-16 14:49:28)

Offline

#45 2016-06-16 15:26:02

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

Yes, on a secondary Windows 10 machine, I did not have any registration issue.
Are you sure the URI is not already registered by another app?

Offline

#46 2016-06-16 15:39:56

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

Yes, i use cmd to check what happen before and after server start/stop.
I have no problems with registration, i can't remove registration via "RemoveUrl".
I've tested in windows 10, 8.1 and windows 7 - same problem.

If RemoveUrl work in your test application, can you share source of test project? So i would be able to see how and where you use RemoveUrl, compile and test on my machine.

Last edited by George (2016-06-16 15:48:02)

Offline

#47 2016-06-19 12:11:44

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

I've implemented another interface with sicPerSession instance implementation.
Also i created custom session class:

TSQLAuthUserEx = class(TSQLAuthUser)
  public
    SessionData: string;
  end;

Model initialization use my class:

fModel := TSQLModel.Create([TSQLAuthGroup, TSQLAuthUserEx], ROOT_NAME);

And i'm trying change SessionData value for session.
So, i've added two server methods (set and get session data):

// Set user session data
function TRestMethodsEx.SetDataToUserSession(data: string): Boolean;
var
  UserSession: TSQLAuthUserEx;
begin
  UserSession := TSQLAuthUserEx.Create(fServer, ServiceContext.Request.SessionUser, True);
  UserSession.SessionData := data; // set session value from client
  try
    Result := fServer.Update(UserSession);
  except
    Result := False;
  end;
  fServer.UnLock(UserSession);
  UserSession.Free;
end;

// Get user session data
function TRestMethodsEx.GetDataFromUserSession(): string;
var
  UserSession: TSQLAuthUserEx;
begin
  UserSession := TSQLAuthUserEx.Create(fServer, ServiceContext.Request.SessionUser);
  Result := UserSession.SessionData; // get session value from session
  UserSession.Free;
end;

// Get user name
function TRestMethodsEx.GetUserSessionLogin(): string;
begin
  Result := UTF8ToString(ServiceContext.Request.SessionUserName); // Returns correct user name
end;

I can't find why GetDataFromUserSession() returns empty string...
Samples and documentation was read before post.

Last edited by George (2016-06-19 12:14:25)

Offline

#48 2016-06-19 13:45:10

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

This is a BLOB field, which is not retrieved by default, to save bandwidth.
Please check the doc and use RetrieveBlob method instead.

Offline

#49 2016-06-19 16:11:34

George
Member
Registered: 2016-04-05
Posts: 142

Re: JSON RESTful web api from DataSnap to mORMot

Code above is server side, not client.
Should i use anyway RetrieveBlob?

Last edited by George (2016-06-19 16:23:46)

Offline

#50 2016-06-19 18:57:22

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,667
Website

Re: JSON RESTful web api from DataSnap to mORMot

Check the TSQLRest.Retrieve docs - common to both client and server side, following Liskov Substitution principle.
Of course you should use RetrieveBlob also if the TSQLRest instance is a server instance!

Offline

Board footer

Powered by FluxBB