Interface based services

ab · 2012-03-07 10:46:28

Here are the key features of the current implementation of services using interfaces in the Synopse mORMot framework:

Service Orientation - Allow loosely-coupled relationship;
Design by contract - Data Contracts are defined in Delphi code as standard interface custom types;
Factory driven - Get an implementation instance from a given interface;
Server factory - You can get an implementation on the server side;
Client factory - You can get a "fake" implementation on the client side, remotely calling the server to execute the process;
Auto marshaling - The contract is transparently implemented: no additional code is needed e.g. on the client side, and will handle simple types (strings, numbers, dates, sets and enumerations) and high-level types (objects, collections, records, dynamic arrays) from Delphi 6 up to XE2;
Flexible - Methods accept per-value or per-reference parameters;
Instance lifetime - An implementation class can be: Created on every call, Shared among all calls, Shared for a particular user or group, Stay alive as long as the client-side interface is not released, or as long as an authentication session exists;
Stateless - Following a standard request/reply pattern;
Signed - The contract is checked to be consistent before any remote execution;
Secure - Every service and/or methods can be enabled or disabled on need;
Safe - Using extended RESTful authentication;
Multi-hosted (with DMZ) - Services are hosted by default within the main ORM server, but can have their own process, with a dedicated connection to the ORM core;
Broker ready - Service meta-data can be optionally revealed by the server;
Multiple transports - All Client-Server protocols of mORMot are available, i.e. direct in-process connection, GDI messages, named pipes, TCP/IP-HTTP;
JSON based - Transmitted data uses JavaScript Object Notation;
Routing choice - Services are identified either at the URI level (the RESTful way), either in a JSON-RPC model (the AJAX way);
AJAX and RESTful - JSON and HTTP combination allows services to be consumed from AJAX rich clients;
Light & fast - Performance and memory consumption are very optimized, in order to ensure scalability and ROI.

Forum discussion about several Blog articles.
See http://blog.synopse.info/post/2012/03/0 … d-services and related.

A dedicated sample has been published.
See http://blog.synopse.info/post/2012/03/0 … ample-code

You can retrieve a PDF version of those articles:
see Interface based Services in mORMot - Overview document.

Junior/RO · 2012-03-12 13:31:37

I have tried your sample and it worked. But I don't know how to rewrite your sample code to work using http.

When I try, client give me a exception '"Calculator" interface or REST routing not supported by server:'

What I am doing wrong?

Can you please write a http version of your sample '14 - Interface based services'? Thank you very much.

PS: I never tried synopse mORMot before, but it seems easier to comunicate with server using mORMot than using DataSnap or Asta. Better than a webservice too. Great work.

Last edited by Junior/RO (2012-03-12 13:39:03)

ab · 2012-03-12 20:40:26

Thanks for your interest.

I've updated the sample to add HTTP client-server, and made modification of the client to handle both HTTP and named pipe communication from code.

See http://synopse.info/fossil/info/0b07bc52d7

Junior/RO · 2012-03-12 22:10:23

Now it's working. Thank you.

By the way, named pipes can be used in a local network or http is the only solution?

Last edited by Junior/RO (2012-03-12 22:14:36)

ab · 2012-03-13 11:39:41

Since Windows Vista/Seven, we faced some security restriction issues which made named pipes failing to communication over a local network.
See http://synopse.info/forum/viewtopic.php?id=43

So up to know, only HTTP/TCP-IP is working for a remote communcation.
But kernel-level http.sys server makes performance very good, as far as we can guess, over a physical network: a 100Mb network is saturated, and even a 1GB network connection is easily saturated, when messages are big enough.

Junior/RO · 2012-03-21 16:30:05

Maybe there are a bug when the return value of a interface based method is a Boolean.

SynCommons.JSONDecode() thinks that a correct value from server is invalid and gives me a exception "Error calling MyBooleanTest.Test remote method".

Interface:

type
  IMyBooleanTest = interface(IInvokable)
    ['{9A60C8ED-CEB2-4E09-87D4-4A16F496E5FE}']
    function Test: Boolean;
  end;

const
  ROOT_NAME = 'root';
  APPLICATION_NAME = 'RestService';

Server.dpr:

type
  TMyBooleanTest = class(TInterfacedObject, IMyBooleanTest)
  public
    function Test: Boolean;
  end;

function TMyBooleanTest.Test: Boolean;
begin
  Result := False;
end;

var
  aModel: TSQLModel;
  aServer: TSQLRestServer;
  aHTTPServer: TSQLite3HttpServer;
begin
  aModel := TSQLModel.Create([],ROOT_NAME);
  try
    aServer := TSQLRestServerFullMemory.Create(aModel,'test.json',false,true);
    try
      aServer.ServiceRegister(TMyBooleanTest,[TypeInfo(IMyBooleanTest)],sicShared);
      aHTTPServer := TSQLite3HttpServer.Create('888',[aServer]);
      try
        writeln('Background server is running.'#10);
        write('Press [Enter] to close the server.');
        readln;
      finally
        aHTTPServer.Free;
      end;
    finally
      aServer.Free;
    end;
  finally
    aModel.Free;
  end;
end.

Client:

procedure TForm1.btnCallClick(Sender: TObject);
var
  MyTest: IMyBooleanTest;
begin
  if Client=nil then begin
    if Model=nil then
      Model := TSQLModel.Create([],ROOT_NAME);
    Client := TSQLite3HttpClient.Create('localhost','888',Model);
    Client.SetUser('User','synopse');
    Client.ServiceRegister([TypeInfo(IMyBooleanTest)],sicShared);
  end;

  if Client.Services['MyBooleanTest'].Get(MyTest) then
     ShowMessage(BoolToStr(MyTest.Test));   // <- "Error calling MyBooleanTest.Test remote method".
end;

ab · 2012-03-21 16:55:42

This was an issue.

It is now fixed, and associated regression tests have been added.
See http://synopse.info/fossil/info/6f77def459

Thanks for the report.

Junior/RO · 2012-03-23 18:19:44

Thank you. It's working now.

I have a new question: need to send a image (.jpg or .tiff, size from 2mb to 5mb) from server to client. I'm thinking about write a remote method that returns a sort of blob, but don't know which type use:

type
  IPictureService = interface(IInvokable)
    ['{9A60C8ED-CEB2-4E09-87D4-4A16F496E5FE}']
    function GetPicture(Index: Integer): <which type? TDynArray? Don't know>;
  end;

EDIT: I found a solution. Return a record. It's the best approach?

Interface

type
  TPicturePackage = record
    FileName: RawUTF8;
    Binary: TByteDynArray;
  end;

  IPictureService = interface(IInvokable)
    ['{9A60C8ED-CEB2-4E09-87D4-4A16F496E5FE}']
    function GetPicture(Index: Integer): TPicturePackage;
  end;

Server:

function TPictureService.GetPicture(Index: Integer): TPicturePackage;
var 
  Stream: TMemoryStream;
begin
  Stream := TMemoryStream.Create;
  try
    Result.FileName := 'biology' + IntToStr(Index) + '.jpg';
    Stream.LoadFromFile(Result.FileName);
    SetLength(Result.Binary, Stream.Size);
    Move(Stream.Memory^, Result.Binary[0], Stream.Size);
  finally
    Stream.Free;
  end;
end;

Client:

    Package := PictureService.GetPicture(1);
    Stream := TMemoryStream.Create;
    try
      Stream.Size := Length(Package.Binary);
      Move(Package.Binary[0], Stream.Memory^, Stream.Size);
      Stream.SaveToFile(Package.FileName);
    finally
      Stream.Free;
    end;

Last edited by Junior/RO (2012-03-23 19:39:01)

ab · 2012-03-23 20:59:43

It will work as expected.

But it will use base64 encoding.
So transferred data will be a bit bigger.

Junior/RO · 2012-03-23 21:01:38

Can you show me a piece of more efficient code? I suspect that you have a better approach

Last edited by Junior/RO (2012-03-23 21:01:50)

ab · 2012-03-23 21:03:01

I'm working on it ...

Junior/RO · 2012-03-24 15:22:00

Another issue: if I declare the interfaces variables as in

type
  TForm1 = class(TForm)
  private
    Service: IPictureService;
  end;

and create a instance like:

procedure TForm1.FormShow(Sender: TObject);
begin
  // Client already created and service registered.
  Client.Services.Info(TypeInfo(IPictureService)).Get(Service);
end;

Then, if I call the Service.GetPicture() 2 or more times, some exceptions will raise when destroying the form (reference counting, OS errors etc). However, if I wrote in Form1.OnDestroy() event:

procedure TForm1.FormDestroy(Sender: TObject);
begin
  Service := nil;
end;

...then no exceptions will rise. And the termination of the application will be faster.

Last edited by Junior/RO (2012-03-24 16:01:26)

mpv · 2012-03-25 09:40:16

In current realization of "Interface based services" any request to service is going by POST method, and all POST methods are blocked in RestServer.URI method by AcquireWrite(SessionID), therefore when using "Interface based services" we do not have a true multithread server - all thread are blocking while one interface method is executing. It's bad Do you plan to do multithreding (may be defining some method in interface what say the interface doesn't need blocking or move LaunchService from AcquireWrite block)?.

Last edited by mpv (2012-03-25 11:31:19)

ab · 2012-03-26 12:14:42

@mpv

You are right.

I'll let services be running outside the global lock.
See http://synopse.info/fossil/info/a56cba70ce for the fix.

Thanks for the remark.

Junior/RO · 2012-03-27 13:06:51

Hi. Could you reproduce the new issue (calling interface based services two or more times with exceptions on exiting) that I reported?

It's better report this type of issue here or in the fossil's tickets feature?

Last edited by Junior/RO (2012-03-27 13:09:20)

ab · 2012-03-27 15:31:22

@Junior You may create a ticket entry.
This is more easy to follow the status, and not forget about it.

I think it is related to the way reference counting is implemented by the compiler.
In fact, in your code, the private interface is released AFTER the Client instance is released.
When the Service instance is freed, it is designed to notify the remote server that the instance is released...
So, due to this fact, you NEED to explicitly release the interface BEFORE Client.Free is called.

That is, you have to code:

procedure TForm1.FormDestroy(Sender: TObject);
begin
  Service := nil;
  Client.Free; // after Service release
end;

This is the only way of implementing it, IMHO.

Since the exact address of Service is accessible when its content is filled, we may add some "weak pointer" strategy to TSQLRestClient services.
That is, Client.Free may do explicitely Service := nil for all its services, if it is asked as such in the Client.Services.Info(TypeInfo(IPictureService)).Get(Service,true) method call.
But I suspect this may be confusing, and would need a better implementation of "interfaces weak reference", at the compiler level.

Therefore, I think there is no issue here, but documentation shall be enhanced in order to explicitly warn about such issues.
That is, client code shall explictly release all global interfaces BEFORE calling Client.Free.

mpv · 2012-03-27 18:59:09

ab wrote:

@mpv
You are right.
I'll let services be running outside the global lock.
See http://synopse.info/fossil/info/a56cba70ce for the fix.
Thanks for the remark.

It's much better!
And adding
case URIMethod of
is a good idea – code becomes much more clear.
There is also a problem with SQL statement execute – I created ticket.

General remark - your framework is the best Delphy code I have seen for 15 years of developing. Very cool!

ab · 2012-03-28 06:59:17

Thanks for the feedback!

I've committed http://synopse.info/fossil/info/6053a9ddc9 and http://synopse.info/fossil/info/0a6f5f3a6a to implement an optional URI-encoding of the SQL statement execution.
I've added reUrlEncodedSQL remote access right, to allow execution of SQL statement from a GET with the content encoded on the URI (as from XMLHTTPRequest).

mpv · 2012-03-28 07:53:32

Yes, I see. Thanks. It's now possible to do params in queries - I post a suggestion.

ab · 2012-03-28 10:02:48

Two modifications:
* now interface based services are responding with GET commands in additional to POST verb (will be more compatible with XMLHTTPRequest for instance);
* Custom returned content (i.e. TServiceCustomAnswer) is now handled as expected by the Delphi client (i.e. TServiceFactoryClient class).

See http://synopse.info/fossil/info/52105f4ebe

@Junior/RO The new TServiceCustomAnswer definition will help transmitting BLOB content without any JSON+Base64 encoding.
This could be a good idea for a picture, e.g.
See also http://blog.synopse.info/post/2012/03/2 … ed-service

profh · 2012-04-08 15:15:13

pls show an example about how to send/get binary data to/from server using an interface-based service.

thanks.

ab · 2012-04-08 16:20:12

You've got sample code in the quoted blog article - see http://blog.synopse.info/post/2012/03/2 … ed-service

And the regression tests, available in the corresponding commit.

mpv · 2013-04-23 11:36:22

I try to rewrite my application to use interface-based services. In my case I need per-thread service instance. Is it possible to add TServiceInstanceImplementation.sicPerThread instance implementation pattern? From from the POV of the client it will look like sicShared as I understand.

ab · 2013-04-23 13:08:13

Why do you need a per-thread instance?

Please add a ticket for this feature request.

mpv · 2013-04-23 14:13:02

in my case each thread must contain a copy of the interpreter with a lot of compiled scripts. Create an instance of the interpreter and compiling scripts is time consuming operation and should be used to perform once for each thread.

ab · 2013-04-23 14:58:07

I suppose forcing to run the service methods in the main thread (optExecInMainThread+optFreeInMainThread) is not good enough for your purpose?

mpv · 2013-04-23 15:05:04

optExecInMainThread absolutely impossible - in this case I got single-thread application. But I need multi-thread... And I don't want to free created instance each time - just create it during first call to service and free when server die.

ab · 2013-04-23 19:10:06

It perfectly make sense!

Ticket http://synopse.info/fossil/info/cb76c866bb is on the road...

mpv · 2013-04-24 05:18:09

AB, i do what I need in thread-safe way (the same you do with threadSafeConnection). So for a moment i can use sicSingle and there is no ned to implement sicPerThread. Sorry to trouble you. I clous feature request? Or we lave it for future need?

ab · 2013-04-24 06:34:52

No problem to do it now.

I think it will be pretty easy to implement it.

Stay tuned!

mpv · 2013-04-24 15:52:26

Some additional ideas....
I have a "static file" transfer METHOD - just transfer files from server to client (something like "Samples\09 - HttpApi web server") . Now it is implemented as a published method of TSQLRestServer descendant

TMyServer = class(TSQLRestServer)
...
published
  procedure static(var Ctxt: TSQLRestServerCallBackParams);
  procedure YUI(var Ctxt: TSQLRestServerCallBackParams);
  .......
end;

I want to rewrite it using interface-based services. This give me ability to connect "static file transfer" functionality to any server (TSQLRestServerFullMemory for example). In fact, I do want to give up all the published methods and rewrite all via interface-based service.
In current implementation I call TMyServer.static method by put
<script charset="utf-8" src="http://localhost:8080/root/static/app.js"></script>
in HTML file.
This is not interface-based service URL format. Yes, I can change this to:
<script charset="utf-8" src="http://localhost:8080/root/static.get?file=app.js"></script> but this is not KISS from HTML POV.

Another example - I have YUI method - realization of YUI interface using my ORM as background.
I want to rewrite it as a method of interface-based service IYUI to.

To solve my tasks I need something like this:

  IStatic = interface(IInvokable)
    ['{84A6479B-4DE8-4B7A-93BB-4CE8935FE03C}']
    function default(const URI: RawUTF8; const asMIME: RawUTF8): TServiceCustomAnswer;
  end;
  Server.ServiceRegister(TStaticFileService,[TypeInfo(IStatic)], sicShared);

"Magic", I want to offer is:
1. "default" method - this is the method that should be called if no ".method" passed in URL but for ServicesRouting=rmREST. This give me possibility to write GET /root/ServiceName.... instead of GET /root/ServiceName.MethodName....
2. "URI" parameter - if parameter with name "URI": RawUTF8 exist in method parameter list value of this parameter retrieved from URL tail. This give me possibility to write GET /root/static/pictures/myimage.gif?asMIME=application\/pdf and server call my function
TStaticFileService.default("/pictures/myimage.gif", "application\/pdf");

What do you think - is it make sense?

ab · 2013-04-25 14:03:50

I've added the new sicPerThread mode.
See http://synopse.info/fossil/info/6308e26c35

Corresponding regression tests are included, and documentation has been updated to include this new instance creation process.

Hope it helps!

About new web-oriented interface-based services outside the pure "JSON/Delphi" world, we may be able to add some new routing/parameter marshalling, as you expect.
But we still need to find the best/KISS way of defining them.

mpv · 2013-04-26 08:43:32

I've added the new sicPerThread mode.

Magnifiquement, as always

we may be able to add some new routing/parameter marshalling, as you expect

I do not want a new routing scheme - I just want to expand a little two existing by adding "default" behavior and one "magic" parameter retrieved from URI and combine it with existing rmREST and rmJSON_RPC

This give me ability to rebuild my server using interface-based services and be compatible with framework not on Syn* level as today, but on mORMot* level. The next step(SpiderMonkey integration to mORMot) will be much easier for me after that..

ab · 2013-04-26 09:44:19

If I understand well, we need only a GET method to retrieve some static content?

Perhaps something more complete may make sense, with "var Ctxt: TSQLRestServerCallBackParams" parameters.
It will allow a known - and already coded - way of handling any HTTP method, and return any content.

Like:

  IStaticRequest = interface(IInvokable)
    ['{84A6479B-4DE8-4B7A-93BB-4CE8935FE03C}']
    procedure _default(var Ctxt: TSQLRestServerCallBackParams);
  end;

... and perhaps not include it to the service interface (i.e. not call Server.ServiceRegister(TStaticFileService,[TypeInfo(IStatic)], sicShared)) but let the feature be available if the implementation class has this interface:

// in mORMot.pas:
type
  IDefaultRequest = interface(IInvokable)
    ['{84A6479B-4DE8-4B7A-93BB-4CE8935FE03C}']
    procedure _default(var Ctxt: TSQLRestServerCallBackParams);
  end;

// used as such:
type
  IMyService = interface(IInvokable)
    ['{796479B-7248-367A-794B-4CE8935F739C}']
     function ComputeAdd(a,b: integer): integer;
  end;

  TStaticFileService = class(TInterfaceObject, IMyService, IDefaultRequest)
  protected
    // will be called if no method is recognized
    procedure _default(var Ctxt: TSQLRestServerCallBackParams);
  public
     // normal service method
     function ComputeAdd(a,b: integer): integer;
  end;

What do you think?

mpv · 2013-04-26 16:19:55

I thought about something like that.

If I understand well, we need only a GET method to retrieve some static content?

In general - no.
For static file service - yes, only GET. But I plane to use similar services realization for my ORM - so I need all type of HTTP request (GET|PUT|POST|....) to be passed to interface-based methods.

About passing Ctxt: TSQLRestServerCallBackParams to _default method - it maybe just for optimization, for interface-based services I already have access to TSQLRestServerCallBackParams via ServiceContext.Session.
The advantage of parameter list in interface methods, i.e. not

IStaticRequest = interface(IInvokable)
    ['{84A6479B-4DE8-4B7A-93BB-4CE8935FE03C}']
    procedure _default(var Ctxt: TSQLRestServerCallBackParams);
end;

but separate parameter list

IMyService = interface(IInvokable)
    ['{796479B-7248-367A-794B-4CE8935F739C}']
     function ComputeAdd(a,b: integer): integer;
    function _default(const URI: RawUTF8; const asMIME: RawUTF8; .....): TServiceCustomAnswer; // only URI parameter retrieved from URL tail (if exist in parameter list), all other - depending of routing schema
end;

TStaticFileService = class(TInterfaceObject, IMyService)
  public
    // will be called if no method is recognized, because only of _default is a method name
     function _default(const URI: RawUTF8; const asMIME: RawUTF8; .....): TServiceCustomAnswer; // only URI parameter retrived from URL tail, all other - depending of routing schema
     // normal service method
     function ComputeAdd(a,b: integer): integer;
  end;

is:
1) I do not need to parse the parameters manually, and
2) I have a _contract_ - it very important if team consist of many developer.

But to have such a procedure would be very good at TSQLRestServer level

TMyServer = class(TSQLRestServer)
published
  procedure _default(var Ctxt: TSQLRestServerCallBackParams);

This give me ability to handle request of type [GET|POST|...] http://myserver/root. For example in my current implementation I return server contract in case of GET http://myserver/root?_contract_ or do redirect for browser in case of GET http://myserver/root to http://myserver/root/static/index.html

procedure TMyServer._default(var Ctxt: TSQLRestServerCallBackParams);
begin
...
  if Ctxt.Method = mGET then begin //static file only for GET request
    if Ctxt.URI='' then begin
      Result := hscMovedPermanently;
      Ctxt.OutCustomHeaders := LOCATION_REDIRECT_HEADER + Ctxt.Call.URL + '/static/index.html';
  .......
  if (Ctxt.Method = mGET) and (Ctxt.Parameters <> nil) and IdemPChar(Ctxt.Parameters, '_CONTRACT_') then begin
    //return global server contract
  end
 .....
  
end;

ab · 2013-04-27 08:07:33

The idea of defining procedure _default(var Ctxt: TSQLRestServerCallBackParams) at an interface level and not at TSQLRestServer level is to process dedicated process at the URI level of the interface.

That is, TMyInterfaceImplementation._default() will be called if the URI root/MyInterface/totoro is asked and there is no method TMyInterfaceImplementation.totoro defined.

But it still make sense to have a callback event at TSQLRestServer level, which may be called for every unknown dll.
Perhaps not a TSQLRestServer virtual method, but a new OnServerNotFoundURI event.
Generally, it could make sense to add some events to TSQLRestServer and TSQLRestClient classes.

mpv · 2013-04-27 12:35:23

This is exactly what I had in mind, the two procedures. One on the TSQLRestServer level ( do it using event OnServerNotFoundURI(Ctxt: TSQLRestServerCallBackParams) as you propose is good). The second is on the service level. But for service level TMyInterfaceImplementation._default() I propose not one parameter Ctxt, but the same way we define parameter for other interface-based methods. ( ok - we dont need magic URI parameter - I can retrive it from ServiceContext variable).

mORMot Open Source

#1 2012-03-07 10:46:28

Interface based services

#2 2012-03-12 13:31:37

Re: Interface based services

#3 2012-03-12 20:40:26

Re: Interface based services

#4 2012-03-12 22:10:23

Re: Interface based services

#5 2012-03-13 11:39:41

Re: Interface based services

#6 2012-03-21 16:30:05

Re: Interface based services

#7 2012-03-21 16:55:42

Re: Interface based services

#8 2012-03-23 18:19:44

Re: Interface based services

#9 2012-03-23 20:59:43

Re: Interface based services

#10 2012-03-23 21:01:38

Re: Interface based services

#11 2012-03-23 21:03:01

Re: Interface based services

#12 2012-03-24 15:22:00

Re: Interface based services

#13 2012-03-25 09:40:16

Re: Interface based services

#14 2012-03-26 12:14:42

Re: Interface based services

#15 2012-03-27 13:06:51

Re: Interface based services

#16 2012-03-27 15:31:22

Re: Interface based services

#17 2012-03-27 18:59:09

Re: Interface based services

#18 2012-03-28 06:59:17

Re: Interface based services

#19 2012-03-28 07:53:32

Re: Interface based services

#20 2012-03-28 10:02:48

Re: Interface based services

#21 2012-04-08 15:15:13

Re: Interface based services

#22 2012-04-08 16:20:12

Re: Interface based services

#23 2013-04-23 11:36:22

Re: Interface based services

#24 2013-04-23 13:08:13

Re: Interface based services

#25 2013-04-23 14:13:02

Re: Interface based services

#26 2013-04-23 14:58:07

Re: Interface based services

#27 2013-04-23 15:05:04

Re: Interface based services

#28 2013-04-23 19:10:06

Re: Interface based services

#29 2013-04-24 05:18:09

Re: Interface based services

#30 2013-04-24 06:34:52

Re: Interface based services

#31 2013-04-24 15:52:26

Re: Interface based services

#32 2013-04-25 14:03:50

Re: Interface based services

#33 2013-04-26 08:43:32

Re: Interface based services

#34 2013-04-26 09:44:19

Re: Interface based services

#35 2013-04-26 16:19:55

Re: Interface based services

#36 2013-04-27 08:07:33

Re: Interface based services

#37 2013-04-27 12:35:23

Re: Interface based services

Board footer