mORMot and Open Source friends
View Ticket
Not logged in
Ticket Hash: aa230e52993ed716b51bc675b69fba1a0fed7afc
Title: Implements one-way callbacks from the server
Status: Fixed Type: Feature_Request
Severity: Severe Priority: Immediate
Subsystem: mORMot Resolution: Implemented
Last Modified: 2015-05-06 17:20:59
Version Found In:
User Comments:
root added on 2014-06-27 08:14:07:

Aim is to implement notification events trigerred from the server side, very easily from Delphi code, even over HTTP (note that WCF does not allow this). It will be the ground of Event Collaboration or Event Sourcing stacks included within mORMot.

See this blog article about the interface-based implementation proposal, and its innovative publish / subscribe pattern - may be included as http.sys API 2.0 websocket protocol


ab added on 2014-12-31 10:32:32:

We enhanced the priority of this feature request, since it is needed for several other feature requests, e.g. synch/replication [3453f314d9] or monitoring [17958b22c02].


ab added on 2015-01-29 12:28:33:

Status Update
From our perspective, we may consider implement one-way callbacks with several ways:

  • Simple pooling mechanism from the client, using a timer - should also be available for SynCrossPlatformRest.pas supported platforms;
  • Create a HTTP server on the client, to receive callbacks (sounds much better than pooling or websockets for production services);
  • ZeroMQ for high performance, cloud-computing ready messaging.

WebSockets or long pooling are not an option in our current roadmap, since we would rather rely on a simple REST connection for the end-user applications, with an application service layer (as proposed by DDD principles). End-user application could simply pool its application service for any pending event. Mobile apps may benefit later on from Google/Apple push notification services. HTTP callback server or ZeroMQ may be used on a trusted intranet, for better performance.

We are currently investigating about supporting ZeroMQ for easy implementation of the publish/subscribe pattern.

See this good introduction to ZeroMQ for a high level presentation.

The ZeroMQ Request/Reply pattern may be used to implement a REST access to mORMot servers instead of HTTP, with the benefit of higher performance, and fair distribution among servers. ZeroMQ devices would add simple and efficient routing of services.

The ZeroMQ Publish/Subscribe pattern may be used to implement our one-way callbacks from the server, with appropriate filtering of the JSON content, on need.


ab added on 2015-02-26 10:48:07:

Implementation Proposal

Proposal is to use interface parameters to interface-based service methods to send a callback to the server.

When the client code supplies an instance of class (possibly self) for this parameter:

type
  IMyCallback = interface(IInvokable)
    ['{5CD49AB1-D5A3-4059-B287-25E939DD2A0B}']
    procedure EventCalledFromServer(const withAValue: integer);
  end;
  IMyService = interface(IInvokable)
    ['{FB31BF33-38C7-45AE-9511-6AC727ADC4F4}']
    procedure MethodWithCallback(const anyParameter: variant; const TheCallBack: IMyCallback);
  end;

type TMyServiceClient = class(TInterfacedObject,IMyCallback) protected fRest: TSQLRestClient; // here the callback will point back to "self", so IMyCallback is implemented now // but you may set a callback from any another class implementing IMyCallback procedure EventCalledFromServer(const withAValue: integer); public procedure Test; end;

procedure TMyServiceClient.Test; var MyService: IMyService; begin if fRest.Services.Resolve(IMyService,MyService) then MyService.MethodWithCallback(1234,self); end;

procedure TMyServiceClient.EventCalledFromServer(const withAValue: integer); begin // will be called back by the server end;

type TMyServiceServer = class(TInterfacedObject,IMyService) protected fCallback: IMyCallback; // for delayed call, in sicClientDriven mode public procedure MethodWithCallback(const anyParameter: variant; const TheCallBack: IMyCallback); procedure AnotherMethod; end;

procedure TMyServiceServer.MethodWithCallback(const anyParameter: variant; const TheCallBack: IMyCallback); begin TheCallBack.EventCalledFromServer(1234); fCallback := TheCallBack; end;

procedure TMyServiceServer.AnotherMethod; begin if assigned(fCallback) then fCallBack.EventCalledFromServer(5678); end;

In practice:

  • A reverse server is created on the client computer, to receive the callback events;
  • The interface parameter will be transmitted to the server serialized as a full reverse URI JSON string or a routing-agnostic JSON object - a "/callback" method-based service at REST client level could be used as entry point, using the existing client authentication context to handle security, and an internal list of callbacks methods matching the URI, with an (optional/future?) incremental requestID;
  • The server will initiate a reverse client on the server computer, to send the callback events;
  • On the server side implementation method, a fake interface instance will be transmitted in the parameter, so that the client may be notified immediately, or later on if the interface reference is stored in a local field variable, with a regular REST/JSON call;
  • By definition, delayed callback would work safely only in sicClientDriven mode, so that the interface reference may be stored safely in the server instance, synchronized with the client instance live time; but any other execution modes should be safe to be used, if no interface reference is made for the delayed calls;
  • By definition, direct access at TSQLRestServer level (i.e. without HTTP) will work as usual, since the used syntax is the exact behavior of interface parameters - it would even allow to stub/mock the callbacks;
  • We would implement this set of callbacks with HTTP only at first, and a reversed HTTP server - Windows messages, named pipes, websockets or ZeroMQ may be implemented later, but our first intent is to add a safe and easy callback mechanism in a cloud of mORMot SOA servers;
  • Routing and message propagation will take place automatically: you could safely chain passing several callback parameters around several mORMot client/server nodes (which may be pretty common in a DDD/nTier architecture), e.g. for transparent error process (we may transmit back Exceptions as variant parameters to the callback).

This pattern will also be the base of any kind of dual-connection process, e.g. regular publish/subscribe mechanism with a list of subscription on the server side, pointing to alive sicClientDriven client connections.


ab added on 2015-02-26 11:01:35:

In addition, a callback passed as a out aService: IMyService parameter (instead of const) may be used to easily implement a catalog system.

type
  IMyService = ...
type
  IMyCatalog = interface(IInvokable)
    ['{5696DC03-2AD6-419C-B894-B9EA1506DD9B}']
    function RetrieveMyServiceInstance(const ShardingCriteria: TID; out Service: IMyCallback): boolean;
  end;

As you can see, we specified an optional criteria, which may be used to spread the service implementation in several servers, in a way close to the MongoDB sharding concept. This would allow clean horizontal scaling of the whole SOA implementation.


ab added on 2015-05-06 17:20:59:

We have almost everything needed.

CQRS can be easily implemented:

Event Source may be implemented easily with SOA interfaces, and their callbacks.

See how callbacks can be defined, using WebSockets for communication and the corresponding samples.