Roadmap: interface-based callbacks for Event Collaboration

ab · 2012-09-06 16:53:28

On the mORMot roadmap, we added a new upcoming feature, to implement one-way callbacks from the server.
That is, add transparent "push" mode to our Service Oriented Architecture framework.

Aim is to implement notification events triggered from the server side, very easily from Delphi code, even over a single HTTP connection - for instance, WCF do not allow this: it will need a dual binding, so will need to open a firewall port and such.

It will be the ground of an Event Collaboration stack included within mORMot, in a KISS way.
Event Collaboration is really a very interesting pattern, and even if not all your application domain should be written using it, some part may definitively benefit from it.
The publish / subscribe pattern provides greater network scalability and a more dynamic SOA implementation: for instance, you can add listeners to your main system events (even third-party developed), without touching your main server.
Or it could be the root of the Event Sourcing part of your business domain: since callbacks can also be executed on the server side (without communication), they can be used to easily add nice features like: complete rebuild, data consolidation (and CQRS), temporal query, event replay, logging, audit, backup, replication.

Feedback for http://blog.synopse.info/post/2012/09/0 … laboration

andrewdynamo · 2012-09-07 06:30:18

Only polling (and long polling) is/will be used?
I know, this is the only possible with HTTP, but with WebSockets (over http) you can do real "push"ing:
http://www.websocket.org/quantum.html

Therefore I made a websocket implementation (for RemObjects, sorry) with Indy 10:
https://plus.google.com/u/0/11013108667 … vicDJLfYtQ

It's not perfect, but tried to keep thing seperate in a special IOHandler etc, so you can probably use it too without RO.

ab · 2012-09-07 08:58:25

You are right, WebSockets may be implemented in the future.
But it is AFAIK implemented as a long polling (or lock-and-wait) scheme, based on HTTP headers.
So it is not exactly real "push"ing, but header-based plumbing based on HTTP one-way connection.

In mORMot, we did want to be truly RESTful, without the necessity to use HTTP.
So we try to avoid working with HTTP headers in our protocol implementations, since we also implement direct in-process, GDI messages and named pipes. In fact, since GDI messages are two-way communications, we would implement a direct callback feature, with no polling; and maybe change the named pipe mode to duplex, or allow direct callback event for in-process/stand alone access.

For AJAX clients (e.g. via Smart), it could make sense to implement a WebSocket ready server for mORMot.
But for Delphi clients, it would not make sense to use it.

We will try to make the most generic implementation, allowing (long) polling for HTTP, but perhaps some dedicated mechanism for other protocols (in-process, GDI, named pipes).
I would like the TSQLRest callback internal methods to be generic/abstract, knowing nothing about polling or the underneath communication used.
So you would never be stuck to one implementation, but open to alternatives.

I think http.sys / HttpAPI (our main HTTP server, much better than Indy or even direct WinSock access) is even not compatible with websockets - see http://nbevans.wordpress.com/2011/12/20 … ows-server - at least until Windows 8.
In fact, WebSockets is pretty controversial: due to its implementation pattern, it is sometimes recognized as a DOS attempt, or invalid HTTP content.
Therefore, perhaps not so high in our priority list.
Of course contribution is welcome!

Thanks for the link and feedback.

andrewdynamo · 2012-09-10 09:19:59

ab wrote:

You are right, WebSockets may be implemented in the future.
But it is AFAIK implemented as a long polling (or lock-and-wait) scheme, based on HTTP headers.
So it is not exactly real "push"ing, but header-based plumbing based on HTTP one-way connection.

No, it is really low level binary TCP connection! (I made a Delphi implementation so I can know it )
With WinSocks you can do the same, because you can send data from client to server and from server to client via the connection handle anytime you want.
(you only need to actively check and wait for incoming data on the client side, I made a single thread for this which can wait for 64 connection at once).
So for Delphi, there is no real benefit to use WebSockets for duplex communication (can be done already, nothing really new) between Delphi server and Delphi clients.

However, we probably gonna use WS in Delphi too, because then we only need ONE http server port for both html4 clients and binary communication for html5 and Delphi clients .

ab · 2012-10-02 12:03:24

andrewdynamo wrote:

However, we probably gonna use WS in Delphi too, because then we only need ONE http server port for both html4 clients and binary communication for html5 and Delphi clients .

But since WS is not compatible with http.sys IOCP-driven kernel-mode server we like to use, and it has some DOS false-positive recognition issues, it sounds as not #1 priority.

ab · 2013-01-24 13:12:04

Some potential use of Event-Driven design, and some implementation patterns are available here:
http://synopse.info/forum/viewtopic.php?id=1054

This mechanism, integrated to the mORMot core, will:
- Publish dedicated event interfaces (i.e. "classic" event-driven process);
- Allow notification of interface based services;
- Allow notification of method-based services;
- Allow notification of CRUD/ORM operations.

Latest should be able to implement direct replication, event-driven storage, or CQRS.

ab · 2015-02-26 10:53:10

I've updated the feature request ticket, with a new implementation proposal.
See the last entry at http://synopse.info/fossil/info/aa230e52993ed7

Proposal is to use interface parameters to interface-based service methods to send a callback to the server.

When the client code supplies an instance of class (possibly self) for this parameter:

type
  IMyCallback = interface(IInvokable)
    ['{5CD49AB1-D5A3-4059-B287-25E939DD2A0B}']
    procedure EventCalledFromServer(const withAValue: integer);
  end;
  IMyService = interface(IInvokable)
    ['{FB31BF33-38C7-45AE-9511-6AC727ADC4F4}']
    procedure MethodWithCallback(const anyParameter: variant; const TheCallBack: IMyCallback);
  end;

type
  TMyServiceClient = class(TInterfacedObject,IMyCallback)
  protected
    fRest: TSQLRestClient;
    // here the callback will point back to "self", so IMyCallback is implemented now
    // but you may set a callback from any another class implementing IMyCallback
    procedure EventCalledFromServer(const withAValue: integer);
  public
    procedure Test;
  end;

procedure TMyServiceClient.Test;
var MyService: IMyService;
begin
  if fRest.Services.Resolve(IMyService,MyService) then
    MyService.MethodWithCallback(1234,self);
end;


procedure TMyServiceClient.EventCalledFromServer(const withAValue: integer);
begin // will be called back by the server
end;

type
  TMyServiceServer = class(TInterfacedObject,IMyService)
  protected
    fCallback: IMyCallback; // for delayed call, in sicClientDriven mode
  public
    procedure MethodWithCallback(const anyParameter: variant; const TheCallBack: IMyCallback);
    procedure AnotherMethod;
  end;

procedure TMyServiceServer.MethodWithCallback(const anyParameter: variant; const TheCallBack: IMyCallback);
begin
  TheCallBack.EventCalledFromServer(1234);
  fCallback := TheCallBack; 
end;

procedure TMyServiceServer.AnotherMethod;
begin
  if assigned(fCallback) then
    fCallBack.EventCalledFromServer(5678);
end;

Sounds pretty fine to me.
What do you think?

itSDS · 2015-02-26 13:18:04

I think it could be fine !

It may be Helpful to show a Progressbar for lengthen Server Operations on the Client

or to ask a question to the Client ?

is it sync as async ?

ab · 2015-02-26 13:53:04

It is a blocking scheme, but not synchronous.
The client callback could be executed whenever the server want, and it would even be by default executed in another thread than the client's.

itSDS · 2015-02-26 13:57:20

Is it possible to create callback function with Result, as on the server ?

ab · 2015-02-26 19:54:55

Of course yes.

In fact we may even make procedures asynchronous and functions synchronous...
This may be handy for events notifications in a publish/subscribe pattern.

mORMot Open Source

#1 2012-09-06 16:53:28

Roadmap: interface-based callbacks for Event Collaboration

#2 2012-09-07 06:30:18

Re: Roadmap: interface-based callbacks for Event Collaboration

#3 2012-09-07 08:58:25

Re: Roadmap: interface-based callbacks for Event Collaboration

#4 2012-09-10 09:19:59

Re: Roadmap: interface-based callbacks for Event Collaboration

#5 2012-10-02 12:03:24

Re: Roadmap: interface-based callbacks for Event Collaboration

#6 2013-01-24 13:12:04

Re: Roadmap: interface-based callbacks for Event Collaboration

#7 2015-02-26 10:53:10

Re: Roadmap: interface-based callbacks for Event Collaboration

#8 2015-02-26 13:18:04

Re: Roadmap: interface-based callbacks for Event Collaboration

#9 2015-02-26 13:53:04

Re: Roadmap: interface-based callbacks for Event Collaboration

#10 2015-02-26 13:57:20

Re: Roadmap: interface-based callbacks for Event Collaboration

#11 2015-02-26 19:54:55

Re: Roadmap: interface-based callbacks for Event Collaboration

Board footer