Replace DataSnap by mORMot on the HTTP layer of a DBX based project

Roberto Schneiders · 2015-10-16 13:51:13

Hello,

I'm trying to do some crazy stuff with mORMot.

I'm working on a project that is basically a Datasnap TCP stateful (IAPPServer) with a lot of DBX (Datasets, Clientdatasets, etc) which I can not get away from, despite wanting to.

We are having serious trouble to scale this thing, so, we want to make a stateless REST server. Amazingly, this is not simple to do with Datasnap REST, I mean, it is possible, but, I will have to do manually all the conversion Dataset -> JSON, JSON -> Dataset (also deltas). So, I'm thinking, If I have to do this, why not use mORMot in the HTTP layer?

First, I tought in use only the ninja JSON serialization of mORMot, but I think it would be very nice to use the http layer too. I am even wondering if I can communicate using binary (I don't have to access this data in any other client, so, the fastest, the better.).

Do you guys have any insight about this?

I haven't tried anything on the HTTP layer yet, but I had some trouble with the JSON serialization and datasets, I'm trying to use the existing datasets helpers. Probably this wasen't the purpose of this classes, or, I just using them wrong, I'm not sure. I tried to do thinks like this:

function TMormotTest.DatasetToJson(Dataset: TDataSet): string;
begin
  Result := SynVirtualDataset.DataSetToJSON(Dataset);
end;

function TMormotTest.JsonToDataSet(json: string): TDataSet;
var
  Dataset: TClientDataSet;
begin
  Dataset:= TCDSUtil.GetEmptyDataSet;
  JSONToClientDataSet(Dataset, json, nil, cdsReplace, false);
  result := Dataset;
end;

The SynVirtualDataset.DataSetToJSON converts the blob fields to base64 and the JSONToClientDataSet does not take this into consideration.

ab · 2015-10-16 15:32:21

Do not use json here.

There is already a binary format with high performance server using SynDBRemote. Try in this direction.

Or you may use websockets as communication layer over a regular mORMot rest server. It supports compression and encryption as binary. And asynchronous callbacks if needed in the future of your system.

Roberto Schneiders · 2015-10-16 16:41:59

Thank you, I will take a look in SynDBRemote.

Roberto Schneiders · 2015-10-16 17:17:35

Using SynDBRemote, I have access to any connection implementation (TFDConnection, TSQLConnection) that allows me to use the existing datasets with it?

Roberto Schneiders · 2015-10-16 17:51:25

I was able to make it work with SynDBDataset. Thanks.

Roberto Schneiders · 2015-10-16 20:03:33

Unfortunately, we have some process (SQL) that have to be made on the server (actualy using datasets events), so, I don't think this will work for us.

Do you have any idea how I can serialize and send a OleVariant through TSQLRestServer?

ab · 2015-10-17 16:00:20

What is stored in the OleVariant?

If it is a string or a number, it will be transmitted by value.

danielkuettner · 2015-10-17 18:43:48

OleVariant is a binary value, not documented.
But why not use ClientDatasetToJSON (http://synopse.info/fossil/info/3c2a6e082a)?

ab · 2015-10-17 20:40:45

An olevariant variable can store a standard variant value ... Or a proprietary type which may contain an invokable instance.

By definition this invokable has no standard json serialization...

Roberto Schneiders · 2015-10-19 11:33:34

In this case, I want to serialize the ClientDataSet Data and Delta properties.

The problem with DatasetToJSON danielkuettner is the JSON, in first place, I would prefer something more optimized for speed. The other problem is that I don't have a method that does the inverse. JSONToClientDataSet dose'n work because of the blobs base64 encoding, as stated on the first post.

danielkuettner · 2015-10-19 13:20:56

@Roberto Schneiders
First I have to thank you, because you was the man, who brought me to mORMot (with your speed comparing article).

IMHO you get the best performance if you use mORMot as it was made for: Interface based services with JSONs. Than you have replaced all the slow and not really stable Datasnap overhead.

The only part you have to do is to work with your CDSs, generate JSONs to send to server (ClientDataSetToJSON is already there) and to convert the JSONs from the server in to the CDSs. Therefore you can use _JSON (e.g.) to get a variant. With that you can set the fields of the CDSs. I think this should not be a performance killer.

In another step you can replace all the CDS-stuff to get a native mORMot multitiered application.

If you plan to send and get the OleVariants you have to work with Blobs, therefore you have to use method based services (I guess) and you see no params and so on. This is not so transparent and I can't see any advantages over all.

PS: the Delta of the CDS could set to another CDS which can be serialized by ClientDatasetToJSON

Last edited by danielkuettner (2015-10-19 13:22:52)

Roberto Schneiders · 2015-10-19 17:30:38

haha, I'm glad you liked it. mORMot is awesome, I switched to another job two years ago (working mostly with Ruby, Go) and no longer had the opportunity to work with it, maybe this will change now.

Yeah, I'm doing some tests with Interfaced Based Services now. I was able to serialize the olevariant data and delta with some crazy code (http://stackoverflow.com/questions/2481 … conversion)

I think I will do just a tiny benchmark to compare to our Datasnap TCP, just for fun.

mORMot Open Source

#1 2015-10-16 13:51:13

Replace DataSnap by mORMot on the HTTP layer of a DBX based project

#2 2015-10-16 15:32:21

Re: Replace DataSnap by mORMot on the HTTP layer of a DBX based project

#3 2015-10-16 16:41:59

Re: Replace DataSnap by mORMot on the HTTP layer of a DBX based project

#4 2015-10-16 17:17:35

Re: Replace DataSnap by mORMot on the HTTP layer of a DBX based project

#5 2015-10-16 17:51:25

Re: Replace DataSnap by mORMot on the HTTP layer of a DBX based project

#6 2015-10-16 20:03:33

Re: Replace DataSnap by mORMot on the HTTP layer of a DBX based project

#7 2015-10-17 16:00:20

Re: Replace DataSnap by mORMot on the HTTP layer of a DBX based project

#8 2015-10-17 18:43:48

Re: Replace DataSnap by mORMot on the HTTP layer of a DBX based project

#9 2015-10-17 20:40:45

Re: Replace DataSnap by mORMot on the HTTP layer of a DBX based project

#10 2015-10-19 11:33:34

Re: Replace DataSnap by mORMot on the HTTP layer of a DBX based project

#11 2015-10-19 13:20:56

Re: Replace DataSnap by mORMot on the HTTP layer of a DBX based project

#12 2015-10-19 17:30:38

Re: Replace DataSnap by mORMot on the HTTP layer of a DBX based project

Board footer