Smart Mobile Studio mORMot class server

LoPiTaL · 2013-01-29 14:21:37

Hi everyone!
I have developed a Smart Mobile Studio class which is able to connect to the mORMot server, and I wanted to share it with you.
The class is able to:
· Handle security.
· Perform all the CRUD operations (delete, update, add and get) with ORM records.
· It is able to execute remote methods, including the usage of var and out parameters.

The SMS projects consists in:
- mORMotServer: which is where the mORMot connector resides.
- Computation: helper computation functions, crc32 and SHA256, extracted from mORMot's blog page:
http://blog.synopse.info/post/2012/04/1 … JavaScript
- JavaScriptHelpers: which contains several low-level functions.

Due to the lack of RTTI in SMS, the definition of the server methods and the ORM records are a bit strange, requiering in both cases two records instead of one (or one interface), but it is still pretty straight forward.

In the package, there are two samples, one using the "Project 4 server" sample that comes with the mORMot framework, to demonstrate the hability to work with ORM records; and another using the "Project 14 server", to demonstrate the hability to execute remote methods.

Also, it is included a unit called HTTPHelpers.pas, which contains a very basic class inherited from TSQLRestServerDB, TSQLRestServerDB_CORS, which implements CORS access control, thus enabling access to the server from different domain and port. This is useful when working from Smart Mobile Studio and testin the javascript application from the web browser. Without this, it won't work. Just replace your TSQLRestServerDB with TSQLRestServerDB_CORS, and that's all.

Requiered versions:
· SmartMobileStudio: 1.1.0.372 (or higher)
· mORMot framework: today's one, 10:47 - 2013-01-29, 9bae7dc630

Link (I am unable to upload it to the forum):
https://www.dropbox.com/s/um7p1j2bwow5q … 0Smart.rar

Best regards,
LoPiTaL

Roberto Schneiders · 2013-01-29 16:01:34

Excellent. Thank you for sharing.

Maybe we can include these classes in the framework, what do you think Arnaud?

I'm not able to compile in version 1.1.0.371 Smart Mobile Studio :
Syntax Error: There's no accessible member with name "properties"! [line: 613, column: 20, file: mORMotServer]

In my case the example is not working. By clicking on connect nothing happens. If I find something I post here.

Roberto Schneiders · 2013-01-29 16:08:30

Interesting. Apparently only works in Internet Explorer.

In Chrome just does not work. In Firefox the following message appears:
This site () is requesting data storage on your computer for offline use

By clicking on allow nothing happens, the message remains on the screen. Connect doesn't work.

ab · 2013-01-29 16:38:43

This is just awesome news!

I do not have dropbox access from here, but I'll check it soon.

Thanks a lot for sharing!

We will certainly review it, and integrate to the main mORMot code.
Perhaps some kind of http-based service browser and a SMS code generator for the interfaces.

I'll also check TSQLRestServerDB_CORS, since it is also on the roadmap.

ab · 2013-01-29 19:04:56

I just checked your code.

This is legend.... dary!

Could we integrate it directly in the official mORMot source tree?

I still can't understand why SMS uses this awful XML .opp format.
Just unmaintainable for any serious project.

LoPiTaL · 2013-01-29 20:53:14

Hi everyone!
Thanks for all the answers!

Roberto Schneiders wrote:

I'm not able to compile in version 1.1.0.371 Smart Mobile Studio :
Syntax Error: There's no accessible member with name "properties"! [line: 613, column: 20, file: mORMotServer]

I have uploaded a new version that solves the problem with the properties field (it was introduced in 1.1.0.372). Now the projects compiles in 1.1.0.371+.

Roberto Schneiders wrote:

Interesting. Apparently only works in Internet Explorer.
In Chrome just does not work. In Firefox the following message appears:
This site () is requesting data storage on your computer for offline use

Doing some tests I have found that Internet Explorer ignores the CORS directives (probably it ignores only the different ports part), while the others follows the CORS as is. Due to this, it work only in IE and not in the others.
To solve this, check the following:
· You must be using, as the project samples servers, the provided TSQLRestServerDB_CORS class instead of TSQLRestServerDB or TSQLRestServerFullMemory or any other, in order to do cross site access (although the cross site is limited to the port). This class is in the HTTPHelpers.pas unit (this unit is for Delphi, not for SMS).
· Be sure to have the lattest (today) version of the repository, or at least, correct manually the following bug (they are only a couple of pointers, easy to workaround): http://synopse.info/fossil/info/91f8f3ec6f

I also found an inconvenient with Firefox: encodeURL() does not encode the simple quotes, while the browser does, doing the signature invalid. I solved it in the newly updated version.

I tried in IE 9, Firefox 18.0.1 and Chrome 24.0, and it works in all three. The only problem was in IE 9, that if I use the IP address instead of 'localhost', it does not work; with 'localhost', it worked fine. With the other two, no more problems.

Roberto Schneiders wrote:

This site () is requesting data storage on your computer for offline use

Yes, this is normal. It appeared also to me.

ab wrote:

I'll also check TSQLRestServerDB_CORS, since it is also on the roadmap.

It is a very very basic implementation of the CORS, just allowing ALL connections from EVERYWHERE. Since the security is handled by the server, this is not really a problem, but it would be nice to be able to choose from where and so...
There is also a nice class to work with headers, making really easy the work with them, and not having to play with Pos, length and so in a single string.

ab wrote:

This is legend.... dary!

Thank you

ab wrote:

Could we integrate it directly in the official mORMot source tree?

Of course! I uploaded it with that in mind. Check that you have my lattest upload (to check it, look in the JavascriptHelpers.pas file and see if there is a function called 'VariantProperties')

ab wrote:

I still can't understand why SMS uses this awful XML .opp format.
Just unmaintainable for any serious project.

Yeahh I know! And if you choose to use external files, the files MUST be placed in the same folder as the project, useless anyway!!!

Thank you for all your comments!! I am glad you liked it!

Best regards,
LoPiTaL

warleyalex · 2013-01-30 02:55:33

That's a nice work!

About Demo 1 - Project 4 - ORM records manage works just fine in Chrome 19.

In Firefox 8, it show a ugly message "This website () is asking to store data on your computer for offline use."

but you can handle this with:
Tools > Options > Advanced > Network --> Tell me when a website asks to store data for offline use.

The problem is: in FF 8, only the "Connect" button works, you're authenticated at least.

Thank you

LoPiTaL · 2013-01-30 08:06:53

Hi!
Under Chrome it seems to work perfectly fine. Under IE it also worked fine.
In Firefox, I have tried in versions 13.0, 14.0 and 18.0 (the updater didn't let me chose any other version ) and also it has worked. Can you try with a newer version of firefox?

Best regards,
LoPiTaL

Last edited by LoPiTaL (2013-01-30 08:07:09)

ab · 2013-01-30 10:06:47

I've added TSQLHttpServer.AccessControlAllowOrigin property to handle cross-site AJAX requests via cross-origin resource sharing (CORS).
By default, it is set to '', so cross-site will be disabled.
Implementation is not complete: it won't check the "Origin: " incoming header, but it will let AJAX request work.

Its sounded better to include it at TSQLHttpServer level, since CORS is a HTTP-dedicated feature.

Therefore, server side of sample "04 - HTTP Client-Server" has been modified as such:

procedure TForm1.FormCreate(Sender: TObject);
begin
  Model := CreateSampleModel;
  DB := TSQLRestServerDB.Create(Model,ChangeFileExt(paramstr(0),'.db3'),true);
  DB.CreateMissingTables(0);
  Server := TSQLHttpServer.Create('8080',[DB]);
  Server.AccessControlAllowOrigin := '*'; // allow cross-site AJAX queries
end;

It now runs your corresponding SMS index.html page.

See http://synopse.info/fossil/info/9d97879a75

mpv · 2013-01-30 13:27:25

CORS is good news. Actually in modern system it used instead JSONP. But AB, maybe better to move CORS support from TSQLHttpServer to THttpServerGeneric?

ab · 2013-01-30 13:40:01

mpv wrote:

maybe better to move CORS support from TSQLHttpServer to THttpServerGeneric?

I asked myself the very same question.

But I suspect that it is mostly a global setting, dedicated to HTTP remote access protocol.
THttpServerGeneric is a TThread, so it is at lower level.
Putting this at TSQLHttpServer level was easier.

It is a first attempt (it does not check incoming "Origin: " header), and is to be improved, in all cases.
I commit this in order to have AJAX cross-site working as expected, to demonstrate integration of SMS and mORMot.

LoPiTaL · 2013-01-30 13:59:34

ab wrote:

I've added TSQLHttpServer.AccessControlAllowOrigin property to handle cross-site AJAX requests via cross-origin resource sharing (CORS).
http://synopse.info/fossil/info/9d97879a75

Sorry, it does not work.
If you reply with the "Access-Control-Allow-Origin: *" header, then all "read" methods will work (this is, GET and OPTIONS). Even the browser does not query for the access control (does not send an OPTIONS method), just send it and accepts it if that header exists.

But when you want to send a POST or like, then the OPTIONS query is sent with a header called "Access-Control-Request-Headers", which contains the requiered headers to be CORS accessible. The server MUST response to that query with another header called "Access-Control-Allow-Headers", which contains an EXACT match of the requiered headers that contained the OPTIONS query.

With this implementation, only CONNECT, DISCONNECT and LIST will work. You cannot ADD, DELETE or UPDATE records.

If you look at my implementation, you will see that when I prepare the headers to be sent, all are constant, in exception of the "Access-Control-Allow-Headers", which I take from the LHeadersIn.

Just wondering, why don't take the THTMLHeaders class and use it within the TSQLHTTPServer? It is just a stringlist and simplifies the work with HTTP headers A LOT. It can parse directly all the headers with the Text property, and after working with them, retrieve them again with the same property.

Best regards,
LoPiTaL

ab · 2013-01-30 15:12:44

It was a quick implementation... I failed totally the OPTIONS process!

THTMLHeaders class is a nice idea, but is also slower than expected, since breaking some implementation patterns of mORMot (like temporary memory allocations, and conversion between string<->RawUTF8). That is why I did not use it.
In order to retrieve the value of an incoming header, I prefer to use FindIniNameValue() and ExistsIniNameValue() which avoid any memory allocation.

I suppose http://synopse.info/fossil/info/aa1fa79b05 should do the trick.

Thanks for your feedback!

LoPiTaL · 2013-01-30 17:10:34

ab wrote:

THTMLHeaders class is a nice idea, but is also slower than expected, since breaking some implementation patterns of mORMot (like temporary memory allocations, and conversion between string<->RawUTF8)

Ok, I got it.

ab wrote:

I suppose http://synopse.info/fossil/info/aa1fa79b05 should do the trick.

Yes! It works perfectly fine now!! Tried in IE 9, Firefox 18.0.1 and Chrome 24.0

Feel free to add the SMS units to the repository!

Best regards,
LoPiTaL

ab · 2013-01-31 16:30:15

I would like to enhance the SmartMobileStudio side code, in order to be able to directly use a class with a method for interface-based services, and perhaps make cleaner the CRUD access.

I will probably also rename mORMotServer.pas into mORMotClient.pas, and all *Server names into *Client, since it is a client class.

And include a compiled .html file within the corresponding sample folders.

LoPiTaL · 2013-02-02 13:53:20

Hi Ab! I also think that the code can be improved. I published it as I get it working, so it actually can be improved.
If you don't mind, I can try to help you with that, since I have already touched the SMS RTL.

ab wrote:

I would like to enhance the SmartMobileStudio side code, in order to be able to directly use a class with a method for interface-based services, and perhaps make cleaner the CRUD access.

This is a bit hard. Since SMS does not have RTTI, the class (or interface) can not be defined just as in Delphi and be registered with the client.
I think that it would be possible to prepare an executable, that can be compiled before the server and launched in the "On After Compilation" event of the server, that could be used to generate an SMS unit with the classes / interfaces ready to be used with the client class in SMS and the mORMot server.
In fact, it should only prepare a class with a private inher class. The outer class (being also interfaced with the server's interface in the case of the interface-based services) will have the declared server methods. The inner class should be the currently "external" class, to be used as the JSON generator / parser.

About the CRUD access, I think that the aproach should be very similar, but the difference here is that the outer class may be the very same to all records (since the methods are the same), just changing the inner class to the correct "external" class.
Of course, from the user's point of view, this would be all invisible (all JSON fields and so being private) and he will only see the declared server methods.

What do you think?

Edit: I was looking now to the SynProject page, and I have seen that it has a built-in Delphi parser. Can it be used in the executable I was talking about? So it would not be necessary to recompile the exe all the times, just passing to it the source files as parameters and it can parse them and generate the corresponding units. The only needed step would be to call the executable in the "On After Compilation" event of the server.

ab wrote:

I will probably also rename mORMotServer.pas into mORMotClient.pas, and all *Server names into *Client, since it is a client class.

Yes, the *server sufix was just refering "connect to server", but yes, it should be client. This is an easy change.

ab wrote:

And include a compiled .html file within the corresponding sample folders.

This can be done just including the compiled .html generated by the SMS compiler. I removed it just to save space, I thought that everybody could just compile it and regenerate it, but I can include it.

Now I haven't got my computer, but the next week I will be working on all of this.

Best regards,
LoPiTaL

Last edited by LoPiTaL (2013-02-02 15:22:19)

ab · 2013-02-02 15:50:50

This is what I had in mind!
That is, have a hidden method able to create some SMS client wrapper code (or C#), from the server side itself.
Similar to the "mex" endpoint to be added for WCF servers, in order to access the WSDL from outside.
I wanted to allow some quick HTTP browsing of the available ORM classes, method-based services and interface-based services, from any browser.
For ORM classes and interface-based services, no need to read the source code, just create the SMS/C# wrapper classes from the RTTI available at runtime on the server.
We may had an option to the server (with command line or external config file), or enable the feature only if the server is run within the Delphi IDE (which could make sense, and seems to be secure).

I'm very thankful about the fact that your shared your code, which works well.
Starting from working code is always much easier than writing something from scratch, and, as you say, the SMS RTL is a bit weird, when it comes to interaction with Javascript and its internal data/types representations.
I saw the questions you posted on the SMS forum - you did spend some time in research!

I'm quite confident we are able to publish some mORMot servers process very easily to work with SMS, with no external .js files (needed e.g. if you want to access to DataSnap or RemObjects).

Did you see that I've added logging on console feature (with colors) to mORMot?
I found it very easy to debug interactively client remote access, e.g. from SMS applications.

Any help and feedback is welcome.

warleyalex · 2013-02-02 19:04:16

Just a word.

If you want to do an app in a web site where you have complete control over the way everything is, and indeed have to manually do everything, then SMS looks good, but Jquery wouldn't be a better choice? You can perform all CRUD operations, you can handle security and it's able to execute remote methods.

LoPiTaL · 2013-02-02 19:24:52

ab wrote:

I wanted to allow some quick HTTP browsing of the available ORM classes, method-based services and interface-based services, from any browser.
For ORM classes and interface-based services, no need to read the source code, just create the SMS/C# wrapper classes from the RTTI available at runtime on the server.

Ok, this is the piece of the puzzle I wasn't seeing, so I needed some parser or external .exe to generate the RTTI of the interfaces. But the RTTI is already in the server!... Nice!

ab wrote:

We may had an option to the server (with command line or external config file), or enable the feature only if the server is run within the Delphi IDE (which could make sense, and seems to be secure).

I think it would be better if implemented as a method only executable by the administrator, using the security features of the server. It would be a demonstration of the security of the server, and would be perfectly integrated in it.

ab wrote:

Did you see that I've added logging on console feature (with colors) to mORMot?
I found it very easy to debug interactively client remote access, e.g. from SMS applications.

Nop, I haven't got my computer with me, so I haven't got access to Delphi nor SMS today. On Monday I will check it

ab wrote:

I'm quite confident we are able to publish some mORMot servers process very easily to work with SMS, with no external .js files (needed e.g. if you want to access to DataSnap or RemObjects).
Any help and feedback is welcome.

Whatever you need, please, don't hesitate to ask!
After doing the changes to the SMS unit we talked on the previous post, I am going to try to create an interface-based service on the server which returns the interface-methods of the server as a SMS unit. If it can be possible, the work with SMS will be very very simplified and even error proven!

When I have something, I will tell you

warleyalex wrote:

Just a word.
If you want to do an app in a web site where you have complete control over the way everything is, and indeed have to manually do everything, then SMS looks good, but Jquery wouldn't be a better choice? You can perform all CRUD operations, you can handle security and it's able to execute remote methods.

warleyalex, yes, working with SMS is just like a wrapper to javascript (and then, also a wrapper for Jquery, as it is at top of javascript). The problem then, is that you have to know two completely different languages: Javascript and Delphi. With SMS, you only need to know Delphi to work in both sides. You have Delphi for the server, Delphi for stand-alone clients and now, Delphi for the web. When we finish what we have been talking here, then you won't need to do anything manually, the security will be managed by the mORMotClient class, and the CRUD operations and the remote methods will be handled by the auto-generated classes, so the working with the web-clients will be simplified.

Best regards,
LoPiTaL

Last edited by LoPiTaL (2013-02-02 19:31:08)

ab · 2013-02-02 20:54:49

What we can do is to make it in two steps:

1. You write some "manual interface" code, for both ORM and interface-based services.
Purpose of it is to be integrated to SMS, and easy to use.

Something which may work like this:

Calculator.Add(A, B, lambda(result) W3EditBox1.Text := 'Result='+IntToStr(result) end);

So you can use const/var/out parameters from the definition of the TCalculation method itself.

For the ORM, having the same naming on both sides (TSQLRecord) and perhaps the same definition, could make sense.
BLOB retrieval is also needed.
Dynamic arrays are an issue here, since they are not AJAX ready (binary + Base64 encoding).

2. Then I create the unit wrapper on the server side, from RTTI and mORMot internal structures.
I will use your SMS side code as reference implementation.

We would need to properly handle "Client-driven" instance life time, also.

What do you think of?

LoPiTaL · 2013-02-03 14:08:06

Perfect! I like the idea.
I'm going to work on it when I arrive home.
When I have something, I will tell you, so you can continue with the second step.

Best regards,
LoPiTaL

andrewdynamo · 2013-02-07 12:03:10

ab wrote:

This is what I had in mind!
That is, have a hidden method able to create some SMS client wrapper code (or C#), from the server side itself.
Similar to the "mex" endpoint to be added for WCF servers, in order to access the WSDL from outside.
I wanted to allow some quick HTTP browsing of the available ORM classes, method-based services and interface-based services, from any browser.
For ORM classes and interface-based services, no need to read the source code, just create the SMS/C# wrapper classes from the RTTI available at runtime on the server.
We may had an option to the server (with command line or external config file), or enable the feature only if the server is run within the Delphi IDE (which could make sense, and seems to be secure).

Any progress on this side? (about the "metadata" endpoint of the mORMot server)
Because then an auto generator could be included in the SMS IDE itself, which can update/create
the client side classes e.g. from a menu option.
The best would be to have full metadata, so if a remote function returns an object, to have a lightweight
client side class generated too for this object (maximum type safety, code completion, etc).

Side note: DataSnap does not offer full metadata for the proxy generators, so only simple function wrappers
can be made . I probably need to make my own metadata retrieval for DS because of this, in order to be able
to generate DS wrappers for the SMS IDE. So maybe we can share some work/ideas etc to get an universal
metadata system? (I know, DS is a competitor but SMS should support as much RPC servers as possible, and this
way we can at least add full mORMot support in the SMS IDE soon too )

ab · 2013-02-07 17:18:44

About this integration, I'm waiting for LoPiTaL reference code.

Including wrapper generation within the SMS IDE does make sense, of course!
mORMot has JSON-based metadata available, which can be used very easily to generate any wrapper.

Client-side auto-generation also - just like with mORMot Delphi implementation of the client-side interface-based services (we do not generate wrappers, but just "fake" classes, from RTTI).
But having the methods signatures available during code writing, using IDE intellisense, is a plus. We can do that on Delphi client, since we have interfaces, so we may be able to share the interface also on SMS client side.

I like also very much the TypeScript approach, and its strong-typing add to javascript.
You can supply the TypeScript compiler a set of "interface headers", to add to the compilation context, when writing your client.
I would like to have similar wrappers created - perhaps the link of auto-generated client-side javascript classes may be written in Smart (if it can be generic enough), then integrated into TypeScript or any "pure" JavaScript client.

Any idea sharing is warmly welcome.
If you have any implementation idea (i.e. typical SMS code which may be used on client side), do not hesitate to share it.

LoPiTaL · 2013-02-07 21:06:07

Sorry for my delay, but I have been refactoring the code a bit, combining both projects so only one copy of the library must be maintained. Also it seems more to the original mORMot code, with calls like FillPrepare, FillOne, GetService, etc...
The ORM part is already working fine, and almost all the interface-based services part. I am testing now with var and class parameters (out parameters are not supported by SMS).
In any time now, I will have the code and will share with you.

Best regards,
LoPiTaL

ab · 2013-02-07 22:47:18

This is great news!

We are waiting for it!

Thank you so much!

LoPiTaL · 2013-02-07 23:26:33

Ok, so it is here!
I have uploaded to Dropbox a new version with the changes we talked.
Changes mainly are:
. Renamed *Server by *Client.
· Added assynchronous / synchronous queries.
· Refactored the methods / interfaces for the interfaced-based services usage. Now it can be used directly interfaces, being the weird TMethod / JMethod pair hidden.
· Modified the TmORMotRecord class, so now it have functions like FillPrepare, FillOne, CreateAndFillPrepare, etc... The pair TmORMotRecord / JmORMOtRecord is hidden also.
· Added and handled sicClientDriven life time.

Several notes:
· Since the shutdown of SMS is not guaranteed to be executen by the web browser, it is necessary the usage of timeouts in the server to manage unclosed connections and clientDriven instances not freed (I guess this is already implemented, but just confirming it).
· I have modified the definition in the interface of the DemoProject4, in order to have an interface with var parameters and class parameters. Also the new definition is meant to be created with sicClientDriven, so the Accum method is usefull.
· SMS does not support out parameters.
· The interface retrieving must be done by name, but this shouldn't be a problem since the name will be autogenerated
· Be warning! Both projects (4 and 14) works with "localhost" as server name, but nº 4 uses port 8080 and nº 14 888, so choose the correct one for each project.

ab:
The autogenerated requiered code is the one found in the two Intf_* units (Intf_ORM.pas and Intf_Services.pas). I tried to be as rigid as possible with the names, always naming the same way things related (ICalculator, TCalculator, JCalculator, JCalculator_Add....). Please, tell me if the code is OK and if it is possible to automate.
Note the "weird" declaration of the var parameters in the JSON struct (i.e. "_VAR_SumResult"). It must be this way (begining with _VAR_), so SMS can detect which variables are "var" and return the correct values.

Any question, problem or suggestion, don't hesitate to ask.
Best regards,
LoPiTaL

ab · 2013-02-08 09:25:48

Great news!
I'm quite confident we will be able to create the wrappers as expected.

Some questions:

1. The SQLFormat() function only accept string parameters to bound - we may have to put "array of const" here, and format numbers, BLOBs or dates as expected by the mORMot inline format.

2. What about using a record, or even a variant, instead of class external (JObject)?
I do not think we need a class for those structures, which purpose is only to make JSON serialization easy (if I understand it correctly).

3. The code is working great, but there is something I still can't understand.
Why is this code working, on SMS client side:

function TCalculator.Add(n1,n2: integer): integer;
 (...)
    Client.Execute(LMethod);
    LJSON:=JCalculatorInterface_Add(LMethod.Field);
    Result:=LJSON.result;
(...)

Is not javascript supposed to be asynchronous?
I did not understand well your synchronous/asynchronous implementation.
Could you explain it a little?

LoPiTaL · 2013-02-08 15:17:22

Hi ab!

1. The "array of const" is buggie, the compiler throws an exception, so I put it aside when I was working in that function. Nevertheless, I returned now to it, and digging a bit (and with more knowledge of the inners of SMS now than then) I have found a workaround, so I will implement it as "array of const". In fact, I have got it already working (but passing all the items to string ).

2. The record is not usable, since there are no pointers in SMS, then you cannot typecast a record to any other kind. So you cannot declare any function expecting a generic record, so the parsing from record to variant MUST be always done with asm end sections.... It is not usefull.
The variant can be used without more problems.

I have chosen to use "class external (JObject)" because I have got the code mistakes thrown by the compiler, and not in runtime, being less error prone. Since it is supposed (now) to be an autogenerated code, it should not contain any errors, but still, I think is clearer to use the class externals. Why don't you like this kind of classes?

3. It is because the calling is done in a synchronous mode. In fact, the interface-based services MUST be run in synchronous mode.
I know that javascript is asynchronous, but the Javascript object used to perform HTTP queries (XMLHttpRequest) supports also synchronous mode (https://developer.mozilla.org/en-US/doc … s_Requests). I don't know where the trick is, but it works and it is supported by every web browser...

About my implementation, in TmORMotClient, there is a property called Assynchronous. If this is set, then the query is performed asynchronously: it returns from the function without having the response.
Also there is an "internal handler", which is set in all the class methods and is where the next code of the function will be executed. Then, when the internal handler is called, this handler calls the external one, that is where the user's code handling the result of the query must be placed.
If this variable is not set, then the internal handler is called inmediately after the query, performs the internal operations and return without calling the external handler.

I.e:

function TmORMotClient.AddRecord(ARecord: TmORMotRecord): Boolean;
  //Internal handler. This method will be called both, in synch and in asynch mode.
  function AddedRecord(AResponse: string): Boolean;
  var
    LVariant: Variant;
  begin
    //Process the response
    REsult:=False;
    if (AResponse <> '') then begin
      LVariant:=JSON2Variant(AResponse);
      if ErrorReturned(LVariant) then begin
        //Error handler is always executed, both in synch and in asynch
        CallErrorHandler(JmORMotError(LVariant));
        Exit;
      end;
    end;

    //External handler is only executed if in asynch mode (the app is not waiting the function's result, so we must call it explicitly).
    CallExternalHandler();
    Result:=True;
  end;

var
  LData: string;
  LResponse: string;
begin
  Result:=False;
  LData:=JObject2JSON(ARecord[0]);

  //Prepare asynchronous callback
  FOnQueryCallBackInternal:=
  procedure(AResponse: string)
  begin
    AddedRecord(AResponse);
  end;

  //Perform the post
  LResponse:=Post(ARecord.TableName, LData);

  //If we are in synchronous mode, call manually the internal handler and return.
  if not FAssynchronous then
    Result:=AddedRecord(LResponse);
end;

A bit tricky, but this way both types of connections are supported, but for the interface-based services!!
Maybe I should refactor the interface-based services a bit more to support it also, though I don't see any advantage in making asynchronous calls in this part of the framework.
What do you think?

Best regards,
LoPiTaL

ab · 2013-02-08 18:05:46

Thanks a lot for the precision.
I understand it by now.
I thought that due to its asynchronous nature, we had to use lambdas to return the values (see my post from some last week, above).
But using blocking communication just does make sense and is much easier to work with.

How do you add a "waiting spinner" in SMS? Is there anything "native"?

No problem about class external(JObject) - it was just a weird syntax, but when generated as JavaScript, it will use a plain javascript object - so no speed penalty.

As far as I can see, you had to circumvent some unexpected compiler problems, so I'm very happy you faced them and killed the beast!

About session closing, what about the window.onbeforeunload event?

Could you refresh your source code (including "array of const" version of SQLFormat), so that I will work on the wizard generation?

I think it is my turn to work, now.

Thanks a lot!

LoPiTaL · 2013-02-08 19:09:01

ab wrote:

But using blocking communication just does make sense and is much easier to work with.

I though that also, so I will not modify any more the interface-based services.

ab wrote:

How do you add a "waiting spinner" in SMS? Is there anything "native"?

What do you mean?

ab wrote:

No problem about class external(JObject) - it was just a weird syntax, but when generated as JavaScript, it will use a plain javascript object - so no speed penalty.

Yes, it is a bit weird, but it does the trick: inheritance and checking in compile time in a JavaScript object - just perfect

ab wrote:

About session closing, what about the window.onbeforeunload event?

Maybe, I have to check it.

ab wrote:

Could you refresh your source code (including "array of const" version of SQLFormat), so that I will work on the wizard generation?

It is done. Tomorrow I will try to implement the features you have, this is, use integers, strings and so on, and not only strings.

When I have something new, I will tell you, but it is almost done, apart from the SQLFormat function and the disconnecting process, isn't it?.

Best regards,
LoPiTaL

Last edited by LoPiTaL (2013-02-08 19:23:58)

ab · 2013-02-08 20:00:36

LoPiTaL wrote:

ab wrote:
How do you add a "waiting spinner" in SMS? Is there anything "native"?
What do you mean?

The animated circle which is displayed as popup on web pages, when something is happening in the background.
If your service is expected to take some time, it could be a good idea to show such an animation.

LoPiTaL wrote:

When I have something new, I will tell you, but it is almost done, apart from the SQLFormat function and the disconnecting process, isn't it?.

Indeed!
Everything sounds very good so far!

LoPiTaL · 2013-02-10 00:22:50

ab wrote:

The animated circle which is displayed as popup on web pages, when something is happening in the background.
If your service is expected to take some time, it could be a good idea to show such an animation.

Yes, there is a class "TW3CustomAnimation" which can be set up to spin an image like that one. I haven't explored too much, so maybe there is something more suitable...

The only problem with this is that it can be done only when connected using assynchronous mode, because the synchronous mode hangs the UI.
To make this possible, I have implemented the assynchronous interface-based service, so now everything can be done assynchronously and the UI stays responsive.

NOTE that I have changed the Intf_Services file!!!

Also I have changed a bit the working of the assynchronous mode making the callbacks per query instead of globally: since the user can press several buttons at a time, the callbacks can be changed while in the query, so now each callback is stored in the HTTP object instead of storing it in the client to prevent the changing of the callback while waiting the answer from the server.
I have also added a new function (LongCall) to test this functionality, and it works pretty fine.

Tomorrow I will implement the SQLFormat.

Best regards,
LoPiTaL

LoPiTaL · 2013-02-10 11:42:27

Ok, I have implemented the Variant2string function, used in the SQLFormat function, so its behaviour is pretty similar to the mORMot one's. The main difference is that in the SMS are not supported Pointers, Classes nor Currency. Thus, it is only supported strings, integer, float and boolean.
Pointers will not be supported (since SMS itself does not support them).
Classes I don't see the need of them in a SQL query, so I haven't implemented it...
Currency is not supported directly in SMS, and since it is just a float number (or int64, don't remember), SMS detects the Variant type as one of those, so it is not possible to generate it. But since it can be done manually before calling the SQLFormat function it is not a big problem (just follow the mORMot implementation).

Best regards,
LoPiTaL

ab · 2013-02-10 11:52:30

Nice.
Parameters are still always inlined with quotes, whereas quotes should be only for strings, not numbers nor null.
So we may need some like:

  LActualSQLWhere := LActualSQLWhere+LInBetween+':(';
  if TVariant.IsString(LVariantBounds[i]) then
    LActualSQLWhere:=LActualSQLWhere+QuotedStr(TVariant.AsString(LVariantBounds[i]))  else
  if TVariant.IsNull(LVariantBounds[i]) then
    LActualSQLWhere:=LActualSQLWhere+'null'  else
    LActualSQLWhere:=LActualSQLWhere+TVariant.AsString(LVariantBounds[i]);
  LActualSQLWhere := LActualSQLWhere+'):';
...

What about date/time? or blobs?
They have a special "magic" at the beginning of the encoded string, and BLOBs are Base-64 encoded.
See JSON_SQLDATE_MAGIC and JSON_BASE64_MAGIC_QUOTE in SynCommons.pas.

I do not expect blobs to be handled now, but I think date/time should be worth encoding.
We may at least define DateToSQL/DateTimeToSQL functions, just as with SynCommons.pas.

LoPiTaL · 2013-02-10 18:34:06

Hi again!
I have almost both, date/times and blobs working.
I can convert from TDateTime to string an viceversa, and the same for the blobs in Base64 encoding.

But I can't make them work when transmitting to the server. I have several questions about it:
1. How is the TDateTime transmitted when used in a query, when it is appended to the URI?
I.e. I want to transmit Date<=:(2013-03-10):
and I actually transmit
Date%3C=:(%27%C2%B1%C2%BF%C3%AF2013-03-10%27)
Is it correct?

2. How is the TDateTime transmitted when used in a record object, this is, when it is transmitted in the BODY of the request? Right now, I have got it transmited as a float (directly the TDateTime variable). Is it correct?

Best regards,
LoPiTaL

ab · 2013-02-10 19:03:11

1. This is not correct - it should be U+FFF1 as Unicode - i.e. %EF%BF%B1 as UTF-8.

2. TDateTime is to be transmitted as Iso8601 TEXT within the JSON content, not floating point number.
See the documentation about this point.

warleyalex · 2013-02-12 02:14:37

About proj n. 04, Content-Type header = "application/xml" shouldn't be = "application/json"?
The response body type is returning XML for DELETE, ADD and DELETE ALL FROM NAME buttons.

In Chrome:
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Content-Type: application/xml
---
In Firefox:
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Content-Type: text/plain; charset=UTF-8

----
About proj n.14, I'm getting a 400 bad request error when I click button "Add, AddComplex, Accum,
LongCall". Only button [+] seems to work here.

I'd like to test this functionality, but seems that I have do add functions (AddComplex, Accum, LongCall) at proj n.14.

ab · 2013-02-12 07:54:41

Yes, you have to use either a server, either another.

When we will adapt the SMS source code, we will certainly publish a dedicated server, publishing both services.

LoPiTaL · 2013-02-12 11:30:32

warleyalex wrote:

I'd like to test this functionality, but seems that I have do add functions (AddComplex, Accum, LongCall) at proj n.14.

The modified interface for that project adding the other functions is also included in the rar package of the SMS project. Just replace the default one by the modified one, recompile, and reexecute the server.

ab: I have found that TDateTime and TBytes (blob) will need a special treatment in the autogenerated code. Both are already working in the ORM side, but I want to try them in the interface-based part before publishing anything. Hopefully today or tomorrow I will publish it. I tell you, so you can wait a bit more before begining the autogenerating procedure.

Also I want to prepare a little Delphi and SMS projects to be used as a Test Framework (TDD ? ) and let the sample SMS project easy to understand, just connecting to the samples 4 and 14 as they are right now, because the SMS project is begining to be quite dificult to follow.

Best regards,
LoPiTaL

ab · 2013-02-12 14:28:19

I'm currently fighting against UniDAC + Zeos database backends, to integrate them with our SynDB classes.
It is working with Oracle, but not with other providers (like Firebird)... yet...

Results are pretty interesting!
http://blog.synopse.info/post/2013/02/1 … RMot-s-ORM

Now, back to SMS.

Eric · 2013-02-14 14:11:59

ab wrote:

I'm currently fighting against UniDAC + Zeos database backends, to integrate them with our SynDB classes.

Impressive stuff!

ab wrote:

Now, back to SMS.

Following this from afar at the moment (spare time goes into improvements & fixes for 1.1).

With 1.1 there is some RTTI capability in SMS (class attributes, reading & writing published fields, invoking published properties getters & setters), it's rather raw and not wrapped up, but it's there

ab · 2013-02-14 17:28:54

RTTI abilities of SMS may be very helpful for clients like ours...
Is there any possibility to get the RTTI from an "interface" definition?
Is it possible to get the published fields RTTI for a SMS class?
This should be all we need by now.

About new TDataset / DB.pas abilities of our SynDB classes, I guess this will benefit also directly DWS database layer.
Firebird is now working via UniDAC and ZEOS layers, but I'm waiting for our direct SynDBFirebird.pas unit to publish some feedback.

andrewdynamo · 2013-02-14 21:18:25

Quick reply: yes also RTTI for interface is generated (but only when used because of smartlinking, can be probably tweaked using settings, eric will know that )
Maybe tomorrow a quick test

andrewdynamo · 2013-02-15 07:36:05

Update: in the upcoming update of SMS 1.1 you can use the following:

  W3Button1.OnClick := lambda
    var rtti := RTTIRawAttributes;
    for var ir :=Low(rtti) to High(rtti) do
    begin
      var attrib := rtti[ir];
      ShowMessage( JSON.Stringify(attrib) );
    end;
  end;

Interface are not directly supported, only via a class:

  ITest = interface
    function GetTest: Integer;
    property Test: Integer read GetTest;
  end;

  TTest = class(ITest)                            <-- rtti reports ITest
  published
    function GetTest: Integer;
    property Test: Integer read GetTest;
  end;

And only properties and fields are supported, no functions (yet?).
So for now, rtti cannot be used for mORMot.

P.S. extended usage info by looking at the unit test code:
http://dwscript.googlecode.com/svn-hist … d_enum.pas
http://dwscript.googlecode.com/svn-hist … y_enum.pas

ab · 2013-02-15 11:30:16

Thanks for the research and the info.

LoPiTaL · 2013-02-15 15:43:47

Interesting the RTTI for SMS!!
In Delphi, the mORMot only uses properties for the records definitions, so maybe this can be all what is needed for the ORM part!! No more need for autogenerating it....
This weekend I will try it.

About the other part, interface-based services, we would need there the methods / functions RTTI to be able to work with it without autogenerating, but anyway, it looks pretty interesting.

I hope this weekend to have prepared the test project and the basic usage project, and maybe among some of the RTTI tests.

Best regards,
LoPiTaL

LoPiTaL · 2013-02-24 18:16:40

Finally!
After a bussy week and a hard time with the SMS' RTTI I have prepared a new release of the client.
The ORM part is pretty similar to the Delphi's one, it is not required to autogenerate anything on this part. Everything is managed through the RTTI, and it works pretty fine . It even can be shared with the Delphi server with some {$IFDEF}s.
The interface-based service is similar to the previous release, so here the autogeneration is requiered until the SMS RTTI supports functions and interfaces also.

I have prepared also a new project, SMSTests, meant to be the test bench of the features of the client. In it, all the features are tested: ORM, interface-based services, connection / disconnection, blobs and dates; everything in both behaviours, synchronously and assynchronous.
The old project, mORMotSampleProject, remains as it was, and it is an easy way to get started with the client.

About the tests, I have tried it with the SMS client, Firefox and Chrome and it worked fine on them. The problem was with internet explorer, which sometimes works fine, sometimes does not work at all (the most times). I don't know why...

Best regards,
LoPiTaL

ab · 2013-02-24 18:23:37

This is JUST AWESOME!

Thanks a lot!

martin.suer · 2014-03-22 15:32:00

Hi,

is the archive in the dropbox mentioned in this thread the latest version of the mORMot/SmartMS-Integration?
Since I can't find it in the mORMot nightly 1.18 build, is it "officially" part of mORMot or not?

With that and Smart 2.0.1 I can access the ORM but have troubles using Interfaced Server.

Martin

ab · 2014-03-22 16:04:51

There is no "official" SMS integration yet.

But I'm working on it.
Stay tuned!

mORMot Open Source

#1 2013-01-29 14:21:37

Smart Mobile Studio mORMot class server

#2 2013-01-29 16:01:34

Re: Smart Mobile Studio mORMot class server

#3 2013-01-29 16:08:30

Re: Smart Mobile Studio mORMot class server

#4 2013-01-29 16:38:43

Re: Smart Mobile Studio mORMot class server

#5 2013-01-29 19:04:56

Re: Smart Mobile Studio mORMot class server

#6 2013-01-29 20:53:14

Re: Smart Mobile Studio mORMot class server

#7 2013-01-30 02:55:33

Re: Smart Mobile Studio mORMot class server

#8 2013-01-30 08:06:53

Re: Smart Mobile Studio mORMot class server

#9 2013-01-30 10:06:47

Re: Smart Mobile Studio mORMot class server

#10 2013-01-30 13:27:25

Re: Smart Mobile Studio mORMot class server

#11 2013-01-30 13:40:01

Re: Smart Mobile Studio mORMot class server

#12 2013-01-30 13:59:34

Re: Smart Mobile Studio mORMot class server

#13 2013-01-30 15:12:44

Re: Smart Mobile Studio mORMot class server

#14 2013-01-30 17:10:34

Re: Smart Mobile Studio mORMot class server

#15 2013-01-31 16:30:15

Re: Smart Mobile Studio mORMot class server

#16 2013-02-02 13:53:20

Re: Smart Mobile Studio mORMot class server

#17 2013-02-02 15:50:50

Re: Smart Mobile Studio mORMot class server

#18 2013-02-02 19:04:16

Re: Smart Mobile Studio mORMot class server

#19 2013-02-02 19:24:52

Re: Smart Mobile Studio mORMot class server

#20 2013-02-02 20:54:49

Re: Smart Mobile Studio mORMot class server

#21 2013-02-03 14:08:06

Re: Smart Mobile Studio mORMot class server

#22 2013-02-07 12:03:10

Re: Smart Mobile Studio mORMot class server

#23 2013-02-07 17:18:44

Re: Smart Mobile Studio mORMot class server

#24 2013-02-07 21:06:07

Re: Smart Mobile Studio mORMot class server

#25 2013-02-07 22:47:18

Re: Smart Mobile Studio mORMot class server

#26 2013-02-07 23:26:33

Re: Smart Mobile Studio mORMot class server

#27 2013-02-08 09:25:48

Re: Smart Mobile Studio mORMot class server

#28 2013-02-08 15:17:22

Re: Smart Mobile Studio mORMot class server

#29 2013-02-08 18:05:46

Re: Smart Mobile Studio mORMot class server

#30 2013-02-08 19:09:01

Re: Smart Mobile Studio mORMot class server

#31 2013-02-08 20:00:36

Re: Smart Mobile Studio mORMot class server

#32 2013-02-10 00:22:50

Re: Smart Mobile Studio mORMot class server

#33 2013-02-10 11:42:27

Re: Smart Mobile Studio mORMot class server

#34 2013-02-10 11:52:30

Re: Smart Mobile Studio mORMot class server

#35 2013-02-10 18:34:06

Re: Smart Mobile Studio mORMot class server

#36 2013-02-10 19:03:11

Re: Smart Mobile Studio mORMot class server

#37 2013-02-12 02:14:37

Re: Smart Mobile Studio mORMot class server

#38 2013-02-12 07:54:41

Re: Smart Mobile Studio mORMot class server

#39 2013-02-12 11:30:32

Re: Smart Mobile Studio mORMot class server

#40 2013-02-12 14:28:19