mORMot Open Source

avista

Member

Registered: 2014-01-06

Posts: 63

Interface Based Services - Called From Javascript (AJAX)

I'm new to the mORMot framework and I'm interested in using it to develop web services that can be called from Javascript applications. 

I'm not interested in ORM at this point, as I only need to leverage existing code to provide the web services (similar to what C# provides).

I modified the Project 14 calculator demo interface to include an extra function as a test, like so:

function LookupCustomer(ACustomerNumber: string): RawUTF8;

which is implemented like this:

function TServiceCalculator.LookupCustomer(ACustomerNumber: string): RawUTF8;
var
  Customer: TCustomer;
begin
  Customer := TCustomer.Create;
  try
    Customer.FirstName := 'First';
    Customer.LastName := 'Last';
    Result := ObjectToJSON(Customer);
  finally
    Customer.Free;
  end;
end;

When called from the client application, the JSON is returned as expected.  However, I have the following questions:

1. When calling the service from the client, these following tests are performed:

      if not Client.ServerTimeStampSynchronize then begin
         ShowMessage(UTF8ToString(Client.LastErrorMessage));
         exit;
       end;
       case ComboProtocol.ItemIndex of
       2:   TSQLRestServerAuthenticationNone.[b]ClientSetUser(Client,'User')[/b];
       else 
         Client.SetUser('User','synopse');
       end;

   If, for example, I need to call the service from a Javascript application, how would I perform these tests from Javascript?  Alternatively, how to I exclude them if I don't need them?

2. How can I route specific URLs to a specific method?  (e.g., if I perform a GET /root/v1/customers/100, how can I route that to my 'LookupCustomer' method call in the server?)

3. When I route a URL, how can I determine whether it was a GET, POST, PUT or DELETE so I know what behavior to implement on the server side?

4. If I need to return an error, the documentation states that for method based services I can use Ctxt.Error().  Is there something similar for interface based services (so I can return the TCustomer as a function result instead of RawUTF8 and still be able to return meaningful JSON errors like Ctxt.Error('Customer does not exist', [HTML_NOTFOUND]')?

5. I need every call to occur in a new thread, so I assume that sicPerThread is the option I need to use.  However, does mORMot support a thread pool? If so, how can I associate a custom object instance with each thread, which needs to be pre-allocated with database connections so it can be reused on subsequent service calls on that thread.  I saw the TServiceRunningContext class, but I'm not sure if I could use it for this purpose.

Thanks in advance for any help provided!

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: Interface Based Services - Called From Javascript (AJAX)

Thanks for your interest, and welcome!

0. Two remarks/suggestions:
- You could have used RawJSON instead of RawUTF8 to avoid a conversion to a JSON string, but direct JSON object return.
- You could also not use a RawJSON result, but just a good TCustomer out parameter:

function TServiceCalculator.LookupCustomer(const ACustomerNumber: string; out Customer: TCustomer): boolean;
begin
  Customer.FirstName := 'First';
  Customer.LastName := 'Last';
  result := true; // mark success
end;

1. Calling ServerTimeStampSynchronize is not mandatory. It is just to test the connection, and to get the time delta between the client and the server.
URI-Authentication is disabled by default. So for plain AJAX requests, it is not mandatory either.

2. Interface-based services URL routing is defined in a KISS principle in mORMot, i.e. fixed to some schemes, which are defined by classes.
For instance, TSQLRestRoutingREST defines /root/interfacename.methodname[/ClientDrivenID].
Or TSQLRestRoutingJSON_RPC defines /root/interfacename with the method name within the JSON input.
You can define your own class, inheriting from TSQLRestServerURIContext. But it won't be direct.
If you want custom routing, use method-based services, which gives access to all execution context.

3. You can get the HTTP call information from the threadvar ServiceContext.Request member.

4. For interface-based services, errors are usually triggered by raising an exception.

5. sicPerThread is about the class implementing the interface lifetime: one class instance will be available per running thread.
Since mORMot uses a thread pool internally, you should override TSQLRestServer.BeginCurrentThread and TSQLRestServer.EndCurrentThread methods to initialize your thread resources (e.g. initialize external database access).

avista

Member

Registered: 2014-01-06

Posts: 63

Re: Interface Based Services - Called From Javascript (AJAX)

I have further questions:

4. For interface-based services, errors are usually triggered by raising an exception.

How is this exception returned to the caller?  Is there a way to generate a specific exception to return the error to the caller in a JSON format as is provided with Ctxt.Error()?

2. Interface-based services URL routing is defined in a KISS principle in mORMot, i.e. fixed to some schemes, which are defined by classes.
For instance, TSQLRestRoutingREST defines /root/interfacename.methodname[/ClientDrivenID].
Or TSQLRestRoutingJSON_RPC defines /root/interfacename with the method name within the JSON input.
You can define your own class, inheriting from TSQLRestServerURIContext. But it won't be direct.
If you want custom routing, use method-based services, which gives access to all execution context.

It looks like both of these classes don't support standard REST URL behavior because they require the method names to be defined in either the body or the URL with the 'dot' notation.  Unfortunately, this won't work for me as I'm creating an API for other developers to access our system and I want the URLs to conform to the generally accepted standard REST URL format. 

If I override TSQLRestServerURIContext, how will that help me?  I'm not clear on what you mean by it won't be direct, unless you mean that the mapping between this class and the interface implementation won't be automatic.  If so, is there a way to manually create the mapping between the URL and the methods that get called (i.e., is there a way to gain control at the point of method dispatch so I can route the URL to the correct method?). 

Ideally, I'd like to map specific HTML verbs to specific method signatures and handle the parameters via the URL, like GET_Customer, POST_Customer, etc. or whatever convention I choose (I assume this is already being done somewhere in the code anyway). If I can override TSQLRestServerURIContext to create custom instance behavior, that would be great. If so, any example code (and/or which methods to override, etc.) would be greatly appreciated. 

Perhaps the only way to do this is by using method based services as you mentioned, but I'd prefer to use the interface based architecture if at all possible.

5. sicPerThread is about the class implementing the interface lifetime: one class instance will be available per running thread.
Since mORMot uses a thread pool internally, you should override TSQLRestServer.BeginCurrentThread and TSQLRestServer.EndCurrentThread methods to initialize your thread resources (e.g. initialize external database access).

Just so I understand, sicPerThread means that on the server side, an instance of my service class that implements the methods will be created per thread in the pool.  If this is the case, then can I simply create a class variable within the service class that references my database connection for each instance.  If so, does the service class instance lifetime get managed in the BeginCurrentThread/EndCurrentThread methods?  I obviously don't want the threads to be terminated until my server application is shutdown, so I want to be sure that I understand the lifetime management of the instance.  Also, are there any limits with the thread pool or does it expand as needed?  Can I specify how many threads are initially in the pool?

Thanks again for the help!

Last edited by avista (2014-01-06 19:21:37)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: Interface Based Services - Called From Javascript (AJAX)

How is this exception returned to the caller?  Is there a way to generate a specific exception to return the error to the caller in a JSON format as is provided with Ctxt.Error()?

The exception is returned as a server error 500 + some JSON object containing the Exception.Message content text.

If I override TSQLRestServerURIContext, how will that help me?

Just override the URIDecodeSOAByInterface/ExecuteSOAByInterface methods to handle the request as expected.
See TSQLRestRoutingREST and TSQLRestRoutingJSON_RPC as example.
I'll soon extend TSQLRestRoutingREST to handle URI level parameters. Stay tuned!

If so, does the service class instance lifetime get managed in the BeginCurrentThread/EndCurrentThread methods?  I obviously don't want the threads to be terminated until my server application is shutdown...

Yes: BeginCurrentThread/EndCurrentThread methods are called when the thread pool is initialized/finalized.
The thread pool will exist until the server is shut down.
You can specify how many thread you define in the thread pool - but 32 threads (the default) is enough to serve 50,000 concurrent connections. See http://blog.synopse.info/post/2013/09/1 … -of-mORMot
So I suppose this is a good default parameter.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: Interface Based Services - Called From Javascript (AJAX)

I've just committed some enhancements.
TSQLRestRoutingREST will now recognize several URI schemes.
See http://synopse.info/fossil/info/7b4dd9f2d3
and blog article http://blog.synopse.info/post/2014/01/0 … d-services

The new  root/Calculator/Add?n1=1&n2=2 alternative could be pretty convenient to be consumed from your REST clients.

Hope it helps.

foncci

Member

Registered: 2013-11-15

Posts: 53

Re: Interface Based Services - Called From Javascript (AJAX)

Hi ab,

Now I'm concerned about the best way to interact with an external database. So far I'm doing this way:

  TServiceIntegrador = class(TInterfacedObject, IServiceIntegrador)
    public
      DB : TFIBDataBase;
      Trans : TFIBTransaction;
      Query : TFIBDataSet;

      Procedure Connect;

      function UpdSQL(aComando, aData : RawJSON) : RawJSON;
      function echo(s : RawJSON) : RawJSON;

      destructor Destroy; override;
  end;

function TServiceIntegrador.UpdSQL(aComando, aData: RawJSON): RawJSON;
begin
  Connect; // Check if there is a connection, if not do it.
  ....
end;

begin
  CompressShaAesSetKey('TesingCipher');
  aModel := TSQLModel.Create([], cServiceName);
  aServer := TSQLRestServerFullMemory.Create(aModel, '', false, false);
  aServer.ServiceRegister(TServiceIntegrador, [TypeInfo(IServiceIntegrador)], sicPerThread). AllowAll; // .AllowAllByName(['User']);
  aHTTPServer := TSQLHttpServer.Create('8888', [aServer],'+',useHttpApiRegisteringURI, cCantThreads, secSynShaAes);

.....
end.

It is working pretty good but now I'm wondering if BeginCurrentThread/EndCurrentThread is a better way. I was looking in the samples but couldn't find anything. Can you provide us with a very simple sample that covers this topic?

By the way, I'm testing the encryption using wireshark/RowCap and nothing. This line fClient.Compression := [hcSynLZ, hcDeflate, hcSynShaAes]; is on the client.
I'm using today build. Am I missing something?

Thanks in advance,

Al

Last edited by foncci (2014-01-08 23:36:39)

avista

Member

Registered: 2014-01-06

Posts: 63

Re: Interface Based Services - Called From Javascript (AJAX)

I've just committed some enhancements.
TSQLRestRoutingREST will now recognize several URI schemes.
See http://synopse.info/fossil/info/7b4dd9f2d3
and blog article http://blog.synopse.info/post/2014/01/0 … d-services

The new  root/Calculator/Add?n1=1&n2=2 alternative could be pretty convenient to be consumed from your REST clients.

Hope it helps.

Yes, that definitely helps .  Thank you very much for the effort to implement this functionality, it is greatly appreciated!

Last edited by avista (2014-01-09 03:22:25)

cybexr

Member

Registered: 2016-09-14

Posts: 81

Re: Interface Based Services - Called From Javascript (AJAX)

Hi ab, Some sicPerThread problems

code based on Sample-16, I made little modification on sicPerThread, the result is not expected.

{ TServiceRemoteSQL }
procedure TServiceRemoteSQL.Connect(aEngine: TRemoteSQLEngine;
  const aServerName, aDatabaseName, aUserID, aPassWord: RawUTF8);
..
begin
  if fProps<>nil then
    raise Exception.Create('Connect called more than once');
  if TYPES[aEngine]=nil then
    raise Exception.CreateFmt('aEngine=%s is not supported',
      [GetEnumName(TypeInfo(TRemoteSQLEngine),ord(aEngine))^]);
  fProps := TYPES[aEngine].Create(aServerName,aDatabaseName,aUserID,aPassWord);
end;

function TServiceRemoteSQL.Execute(const aSQL: RawUTF8; aExpectResults, aExpanded: Boolean): RawJSON;
var res: ISQLDBRows;
begin
  if fProps=nil then
    Connect(rseOracle, '10.20.40.251/orcl', '', 'ydjg508_mid', 'pde');
  res := fProps.ExecuteInlined(aSQL,aExpectResults);
  if res=nil then
    result := '' else
    result := res.FetchAllAsJSON(aExpanded);
end;

destructor TServiceRemoteSQL.Destroy;
begin
  FreeAndNil(fProps);
  inherited;
end;


    aServer := TSQLRestServerFullMemory.Create(aModel,false);
      aServer.ServiceRegister(TServiceRemoteSQL,[TypeInfo(IRemoteSQL)],sicPerThread);
      aHTTPServer := TSQLHttpServer.Create(PORT_NAME,[aServer],'+',useHttpSocket); 

The server log displayed by Synlogview image:

The server log :

20180522 09343026 "O trace mORMot.TSQLRestServerFullMemory(020F29A0) BeginCurrentThread(THttpServerResp) root=root ThreadID=00001D40 ThreadCount=126
20180522 09343908 "O trace mORMot.TSQLRestServerFullMemory(020F29A0) EndCurrentThread(THttpServerResp) ThreadID=00001D40 ThreadCount=102

20180522 09345313 !$ trace mORMot.TSQLRestServerFullMemory(020F29A0) BeginCurrentThread(THttpServerResp) root=root ThreadID=00001D40 ThreadCount=111
20180522 09345650 !$ trace mORMot.TSQLRestServerFullMemory(020F29A0) EndCurrentThread(THttpServerResp) ThreadID=00001D40 ThreadCount=111

Client is apache JMeter, with 100 thread simultaneously, each thread will send 5 HTTP request body (JSON encoded SQL expected by the RemoteSQL.Execute URL).

It seems like RESTServer randomly pick one thread  then create TServiceRemoteSQL and process DB logics then release it.
With my expectation, the second picture #68 thread with threadID=00001D40 should run TServiceRemoteSQL.Execute and the fProps should be assigned .
But it seems the TServiceRemoteSQL instance is destroyed by previous #175 thread.

Last edited by cybexr (2018-05-24 04:29:58)

cybexr

Member

Registered: 2016-09-14

Posts: 81

Re: Interface Based Services - Called From Javascript (AJAX)

HTTPAPI mode works fine,  32 thread serves all 100-Jmeter clients, DB connection is reuesed by server thread.

The prev problem seems occurs with THttpServer (socket based ), the thread pool only serve reqest, but further calc&rest logic is dispatched with dedicate THttpServerResp thread everytime.
So when THttpServer working with sicPerThread mode SOA-service,  the SOA-interface instance lifetime is so short.

  /// a simple Thread Pool, used for fast handling HTTP requests of a THttpServer
  // - will handle multi-connection with less overhead than creating a thread
  // for each incoming request
  // - will create a THttpServerResp response thread, if the incoming request is
  // identified as HTTP/1.1 keep alive, or HTTP body length is bigger than 1 MB
  TSynThreadPoolTHttpServer = class(TSynThreadPool)

    // here aContext is a pointer(TSocket=THandle) value
    procedure Task(aCaller: TSynThread; aContext: Pointer); override;
      if (fServer.ServerKeepAliveTimeOut>0) and
         (fServer.fInternalHttpServerRespList.Count<THREADPOOL_MAXWORKTHREADS) and
         (ServerSock.KeepAliveClient or
          (ServerSock.ContentLength>THREADPOOL_BIGBODYSIZE)) then begin
        // HTTP/1.1 Keep Alive (including WebSockets) or posted data > 1 MB
        // -> process in dedicated background thread
        fServer.fThreadRespClass.Create(ServerSock,fServer);

and my question is why THttpServer internally desinged like this ?

cybexr

Member

Registered: 2016-09-14

Posts: 81

Re: Interface Based Services - Called From Javascript (AJAX)

In order to stick SOA-method thread enviorment within TSynThreadPool   , trying modify THREADPOOL_MAXWORKTHREADS = 0,

then test with Jmeter (100 client-thread concurrent ) , totaly 1000 http-request, result in about 30% broken connection ( Non HTTP response message: Software caused connection abort: recv failed ).

maybe the problem occurs at  function TSynThreadPool.Push(aContext: pointer): boolean;
  {$ifdef USE_WINIOCP}
  result := PostQueuedCompletionStatus(fRequestQueue,0,0,aContext);
  {$else}

and now the syncrtsock-IOCP is not fullfeature,   https://synopse.info/forum/viewtopic.php?id=4049.

It seems, HTTPAPI @windows + SOA sicPerthread is more proper.

Last edited by cybexr (2018-06-06 09:33:17)