#1 Re: mORMot 1 » mORMot REST WebService Cross Platform Client » 2018-06-05 10:11:43

Hello again!
Everything is working fine, but we are facing an issue; we would like to send from the client (cross-platform) a picture to the server. Everything is packed in a TDataSet and converted to JSON.
So the client is sending with the TJSONWriter from SynCommons:

var
  Data: PByte;
  blob: RawByteString;
Data := @DS.Fields.Fields[col].AsBytes[0];
with FromVarBlob(Data)
do SetString(blob, Ptr, Len);
Writer.WrBase64(Pointer(blob), Length(blob), True);

(This code is just like you are doing when transforming SQLDBRows to JSON when the server communicates with the client.)

The server receives the JSON and we use JSONToDataSet in order to get the TDataSet back.
But we could not achieve to get back the blob. We tried with TMemoryStream (TBlobField.SaveToStream), or even with TMemoryStream.WriteBuffer or Base64JSONStringToBytes...

Do you have an idea of what we are doing wrong?

#2 Re: mORMot 1 » mORMot REST WebService Cross Platform Client » 2018-03-07 10:18:34

Great!
Would you like to have the code we modified in order to change the JSON creation (adding the datatypes) in TSQLDBStatement.FetchAllToJSON when the expanded is False?
And also our function JSONToDataSet which fills a TDataSet with the JSON using the JSON with the types in it?

#3 Re: mORMot 1 » mORMot REST WebService Cross Platform Client » 2018-03-06 10:40:52

Finally we achieved to do that with :

  if ((DS.Fields[i].DataType = ftBlob) and (JSONTable.RowValues[i] <> Null)) then
  begin
    Base64JSONStringToBytes(JSONTable.RowValues[i], bytes);
    TBlobField(DS.Fields[i]).Value := TBytes(bytes);
  end

#4 Re: mORMot 1 » mORMot REST WebService Cross Platform Client » 2018-03-06 10:09:06

Hello again,

Okay, so we are still struggling a little bit in order to achieve our goal.
So far, we have changed a little bit the TSQLDBStatement.FetchAllToJSON function in order to save the data types in the JSON as the following :

{"fieldCount":2, "values": ["col1", "col2", "type1", "type2", "val11", "val12", "val21", "val22", ...] }

And we have created a function called JSONToDataSet which from a JSON string fills a TDataSet with the data in the JSON, using a TJSONTable for helping reading the JSON.

The problem is with handling blob fields. In the JSON it is stored base64 encoded, but in order to be used in the application, it should be decoded.
(we are doing : TBlobField(DataSet.Fields[ i ]).Value := JSONTable.RowValues[ i ])
Do you know how to do that ?
Thank you very much for your help.

#5 mORMot 1 » mORMot REST WebService Cross Platform Client » 2018-03-05 08:36:53

Ozim16
Replies: 7

Hello everyone,

Our project has a server (TSQLRestServerFullMemory) with a service defined, and functions returining RawJSON.
We have a client (Windows) with Delphi.

These methods returning RawJSON are returning Rows of DataSet with ISQLDBRows : Rows.FetchAllAsJSON(false).

The code works great on Windows, but with Android... In fact, we used SynCommons, mORMot and mORMotVCL in order to have the RawJSON type and the JSONToDataSet function.
With Android and other mobile devices, we should use SynCrossPlatform*. But all our functions do not work anymore. For example we had on the client side :

function ...(int id) : TDataSet;
begin
  JSON   := Methods.GetTopics(id);
  Result  := JSONToDataSet(Nil, JSON);
end

What should we use now ? Is it still possible to keep our IMethods interface, with functions returning RawJSON ?

Our server is a web services which returns results of SQL requests with datasets, in fact the client needs data from the database and the server gives him the datasets as JSON.

Thanks for your help.

#6 Re: mORMot 1 » mORMot Delphi REST Server without ORM » 2018-02-28 13:45:31

Hello again, we have another problem unfortunately...

The code works great on Windows, but with Android... In fact, we used SynCommons, mORMot and mORMotVCL in order to have the RawJSON type and the JSONToDataSet function.
With Android and other mobile devices, we should use SynCrossPlatform*. But all our functions do not work anymore. For example we had :

function ...(int id) : TDataSet;
begin
  JSON   := Methods.GetTopics(id);
  Result  := JSONToDataSet(Nil, JSON);
end

What should we use now ? Is it still possible to keep our IMethods interface, with functions returning RawJSON ?

Our server is a web services which returns results of SQL requests with datasets, in fact the client needs data from the database and the server gives him the datasets as JSON (with SQLToJSON).

Thanks for your help.

#7 Re: mORMot 1 » mORMot Delphi REST Server without ORM » 2018-02-16 11:20:13

Hello everyone,

The server and the client are working fine. We made some stress tests with it and we encountered some problems.
For the tests, first we ran the server and the client on the same computer. The server sends to the client results from the database (arount 40 000 rows) as JSON (the size is around 8,6Mo).
Then the client sends to the server a String and the server inserts a new row in a table of the database (for log reasons)
The client is running the request for 10 times. 40 clients are running at the same time.

The server stops the connections with the error :

 ECrtSocket --> 'SndLow 10054 WSAECONNRESET [Une connexion existante a dû être fermée par l'hôte distant]' 

Then we also tested running the client on a distant computer, and with only 20 clients, we have the following error :

 EWebSockets --> 'SockInPending() Error 10053 on :'
EcrtSocket --> 'SndLow 10053 WSAECONNABORTED [Une connexion établie a été abandonnée par un logiciel de votre ordinateur hôte]' 

So we would like to know what is the reason of such behaviour and how to have a stack of requests (if the server is struggling with too much requests) in order to have every client waiting for its answer.
Maybe the test is not really realistic because no client would ask for as much data.

Here is the Server definition :

fModel := TSQLModel.Create([], 'service');
fRestServer := TSQLRestServerFullMemory.Create(fModel, false);
ServiceFactoryServer := fRestServer.ServiceDefine(TRestMethods, [IRestMethods], sicSingle);
ServiceFactoryServer.SetOptions([], [optErrorOnMissingParam]);

fHTTPServer := TSQLHttpServer.Create(AnsiString(Port), [fRestServer], '+', useBidirSocket, 32, secSynShaAes);
fHttpServer.AccessControlAllowOrigin := '*'; // Allow cross-site AJAX Queries
TWebSocketServerRest(fHTTPServer.HttpServer).ServerKeepAliveTimeOut := 3000;
{ WebSocketServerRest := } fHTTPServer.WebSocketsEnable(fRestServer, '', True);

And for the Client definition :

fModel := TSQLModel.Create([], ROOT_NAME);
fClient := TSQLHttpClientWebsockets.Create(
              ServerName,      // Server
              Port,            // Port
              fModel,          // Model
              5000,     // Send Timeout
              5000,  // Receive Timeout
              10000); // Connect Timeout

TSQLHttpClientWebsockets(fClient).KeepAliveMS := 3000;
TSQLHttpClientWebsockets(fClient).Compression := [hcSynShaAes]; // Cryptage AES
(fClient as TSQLHttpClientWebsockets).WebSocketsUpgrade('', True);

// Service initialization
fClient.ServiceDefine([IRestMethods], sicSingle);
Result := fClient.Services.Resolve(IRestMethods, RestMethods); // same result, but no chance to make mistake with service name

Thank you very much for your help.

#8 Re: mORMot 1 » mORMot Delphi REST Server without ORM » 2018-02-14 15:32:15

Hi again.
We are working on the client side now and its works fine.
The service method returns a RawJSON (which was sent with DbConnection.Execute(...).FetchAllAsJSON(true)).
In order to store the data on the client side we are doing :
var values: TDocVariantData;
values.InitJSON(res, JSON_OPTIONS[true]);
DataSource1.DataSet := TDocVariantArrayDataSet.Create(Self,values.Values,[],[]);
DataSource1.DataSet.Open;

Is it the correct way to achieve it or is there a better/faster way to do ?

Thanks

#9 Re: mORMot 1 » mORMot Delphi REST Server without ORM » 2018-02-13 16:11:47

Finally we have found a solution for this problem.
For those who have the same problem, just add "ZConnection" in the uses statement where the TSQLDBZEOSConnectionProperties is created.

#10 Re: mORMot 1 » mORMot Delphi REST Server without ORM » 2018-02-13 16:00:08

Hello,

Thank you for your answers. Right now, we are testing the Interface-based services, it works good but we are struggling trying to connect the TSQLDBZEOSConnectionProperties to our database.
The database is on Firebird 3.0 on Port 3055 (localhost), everytime we have an error : "Classe d'exception EZSQLException avec un message 'Requested database driver was not found'". The connection URL is :
'zdbc:firebird-3.0://localhost:3055/D:\ATOM\database\ATOM.FDB?username=SYSDBA;password=masterkey;LibLocation=C:\Firebird_3_0\WOW64\fbclient.dll'
We tested with adding the ZConnexion in a form, it connects well and its URL is exactly the same as above.
By putting breakpoints, we saw that no Driver is in FDrivers property of TZDriverManager. Is something missing ? Maybe something like 'registerDriver ?

#11 mORMot 1 » mORMot Delphi REST Server without ORM » 2018-02-12 22:01:22

Ozim16
Replies: 11

Hello everyone,

We are working on a project where we would like to have a Server and a Client. Both will be in Delphi, the Server is a VCL application (should be for Windows), and the Client is a FireMonkey application.
First we tested DataSnap, but then we found out that it wasn't very performant and scalable, and because we will need to handle a lot of connections with Clients on smartphones or tablets, we would like to use mORMot to do the job.
Our problem is we would like to use mORMot only for its REST Server and Client framework without the ORM; in fact server methods will send Datasets to the client in JSON, and filling the Datasets with TFDQuery (our DataBase is in FireBird).
Is it possible to do so ? If yes which components should we use ?
According to the examples in mORMot's repository, should we use the TServerSQLRestServerFullMemory ? Do you have any code example which fits our need ?
Thanks a lot !

Board footer

Powered by FluxBB