Problem with client application

array81 · 2011-04-01 10:14:50

I decided to write this post because I still have many problems with my application and with SQLite framework. My application can works as standalone application, server application or client application. It works very well as standalone application or server application. I have problem only with try to run it as client application.

I post some piece of code:

var
  Server: TSQLite3HttpServer;
  Database: TSQLRestClientURI;
  DatabaseServer: TSQLRestServerDB;
  Model: TSQLModel;

when the user choose the "run mode", the application call one of these functions:

// RUN AS STANDALONE
function TMainForm.LoadLocalDatabase(Filename: string; Overwrite: Boolean): Boolean;
begin
  Result := False;

  if FileExists(Filename) then
    begin
      if Overwrite then
        begin
          DeleteFile(Filename);
        end;
    end;

  CloseDatabase();
  ResetData();

  if Database <> nil then
    Database.Free;

  try
    Database := TSQLRestClientDB.Create(Model, nil, Filename, TSQLRestServerDB);
    TSQLRestServerDB(TSQLRestClientDB(Database).Server).CreateMissingTables(0);
    Result := True;
    RefreshAllGUI;
  except
    on E: Exception do
      begin
        // ERROR
      end;
  end;
end;

function TMainForm.LoadClientDatabase(Server: string; Port: Integer; UserName, Password: string): Boolean;
begin
  Result := False;

  CloseDatabase();
  ResetData();

  if not Ping(Server, 1000) then
    Exit;

  if Database <> nil then
    Database.Free;

  try
    Database := TSQLite3HttpClient.Create(Server, IntToStr(Port), Model);
    Result := True;
    RefreshAllGUI;
  except
    on E: Exception do
      begin
        // ERROR
      end;
  end;
end;

function TMainForm.LoadServerDatabase(Port: Integer; Filename: string; Overwrite: Boolean): Boolean;
begin
  Result := False;

  if FileExists(Filename) then
    begin
      if Overwrite then
        begin
          DeleteFile(Filename);
        end;
    end;

  CloseDatabase();
  ResetData();

  if Database <> nil then
    Database.Free;

  if Server <> nil then
    Server.Free;

  try
    CambiaStato(clBusy);
    Database := TSQLRestClientDB.Create(Model, nil, Filename, TSQLRestServerDB);
    DatabaseServer := TSQLRestClientDB(Database).Server;
    DatabaseServer.CreateMissingTables(0);
    DatabaseServer.OnUpdateEvent := DatabaseAggiornatoEvento;
    Server := TSQLite3HttpServer.Create(IntToStr(Port), DatabaseServer);
    Result := True;
    RefreshAllGUI;
  except
    on E: Exception do
      begin
        // ERROR
      end;
  end;
end;

where Model is create on MainForm.Create:

Model := GetDatabaseModel();

When use the application as client I have many problem, very often that the application get or set data from/to database is very slow (sometime it have a small freeze), sometime is impossible add or update a simple data... (all these with antivirus and firewall disabled). The errors are not constant but random.

I just have post this problem and the Administrator tried to help me (thanks again AB) but I have this problem again, we thought at the speed of LAN (I have test it on 2 LAN, one with all Windows XP computers and one with all Windows 7 computer).

However I don't think the problem is the LAN speed, because I use my LAN in my Office without any problem with other software (I have different software that need of LAN connection).

Besides I have made a fast test, I have try to run the application as standaline application then I have used as database a file on another PC (through LAN), in this way all data traveling on the LAN. In this way the application works very well, without problem, fast and without error.

Can someone give me some suggestions?

ab · 2011-04-01 12:38:47

Still with the new http.sys server and latest version of the client?

array81 · 2011-04-01 13:27:23

I use last version of the framework (I have update it today).

On last version http.sys server is used by default on Windows XP, right? I make my test on all Windows XP LAN so I think the framework use the new http.sys option.

array81 · 2011-04-14 17:27:39

Can the new log system help me with my problem? How?
I receive different type of error ("sndlow -10053", "sockreculn 10038", "synln 0", ...) but I don't understand it.

array81 · 2011-04-15 13:04:15

On my Home LAN (Windows 7) I have uninstall the antivirus (Microsoft Essential Security) then I have open the port 8080 on my firewall (the default Windows firewall), I have do it on all my LAN computers.
All these not change nothing, I have try to load some data from client: to load about 10 kB I need of about 3 seconds. To load the data I use about 12 Database.ExecuteList.

ab · 2011-04-16 06:11:51

array81 wrote:

I receive different type of error ("sndlow -10053", "sockreculn 10038", "synln 0", ...) but I don't understand it.

Those errors are WinSock errors:

WSAECONNABORTED - Error 10053
See http://support.ipswitch.com/kb/WSK-19980702-EM02.htm

WSAENOTSOCK - Error 10038
See http://support.ipswitch.com/kb/WSK-19980714-EM22.htm

The first one is like a low-level communication error.
There are some problems in the TCP layer of the protocol: this is a far lowest level than our HTTP use of the stack.

array81 · 2011-04-16 08:30:34

So, the problem is not the framework, right? Do you have some ideas about the causes? I have made my test on 2 very different LAN (first LAN, old computers with Windows XP and LAN cable, second LAN, all new computers with Windows 7 and WI-FI LAN) with the some reseult, is this strange?

Thanks

ab · 2011-04-16 09:51:51

With http.sys as the server, there should be no speed problem at all from this side.

I just tried to use WinINet instead of our SynCrtSock routines for the client.
See http://synopse.info/fossil/info/e1b2effaf9
You can test it by defining USEWININET conditional for the whole application, then using a TSQLite3HttpClientWinINet class instead of TSQLite3HttpClient for the client.

Locally, it performs very bad, 10-20 times slower than our SynCrtSock.
But you could make a test with your remote network.

array81 · 2011-04-16 14:34:58

I have try USEWININET.

In this way my application is very speed that before! No error until now, very small lag time (about 1 second). I have repeat my last test to load the same 10 KB of data I need a time between (0.1 and 0.5 seconds (I have test some time) and not 3 second.

This is very strange!

NOTE in this moment I have make the test only on my home lan (windows 7), monday I will test it on office lan (windows xp).

Can you explain what this?

Is there a way to understad if the framework use or not use http.sys?

I will make some other test.

Last edited by array81 (2011-04-16 15:16:57)

array81 · 2011-04-18 07:26:10

First test on Windows XP LAN, as for Windows 7 LAN my application doesn't have error and it about x3 time more faster.

ab · 2011-04-18 07:59:32

So there should be a problem with the timeout or configuration of the standard client using SynCrtSock.

If run locally on the same computer (for instance in the TestSQL3.dpr test program), SynCrtSock is 10 times faster than WinINet.
If run remotely on your networks, SynCrtSock is 3 times slower than WinINet.

I was not able yet to figure out what's wrong.
The configuration of the socket seems exactly the same as with Indy and with Synapse HTTP components...
If you have any idea, I'm open to any suggestion.

array81 · 2011-04-18 08:56:03

If the socket is exactly the same as with Indy and with Synapse HTTP components will not be easy to find the problem. In the past I just have use Indy component on my LAN but I don't remember similar problem.

Two things:

1) Have you ever tested the framework on a real LAN? To understand if this problem is a common problem of LAN use or a paricular problem (for only some user like me).

2) Until you not found the problem, can you complete the TSQLite3HttpClientWinINet class? (better than nothing, in alternative I need change database framework because in this moment with SynCrtSock my program is unusable).

ab · 2011-04-18 09:35:29

1) Yes, I did, and didn't find such issues.

2) I'm finishing TSQLite3HttpClientWinINet:
* TSQLite3HttpClientWinINet now handle optional incoming and outgoing data compression, and fix some potential issues in Unicode Delphi
* TSQLite3HttpClientWinINet is now available by default (WITHWININET conditional is defined)
See http://synopse.info/fossil/info/047920baba

And I'll certainly handle WinHTTP API as an alternative to WinINet. It was told to be faster, and can used within services or server applications.
The code should be made common to WinHTTP and WinINet, this the global approach from the API is the same.

ab · 2011-04-18 13:13:26

Here we are.
I've finished a port of a client class using the WinHTTP API.
It seems definitively very fast, much faster than TSQLite3HttpClientWinINet and even faster than the previous TSQLite3HttpClient class over a network.

See http://synopse.info/fossil/info/9f663b344e and http://synopse.info/fossil/info/59de047ba4

Could you please take a look at this over your network, and tell us about the resulting speed?
I hope this will be a solution to our performance issues.

array81 · 2011-04-18 14:53:13

Hi,

I have try the new version on my office LAN (Windows XP) I cannot measure the speed because I don't have installed a http traffic check on my office LAN but it very fast, more faster that TSQLite3HttpClientWinINet, I don't have lag time any more.

I need make some other test because I have get 1 error on 10 attempts. I need to understand if it's my error or a framework error.
This evening I hope to finish my test and post a reply.

ab · 2011-04-18 15:42:00

Make sure that you got the latest version for your tests.

I just made some corrections, and made TSQLite3HttpClientWinHTTP the default official TSQLite3HttpClient.

I'll wait for your testing before releasing officially the 1.13 version of the framework.

array81 · 2011-04-18 17:08:22

I small thing before my real report post (this evening).

When compile my application start the console, there is not data on console, only a black console. Can I disable it?

ab · 2011-04-18 17:35:08

Yes, I think I disabled all the console write in the new version of the unit.

So it will be OK to disable the console.

ab · 2011-04-18 18:08:52

There was a console allocated... it's now disabled.

And I enhanced the error handling during HTTP request.

Please make sure you have downloaded latest version.
At least http://synopse.info/fossil/info/1ded4137f4

array81 · 2011-04-18 23:19:32

As promised I have the latest version of the framework (ab5e2473ec).

Very good speed and I don't have found error.

I actually had a small error, however I am convinced that depends to my code. In fact in the past, with the big lag time, I could not use TSQLRestClientURI.UpdateFromServer so I used message between server and client by TCP protocol, when TSQLRestServer.OnUpdateEvent was called on server application the server application sent a message to all client then the clients loaded data from database.
However now, without lag this system get me some problem.

I have disabled it and now I don't have any problem. So I think you framework now works on my 2 LAN Many thanks.

Two things:
1) to replace my "update system" I need use TSQLRestServer.OnUpdateEvent on server and TSQLRestClientURI.UpdateFromServer on clients, right?
2) with the new WinHTTP API use, do I need always "TestSQL3Register.dpr" system?

Thanks again.

ab · 2011-04-19 06:45:04

I'm happy to hear that your performance problems are gone.
I think we made some progress here. WinHTTP will be the default client for revision 1.13 of the framework.

array81 wrote:

1) to replace my "update system" I need use TSQLRestServer.OnUpdateEvent on server and TSQLRestClientURI.UpdateFromServer on clients, right?

Yes, you can do that this way. But be sure your update pattern is really stateless.

array81 wrote:

2) with the new WinHTTP API use, do I need always "TestSQL3Register.dpr" system?

WinHTTP is for the client side only.
So it doesn't change the server part, so if you want to use HTTP Server API (http.sys) you still need to register your urls on Vista/Seven.

array81 · 2011-04-19 10:02:37

I think I don't understand what mean "But be sure your update pattern is really stateless."

ab · 2011-04-19 11:20:10

array81 wrote:

I think I don't understand what mean "But be sure your update pattern is really stateless."

Be sure that the Server is the only reference, and that the Client is able to retrieve any pending update from the Server side.
That is, always modify a record content on a server side, then refresh the client to retrieve the modified value.
Do not modify the client side directly, but always pass through the Server.
The UI components of the framework follow these principles.

Client-side modification could be performed, but must be made in a separated autonomous table/database.

This will avoid any synchronization problems between concurrent client modification.

See "REST is Stateless" paragraphs in the documentation (just updated these days for the upcoming 1.13 release).

array81 · 2011-04-19 13:22:32

OK, when I change something from client I must update only the database (no client GUI) then I must use only TSQLRestClientURI.UpdateFromServer to update the client GUI, right?

This approach is not very intuitive but I understand that is the most correct. Now I must some test to try to understand how TSQLRestClientURI.UpdateFromServer works

A last things, if I use the previous approach, how do my application can update the GUI if it works a standalone application? Can I use TSQLRestServer.OnUpdateEvent as "server mode"? I don't think this is possible but I'd like use a common code.

ab · 2011-04-19 13:37:27

array81 wrote:

A last things, if I use the previous approach, how do my application can update the GUI if it works a standalone application? Can I use TSQLRestServer.OnUpdateEvent as "server mode"? I don't think this is possible but I'd like use a common code.

If the stand alone mode is exclusive to the server mode (that is the data is not shared between the two modes), there is no problem to handle the standalone application as "server mode". Think as the stand alone application like a client/server with both sides on the same computer: for the user, it's only one program, but from the code, it has two sides one server and one client at the same time.

array81 · 2011-04-19 13:43:54

OK, I have understand I need use TSQLRestServerDB and TSQLRestClientURI but LOCALHOST as host.

It's possible add onther example to framework to see TSQLRestClientURI.UpdateFromServer work with TSQLTableJSON, a simple grid.

A simple demo with a server a client with grid and update function for the client.

array81 · 2011-04-20 17:18:17

I need some advice for client update. I have about 10 grids on my application, some of these grids are "complex" grids with tree structure and other features.

I use some TSQLTableJSON to fill them, then I destroy all my TSQLTableJSON. This is wrong, right? I should not destroy it and use it to call TSQLRestClientURI.UpdateFromServer, right?

A second thing: if my standalone application works like a server/client application (on the same machine) I can use always the some TSQLRestClientURI.UpdateFromServer function to upgrade the GUI and avoid OnUpdateEvent event, right?

Last edited by array81 (2011-04-20 17:25:50)

ab · 2011-04-20 18:23:41

array81 wrote:

I use some TSQLTableJSON to fill them, then I destroy all my TSQLTableJSON. This is wrong, right? I should not destroy it and use it to call TSQLRestClientURI.UpdateFromServer, right?

Yes, UpdateFromServer is the right way of doing it.
Because if there is no data modification from the server, it won't recreate the table content.
But if you are sure that everytime you destroy it the data DID change, there won't be any big difference.

array81 wrote:

A second thing: if my standalone application works like a server/client application (on the same machine) I can use always the some TSQLRestClientURI.UpdateFromServer function to upgrade the GUI and avoid OnUpdateEvent event, right?

Yes, you should use the same UpdateFromServer method.
Think as the GUI of your program like a Client software.
From the User point of view, it won't make a difference, but from the code, it will be a somewhat a bit difficult at first to get it right, but you'll find it great later on:
- because you can share the same GUI code for both stand alone and remote client mode;
- you'll maintain a better n-Tier architecture, i.e. separate the GUI from the database and business logic.

I've emphasized those architecture choice in the latest state of the documentation, to be officially released with version 1.13.

mORMot Open Source

#1 2011-04-01 10:14:50

Problem with client application

#2 2011-04-01 12:38:47

Re: Problem with client application

#3 2011-04-01 13:27:23

Re: Problem with client application

#4 2011-04-14 17:27:39

Re: Problem with client application

#5 2011-04-15 13:04:15

Re: Problem with client application

#6 2011-04-16 06:11:51

Re: Problem with client application

#7 2011-04-16 08:30:34

Re: Problem with client application

#8 2011-04-16 09:51:51

Re: Problem with client application

#9 2011-04-16 14:34:58

Re: Problem with client application

#10 2011-04-18 07:26:10

Re: Problem with client application

#11 2011-04-18 07:59:32

Re: Problem with client application

#12 2011-04-18 08:56:03

Re: Problem with client application

#13 2011-04-18 09:35:29

Re: Problem with client application

#14 2011-04-18 13:13:26

Re: Problem with client application

#15 2011-04-18 14:53:13

Re: Problem with client application

#16 2011-04-18 15:42:00

Re: Problem with client application

#17 2011-04-18 17:08:22

Re: Problem with client application

#18 2011-04-18 17:35:08

Re: Problem with client application

#19 2011-04-18 18:08:52

Re: Problem with client application

#20 2011-04-18 23:19:32

Re: Problem with client application

#21 2011-04-19 06:45:04

Re: Problem with client application

#22 2011-04-19 10:02:37

Re: Problem with client application

#23 2011-04-19 11:20:10

Re: Problem with client application

#24 2011-04-19 13:22:32

Re: Problem with client application

#25 2011-04-19 13:37:27

Re: Problem with client application

#26 2011-04-19 13:43:54

Re: Problem with client application

#27 2011-04-20 17:18:17

Re: Problem with client application

#28 2011-04-20 18:23:41

Re: Problem with client application

Board footer