mORMot Open Source

island

Member

Registered: 2017-02-20

Posts: 10

Table with more than 64 fields not filled over HTTPClient

Server

  // set logging abilities
  SQLite3Log.Family.Level := LOG_VERBOSE;
  SQLite3Log.Family.EchoToConsole := LOG_VERBOSE;
  SQLite3Log.Family.PerThreadLog := ptIdentifiedInOnFile;
  FDPhysMSSQLDriverLink := TFDPhysMSSQLDriverLink.Create(nil);

  // define the log level
  with TSQLLog.Family do begin
    Level := LOG_VERBOSE;
    EchoToConsole := LOG_VERBOSE; // log all events to the console
    PerThreadLog := ptIdentifiedInOnFile;
  end;
  // manual switch to console mode
  AllocConsole;
  TextColor(ccLightGray); // needed to notify previous AllocConsole

  aProps := TOleDBMSSql2012ConnectionProperties.Create('server-ip\SQLEXPRESS2008','TEST','TESTADMIN','123');
  try
    // get the shared data model
    Model := CreateModel;
    VirtualTableExternalRegisterAll(Model,aProps);
    try
      // create the main mORMot server
       aRestServer := TSQLRestServerDB.Create(Model,true); // authentication=true
      try
       aRestServer.AuthenticationRegister(TSQLRestServerAuthenticationNone);
       // create tables or fields if missing
       aRestServer.CreateMissingTables;
       InsertTab70; //   aRestServer.Add(test70,true); <== OK
       // serve aRestServer data over HTTP
       aHttpServer := TSQLHttpServer.Create(SERVER_PORT,[aRestServer],'+',useHttpApiRegisteringURI);
       try
         aHttpServer.AccessControlAllowOrigin := '*'; // allow cross-site AJAX queries
         writeln('Background server is running.'#10);
         write('Press [Enter] to close the server.');
         readln;
       finally
         aHttpServer.Free;
       end;
     finally
       aRestServer.Free;
     end;
    finally
     Model.Free;
    end;
  finally
    aProps.Free;
    FDPhysMSSQLDriverLink.Free;
  end;

Client:

  Client := TSQLHttpClientWinHTTP.Create(dbname,port,Model);


 a70 : TSQLTest70;


begin
    a70 := TSQLAnw.Create(Client,'Name=?',['test']);
    if a70.ID = 0 then
    begin
     a70.name := 'test';
     result := Client.Add(a70,true); 
     // Result = id-Number, it seems everythink OK, but it isn't.
     // every fields are null
    .
    .
    .

The same code with  a table with 64 fields works.
In Delphi Berlin 10.1 i set the definition MAX_SQL_FIELDS_128

I can't find my mistake.

Thank's for help

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,669

Website

Re: Table with more than 64 fields not filled over HTTPClient

Please provide more information: what is the generated JSON on client side, the SQL INSERT on the server side, etc...

And, please don't put code or big logs in the forum thread itself, as stated by the forum rules - see https://synopse.info/forum/misc.php?action=rules

island

Member

Registered: 2017-02-20

Posts: 10

Re: Table with more than 64 fields not filled over HTTPClient

I have made another test.
Table with 70 fields. Fieldnames f1 ... f70 = everyting OK.
Log: "insert into dbo.A70 (ID,.... all field and values
I change the name to f1longfieldname .. f70longfieldname => that the problem
Log:
DB "insert into dbo.A70 (ID) values(?)
SQL insert into dbo.A70 (ID) values (2)

No more fields and values

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,669

Website

Re: Table with more than 64 fields not filled over HTTPClient

I'm sorry I don't understand what you wrote...

What is the transmitted JSON?
What is the difference between f1..f70 and f1longfieldname...f70longfieldname ?

island

Member

Registered: 2017-02-20

Posts: 10

Re: Table with more than 64 fields not filled over HTTPClient

In mORMot.pas
function TSQLRecord.getJasonValues ...
i see the J.DataString with all fieldnames and values
result=rowid without error

island

Member

Registered: 2017-02-20

Posts: 10

Re: Table with more than 64 fields not filled over HTTPClient

The difference is short and long fieldname.
There must be a problem memoryproblem

TSQLA70 = class(TSQLRecord)
 private
  ff1 : RawUTF8;
  .
  .
  .
  f70 : RawUTF8;
  published
   property f1 : RawUTF8 index 100 read ff1 write ff1;
   .
   .
   .
   property f70 : RawUTF8 index 100 read ff70 write ff70;
 end;

Table with definition works

TSQLA70 = class(TSQLRecord)
 private
  ff1longfieldname : RawUTF8;
  .
  .
  .
  f70longfieldname : RawUTF8;
  published
   property f1longfieldname : RawUTF8 index 100 read ff1longfieldname write ff1longfieldname;
   .
   .
   .
   property f70longfieldname : RawUTF8 index 100 read ff70longfieldname write ff70longfieldname;
 end;

Table with definition doesn't work

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,669

Website

Re: Table with more than 64 fields not filled over HTTPClient

What is wrong in TJSONObjectDecoder.EncodeAsSQLPrepared ?

island

Member

Registered: 2017-02-20

Posts: 10

Re: Table with more than 64 fields not filled over HTTPClient

Fieldcount is 1 but it have to be 70

island

Member

Registered: 2017-02-20

Posts: 10

Re: Table with more than 64 fields not filled over HTTPClient

The problem is between client and server communication, because on serverside i can insert
data into the table without problems.
Tabledefinition with a lot of fields with long fieldnames may be the real couse and not fieldcount > 64
Perhaps only on my system. But i can't find my mistake.

Thank's for your help.

island

Member

Registered: 2017-02-20

Posts: 10

Re: Table with more than 64 fields not filled over HTTPClient

The Problem:
Datasize > 1024 will be compress on clientside (without any messages)
On server side nothing happend. The very bad sideeffect = server fill null data in all fields and no error.
That is not a good idea.
See Documentation 11.4.2
All this happen if you do not register the server like Sample Project04ServerRegister
I never have calculate with such problems. 2 days over.
The solution
Register the server like Sample Project04ServerRegister

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,669

Website

Re: Table with more than 64 fields not filled over HTTPClient

??? do you mean that the HTTP socket server don't handle data > 1KB ???
I doubt it is the case, since we fetch 4KB of JSON during the regression tests, AFAIR.

island

Member

Registered: 2017-02-20

Posts: 10

Re: Table with more than 64 fields not filled over HTTPClient

No. it handle > 1KB but it compressed the data on clientside in SynCrtSock, and the Server recieve this data without
encoding or better decompressed.

In  mORMotHttpServer Line 954 call.inBody := Ctxt.InConcent
there are the compressed data or decode data. Like êïÆ#5#7 and so on, not usefull.
The client add the compress mode into the POST message.

island

Member

Registered: 2017-02-20

Posts: 10

Re: Table with more than 64 fields not filled over HTTPClient

I tested again.
Server not     registered. The debugger show TSQLHttpServer: http.sys URI registration error #5 for http://+888/root (administratior rights neede, at least once to register the URI)
but it runs with the problem > 1 KB
Server registered.
The erro message do not appear.
No problems. Serverside Ctxt.InConcent = decompressed and readable data