[mongodb] Sample 28 Conversion

Sabbiolina · 2014-09-19 21:55:24

Hi AB, I'm trying to modify the sample 28 to use MongoDB as the database.

I do not know how to convert this function: VirtualTableExternalRegisterAll (Model, aProps);

Or is there another method?

Thanks

ab · 2014-09-20 07:15:58

Take a look at the SAD 1.18 PDF corresponding paragraphs.

Especially the "MongoDB + ORM = ODM" chapter.
You would read:

Documentation wrote:

On the server side (there won't be any difference for the client), you define a TMongoDBClient, and assign it to a given TSQLRecord class, via a call to StaticMongoDBRegister():
MongoClient := TMongoClient.Create('localhost',27017);
DB := MongoClient.Database['dbname'];
Model := TSQLModel.Create([TSQLORM]);
Client := TSQLRestClientDB.Create(Model,nil,':memory:',TSQLRestServerDB);
if StaticMongoDBRegister(TSQLORM,fClient.Server,fDB,'collectionname')=nil then
raise Exception.Create('Error');
And... that's all!

Sabbiolina · 2014-09-20 07:44:51

Which section are you referring to?

I wrote this:

  MongoClient := TMongoClient.Create('localhost',27017);
  DB := MongoClient.Database['testODM'];
  Model := TSQLModel.Create([TSQLORM]);
  Client := TSQLRestClientDB.Create(Model,nil,':memory:',TSQLRestServerDB);
  if StaticMongoDBRegister(TSQLORM,Client.Server,DB,'mytestcollection')=nil then
  begin
   writeln('Error !'#10);
   exit;
  end;

  try
   // create the main mORMot server
   aRestServer := TSQLRestServerDB.Create(Model,':memory:',false); // authentication=false
   try
    aRestServer.CreateMissingTables; // create tables or fields if missing
    // serve aRestServer data over HTTP
    aHttpServer := TSQLHttpServer.Create(SERVER_PORT,[aRestServer],'+',useHttpApiRegisteringURI);
    try
     aHttpServer.AccessControlAllowOrigin := '*'; // allow cross-site AJAX queries
     writeln('Background server is running.'#10);
     write('Press [Enter] to close the server.');
     readln;
    finally aHttpServer.Free; end;
   finally aRestServer.Free; end;
  finally Model.Free; end;

The Db and the collection are written in mongodb regularly, but "CreateMissingTables" does nothing, since there is a link between the aRestServer and MongoDB.

In fact, I was wondering how to do it.
VirtualTableExternalRegisterAll has TSQLDBConnectionProperties as a parameter, but that, looking SAD document is not related to mongodb.

So how to link?

ab · 2014-09-20 07:46:49

TSQLDBConnectionProperties is for SQL databases, as its name states.
It is pointless to use with MongoDB.

Similarly, "CreateMissingTables" has nothing to do with MongoDB tables.
MongoDB is a "schema-less" database.

Sabbiolina · 2014-09-20 08:33:18

I've lost something ...

Server:

/// minimal REST server for a list of Persons stored on MONGODB
program RESTserver;

{$APPTYPE CONSOLE}

uses
  SynCommons,          // framework core
  mORMot,              // RESTful server & ORM
  mORMotSQLite3,       // SQLite3 engine as ORM core
  SynSQLite3Static,    // staticaly linked SQLite3 engine
  mORMotDB,            // ORM using external DB
  mORMotHttpServer,    // HTTP server for RESTful server
  SynDB,               // external DB core
  SynMongoDB,mORMotMongoDB;

const
 SERVER_PORT='888';

type
 TSQLORM = class(TSQLRecord)
  private
    fAge: integer;
    fName: RawUTF8;
    fDate: TDateTime;
    fValue: variant;
    fInts: TIntegerDynArray;
    fCreateTime: TCreateTime;
    fData: TSQLRawBlob;
  published
    property Name: RawUTF8 read fName write fName stored AS_UNIQUE;
    property Age: integer read fAge write fAge;
    property Date: TDateTime read fDate write fDate;
    property Value: variant read fValue write fValue;
    property Ints: TIntegerDynArray index 1 read fInts write fInts;
    property Data: TSQLRawBlob read fData write fData;
    property CreateTime: TCreateTime read fCreateTime write fCreateTime;
  end;

var
  aRestServer: TSQLRestServerDB;
  aHttpServer: TSQLHttpServer;
  MongoClient : TMongoClient;
  Client: TSQLRestClientDB;
  DB: TMongoDatabase;
  Model :TSQLModel;
begin
  // define the log level
  with TSQLLog.Family do begin
    Level := LOG_VERBOSE;
    EchoToConsole := LOG_VERBOSE; // log all events to the console
  end;

  MongoClient := TMongoClient.Create('localhost',27017);
  DB := MongoClient.Database['testODM'];
  Model := TSQLModel.Create([TSQLORM]);
  Client := TSQLRestClientDB.Create(Model,nil,':memory:',TSQLRestServerDB);
  if StaticMongoDBRegister(TSQLORM,Client.Server,DB,'mytestcollection')=nil then
  begin
   writeln('Error !'#10);
   exit;
  end;
  try
   // create the main mORMot server
   aRestServer := TSQLRestServerDB.Create(Model,':memory:',false); // authentication=false
   try
    aRestServer.CreateMissingTables; // create tables or fields if missing
    // serve aRestServer data over HTTP
    aHttpServer := TSQLHttpServer.Create(SERVER_PORT,[aRestServer],'+',useHttpApiRegisteringURI);
    try
     aHttpServer.AccessControlAllowOrigin := '*'; // allow cross-site AJAX queries
     writeln('Background server is running.'#10);
     write('Press [Enter] to close the server.');
     readln;
    finally aHttpServer.Free; end;
   finally aRestServer.Free; end;
  finally Model.Free; end;

end.

Client:

program RESTclient;

{$APPTYPE CONSOLE}

uses
  SynCommons,          // framework core
  mORMot,              // RESTful server & ORM
  mORMotHttpClient;    // HTTP client to a mORMot RESTful server

const
  SERVER_ROOT = 'root';
  SERVER_PORT = '888';

type
 TSQLORM = class(TSQLRecord)
  private
    fAge: integer;
    fName: RawUTF8;
    fDate: TDateTime;
    fValue: variant;
    fInts: TIntegerDynArray;
    fCreateTime: TCreateTime;
    fData: TSQLRawBlob;
  published
    property Name: RawUTF8 read fName write fName stored AS_UNIQUE;
    property Age: integer read fAge write fAge;
    property Date: TDateTime read fDate write fDate;
    property Value: variant read fValue write fValue;
    property Ints: TIntegerDynArray index 1 read fInts write fInts;
    property Data: TSQLRawBlob read fData write fData;
    property CreateTime: TCreateTime read fCreateTime write fCreateTime;
  end;

function DataModel: TSQLModel;
begin
  result := TSQLModel.Create([TSQLORM],SERVER_ROOT);
  TSQLORM.AddFilterOrValidate('Name',TSynValidateText.Create); // ensure exists
end;


var aModel: TSQLModel;
    aClient: TSQLHttpClient;
    aSQLORM: TSQLORM;
    aID: integer;
begin
  aModel := DataModel;
  try
    aClient := TSQLHttpClientWinHTTP.Create('localhost',SERVER_PORT,aModel);
    try
      writeln('Add a new TSQLORM');
      aSQLORM := TSQLORM.Create;
      try
        Randomize;
        aSQLORM.Name := 'Name'+Int32ToUtf8(Random(10000));
        aID := aClient.Add(aSQLORM,true);
      finally aSQLORM.Free; end;
      writeln('Added TSQLORM.ID=',aID);

      aSQLORM := TSQLORM.Create(aClient,aID);
      try
        writeln('Name read for ID=',aSQLORM.ID,' from DB = "',aSQLORM.Name,'"');
      finally aSQLORM.Free; end;
    finally aClient.Free; end;
    write(#10'Press [Enter] to quit');
    readln;
  finally aModel.Free; end;
end.

client response:
Add a new TSQLORM
Added TSQLORM.ID=1
Name read for ID=1 from DB = "Name4488"

Press [Enter] to quit

It works, but nothing in mongodb.
In memory table only I think.

this is server log:
20140920 10301428 + TSQLRestServerDB(006D89F0).005892C0
20140920 10301428 call TSQLRestServerDB(006D89F0) TimeStamp
20140920 10301428 srvr TSQLRestServerDB(006D89F0) GET TimeStamp -> 200
20140920 10301428 - 00.002.355
20140920 10301428 + TSQLRestServerDB(006D89F0).005892C0
20140920 10301428 SQL TSQLRestServerDB(006D89F0) INSERT INTO ORM (Name,Age,Date,Value,Ints,CreateTime) VALUES (:('Name4488'):,:(0):,:(''):,:(null):,:('?BAAA'):,:(135193331598):); prepared with 6 params
20140920 10301428 DB TSQLDatabase(007CC338) LastInsertRowID=1
20140920 10301428 srvr TSQLRestServerDB(006D89F0) POST ORM -> 201
20140920 10301428 - 00.002.421
20140920 10301428 + TSQLRestServerDB(006D89F0).005892C0
20140920 10301428 cache TSQLDatabase(007CC338) not in cache
20140920 10301429 SQL TSQLRestServerDB(006D89F0) SELECT ID,Name,Age,Date,Value,Ints,CreateTime FROM ORM WHERE RowID=:(1):; prepared with 1 param
20140920 10301429 res TSQLDatabase(007CC338) [{"ID":1,"Name":"Name4488","Age":0,"Date":null,"Value":null,"Ints":"?BAAA","CreateTime":135193331598}]
20140920 10301429 srvr TSQLRestServerDB(006D89F0) GET ORM -> 200
20140920 10301429 - 00.002.815

ab · 2014-09-20 08:56:39

StaticMongodbRegister should be run BEFORE the server is started.

Sabbiolina · 2014-09-20 09:54:38

I'm sorry, I do not understand.

StaticMongoDBRegister call before the start of the server.

What server are you referring to?

ab · 2014-09-20 13:40:25

Your code is just broken.

You have two servers.

And MongoDB is just registered for

  Client := TSQLRestClientDB.Create(Model,nil,':memory:',TSQLRestServerDB);
  if StaticMongoDBRegister(TSQLORM,Client.Server,DB,'mytestcollection')=nil then

So the main aRestServer is not linked at all to MongoDB.

Correct code may be:

  MongoClient := TMongoClient.Create('localhost',27017);
  DB := MongoClient.Database['testODM'];
  Model := TSQLModel.Create([TSQLORM]);
  try
   // create the main mORMot server
   aRestServer := TSQLRestServerDB.Create(Model,':memory:',false); // authentication=false
   try
    // register the table to use MongoDB
    if StaticMongoDBRegister(TSQLORM,aRestServer,DB,'mytestcollection')=nil then
    begin
    writeln('Error !'#10);
     exit;
    end;
    aRestServer.CreateMissingTables; // create tables or fields if missing
    // serve aRestServer data over HTTP
    aHttpServer := TSQLHttpServer.Create(SERVER_PORT,[aRestServer],'+',useHttpApiRegisteringURI);
    try
     aHttpServer.AccessControlAllowOrigin := '*'; // allow cross-site AJAX queries
     writeln('Background server is running.'#10);
     write('Press [Enter] to close the server.');
     readln;
    finally aHttpServer.Free; end;
   finally aRestServer.Free; end;
  finally Model.Free; end;
  MongoClient.Free;

Sabbiolina · 2014-09-20 13:55:18

I had already tried to do that solution.

but at the first client call I get an out of memory error.

and I see the processor consumption rise rapidly.

it seems that there is an infinite recursion

nothing is saved in mongodb and it takes a minute to finish the server.

now I'm debugging: procedure TBSONWriter.BSONWriteFromJSON

Sabbiolina · 2014-09-20 14:11:44

I found the problem:

JSON='[]';

function TBSONWriter.BSONWriteDocFromJSON(JSON: PUTF8Char; aEndOfObject: PUTF8Char;
  out Kind: TBSONElementType; DoNotTryExtendedMongoSyntax: boolean): PUTF8Char;
var Start, ndx: cardinal;
    EndOfObject: AnsiChar;
    Name: RawUTF8;
begin
  result := nil;
  if JSON=nil then
    exit;
  if JSON^ in [#1..' '] then repeat inc(JSON) until not(JSON^ in [#1..' ']);
  case JSON^ of
  '[': begin
    Kind := betArray;
    Start := BSONDocumentBegin;
    repeat inc(JSON) until not(JSON^ in [#1..' ']);
    ndx := 0;
    repeat
      UInt32ToUtf8(ndx,Name);
      BSONWriteFromJSON(Name,JSON,@EndOfObject,DoNotTryExtendedMongoSyntax);
      if JSON=nil then
        exit; // invalid content
      inc(ndx);
    until EndOfObject=']'; <---- EndoOfObject is #0
  end;

 [...]

the value returned by BSONWriteFromJSON in EndOfObject is # 0

but with:

until (EndOfObject=']') or (EndOfObject=#0) ;

WORKS!!!
the record in mongodb is written.

Addendum:
{
"_id" : 1,
"Name" : "Name8558",
"Age" : 0,
"Date" : ISODate("1899-12-30T00:00:00.000Z"),
"Value" : null,
"Ints" : [
null <-- wrong ?
],
"CreateTime" : NumberLong(135193354702)
}

ab · 2014-09-21 08:34:28

See http://synopse.info/fossil/info/03cd22a220

Still the "Ints":[null] issue to find out.

mORMot Open Source

#1 2014-09-19 21:55:24

[mongodb] Sample 28 Conversion

#2 2014-09-20 07:15:58

Re: [mongodb] Sample 28 Conversion

#3 2014-09-20 07:44:51

Re: [mongodb] Sample 28 Conversion

#4 2014-09-20 07:46:49

Re: [mongodb] Sample 28 Conversion

#5 2014-09-20 08:33:18

Re: [mongodb] Sample 28 Conversion

#6 2014-09-20 08:56:39

Re: [mongodb] Sample 28 Conversion

#7 2014-09-20 09:54:38

Re: [mongodb] Sample 28 Conversion

#8 2014-09-20 13:40:25

Re: [mongodb] Sample 28 Conversion

#9 2014-09-20 13:55:18

Re: [mongodb] Sample 28 Conversion

#10 2014-09-20 14:11:44

Re: [mongodb] Sample 28 Conversion

#11 2014-09-21 08:34:28

Re: [mongodb] Sample 28 Conversion

Board footer