mORMot Open Source

DigDiver

Member

Registered: 2013-04-29

Posts: 137

MongoDB .GetJSONValues(%): missing column - count=% expected:%

There is a big problem with using MongoDB when adding a new property into the model.

For example, we have a model like this:

 Type
  TDownloads = class(TSQLRecord)
    private
     FUserID         : Int64;
     FManual         : Boolean;
     FFileName       : RawUTF8;
     FSignedURL      : RawUTF8;
     FCreated        : TDateTime;
     FExpirationDate : TDateTime;
     FStatus         : RawUTF8;
    published
     property UserID         : Int64      read FUserID         write FUserID;
     property Manual         : Boolean    read FManual         write FManual;
     property FileName       : RawUTF8    read FFileName       write FFileName;
     property SignedURL      : RawUTF8    read FSignedURL      write FSignedURL;
     property Created        : TDateTime  read FCreated        write FCreated;
     property ExpirationDate : TDateTime  read FExpirationDate write FExpirationDate;
     property Status         : RawUTF8    read FStatus         write FStatus;
  end;

We add some records to the DB. In the some situations, I need to expand the model (add a new property). In my example -   property Success : Boolean  read FSuccess    write FSuccess;

 Type
  TDownloads = class(TSQLRecord)
    private
     FUserID         : Int64;
     FManual         : Boolean;
     FFileName       : RawUTF8;
     FSignedURL      : RawUTF8;
     FCreated        : TDateTime;
     FExpirationDate : TDateTime;
     FStatus         : RawUTF8;
     FSuccess        : Boolean;
    published
     property UserID         : Int64      read FUserID         write FUserID;
     property Manual         : Boolean    read FManual         write FManual;
     property FileName       : RawUTF8    read FFileName       write FFileName;
     property SignedURL      : RawUTF8    read FSignedURL      write FSignedURL;
     property Created        : TDateTime  read FCreated        write FCreated;
     property ExpirationDate : TDateTime  read FExpirationDate write FExpirationDate;
     property Status         : RawUTF8    read FStatus         write FStatus;
     property Success        : Boolean    read FSuccess        write FSuccess;
  end;

And after this modification, I cannot get records from DB at all.

In the function TSQLRestStorageMongoDB.GetJSONValues the exception  missing column - count=% expected:% is raised:

      if col<>colCount then
        raise EORMMongoDBException.CreateUTF8(
          '%.GetJSONValues(%): missing column - count=% expected:%',
          [self,StoredClass,col,colCount]);

When I use SQLite instead of MongoDB, there is no such a problem.

igors233

Member

Registered: 2012-09-10

Posts: 241

Re: MongoDB .GetJSONValues(%): missing column - count=% expected:%

Just and idea, have you called CreateMissingTables for your ORM server?

DigDiver

Member

Registered: 2013-04-29

Posts: 137

Re: MongoDB .GetJSONValues(%): missing column - count=% expected:%

Just and idea, have you called CreateMissingTables for your ORM server?

Yes, of course

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,666

Website

Re: MongoDB .GetJSONValues(%): missing column - count=% expected:%

Yes, this implementation is a bit paranoid... and doesn't work when you add a published field!

This is due to how MongoDB is a schema-less storage.
We don't change existing data when a new field is added.

I've fixed TSQLRestStorageMongoDB.GetJSONValues() to write null for any missing field, instead of raising an exception.
See http://synopse.info/fossil/info/21a06997d6

Thanks for the report.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,666

Website

Re: MongoDB .GetJSONValues(%): missing column - count=% expected:%

See also http://synopse.info/fossil/info/2eb07d7436
This time, it fixes the method when a field is removed from the TSQLRecord type definition.

DigDiver

Member

Registered: 2013-04-29

Posts: 137

Re: MongoDB .GetJSONValues(%): missing column - count=% expected:%

Thanks! Works fine.