mORMot Open Source

AOG

Member

Registered: 2014-02-24

Posts: 490

MongoDB : picture gallery

Hello All,

I am trying to setup a simple picture gallery with mORMot and MongoDB.
Thumbnails have to be stored inside MongoDB.
Original picture is accessed by 'Path' if needed.

  TPhoto = packed record
    Picture: TSQLRawBlob;
    ModDateTime : TDateTime;
    Path : RawUTF8;
  end;
  TPhotos = array of TPhoto;

  TSQLUSER = class(TSQLRecord)
  private
    fName: RawUTF8;
    fDate: TDateTime;
    fPhotos : TPhotos;
  published
    property Name: RawUTF8 read fName write fName stored AS_UNIQUE;
    property Date: TDateTime read fDate write fDate;
    property Photos: TPhotos index 1 read fPhotos write fPhotos;
  end;

Using the above works ok, but I cannot get any data into 'Picture'.

var
  R: TSQLUSER;
  i:integer;
  pic: RawByteString;
  Photo:TPhoto;
begin
  R := TSQLUSER.Create;

  R.Name := 'TestName';

  // pic is loaded with thumbnail image data
  Photo.ModDateTime:=Now;
  Photo.Picture:=pic;
  R.DynArray('Photos').Add(Photo);

  i:=Client.Add(R,True);

  //Client.UpdateBlobFields(R); // does not work
  //Client.UpdateBlob(TSQLUSER,i,'Photos',pic); // does not work
  //Client.UpdateBlob(TSQLUSER,i,'Picture',pic); // does not work
end;

Does anybody know how to update the blobdata of 'Picture' with a thumbnail ?.

Thanks, Alfred.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: MongoDB : picture gallery

What is the generated JSON content?

Could you try with

Picture: RawByteString;

as field definition?

I suppose that since TSQLRawBlob is defined in mORMot.pas, it is available for TSQLRecord published fields only, not for record / dynamic array serialization.
The expected type for BLOB is RawByteString within a record or a dynamic array.

I've just enhanced MongoDB and record JSON serialization blob support.
TSQLRawBlob should be recognized as BLOB even for record / dynamic array serialization now.
See http://synopse.info/fossil/info/8f8c9a3c42

AOG

Member

Registered: 2014-02-24

Posts: 490

Re: MongoDB : picture gallery

The database itself (the fields) contained an array (name: 'Photos'), containing, among others, a string type with only 4 characters with name 'Picture'.
So not a binary datatype, but a string.

(I am using RoboMongo to view the database contents; I did not view the JSON contents).

However, I solved the problem by using:

Photo.Picture:=TSQLRawBlobToBlob(pic);

No UpdateBlob whatsoever is needed, while this 'Picture'-field is not saved or treated as a blob. It is a string.

I think this is the same method (or produces in the same result) as your suggestion !

Retrieving by using:

Photo.Picture:=BlobToTSQLRawBlob(pic);

For now, it works. And I am happy with it !

A single question remains:
Is it true that only simple types are allowed (or processed correctly) inside a dynamic array inside a SQLrecord ?

Edit:
I saw that a TDateTime inside a dynamic array is also converted into a string.
However, MongoDB allows for perfect sharding: every native datatype is allowed for nesting (arrays).
Could this be a future feature of the mORMot/MongoDB combination ?

Edit2:
I will try your update concerning Blobs !!
(sorry for missing your comments on this update)

Thanks.

ps: many thanks for adding MongoDB ... it allows for very nice and flexible database storage in combination with your very nice framework !!

Last edited by AOG (2014-05-24 08:59:40)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: MongoDB : picture gallery

In fact, writing by hand TSQLRawBlobToBlob() and BlobToTSQLRawBlob() is the same as declaring the field as RawByteString.

Now, data sent to MongoDB from JSON will recognize strict 'YYYY-MM-DDThh:mm:ss' or 'YYYY-MM-DD' or 'Thh:mm:ss' patterns as betDateTime, as e.g. generated by TTextWriter.AddDateTime() or RecordSaveJSON().
See http://synopse.info/fossil/info/2d3aef367e

Which version of Delphi are you using?

AOG

Member

Registered: 2014-02-24

Posts: 490

Re: MongoDB : picture gallery

Wow.

After using your latest software (from today), I can report:

* dates inside a dynamic array are stored as dates !
* blobs inside a dynamic array are stored as binaries !

THANK YOU VERY MUCH !!
This makes my life very easy. And my software much more useful.

To be honest: it's all about sharding.
Present data is so hard to fit in a standard scheme. Speaking for myself, I am always in need for dynamic (array) storage. And this means sharding in most cases.
But sharding using a special formatting system makes long term support hard (even when using JSON in a text-field).

Sharding with MongoDB (more or less native by the use of arrays and nesting) is much more secure and reliable.
And the data itself is also usefull and understandable for non-mORMot-users.

So, thanks again.

I am using XE4.

And I am refactoring a flat-file database into something ORMish with mORMot.
And the mORMot-MongoDB is a perfect (dynamic) fit.

Alfred.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: MongoDB : picture gallery

Happy it works well and fit your needs!

Do not forget that you can use SQlite3 as internal back-end, with all data sharding features, without the need of having a MongoDB server installed.
And performance may be even higher than MongoDB.

WhileNotEof

Member

Registered: 2014-04-18

Posts: 3

Re: MongoDB : picture gallery

This means that it is possible to use SQLLite document management system?. I am not sure about  the database to use, and I prefer to use one with a minimal installation.

Last edited by WhileNotEof (2014-05-24 16:41:31)

AOG

Member

Registered: 2014-02-24

Posts: 490

Re: MongoDB : picture gallery

Yes. That's mORMot.

Just change something like

fMongoClient := TMongoClient.Create('SERVER_IP',27017);
fDB := fMongoClient.Database[DB_NAME];
fClient := TSQLRestClientDB.Create(fModel,nil,':memory:',TSQLRestServerDB);
StaticMongoDBRegister(TSQLORM,fClient.Server,fDB,'mORMot');

into

fClient := TSQLRestClientDB.Create(fModel,nil,DB_NAME,TSQLRestServerDB);

to go from remote MongoDB to local SQLite. All other coding remains the same !! A very very nice feature of mORMot.

See Sample 15 (External DB Performance) and Sample 24 (MongoDB)

Good luck.

Last edited by AOG (2014-05-25 08:10:35)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: MongoDB : picture gallery

You are right, SQLite3 will be much easier to deploy.
No installation at all required: just launch the server exe, and that's all.

Then you will be able to switch later to MongoDB, if needed, e.g. if your data becomes really huge (more than 10 GB) - but in this case, MongoDB will need a lot of RAM, and will consume more disk space than SQlite3.
And I guess SQLite3 (with the proper indexes) could be faster, especially at reading - see http://blog.synopse.info/post/2014/05/0 … -benchmark
But if you use MongoDB directly, you can have indexes on sub-properties (i.e. within the variants or dynamic arrays), whereas it is not possible with SQlite3, which allows only to index main properties.