mORMot Open Source

Bascy

Member

From: The Netherlands

Registered: 2012-06-22

Posts: 108

Writing a TCollection to a TEXT field is not retrieved correctly

I have created a TSQLRecord with a TInterfaceCollection as one of its published properties.
When this property is written to the underlying database (our own NexusDB implementation in this case) it results in a neat JSON string

[{"ValueDataObject":"X'6F626A65637420544C435353696D706C6556616C75650D0A20204173537472696E67203D20274469742069732065656E2074656B73746A65270D0A656E640D0A'","AttrDefDataObjct":"X'6F626A6563742054574244796E4174747269627574654465660D0A20204E616D65203D202756616C75654E616D6531270D0A20205742417474726962757465436C617373203D2027544C435353696D706C6556616C7565270D0A656E640D0A'"},{"ValueDataObject":"X'6F626A65637420544C435353696D706C6556616C75650D0A20204173466C6F6174203D2033342E3636353030303030303030303030303030300D0A656E640D0A'","AttrDefDataObjct":"X'6F626A6563742054574244796E4174747269627574654465660D0A20204E616D65203D202756616C75654E616D6532270D0A20205742417474726962757465436C617373203D2027544C435353696D706C6556616C7565270D0A656E640D0A'"}]

When i try to retrieve a record from this model, it does not work anymore :-(
The JSONToObject function in Mormot.pas is called from TSQLPropInfoRTTIObject.SetValue with the following value of From:

'ï¿°W3siVmFsdWVEYXRhT2JqZWN0IjoiWCc2RjYyNkE2NTYzNzQyMDU0NEM0MzUzNTM2OTZENzA2QzY1NTY2MTZDNzU2NTBEMEEyMDIwNDE3MzUzNzQ3MjY5NkU2NzIwM0QyMDI3NDQ2OTc0MjA2OTczMjA2NTY1NkUyMDc0NjU2QjczNzQ2QTY1MjcwRDBBNjU2RTY0MEQwQSciLCJBdHRyRGVmRGF0YU9iamN0IjoiWCc2RjYyNkE2NTYzNzQyMDU0NTc0MjQ0Nzk2RTQxNzQ3NDcyNjk2Mjc1NzQ2NTQ0NjU2NjBEMEEyMDIwNEU2MTZENjUyMDNEMjAyNzU2NjE2Qzc1NjU0RTYxNkQ2NTMxMjcwRDBBMjAyMDU3NDI0MTc0NzQ3MjY5NjI3NTc0NjU0MzZDNjE3MzczMjAzRDIwMjc1NDRDNDM1MzUzNjk2RDcwNkM2NTU2NjE2Qzc1NjUyNzBEMEE2NTZFNjQwRDBBJyJ9LHsiVmFsdWVEYXRhT2JqZWN0IjoiWCc2RjYyNkE2NTYzNzQyMDU0NEM0MzUzNTM2OTZENzA2QzY1NTY2MTZDNzU2NTBEMEEyMDIwNDE3MzQ2NkM2RjYxNzQyMDNEMjAzMzM0MkUzNjM2MzUzMDMwMzAzMDMwMzAzMDMwMzAzMDMwMzAzMDMwMzAwRDBBNjU2RTY0MEQwQSciLCJBdHRyRGVmRGF0YU9iamN0IjoiWCc2RjYyNkE2NTYzNzQyMDU0NTc0MjQ0Nzk2RTQxNzQ3NDcyNjk2Mjc1NzQ2NTQ0NjU2NjBEMEEyMDIwNEU2MTZENjUyMDNEMjAyNzU2NjE2Qzc1NjU0RTYxNkQ2NTMyMjcwRDBBMjAyMDU3NDI0MTc0NzQ3MjY5NjI3NTc0NjU0MzZDNjE3MzczMjAzRDIwMjc1NDRDNDM1MzUzNjk2RDcwNkM2NTU2NjE2Qzc1NjUyNzBEMEE2NTZFNjQwRDBBJyJ9XQ=='

As far as i can see this is the result after calling WRBase64() in the Writer, and this is not reversed when reading.

There was another strange thing I noticed while debugging this:

When TSQLRecordProperties.Create() is called, fields with an SQLFieldType of sftObject are handled as a SimpleField, but they are actually stored as a TEXT field (Blob Memo in the case of our NexusDB implementation). Doesnt this create problems when reading back, as I had expected these properties of unknown size to be read only after calling RetrieveBlobFields()... ?

Greetings
Bas

Last edited by Bascy (2013-01-21 13:59:50)

Bascy

Member

From: The Netherlands

Registered: 2012-06-22

Posts: 108

Re: Writing a TCollection to a TEXT field is not retrieved correctly

Again .. we managed to create our own problems here hahaha

The above problem was caused by the TSQLDBNexusDBStatement.ColumnTypeNativeToDB() implementation. There the DB types ftMemo which is used for unindexed UTF8 fields was mapped to SynDB.ftBlob in stead to SynDB.ftUTF8 ...

Does raise the question why these Object serializations into TEXT fields are always read by default (unlike regular Blob fields).. a serialized collection can get  quite large ..

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,240

Website

Re: Writing a TCollection to a TEXT field is not retrieved correctly

Does raise the question why these Object serializations into TEXT fields are always read by default (unlike regular Blob fields).. a serialized collection can get  quite large ..

Lazy loading is just for plain BLOBs.
Collections or dynamic arrays are retrieved with the main data.
This is what we call "share-nothing" or "data sharding" in the documentation.

If you want lazy loading, use a one-to-many relationship.