#1 2020-12-24 19:16:41

leus
Member
Registered: 2012-09-05
Posts: 79

Blobs and BatchUpdater

Hello,

Is it possible to transfer BLOB fields in a Batch Update operation?

Offline

#2 2020-12-25 05:57:47

edwinsn
Member
Registered: 2010-07-02
Posts: 1,215

Re: Blobs and BatchUpdater

Yes, use ALL_FIELDS or your custom field bits for the `Add` and `Update` methods


Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.

Offline

#3 2020-12-25 10:42:52

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,238
Website

Re: Blobs and BatchUpdater

You can specify the fields.

But note that the Blob content will be serialized as JSON during transmission so it is only good for small Blob size .

Offline

#4 2020-12-26 20:14:39

leus
Member
Registered: 2012-09-05
Posts: 79

Re: Blobs and BatchUpdater

ab wrote:

You can specify the fields.

But note that the Blob content will be serialized as JSON during transmission so it is only good for small Blob size .

This is interesting.

I need to do this operation once a day, and we are talking between 1,000 and 10,000 records with a blob size that may vary between 2kb up to 100kb (I don't have the number yet). Assuming worst case scenario, we are talking about 1GB in raw binary before any JSON transformation. Is it better to transfer one-by-one in this case?

Offline

#5 2020-12-27 04:58:39

edwinsn
Member
Registered: 2010-07-02
Posts: 1,215

Re: Blobs and BatchUpdater

leus wrote:
ab wrote:

You can specify the fields.

But note that the Blob content will be serialized as JSON during transmission so it is only good for small Blob size .

This is interesting.

I need to do this operation once a day, and we are talking between 1,000 and 10,000 records with a blob size that may vary between 2kb up to 100kb (I don't have the number yet). Assuming worst case scenario, we are talking about 1GB in raw binary before any JSON transformation. Is it better to transfer one-by-one in this case?

Sending 1GB at a time is obviously not a good idea. You can check `TSQLRestBatch.SizeBytes` and when it reaches a certain number, for example, 5MB, then you send it, and start another batch for the remaining data.


Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.

Offline

#6 2020-12-27 14:18:27

leus
Member
Registered: 2012-09-05
Posts: 79

Re: Blobs and BatchUpdater

edwinsn wrote:

Sending 1GB at a time is obviously not a good idea. You can check `TSQLRestBatch.SizeBytes` and when it reaches a certain number, for example, 5MB, then you send it, and start another batch for the remaining data.

Thank you, I'll take a look.

Last edited by leus (2020-12-27 14:18:48)

Offline

#7 2020-12-27 14:54:27

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,238
Website

Re: Blobs and BatchUpdater

I would use a binary layout with a method-based service for such uploads.

Offline

Board footer

Powered by FluxBB