mORMot Open Source

dexter

Member

Registered: 2015-04-24

Posts: 53

Download big files from server

Hi,

I'm generating big files on server side and need to download them to client.

On server side:

Ctxt.ReturnFile(FileName, false, 'application/octet-stream');

On client side:

    if FClient.CallBackGet('GetFile', ['FileName', aFileName], FileContent) <> HTTP_SUCCESS then
      raise Exception.CreateFmt('Cannot get result file from server: %s', [FileContent]);

  FileFromString(FileContent, 'somefile.big', true);

It works ok, but my concern is the memory consumption when files are 1GB+ in size.
How can I save the downloaded content to a stream "on the fly" and not having the file's content all at once in the RAM (in FileContent variable)?

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Download big files from server

On mORMot 1, the usual way is to use a reverse proxy then let it return  static files.
There are nginx-related methods when you setup the HTTP server.

For mORMot 2, the new async HTTP Server is able to stream the file directly (when files are > 1MB).
But the best way on production may be still to define a nginx reverse proxy.

macfly

Member

From: Brasil

Registered: 2016-08-20

Posts: 374

Re: Download big files from server

Hi ab,

To use the new async features with TRestHttpServer just use useHttpAsync option?

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Download big files from server

Yes, this will trigger the new HTTP server.

You can set useHttpAsync for the async HTTP server or useBidirAsync  for the async WebSockets server.

macfly

Member

From: Brasil

Registered: 2016-08-20

Posts: 374

Re: Download big files from server

Thanks ab,

I was initially searching for an TRestHttpAsyncServer , but it's even simpler than that.

squirrel

Member

Registered: 2015-08-13

Posts: 155

Re: Download big files from server

Hi ab

Streaming the file directly from the server is great.  I wish this was possible with mORMot 1.  If I may get back to the second half of dexter's question - saving of the file by the client.  Is there a way for the client (ie TWinHTTP.Request) to stream the bytes received from the server directly to file/disk without caching it in memory first?

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Download big files from server

Not directly in the REST/SOA interface.

But the new WGet() methods are meant for such complex client process, including a lot of additional features like progress callbacks, hashing and even resuming!
For a huge content downloading, it makes sense to use a dedicated call with its own connection, perhaps in a background thread.