You are not logged in.
Pages: 1
One of my servers, proxied by Cloudflare, doesn't return Content-Length when queried (I don't know why, that's a whole other ball of fluff). When I attempt downloading a file from that server with MGet the download fails but there is no error indication, just a zero-length file created locally.
Debug output:
D:\mORMot\src\tools\mget\Win32\Debug\Output>mget https://www.simplysup.com/download/dl/trjsetup.exe /debug
20240103 01420959 ! info SetThreadName 3ae4=Main
20240103 01420959 ! + mormot.tools.mget.TMGetProcess(02f52a40).Execute https://www.simplysup.com/download/dl/trjsetup.exe
20240103 01421001 ! trace mormot.net.client.THttpClientSocket(02edffe0) open(www.simplysup.com:443) sock=0268 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 TLSv1.2
20240103 01421001 ! trace mormot.net.client.THttpClientSocket(02edffe0) WGet download/dl/trjsetup.exe: start downloading D:\mORMot\src\tools\mget\Win32\Debug\Output\trjsetup.exe.part
20240103 01421001 ! trace mormot.net.client.THttpClientSocket(02edffe0) RequestInternal GET www.simplysup.com:443//download/dl/trjsetup.exe flags= retry=0
20240103 01421003 ! trace mormot.net.client.THttpClientSocket(02edffe0) GetHeader flags=ptk len=-1 application/x-msdownload
20240103 01421003 ! trace mormot.net.client.THttpClientSocket(02edffe0) RequestInternal status=200 keepalive=30000 flags=ptk in 43.08ms
20240103 01421003 ! trace mormot.tools.mget.TMGetProcess(02f52a40) Execute: WGet=D:\mORMot\src\tools\mget\Win32\Debug\Output\trjsetup.exe
20240103 01421003 ! - 00.105.011
WGet (1.21.1) downloads the file OK.
Offline
The server uses
Transfer-Encoding: chunked
which should be supported.
The regular Get() calls works as expected, but I guess wget requires a length to work as expected...
I will investigate further.
Offline
Please try
https://github.com/synopse/mORMot2/commit/0a01bc35
There are Internet issues on my location currently, so I am not able to fully validate the fix.
Your feedback is welcome.
Offline
Thanks Arnaud, I can confirm the fix works:
D:\mORMot\src\tools\mget\Win32\Debug\Output>mget https://www.simplysup.com/download/dl/trjsetup.exe /debug
20240103 17504852 ! info SetThreadName 4fc0=Main
20240103 17504852 ! + mormot.tools.mget.TMGetProcess(03122b30).Execute https://www.simplysup.com/download/dl/trjsetup.exe
20240103 17504858 ! trace mormot.net.client.THttpClientSocket(030affe0) open(www.simplysup.com:443) sock=0268 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 TLSv1.2
20240103 17504858 ! trace mormot.net.client.THttpClientSocket(030affe0) WGet download/dl/trjsetup.exe: start downloading D:\mORMot\src\tools\mget\Win32\Debug\Output\trjsetup.exe.part
20240103 17504858 ! trace mormot.net.client.THttpClientSocket(030affe0) RequestInternal GET www.simplysup.com:443//download/dl/trjsetup.exe flags= retry=0
20240103 17504861 ! trace mormot.net.client.THttpClientSocket(030affe0) GetHeader flags=ptk len=-1 application/x-msdownload
20240103 17504861 ! trace mormot.core.buffers.TStreamRedirect(0303de90) trjsetup.exe 25.7KB read 547.9KB/s ...
20240103 17504961 ! trace mormot.core.buffers.TStreamRedirect(0303de90) trjsetup.exe 7.3MB read 7MB/s ...
20240103 17505061 ! trace mormot.core.buffers.TStreamRedirect(0303de90) trjsetup.exe 13.7MB read 6.7MB/s ...
20240103 17505161 ! trace mormot.core.buffers.TStreamRedirect(0303de90) trjsetup.exe 20.1MB read 6.6MB/s ...
20240103 17505201 ! trace mormot.net.client.THttpClientSocket(030affe0) GetBody len=21425456
20240103 17505201 ! trace mormot.net.client.THttpClientSocket(030affe0) RequestInternal status=200 keepalive=30000 flags=ptk in 3.09s
20240103 17505201 ! trace mormot.core.buffers.TStreamRedirect(0303de90) trjsetup.exe 20.4MB done in 3.07s ( 6.6MB/s )
20240103 17505201 ! trace mormot.tools.mget.TMGetProcess(03122b30) Execute: WGet=D:\mORMot\src\tools\mget\Win32\Debug\Output\trjsetup.exe
20240103 17505201 ! - 03.176.051
Offline
I'm afraid to be wrong, but maybe it makes sense to add the ability to download files over SMB to MGet.
Or move this feature into a separate utility, something like
smbget - wget-like utility for download files over SMB
https://manpages.ubuntu.com/manpages/tr … get.1.html
Last edited by vs (2024-02-07 18:50:07)
Offline
A pure SMB client is out-of-scope...
But https://searchcode.com/codesearch/view/64390136/ may help.
A network file system access (i.e. using plain FileOpen/FileRead) instead of sockets + HTTP, with our P2P automated fallback, is (easily) feasible.
My guess is that we could a "protocol-plugin-system" to work with schemes other than http/https in mget URIs.
We could had e.g. file://.... and smb:// protocols.
Offline
Great as always, ab!
Thank you!
Generally speaking also plus FTP, FTPS, and SFTP
ftp://[user[:password]@]server[:port]/path/to/remote/resource.mpeg
sftp://[user[:password]@]server[:port]/path/to/remote/resource.mpeg
Offline
Preliminary 'file://' support:
https://github.com/synopse/mORMot2/commit/489436af
Note that on Windows, it would use proper file URIs so could redirect to a remote folder:
https://learn.microsoft.com/en-us/archi … in-windows
On POSIX, you need to have mounted the file to a given position.
Offline
Incredible. Thank you!
I'll try to do some tests.
Offline
Pages: 1