#1 2016-09-05 12:25:26

ecofast
Member
Registered: 2016-09-05
Posts: 2

About ZLib/SynLZ/Snappy/LZ4

Hello, I did some test using System.ZLib(XE10 Berlin)、SynLZ、Snappy and LZ4, like https://github.com/Hugie/lz4-delphi did.
And the result shows that LZ4 is ABSOLUTELY the WINNER.

It seems that LZ4 HC has a more quick decompress speed and better compress ratio, but I can not find  any edition of Delphi, Could anyone give some help?
Thanks!

Offline

#2 2016-09-05 17:19:50

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 9,336
Website

Re: About ZLib/SynLZ/Snappy/LZ4

Not so sure.
See http://synopse.info/forum/viewtopic.php?pid=3374#p3374

And forget ZDelphi Zlib unit. Try SynZip.
Or our SynZLibSSE.pas under FPC/Win64 only by now.

Offline

#3 2016-09-05 20:39:51

emk
Member
Registered: 2013-10-24
Posts: 45

Re: About ZLib/SynLZ/Snappy/LZ4

I would like to use mORMot Win32 server because many RTL/mORMot functions are asm optimised and thanks to mORMot good arhitecture memory need is low wink  If, I would compile for Win64 I'll loose many of them. What is the solution for compression to use  a mobile client (iOS/Android) in conjuction with mORMot method based service? Delphi zlib seems to slow (I dindn't test it), SynLZ it's not crossplatform, Cloudflare zlib is only for FPC/Win64).

1. Please make SynLZ crossplatform or write some Snappy wrapper(crossplatform) and define some "synaessnappy" header; snappy seems to be on the same level with SynLZ.
2. What compression to use for AJAX + Win32. Do you intend to enhance the speed of zlib for Win32?

Thank you.

Offline

#4 2016-09-06 04:24:27

ecofast
Member
Registered: 2016-09-05
Posts: 2

Re: About ZLib/SynLZ/Snappy/LZ4

ab wrote:

Not so sure.
See http://synopse.info/forum/viewtopic.php?pid=3374#p3374

And forget ZDelphi Zlib unit. Try SynZip.
Or our SynZLibSSE.pas under FPC/Win64 only by now.

Hi ab,
    Through the way https://github.com/atelierw/LZ4Delphi did, i modified https://github.com/Hugie/lz4-delphi, add function LZ4_compressHC(const ASource: Pointer; ADestination: Pointer; AInputSize: Integer): Integer; cdecl; external name '_LZ4_compressHC'; in lz4d.lz4 unit, and it works!
    Here's my testing result: http://www.cnblogs.com/ecofast/p/5844016.html
    I DO love LZ4 HC.

Offline

#5 2016-09-06 05:42:41

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 9,336
Website

Re: About ZLib/SynLZ/Snappy/LZ4

Yes, LZ4HC is comparable to UCL or LZMA: slow compression, fast decompression, good ratio.

For our client/server process, we do mainly compression on the server side, so for us, compression speed and compression ratio do matter.
The client decompression speed is not a problem, for server application scaling, in practice.
All those algorithms are much faster to decompress than to download the content!

This was the whole idea of SynLZ: make the best compression speed possible, with the best compression ratio possible, in pure pascal.
Of course, we target server process, with a highly number of concurrent requests...

Offline

#6 2016-09-06 15:40:31

emk
Member
Registered: 2013-10-24
Posts: 45

Re: About ZLib/SynLZ/Snappy/LZ4

Any plans to support SynLZ on FMX platform? Sometimes I will need to transfer some large blocks to iOS and Android devices.
Thank you.

Offline

#7 2017-08-10 12:08:03

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 9,336
Website

Re: About ZLib/SynLZ/Snappy/LZ4

SynLZ support for NextGen compiler is now available in the "CrossPlatform" sub-folder.
See https://github.com/synopse/mORMot/blob/ … mSynLZ.pas

Feeback is welcome, since we don't use Delphi for iOS and Android here!

Offline

#8 2017-08-10 13:07:43

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 9,336
Website

Re: About ZLib/SynLZ/Snappy/LZ4

I've also written a new x64 asm optimized version of SynLZ, and profiled the existing x86 asm to be even faster than previously.
For a 100MB text log file, SynLZ is faster than Snappy, and compress better (93% instead of 84%).
For other kind of files, Snappy is slightly faster, but SynLZ compresses better, and most of the time faster.

For Win32:

Win32 Processing DragonFly-devpcm.log = 98.7 MB for 1 times
 Snappy compress in 125.07ms, ratio=84%, 789.3 MB/s
 Snappy uncompress in 70.35ms, 1.3 GB/s
 SynLZ compress in 103.61ms, ratio=93%, 952.8 MB/s
 SynLZ uncompress in 68.71ms, 1.4 GB/s

For Win64:

Win64 Processing DragonFly-devpcm.log = 98.7 MB for 1 times
 Snappy compress in 107.13ms, ratio=84%, 921.5 MB/s
 Snappy uncompress in 61.06ms, 1.5 GB/s
 SynLZ compress in 97.25ms, ratio=93%, 1015.1 MB/s
 SynLZ uncompress in 61.27ms, 1.5 GB/s

Anyway, the main benefit of SynLZ is that it was designed in plain pascal, so it is clearly cross-platform and well integrated with Delphi/FPC (no external .o).

Offline

#9 2017-08-11 14:40:31

emk
Member
Registered: 2013-10-24
Posts: 45

Re: About ZLib/SynLZ/Snappy/LZ4

Perfect! Didn't try but thank you very much.

Offline

#10 2017-08-14 10:58:07

Johan
Member
Registered: 2014-11-04
Posts: 5

Re: About ZLib/SynLZ/Snappy/LZ4

I still see a few opportunities for improvement. You use plain x64 instructions for zero init. These can be optimized by using SSE 128 registers. 
I'd have to look into it more.

Offline

#11 2017-08-15 07:07:43

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 9,336
Website

Re: About ZLib/SynLZ/Snappy/LZ4

The x86 code is for plain x86 with no SSE instructions. This is on purpose.

Offline

Board footer

Powered by FluxBB