#1 2025-01-31 09:42:25

okoba
Member
Registered: 2019-09-29
Posts: 126

pigz support?

I was looking for faster streaming of gzip and as libdeflate does not support streaming, I want to ask if anyone here has experience with pigz?

Offline

#2 2025-01-31 11:21:45

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,845
Website

Re: pigz support?

No experience nor plan to use it.

Also found out https://pypi.org/project/mgzip/
which sounds not a bad idea.

But all those libraries are for gzip only, not deflate/zip, and they seem to use a specific extra flag.

Offline

#3 2025-01-31 12:39:48

okoba
Member
Registered: 2019-09-29
Posts: 126

Re: pigz support?

Then ZSTD is the way to go.
Thank you.

Offline

#4 2025-01-31 14:41:06

zen010101
Member
Registered: 2024-06-15
Posts: 99

Re: pigz support?

okoba wrote:

Then ZSTD is the way to go.

https://github.com/L-Angel/compress-demo

This is a comparison of several common compression algorithms, including Snappy and zstd. The conclusion is:

zstd is similar to snappy. And according to the test of ab, synlz is better than snappy. https://blog.synopse.info/?post/2017/08 … form-SynLZ

Offline

#5 2025-01-31 14:47:09

okoba
Member
Registered: 2019-09-29
Posts: 126

Re: pigz support?

@ab I sent a pull request for it.
As ZSTD is now supported by almost all browsers, it can be a good boost for mORMot servers: https://developer.mozilla.org/en-US/doc … patibility
Also much beneficial for compressing log and backup files.

Last edited by okoba (2025-01-31 14:50:29)

Offline

#6 2025-01-31 14:49:06

okoba
Member
Registered: 2019-09-29
Posts: 126

Re: pigz support?

@zen010101 the test mentioned is very old.
Here is a good one: https://morotti.github.io/lzbench-web/

Offline

#7 2025-01-31 14:51:22

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,845
Website

Re: pigz support?

Does not include libdeflate.

Offline

#8 2025-01-31 14:53:41

okoba
Member
Registered: 2019-09-29
Posts: 126

Re: pigz support?

Offline

Board footer

Powered by FluxBB