You are not logged in.
- Topics: Active | Unanswered
Pages: 1
#1 2023-09-12 10:36:28
- okoba
- Member
- Registered: 2019-09-29
- Posts: 118
ZSTD support
Currently mORMot supports many compression algorithm, and I wanted to ask if there is a plan for ZSTD support?
Offline
#3 2023-09-12 12:06:59
- okoba
- Member
- Registered: 2019-09-29
- Posts: 118
Re: ZSTD support
Yes I benchmarked all compression library available in mORMot and ZSTD was faster and had better compression. libdeflate is great, but ZSTD can do better.
Although I agree, libdeflate is the good way especially for server needs. But for custom client and server codes, ZSTD can be the better choice.
I could take care of adding ZSTD to mORMot 2 in the current TAlgoCompress style, but the problem is mostly managing to prepare the best static libraries (including custom emmory manger like you done for SQLite and others, or shared C Lib codes). I don't know C and building their libraries is just a pain.
If thats possible for you to prepare that, I can write the code for the current official dll.
Offline
#4 2023-09-12 12:12:51
Re: ZSTD support
I am no C expert but I could do try to compile the code as we do for SQlite3.
My guess is that https://github.com/YWtheGod/ZSTD4Delphi … d4delphi.c may help as a startup main source.
Online
#5 2023-09-12 12:40:23
- okoba
- Member
- Registered: 2019-09-29
- Posts: 118
Re: ZSTD support
Oh believe me, compare to me, you are a god of C 
I will look into it.
Offline
#6 2023-09-13 06:05:58
- TPrami
- Member
- Registered: 2010-07-06
- Posts: 119
Re: ZSTD support
Also found this:
https://github.com/DenisAnisimov/ZSTD.pas
There is also LZ4 implementation.
Have no idea of the quality tough.
Both are interesting compression algorithms for sure.
Offline
#7 2023-09-13 07:22:37
- igors233
- Member
- Registered: 2012-09-10
- Posts: 241
Re: ZSTD support
Since 7z is already supported I think it would be better to check if 7z ZS is supported as well.
7z ZS supports ZTSD, brotli and other codecs and so it would give more options to users.
Brotli is also becoming more and more supported by webservers or API providers.
Offline
#9 2023-09-13 10:53:45
- igors233
- Member
- Registered: 2012-09-10
- Posts: 241
Re: ZSTD support
7z by default works on buffers/streams (IInStream, IOutStream) which caller decides how to implement, as file stream or memory stream.
Last edited by igors233 (2023-09-13 10:53:59)
Offline
#11 2023-09-13 11:20:36
- igors233
- Member
- Registered: 2012-09-10
- Posts: 241
Re: ZSTD support
ArchiveFile is also defined as working with IInStream (see IArchiveOpenCallback) so 7z archive can fully be in memory.
But I understand what you mean, overhead of having simple buffer treatead as archive instead of compressed buffer, I think that overhead isn't large and is worth having any compression supported.
> And the 7z library we support is a dll for Windows only.
7z does have debian/linux implementation, though I do not know how compatible it is or if 7z ZS has linux support.
BTW there's also LZMA2 compression (default 7z algo), InnoSetup has Pascal implementation of it and there are some other implementation but I don't know if they're cross platform.
Offline
#13 2023-09-13 17:07:17
- igors233
- Member
- Registered: 2012-09-10
- Posts: 241
Re: ZSTD support
> Here are interesting benchmarks from Tommi on the subject too.
I've read somewhere that comparing speeds with different settings is wrong (apples/oranges) so all test should be done using same window sizes.
Following link has that test (brotli is faster for compression, zstd for decompression):
https://peazip.github.io/fast-compressi … ndard.html
Search for table marked: Fixed 128 MB window size for both algorithms
Last edited by igors233 (2023-09-13 17:15:57)
Offline
Pages: 1