Need help - SynZip fails on FPC(trunc) Win64 target

mpv · 2018-02-15 14:53:59

@hnb @ab
I'm found a case SynZip SOMETIMES fail to uncompress http response under FPC win64 (delphi if OK, fpc/Linux64 is OK).
After many attempts I'm reproduce situation in debug mode and dump a wrong gzipped file can be downloaded here (266 bytes)
The test case:

program Project1;
{$mode DELPHI}{$H+}
{$I Synopse.inc}
uses
  {$IFDEF UNIX}{$IFDEF UseCThreads}
  cthreads,
  {$ENDIF}{$ENDIF}
  Classes,
  SynZip,
  SynCommons;

var
    s: RawByteString;
    a: AnsiString;
begin
   s := StringFromFile('wrong.gzip');
   a := CompressGZip(s, false);
end.

Result: Error -5 during uncompress mem

Error is raised inside paszlib/src/zinflate unit (inlined) and I'm not understand how to catch it
So I ask for a HELP

P.S. 99% of requests uncompressed well

ab · 2018-02-15 15:45:09

This is a FPC zlib bug...

mpv · 2018-02-15 19:12:13

OK, when let's use a SynZLibSSE by default for FPC/Win64 target? At last all my integration test is passed in this case. I create a pull request 82

ab · 2018-02-15 23:26:53

SynZLibSSE is only for SSE4 cpus if I remember correctly.

mpv · 2018-02-16 06:41:48

Yes, but this mean all Intel Core i3 and UP (2010) & AMD Opteron and UP (2011). If we spoke about windows x64 target - almost all computers.
In any case IMHO better that it does not work at all than does not work stably.
This bug is really critical for me, because it depends on user input. User enter something on form (in browser), browser gzip request and server can't unzip it

Last edited by mpv (2018-02-16 06:55:41)

hnb · 2018-02-16 07:24:02

ab wrote:

This is a FPC zlib bug...

maybe is worth to report this on FPC mantis bug tracker?

mpv · 2018-02-16 08:27:56

I check without SynZip - all work without errors. So problem in SynZip. I continue debugging

ab · 2018-02-16 08:49:48

Or we may just supply the "standard" .o files for FPC on both Win32 and Win64, just like with Delphi.

mpv · 2018-02-16 09:24:39

Catch it! See fix
USEZLIBSSE returned back to original version(commented)
Also I add a small ifndef to not redefine a ZLIB_VERSION from zbase

ab · 2018-02-18 16:57:58

Please check https://github.com/synopse/mORMot/commi … ff58e03490

mpv · 2018-02-18 18:37:54

I found some problems with this commit (can't even compile to verify):
1) Unicode Delphi (Win32) PtrUInt defined twice with a different declaration - once SynCrtSock:348 (PtrUInt = cardinal), second SynZip:275 (PtrUInt = NativeUInt)
2) FPC Win64. If USEZLIBSSE defined - folder fpc-win64/sse is missing (please, add it to github). If not defined - I got the strange compiler errors: "Error: Undefined symbol: memcpy" (and the same about memset, malloc, free)

Last edited by mpv (2018-02-18 18:44:19)

ab · 2018-02-18 21:20:18

The missing files were added to the official sqlite3fpc.7z but you are right: I didn't add them to github.

Please also check https://synopse.info/fossil/info/8b8cf321c5

mpv · 2018-02-18 22:37:20

Delphi win32 is Ok now - thanks!
But FPC win64 fails with Undefined symbol: memcpy (memset, malloc, free) even for a program from the first topic post with USEZLIBSSE and without USEZLIBSSE. Either something wrong with FPC linker of with *.o files or with defines. Even don't know where to start...

ab · 2018-02-18 23:40:07

Please check https://synopse.info/fossil/info/5881f48f25

mpv · 2018-02-19 08:19:22

I checked the following cases: Delphi(XE2) Win32; FPC(trunc) Linux64 w/o USEZLIBSSE; FPC(trunc) win64 with and without USEZLIBSSE; All works. Many-many thanks for your work!

macfly · 2021-06-16 20:52:05

i'm working on my first application using mormot2 + orm and i'm getting this same error when compiling.

Undefined symbol: memcpy (and the same about memset, malloc, free)

Lazarus 2.0.10 Win32
FPC 3.2.0

Win32 - Same error
Cross compile Win64 - Same error
Cross compile Linux64 - OK

Mormot2 with latest updates on git and latest static files (from git releases).

Last edited by macfly (2021-06-16 20:54:43)

ab · 2021-06-17 07:37:30

Did you include mormot.lib.static.pas to your project?

I am not able to reproduce here. No problem to compile and run mormot2tests.lpi on Win32/Win64.
https://gist.github.com/synopse/550bb7b … d61c6550f8

macfly · 2021-06-17 11:49:28

Sorry for taking your time.

I was ignoring that it was necessary to include mormot.lib.static.pas.

Solved.

ab · 2021-06-17 14:34:01

This unit should have been included by default with mormot.core.zip.
I have just ensured mormot.lib.static is included to properly link libdeflate.

macfly · 2021-06-18 20:14:00

Thanks ab,

I updated and removed mormot.lib.static.pas from uses.

Now it compiles for Win32 but Win64 keeps asking for these functions, because LIBDEFLATESTATIC is not defined for it.

But it's ok, I'll leave mormot.lib.static.pas in the project.

mormot.defines.inc

 {$ifdef OSWINDOWS}
    {$ifdef CPUX86}
      {$define LIBDEFLATESTATIC}
      {$define LIBQUICKJSSTATIC}
    {$endif CPUX86}
    {$ifdef CPUX64}
      {.$define LIBDEFLATESTATIC} // Win64 + FPC 3.2 = internal error 200603061 <<-------
      {$define LIBQUICKJSSTATIC}
    {$endif CPUX64}
  {$endif OSWINDOWS}

ab · 2021-06-19 10:35:05

mormot.lib.static should be included now for Win64

mORMot Open Source

#1 2018-02-15 14:53:59

Need help - SynZip fails on FPC(trunc) Win64 target

#2 2018-02-15 15:45:09

Re: Need help - SynZip fails on FPC(trunc) Win64 target

#3 2018-02-15 19:12:13

Re: Need help - SynZip fails on FPC(trunc) Win64 target

#4 2018-02-15 23:26:53

Re: Need help - SynZip fails on FPC(trunc) Win64 target

#5 2018-02-16 06:41:48

Re: Need help - SynZip fails on FPC(trunc) Win64 target

#6 2018-02-16 07:24:02

Re: Need help - SynZip fails on FPC(trunc) Win64 target

#7 2018-02-16 08:27:56

Re: Need help - SynZip fails on FPC(trunc) Win64 target

#8 2018-02-16 08:49:48

Re: Need help - SynZip fails on FPC(trunc) Win64 target

#9 2018-02-16 09:24:39

Re: Need help - SynZip fails on FPC(trunc) Win64 target

#10 2018-02-18 16:57:58

Re: Need help - SynZip fails on FPC(trunc) Win64 target

#11 2018-02-18 18:37:54

Re: Need help - SynZip fails on FPC(trunc) Win64 target

#12 2018-02-18 21:20:18

Re: Need help - SynZip fails on FPC(trunc) Win64 target

#13 2018-02-18 22:37:20

Re: Need help - SynZip fails on FPC(trunc) Win64 target

#14 2018-02-18 23:40:07

Re: Need help - SynZip fails on FPC(trunc) Win64 target

#15 2018-02-19 08:19:22

Re: Need help - SynZip fails on FPC(trunc) Win64 target

#16 2021-06-16 20:52:05

Re: Need help - SynZip fails on FPC(trunc) Win64 target

#17 2021-06-17 07:37:30

Re: Need help - SynZip fails on FPC(trunc) Win64 target

#18 2021-06-17 11:49:28

Re: Need help - SynZip fails on FPC(trunc) Win64 target

#19 2021-06-17 14:34:01

Re: Need help - SynZip fails on FPC(trunc) Win64 target

#20 2021-06-18 20:14:00

Re: Need help - SynZip fails on FPC(trunc) Win64 target

#21 2021-06-19 10:35:05

Re: Need help - SynZip fails on FPC(trunc) Win64 target

Board footer