#1 2018-02-15 14:53:59

mpv
Member
From: Ukraine
Registered: 2012-03-24
Posts: 1,306
Website

Need help - SynZip fails on FPC(trunc) Win64 target

@hnb @ab
I'm found a case SynZip SOMETIMES fail to uncompress http response under FPC win64 (delphi if OK, fpc/Linux64 is OK).
After many attempts I'm reproduce situation in debug mode and dump a wrong gzipped file can be downloaded here (266 bytes)
The test case:

program Project1;
{$mode DELPHI}{$H+}
{$I Synopse.inc}
uses
  {$IFDEF UNIX}{$IFDEF UseCThreads}
  cthreads,
  {$ENDIF}{$ENDIF}
  Classes,
  SynZip,
  SynCommons;

var
    s: RawByteString;
    a: AnsiString;
begin
   s := StringFromFile('wrong.gzip');
   a := CompressGZip(s, false);
end.

Result: Error -5 during uncompress mem

Error is raised inside paszlib/src/zinflate unit (inlined) and I'm not understand how to catch it
So I ask for a HELP

P.S. 99% of requests uncompressed well

Offline

#2 2018-02-15 15:45:09

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 13,268
Website

Re: Need help - SynZip fails on FPC(trunc) Win64 target

This is a FPC zlib bug...

Online

#3 2018-02-15 19:12:13

mpv
Member
From: Ukraine
Registered: 2012-03-24
Posts: 1,306
Website

Re: Need help - SynZip fails on FPC(trunc) Win64 target

OK, when let's use a SynZLibSSE by default for FPC/Win64 target? At last all my integration test is passed in this case. I create a pull request 82

Offline

#4 2018-02-15 23:26:53

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 13,268
Website

Re: Need help - SynZip fails on FPC(trunc) Win64 target

SynZLibSSE is only for SSE4 cpus if I remember correctly.

Online

#5 2018-02-16 06:41:48

mpv
Member
From: Ukraine
Registered: 2012-03-24
Posts: 1,306
Website

Re: Need help - SynZip fails on FPC(trunc) Win64 target

Yes, but this mean all Intel Core i3 and UP (2010) & AMD Opteron and UP (2011). If we spoke about windows x64 target - almost all computers.
In any case IMHO better that it does not work at all than does not work stably.
This bug is really critical for me, because it depends on user input. User enter something on form (in browser), browser gzip request and server can't unzip it

Last edited by mpv (2018-02-16 06:55:41)

Offline

#6 2018-02-16 07:24:02

hnb
Member
Registered: 2015-06-15
Posts: 291

Re: Need help - SynZip fails on FPC(trunc) Win64 target

ab wrote:

This is a FPC zlib bug...

maybe is worth to report this on FPC mantis bug tracker?


best regards,
Maciej Izak

Offline

#7 2018-02-16 08:27:56

mpv
Member
From: Ukraine
Registered: 2012-03-24
Posts: 1,306
Website

Re: Need help - SynZip fails on FPC(trunc) Win64 target

I check without SynZip - all work without errors. So problem in SynZip. I continue debugging

Offline

#8 2018-02-16 08:49:48

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 13,268
Website

Re: Need help - SynZip fails on FPC(trunc) Win64 target

Or we may just supply the "standard" .o files for FPC on both Win32 and Win64, just like with Delphi.

Online

#9 2018-02-16 09:24:39

mpv
Member
From: Ukraine
Registered: 2012-03-24
Posts: 1,306
Website

Re: Need help - SynZip fails on FPC(trunc) Win64 target

Catch it! See fix
USEZLIBSSE returned back to original version(commented)
Also I add a small ifndef to not redefine a ZLIB_VERSION  from zbase

Offline

#10 2018-02-18 16:57:58

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 13,268
Website

Re: Need help - SynZip fails on FPC(trunc) Win64 target

Online

#11 2018-02-18 18:37:54

mpv
Member
From: Ukraine
Registered: 2012-03-24
Posts: 1,306
Website

Re: Need help - SynZip fails on FPC(trunc) Win64 target

I found some problems with this commit (can't even compile to verify):
1) Unicode Delphi (Win32) PtrUInt defined twice with a different declaration - once SynCrtSock:348 (PtrUInt = cardinal), second SynZip:275 (PtrUInt = NativeUInt)
2) FPC Win64. If USEZLIBSSE defined - folder fpc-win64/sse is missing (please, add it to github). If not defined - I got the strange compiler errors: "Error: Undefined symbol: memcpy" (and the same about memset, malloc, free)

Last edited by mpv (2018-02-18 18:44:19)

Offline

#12 2018-02-18 21:20:18

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 13,268
Website

Re: Need help - SynZip fails on FPC(trunc) Win64 target

The missing files were added to the official sqlite3fpc.7z but you are right: I didn't add them to github.

Please also check https://synopse.info/fossil/info/8b8cf321c5

Online

#13 2018-02-18 22:37:20

mpv
Member
From: Ukraine
Registered: 2012-03-24
Posts: 1,306
Website

Re: Need help - SynZip fails on FPC(trunc) Win64 target

Delphi win32 is Ok now - thanks!
But FPC win64 fails with Undefined symbol: memcpy (memset, malloc, free) even for a program from the first topic post with USEZLIBSSE and without USEZLIBSSE. Either something wrong with FPC linker of with *.o files or with defines. Even don't know where to start...

Offline

#14 2018-02-18 23:40:07

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 13,268
Website

Re: Need help - SynZip fails on FPC(trunc) Win64 target

Online

#15 2018-02-19 08:19:22

mpv
Member
From: Ukraine
Registered: 2012-03-24
Posts: 1,306
Website

Re: Need help - SynZip fails on FPC(trunc) Win64 target

I checked the following cases: Delphi(XE2) Win32; FPC(trunc) Linux64 w/o USEZLIBSSE; FPC(trunc) win64 with and without USEZLIBSSE; All works. Many-many thanks for your work!

Offline

#16 2021-06-16 20:52:05

macfly
Member
From: Brasil
Registered: 2016-08-20
Posts: 374

Re: Need help - SynZip fails on FPC(trunc) Win64 target

i'm working on my first application using mormot2 + orm and i'm getting this same error when compiling.

Undefined symbol: memcpy (and the same about memset, malloc, free)

Lazarus 2.0.10 Win32
FPC 3.2.0

Win32 - Same error
Cross compile Win64 - Same error
Cross compile Linux64 - OK

Mormot2 with latest updates on git and latest static files (from git releases).

Last edited by macfly (2021-06-16 20:54:43)

Offline

#17 2021-06-17 07:37:30

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 13,268
Website

Re: Need help - SynZip fails on FPC(trunc) Win64 target

Did you include mormot.lib.static.pas to your project?

I am not able to reproduce here. No problem to compile and run mormot2tests.lpi on Win32/Win64.
https://gist.github.com/synopse/550bb7b … d61c6550f8

Online

#18 2021-06-17 11:49:28

macfly
Member
From: Brasil
Registered: 2016-08-20
Posts: 374

Re: Need help - SynZip fails on FPC(trunc) Win64 target

Sorry for taking your time.

I was ignoring that it was necessary to include mormot.lib.static.pas.

Solved.

Offline

#19 2021-06-17 14:34:01

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 13,268
Website

Re: Need help - SynZip fails on FPC(trunc) Win64 target

This unit should have been included by default with mormot.core.zip.
I have just ensured mormot.lib.static is included to properly link libdeflate.

Online

#20 2021-06-18 20:14:00

macfly
Member
From: Brasil
Registered: 2016-08-20
Posts: 374

Re: Need help - SynZip fails on FPC(trunc) Win64 target

Thanks ab,

I updated and removed mormot.lib.static.pas from uses.

Now it compiles for Win32 but Win64 keeps asking for these functions, because LIBDEFLATESTATIC is not defined for it.

But it's ok, I'll leave mormot.lib.static.pas in the project.

mormot.defines.inc

 {$ifdef OSWINDOWS}
    {$ifdef CPUX86}
      {$define LIBDEFLATESTATIC}
      {$define LIBQUICKJSSTATIC}
    {$endif CPUX86}
    {$ifdef CPUX64}
      {.$define LIBDEFLATESTATIC} // Win64 + FPC 3.2 = internal error 200603061 <<-------
      {$define LIBQUICKJSSTATIC}
    {$endif CPUX64}
  {$endif OSWINDOWS}    

Offline

#21 2021-06-19 10:35:05

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 13,268
Website

Re: Need help - SynZip fails on FPC(trunc) Win64 target

mormot.lib.static should be included now for Win64

Online

Board footer

Powered by FluxBB