mORMot Open Source

jacobhoover

Member

Registered: 2010-11-04

Posts: 3

SynZip and Delphi 2010

I'm just wondering if anyone else was having issues generating a Zip compatible archive with the SynZip unit? Using code such as

  Zip := TZipWrite.Create(FileName);
  try
    Zip.AddDeflated(AFileName);
    Zip.AddDeflated(BFileName);
    Zip.AddDeflated(CFileName);
  finally
    FreeAndNil(Zip);
  end;

results in an archive that the structure can be seen by other archivers like WinAce, but all files in the archive fail to pass verification tests and cannot be extracted. (They failed the CRC test.)

I went back in version control, and used the prior CRC calculations (via crc32.obj instead of the runtime built CRC table) which then renders correct and valid Zip archives.

Is it possible that there is a need for specific compiler directives in order for the newer optimized code to function correctly with Delphi 2010?

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: SynZip and Delphi 2010

It's very strange....

By me, the zip archives seems correct out. Total Commander likes the crc32 of these files.
There are some dedicated regression tests against the crc32 calculation, using both slow pascal (and accurate) and optimized version.

I'll check for any problem with Delphi 2010.
But with Delphi 2010, the following regression tests passed without any issue:

procedure TTestCompression.GZipFormat;
var Z: TSynZipCompressor;
    L,n: integer;
    P: PAnsiChar;
    crc2: Cardinal;
begin
  Check(crc32(0,@crc32Tab,5)=$DF4EC16C,'crc32');
  Check(UpdateCrc32(0,@crc32Tab,5)=$DF4EC16C,'crc32');
  Check(crc32(0,@crc32Tab,1024)=$6FCF9E13,'crc32');
  Check(UpdateCrc32(0,@crc32Tab,1024)=$6FCF9E13);
  Check(crc32(0,@crc32Tab,1024-5)=$70965738,'crc32');
  Check(UpdateCrc32(0,@crc32Tab,1024-5)=$70965738);
  Check(crc32(0,pointer(PtrInt(@crc32Tab)+1),2)=$41D912FF,'crc32');
  Check(UpdateCrc32(0,pointer(PtrInt(@crc32Tab)+1),2)=$41D912FF);
  Check(crc32(0,pointer(PtrInt(@crc32Tab)+3),1024-5)=$E5FAEC6C,'crc32');
  Check(UpdateCrc32(0,pointer(PtrInt(@crc32Tab)+3),1024-5)=$E5FAEC6C);
  M := SynCommons.THeapMemoryStream.Create;
  Z := TSynZipCompressor.Create(M,6,szcfGZ);
  L := length(Data);
  P := Pointer(Data);
  crc := 0;
  crc2 := 0;
  while L<>0 do begin
    if L>1000 then
      n := 1000 else
      n := L;
    Z.Write(P^,n); // compress by little chunks to test streaming
    crc := crc32(crc,P,n);
    crc2 := UpdateCrc32(crc2,P,n);
    inc(P,n);
    dec(L,n);
  end;
  Check(crc=Z.CRC,'crc32');
  Check(crc2=crc,'crc32');
  Z.Free;
  Check(GZRead(M.Memory,M.Position)=Data);
end;

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: SynZip and Delphi 2010

I've added some more regression tests in TTestCompression.ZipFormat but I still can't reproduce your problem...

Did you try to undefine the BYFOUR conditional in line 3927 of SynZip.pas?

jacobhoover

Member

Registered: 2010-11-04

Posts: 3

Re: SynZip and Delphi 2010

Undefining BYFOUR had no effect. THe only unit I am using is the current version of SynZip.

Looking at the package it's in, the compiler defines are as such:

{$ALIGN 8}
{$ASSERTIONS ON}
{$BOOLEVAL OFF}
{$DEBUGINFO ON}
{$EXTENDEDSYNTAX ON}
{$IMPORTEDDATA ON}
{$IOCHECKS ON}
{$LOCALSYMBOLS ON}
{$LONGSTRINGS ON}
{$OPENSTRINGS ON}
{$OPTIMIZATION OFF}
{$OVERFLOWCHECKS ON}
{$RANGECHECKS ON}
{$REFERENCEINFO ON}
{$SAFEDIVIDE OFF}
{$STACKFRAMES ON}
{$TYPEDADDRESS OFF}
{$VARSTRINGCHECKS ON}
{$WRITEABLECONST OFF}
{$MINENUMSIZE 1}
{$IMAGEBASE $400000}
{$RUNONLY}
{$IMPLICITBUILD OFF}
{$DEFINE EUREKALOG}
{$DEFINE EUREKALOG_VER6}

I was going to run your unit tests, but they aren't quite right.

I ended dumbing it down to:
  Assert((crc32(0,get_crc_table,5)=$DF4EC16C),'crc32');
  Assert((crc32(0,get_crc_table,1024)=$6FCF9E13),'crc32');
  Assert((crc32(0,get_crc_table,1024-5)=$70965738),'crc32');

And the very first assertion fails.

jacobhoover

Member

Registered: 2010-11-04

Posts: 3

Re: SynZip and Delphi 2010

Now I'm at a loss. I seems for some reason the initializer was not running, as the SynZip CRC table was not being populated.  I was going to do a test betwen the SynZip table, and a static table to see if something was wrong when I noticed this.

I again did a clean, and a rebuild all, and now the new source is functioning. (Including the previously failing assertions.)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: SynZip and Delphi 2010

You should had a problem in your configuration.

I just made a test with the latest version from the source repository with Delphi 6, 7, 2007 and 2010, and it compiles and runs fine.
All tests passed...

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: SynZip and Delphi 2010

I can see that you use range checking ON.
It's not a good idea when you use our units.
Some part rely on range checking OFF, for speed, code generation and string access (we need to process the last #0 of a string, for instance).