#1 2020-03-26 17:13:36

Eric
Member
Registered: 2012-11-26
Posts: 129
Website

SynZip slow when unzipping executable because of Microsoft Antivirus

Hi,

I am experiencing very slow unzip times when unzipping executables (.exe, .dll) with SynZip, while unzipping the same archive with 7Zip is very fast.

Looking at the Windows task manager, all the CPU time is spent in the Microsoft Antivirus scanner, apparently every time SynZip writes a block, the AV does a scan.

By comparison, 7Zip only shows antivirus activity when the unzip is complete.

I have tried setting exclusive mode for the TFileStream in the .Unzip() method, but that showed no improvement...
I also tried to unzip in memory, then write all at once, no improvement either...

Any other ideas ?

Offline

#2 2020-03-26 17:25:09

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 13,504
Website

Re: SynZip slow when unzipping executable because of Microsoft Antivirus

Perhaps 7Zip is detected and white-listed by the AV, not your executable, which is not trusted.

Did you try to add a digital signature to it?

What if you unzip into a .toto file, not .exe file, and then rename it once finished?

Offline

#3 2020-03-27 07:39:21

Eric
Member
Registered: 2012-11-26
Posts: 129
Website

Re: SynZip slow when unzipping executable because of Microsoft Antivirus

No digital signature on the unzipper, but running a full scan on the extracted binaries is very fast, barely registering in CPU usage, and AFAICT it happens for 7zip as well.
(but the unzipped binaries are signed)

Apparently some other zip extractors run into the same issue (https://thomasmullaly.codes/2017/11/19/ … hocolatey/)

I have tried renaming the file, no effect.
I also tried writing the first 16 kB header with zeroes, and then the rest of the file with content, this seems enough to prevent the slowdown. But when opening the file again to write the missing 16 kB headers, the MsMpEng slowdown kicks in again.

I also investigated when the slowdown occurs, it's on the FileClose, writing the data itself is fast.

When delaying the close, by not freeing the TFileStream and not doing the FileClose, then it is possible to unzip all the files at high speed.
Tthe solution might be to defer all the FileClose to a background thread or an asynchronous process of some sort...

Last edited by Eric (2020-03-27 07:40:21)

Offline

#4 2020-03-27 07:50:45

Eric
Member
Registered: 2012-11-26
Posts: 129
Website

Re: SynZip slow when unzipping executable because of Microsoft Antivirus

Another clue possibly:

Just tried with an old unzip.exe (http://gnuwin32.sourceforge.net/packages/unzip.htm), it shows a very high MsMpEng.exe activity, but the extraction is overall quite fast (5-6 seconds, vs 40 seconds with SynZip in a Delphi binary)
By reference 7zip takes 1-2 seconds with minimal MsMpEng activity (comparable to the activity when scanning the extracted files)

Offline

#5 2020-03-27 20:31:22

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 13,504
Website

Re: SynZip slow when unzipping executable because of Microsoft Antivirus

Both with Win32 and WIn64 executables?

Offline

#6 2020-03-30 07:06:04

Eric
Member
Registered: 2012-11-26
Posts: 129
Website

Re: SynZip slow when unzipping executable because of Microsoft Antivirus

No difference between Win32 et Win64...

I simplified the code to just this, to try variation of the locking and access mode (with no effect)

var buf := z.UnZip(i);
var h := CreateFile(PChar(fileName), GENERIC_READ or GENERIC_WRITE,
                    0, nil, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0);
WriteFile(h, buf[1], Length(buf), nb, nil);
Assert(nb = Length(buf));
CloseHandle(h);   // <-------- this is the slow part, and only it

Offline

#7 2020-03-30 12:33:42

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 13,504
Website

Re: SynZip slow when unzipping executable because of Microsoft Antivirus

I guess it would occur with any other library than SynZip.

Sounds like a MS AV paranoid check...

Offline

#8 2020-03-31 07:45:13

Eric
Member
Registered: 2012-11-26
Posts: 129
Website

Re: SynZip slow when unzipping executable because of Microsoft Antivirus

It is not a SynZip issues yes, but it does not happen (as bad) with various unzip.exe I tested.
Delphi RTL must be doing something that triggers MS AV in a bad way... sad

Offline

#9 2020-03-31 08:27:08

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 13,504
Website

Re: SynZip slow when unzipping executable because of Microsoft Antivirus

Did you try to compile with FPC?
Perhaps the MS AV heuristic detects Delphi but not FPC...
More viruses were written in Delphi than in FPC I guess.
Or try to use a dll to unzip?

Offline

#10 2021-02-11 15:02:53

Eric
Member
Registered: 2012-11-26
Posts: 129
Website

Re: SynZip slow when unzipping executable because of Microsoft Antivirus

About one year later, MS AV is still present, with varying degrees of occurrence depending on binary...
I got my executable whitelisted, but it's not enough, apparently the only surefire way is to add an exclusion.

Offline

Board footer

Powered by FluxBB