#1 2023-05-10 15:39:03

WG42
Member
Registered: 2023-05-10
Posts: 7

Unit mormot.lib.win7zip, support for -ssw switch and compression issue

Currently I am refactoring my software to make use of mORMot2. I started years ago with TZipFile and moved later to Jedi JCL JclCompression.
mORMot2 works great so far, thank you for that.

1)
A user wrote me an email. Compressing files stops, when trying to add a file which is currently opened by another program.
When using the -ssw switch (I suppose he tried with 7z CLI) everything works fine.
> is it possible to add custom switch support? Or at least the -ssw switch for the moment so opened files can be compressed?

2)
When setting CompressionLevel to 0 (SAVE), CompressionMethod to m7Copy and setting up some more options to create an archiv, this archive gets created as expected in Save/Copy mode.
As soon as you add a password or encrypt headers, CompressionMethod will be set automatically to LZMA.

Last edited by WG42 (2023-05-10 19:00:46)

Offline

#2 2023-05-10 16:42:59

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,183
Website

Re: Unit mormot.lib.win7zip, support for -ssw switch and compression issue

Thanks for the feedback.

Note that there is no "switch" whatsoever in the 7Z API.
Even no simple methods to call - just a complex net of interfaces. wink

1) The files are opened as fmOpenReadDenyNone = fmOpenRead or fmShareDenyNone
in T7zWriter.GetStream() so they should be opened once currently opened by another program.

2) I have no idea where this behavior comes from.
Is the 7z CLI not doing the same?

Offline

#3 2023-05-10 16:59:34

WG42
Member
Registered: 2023-05-10
Posts: 7

Re: Unit mormot.lib.win7zip, support for -ssw switch and compression issue

Thank you for your answer.

To 1) I will test that using mORMot2.
The behavior "compression stops when trying to compress a file in use/opened by another program" occurs when using JclCompression I think.

When using 7z GUI normally, everything is working as set in the options. I can set SAVE mode (0), Encrypt headers and a Password. The archive compression method is Copy instead of LZMA.
When using mORMot2, once you set a password or encrypt headers, LZMA is used.

Offline

#4 2023-05-10 17:55:38

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,183
Website

Re: Unit mormot.lib.win7zip, support for -ssw switch and compression issue

In respect to JclCompression, we did fix some bugs during the rewrite as mormot..lib.win7zip.pas - and I hope we did not add any new bug.
wink

Was encrypt + save working fine with JclCompression?

Offline

#5 2023-05-10 17:59:49

WG42
Member
Registered: 2023-05-10
Posts: 7

Re: Unit mormot.lib.win7zip, support for -ssw switch and compression issue

Encrypt Headers + Save (Level 0) and Copy mode works with JclCompression.

Tested it also with mORMot2:
- FileFormat > fh7z
- SetCompressionLevel > 0
- SetCompressionMethod7z > m7Copy
- SetPassword > password

Everything fine up to here, now add EncryptHeaders7z(True); and LZMA is applied.

Why I use mORMot2 and not JclCompression:
- it's being worked on - thank you so much for that!
- it's from france - me too... sort of
- it's better in my opinion
- many more reasons

Last edited by WG42 (2023-05-10 18:07:10)

Offline

#6 2023-05-10 18:12:05

tbo
Member
Registered: 2015-04-20
Posts: 335

Re: Unit mormot.lib.win7zip, support for -ssw switch and compression issue

ab wrote:

1) The files are opened as fmOpenReadDenyNone = fmOpenRead or fmShareDenyNone
in T7zWriter.GetStream() so they should be opened once currently opened by another program.

If you want to save a folder with function I7zWriter.AddFiles and open a file of this folder before, an exception is thrown:

var fileStream: THandleStream := TFileStreamEx.Create('c:\mORMot2\src\lib\mormot.lib.win7zip.pas', fmOpenRead);
try
  var libWriter: I7zWriter := New7zWriter(fh7z);
  libWriter.AddFiles('c:\mORMot2\src\', 'mORMot2_2023-05-01', '*.pas;*.inc', True);
  libWriter.SaveToFile(MakePath([Executable.ProgramFilePath, 'TestMultiData.7z']));
finally
  fileStream.Free;
end;

The two exceptions are as follows:

ExceptionMessage="TFileStreamEx.Create(c:\mORMot2\src\lib\mormot.lib.win7zip.pas) failed as ERROR_SHARING_VIOLATION"
ExceptionName="EOSException"
ExceptionAddress=763A8FC2
---------------------------
ExceptionMessage="T7zWriter.SaveToStream error 8007000E (Für diesen Vorgang sind nicht genügend Speicherressourcen verfügbar)"
ExceptionName="E7Zip"
ExceptionAddress=763A8FC2

The behavior is normal, unless the file would have been opened with fmOpenRead or fmShareDenyWrite/ or fmShareDenyNone. The question would be rather, couldn't you skip this file without canceling the whole process?

With best regards
Thomas

Last edited by tbo (2023-05-10 18:16:54)

Offline

#7 2023-05-10 18:30:38

WG42
Member
Registered: 2023-05-10
Posts: 7

Re: Unit mormot.lib.win7zip, support for -ssw switch and compression issue

Maybe the ProgressCallback could result/send something once a file is done compressing or once a file has not been compressed due to an error. Like this in ProgressCallback we could catch these errors for logging.
it would be nice to have a more powerful ProgressCallback event. Is a file done compressing, did it work or not, which file has been compressed (name of it).

Last edited by WG42 (2023-05-10 18:44:42)

Offline

#8 2023-05-10 18:36:20

tbo
Member
Registered: 2015-04-20
Posts: 335

Re: Unit mormot.lib.win7zip, support for -ssw switch and compression issue

WG42 wrote:

Maybe the ProgressCallback could result/send something once a file is done compressing or once a file has not been compressed due to an error. Like this in ProgressCallback we could catch these errors for logging.

Or an additional wish, the function AddFiles would have as return a list of files that could not be packed.

With best regards
Thomas

Offline

#9 2023-05-11 07:52:26

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,183
Website

Re: Unit mormot.lib.win7zip, support for -ssw switch and compression issue

If the file is opened with plain fmOpenRead then no other process would be able to open it - for sure.

Please try https://github.com/synopse/mORMot2/commit/9674e7ee
- the exception should be intercepted for locked files during  the process
- AddFiles() will return the list of locked files which were not marked to be added

Offline

#10 2023-05-11 14:13:44

tbo
Member
Registered: 2015-04-20
Posts: 335

Re: Unit mormot.lib.win7zip, support for -ssw switch and compression issue

ab wrote:

Please try https://github.com/synopse/mORMot2/commit/9674e7ee
- the exception should be intercepted for locked files during  the process
- AddFiles() will return the list of locked files which were not marked to be added

Commit 5441 (9674e7e) works as expected. Thanks for this. The following test was passed:

var fileStream: THandleStream := Nil;
try
  fileStream := TFileStreamEx.Create('c:\mORMot2\src\lib\mormot.lib.win7zip.pas', fmOpenRead);
  var libWriter: I7zWriter := New7zWriter(fh7z);
  var skippedFiles: TFileNameDynArray := libWriter.AddFiles('c:\mORMot2\src\', '', '*.pas;*.inc', True);
  if Length(skippedFiles) > 0 then
    ShowMessage(Format('Skipped Files: %d', [Length(skippedFiles)]));

  libWriter.SaveToFile(MakePath([Executable.ProgramFilePath, 'TestData.7z']));
finally
  fileStream.Free;
end;

IsFileReadable function added in AddFiles makes no measurable difference in timing.

With best regards
Thomas

Last edited by tbo (2023-05-11 14:23:55)

Offline

#11 2023-05-11 15:51:20

WG42
Member
Registered: 2023-05-10
Posts: 7

Re: Unit mormot.lib.win7zip, support for -ssw switch and compression issue

Thank you very much. Just wanted to know if this would be possible too

Maybe the ProgressCallback could result/send something once a file is done compressing or once a file has not been compressed due to an error. Like this in ProgressCallback we could catch these errors for logging.
it would be nice to have a more powerful ProgressCallback event. Is a file done compressing, did it work or not, which file has been compressed (name of it).

A more powerful OnProgress event would be so good!

Didn't fully understand about ssw switches and options like that in general. Is it possible to add that too?
Another software written in Delphi offers such an option to pass options like -ssw.

Passing custom parameters to the 7z.dll would also solve some problems users might get.

Last edited by WG42 (2023-05-11 16:51:01)

Offline

#12 2023-05-11 16:44:56

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,183
Website

Re: Unit mormot.lib.win7zip, support for -ssw switch and compression issue

@WG42
It is not possible to "pass custom parameters to 7z.dll" because, as I wrote above, the 7z.dll has no knowledge of the 7z.exe parameters whatsoever.

Offline

#13 2023-05-11 16:50:06

WG42
Member
Registered: 2023-05-10
Posts: 7

Re: Unit mormot.lib.win7zip, support for -ssw switch and compression issue

Sorry I missunderstood something. In this other software you can just input something like x=20 or mt=4. This input is parsed and set later normally. So... nothing for the library.

Last edited by WG42 (2023-05-11 16:56:51)

Offline

#14 2023-05-24 15:35:59

WG42
Member
Registered: 2023-05-10
Posts: 7

Re: Unit mormot.lib.win7zip, support for -ssw switch and compression issue

Could you add the skippedFile functionality also to the AddFile() function?

function T7zWriter.AddFile(const Filename: TFileName; const ZipName: RawUtf8): string;
var
  item: T7zItem;
  Handle: THandle;
begin
  Handle := FileOpen(Filename, fmOpenReadDenyNone);
  if not ValidHandle(Handle) then
    exit;
  // result := ValidHandle(Handle);
  // if not result then
  //   exit;

  if IsFileReadable(Filename) then
   begin
    item := T7zItem.Create;
    item.SourceMode := smFile;
    item.FileName := Filename;
    item.ZipName := ZipName;
    GetFileTime(Handle, @item.CreationTime, nil, @item.LastWriteTime);
    item.Size := FileSize(Handle);
    CloseHandle(Handle);
    item.Attributes := GetFileAttributes(pointer(Filename));
    item.IsFolder := item.Attributes and faDirectory <> 0;
    item.IsAnti := false;
    item.Ownership := soOwned;
    item.UpdateItemIndex := -1;
    AddOrReplace(item);
   end
   else
     // notify the caller that this file is locked
     result := Filename;
end;

Maybe there could be a list, not a function result, in which AddFile() and AddFiles() add files to skip.
So when I use 100 times AddFile() (because I have my own list I go through to add files) then I would like to check only 1 files to skip list after these 100 AddFile() calls.

Last edited by WG42 (2023-05-24 16:18:57)

Offline

#15 2023-05-24 16:15:30

tbo
Member
Registered: 2015-04-20
Posts: 335

Re: Unit mormot.lib.win7zip, support for -ssw switch and compression issue

WG42 wrote:

Could you add the skippedFile(s) functionality also to the AddFile() function?

This is not necessary because AddFile() does not include a locked file. See following example:

var fileStream: THandleStream := Nil;
try
  var testFile: TFileName := 'c:\mORMot2\src\lib\mormot.lib.win7zip.pas';
  fileStream := TFileStreamEx.Create(testFile, fmOpenRead);

  var skippedFiles: TFileNameDynArray;
  var libWriter: I7zWriter := New7zWriter(fh7z);
  libWriter.SetCompressionLevel(3);
  if libWriter.AddFile(testFile, StringToUtf8(ExtractFileName(testFile))) then
    libWriter.SaveToFile(MakePath([Executable.ProgramFilePath, 'TestData.7z']))
  else
    AddString(TStringDynArray(skippedFiles), testFile);

  for var fileName: TFileName in libWriter.AddFiles('c:\mORMot2\src\', '', '*.pas;*.inc', True) do
    AddString(TStringDynArray(skippedFiles), fileName);

  ShowMessage(Format('File(s) could not be added! Count: %d', [Length(skippedFiles)]));
finally
  fileStream.Free;
end;

With best regards
Thomas

Last edited by tbo (2023-05-24 16:16:46)

Offline

Board footer

Powered by FluxBB