SynZip with UTF8 file names inside archive

mpv · 2021-11-16 16:06:53

current SynZip implementation always sets an "operating system of origin"
- in mORMot1 to 0 (DOS)
- in mORMot2 to ZIP_OS constant what depends on OS compilation target), but for windows target - 0

If "operating system of origin" is 0, even in case UFT8 flag is sets for file name, most of clients ( verified on Win with winrar, 7zip, explorer and on Linux with zipinfo tool) displays wrong file name (ignores UTF8 bit and display as ASCII)

I made a PR in mORMot1 - https://github.com/synopse/mORMot/pull/414 what force OS to Unix, in mORMot2 I propose to set ZIP_OS to 3 for Windows

mpv · 2021-11-16 18:50:25

I attach 2 files to PR, both with cyrillic file names inside, one before fix and one - after fix

ab · 2021-11-16 21:17:13

My guess is that 10 = Windows NTFS should be used instead.
See https://pkware.cachefly.net/webdocs/cas … PPNOTE.TXT

4.4.2.2 The current mappings are:
0 - MS-DOS and OS/2 (FAT / VFAT / FAT32 file systems)
1 - Amiga 2 - OpenVMS
3 - UNIX 4 - VM/CMS
5 - Atari ST 6 - OS/2 H.P.F.S.
7 - Macintosh 8 - Z-System
9 - CP/M 10 - Windows NTFS
11 - MVS (OS/390 - Z/OS) 12 - VSE
13 - Acorn Risc 14 - VFAT
15 - alternate MVS 16 - BeOS
17 - Tandem 18 - OS/400
19 - OS X (Darwin) 20 thru 255 - unused

ab · 2021-11-16 21:22:27

I have merged your pull request for mORMot 1.

And set ZIP_OS = 10 on Windows for mORMot 2.

mpv · 2021-11-17 11:16:44

It's strange, but when I sets madeBy flag to $0A14 then zipinfo -v fileName.zip shows OS as TOPS-20 (even don't know before what such OS exists), not an NTFS as in spec, but Unicode names works in this case

$ zipinfo -v utf8.zip 
....

Central directory entry #1:
---------------------------

  папка/новыйФайл.txt

  offset of local header from start of archive:   0
  file system or operating system of origin:      TOPS-20
  version of encoding software:                   2.0
  minimum file system compatibility required:     MS-DOS, OS/2 or NT FAT

From zipinfo sources https://github.com/LuaDist/unzip/blob/m … fo.c#L1003 for NTFS 11 ($0B) is used (the same zipinfo.c is in apple repository)

If I sets madeBy flag to $0B14 zipinfo show OS as NTFS but filename is broken

Central directory entry #1:
---------------------------

  ╨┐╨░╨┐╨║╨░/╨╜╨╛╨▓╤Л╨╣╨д╨░╨╣╨╗.txt

  offset of local header from start of archive:   0
  file system or operating system of origin:      NTFS
  version of encoding software:                   2.0
  minimum file system compatibility required:     MS-DOS, OS/2 or NT FAT

So I really don't know where is a source of truth.

P.S.
$03 - Unix works as expected for both OS info and file name encoding

Last edited by mpv (2021-11-17 11:20:32)

ab · 2021-11-17 13:57:37

What does WinZip emit as version madeBy flag value?

mpv · 2023-03-22 18:56:52

Another patch for non-english file names in zip - see mORMot1 MR444. We discover what build-in zip on latest Windows10 Home put such names into Unicode Path Extra Field.

Good to be ported also in mORMot2

Last edited by mpv (2023-03-22 18:58:04)

mpv · 2023-03-23 13:14:16

I attach test zip file into https://github.com/synopse/mORMot/pull/444

ab · 2023-03-23 20:43:05

Yes, I have made a similar modification in mORMot 2.
But diverse because we need to support also Zip64 which is also in an extended file info extra block.
https://github.com/synopse/mORMot2/commit/76ca69e1c3a78

mORMot Open Source

#1 2021-11-16 16:06:53

SynZip with UTF8 file names inside archive

#2 2021-11-16 18:50:25

Re: SynZip with UTF8 file names inside archive

#3 2021-11-16 21:17:13

Re: SynZip with UTF8 file names inside archive

#4 2021-11-16 21:22:27

Re: SynZip with UTF8 file names inside archive

#5 2021-11-17 11:16:44

Re: SynZip with UTF8 file names inside archive

#6 2021-11-17 13:57:37

Re: SynZip with UTF8 file names inside archive

#7 2023-03-22 18:56:52

Re: SynZip with UTF8 file names inside archive

#8 2023-03-23 13:14:16

Re: SynZip with UTF8 file names inside archive

#9 2023-03-23 20:43:05

Re: SynZip with UTF8 file names inside archive

Board footer