No such file or directory (Unicode chars)

[sergeevabc]

bzip3 1.5.1, Windows 7 SP1.

$ dir /b *.txt
vögel.txt

$ bzip3.exe *.txt
Error: failed to open input file `*.txt': Invalid argument

$ bzip3.exe vögel.txt
Error: failed to open input file `vogel.txt': No such file or directory

However, some other modern CLI compression apps work fine.

$ 7z.exe a hello vögel.txt
7-Zip 24.09 (x64) : Copyright (c) 1999-2024 Igor Pavlov : 2024-11-29
Scanning the drive:
1 file, 5 bytes (1 KiB)
Creating archive: hello.7z
Add new data to archive: 1 file, 5 bytes (1 KiB)
Files read from disk: 1
Archive size: 131 bytes (1 KiB)
Everything is Ok

$ zpaqfranz.exe a hello vögel.txt
zpaqfranz v60.10w-JIT,-GUI,-L,HW BLAKE3,SHA1/2,4,SFX64 v55.1,(2024-12-15)
franz:-noconsole
Creating hello.zpaq at offset 0 + 0
Add 2025-02-10 12:03:51         1                  5 (   5.00  B) 2T (0 dirs)
1 +added, 0 -removed.
0 + (5 -> 5 ->  829) = 829  @ 64.00  B/s
Files added +1
0.093s (00:00:00,4.03MB) (all OK)

[iczelia]

Seems to work on Linux, probably something with the C start files supplied by MinGW is wrong.

 % head -c 999 /dev/urandom | base64 > vögel.txt
 % ls -la vögel.txt
-rw-rw-r-- 1 palaiologos palaiologos 1350 Feb 10 10:18 vögel.txt
 % bzip3 -v vögel.txt
 vögel.txt:	1350 -> 1127 bytes, 83.48%, 6.68 bpb

[iczelia]

I tried to reproduce this using my local MSYS2 install using the bzip3 1.5.1 tarball on Windows 10. However, the errors that I am getting are quite magical:

• MSYS2 MINGW32: https://pastebin.com/L6SLAf1F; the error is error adding symbols: archive has no index; run ranlib to add one, but using ranlib to give the archive file/static library extra chakra or whatever doesn't seem to help it. Doesn't seem to be my fault.
• MSYS2 MINGW64: https://pastebin.com/BNuvcQnz; the error is in the stub generated by libtool (no _spawnv nor _P_WAIT). Doesn't seem to be my fault either.
• MSYS2: https://pastebin.com/afVdMt6L; compiler segfault as far as I reckon, in the code generation phase.
• MSYS2 CLANG64: https://pastebin.com/LWw5MBVQ; same issue as with MINGW64.
• MSYS2 UCRT64: https://pastebin.com/aUBe3Aqz; same issue as with MINGW64.

The build from the releases tab seems to work OK outside of some locale issues when printing to the terminal:

C:\Users\Kamila\Downloads>echo vögel > vögel.txt

C:\Users\Kamila\Downloads>bzip3-x86_64.exe -vf vögel.txt
 v÷gel.txt:     8 -> 33 bytes, 412.50%, 33.00 bpb

C:\Users\Kamila\Downloads>dir vögel*
 Volume in drive C has no label.
 Volume Serial Number is C66C-6DF0

 Directory of C:\Users\Kamila\Downloads

10/02/2025  14:23                 8 vögel.txt
10/02/2025  14:23                33 vögel.txt.bz3
               2 File(s)             41 bytes
               0 Dir(s)  741.135.855.616 bytes free

C:\Users\Kamila\Downloads>

I will investigate this at some point. But I think that the specific issue that you are observing is specific to W7 SP1.

[therube]

Win7, SP1. en-US, bzip3-x86_64.exe & no issues with:

bzip3-x86_64.exe -e vögel.TXT

bzip3-x86_64.exe -tv vögel.TXT.bz3
v÷gel.TXT.bz3: OK, 285 -> 443 bytes, 64.33%, 5.15 bpb

(Not that I know about such things, but I'd think locale, code pages, something in that arena.

@sergeevabc, can you, DIR vögel.txt ?
DIR may see *.txt but it might not see, ö - on your end.)

[iczelia]

@therube
Thanks for your feedback, this is also what I am observing myself right now. I wish that the situation with UTF-8 in MinGW and others was clearer for legacy Windows. That said, it makes sense that they are not officially supported anymore.
I have a feeling like OP's locale is not en-US which causes this behaviour.

[sergeevabc]

can you DIR vögel.txt ?

$ chcp
Active code page: 866

$ dir vögel.txt
Directory of C:\Test     
2025-02-22  03:58                 7 vögel.txt     
               1 File(s)              7 bytes

$ type vögel.txt
hello

$ xxhsum vögel.txt
13d0d0c8bac410af  vögel.txt

$ b3sum vögel.txt
2b5974a9767a33204c2d1583bb4b1ba70f24e2970cbc522073df001272525954  vögel.txt

$ gpg --verbose --batch --passphrase 123 -c vögel.txt
gpg: enabled compatibility flags:
gpg: using cipher AES256.CFB
gpg: writing to 'vögel.txt.gpg'

$ pwsh -c "${c:vögel.txt} = ${c:vögel.txt} -replace 'hello','goodbye'"

$ bat vögel.txt
───────┬─────────────────────────────────
       │ File: vögel.txt                 
───────┼─────────────────────────────────
   1   │ goodbye
───────┴─────────────────────────────────

$ ruplacer goodbye hello vögel.txt --go
vögel.txt:1 - goodbye
vögel.txt:1 + hello
Performed 1 replacement on 1 matching file

$ python -c "list(map(print, open('vögel.txt')))"
hello

$ rg -g vögel.txt .
vögel.txt
1:hello  

$ brename -p vögel -r pájaros
Renaming paths...                 
  [DONE] vögel.txt -> pájaros.txt 
1 path(s) renamed in 0.037 seconds

$ f2 -f pájaros -r vögel
SUCCESS  Renamed 'pájaros.txt' to 'vögel.txt'

$ croc vögel.txt
Did you mean to send 'vögel.txt'? (Y/n) 

[iczelia]

Interesting. Where does xxhsum come from? Are all of these also built using MinGW?

[therube]

(Again, I know nothing of this stuff, but...)

I changed to various code pages.
Default 437, 866, 1252, 65001 ...
All worked on my end.

I find it real odd that all other programs do work. Would not have expected that.
(As do dir, type, xxshsum, b3sum, 7-zip, arj... on my end.)

I take it that it is not only the ö character that you have issues with.
That a file name like, "ñ á é í ó ú.txt" would present the same issues on your end?

If you set chcp to say, 65001, does bzip3 still have the same issue?

xxhsum is, https://github.com/Cyan4973/xxHash.
& this looks to be a MinGW of that, https://packages.msys2.org/packages/mingw-w64-x86_64-xxhash.

And just a couple texts for perusal (might or might not have any bearing).
https://stackoverflow.com/questions/388490/how-can-i-use-unicode-characters-on-the-windows-command-line
https://superuser.com/questions/476740/error-while-zipping-files-with-unicode-characters-in-names-with-win7s-send-to

[iczelia]

No patches to xxhash on MinGW's end. That said, XXH includes a lot of platform-specific code for Windows support and particularly Unicode.

[sergeevabc]

Dear @kspalaiologos, please check the following threads:

• Does not accept unicode file path Cyan4973/xxHash#293
  how we made xxhash Unicode-aware even on Windows XP
• zstd fails to process some filenames on Windows [we need a hero] facebook/zstd#3882
  zpaqfranz's author shares a custom Unicode function

@therube
If you set chcp to say, 65001, does bzip3 still have the same issue?

Yes, alas. But I believe the solution is somewhere out there. ^^^

[sergeevabc]

A little amateurish benchmark for the New Year tree. :)