Skip to content

Latest commit

 

History

11 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 

Repository files navigation

multibyte_identifiers — UTF-8 variable names for GNU Bash

An experimental patch for GNU Bash 5.3 that lets shell variable names contain non-ASCII characters, behind a system-wide switch only root can operate.

Straße=1
GRÖẞE=zwölf
Zähler=5; (( Zähler++ ))
คำ=word
declare -A Übersetzung=([rot]=red)
echo "$Straße ${#GRÖẞE} ${Übersetzung[rot]}"

With the switch off — the default — behaviour is byte-for-byte identical to an unpatched Bash.

Status

Proposed on bug-bash@gnu.org in July 2026 and revised through five rounds in response to review by Robert Elz and Andreas Schwab. Not merged upstream, and possibly never will be; the discussion is public in the bug-bash archive. Treat this as a working prototype, not as a supported feature.

Which standards, and why

The hard question for a feature like this is not whether to allow non-ASCII names but which ones. Above U+007F the shell grammar no longer forces a boundary, so one has to be chosen. This patch takes the one the wider ecosystem has converged on rather than inventing a private repertoire:

UAX #31, Unicode Identifier and Pattern Syntax — the default identifier syntax: a name starts with any XID_Start character or _, and continues with XID_Continue characters. This is what C23, C++ (P1949R7), Rust and Python are all built on, so "what counts as an identifier character" is a question with one published answer instead of one answer per language. It also comes with a stability guarantee: a string that is a valid default identifier today stays one in every later Unicode version, so the table can only ever grow.

UAX #15, Normalization — names must be in Normalization Form C, enforced with the NFC quick check. This is the part that matters most in a shell: it means identifier equality stays byte equality. Precomposed ä and a + U+0308 render identically in every font, and without this rule they would be two different variables. grep, diff and human readers stay reliable.

UAX #31 §2.3, default-ignorable characters — variation selectors, ZWJ/ZWNJ, the Hangul fillers, CGJ and soft hyphen are excluded even where the UCD leaves them in XID_Continue. These characters ask to be rendered invisibly, so allowing them would let two distinct names look identical on screen — the "Trojan Source" class of problem (CVE-2021-42574).

UTS #39 §3.1, general security profile — only characters with Identifier_Status=Allowed are kept, which removes obsolete, rarely used and confusable characters. U+00B5 MICRO SIGN, confusable with U+03BC GREEK SMALL LETTER MU, is the representative case.

That last profile needed an adjustment, and it is worth stating plainly because it was a bug in earlier revisions. UTS #39 is derived with NFKC in mind and restricts everything NFKC would rewrite, while this patch requires NFC. Applying it wholesale removed ordinary letters of scripts in daily use — without U+0E33 THAI CHARACTER SARA AM, frequent Thai words such as คำ, ทำ and น้ำ could not be used as names at all. The generator therefore re-admits a restricted character when it is NFC-stable, has a compatibility mapping to two or more characters, and every character of that mapping lies in its own block and is itself allowed. That currently yields eleven characters (Thai, Lao, Armenian, Arabic and Kana), listed by name in the generator with a self-check that fails if a UCD update changes the set. Fullwidth forms, mathematical alphanumerics, the Latin digraphs and the Arabic and Armenian presentation forms stay excluded.

Deliberately not adopted: confusable detection (UTS #39 §4). It compares identifiers pairwise within a compilation unit, and a shell has none — names arrive from the environment, read, eval and sourced files at arbitrary times. Rust warns about confusables per crate; that has no shell equivalent.

Security model

The consent file /etc/bash_multibyte_identifiers is the system-wide switch. Every newly started shell adopts its state, so once an administrator has turned the feature on, every user simply uses it — no shopt needed.

Operation Command Who
Enable system-wide sudo bash-mbident enable root only
Disable system-wide sudo bash-mbident disable root only
Query system state bash-mbident status anyone (exit 0/1)
Use non-ASCII names — automatic when enabled anyone
Query shell state shopt multibyte_identifiers anyone
Override in one shell shopt -s / -u multibyte_identifiers root only

A UTF-8 locale is also required; switching the locale away from UTF-8 at run time quietly suspends the extended syntax. bash-mbident enable warns that such scripts are not portable and asks for confirmation, using the locale's yesexpr so that j works in a German locale.

Building

Requires build-essential and bison. The patch applies to the official GNU tarball, not to a checkout of this repository.

wget https://ftp.gnu.org/gnu/bash/bash-5.3.tar.gz
tar xf bash-5.3.tar.gz
cd bash-5.3

git apply -v ../patches/v5-0001-Add-multibyte-UTF-8-shell-variable-names-behind-a.patch
grep MBID_.*VERSION mbidtbl.h     # must print two lines; if not, stop here

./configure
make -j"$(nproc)"

git apply works outside a git repository and sets the executable bit on the new scripts; with patch -p1 you have to run chmod +x support/bash-mbident.sh afterwards. If git apply reports an error, do not continue — configure and make will happily build an unpatched shell.

Then enable it and try it out:

sudo install -m 755 support/bash-mbident.sh /usr/local/bin/bash-mbident
sudo bash-mbident enable
./bash                            # a *new* shell reads the switch at startup
Straße=1; echo "$Straße"

sudo make install is not recommended for a trial: it puts a second Bash into /usr/local/bin, which may then take precedence over the system shell.

Verifying a build

printf '\xce\xbc=7\n'             | ./bash ; echo "MU (U+03BC):        $?"  # 0
printf '\xc2\xb5=7\n'             | ./bash ; echo "MICRO (U+00B5):     $?"  # 127
printf '\xe0\xb8\x84\xe0\xb8\xb3=7\n' | ./bash ; echo "THAI kham:      $?"  # 0
printf '\xef\xbc\xa1=7\n'         | ./bash ; echo "FULLWIDTH A:        $?"  # 127
printf '\xef\xbb\xbb=7\n'         | ./bash ; echo "ARABIC LIGATURE:    $?"  # 127

Exit 0 means the name was accepted, 127 means it was rejected. The full regression suite runs with make tests; tests/run-mbident skips itself cleanly where the prerequisites are missing.

Regenerating the Unicode table

mbidtbl.h is generated but kept in the tree, so that a normal build needs no Python — the same reasoning by which Bash ships y.tab.c so a build needs no bison. After a Unicode update:

make regen-mbidtbl

The recipe runs support/mkmbidtbl.py against support/unicode-identifier-status.txt (the UTS #39 data file, vendored so the build needs no network). The XID and normalization data come from the UCD shipped with the Python interpreter used to run the generator, so a newer Python picks up a newer UCD. Both source versions are recorded in the generated header; they need not match, since intersecting with a newer security profile can only shrink what the UCD in use permits.

Known limitations

These are properties of the design, documented rather than hidden:

  • Non-portable by construction. Scripts using such names will not run on other shells or on an unpatched Bash. That is what the warning at enable time and the experimental label are for.
  • Combining marks cannot start a name. UAX #31 excludes Mn from XID_Start, so the Thai vowel sign SARA I cannot begin an identifier where Latin i can. This asymmetry comes from Unicode, but adopting Unicode's profile was a choice made here.
  • Some scripts allow several identically rendering spellings. NFC removes the canonical-equivalence class only. Thai, for instance, permits either order of NIKHAHIT and a tone mark, and no normalization form unifies those — so a word can be encoded more than one way and each is a distinct variable.
  • Confusables remain, as they do in ASCII with l, I and 1.
  • Right-to-left names reorder neighbouring characters when displayed, as any RTL text does in a terminal.
  • The option takes effect at parse time. bash -c 'shopt -s multibyte_identifiers && Straße=1' fails because the whole list is parsed before shopt runs — the same behaviour extglob has. In a script, or with ;, it works. zsh documents the identical caveat for its own option.

Repository layout

patches/     the patch, one self-contained file against bash-5.3

Licence

The patch modifies GNU Bash and is offered under the GNU General Public License, version 3 or later, the same terms as Bash itself.

About

Bash 5.3 with the possibility to use Unicode in Identifiers

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors