perldoc > Unicode::UCD

man perldoc info pydoc3 ri search

📖 NAME

Unicode::UCD - Unicode character database

🚀 Quick Reference

Use Case	Command	Description
🔍 Get character info	charinfo($codepoint)	Returns hash reference with name, category, combining, bidi, case mappings, etc.
🎯 Get specific property	charprop($codepoint, $property)	Returns value of any Unicode property for a code point.
📋 Get all properties	charprops_all($codepoint)	Returns hash of all property values for a code point.
🔠 Case folding	casefold($codepoint)	Returns locale-independent case folding mapping (full, simple, Turkic).
🔡 Case specifications	casespec($codepoint)	Returns potentially multi-char case mappings (lower, title, upper) with conditions.
📦 Get block	charblock($codepoint) or charblock('BlockName')	Returns block name or range set for a block.
✍️ Get script	charscript($codepoint) or charscript('ScriptName')	Returns script name or range set.
🗂️ List all blocks	charblocks()	Returns hash of block names to code point ranges.
🗂️ List all scripts	charscripts()	Returns hash of script names to code point ranges.
🔢 Test range membership	charinrange($range_set, $codepoint)	Returns true if code point is in a range set.
📊 Get general categories	general_categories()	Returns hash of short→long category names.
↔️ Get bidi types	bidi_types()	Returns hash of short→long bidi type names.
🚫 Composition exclusion (discouraged)	compexcl($codepoint)	Returns true if code point should not be produced by composition normalization.
🏷️ Named sequences	namedseq('Name')	Returns string of code points for a named sequence, or hash of all.
🔢 Numeric value	num($string)	Returns numeric value of a Unicode string, or undef. Works with digits and fractions.
🏷️ Property aliases	prop_aliases($name)	Returns list of synonyms for a property name.
📋 Property values	prop_values($property)	Returns list of legal values for a property (if restricted).
🏷️ Property value aliases	prop_value_aliases($property, $value)	Returns list of synonyms for a property value.
📊 Inversion list	prop_invlist($property)	Returns inversion list for binary property or property=value.
🗺️ Inversion map	prop_invmap($property)	Returns complete mapping as two parallel arrays, format, and default.
🔍 Search inversion list	search_invlist(\@invlist, $codepoint)	Returns index of range containing code point.
📅 Unicode version	Unicode::UCD::UnicodeVersion	Returns version string of the Unicode Character Database.

🔧 SYNOPSIS

use Unicode::UCD 'charinfo';
my $charinfo   = charinfo($codepoint);

use Unicode::UCD 'charprop';
my $value  = charprop($codepoint, $property);

use Unicode::UCD 'charprops_all';
my $all_values_hash_ref = charprops_all($codepoint);

use Unicode::UCD 'casefold';
my $casefold = casefold($codepoint);

use Unicode::UCD 'all_casefolds';
my $all_casefolds_ref = all_casefolds();

use Unicode::UCD 'casespec';
my $casespec = casespec($codepoint);

use Unicode::UCD 'charblock';
my $charblock  = charblock($codepoint);

use Unicode::UCD 'charscript';
my $charscript = charscript($codepoint);

use Unicode::UCD 'charblocks';
my $charblocks = charblocks();

use Unicode::UCD 'charscripts';
my $charscripts = charscripts();

use Unicode::UCD qw(charscript charinrange);
my $range = charscript($script);
print "looks like $script\n" if charinrange($range, $codepoint);

use Unicode::UCD qw(general_categories bidi_types);
my $categories = general_categories();
my $types = bidi_types();

use Unicode::UCD 'prop_aliases';
my @space_names = prop_aliases("space");

use Unicode::UCD 'prop_value_aliases';
my @gc_punct_names = prop_value_aliases("Gc", "Punct");

use Unicode::UCD 'prop_values';
my @all_EA_short_names = prop_values("East_Asian_Width");

use Unicode::UCD 'prop_invlist';
my @puncts = prop_invlist("gc=punctuation");

use Unicode::UCD 'prop_invmap';
my ($list_ref, $map_ref, $format, $missing)
                                  = prop_invmap("General Category");

use Unicode::UCD 'search_invlist';
my $index = search_invlist(\@invlist, $code_point);

# The following function should be used only internally in
# implementations of the Unicode Normalization Algorithm, and there
# are better choices than it.
use Unicode::UCD 'compexcl';
my $compexcl = compexcl($codepoint);

use Unicode::UCD 'namedseq';
my $namedseq = namedseq($named_sequence_name);

my $unicode_version = Unicode::UCD::UnicodeVersion();

my $convert_to_numeric =
          Unicode::UCD::num("\N{RUMI DIGIT ONE}\N{RUMI DIGIT TWO}");

📝 DESCRIPTION

The Unicode::UCD module offers a series of functions that provide a simple interface to the Unicode Character Database.

🔢 Code Point Argument

Some of the functions are called with a code point argument, which is either a decimal or a hexadecimal scalar designating a code point in the platform's native character set (extended to Unicode), or a string containing "U+" followed by hexadecimals designating a Unicode code point. A leading 0 will force a hexadecimal interpretation, as will a hexadecimal digit that isn't a decimal digit.

Examples:

223     # Decimal 223 in native character set
0223    # Hexadecimal 223, native (= 547 decimal)
0xDF    # Hexadecimal DF, native (= 223 decimal)
'0xDF'  # String form of hexadecimal (= 223 decimal)
'U+DF'  # Hexadecimal DF, in Unicode's character set
                          (= LATIN SMALL LETTER SHARP S)

Note that the largest code point in Unicode is U+10FFFF.

📄 charinfo()

use Unicode::UCD 'charinfo';
my $charinfo = charinfo(0x41);

This returns information about the input "code point argument" as a reference to a hash of fields as defined by the Unicode standard. If the "code point argument" is not assigned or is a non-character, undef is returned. Fields that aren't applicable are empty.

Fields:

• code – input native code point in hexadecimal, at least 4 hexdigits
• name – name in upper case; empty for Surrogate/Private Use, or <control> for others
• category – short general category (e.g. Lu, Nd)
• combining – combining class number
• bidi – bidirectional type (e.g. L, R)
• decomposition – decomposition string (codes separated by spaces, possibly with type)
• decimal – integer numeric value if decimal digit
• digit – integer numeric value if other digit
• numeric – numeric value (integer or fraction like "1/4")
• mirrored – "Y" or "N"
• unicode10 – Unicode 1.0 name if different
• comment – always empty as of Unicode 6.0
• upper – uppercase mapping (single char) or empty
• lower – lowercase mapping (single char) or empty
• title – titlecase mapping (single char) or empty
• block – block name (old-style)
• script – script name

Note: You cannot do (de)composition and casing solely from these fields; use casespec() and Unicode::Normalize.

🎯 charprop()

use Unicode::UCD 'charprop';
print charprop(0x41, "Gc"), "\n";
print charprop(0x61, "General_Category"), "\n";

Returns the value of the Unicode property (any synonym) for the code point. The return value is a scalar (string or number). For properties with synonyms, returns the longest, most descriptive form. More "cooked" than charinfo.

Special notes:

• Block – returns new-style block name
• Decomposition_Mapping – may be intermediate; does not include type
• Name_Alias – multiple synonyms joined by comma
• Numeric_Value – fractions converted to float
• Script_Extensions – multiple scripts joined by comma
• For Perl extensions not expressible in compound form (e.g., "Any"), returns undef.

📋 charprops_all()

use Unicode::UCD 'charprops_all';
my $all_properties_of_A_hash_ref = charprops_all("U+41");

Returns a reference to a hash with all distinct Unicode properties (no Perl extensions) as keys and their values as returned by charprop(). Expensive in time and memory.

📦 charblock()

use Unicode::UCD 'charblock';
my $charblock = charblock(0x41);
my $charblock = charblock(1234);
my $charblock = charblock(0x263a);
my $charblock = charblock("U+263a");
my $range     = charblock('Armenian');

With a code point argument, returns the block name (old-style). If the code point is unassigned, returns the block it would belong to. If the argument is a block name, returns a range set (array of [start, end, block_name]) for that block. Returns undef if unknown block.

✍️ charscript()

use Unicode::UCD 'charscript';
my $charscript = charscript(0x41);
my $charscript = charscript(1234);
my $charscript = charscript("U+263a");
my $range      = charscript('Thai');

With a code point argument, returns the script name. If unassigned or early Unicode, returns "Unknown". If argument is a script name, returns a range set. Returns undef if unknown script. Note: Use Script_Extensions (charprop) for improved results.

🗂️ charblocks()

use Unicode::UCD 'charblocks';
my $charblocks = charblocks();

Returns a reference to a hash with old-style block names as keys and code point ranges (as from charblock()) as values. Alternative: prop_invmap("block") or prop_values("Block").

🗂️ charscripts()

use Unicode::UCD 'charscripts';
my $charscripts = charscripts();

Returns a reference to a hash with script names as keys and code point ranges as values. Alternative: prop_invmap("scx") for Script_Extensions.

🔢 charinrange()

use Unicode::UCD qw(charscript charinrange);
$range = charscript('Hiragana');
print "looks like hiragana\n" if charinrange($range, $codepoint);

Tests whether a code point is in a range set as returned by charblock() or charscript().

📊 general_categories()

use Unicode::UCD 'general_categories';
my $categories = general_categories();

Returns a reversible hash of short general category names (e.g., "Lu") to long names (e.g., "UppercaseLetter"). Alternative: prop_values("Gc") and prop_value_aliases().

↔️ bidi_types()

use Unicode::UCD 'bidi_types';
my $categories = bidi_types();

Returns a reversible hash of short bidi type names (e.g., "L") to long names (e.g., "Left-to-Right"). Alternative: prop_values("Bidi_Class") and prop_value_aliases().

🚫 compexcl() (Discouraged)

use Unicode::UCD 'compexcl';
my $compexcl = compexcl(0x09dc);

Returns true if the code point should not be produced by composition normalization. Better alternatives: chr(0x09dc) =~ /\p{Comp_Ex}/ or /\p{Full_Composition_Exclusion}/. Returns false otherwise. Undef if Unicode version is too early.

🔠 casefold()

use Unicode::UCD 'casefold';
my $casefold = casefold(0xDF);
if (defined $casefold) {
    my @full_fold_hex = split / /, $casefold->{'full'};
    my $full_fold_string = join "", map {chr(hex($_))} @full_fold_hex;
    my @turkic_fold_hex = split / /, ($casefold->{'turkic'} ne "")
                                    ? $casefold->{'turkic'} : $casefold->{'full'};
    my $turkic_fold_string = join "", map {chr(hex($_))} @turkic_fold_hex;
}
if (defined $casefold && $casefold->{'simple'} ne "") {
    my $simple_fold_hex = $casefold->{'simple'};
    my $simple_fold_string = chr(hex($simple_fold_hex));
}

Returns (almost) locale-independent case folding. Returns undef if no folding. Hash fields:

• code – code point in hex
• full – one or more codes (spaces) for full fold
• simple – single code folding if available, else empty
• mapping – same as simple if non-empty, else full (backward compat)
• status – "C" (common), "S" (distinct), "F" (only full), "I" (special for dotted I)
• turkic – special folding for Turkic languages

For best results use full field. The core function fc() is faster for strings.

📋 all_casefolds()

use Unicode::UCD 'all_casefolds';
my $all_folds_ref = all_casefolds();
foreach my $char_with_casefold (sort { $a  $b } keys %$all_folds_ref) {
    printf "%04X:", $char_with_casefold;
    my $casefold = $all_folds_ref->{$char_with_casefold};
    # ... get folds as in casefold() example
}

Returns a reference to a hash of all characters with case folds (keys are decimal ordinals, values are hash references identical to casefold() output).

🔡 casespec()

use Unicode::UCD 'casespec';
my $casespec = casespec(0xFB00);

Returns potentially locale-dependent case mappings (lower, title, upper) that may be multi-character. Returns undef if all mappings are single-char and locale-independent. Hash fields:

• code – code point in hex
• lower, title, upper – codes separated by spaces
• condition – list of locales and/or contexts, or undef if always valid

If there are locale-specific rules, additional keys like "lt", "tr", "az" contain hash references with the same structure.

🏷️ namedseq()

use Unicode::UCD 'namedseq';
my $namedseq = namedseq("KATAKANA LETTER AINU P");
my @namedseq = namedseq("KATAKANA LETTER AINU P");
my %namedseq = namedseq();

Scalar context: returns string of code points for named sequence, or undef. List context: returns list of ordinals. No arguments in list context: returns hash of all named sequences (names to strings). Works only on officially approved named sequences. Note: chnames::string_vianame() is more general.

🔢 num()

use Unicode::UCD 'num';
my $val = num("123");
my $ FRACTION 1/4}");
my $val = num("12a", \$valid_length);  # $valid_length contains 2

Returns numeric value of a Unicode string, or undef if not completely valid. With optional second parameter (reference to scalar), sets it to length of valid initial substring. For single characters, returns Unicode numeric value. For multi-character strings, all characters must be decimal digits from the same script and same form. Handles fractions like "1/4". Sub- and superscripts not recognized as numbers.

🏷️ prop_aliases()

use Unicode::UCD 'prop_aliases';
my ($short_name, $full_name, @other_names) = prop_aliases("space");
my $same_full_name = prop_aliases("Space");     # Scalar context
my ($same_short_name) = prop_aliases("Space");  # gets 0th element

Returns long name (scalar context) or list of all synonyms (short name first, then long, then others). Input is loosely matched (ignores case, hyphens, underscores). Returns undef for unknown names. Does not recognize "Is_" prefix for standard properties but does for Perl extensions. Discouraged forms are accepted as input but not returned; accepted alternatives are given.

📋 prop_values()

use Unicode::UCD 'prop_values';
print "AHex values are: ", join(", ", prop_values("AHex")), "\n";

Returns list of legal values for a property if it has a restricted set (e.g., binary properties, General Category). Returns undef if not restricted. Input property name is loosely matched. For Block property, returns new-style block names.

🏷️ prop_value_aliases()

use Unicode::UCD 'prop_value_aliases';
my ($short_name, $full_name, @other_names) = prop_value_aliases("Gc", "Punct");
my $same_full_name = prop_value_aliases("Gc", "P");   # Scalar context

Returns long name (scalar) or list of all synonyms for a property value. Input parameters are loosely matched. Returns undef if unknown. For properties without synonyms, returns the input value (possibly normalized). For Block property, returns new-style block names.

📊 prop_invlist()

use feature 'say';
use Unicode::UCD 'prop_invlist';
say join ", ", prop_invlist("Any");

Returns an inversion list (list of code points) for a binary property or property=value pair. Unknown input returns undef in scalar, empty list in list. Inversion list: even indices start ranges with property, odd indices start ranges without. The list may include code points above 0x10FFFF; you can trim by adding/poping 0x110000. Does not know user-defined or Perl internal properties.

Example with property=value:

say join ", ", prop_invlist("Script_Extensions=Shavian");
# prints: 66640, 66688
say join ", ", prop_invlist("ASCII_Hex_Digit=No");
# prints: 0, 48, 58, 65, 71, 97, 103

🗺️ prop_invmap()

use Unicode::UCD 'prop_invmap';
my ($list_ref, $map_ref, $format, $default) = prop_invmap("General Category");

Returns complete mapping for a property as two parallel arrays: code point range beginnings and corresponding values. Also returns format string and default value. Format can be:

• "s" – all map elements are simple scalars (e.g., Block)
• "sl" – some map elements are lists of scalars (e.g., Script_Extensions, Name_Alias)
• "a" – all map elements are integers that need adjustment by offset (e.g., Simple_Uppercase_Mapping)
• "al" – mix of "a" and lists (e.g., Uppercase_Mapping)
• "ae" – like "a" but some elements are empty string (e.g., Perl_Decimal_Digit)
• "ale" – mix of "al" and empty string (e.g., NFKC_Casefold)
• "ar" – all map elements are rational numbers or "NaN" (e.g., Nv)
• "n" – Name property with special entries (CJK, hangul, control)
• "ad" – Decomposition_Mapping like "al" but with <hangul syllable> entries

The fourth element $default is used with "a" formats; it's the value for most code points. For properties that need adjustment, only scalar integer entries should be adjusted. Use search_invlist() for binary search. Does not know user-defined properties.

Example for Block (Unicode 6.0):

Index  @blocks_ranges  @blocks_maps
0        0x0000      Basic Latin
1        0x0080      Latin-1 Supplement
...
242      0x110000    No_Block

Getting every available name (instead of reading Name.pl directly):

my (%name, %cp, %cps, $n);
# All codepoints
foreach my $cat (qw( Name Name_Alias )) {
    my ($codepoints, $names, $format, $default) = prop_invmap($cat);
    foreach my $i (0 .. @$codepoints - 2) {
        my ($cp, $n) = ($codepoints->[$i], $names->[$i]);
        foreach my $name (ref $n ? @$n : $n) {
            $name{$cp} //= $name;
            $cp{$name} //= $cp;
        }
    }
}
# Named sequences
{   my %ns = namedseq();
    foreach my $name (sort { $ns{$a} cmp $ns{$b} } keys %ns) {
        $cp{$name} //= [ map { ord } split "" => $ns{$name} ];
    }
}

🔍 search_invlist()

use Unicode::UCD qw(prop_invmap prop_invlist);
use Unicode::UCD 'search_invlist';

my @invlist = prop_invlist($property_name);
print $code_point, ((search_invlist(\@invlist, $code_point) // -1) % 2)
                    ? " isn't" : " is",
    " in $property_name\n";

my ($blocks_ranges_ref, $blocks_map_ref) = prop_invmap("Block");
my $index = search_invlist($blocks_ranges_ref, $code_point);
print "$code_point is in block ", $blocks_map_ref->[$index], "\n";

Searches an inversion list for a code point argument. Returns the index of the range containing the code point (satisfies list[i]

📖 NAME 🚀 Quick Reference 🔧 SYNOPSIS 📝 DESCRIPTION