2026-05-21 22:09 @216.73.216.105 CrawledBy Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
NAME
Unicode::Stringprep - Preparation of Internationalized Strings
(RFC 3454)
SYNOPSIS
use Unicode::Stringprep;
use Unicode::Stringprep::Mapping;
use Unicode::Stringprep::Prohibited;
my $prepper = Unicode::Stringprep->new(
3.2,
[ { 32 => '<SPACE>'}, ],
'KC',
[ @Unicode::Stringprep::Prohibited::C12, @Unicode::Stringprep::Prohibited::C22,
@Unicode::Stringprep::Prohibited::C3, @Unicode::Stringprep::Prohibited::C4,
@Unicode::Stringprep::Prohibited::C5, @Unicode::Stringprep::Prohibited::C6,
@Unicode::Stringprep::Prohibited::C7, @Unicode::Stringprep::Prohibited::C8,
@Unicode::Stringprep::Prohibited::C9 ],
1, 0 );
$output = $prepper->($input)
DESCRIPTION
This module implements the *stringprep* framework for preparing Unicode
text strings in order to increase the likelihood that string input and
string comparison work in ways that make sense for typical users
throughout the world. The *stringprep* protocol is useful for protocol
identifier values, company and personal names, internationalized domain
names, and other text strings.
The *stringprep* framework does not specify how protocols should prepare
text strings. Protocols must create profiles of stringprep in order to
fully specify the processing options.
FUNCTIONS
This module provides a single function, "new", that creates a perl
function implementing a *stringprep* profile.
This module exports nothing.
new($unicode_version, $mapping_tables, $unicode_normalization,
$prohibited_tables, $bidi_check, $unassigned_check)
Creates a "bless"ed function reference that implements a stringprep
profile.
This function takes the following parameters:
$unicode_version
The Unicode version specified by the stringprep profile.
Currently, this parameter must be 3.2 (numeric).
$mapping_tables
The mapping tables used for stringprep.
The parameter may be a reference to a hash or an array, or
"undef". A hash must map Unicode codepoints (as integers, e. g.
0x0020 for U+0020) to replacement strings (as perl strings). An
array may contain pairs of Unicode codepoints and replacement
strings as well as references to nested hashes and arrays.
Unicode::Stringprep::Mapping provides the tables from RFC 3454,
Appendix B.
For further information on the mapping step, see RFC 3454,
section 3.
$unicode_normalization
The Unicode normalization to be used.
Currently, "undef"/'' (no normalization) and 'KC' (compatibility
composed) are specified for *stringprep*.
For further information on the normalization step, see RFC 3454,
section 4.
Normalization form KC will also enable checks for some problem
sequences for which the normalization can't be implemented in an
interoperable way.
For more information, see "CAVEATS" below.
$prohibited_tables
The list of prohibited output characters for stringprep.
The parameter may be a reference to an array, or "undef". The
array contains pairs of codepoints, which define the start and
end of a Unicode character range (as integers). The end
character may be "undef", specifying a single-character range.
The array may also contain references to nested arrays.
Unicode::Stringprep::Prohibited provides the tables from
RFC 3454, Appendix C.
For further information on the prohibition checking step, see
RFC 3454, section 5.
$bidi_check
Whether to employ checks for confusing bidirectional text. A
boolean value.
For further information on the bidi checking step, see RFC 3454,
section 6.
$unassigned_check
Whether to check for and prohibit unassigned characters. A
boolean value.
The check must be used when creating *stored* strings. It should
not be used for *query* strings, increasing the chance that
newly assigned characters work as expected.
For further information on *stored* and *query* strings, see
RFC 3454, section 7.
The function returned can be called with a single parameter, the
string to be prepared, and returns the prepared string. It will die
if the input string cannot be successfully prepared because it would
contain invalid output (so use "eval" if necessary).
For performance reasons, it is strongly recommended to call the
"new" function as few times as possible, i. e. exactly once per
*stringprep* profile. It might also be better not to use this module
directly but to use (or write) a module implementing a profile, such
as Authen::SASL::SASLprep.
IMPLEMENTING PROFILES
You can easily implement a *stringprep* profile without subclassing:
package ACME::ExamplePrep;
use Unicode::Stringprep;
use Unicode::Stringprep::Mapping;
use Unicode::Stringprep::Prohibited;
*exampleprep = Unicode::Stringprep->new(
3.2,
[ \@Unicode::Stringprep::Mapping::B1, ],
'',
[ \@Unicode::Stringprep::Prohibited::C12,
\@Unicode::Stringprep::Prohibited::C22, ],
1,
);
This binds "ACME::ExamplePrep::exampleprep" to the function created by
"Unicode::Stringprep->new".
Usually, it is not necessary to subclass this module. Sublassing this
module is not recommended.
DATA TABLES
The following modules contain the data tables from RFC 3454. These
modules are automatically loaded when loading "Unicode::Stringprep".
* Unicode::Stringprep::Unassigned
@Unicode::Stringprep::Unassigned::A1 # Appendix A.1
* Unicode::Stringprep::Mapping
@Unicode::Stringprep::Mapping::B1 # Appendix B.1
@Unicode::Stringprep::Mapping::B2 # Appendix B.2
@Unicode::Stringprep::Mapping::B2 # Appendix B.3
* Unicode::Stringprep::Prohibited
@Unicode::Stringprep::Prohibited::C11 # Appendix C.1.1
@Unicode::Stringprep::Prohibited::C12 # Appendix C.1.2
@Unicode::Stringprep::Prohibited::C21 # Appendix C.2.1
@Unicode::Stringprep::Prohibited::C22 # Appendix C.2.2
@Unicode::Stringprep::Prohibited::C3 # Appendix C.3
@Unicode::Stringprep::Prohibited::C4 # Appendix C.4
@Unicode::Stringprep::Prohibited::C5 # Appendix C.5
@Unicode::Stringprep::Prohibited::C6 # Appendix C.6
@Unicode::Stringprep::Prohibited::C7 # Appendix C.7
@Unicode::Stringprep::Prohibited::C8 # Appendix C.8
@Unicode::Stringprep::Prohibited::C9 # Appendix C.9
* Unicode::Stringprep::BiDi
@Unicode::Stringprep::BiDi::D1 # Appendix D.1
@Unicode::Stringprep::BiDi::D2 # Appendix D.2
CAVEATS
In Unicode 3.2 to 4.0.1, the specification of UAX #15: Unicode
Normalization Forms for forms NFC and NFKC is not logically
self-consistent. This has been fixed in Corrigendum #5
(<http://unicode.org/versions/corrigendum5.html>).
Unfortunately, this yields two ways to implement NFC and NFKC in Unicode
3.2, on which the Stringprep standard is based: one based on a literal
interpretation of the original specification and one based on the
corrected specification. The output of these implementations differs for
a small class of strings, all of which can't appear in meaningful text.
See UAX #15, section 19
<http://unicode.org/reports/tr15/#Stability_Prior_to_Unicode41> for
details.
This module will check for these strings and, if normalization is done,
prohibit them in output as it is not possible to interoperate under
these circumstandes.
Please note that due to this, the *normalization* step may cause the
preparation to fail. That is, the preparation function may die even if
there are no prohibited characters and no checks for bidi sequences and
unassigned characters, which may be surprising.
AUTHOR
Claus Färber <CFAERBER AT cpan.org>
LICENSE
Copyright 2007-2009 Claus Färber.
This library is free software; you can redistribute it and/or modify it
under the same terms as Perl itself.
SEE ALSO
Unicode::Normalize, RFC 3454 (<http://www.ietf.org/rfc/rfc3454.txt>)
Generated by phpMan Author: Che Dong On Apache Under GNU General Public License - MarkDown Format
2026-05-21 22:09 @216.73.216.105 CrawledBy Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)