2026-07-31 10:39 @216.73.217.152
CrawledBy Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
Enhanced by LLM: deepseek-v4-flash / taotoken.net / www.chedong.com - original format
recode — This recoding library converts files between various coded character sets and surface encodings. When this cannot be achieved exactly, it may get rid of the offending characters or fall back on approximations. The library recognises or produces more than 300 different character sets and is able to convert files between almost any pair. Most RFC 1345 character sets, and all 'libiconv' character sets, are supported. The 'recode' program is a handy front-end to the library.
| Use Case | Command | Description |
|---|---|---|
| 🔍 Filter Mac to Latin-1 | recode mac | Convert Macintosh file (stdin) to Latin-1 (stdout) |
| 🔍 Filter IBM-PC to Latin-1 | recode pc | Convert IBM-PC/CR-LF to Latin-1 |
| 🔍 Remove Quoted-Printable | recode /qp | Strip QP encoding from input |
| 🔍 Reverse: Latin-1 to Mac | recode ..mac | Convert Latin-1 to Macintosh/CR |
| 🔍 Reverse: Latin-1 to IBM-PC | recode ..pc | Convert Latin-1 to IBM-PC/CR-LF |
| 🔍 Force recoding despite errors | recode -f BEFORE..AFTER | Proceed even if irreversible |
| 📋 List charsets and surfaces | recode -l | Show all available charsets/aliases |
| 📋 List with grep filter | recode -l | grep -i greek | Find Greek-related charsets |
| 📋 Show charset table (decimal) | recode -ld l1 | Display Latin-1 in decimal |
| 📋 Show charset table (full) | recode -lf l1 | Display Latin-1 with comments |
| 🔎 Guess unknown charset | recode -k 130:233 | Identify charset from known mappings |
| 🔄 Overwrite file in place | recode BEFORE..AFTER FILE | Recode FILE over itself |
| 🔄 Filter with shell redirection | recode BEFORE..AFTER <in >out | Recode stdin to stdout |
| 📝 Recode only diacritics | recode -d l1..tex | Limit to non-English letters (LaTeX) |
| 📝 Recode strings/comments only | recode -Spo pc/..u8 | Only recode msgstr in .po files |
| 🔬 Verbose step listing | recode -v BEFORE..AFTER | Show intermediate steps and quality |
| 🧪 Strict mode (lossy) | recode -s BEFORE..AFTER | Drop unmapped characters |
When you just want to use 'recode' quickly:
recode mac reads Macintosh file from stdin, outputs Latin-1 to stdout. Equivalent to any of: recode mac.., recode mac..l1, recode Macintosh..ISO_8859-1recode pc (default surface CR-LF), or recode pc/ (no surface conversion)recode /qprecode ..mac (Latin-1 to Mac), recode ..pc (Latin-1 to IBM-PC), recode ../qp (add QP)-f, e.g. recode -f macUse recode -l to list all charsets/surfaces, recode --help for option summary.
Charset — A particular association between computer codes (small integers starting at 0) and a repertoire of intended characters. Roughly a table giving meaning to each allowable code value.
Surface — A surface transformation of a charset stream. Any kind of (usually reversible) mapping associating physical bits to a stream of characters. Examples: end-of-line encoding, Base64, gzip compression. The "trivial" surface puts characters into fixed-width 8-bit chunks.
Purpose — Convert files between various charsets and surfaces. When exact conversion is impossible, the program may drop offending characters or fall back on approximations. The library recognises around 175 charsets under 500 names, and handles a dozen surfaces.
Recoding is possible between many charsets, described by RFC 1345 tables or available in the 'iconv' library. The 'recode' library also handles specialised charsets:
Exceptions — Cannot recode from 'flat', 'count-characters' or 'dump-with-names'; cannot recode from/to 'data', 'tree' or ':libiconv:'. Surfaces and charsets live in disjoint recoding spaces.
Surfaces are tricks used to fit a charset into practical constraints (7-bit medium, variable-length codes, special newline treatment, etc.). A "pure charset" is a charset free of any surface. When a specialised surface is practical to consider as a charset, this is preferred.
Send suggestions, documentation errors and bug reports to recode-bugs AT iro.ca or directly to pinard AT iro.ca, François Pinard. Provide good, solid, verifiable references for tables when contributing new charsets or surfaces.
General format:
recode [OPTION]... [CHARSET | REQUEST [FILE]... ]For simple cases, the request looks like:
BEFORE..AFTEREach FILE is read assuming charset BEFORE, recoded over itself to charset AFTER. If no FILE, acts as a Unix filter (stdin to stdout).
Example — prepare a distribution from Latin-1 to MSDOS:
mkdir package
cp -p Makefile *.[ch] package
recode Latin-1..MSDOS package/*
zoo ah package.zoo package/*
rm -rf packageExample — filter without altering original:
recode -d l1..tex <datum.txt >datum.texCharset names may contain any character besides comma, forward slash, or two periods in a row. The complete syntax allows intermediate charsets:
BEFORE..INTERIM1..INTERIM2..AFTERSub-requests may be separated by commas:
BEFORE..INTERIM1,INTERIM1..INTERIM2,INTERIM2..AFTERSurfaces are introduced with slashes:
PURE-CHARSET/SURFACE1/SURFACE2...Some charsets have implied surfaces. For example, 'pc' has 'CR-LF' as its usual surface. The request pc/..l1 defeats automatic removal of surfaces.
Charset names, surface names, or aliases may be abbreviated to any unambiguous prefix. 'recode' ignores all characters besides letters and digits when recognising names, and is case-insensitive.
If a charset name is omitted, the value of the DEFAULT_CHARSET environment variable is used (falling back to the current locale's encoding).
--version — Print version number and exit--help — Print help page and exit-C / --copyright — Print copyright and copying conditions-h[LANGUAGE/][NAME] / --header=[LANGUAGE/][NAME] — Generate a source file (C or Perl) with a recoding table-k PAIRS / --known=PAIRS — Guess unknown charset from known character mappings-l[FORMAT] / --list[=FORMAT] — List charsets (one per line with aliases), or show detailed table for a specific charset-T / --find-subsets — Find subsets among UCS-2 based charsetsFORMAT values for -l: decimal (default), octal, hexadecimal, full.
-c / --colons — Use colon ':' instead of double-quote '"' for diaeresis in Texte charset-g / --graphics — Approximate IBM-PC box-drawing characters with ASCII (implies -f)-t / --touch — Update file timestamps to recoding time (default preserves timestamps)-v / --verbose — Print intermediate charset list, recoding quality, and per-file progress-x CHARSET / --ignore=CHARSET — Disable recoding paths through a specific charset-f / --force — Run irreversible recodings to completion, don't exit non-zero for irreversibility-q / --quiet / --silent — Inhibit warning messages about irreversible recodings-s / --strict — Drop characters not explicitly mapped (lossy). Without -f, strict mode causes failure on unmapped charactersReversibility notes:
-s is used\^\i{} vs \^{\i}) may differWithout -s, 'recode' fills mappings with invented correspondences, preferring small permutation cycles. This may produce funny characters but aids reversibility. Use -s to avoid this.
--sequence=memory — Use in-memory buffers for intermediate results-i / --sequence=files — Use intermediate files (default for overwrite mode)-p / --sequence=pipe — Fork processes interconnected with pipes (default for filter mode)-d / --diacritics — Limit conversion to non-ASCII (HTML) or non-English (LaTeX) characters-S[LANGUAGE] / --source[=LANGUAGE] — Recode only comments and strings in C or PO filesRecode the whole buffer from IBM-PC to current charset:
C-x h C-u M-| recode ibmpc RETBind keys to recode region from Easy French to Latin-1:
(global-set-key "\C-cT" 'recode-texte)
(defun recode-texte (flag)
(interactive "P")
(shell-command-on-region
(region-beginning) (region-end)
(concat "recode " (if flag "..txte" "txte")) t)
(exchange-point-and-mark))When 'recode' doesn't provide satisfying results, common causes: incorrect call, doubts about correct results, or mangled input files. To debug:
-v to see the recoding stepsrecode -v UTF-8..ISO_10646-UCS-2 <nicepage >temporary-f to force output despite invalid input-dThe 'recode' program is an application of the recoding library. To use the library in C programs:
#include <recode.h>Link with -lrecode.
The library has four levels: outer, request, task, and charset functions.
Prepare the library for use. Example (minimal program that does nothing useful):
#include <stdbool.h>
#include <recode.h>
const char *program_name;
int
main (int argc, char *const *argv)
{
program_name = argv[0];
RECODE_OUTER outer = recode_new_outer (true);
recode_delete_outer (outer);
exit (0);
}Functions:
RECODE_OUTER recode_new_outer (AUTO_ABORT) — Initialise the library. If AUTO_ABORT is true, the library issues diagnostics and aborts on errors.bool recode_delete_outer (OUTER) — Reclaim memory allocated by recode_new_outerThe program_name variable is used by the library when it diagnoses and aborts.
Cover most recoding needs. Example — filter ibmpc to latin1:
#include <stdio.h>
#include <stdbool.h>
#include <recode.h>
const char *program_name;
int
main (int argc, char *const *argv)
{
program_name = argv[0];
RECODE_OUTER outer = recode_new_outer (true);
RECODE_REQUEST request = recode_new_request (outer);
bool success;
recode_scan_request (request, "ibmpc..latin1");
success = recode_file_to_file (request, stdin, stdout);
recode_delete_request (request);
recode_delete_outer (outer);
exit (success ? 0 : 1);
}Functions:
RECODE_REQUEST recode_new_request (OUTER) — Initialise a request variablebool recode_delete_request (REQUEST) — Un-initialise a requestbool recode_scan_request (REQUEST, "STRING") — Parse a recoding request stringrecode_string (REQUEST, STRING) — Recode a string, returns freshly allocated result or NULLchar *recode_string_to_buffer (REQUEST, INPUT_STRING, &OUTPUT_BUFFER, &OUTPUT_LENGTH, &OUTPUT_ALLOCATED) — Recode string to bufferbool recode_string_to_file (REQUEST, INPUT_FILE, OUTPUT_FILE) — Recode string to filebool recode_buffer_to_buffer (REQUEST, INPUT_BUFFER, INPUT_LENGTH, &OUTPUT_BUFFER, &OUTPUT_LENGTH, &OUTPUT_ALLOCATED) — Buffer to bufferbool recode_buffer_to_file (REQUEST, INPUT_BUFFER, INPUT_LENGTH, OUTPUT_FILE) — Buffer to filebool recode_file_to_buffer (REQUEST, INPUT_FILE, &OUTPUT_BUFFER, &OUTPUT_LENGTH, &OUTPUT_ALLOCATED) — File to bufferbool recode_file_to_file (REQUEST, INPUT_FILE, OUTPUT_FILE) — File to fileFields of struct recode_request (accessible via recodext.h):
verbose_flag — Echo steps to stderr (default false)diaeresis_char — Diaeresis character for 'texte' charset ('"' or ':')make_header_flag — Produce source table instead of recodingdiacritics_only — For HTML/LaTeX, only recode diacriticsascii_graphics — Approximate IBM-PC box-drawing with ASCIIAllow more explicit control over input/output. Example — filter ibmpc to latin1 at task level:
#include <stdio.h>
#include <stdbool.h>
#include <recodext.h>
const char *program_name;
int
main (int argc, char *const *argv)
{
program_name = argv[0];
RECODE_OUTER outer = recode_new_outer (false);
RECODE_REQUEST request = recode_new_request (outer);
RECODE_TASK task;
bool success;
recode_scan_request (request, "ibmpc..latin1");
task = recode_new_task (request);
task->input.file = "";
task->output.file = "";
success = recode_perform_task (task);
recode_delete_task (task);
recode_delete_request (request);
recode_delete_outer (outer);
exit (success ? 0 : 1);
}Functions:
RECODE_TASK recode_new_task (REQUEST) — Initialise a taskbool recode_delete_task (TASK) — Un-initialise a taskrecode_perform_task (TASK) — Execute the recodingKey fields of struct task_request (via recodext.h):
request — Current recoding requestinput.name / input.file — Input file name or FILE pointerinput.buffer / input.cursor / input.limit — In-memory input bufferoutput.name / output.file — Output file name or FILE pointeroutput.buffer / output.cursor / output.limit — In-memory output bufferstrategy — Sequencing strategy (RECODE_STRATEGY_UNDECIDED, RECODE_SEQUENCE_IN_MEMORY, RECODE_SEQUENCE_WITH_FILES, RECODE_SEQUENCE_WITH_PIPE)byte_order_mark — Expect/produce BOM for UCS-2/UTF-16 (default true)fail_level — Error level at which to report failureabort_level — Error level at which to interrupt processingerror_so_far — Maximum error level met so farInternal functions made external:
RECODE_CHARSET find_charset (NAME, CLEANING-TYPE);
bool list_all_charsets (CHARSET);
bool list_concise_charset (CHARSET, LIST-FORMAT);
bool list_full_charset (CHARSET);Error levels (increasing severity):
RECODE_NO_ERROR — No errorRECODE_NOT_CANONICAL — Alternative coding used (non-canonical but reversible in meaning)RECODE_AMBIGUOUS_OUTPUT — Reverse recoding would produce ambiguous outputRECODE_UNTRANSLATABLE — Character has no representation in output charsetRECODE_INVALID_INPUT — Input does not comply with declared codingRECODE_SYSTEM_ERROR — Underlying system error (currently unused)RECODE_USER_ERROR — Programmer/user requested something invalid (currently unused)RECODE_INTERNAL_ERROR — Internal library bug (currently unused)RECODE_MAXIMUM_ERROR — Sentinel valueSet fail_level and abort_level in the task structure to control behaviour.
ISO 10646 defines a universal character set (UCS) based on wide characters, offering possibilities for 2^31 characters. In 'recode', surfaces of UCS are presented as genuine charsets:
ISO-10646-UCS-2, UCS-2, BMP, rune, u2. Begins with byte order mark 0xFEFF.ISO-10646-UCS-4, UCS, UCS-4, ISO_10646, 10646, u4.UNICODE-1-1-UTF-7, UTF-7, TF-7, u7.UTF-8, UTF-2, UTF-FSS, FSS_UTF, TF-8, u8.UTF-16, Unicode, TF-16, u6.The 'recode' library incorporates most code and tables from the portable 'iconv' library by Bruno Haible. The :libiconv: charset is a conceptual pivot within the iconv part. Using -x: or --ignore=:libiconv: disables iconv paths.
Supported charsets include: US-ASCII, UTF-8, UCS-2, UCS-2BE, UCS-2LE, UCS-4, UCS-4BE, UCS-4LE, UTF-16, UTF-16BE, UTF-16LE, UTF-7, ISO-8859-1 through -16, KOI8-R, KOI8-U, KOI8-RU, Windows CP1250-1258, DOS CP850/CP866, MacRoman/MacCentralEurope/MacCroatian/MacIceland/MacRomania/MacCyrillic/MacUkraine/MacGreek/MacTurkish/MacHebrew/MacArabic/MacThai, HP-ROMAN8, NEXTSTEP, ARMSCII-8, Georgian-Academy/PS, MuleLao-1, CP1133, TIS-620, CP874, VISCII, TCVN, and many CJK charsets (JIS, GB, KSC, EUC, SJIS, BIG5, etc.).
Full alias lists are provided in the original documentation for each charset.
An important part of the tabular charset knowledge comes from RFC 1345 ("Character Mnemonics & Character Sets", K. Simonsen, June 1992). The 'recode' library implements most of RFC 1345, with these exceptions:
dk-us and us-dk (overload character positions)ANSI_X3.110-1983, ISO_6937-2-add, T.101-G2, T.61-8bit, iso-ir-90, videotex-supplGB_2312-80, JIS_C6226-1978, JIS_C6226-1983, JIS_X0212-1990, KS_C_5601-1987isoir91 as NATS-DANO (not JIS_C6229-1984-a) and isoir92 as NATS-DANO-ADD (not JIS_C6229-1984-b)The RFC1345 charset (aliases: mnemonic, 1345) uses ISO 646 mnemonics with ampersand prefixes. The original documentation lists over 200 charsets with their aliases and sources.
ANSI_X3.4-1968, aliases: ANSI_X3.4-1986, ASCII, IBM367, ISO646-US, ISO_646.irv:1991, US-ASCII, cp367, iso-ir-6, us. Shortest: us. Tables: recode -lf us (full), -ld (decimal), -lo (octal), -lh (hex).CP819, IBM819, ISO-8859-1, ISO_8859-1, iso-ir-100, l1, Latin-1. Shortest: l1. Tables: recode -lf l1 etc.BS.EBCDIC (dd conv=ebcdic compatible), EBCDIC-CCC (Concurrent Computer Corp), EBCDIC-IBM (dd conv=ibm compatible, with corrections). RFC 1345 brings 15+21 EBCDIC charsets.dos, MSDOS, pc. Implied surface: CR-LF. Shortest: pc.QNX.NOS.ap. Implied surface: CR. RFC 1345 brings macintosh and macintosh_ce (aliases: mac, macce) with CR surface.XML-standalone (h0), HTML_1.1 (h1), HTML_2.0 (h2, RFC1866), HTML-i18n (RFC2070), HTML_3.2 (h3), HTML_4.0 (h4/h).ltex. Converts to/from LaTeX macro calls. Use -d to limit to non-English letters.texi, ti. Partially implemented.TCVN, VISCII, VPS, VIQR, VNI. CP1129 and CP1258 not yet available.AFRFUL-102-BPI_OCIL (Bambara, Ewondo, Fulfude), AFRLIN-104-BPI_OCIL (Lingala, Sango, Wolof), AFRL1-101-BPI_OCIL (transliterated Latin-1).KEYBCS2 (Kamenicky), CORK (T1), KOI-8_CS2. Also via RFC 1345: CP1251, CSN_369103, ECMA-cyrillic, IBM880, INIS-cyrillic, ISO-8859-5, KOI-7, KOI-8, KOI8-R, KOI8-RU, KOI8-U.txte. 7-bit code, diacritics noted with apostrophe, backtick, caret, double quote, comma.21-Permutation (swap bytes, alias swabytes), 4321-Permutation (reverse quadruples)CR (Macintosh, value 13), CR-LF (Microsoft, alias cl, also discards Ctrl-Z on removal)Base64 (aliases: b64, 64), Quoted-Printable (aliases: quote-printable, QP)Octal-1 (o1/o), Octal-2 (o2), Octal-4 (o4), Decimal-1 (d1/d), Decimal-2 (d2), Decimal-4 (d4), Hexadecimal-1 (x1/x), Hexadecimal-2 (x2), Hexadecimal-4 (x4)test7 (128 bytes 0-127), test8 (256 bytes 0-255), test15 (64509 double bytes, excluding surrogate/risky UCS-2), test16 (65536 double bytes)The 'recode' driver constructs a table of all conversion routines ("single steps") as nodes in a directed graph. A cost is attributed to each arc. Given start and goal charsets, 'recode' computes the most economical route. Optimisation merges simple subsequences, and "double steps" (via UCS-2) are dynamically merged into efficient reversible single steps. Statistics: average steps after optimisation is ~1.8 (reversible) or ~1.3 (strict mode).
Create a .c file (or .l Flex file for multi-character recognition) with two conversion functions (from an old charset to the new one, and from the new charset to an old one). The initialisation function module_CHARSET calls declare_step. Predefined functions: file_one_to_one (for 1-to-1 tables), file_one_to_many (for 1-to-many). Edit Makefile.am to add to C_STEPS or L_STEPS.
Similar to adding a charset, but transformations are from/to the special charsets data or tree. declare_step recognises these and installs the charset as a surface.
Shared library — All tables are precompiled into binaries and made into a shared library, avoiding external data files and disk accesses. Initialisation overhead is acceptable.
Central charset — UCS-2 is almost always used as a trampoline between charsets, but is optimised out into single-step permutation tables. The library is not forcefully restrained to UCS-2.
Why not 'iconv' — The iconv buffer-to-buffer model is hard to synchronise across long recoding sequences. The stream-based approach is simpler and more efficient. Iconv can be emulated if needed.
Alphabetical list of concepts: abbreviated names, adding new charsets/surfaces, African charsets, aliases, ambiguous output, ASCII table, average number of recoding steps, bool data type, box-drawing characters, bug reports, byte order mark, caret ASCII code, CDC charsets, chaining, character entities, charset level functions, codepages, combining characters, commutativity of surfaces, contributing, conversions (unavailable), counting characters, CR surface, CR-LF surface, Cyrillic charsets, debugging surfaces, default charset, description of individual characters, diacritics, diaeresis, disable map filling, double step, dumping characters, Easy French, EBCDIC, end of line format, endianness, error handling, file sequencing, file time stamps, filter operation, force recoding, French description, guessing charsets, Haible Bruno, HTML, IBM codepages, iconv, identifying subsets, ignore charsets, implied surfaces, impossible conversions, interface with iconv, intermediate charsets, internal functions, invalid input, invocation synopsis, irreversible recoding, ISO 10646, LaTeX, Latin charsets, libiconv, listing charsets, Macintosh, map filling, MIME, MS-DOS, MULE, multiplexed charsets, new charsets/surfaces, NeXT, non-canonical input, NOS 6/12 code, numeric character references, outer level functions, partial conversion, permutations, pipe sequencing, pseudo-charsets, pure charset, quality of recoding, recoding library, recoding path rejection, recoding steps statistics, request level functions, reversibility, RFC 1345, RFC 2045, sequencing, SGML, shared library, silent operation, single step, source file generation, strict operation, structural surfaces, subsets, super-charsets, suppressing diagnostics, surface, task level functions, TeX, Texinfo, threshold for error reporting, time stamps, trivial surface, tutorial, UCS, Unicode, unknown charsets, untranslatable input, valid characters in charset names, verbose operation, Vietnamese, Web, XML.
--colons / -c — Recoding (line 11)--copyright / -C — Listings (line 19)--diacritics / -d — Mixed (line 20)--find-subsets / -T — Listings (line 198)--force / -f — Reversibility (line 10)--graphics / -g — Recoding (line 16)--header / -h — Listings (line 26)--help — Listings (line 14)--ignore / -x — Recoding (line 60)--known / -k — Listings (line 52)--list / -l — Listings (line 141)--quiet / -q / --silent — Reversibility (line 37)--sequence — Sequencing (line 22)--source / -S — Mixed (line 39)--strict / -s — Reversibility (line 49)--touch / -t — Recoding (line 26)--verbose / -v — Recoding (line 35)--version — Listings (line 10)Alphabetical index of important functions, data structures, and variables in the recoding library: abort_level, ascii_graphics, byte_order_mark, declare_step, DEFAULT_CHARSET, diacritics_only, diaeresis_char, error_so_far, fail_level, file_one_to_many, file_one_to_one, find_charset, LANG, LANGUAGE, list_all_charsets, list_concise_charset, list_full_charset, make_header_flag, RECODE_AMBIGUOUS_OUTPUT, recode_buffer_to_buffer, recode_buffer_to_file, recode_delete_outer, recode_delete_request, recode_delete_task, recode_file_to_buffer, recode_file_to_file, recode_filter_close, recode_filter_open, recode_format_table, RECODE_INTERNAL_ERROR, RECODE_INVALID_INPUT, RECODE_MAXIMUM_ERROR, recode_new_outer, recode_new_request, recode_new_task, RECODE_NOT_CANONICAL, RECODE_NO_ERROR, RECODE_OUTER, recode_perform_task, RECODE_REQUEST, recode_request, recode_scan_request, RECODE_SEQUENCE_IN_MEMORY, RECODE_SEQUENCE_WITH_FILES, RECODE_SEQUENCE_WITH_PIPE, RECODE_STRATEGY_UNDECIDED, recode_string, recode_string_to_buffer, recode_string_to_file, RECODE_SYSTEM_ERROR, RECODE_TASK, RECODE_UNTRANSLATABLE, RECODE_USER_ERROR, strategy, task_request, verbose_flag.
This is an alphabetical list of all the charsets and surfaces supported by 'recode', and their aliases. The full index covers over 600 entries including all standard aliases, EBCDIC variants, IBM code pages, ISO standards, RFC-defined names, and surface names. Key entries include: ASCII, UTF-8, ISO-8859-*, CP125*, IBM*, EBCDIC, MacRoman, KOI8-R, Base64, Quoted-Printable, and many more. Refer to the original documentation for the complete list.
Generated by phpman v4.9.26-1-g511901d · Markdown · JSON · MCP Author: Che Dong Under GNU General Public License
2026-07-31 10:39 @216.73.217.152
CrawledBy Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
Enhanced by LLM: deepseek-v4-flash / taotoken.net / www.chedong.com - original format