{ "mode": "info", "parameter": "recode", "section": "", "url": "https://www.chedong.com/phpMan.php/info/recode/json", "generated": "2026-11-10T07:35:17Z", "sections": { "File: recode.info, Node: Top, Next: Tutorial, Prev: (dir), Up: (dir)": { "content": "", "subsections": [] }, "'recode'": { "content": "", "subsections": [] }, "This recoding library converts files between various coded character": { "content": "sets and surface encodings. When this cannot be achieved exactly, it\nmay get rid of the offending characters or fall back on approximations.", "subsections": [] }, "The library recognises or produces more than 300 different character": { "content": "sets and is able to convert files between almost any pair. Most\nRFC 1345 character sets, and all 'libiconv' character sets, are\nsupported. The 'recode' program is a handy front-end to the library.\n\nThe current 'recode' release is 3.6.\n\n* Menu:\n\n* Tutorial:: Quick Tutorial\n* Introduction:: Terminology and purpose\n* Invoking recode:: How to use this program\n* Library:: A recoding library\n* Universal:: The universal charset\n* libiconv:: The 'iconv' library\n* Tabular:: Tabular sources (RFC 1345)\n* ASCII misc:: ASCII and some derivatives\n* IBM and MS:: Some IBM or Microsoft charsets\n* CDC:: Charsets for CDC machines\n* Micros:: Other micro-computer charsets\n* Miscellaneous:: Various other charsets\n* Surfaces:: All about surfaces\n* Internals:: Internal aspects\n* Concept Index:: Concept Index\n* Option Index:: Option Index\n* Library Index:: Library Index\n* Charset and Surface Index:: Charset and Surface Index\n\n-- The Detailed Node Listing --\n", "subsections": [] }, "Terminology and purpose": { "content": "* Charset overview:: Overview of charsets\n* Surface overview:: Overview of surfaces\n* Contributing:: Contributions and bug reports\n", "subsections": [] }, "How to use this program": { "content": "* Synopsis:: Synopsis of 'recode' call\n* Requests:: The REQUEST parameter\n* Listings:: Asking for various lists\n* Recoding:: Controlling how files are recoded\n* Reversibility:: Reversibility issues\n* Sequencing:: Selecting sequencing methods\n* Mixed:: Using mixed charset input\n* Emacs:: Using 'recode' within Emacs\n* Debugging:: Debugging considerations\n\nA recoding library\n\n* Outer level:: Outer level functions\n* Request level:: Request level functions\n* Task level:: Task level functions\n* Charset level:: Charset level functions\n* Errors:: Handling errors\n", "subsections": [] }, "The universal charset": { "content": "* UCS-2:: Universal Character Set, 2 bytes\n* UCS-4:: Universal Character Set, 4 bytes\n* UTF-7:: Universal Transformation Format, 7 bits\n* UTF-8:: Universal Transformation Format, 8 bits\n* UTF-16:: Universal Transformation Format, 16 bits\n* count-characters:: Frequency count of characters\n* dump-with-names:: Fully interpreted UCS dump\n\nASCII and some derivatives\n\n* ASCII:: Usual ASCII\n* ISO 8859:: ASCII extended by Latin Alphabets\n* ASCII-BS:: ASCII 7-bits, 'BS' to overstrike\n* flat:: ASCII without diacritics nor underline\n", "subsections": [] }, "Some IBM or Microsoft charsets": { "content": "* EBCDIC:: EBCDIC codes\n* IBM-PC:: IBM's PC code\n* Icon-QNX:: Unisys' Icon code\n", "subsections": [] }, "Charsets for CDC machines": { "content": "* Display Code:: Control Data's Display Code\n* CDC-NOS:: ASCII 6/12 from NOS\n* Bang-Bang:: ASCII \"bang bang\"\n", "subsections": [] }, "Other micro-computer charsets": { "content": "* Apple-Mac:: Apple's Macintosh code\n* AtariST:: Atari ST code\n", "subsections": [] }, "Various other charsets": { "content": "* HTML:: World Wide Web representations\n* LaTeX:: LaTeX macro calls\n* Texinfo:: GNU project documentation files\n* Vietnamese:: Vietnamese charsets\n* African:: African charsets\n* Others:: Cyrillic and other charsets\n* Texte:: Easy French conventions\n* Mule:: Mule as a multiplexed charset\n", "subsections": [] }, "All about surfaces": { "content": "* Permutations:: Permuting groups of bytes\n* End lines:: Representation for end of lines\n* MIME:: MIME contents encodings\n* Dump:: Interpreted character dumps\n* Test:: Artificial data for testing\n", "subsections": [] }, "Internal aspects": { "content": "* Main flow:: Overall organisation\n* New charsets:: Adding new charsets\n* New surfaces:: Adding new surfaces\n* Design:: Comments on the library design\n\n", "subsections": [] }, "File: recode.info, Node: Tutorial, Next: Introduction, Prev: Top, Up: Top": { "content": "", "subsections": [] }, "1 Quick Tutorial": { "content": "", "subsections": [] }, "So, really, you just are in a hurry to use 'recode', and do not feel": { "content": "like studying this manual? Even reading this paragraph slows you down?", "subsections": [] }, "We might have a problem, as you will have to do some guess work, and": { "content": "might not become very proficient unless you have a very solid\nintuition....\n\nLet me use here, as a quick tutorial, an actual reply of mine to a\n'recode' user, who writes:\n\nMy situation is this--I occasionally get email with special\ncharacters in it. Sometimes this mail is from a user using IBM\nsoftware and sometimes it is a user using Mac software. I myself\nam on a SPARC Solaris machine.\n\nYour situation is similar to mine, except that I often receive\nemail needing recoding, that is, much more than occasionally! The\nusual recodings I do are Mac to Latin-1, IBM page codes to Latin-1,", "subsections": [] }, "Easy-French to Latin-1, remove Quoted-Printable, remove Base64. These": { "content": "are so frequent that I made myself a few two-keystroke Emacs commands to\nfilter the Emacs region. This is very convenient for me. I also resort\nto many other email conversions, yet more rarely than the frequent cases\nabove.\n\nIt seems like this should be doable using 'recode'. However,\nwhen I try something like 'grecode mac macfile.txt' I get nothing\nout--no error, no output, nothing.\n\nPresuming you are using some recent version of 'recode', the command:\n\nrecode mac macfile.txt\n\nis a request for recoding 'macfile.txt' over itself, overwriting the\noriginal, from Macintosh usual character code and Macintosh end of\nlines, to Latin-1 and Unix end of lines. This is overwrite mode. If\nyou want to use 'recode' as a filter, which is probably what you need,\nrather do:\n\nrecode mac\n\nand give your Macintosh file as standard input, you'll get the Latin-1\nfile on standard output. The above command is an abbreviation for any\nof:\n\nrecode mac..\nrecode mac..l1\nrecode mac..Latin-1\nrecode mac/CR..Latin-1/\nrecode Macintosh..ISO8859-1\nrecode Macintosh/CR..ISO8859-1/\n\nThat is, a 'CR' surface, encoding newlines with ASCII , is first\nto be removed (this is a default surface for 'mac'), then the Macintosh\ncharset is converted to Latin-1 and no surface is added to the result\n(there is no default surface for 'l1'). If you want 'mac' code\nconverted, but you know that newlines are already coded the Unix way,\njust do:\n\nrecode mac/\n\nthe slash then overriding the default surface with empty, that is, none.\nHere are other easy recipes:\n\nrecode pc to filter IBM-PC code and CR-LF (default) to Latin-1\nrecode pc/ to filter IBM-PC code to Latin-1\nrecode 850 to filter code page 850 and CR-LF (default) to Latin-1\nrecode 850/ to filter code page 850 to Latin-1\nrecode /qp to remove quoted printable\n\nThe last one is indeed equivalent to any of:\n\nrecode /qp..\nrecode l1/qp..l1/\nrecode ISO8859-1/Quoted-Printable..ISO8859-1/\n\nHere are some reverse recipes:\n\nrecode ..mac to filter Latin-1 to Macintosh code and CR (default)\nrecode ..mac/ to filter Latin-1 to Macintosh code\nrecode ..pc to filter Latin-1 to IBM-PC code and CR-LF (default)\nrecode ..pc/ to filter Latin-1 to IBM-PC code\nrecode ..850 to filter Latin-1 to code page 850 and CR-LF (default)\nrecode ..850/ to filter Latin-1 to code page 850\nrecode ../qp to force quoted printable\n\nIn all the above calls, replace 'recode' by 'recode -f' if you want\nto proceed despite recoding errors. If you do not use '-f' and there is\nan error, the recoding output will be interrupted after first error in\nfilter mode, or the file will not be replaced by a recoded copy in\noverwrite mode.\n\nYou may use 'recode -l' to get a list of available charsets and\nsurfaces, and 'recode --help' to get a quick summary of options. The\nabove output is meant for those having already read this manual, so let\nme dare a suggestion: why could not you find a few more minutes in your\nschedule to peek further down, right into the following chapters!\n", "subsections": [] }, "File: recode.info, Node: Introduction, Next: Invoking recode, Prev: Tutorial, Up: Top": { "content": "", "subsections": [] }, "2 Terminology and purpose": { "content": "A few terms are used over and over in this manual, our wise reader will\nlearn their meaning right away. Both ISO (International Organization\nfor Standardisation) and IETF (Internet Engineering Task Force) have\ntheir own terminology, this document does not try to stick to either one\nin a strict way, while it does not want to throw more confusion in the\nfield. On the other hand, it would not be efficient using paraphrases\nall the time, so 'recode' coins a few short words, which are explained\nbelow.\n\nA \"charset\", in the context of 'recode', is a particular association\nbetween computer codes on one side, and a repertoire of intended\ncharacters on the other side. Codes are usually taken from a set of\nconsecutive small integers, starting at 0. Some characters have a\ngraphical appearance (glyph) or displayable effect, others have special\nuses like, for example, to control devices or to interact with\nneighbouring codes to specify them more precisely. So, a charset is\nroughly one of those tables, giving a meaning to each of the codes from\nthe set of allowable values. MIME also uses the term charset with\napproximately the same meaning. It does not exactly corresponds to\nwhat ISO calls a \"coded character set\", that is, a set of characters\nwith an encoding for them. An coded character set does not necessarily\nuse all available code positions, while a MIME charset usually tries to\nspecify them all. A MIME charset might be the union of a few disjoint\ncoded character sets.\n\nA \"surface\" is a term used in 'recode' only, and is a short for\nsurface transformation of a charset stream. This is any kind of\nmapping, usually reversible, which associates physical bits in some\nmedium for a stream of characters taken from one or more charsets\n(usually one). A surface is a kind of varnish added over a charset so\nit fits in actual bits and bytes. How end of lines are exactly encoded\nis not really pertinent to the charset, and so, there is surface for end\nof lines. 'Base64' is also a surface, as we may encode any charset in\nit. Other examples would 'DES' enciphering, or 'gzip' compression (even\nif 'recode' does not offer them currently): these are ways to give a\nreal life to theoretical charsets. The \"trivial\" surface consists into\nputting characters into fixed width little chunks of bits, usually eight\nsuch bits per character. But things are not always that simple.\n\nThis 'recode' library, and the program by that name, have the purpose\nof converting files between various charsets and surfaces. When this\ncannot be done in exact ways, as it is often the case, the program may\nget rid of the offending characters or fall back on approximations.", "subsections": [] }, "This library recognises or produces around 175 such charsets under 500": { "content": "names, and handle a dozen surfaces. Since it can convert each charset\nto almost any other one, many thousands of different conversions are\npossible.\n\nThe 'recode' program and library do not usually know how to split and\nsort out textual and non-textual information which may be mixed in a\nsingle input file. For example, there is no surface which currently\naddresses the problem of how lines are blocked into physical records,\nwhen the blocking information is added as binary markers or counters\nwithin files. So, 'recode' should be given textual streams which are\nrather pure.\n\nThis tool pays special attention to superimposition of diacritics for\nsome French representations. This orientation is mostly historical, it\ndoes not impair the usefulness, generality or extensibility of the\nprogram. 'recode' is both a French and English word. For those who pay\nattention to those things, the proper pronunciation is French (that is,\n'racud', with 'a' like in 'above', and 'u' like in 'cut').\n\nThe program 'recode' has been written by Franc,ois Pinard. With time,\nit got to reuse works from other contributors, and notably, those of\nKeld Simonsen and Bruno Haible.\n\n* Menu:\n\n* Charset overview:: Overview of charsets\n* Surface overview:: Overview of surfaces\n* Contributing:: Contributions and bug reports\n", "subsections": [] }, "File: recode.info, Node: Charset overview, Next: Surface overview, Prev: Introduction, Up: Introduction": { "content": "", "subsections": [ { "name": "2.1 Overview of charsets", "content": "" } ] }, "Recoding is currently possible between many charsets, the bulk of which": { "content": "is described by RFC 1345 tables or available in the 'iconv' library.\n*Note Tabular::, and *note libiconv::. The 'recode' library also\nhandles some charsets in some specialised ways. These are:\n\n* 6-bit charsets based on CDC display code: 6/12 code from NOS;\nbang-bang code from Universite' de Montre'al;\n\n* 7-bit ASCII: without any diacritics, or else: using backspace for\noverstriking; Unisys' Icon convention; TeX/LaTeX coding; easy\nFrench conventions for electronic mail;\n\n* 8-bit extensions to ASCII: ISO Latin-1, Atari ST code, IBM's code\nfor the PC, Apple's code for the Macintosh;\n\n* 8-bit non-ASCII codes: three flavours of EBCDIC;\n\n* 16-bit or 31-bit universal characters, and their transfer\nencodings.\n\nThe introduction of RFC 1345 in 'recode' has brought with it a few\ncharsets having the functionality of older ones, but yet being different\nin subtle ways. The effects have not been fully investigated yet, so\nfor now, clashes are avoided, the old and new charsets are kept well\nseparate.\n\nConversion is possible between almost any pair of charsets. Here is\na list of the exceptions. One may not recode from the 'flat',\n'count-characters' or 'dump-with-names' charsets, nor from or to the\n'data', 'tree' or ':libiconv:' charsets. Also, if we except the 'data'\nand 'tree' pseudo-charsets, charsets and surfaces live in disjoint\nrecoding spaces, one cannot really transform a surface into a charset or\nvice-versa, as surfaces are only meant to be applied over charsets, or\nremoved from them.\n", "subsections": [] }, "File: recode.info, Node: Surface overview, Next: Contributing, Prev: Charset overview, Up: Introduction": { "content": "", "subsections": [ { "name": "2.2 Overview of surfaces", "content": "" } ] }, "For various practical considerations, it sometimes happens that the": { "content": "codes making up a text, written in a particular charset, cannot simply\nbe put out in a file one after another without creating problems or\nbreaking other things. Sometimes, 8-bit codes cannot be written on a\n7-bit medium, variable length codes need kind of envelopes, newlines\nrequire special treatment, etc. We sometimes have to apply \"surfaces\"\nto a stream of codes, which surfaces are kind of tricks used to fit the\ncharset into those practical constraints. Moreover, similar surfaces or\ntricks may be useful for many unrelated charsets, and many surfaces can\nbe used at once over a single charset.\n\nSo, 'recode' has machinery to describe a combination of a charset\nwith surfaces used over it in a file. We would use the expression \"pure\ncharset\" for referring to a charset free of any surface, that is, the\nconceptual association between integer codes and character intents.\n\nIt is not always clear if some transformation will yield a charset or\na surface, especially for those transformations which are only\nmeaningful over a single charset. The 'recode' library is not overly\npicky as identifying surfaces as such: when it is practical to consider\na specialised surface as if it were a charset, this is preferred, and\ndone.\n", "subsections": [] }, "File: recode.info, Node: Contributing, Prev: Surface overview, Up: Introduction": { "content": "", "subsections": [ { "name": "2.3 Contributions and bug reports", "content": "" } ] }, "Even being the 'recode' author and current maintainer, I am no": { "content": "specialist in charset standards. I only made 'recode' along the years\nto solve my own needs, but felt it was applicable for the needs of\nothers. Some FSF people liked the program structure and suggested to\nmake it more widely available. I often rely on 'recode' users\nsuggestions to decide what is best to be done next.\n\nProperly protecting 'recode' about possible copyright fights is a\npain for me and for contributors, but we cannot avoid addressing the\nissue in the long run. Besides, the Free Software Foundation, which\nmandates the GNU project, is very sensible to this matter. GNU\nstandards suggest that we stay cautious before looking at copyrighted\ncode. The safest and simplest way for me is to gather ideas and\nreprogram them anew, even if this might slow me down considerably. For\ncontributions going beyond a few lines of code here and there, the FSF\ndefinitely requires employer disclaimers and copyright assignments in\nwriting.\n\nWhen you contribute something to 'recode', please explain what it\nis about. Do not take for granted that I know those charsets which are\nfamiliar to you. Once again, I'm no expert, and you have to help me.", "subsections": [] }, "Your explanations could well find their way into this documentation,": { "content": "too. Also, for contributing new charsets or new surfaces, as much as\npossible, please provide good, solid, verifiable references for the\ntables you used(1).\n\nMany users contributed to 'recode' already, I am grateful to them for\ntheir interest and involvement. Some suggestions can be integrated\nquickly while some others have to be delayed, I have to draw a line\nsomewhere when time comes to make a new release, about what would go in\nit and what would go in the next.\n\nPlease send suggestions, documentation errors and bug reports to\n or, if you prefer, directly to\n, Franc,ois Pinard. Do not be afraid to report\ndetails, because this program is the mere aggregation of hundreds of\ndetails.\n\n---------- Footnotes ----------\n\n(1) I'm not prone at accepting a charset you just invented, and which\nnobody uses yet: convince your friends and community first!\n", "subsections": [] }, "File: recode.info, Node: Invoking recode, Next: Library, Prev: Introduction, Up: Top": { "content": "", "subsections": [] }, "3 How to use this program": { "content": "", "subsections": [] }, "With the synopsis of the 'recode' call, we stress the difference between": { "content": "using this program as a file filter, or recoding many files at once.", "subsections": [] }, "The first parameter of any call states the recoding request, and this": { "content": "deserves a section on its own. Options are then presented, but somewhat\ngrouped according to the related functionalities they control.\n\n* Menu:\n\n* Synopsis:: Synopsis of 'recode' call\n* Requests:: The REQUEST parameter\n* Listings:: Asking for various lists\n* Recoding:: Controlling how files are recoded\n* Reversibility:: Reversibility issues\n* Sequencing:: Selecting sequencing methods\n* Mixed:: Using mixed charset input\n* Emacs:: Using 'recode' within Emacs\n* Debugging:: Debugging considerations\n", "subsections": [] }, "File: recode.info, Node: Synopsis, Next: Requests, Prev: Invoking recode, Up: Invoking recode": { "content": "", "subsections": [ { "name": "3.1 Synopsis of 'recode' call", "content": "The general format of the program call is one of:\n\nrecode [OPTION]... [CHARSET | REQUEST [FILE]... ]\n\nSome calls are used only to obtain lists produced by 'recode' itself,\nwithout actually recoding any file. They are recognised through the\nusage of listing options, and these options decide what meaning should\nbe given to an optional CHARSET parameter. *Note Listings::.\n\nIn other calls, the first parameter (REQUEST) always explains which\ntransformations are expected on the files. There are many variations to\nthe aspect of this parameter. We will discuss more complex situations\nlater (*note Requests::), but for many simple cases, this parameter\nmerely looks like this(1):\n\nBEFORE..AFTER\n\nwhere BEFORE and AFTER each gives the name of a charset. Each FILE will\nbe read assuming it is coded with charset BEFORE, it will be recoded\nover itself so to use the charset AFTER. If there is no FILE on the\n'recode' command, the program rather acts as a Unix filter and\ntransforms standard input onto standard output.\n\nThe capability of recoding many files at once is very convenient." } ] }, "For example, one could easily prepare a distribution from Latin-1 to": { "content": "MSDOS, this way:\n\nmkdir package\ncp -p Makefile *.[ch] package\nrecode Latin-1..MSDOS package/*\nzoo ah package.zoo package/*\nrm -rf package\n\n(In this example, the non-mandatory '-p' option to 'cp' is for\npreserving timestamps, and the 'zoo' program is an archiver from Rahul", "subsections": [] }, "Dhesi which once was quite popular.)": { "content": "The filter operation is especially useful when the input files should\nnot be altered. Let us make an example to illustrate this point.", "subsections": [] }, "Suppose that someone has a file named 'datum.txt', which is almost a TeX": { "content": "file, except that diacriticised characters are written using Latin-1.", "subsections": [] }, "To complete the recoding of the diacriticised characters only and": { "content": "produce a file 'datum.tex', without destroying the original, one could\ndo:\n\ncp -p datum.txt datum.tex\nrecode -d l1..tex datum.tex\n\nHowever, using 'recode' as a filter will achieve the same goal more\nneatly:\n\nrecode -d l1..tex datum.tex\n\nThis example also shows that 'l1' could be used instead of 'Latin-1';\ncharset names often have such aliases.\n\n---------- Footnotes ----------\n\n(1) In previous versions or 'recode', a single colon ':' was used\ninstead of the two dots '..' for separating charsets, but this was\ncreating problems because colons are allowed in official charset names.", "subsections": [] }, "The old request syntax is still recognised for compatibility purposes,": { "content": "but is deprecated.\n", "subsections": [] }, "File: recode.info, Node: Requests, Next: Listings, Prev: Synopsis, Up: Invoking recode": { "content": "", "subsections": [ { "name": "3.2 The REQUEST parameter", "content": "" } ] }, "In the case where the REQUEST is merely written as BEFORE..AFTER, then": { "content": "BEFORE and AFTER specify the start charset and the goal charset for the\nrecoding.\n\nFor 'recode', charset names may contain any character, besides a\ncomma, a forward slash, or two periods in a row. But in practice,\ncharset names are currently limited to alphabetic letters (upper or\nlower case), digits, hyphens, underlines, periods, colons or round\nparentheses.\n\nThe complete syntax for a valid REQUEST allows for unusual things,\nwhich might surprise at first. (Do not pay too much attention to these\nfacilities on first reading.) For example, REQUEST may also contain\nintermediate charsets, like in the following example:\n\nBEFORE..INTERIM1..INTERIM2..AFTER\n\nmeaning that 'recode' should internally produce the INTERIM1 charset\nfrom the start charset, then work out of this INTERIM1 charset to\ninternally produce INTERIM2, and from there towards the goal charset.", "subsections": [] }, "In fact, 'recode' internally combines recipes and automatically uses": { "content": "interim charsets, when there is no direct recipe for transforming BEFORE\ninto AFTER. But there might be many ways to do it. When many routes\nare possible, the above \"chaining\" syntax may be used to more precisely\nforce the program towards a particular route, which it might not have\nnaturally selected otherwise. On the other hand, because 'recode' tries\nto choose good routes, chaining is only needed to achieve some rare,\nunusual effects.\n\nMoreover, many such requests (sub-requests, more precisely) may be\nseparated with commas (but no spaces at all), indicating a sequence of\nrecodings, where the output of one has to serve as the input of the\nfollowing one. For example, the two following requests are equivalent:\n\nBEFORE..INTERIM1..INTERIM2..AFTER\nBEFORE..INTERIM1,INTERIM1..INTERIM2,INTERIM2..AFTER\n", "subsections": [] }, "In this example, the charset input for any recoding sub-request is": { "content": "identical to the charset output by the preceding sub-request. But it\ndoes not have to be so in the general case. One might wonder what would\nbe the meaning of declaring the charset input for a recoding sub-request\nof being of different nature than the charset output by a preceding\nsub-request, when recodings are chained in this way. Such a strange\nusage might have a meaning and be useful for the 'recode' expert, but\nthey are quite uncommon in practice.\n\nMore useful is the distinction between the concept of charset, and\nthe concept of surfaces. An encoded charset is represented by:\n\nPURE-CHARSET/SURFACE1/SURFACE2...\n\nusing slashes to introduce surfaces, if any. The order of application\nof surfaces is usually important, they cannot be freely commuted. In\nthe given example, SURFACE1 is first applied over the PURE-CHARSET, then\nSURFACE2 is applied over the result. Given this request:\n\nBEFORE/SURFACE1/SURFACE2..AFTER/SURFACE3\n\nthe 'recode' program will understand that the input files should have\nSURFACE2 removed first (because it was applied last), then SURFACE1\nshould be removed. The next step will be to translate the codes from\ncharset BEFORE to charset AFTER, prior to applying SURFACE3 over the\nresult.\n\nSome charsets have one or more implied surfaces. In this case, the\nimplied surfaces are automatically handled merely by naming the charset,\nwithout any explicit surface to qualify it. Let's take an example to\nillustrate this feature. The request 'pc..l1' will indeed decode MS-DOS\nend of lines prior to converting IBM-PC codes to Latin-1, because 'pc'\nis the name of a charset(1) which has 'CR-LF' for its usual surface.", "subsections": [] }, "The request 'pc/..l1' will not decode end of lines, since the slash": { "content": "introduces surfaces, and even if the surface list is empty, it\neffectively defeats the automatic removal of surfaces for this charset.\nSo, empty surfaces are useful, indeed!\n\nBoth charsets and surfaces may have predefined alternate names, or\naliases. However, and this is rather important to understand, implied\nsurfaces are attached to individual aliases rather than on genuine\ncharsets. Consequently, the official charset name and all of its\naliases do not necessarily share the same implied surfaces. The charset\nand all its aliases may each have its own different set of implied\nsurfaces.\n\nCharset names, surface names, or their aliases may always be\nabbreviated to any unambiguous prefix. Internally in 'recode',\ndisambiguating tables are kept separate for charset names and surface\nnames.\n\nWhile recognising a charset name or a surface name (or aliases\nthereof), 'recode' ignores all characters besides letters and digits, so\nfor example, the hyphens and underlines being part of an official\ncharset name may safely be omitted (no need to un-confuse them!). There\nis also no distinction between upper and lower case for charset or\nsurface names.\n\nOne of the BEFORE or AFTER keywords may be omitted. If the double\ndot separator is omitted too, then the charset is interpreted as the\nBEFORE charset.(2)\n\nWhen a charset name is omitted or left empty, the value of the\n'DEFAULTCHARSET' variable in the environment is used instead. If this\nvariable is not defined, the 'recode' library uses the current locale's\nencoding. On POSIX compliant systems, this depends on the first\nnon-empty value among the environment variables LCALL, LCCTYPE, LANG,\nand can be determined through the command 'locale charmap'.\n\nIf the charset name is omitted but followed by surfaces, the surfaces\nthen qualify the usual or default charset. For example, the request\n'../x' is sufficient for applying an hexadecimal surface to the input\ntext(3).\n\nThe allowable values for BEFORE or AFTER charsets, and various\nsurfaces, are described in the remainder of this document.\n\n---------- Footnotes ----------\n\n(1) More precisely, 'pc' is an alias for the charset 'IBM-PC'.\n\n(2) Both BEFORE and AFTER may be omitted, in which case the double\ndot separator is mandatory. This is not very useful, as the recoding\nreduces to a mere copy in that case.\n\n(3) MS-DOS is one of those systems for which the default charset has\nimplied surfaces, 'CR-LF' here. Such surfaces are automatically removed\nor applied whenever the default charset is read or written, exactly as\nit would go for any other charset. In the example above, on such\nsystems, the hexadecimal surface would then replace the implied\nsurfaces. For adding an hexadecimal surface without removing any, one\nshould write the request as '/../x'.\n", "subsections": [] }, "File: recode.info, Node: Listings, Next: Recoding, Prev: Requests, Up: Invoking recode": { "content": "", "subsections": [ { "name": "3.3 Asking for various lists", "content": "" } ] }, "Many options control listing output generated by 'recode' itself, they": { "content": "are not meant to accompany actual file recodings. These options are:\n\n'--version'\nThe program merely prints its version numbers on standard output,\nand exits without doing anything else.\n\n'--help'\nThe program merely prints a page of help on standard output, and\nexits without doing any recoding.\n\n'-C'\n'--copyright'\nGiven this option, all other parameters and options are ignored.\nThe program prints briefly the copyright and copying conditions.\nSee the file 'COPYING' in the distribution for full statement of\nthe Copyright and copying conditions.\n\n'-h[LANGUAGE/][NAME]'\n'--header[=[LANGUAGE/][NAME]]'\nInstead of recoding files, 'recode' writes a LANGUAGE source file\non standard output and exits. This source is meant to be included\nin a regular program written in the same programming LANGUAGE: its\npurpose is to declare and initialise an array, named NAME, which\nrepresents the requested recoding. The only acceptable values for\nLANGUAGE are 'c' or 'perl', and may may be abbreviated. If\nLANGUAGE is not specified, 'c' is assumed. If NAME is not\nspecified, then it defaults to 'BEFOREAFTER'. Strings BEFORE and\nAFTER are cleaned before being used according to the syntax of\nLANGUAGE.\n\nEven if 'recode' tries its best, this option does not always\nsucceed in producing the requested source table. It will however,\nprovided the recoding can be internally represented by only one\nstep after the optimisation phase, and if this merged step conveys\na one-to-one or a one-to-many explicit table. Also, when\nattempting to produce sources tables, 'recode' relaxes its checking\na tiny bit: it ignores the algorithmic part of some tabular\nrecodings, it also avoids the processing of implied surfaces. But\nthis is all fairly technical. Better try and see!\n\nBeware that other options might affect the produced source tables,\nthese are: '-d', '-g' and, particularly, '-s'.\n\n'-k PAIRS'\n'--known=PAIRS'\nThis particular option is meant to help identifying an unknown\ncharset, using as hints some already identified characters of the\ncharset. Some examples will help introducing the idea.\n\nLet's presume here that 'recode' is run in an ISO-8859-1 locale,\nand that 'DEFAULTCHARSET' is unset in the environment. Suppose\nyou have guessed that code 130 (decimal) of the unknown charset\nrepresents a lower case 'e' with an acute accent. That is to say\nthat this code should map to code 233 (decimal) in the usual\ncharset. By executing:\n\nrecode -k 130:233\n\nyou should obtain a listing similar to:\n\nAtariST atarist\nCWI cphu cwi cwi2\nIBM437 437 cp437 ibm437\nIBM850 850 cp850 ibm850\nIBM851 851 cp851 ibm851\nIBM852 852 cp852 ibm852\nIBM857 857 cp857 ibm857\nIBM860 860 cp860 ibm860\nIBM861 861 cp861 cpis ibm861\nIBM863 863 cp863 ibm863\nIBM865 865 cp865 ibm865\n\nYou can give more than one clue at once, to restrict the list\nfurther. Suppose you have also guessed that code 211 of the\nunknown charset represents an upper case 'E' with diaeresis, that\nis, code 203 in the usual charset. By requesting:\n\nrecode -k 130:233,211:203\n\nyou should obtain:\n\nIBM850 850 cp850 ibm850\nIBM852 852 cp852 ibm852\nIBM857 857 cp857 ibm857\n\nThe usual charset may be overridden by specifying one non-option\nargument. For example, to request the list of charsets for which\ncode 130 maps to code 142 for the Macintosh, you may ask:\n\nrecode -k 130:142 mac\n\nand get:\n\nAtariST atarist\nCWI cphu cwi cwi2\nIBM437 437 cp437 ibm437\nIBM850 850 cp850 ibm850\nIBM851 851 cp851 ibm851\nIBM852 852 cp852 ibm852\nIBM857 857 cp857 ibm857\nIBM860 860 cp860 ibm860\nIBM861 861 cp861 cpis ibm861\nIBM863 863 cp863 ibm863\nIBM865 865 cp865 ibm865\n\nwhich, of course, is identical to the result of the first example,\nsince the code 142 for the Macintosh is a small 'e' with acute.\n\nMore formally, option '-k' lists all possible before charsets for\nthe after charset given as the sole non-option argument to\n'recode', but subject to restrictions given in PAIRS. If there is\nno non-option argument, the after charset is taken to be the\ndefault charset for this 'recode'.\n\nThe restrictions are given as a comma separated list of pairs, each\npair consisting of two numbers separated by a colon. The numbers\nare taken as decimal when the initial digit is between '1' and '9';\n'0x' starts an hexadecimal number, or else '0' starts an octal\nnumber. The first number is a code in any before charset, while\nthe second number is a code in the specified after charset. If\nthe first number would not be transformed into the second number by\nrecoding from some before charset to the after charset, then\nthis before charset is rejected. A before charset is listed\nonly if it is not rejected by any pair. The program will only test\nthose before charsets having a tabular style internal description\n(*note Tabular::), so should be the selected after charset.\n\nThe produced list is in fact a subset of the list produced by the\noption '-l'. As for option '-l', the non-option argument is\ninterpreted as a charset name, possibly abbreviated to any non\nambiguous prefix.\n\n'-l[FORMAT]'\n'--list[=FORMAT]'\nThis option asks for information about all charsets, or about one\nparticular charset. No file will be recoded.\n\nIf there is no non-option arguments, 'recode' ignores the FORMAT\nvalue of the option, it writes a sorted list of charset names on\nstandard output, one per line. When a charset name have aliases or\nsynonyms, they follow the true charset name on its line, sorted\nfrom left to right. Each charset or alias is followed by its\nimplied surfaces, if any. This list is over two hundred lines. It\nis best used with 'grep -i', as in:\n\nrecode -l | grep -i greek\n\nThere might be one non-option argument, in which case it is\ninterpreted as a charset name, possibly abbreviated to any non\nambiguous prefix. This particular usage of the '-l' option is\nobeyed only for charsets having a tabular style internal\ndescription (*note Tabular::). Even if most charsets have this\nproperty, some do not, and the option '-l' cannot be used to detail\nthese particular charsets. For knowing if a particular charset can\nbe listed this way, you should merely try and see if this works.\nThe FORMAT value of the option is a keyword from the following\nlist. Keywords may be abbreviated by dropping suffix letters, and\neven reduced to the first letter only:\n\n'decimal'\nThis format asks for the production on standard output of a\nconcise tabular display of the charset, in which character\ncode values are expressed in decimal.\n\n'octal'\nThis format uses octal instead of decimal in the concise\ntabular display of the charset.\n\n'hexadecimal'\nThis format uses hexadecimal instead of decimal in the concise\ntabular display of the charset.\n\n'full'\nThis format requests an extensive display of the charset on\nstandard output, using one line per character showing its\ndecimal, hexadecimal, octal and 'UCS-2' code values, and also\na descriptive comment which should be the 10646 name for the\ncharacter.\n\nThe descriptive comment is given in English and ASCII, yet if\nthe English description is not available but a French one is,\nthen the French description is given instead, using Latin-1.\nHowever, if the 'LANGUAGE' or 'LANG' environment variable\nbegins with the letters 'fr', then listing preference goes to\nFrench when both descriptions are available.\n\nWhen option '-l' is used together with a CHARSET argument, the\nFORMAT defaults to 'decimal'.\n\n'-T'\n'--find-subsets'\nThis option is a maintainer tool for evaluating the redundancy of\nthose charsets, in 'recode', which are internally represented by an\n'UCS-2' data table. After the listing has been produced, the\nprogram exits without doing any recoding. The output is meant to\nbe sorted, like this: 'recode -T | sort'. The option triggers\n'recode' into comparing all pairs of charsets, seeking those which\nare subsets of others. The concept and results are better\nexplained through a few examples. Consider these three sample\nlines from '-T' output:\n\n[ 0] IBM891 == IBM903\n[ 1] IBM1004 < CP1252\n[ 12] INVARIANT < CSAZ243.4-1985-1\n\nThe first line means that 'IBM891' and 'IBM903' are completely\nidentical as far as 'recode' is concerned, so one is fully\nredundant to the other. The second line says that 'IBM1004' is\nwholly contained within 'CP1252', yet there is a single character\nwhich is in 'CP1252' without being in 'IBM1004'. The third line\nsays that 'INVARIANT' is wholly contained within\n'CSAZ243.4-1985-1', but twelve characters are in\n'CSAZ243.4-1985-1' without being in 'INVARIANT'. The whole output\nmight most probably be reduced and made more significant through a\ntransitivity study.\n", "subsections": [] }, "File: recode.info, Node: Recoding, Next: Reversibility, Prev: Listings, Up: Invoking recode": { "content": "", "subsections": [ { "name": "3.4 Controlling how files are recoded", "content": "" } ] }, "The following options have the purpose of giving the user some fine": { "content": "grain control over the recoding operation themselves.\n\n'-c'\n'--colons'\nWith 'Texte' Easy French conventions, use the column ':' instead of\nthe double-quote '\"' for marking diaeresis. *Note Texte::.\n\n'-g'\n'--graphics'\nThis option is only meaningful while getting out of the 'IBM-PC'\ncharset. In this charset, characters 176 to 223 are used for\nconstructing rulers and boxes, using simple or double horizontal or\nvertical lines. This option forces the automatic selection of\nASCII characters for approximating these rulers and boxes, at cost\nof making the transformation irreversible. Option '-g' implies\n'-f'.\n\n'-t'\n'--touch'\nThe touch option is meaningful only when files are recoded over\nthemselves. Without it, the time-stamps associated with files are\npreserved, to reflect the fact that changing the code of a file\ndoes not really alter its informational contents. When the user\nwants the recoded files to be time-stamped at the recoding time,\nthis option inhibits the automatic protection of the time-stamps.\n\n'-v'\n'--verbose'\nBefore doing any recoding, the program will first print on the\n'stderr' stream the list of all intermediate charsets planned for\nrecoding, starting with the BEFORE charset and ending with the\nAFTER charset. It also prints an indication of the recoding\nquality, as one of the word 'reversible', 'one to one', 'one to\nmany', 'many to one' or 'many to many'.\n\nThis information will appear once or twice. It is shown a second\ntime only when the optimisation and step merging phase succeeds in\nreplacing many single steps by a new one.\n\nThis option also has a second effect. The program will print on\n'stderr' one message per recoded FILE, so as to keep the user\ninformed of the progress of its command.\n\nAn easy way to know beforehand the sequence or quality of a\nrecoding is by using the command such as:\n\nrecode -v BEFORE..AFTER < /dev/null\n\nusing the fact that, in 'recode', an empty input file produces an\nempty output file.\n\n'-x CHARSET'\n'--ignore=CHARSET'\nThis option tells the program to ignore any recoding path through\nthe specified CHARSET, so disabling any single step using this\ncharset as a start or end point. This may be used when the user\nwants to force 'recode' into using an alternate recoding path (yet\nusing chained requests offers a finer control, *note Requests::).\n\nCHARSET may be abbreviated to any unambiguous prefix.\n", "subsections": [] }, "File: recode.info, Node: Reversibility, Next: Sequencing, Prev: Recoding, Up: Invoking recode": { "content": "", "subsections": [ { "name": "3.5 Reversibility issues", "content": "The following options are somewhat related to reversibility issues:\n\n'-f'\n'--force'\nWith this option, irreversible or otherwise erroneous recodings are\nrun to completion, and 'recode' does not exit with a non-zero\nstatus if it would be only because irreversibility matters. *Note\nReversibility::.\n\nWithout this option, 'recode' tries to protect you against recoding\na file irreversibly over itself(1). Whenever an irreversible\nrecoding is met, or any other recoding error, 'recode' produces a\nwarning on standard error. The current input file does not get\nreplaced by its recoded version, and 'recode' then proceeds with\nthe recoding of the next file.\n\nWhen the program is merely used as a filter, standard output will\nhave received a partially recoded copy of standard input, up to the\nfirst error point. After all recodings have been done or\nattempted, and if some recoding has been aborted, 'recode' exits\nwith a non-zero status.\n\nIn releases of 'recode' prior to version 3.5, this option was\nalways selected, so it was rather meaningless. Nevertheless, users\nwere invited to start using '-f' right away in scripts calling\n'recode' whenever convenient, in preparation for the current\nbehaviour.\n\n'-q'\n'--quiet'\n'--silent'\nThis option has the sole purpose of inhibiting warning messages\nabout irreversible recodings, and other such diagnostics. It has\nno other effect, in particular, it does not prevent recodings to\nbe aborted or 'recode' to return a non-zero exit status when\nirreversible recodings are met.\n\nThis option is set automatically for the children processes, when\nrecode splits itself in many collaborating copies. Doing so, the\ndiagnostic is issued only once by the parent. See option '-p'.\n\n'-s'\n'--strict'\nBy using this option, the user requests that 'recode' be very\nstrict while recoding a file, merely losing in the transformation\nany character which is not explicitly mapped from a charset to\nanother. Such a loss is not reversible and so, will bring 'recode'\nto fail, unless the option '-f' is also given as a kind of\ncounter-measure.\n\nUsing '-s' without '-f' might render the 'recode' program very\nsusceptible to the slighest file abnormalities. Despite the fact\nthat it might be irritating to some users, such paranoia is\nsometimes wanted and useful.\n\nEven if 'recode' tries hard to keep the recodings reversible, you\nshould not develop an unconditional confidence in its ability to do so." } ] }, "You ought to keep only reasonable expectations about reverse": { "content": "recodings. In particular, consider:\n\n* Most transformations are fully reversible for all inputs, but lose\nthis property whenever '-s' is specified.\n\n* A few transformations are not meant to be reversible, by design.\n\n* Reversibility sometimes depends on actual file contents and cannot\nbe ascertained beforehand, without reading the file.\n\n* Reversibility is never absolute across successive versions of this\nprogram. Even correcting a small bug in a mapping could induce\nslight discrepancies later.\n\n* Reversibility is easily lost by merging. This is best explained\nthrough an example. If you reversibly recode a file from charset A\nto charset B, then you reversibly recode the result from charset B\nto charset C, you cannot expect to recover the original file by\nmerely recoding from charset C directly to charset A. You will\ninstead have to recode from charset C back to charset B, and only\nthen from charset B to charset A.\n\n* Faulty files create a particular problem. Consider an example,\nrecoding from 'IBM-PC' to 'Latin-1'. End of lines are represented\nas '\\r\\n' in 'IBM-PC' and as '\\n' in 'Latin-1'. There is no way by\nwhich a faulty 'IBM-PC' file containing a '\\n' not preceded by '\\r'\nbe translated into a 'Latin-1' file, and then back.\n\n* There is another difficulty arising from code equivalences. For\nexample, in a 'LaTeX' charset file, the string '\\^\\i{}' could be\nrecoded back and forth through another charset and become '\\^{\\i}'.\nEven if the resulting file is equivalent to the original one, it is\nnot identical.\n\nUnless option '-s' is used, 'recode' automatically tries to fill\nmappings with invented correspondences, often making them fully\nreversible. This filling is not made at random. The algorithm tries to\nstick to the identity mapping and, when this is not possible, it prefers\ngenerating many small permutation cycles, each involving only a few\ncodes.\n\nFor example, here is how 'IBM-PC' code 186 gets translated to\n'control-U' in 'Latin-1'. 'Control-U' is 21. Code 21 is the 'IBM-PC'\nsection sign, which is 167 in 'Latin-1'. 'recode' cannot reciprocate\n167 to 21, because 167 is the masculine ordinal indicator within\n'IBM-PC', which is 186 in 'Latin-1'. Code 186 within 'IBM-PC' has no\n'Latin-1' equivalent; by assigning it back to 21, 'recode' closes this\nshort permutation loop.\n\nAs a consequence of this map filling, 'recode' may sometimes produce\nfunny characters. They may look annoying, they are nevertheless\nhelpful when one changes his (her) mind and wants to revert to the prior\nrecoding. If you cannot stand these, use option '-s', which asks for a\nvery strict recoding.\n\nThis map filling sometimes has a few surprising consequences, which\nsome users wrongly interpreted as bugs. Here are two examples.\n\n1. In some cases, 'recode' seems to copy a file without recoding it.\nBut in fact, it does. Consider a request:\n\nrecode l1..us < File-Latin1 > File-ASCII\ncmp File-Latin1 File-ASCII\n\nthen 'cmp' will not report any difference. This is quite normal.\n'Latin-1' gets correctly recoded to ASCII for charsets\ncommonalities (which are the first 128 characters, in this case).\nThe remaining last 128 'Latin-1' characters have no ASCII\ncorrespondent. Instead of losing them, 'recode' elects to map them\nto unspecified characters of ASCII, so making the recoding\nreversible. The simplest way of achieving this is merely to keep\nthose last 128 characters unchanged. The overall effect is copying\nthe file verbatim.\n\nIf you feel this behaviour is too generous and if you do not wish\nto care about reversibility, simply use option '-s'. By doing so,\n'recode' will strictly map only those 'Latin-1' characters which\nhave an ASCII equivalent, and will merely drop those which do not.\nThen, there is more chance that you will observe a difference\nbetween the input and the output file.\n\n2. Recoding the wrong way could sometimes give the false impression\nthat recoding has almost been done properly. Consider the\nrequests:\n\nrecode 437..l1 < File-Latin1 > Temp1\nrecode 437..l1 < Temp1 > Temp2\n\nso declaring wrongly 'File-Latin1' to be an IBM-PC file, and\nrecoding to 'Latin-1'. This is surely ill defined and not\nmeaningful. Yet, if you repeat this step a second time, you might\nnotice that many (not all) characters in 'Temp2' are identical to\nthose in 'File-Latin1'. Sometimes, people try to discover how\n'recode' works by experimenting a little at random, rather than\nreading and understanding the documentation; results such as this\nare surely confusing, as they provide those people with a false\nfeeling that they understood something.\n\nReversible codings have this property that, if applied several\ntimes in the same direction, they will eventually bring any\ncharacter back to its original value. Since 'recode' seeks small\npermutation cycles when creating reversible codings, besides\ncharacters unchanged by the recoding, most permutation cycles will\nbe of length 2, and fewer of length 3, etc. So, it is just\nexpectable that applying the recoding twice in the same direction\nwill recover most characters, but will fail to recover those\nparticipating in permutation cycles of length 3. On the other end,\nrecoding six times in the same direction would recover all\ncharacters in cycles of length 1, 2, 3 or 6.\n\n---------- Footnotes ----------\n\n(1) There are still some cases of ambiguous output which are rather\ndifficult to detect, and for which the protection is not active.\n", "subsections": [] }, "File: recode.info, Node: Sequencing, Next: Mixed, Prev: Reversibility, Up: Invoking recode": { "content": "", "subsections": [ { "name": "3.6 Selecting sequencing methods", "content": "" } ] }, "This program uses a few techniques when it is discovered that many": { "content": "passes are needed to comply with the REQUEST. For example, suppose that\nfour elementary steps were selected at recoding path optimisation time.", "subsections": [] }, "Then 'recode' will split itself into four different interconnected": { "content": "tasks, logically equivalent to:\n\nSTEP1 OUTPUT\n\nThe splitting into subtasks is often done using Unix pipes. But the\nsplitting may also be completely avoided, and rather simulated by using\nmemory buffer, or intermediate files. The various '--sequence=STRATEGY'\noptions gives you control over the flow methods, by replacing STRATEGY\nwith 'memory', 'pipe' or 'files'. So, these options may be used to\noverride the default behaviour, which is also explained below.\n\n'--sequence=memory'\nWhen the recoding requires a combination of two or more elementary\nrecoding steps, this option forces many passes over the data, using\nin-memory buffers to hold all intermediary results.\n\n'-i'\n'--sequence=files'\nWhen the recoding requires a combination of two or more elementary\nrecoding steps, this option forces many passes over the data, using\nintermediate files between passes. This is the default behaviour\nwhen files are recoded over themselves. If this option is selected\nin filter mode, that is, when the program reads standard input and\nwrites standard output, it might take longer for programs further\ndown the pipe chain to start receiving some recoded data.\n\n'-p'\n'--sequence=pipe'\nWhen the recoding requires a combination of two or more elementary\nrecoding steps, this option forces the program to fork itself into\na few copies interconnected with pipes, using the 'pipe(2)' system\ncall. All copies of the program operate in parallel. This is the\ndefault behaviour in filter mode. If this option is used when\nfiles are recoded over themselves, this should also save disk space\nbecause some temporary files might not be needed, at the cost of\nmore system overhead.\n\nIf, at installation time, the 'pipe(2)' call is said to be\nunavailable, selecting option '-p' is equivalent to selecting\noption '-i'. (This happens, for example, on MS-DOS systems.)\n", "subsections": [] }, "File: recode.info, Node: Mixed, Next: Emacs, Prev: Sequencing, Up: Invoking recode": { "content": "", "subsections": [ { "name": "3.7 Using mixed charset input", "content": "" } ] }, "In real life and practice, textual files are often made up of many": { "content": "charsets at once. Some parts of the file encode one charset, while\nother parts encode another charset, and so forth. Usually, a file does\nnot toggle between more than two or three charsets. The means to\ndistinguish which charsets are encoded at various places is not always\navailable. The 'recode' program is able to handle only a few simple\ncases of mixed input.\n\nThe default 'recode' behaviour is to expect pure charset files, to be\nrecoded as other pure charset files. However, the following options\nallow for a few precise kinds of mixed charset files.\n\n'-d'\n'--diacritics'\nWhile converting to or from one of 'HTML' or 'LaTeX' charset, limit\nconversion to some subset of all characters. For 'HTML', limit\nconversion to the subset of all non-ASCII characters. For 'LaTeX',\nlimit conversion to the subset of all non-English letters. This is\nparticularly useful, for example, when people create what would be\nvalid 'HTML', TeX or LaTeX files, if only they were using provided\nsequences for applying diacritics instead of using the\ndiacriticised characters directly from the underlying character\nset.\n\nWhile converting to 'HTML' or 'LaTeX' charset, this option assumes\nthat characters not in the said subset are properly coded or\nprotected already, 'recode' then transmit them literally. While\nconverting the other way, this option prevents translating back\ncoded or protected versions of characters not in the said subset.\n*Note HTML::. *Note LaTeX::.\n\n'-S[LANGUAGE]'\n'--source[=LANGUAGE]'\nThe bulk of the input file is expected to be written in 'ASCII',\nexcept for parts, like comments and string constants, which are\nwritten using another charset than 'ASCII'. When LANGUAGE is 'c',\nthe recoding will proceed only with the contents of comments or\nstrings, while everything else will be copied without recoding.\nWhen LANGUAGE is 'po', the recoding will proceed only within\ntranslator comments (those having whitespace immediately following\nthe initial '#') and with the contents of 'msgstr' strings.\n\nFor the above things to work, the non-'ASCII' encoding of the\ncomment or string should be such that an 'ASCII' scan will\nsuccessfully find where the comment or string ends.\n\nEven if 'ASCII' is the usual charset for writing programs, some\ncompilers are able to directly read other charsets, like 'UTF-8',\nsay. There is currently no provision in 'recode' for reading mixed\ncharset sources which are not based on 'ASCII'. It is probable\nthat the need for mixed recoding is not as pressing in such cases.\n\nFor example, after one does:\n\nrecode -Spo pc/..u8 < INPUT.po > OUTPUT.po\n\nfile 'OUTPUT.po' holds a copy of 'INPUT.po' in which only\ntranslator comments and the contents of 'msgstr' strings have been\nrecoded from the 'IBM-PC' charset to pure 'UTF-8', without\nattempting conversion of end-of-lines. Machine generated comments\nand original 'msgid' strings are not to be touched by this\nrecoding.\n\nIf LANGUAGE is not specified, 'c' is assumed.\n", "subsections": [] }, "File: recode.info, Node: Emacs, Next: Debugging, Prev: Mixed, Up: Invoking recode": { "content": "", "subsections": [ { "name": "3.8 Using 'recode' within Emacs", "content": "" } ] }, "The fact 'recode' is a filter makes it quite easy to use from within GNU": { "content": "", "subsections": [] }, "Emacs. For example, recoding the whole buffer from the 'IBM-PC' charset": { "content": "to current charset ('Latin-1' on Unix) is easily done with:\n\nC-x h C-u M-| recode ibmpc RET\n\n'C-x h' selects the whole buffer, and 'C-u M-|' filters and replaces the\ncurrent region through the given shell command. Here is another\nexample, binding the keys 'C-c T' to the recoding of the current region\nfrom Easy French to 'Latin-1' (on Unix) and the key 'C-u C-c T' from\n'Latin-1' (on Unix) to Easy French:\n\n(global-set-key \"\\C-cT\" 'recode-texte)\n\n(defun recode-texte (flag)\n(interactive \"P\")\n(shell-command-on-region\n(region-beginning) (region-end)\n(concat \"recode \" (if flag \"..txte\" \"txte\")) t)\n(exchange-point-and-mark))\n", "subsections": [] }, "File: recode.info, Node: Debugging, Prev: Emacs, Up: Invoking recode": { "content": "", "subsections": [ { "name": "3.9 Debugging considerations", "content": "" } ] }, "It is our experience that when 'recode' does not provide satisfying": { "content": "results, either 'recode' was not called properly, correct results raised\nsome doubts nevertheless, or files to recode were somewhat mangled.\nGenuine bugs are surely possible.\n\nUnless you already are a 'recode' expert, it might be a good idea to\nquickly revisit the tutorial (*note Tutorial::) or the prior sections in\nthis chapter, to make sure that you properly formatted your recoding\nrequest. In the case you intended to use 'recode' as a filter, make\nsure that you did not forget to redirect your standard input (through\nusing the '<' symbol in the shell, say). Some 'recode' false mysteries\nare also easily explained, *Note Reversibility::.\n\nFor the other cases, some investigation is needed. To illustrate how\nto proceed, let's presume that you want to recode the 'nicepage' file,\ncoded 'UTF-8', into 'HTML'. The problem is that the command 'recode\nu8..h nicepage' yields:\n\nrecode: Invalid input in step `UTF-8..ISO-10646-UCS-2'\n\nOne good trick is to use 'recode' in filter mode instead of in file\nreplacement mode, *Note Synopsis::. Another good trick is to use the\n'-v' option asking for a verbose description of the recoding steps. We\ncould rewrite our recoding call as 'recode -v u8..h temporary\nrecode -v ISO10646-UCS-2..HTML4.0 ', then 'recode' might have done a bit more that you\nwanted. In this case, your input file was half-'UTF-8', half-'HTML'\nalready, that is, a mixed file (*note Mixed::). There is a special '-d'\nswitch for this case. So, your might be end up calling 'recode -fd\nnicepage'. Until you are quite sure that you accept overwriting your\ninput file whatever what, I recommend that you stick with filter mode.\n\nIf, after such experiments, you seriously think that the 'recode'\nprogram does not behave properly, there might be a genuine bug in the\nprogram itself, in which case I invite you to to contribute a bug\nreport, *Note Contributing::.\n", "subsections": [] }, "File: recode.info, Node: Library, Next: Universal, Prev: Invoking recode, Up: Top": { "content": "", "subsections": [] }, "4 A recoding library": { "content": "", "subsections": [] }, "The program named 'recode' is just an application of its recoding": { "content": "library. The recoding library is available separately for other C\nprograms. A good way to acquire some familiarity with the recoding\nlibrary is to get acquainted with the 'recode' program itself.\n\nTo use the recoding library once it is installed, a C program needs\nto have a line:\n\n#include \n\nnear its beginning, and the user should have '-lrecode' on the linking\ncall, so modules from the recoding library are found.\n\nThe library is still under development. As it stands, it contains\nfour identifiable sets of routines: the outer level functions, the\nrequest level functions, the task level functions and the charset level\nfunctions. There are discussed in separate sections.\n\nFor effectively using the recoding library in most applications, it\nshould be rarely needed to study anything beyond the main initialisation\nfunction at outer level, and then, various functions at request level.\n\n* Menu:\n\n* Outer level:: Outer level functions\n* Request level:: Request level functions\n* Task level:: Task level functions\n* Charset level:: Charset level functions\n* Errors:: Handling errors\n", "subsections": [] }, "File: recode.info, Node: Outer level, Next: Request level, Prev: Library, Up: Library": { "content": "", "subsections": [ { "name": "4.1 Outer level functions", "content": "" } ] }, "The outer level functions mainly prepare the whole recoding library for": { "content": "use, or do actions which are unrelated to specific recodings. Here is\nan example of a program which does not really make anything useful.\n\n#include \n#include \n\nconst char *programname;\n\nint\nmain (int argc, char *const *argv)\n{\nprogramname = argv[0];\nRECODEOUTER outer = recodenewouter (true);\n\nrecodedeleteouter (outer);\nexit (0);\n}\n\nThe header file '' declares an opaque 'RECODEOUTER'\nstructure, which the programmer should use for allocating a variable in\nhis program (let's assume the programmer is a male, here, no prejudice\nintended). This 'outer' variable is given as a first argument to all\nouter level functions.\n\nThe '' header file uses the Boolean type setup by the\nsystem header file ''. But this header file is still fairly\nnew in C standards, and likely does not exist everywhere. If you system\ndoes not offer this system header file yet, the proper compilation of\nthe '' file could be guaranteed through the replacement of the\ninclusion line by:\n\ntypedef enum {false = 0, true = 1} bool;\n\nPeople wanting wider portability, or Autoconf lovers, might arrange\ntheir 'configure.in' for being able to write something more general,\nlike:\n\n#if STDCHEADERS\n# include \n#endif\n\n/* Some systems do not define EXIT*, even with STDCHEADERS. */\n#ifndef EXITSUCCESS\n# define EXITSUCCESS 0\n#endif\n#ifndef EXITFAILURE\n# define EXITFAILURE 1\n#endif\n/* The following test is to work around the gross typo in systems like Sony\nNEWS-OS Release 4.0C, whereby EXITFAILURE is defined to 0, not 1. */\n#if !EXITFAILURE\n# undef EXITFAILURE\n# define EXITFAILURE 1\n#endif\n\n#if HAVESTDBOOLH\n# include \n#else\ntypedef enum {false = 0, true = 1} bool;\n#endif\n\n#include \n\nconst char *programname;\n\nint\nmain (int argc, char *const *argv)\n{\nprogramname = argv[0];\nRECODEOUTER outer = recodenewouter (true);\n\nrecodetermouter (outer);\nexit (EXITSUCCESS);\n}\n\nbut we will not insist on such details in the examples to come.\n\n* Initialisation functions\n\nRECODEOUTER recodenewouter (AUTOABORT);\nbool recodedeleteouter (OUTER);\n\nThe recoding library absolutely needs to be initialised before\nbeing used, and 'recodenewouter' has to be called once, first.\nBesides the OUTER it is meant to initialise, the function accepts a\nBoolean argument whether or not the library should automatically\nissue diagnostics on standard and abort the whole program on\nerrors. When AUTOABORT is 'true', the library later conveniently\nissues diagnostics itself, and aborts the calling program on\nerrors. This is merely a convenience, because if this parameter\nwas 'false', the calling program should always take care of\nchecking the return value of all other calls to the recoding\nlibrary functions, and when any error is detected, issue a\ndiagnostic and abort processing itself.\n\nRegardless of the setting of AUTOABORT, all recoding library\nfunctions return a success status. Most functions are geared for\nreturning 'false' for an error, and 'true' if everything went fine.\nFunctions returning structures or strings return 'NULL' instead of\nthe result, when the result cannot be produced. If AUTOABORT is\nselected, functions either return 'true', or do not return at all.\n\nAs in the example above, 'recodenewouter' is called only once in\nmost cases. Calling 'recodenewouter' implies some overhead, so\ncalling it more than once should preferably be avoided.\n\nThe termination function 'recodedeleteouter' reclaims the memory\nallocated by 'recodenewouter' for a given OUTER variable.\nCalling 'recodedeleteouter' prior to program termination is more\naesthetic then useful, as all memory resources are automatically\nreclaimed when the program ends. You may spare this terminating\ncall if you prefer.\n\n* The 'programname' declaration\n\nAs we just explained, the user may set the 'recode' library so\nthat, in case of problems error, it issues the diagnostic itself\nand aborts the whole processing. This capability may be quite\nconvenient. When this feature is used, the aborting routine\nincludes the name of the running program in the diagnostic. On the\nother hand, when this feature is not used, the library merely\nreturn error codes, giving the library user fuller control over all\nthis. This behaviour is more like what usual libraries do: they\nreturn codes and never abort. However, I would rather not force\nlibrary users to necessarily check all return codes themselves, by\nleaving no other choice. In most simple applications, letting the\nlibrary diagnose and abort is much easier, and quite welcome. This\nis precisely because both possibilities exist that the\n'programname' variable is needed: it may be used by the library\nwhen the user sets it to diagnose itself.\n", "subsections": [] }, "File: recode.info, Node: Request level, Next: Task level, Prev: Outer level, Up: Library": { "content": "", "subsections": [ { "name": "4.2 Request level functions", "content": "" } ] }, "The request level functions are meant to cover most recoding needs": { "content": "programmers may have; they should provide all usual functionality.", "subsections": [] }, "Their API is almost stable by now. To get started with request level": { "content": "functions, here is a full example of a program which sole job is to\nfilter 'ibmpc' code on its standard input into 'latin1' code on its\nstandard output.\n\n#include \n#include \n#include \n\nconst char *programname;\n\nint\nmain (int argc, char *const *argv)\n{\nprogramname = argv[0];\nRECODEOUTER outer = recodenewouter (true);\nRECODEREQUEST request = recodenewrequest (outer);\nbool success;\n\nrecodescanrequest (request, \"ibmpc..latin1\");\n\nsuccess = recodefiletofile (request, stdin, stdout);\n\nrecodedeleterequest (request);\nrecodedeleteouter (outer);\n\nexit (success ? 0 : 1);\n}\n\nThe header file '' declares a 'RECODEREQUEST' structure,\nwhich the programmer should use for allocating a variable in his\nprogram. This REQUEST variable is given as a first argument to all\nrequest level functions, and in most cases, may be considered as opaque.\n\n* Initialisation functions\n\nRECODEREQUEST recodenewrequest (OUTER);\nbool recodedeleterequest (REQUEST);\n\nNo REQUEST variable may not be used in other request level\nfunctions of the recoding library before having been initialised by\n'recodenewrequest'. There may be many such REQUEST variables, in\nwhich case, they are independent of one another and they all need\nto be initialised separately. To avoid memory leaks, a REQUEST\nvariable should not be initialised a second time without calling\n'recodedeleterequest' to \"un-initialise\" it.\n\nLike for 'recodedeleteouter', calling 'recodedeleterequest'\nprior to program termination, in the example above, may be left\nout.\n\n* Fields of 'struct recoderequest'\n\nHere are the fields of a 'struct recoderequest' which may be\nmeaningfully changed, once a REQUEST has been initialised by\n'recodenewrequest', but before it gets used. It is not very\nfrequent, in practice, that these fields need to be changed. To\naccess the fields, you need to include 'recodext.h' instead of\n'recode.h', in which case there also is a greater chance that you\nneed to recompile your programs if a new version of the recoding\nlibrary gets installed.\n\n'verboseflag'\nThis field is initially 'false'. When set to 'true', the\nlibrary will echo to stderr the sequence of elementary\nrecoding steps needed to achieve the requested recoding.\n\n'diaeresischar'\nThis field is initially the ASCII value of a double quote '\"',\nbut it may also be the ASCII value of a colon ':'. In 'texte'\ncharset, some countries use double quotes to mark diaeresis,\nwhile other countries prefer colons. This field contains the\ndiaeresis character for the 'texte' charset.\n\n'makeheaderflag'\nThis field is initially 'false'. When set to 'true', it\nindicates that the program is merely trying to produce a\nrecoding table in source form rather than completing any\nactual recoding. In such a case, the optimisation of step\nsequence can be attempted much more aggressively. If the step\nsequence cannot be reduced to a single step, table production\nwill fail.\n\n'diacriticsonly'\nThis field is initially 'false'. For 'HTML' and 'LaTeX'\ncharset, it is often convenient to recode the diacriticized\ncharacters only, while just not recoding other HTML code using\nampersands or angular brackets, or LaTeX code using\nbackslashes. Set the field to 'true' for getting this\nbehaviour. In the other charset, one can edit text as well as\nHTML or LaTeX directives.\n\n'asciigraphics'\nThis field is initially 'false', and relate to characters 176\nto 223 in the 'ibmpc' charset, which are use to draw boxes.\nWhen set to 'true', while getting out of 'ibmpc', ASCII\ncharacters are selected so to graphically approximate these\nboxes.\n\n* Study of request strings\n\nbool recodescanrequest (REQUEST, \"STRING\");\n\nThe main role of a REQUEST variable is to describe a set of\nrecoding transformations. Function 'recodescanrequest' studies\nthe given STRING, and stores an internal representation of it into\nREQUEST. Note that STRING may be a full-fledged 'recode' request,\npossibly including surfaces specifications, intermediary charsets,\nsequences, aliases or abbreviations (*note Requests::).\n\nThe internal representation automatically receives some\npre-conditioning and optimisation, so the REQUEST may then later be\nused many times to achieve many actual recodings. It would not be\nefficient calling 'recodescanrequest' many times with the same\nSTRING, it is better having many REQUEST variables instead.\n\n* Actual recoding jobs\n\nOnce the REQUEST variable holds the description of a recoding\ntransformation, a few functions use it for achieving an actual\nrecoding. Either input or output of a recoding may be string, an\nin-memory buffer, or a file.\n\nFunctions with names like 'recodeINPUT-TYPEtoOUTPUT-TYPE'\nrequest an actual recoding, and are described below. It is easy to\nremember which arguments each function accepts, once grasped some\nsimple principles for each possible TYPE. However, one of the\nrecoding function escapes these principles and is discussed\nseparately, first.\n\nrecodestring (REQUEST, STRING);\n\nThe function 'recodestring' recodes STRING according to REQUEST,\nand directly returns the resulting recoded string freshly\nallocated, or 'NULL' if the recoding could not succeed for some\nreason. When this function is used, it is the responsibility of\nthe programmer to ensure that the memory used by the returned\nstring is later reclaimed.\n\nchar *recodestringtobuffer (REQUEST,\nINPUTSTRING,\n&OUTPUTBUFFER, &OUTPUTLENGTH, &OUTPUTALLOCATED);\nbool recodestringtofile (REQUEST,\nINPUTFILE,\nOUTPUTFILE);\nbool recodebuffertobuffer (REQUEST,\nINPUTBUFFER, INPUTLENGTH,\n&OUTPUTBUFFER, &OUTPUTLENGTH, &OUTPUTALLOCATED);\nbool recodebuffertofile (REQUEST,\nINPUTBUFFER, INPUTLENGTH,\nOUTPUTFILE);\nbool recodefiletobuffer (REQUEST,\nINPUTFILE,\n&OUTPUTBUFFER, &OUTPUTLENGTH, &OUTPUTALLOCATED);\nbool recodefiletofile (REQUEST,\nINPUTFILE,\nOUTPUTFILE);\n\nAll these functions return a 'bool' result, 'false' meaning that\nthe recoding was not successful, often because of reversibility\nissues. The name of the function well indicates on which types it\nreads and which type it produces. Let's discuss these three types\nin turn.\n\nstring\n\nA string is merely an in-memory buffer which is terminated by\na 'NUL' character (using as many bytes as needed), instead of\nbeing described by a byte length. For input, a pointer to the\nbuffer is given through one argument.\n\nIt is notable that there is no 'tostring' functions. Only\none function recodes into a string, and it is 'recodestring',\nwhich has already been discussed separately, above.\n\nbuffer\n\nA buffer is a sequence of bytes held in computer memory. For\ninput, two arguments provide a pointer to the start of the\nbuffer and its byte size. Note that for charsets using many\nbytes per character, the size is given in bytes, not in\ncharacters.\n\nFor output, three arguments provide the address of three\nvariables, which will receive the buffer pointer, the used\nbuffer size in bytes, and the allocated buffer size in bytes.\nIf at the time of the call, the buffer pointer is 'NULL', then\nthe allocated buffer size should also be zero, and the buffer\nwill be allocated afresh by the recoding functions. However,\nif the buffer pointer is not 'NULL', it should be already\nallocated, the allocated buffer size then gives its size. If\nthe allocated size gets exceeded while the recoding goes, the\nbuffer will be automatically reallocated bigger, probably\nelsewhere, and the allocated buffer size will be adjusted\naccordingly.\n\nThe second variable, giving the in-memory buffer size, will\nreceive the exact byte size which was needed for the recoding.\nA 'NUL' character is guaranteed at the end of the produced\nbuffer, but is not counted in the byte size of the recoding.\nBeyond that 'NUL', there might be some extra space after the\nrecoded data, extending to the allocated buffer size.\n\nfile\n\nA file is a sequence of bytes held outside computer memory,\nbut buffered through it. For input, one argument provides a\npointer to a file already opened for read. The file is then\nread and recoded from its current position until the end of\nthe file, effectively swallowing it in memory if the\ndestination of the recoding is a buffer. For reading a file\nfiltered through the recoding library, but only a little bit\nat a time, one should rather use 'recodefilteropen' and\n'recodefilterclose' (these two functions are not yet\navailable).\n\nFor output, one argument provides a pointer to a file already\nopened for write. The result of the recoding is written to\nthat file starting at its current position.\n\nThe following special function is still subject to change:\n\nvoid recodeformattable (REQUEST, LANGUAGE, \"NAME\");\n\nand is not documented anymore for now.\n", "subsections": [] }, "File: recode.info, Node: Task level, Next: Charset level, Prev: Request level, Up: Library": { "content": "", "subsections": [ { "name": "4.3 Task level functions", "content": "" } ] }, "The task level functions are used internally by the request level": { "content": "functions, they allow more explicit control over files and memory\nbuffers holding input and output to recoding processes. The interface\nspecification of task level functions is still subject to change a bit.\n\nTo get started with task level functions, here is a full example of a\nprogram which sole job is to filter 'ibmpc' code on its standard input\ninto 'latin1' code on its standard output. That is, this program has\nthe same goal as the one from the previous section, but does its things\na bit differently.\n\n#include \n#include \n#include \n\nconst char *programname;\n\nint\nmain (int argc, char *const *argv)\n{\nprogramname = argv[0];\nRECODEOUTER outer = recodenewouter (false);\nRECODEREQUEST request = recodenewrequest (outer);\nRECODETASK task;\nbool success;\n\nrecodescanrequest (request, \"ibmpc..latin1\");\n\ntask = recodenewtask (request);\ntask->input.file = \"\";\ntask->output.file = \"\";\nsuccess = recodeperformtask (task);\n\nrecodedeletetask (task);\nrecodedeleterequest (request);\nrecodedeleteouter (outer);\n\nexit (success ? 0 : 1);\n}\n\nThe header file '' declares a 'RECODETASK' structure,\nwhich the programmer should use for allocating a variable in his\nprogram. This 'task' variable is given as a first argument to all task\nlevel functions. The programmer ought to change and possibly consult a\nfew fields in this structure, using special functions.\n\n* Initialisation functions\n\nRECODETASK recodenewtask (REQUEST);\nbool recodedeletetask (TASK);\n\nNo TASK variable may be used in other task level functions of the\nrecoding library without having first been initialised with\n'recodenewtask'. There may be many such TASK variables, in which\ncase, they are independent of one another and they all need to be\ninitialised separately. To avoid memory leaks, a TASK variable\nshould not be initialised a second time without calling\n'recodedeletetask' to \"un-initialise\" it. This function also\naccepts a REQUEST argument and associates the request to the task.\nIn fact, a task is essentially a set of recoding transformations\nwith the specification for its current input and its current\noutput.\n\nThe REQUEST variable may be scanned before or after the call to\n'recodenewtask', it does not matter so far. Immediately after\ninitialisation, before further changes, the TASK variable\nassociates REQUEST empty in-memory buffers for both input and\noutput. The output buffer will later get allocated automatically\non the fly, as needed, by various task processors.\n\nEven if a call to 'recodedeletetask' is not strictly mandatory\nbefore ending the program, it is cleaner to always include it.\nMoreover, in some future version of the recoding library, it might\nbecome required.\n\n* Fields of 'struct taskrequest'\n\nHere are the fields of a 'struct taskrequest' which may be\nmeaningfully changed, once a TASK has been initialised by\n'recodenewtask'. In fact, fields are expected to change. Once\nagain, to access the fields, you need to include 'recodext.h'\ninstead of 'recode.h', in which case there also is a greater\nchance that you need to recompile your programs if a new version of\nthe recoding library gets installed.\n\n'request'\n\nThe field 'request' points to the current recoding request,\nbut may be changed as needed between recoding calls, for\nexample when there is a need to achieve the construction of a\nresulting text made up of many pieces, each being recoded\ndifferently.\n\n'input.name'\n'input.file'\n\nIf 'input.name' is not 'NULL' at start of a recoding, this is\na request that a file by that name be first opened for reading\nand later automatically closed once the whole file has been\nread. If the file name is not 'NULL' but an empty string, it\nmeans that standard input is to be used. The opened file\npointer is then held into 'input.file'.\n\nIf 'input.name' is 'NULL' and 'input.file' is not, than\n'input.file' should point to a file already opened for read,\nwhich is meant to be recoded.\n\n'input.buffer'\n'input.cursor'\n'input.limit'\n\nWhen both 'input.name' and 'input.file' are 'NULL', three\npointers describe an in-memory buffer containing the text to\nbe recoded. The buffer extends from 'input.buffer' to\n'input.limit', yet the text to be recoded only extends from\n'input.cursor' to 'input.limit'. In most situations,\n'input.cursor' starts with the value that 'input.buffer' has.\n(Its value will internally advance as the recoding goes, until\nit reaches the value of 'input.limit'.)\n\n'output.name'\n'output.file'\n\nIf 'output.name' is not 'NULL' at start of a recoding, this is\na request that a file by that name be opened for write and\nlater automatically closed after the recoding is done. If the\nfile name is not 'NULL' but an empty string, it means that\nstandard output is to be used. The opened file pointer is\nthen held into 'output.file'. If several passes with\nintermediate files are needed to produce the recoding, the\n'output.name' file is opened only for the final pass.\n\nIf 'output.name' is 'NULL' and 'output.file' is not, then\n'output.file' should point to a file already opened for write,\nwhich will receive the result of the recoding.\n\n'output.buffer'\n'output.cursor'\n'output.limit'\n\nWhen both 'output.name' and 'output.file' are 'NULL', three\npointers describe an in-memory buffer meant to receive the\ntext, once it is recoded. The buffer is already allocated\nfrom 'output.buffer' to 'output.limit'. In most situations,\n'output.cursor' starts with the value that 'output.buffer'\nhas. Once the recoding is done, 'output.cursor' will point at\nthe next free byte in the buffer, just after the recoded text,\nso another recoding could be called without changing any of\nthese three pointers, for appending new information to it.\nThe number of recoded bytes in the buffer is the difference\nbetween 'output.cursor' and 'output.buffer'.\n\nEach time 'output.cursor' reaches 'output.limit', the buffer\nis reallocated bigger, possibly at a different location in\nmemory, always held up-to-date in 'output.buffer'. It is\nstill possible to call a task level function with no output\nbuffer at all to start with, in which case all three fields\nshould have 'NULL' as a value. This is the situation\nimmediately after a call to 'recodenewtask'.\n\n'strategy'\nThis field, which is of type 'enum recodesequencestrategy',\ntells how various recoding steps (passes) will be\ninterconnected. Its initial value is\n'RECODESTRATEGYUNDECIDED', which is a constant defined in\nthe header file ''. Other possible values are:\n\n'RECODESEQUENCEINMEMORY'\nKeep intermediate recodings in memory.\n'RECODESEQUENCEWITHFILES'\nDo not fork, use intermediate files.\n'RECODESEQUENCEWITHPIPE'\nFork processes connected with 'pipe(2)'.\n\nThe best for now is to leave this field alone, and let the\nrecoding library decide its strategy, as many combinations\nhave not been tested yet.\n\n'byteordermark'\nThis field, which is preset to 'true', indicates that a byte\norder mark is to be expected at the beginning of any canonical\n'UCS-2' or 'UTF-16' text, and that such a byte order mark\nshould be also produced for these charsets.\n\n'faillevel'\nThis field, which is of type 'enum recodeerror' (*note\nErrors::), sets the error level at which task level functions\nshould report a failure. If an error being detected is equal\nor greater than 'faillevel', the function will eventually\nreturn 'false' instead of 'true'. The preset value for this\nfield is 'RECODENOTCANONICAL', that means that if not reset\nto another value, the library will report failure on any\nerror.\n\n'abortlevel'\nThis field, which is of type 'enum recodeerror' (*note\nErrors::), sets the error level at which task level functions\nshould immediately interrupt their processing. If an error\nbeing detected is equal or greater than 'abortlevel', the\nfunction returns immediately, but the returned value ('true'\nor 'false') is still is decided from the setting of\n'faillevel', not 'abortlevel'. The preset value for this\nfield is 'RECODEMAXIMUMERROR', that means that is not reset\nto another value, the library will never interrupt a recoding\ntask.\n\n'errorsofar'\nThis field, which is of type 'enum recodeerror' (*note\nErrors::), maintains the maximum error level met so far while\nthe recoding task was proceeding. The preset value is\n'RECODENOERROR'.\n\n* Task execution\n\nrecodeperformtask (TASK);\nrecodefilteropen (TASK, FILE);\nrecodefilterclose (TASK);\n\nThe function 'recodeperformtask' reads as much input as possible,\nand recode all of it on prescribed output, given a properly\ninitialised TASK.\n\nFunctions 'recodefilteropen' and 'recodefilterclose' are only\nplanned for now. They are meant to read input in piecemeal ways.\nEven if functionality already exists informally in the library, it\nhas not been made available yet through such interface functions.\n", "subsections": [] }, "File: recode.info, Node: Charset level, Next: Errors, Prev: Task level, Up: Library": { "content": "", "subsections": [ { "name": "4.4 Charset level functions", "content": "" } ] }, "Many functions are internal to the recoding library. Some of them have": { "content": "been made external and available, for the 'recode' program had to retain\nall its previous functionality while being transformed into a mere\napplication of the recoding library. These functions are not really\ndocumented here for the time being, as we hope that many of them will\nvanish over time. When this set of routines will stabilise, it would be\nconvenient to document them as an API for handling charset names and\ncontents.\n\nRECODECHARSET findcharset (NAME, CLEANING-TYPE);\nbool listallcharsets (CHARSET);\nbool listconcisecharset (CHARSET, LIST-FORMAT);\nbool listfullcharset (CHARSET);\n", "subsections": [] }, "File: recode.info, Node: Errors, Prev: Charset level, Up: Library": { "content": "", "subsections": [ { "name": "4.5 Handling errors", "content": "" } ] }, "The 'recode' program, while using the 'recode' library, needs to control": { "content": "whether recoding problems are reported or not, and then reflect these in\nthe exit status. The program should also instruct the library whether\nthe recoding should be abruptly interrupted when an error is met (so\nsparing processing when it is known in advance that a wrong result would\nbe discarded anyway), or if it should proceed nevertheless. Here is how\nthe library groups errors into levels, listed here in order of\nincreasing severity.\n\n'RECODENOERROR'\n\nNo error was met on previous library calls.\n\n'RECODENOTCANONICAL'\n\nThe input text was using one of the many alternative codings for\nsome phenomenon, but not the one 'recode' would have canonically\ngenerated. So, if the reverse recoding is later attempted, it\nwould produce a text having the same meaning as the original\ntext, yet not being byte identical.\n\nFor example, a 'Base64' block in which end-of-lines appear\nelsewhere that at every 76 characters is not canonical. An\ne-circumflex in TeX which is coded as '\\^{e}' instead of '\\^e' is\nnot canonical.\n\n'RECODEAMBIGUOUSOUTPUT'\n\nIt has been discovered that if the reverse recoding was attempted\non the text output by this recoding, we would not obtain the\noriginal text, only because an ambiguity was generated by accident\nin the output text. This ambiguity would then cause the wrong\ninterpretation to be taken.\n\nHere are a few examples. If the 'Latin-1' sequence 'e^' is\nconverted to Easy French and back, the result will be interpreted\nas e-circumflex and so, will not reflect the intent of the original\ntwo characters. Recoding an 'IBM-PC' text to 'Latin-1' and back,\nwhere the input text contained an isolated 'LF', will have a\nspurious 'CR' inserted before the 'LF'.\n\nCurrently, there are many cases in the library where the production\nof ambiguous output is not properly detected, as it is sometimes a\ndifficult problem to accomplish this detection, or to do it\nspeedily.\n\n'RECODEUNTRANSLATABLE'\n\nOne or more input character could not be recoded, because there is\njust no representation for this character in the output charset.\n\nHere are a few examples. Non-strict mode often allows 'recode' to\ncompute on-the-fly mappings for unrepresentable characters, but\nstrict mode prohibits such attribution of reversible translations:\nso strict mode might often trigger such an error. Most 'UCS-2'\ncodes used to represent Asian characters cannot be expressed in\nvarious Latin charsets.\n\n'RECODEINVALIDINPUT'\n\nThe input text does not comply with the coding it is declared to\nhold. So, there is no way by which a reverse recoding would\nreproduce this text, because 'recode' should never produce invalid\noutput.\n\nHere are a few examples. In strict mode, 'ASCII' text is not\nallowed to contain characters with the eight bit set. 'UTF-8'\nencodings ought to be minimal(1).\n\n'RECODESYSTEMERROR'\n\nThe underlying system reported an error while the recoding was\ngoing on, likely an input/output error. (This error symbol is\ncurrently unused in the library.)\n\n'RECODEUSERERROR'\n\nThe programmer or user requested something the recoding library is\nunable to provide, or used the API wrongly. (This error symbol is\ncurrently unused in the library.)\n\n'RECODEINTERNALERROR'\n\nSomething really wrong, which should normally never happen, was\ndetected within the recoding library. This might be due to genuine\nbugs in the library, or maybe due to un-initialised or overwritten\narguments to the API. (This error symbol is currently unused in the\nlibrary.)\n\n'RECODEMAXIMUMERROR'\n\nThis error code should never be returned, it is only internally\nused as a sentinel for the list of all possible error codes.\n\nOne should be able to set the error level threshold for returning\nfailure at end of recoding, and also the threshold for immediate\ninterruption. If many errors occur while the recoding proceed, which\nare not severe enough to interrupt the recoding, then the most severe\nerror is retained, while others are forgotten(2). So, in case of an\nerror, the possible actions currently are:\n\n* do nothing and let go, returning success at end of recoding,\n* just let go for now, but return failure at end of recoding,\n* interrupt recoding right away and return failure now.\n\n*Note Task level::, and particularly the description of the fields\n'faillevel', 'abortlevel' and 'errorsofar', for more information\nabout how errors are handled.\n\n---------- Footnotes ----------\n\n(1) The minimality of an 'UTF-8' encoding is guaranteed on output,\nbut currently, it is not checked on input.\n\n(2) Another approach would have been to define the level symbols as\nmasks instead, and to give masks to threshold setting routines, and to\nretain all errors--yet I never met myself such a need in practice, and\nso I fear it would be overkill. On the other hand, it might be\ninteresting to maintain counters about how many times each kind of error\noccurred.\n", "subsections": [] }, "File: recode.info, Node: Universal, Next: libiconv, Prev: Library, Up: Top": { "content": "", "subsections": [] }, "5 The universal charset": { "content": "", "subsections": [] }, "Standard ISO 10646 defines a universal character set, intended to": { "content": "encompass in the long run all languages written on this planet. It is\nbased on wide characters, and offer possibilities for two billion\ncharacters (2^31).\n\nThis charset was to become available in 'recode' under the name\n'UCS', with many external surfaces for it. But in the current version,\nonly surfaces of 'UCS' are offered, each presented as a genuine charset\nrather than a surface. Such surfaces are only meaningful for the 'UCS'\ncharset, so it is not that useful to draw a line between the surfaces\nand the only charset to which they may apply.\n\n'UCS' stands for Universal Character Set. 'UCS-2' and 'UCS-4' are\nfixed length encodings, using two or four bytes per character\nrespectively. 'UTF' stands for 'UCS' Transformation Format, and are\nvariable length encodings dedicated to 'UCS'. 'UTF-1' was based on\nISO 2022, it did not succeed(1). 'UTF-2' replaced it, it has been\ncalled 'UTF-FSS' (File System Safe) in Unicode or Plan9 context, but is\nbetter known today as 'UTF-8'. To complete the picture, there is\n'UTF-16' based on 16 bits bytes, and 'UTF-7' which is meant for\ntransmissions limited to 7-bit bytes. Most often, one might see 'UTF-8'\nused for external storage, and 'UCS-2' used for internal storage.\n\nWhen 'recode' is producing any representation of 'UCS', it uses the\nreplacement character 'U+FFFD' for any valid character which is not\nrepresentable in the goal charset(2). This happens, for example, when\n'UCS-2' is not capable to echo a wide 'UCS-4' character, or for a\nsimilar reason, an 'UTF-8' sequence using more than three bytes. The\nreplacement character is meant to represent an existing character. So,\nit is never produced to represent an invalid sequence or ill-formed\ncharacter in the input text. In such cases, 'recode' just gets rid of\nthe noise, while taking note of the error in its usual ways.\n\nEven if 'UTF-8' is an encoding, really, it is the encoding of a\nsingle character set, and nothing else. It is useful to distinguish\nbetween an encoding (a surface within 'recode') and a charset, but\nonly when the surface may be applied to several charsets. Specifying a\ncharset is a bit simpler than specifying a surface in a 'recode'\nrequest. There would not be a practical advantage at imposing a more\ncomplex syntax to 'recode' users, when it is simple to assimilate\n'UTF-8' to a charset. Similar considerations apply for 'UCS-2',\n'UCS-4', 'UTF-16' and 'UTF-7'. These are all considered to be charsets.\n\n* Menu:\n\n* UCS-2:: Universal Character Set, 2 bytes\n* UCS-4:: Universal Character Set, 4 bytes\n* UTF-7:: Universal Transformation Format, 7 bits\n* UTF-8:: Universal Transformation Format, 8 bits\n* UTF-16:: Universal Transformation Format, 16 bits\n* count-characters:: Frequency count of characters\n* dump-with-names:: Fully interpreted UCS dump\n\n---------- Footnotes ----------\n\n(1) It is not probable that 'recode' will ever support 'UTF-1'.\n\n(2) This is when the goal charset allows for 16-bits. For shorter\ncharsets, the '--strict' ('-s') option decides what happens: either the\ncharacter is dropped, or a reversible mapping is produced on the fly.\n", "subsections": [] }, "File: recode.info, Node: UCS-2, Next: UCS-4, Prev: Universal, Up: Universal": { "content": "", "subsections": [ { "name": "5.1 Universal Character Set, 2 bytes", "content": "" } ] }, "One surface of 'UCS' is usable for the subset defined by its first sixty": { "content": "thousand characters (in fact, 31 * 2^11 codes), and uses exactly two\nbytes per character. It is a mere dump of the internal memory\nrepresentation which is natural for this subset and as such, conveys\nwith it endianness problems.\n\nA non-empty 'UCS-2' file normally begins with a so called \"byte order\nmark\", having value '0xFEFF'. The value '0xFFFE' is not an 'UCS'\ncharacter, so if this value is seen at the beginning of a file, 'recode'\nreacts by swapping all pairs of bytes. The library also properly reacts\nto other occurrences of '0xFEFF' or '0xFFFE' elsewhere than at the\nbeginning, because concatenation of 'UCS-2' files should stay a simple\nmatter, but it might trigger a diagnostic about non canonical input.\n\nBy default, when producing an 'UCS-2' file, 'recode' always outputs\nthe high order byte before the low order byte. But this could be easily\noverridden through the '21-Permutation' surface (*note Permutations::).\nFor example, the command:\n\nrecode u8..u2/21 < INPUT > OUTPUT\n\nasks for an 'UTF-8' to 'UCS-2' conversion, with swapped byte output.\n\nUse 'UCS-2' as a genuine charset. This charset is available in\n'recode' under the name 'ISO-10646-UCS-2'. Accepted aliases are\n'UCS-2', 'BMP', 'rune' and 'u2'.\n\nThe 'recode' library is able to combine 'UCS-2' some sequences of\ncodes into single code characters, to represent a few diacriticized\ncharacters, ligatures or diphtongs which have been included to ease\nmapping with other existing charsets. It is also able to explode such\nsingle code characters into the corresponding sequence of codes. The\nrequest syntax for triggering such operations is rudimentary and\ntemporary. The 'combined-UCS-2' pseudo character set is a special form\nof 'UCS-2' in which known combinings have been replaced by the simpler\ncode. Using 'combined-UCS-2' instead of 'UCS-2' in an after position\nof a request forces a combining step, while using 'combined-UCS-2'\ninstead of 'UCS-2' in a before position of a request forces an\nexploding step. For the time being, one has to resort to advanced\nrequest syntax to achieve other effects. For example:\n\nrecode u8..co,u2..u8 < INPUT > OUTPUT\n\ncopies an 'UTF-8' INPUT over OUTPUT, still to be in 'UTF-8', yet merging\ncombining characters into single codes whenever possible.\n", "subsections": [] }, "File: recode.info, Node: UCS-4, Next: UTF-7, Prev: UCS-2, Up: Universal": { "content": "", "subsections": [ { "name": "5.2 Universal Character Set, 4 bytes", "content": "" } ] }, "Another surface of 'UCS' uses exactly four bytes per character, and is a": { "content": "mere dump of the internal memory representation which is natural for\nthe whole charset and as such, conveys with it endianness problems.\n\nUse it as a genuine charset. This charset is available in 'recode'\nunder the name 'ISO-10646-UCS-4'. Accepted aliases are 'UCS', 'UCS-4',\n'ISO10646', '10646' and 'u4'.\n", "subsections": [] }, "File: recode.info, Node: UTF-7, Next: UTF-8, Prev: UCS-4, Up: Universal": { "content": "", "subsections": [ { "name": "5.3 Universal Transformation Format, 7 bits", "content": "'UTF-7' comes from IETF rather than ISO, and is described by RFC 2152,\nin the MIME series. The 'UTF-7' encoding is meant to fit 'UCS-2' over\nchannels limited to seven bits per byte. It proceeds from a mix between\nthe spirit of 'Quoted-Printable' and methods of 'Base64', adapted to\nUnicode contexts.\n\nThis charset is available in 'recode' under the name\n'UNICODE-1-1-UTF-7'. Accepted aliases are 'UTF-7', 'TF-7' and 'u7'.\n" } ] }, "File: recode.info, Node: UTF-8, Next: UTF-16, Prev: UTF-7, Up: Universal": { "content": "", "subsections": [ { "name": "5.4 Universal Transformation Format, 8 bits", "content": "" } ] }, "Even if 'UTF-8' does not originally come from IETF, there is now": { "content": "RFC 2279 to describe it. In letters sent on 1995-01-21 and 1995-04-20,\nMarkus Kuhn writes:\n\n'UTF-8' is an 'ASCII' compatible multi-byte encoding of the\nISO 10646 universal character set ('UCS'). 'UCS' is a 31-bit\nsuperset of all other character set standards. The first 256\ncharacters of 'UCS' are identical to those of ISO 8859-1 (Latin-1).\nThe 'UCS-2' encoding of UCS is a sequence of bigendian 16-bit\nwords, the 'UCS-4' encoding is a sequence of bigendian 32-bit\nwords. The 'UCS-2' subset of ISO 10646 is also known as \"Unicode\".\nAs both 'UCS-2' and 'UCS-4' require heavy modifications to\ntraditional 'ASCII' oriented system designs (e.g. Unix), the\n'UTF-8' encoding has been designed for these applications.\n\nIn 'UTF-8', only 'ASCII' characters are encoded using bytes below\n128. All other non-ASCII characters are encoded as multi-byte\nsequences consisting only of bytes in the range 128-253. This\navoids critical bytes like 'NUL' and '/' in 'UTF-8' strings, which\nmakes the 'UTF-8' encoding suitable for being handled by the\nstandard C string library and being used in Unix file names. Other\nproperties include the preserved lexical sorting order and that\n'UTF-8' allows easy self-synchronisation of software receiving\n'UTF-8' strings.\n\n'UTF-8' is the most common external surface of 'UCS', each character\nuses from one to six bytes, and is able to encode all 2^31 characters of\nthe 'UCS'. It is implemented as a charset, with the following\nproperties:\n\n* Strict 7-bit 'ASCII' is completely invariant under 'UTF-8', and\nthose are the only one-byte characters. 'UCS' values and 'ASCII'\nvalues coincide. No multi-byte characters ever contain bytes less\nthan 128. 'NUL' is 'NUL'. A multi-byte character always starts\nwith a byte of 192 or more, and is always followed by a number of\nbytes between 128 to 191. That means that you may read at random\non disk or memory, and easily discover the start of the current,\nnext or previous character. You can count, skip or extract\ncharacters with this only knowledge.\n\n* If you read the first byte of a multi-byte character in binary, it\ncontains many '1' bits in successions starting with the most\nsignificant one (from the left), at least two. The length of this\n'1' sequence equals the byte size of the character. All succeeding\nbytes start by '10'. This is a lot of redundancy, making it fairly\neasy to guess that a file is valid 'UTF-8', or to safely state that\nit is not.\n\n* In a multi-byte character, if you remove all leading '1' bits of\nthe first byte of a multi-byte character, and the initial '10' bits\nof all remaining bytes (so keeping 6 bits per byte for those), the\nremaining bits concatenated are the UCS value.\n\nThese properties also have a few nice consequences:\n\n* Conversion to/from values is algorithmically simple, and reasonably\nspeedy.\n\n* A sequence of N bytes can hold characters needing up to 2 + 5N bits\nin their 'UCS' representation. Here, N is a number between 1 and\n6. So, 'UTF-8' is most economical when mapping ASCII (1 byte),\nfollowed by 'UCS-2' (1 to 3 bytes) and 'UCS-4' (1 to 6 bytes).\n\n* The lexicographic sorting order of 'UCS' strings is preserved.\n\n* Bytes with value 254 or 255 never appear, and because of that,\nthese are sometimes used when escape mechanisms are needed.\n\nIn some case, when little processing is done on a lot of strings, one\nmay choose for efficiency reasons to handle 'UTF-8' strings directly\neven if variable length, as it is easy to get start of characters.", "subsections": [] }, "Character insertion or replacement might require moving the remainder of": { "content": "the string in either direction. In most cases, it is faster and easier\nto convert from 'UTF-8' to 'UCS-2' or 'UCS-4' prior to processing.\n\nThis charset is available in 'recode' under the name 'UTF-8'.\nAccepted aliases are 'UTF-2', 'UTF-FSS', 'FSSUTF', 'TF-8' and 'u8'.\n", "subsections": [] }, "File: recode.info, Node: UTF-16, Next: count-characters, Prev: UTF-8, Up: Universal": { "content": "", "subsections": [ { "name": "5.5 Universal Transformation Format, 16 bits", "content": "" } ] }, "Another external surface of 'UCS' is also variable length, each": { "content": "character using either two or four bytes. It is usable for the subset\ndefined by the first million characters (17 * 2^16) of 'UCS'.\n\nMartin J. Du\"rst writes (to , on 1995-03-28):\n\n'UTF-16' is another method that reserves two times 1024 codepoints\nin Unicode and uses them to index around one million additional\ncharacters. 'UTF-16' is a little bit like former multibyte codes,\nbut quite not so, as both the first and the second 16-bit code\nclearly show what they are. The idea is that one million\ncodepoints should be enough for all the rare Chinese ideograms and\nhistorical scripts that do not fit into the Base Multilingual Plane\nof ISO 10646 (with just about 63,000 positions available, now that\n2,000 are gone).\n\nThis charset is available in 'recode' under the name 'UTF-16'.\nAccepted aliases are 'Unicode', 'TF-16' and 'u6'.\n", "subsections": [] }, "File: recode.info, Node: count-characters, Next: dump-with-names, Prev: UTF-16, Up: Universal": { "content": "", "subsections": [ { "name": "5.6 Frequency count of characters", "content": "A device may be used to obtain a list of characters in a file, and how\nmany times each character appears. Each count is followed by the\n'UCS-2' value of the character and, when known, the RFC 1345 mnemonic\nfor that character.\n\nThis charset is available in 'recode' under the name\n'count-characters'.\n\nThis 'count' feature has been implemented as a charset. This may\nchange in some later version, as it would sometimes be convenient to\ncount original bytes, instead of their 'UCS-2' equivalent.\n" } ] }, "File: recode.info, Node: dump-with-names, Prev: count-characters, Up: Universal": { "content": "", "subsections": [ { "name": "5.7 Fully interpreted UCS dump", "content": "" } ] }, "Another device may be used to get fully interpreted dumps of an 'UCS-2'": { "content": "stream of characters, with one 'UCS-2' character displayed on a full\noutput line. Each line receives the RFC 1345 mnemonic for the character\nif it exists, the 'UCS-2' value of the character, and a descriptive\ncomment for that character. As each input character produces its own\noutput line, beware that the output file from this conversion may be\nmuch, much bigger than the input file.\n\nThis charset is available in 'recode' under the name\n'dump-with-names'.\n\nThis 'dump-with-names' feature has been implemented as a charset\nrather than a surface. This is surely debatable. The current\nimplementation allows for dumping charsets other than 'UCS-2'. For\nexample, the command 'recode l2..full < INPUT' implies a necessary\nconversion from 'Latin-2' to 'UCS-2', as 'dump-with-names' is only\nconnected out from 'UCS-2'. In such cases, 'recode' does not display\nthe original 'Latin-2' codes in the dump, only the corresponding 'UCS-2'\nvalues. To give a simpler example, the command\n\necho 'Hello, world!' | recode us..dump\n\nproduces the following output:\n\nUCS2 Mne Description\n\n0048 H latin capital letter h\n0065 e latin small letter e\n006C l latin small letter l\n006C l latin small letter l\n006F o latin small letter o\n002C , comma\n0020 SP space\n0077 w latin small letter w\n006F o latin small letter o\n0072 r latin small letter r\n006C l latin small letter l\n0064 d latin small letter d\n0021 ! exclamation mark\n000A LF line feed (lf)\n\nThe descriptive comment is given in English and 'ASCII', yet if the", "subsections": [] }, "English description is not available but a French one is, then the": { "content": "", "subsections": [] }, "French description is given instead, using 'Latin-1'. However, if the": { "content": "'LANGUAGE' or 'LANG' environment variable begins with the letters 'fr',\nthen listing preference goes to French when both descriptions are\navailable.\n\nHere is another example. To get the long description of the code 237\nin Latin-5 table, one may use the following command.\n\necho -n 237 | recode l5/d..dump\n\nIf your 'echo' does not grok '-n', use 'echo 237\\c' instead. Here is\nhow to see what Unicode 'U+03C6' means, while getting rid of the title\nlines.\n\necho -n 0x03C6 | recode u2/x2..dump | tail +3\n", "subsections": [] }, "File: recode.info, Node: libiconv, Next: Tabular, Prev: Universal, Up: Top": { "content": "", "subsections": [] }, "6 The 'iconv' library": { "content": "", "subsections": [] }, "The 'recode' library itself contains most code and tables from the": { "content": "portable 'iconv' library, written by Bruno Haible. In fact, many\ncapabilities of the 'recode' library are duplicated because of this\nmerging, as the older 'recode' and 'iconv' libraries share many\ncharsets. We discuss, here, the issues related to this duplication, and\nother peculiarities specific to the 'iconv' library. The plan is to\nremove duplications and better merge specificities, as 'recode' evolves.\n\nAs implemented, if a recoding request can be satisfied by the\n'recode' library both with and without its 'iconv' library part, it is\nlikely that the 'iconv' library will be used. To sort out if the\n'iconv' is indeed used of not, just use the '-v' or '--verbose' option,\n*note Recoding::.\n\nThe ':libiconv:' charset represents a conceptual pivot charset within\nthe 'iconv' part of the 'recode' library (in fact, this pivot exists,\nbut is not directly reachable). This charset has a mere ':' (a colon)\nfor an alias. It is not allowed to recode from or to this charset\ndirectly. But when this charset is selected as an intermediate, usually\nby automatic means, then the 'iconv' part of the 'recode' library is\ncalled to handle the transformations. By using an '--ignore=:libiconv:'\noption on the 'recode' call or equivalently, but more simply, '-x:',\n'recode' is instructed to fully avoid this charset as an intermediate,\nwith the consequence that the 'iconv' part of the library is defeated.\nConsider these two calls:\n\nrecode l1..1250 < INPUT > OUTPUT\nrecode -x: l1..1250 < INPUT > OUTPUT\n\nBoth should transform INPUT from 'ISO-8859-1' to 'CP1250' on OUTPUT.", "subsections": [] }, "The first call uses the 'iconv' part of the library, while the second": { "content": "call avoids it. Whatever the path used, the results should normally be\nidentical. However, there might be observable differences. Most of\nthem might result from reversibility issues, as the 'iconv' engine,\nwhich the 'recode' library directly uses for the time being, does not\naddress reversibility. Even if much less likely, some differences might\nresult from slight errors in the tables used, such differences should\nthen be reported as bugs.\n\nOther irregularities might be seen in the area of error detection and\nrecovery. The 'recode' library usually tries to detect canonicity\nerrors in input, and production of ambiguous output, but the 'iconv'\npart of the library currently does not. Input is always validated,\nhowever. The 'recode' library may not always react properly when its\n'iconv' part has no translation for a given character.\n\nWithin a collection of names for a single charset, the 'recode'\nlibrary distinguishes one of them as being the genuine charset name,\nwhile the others are said to be aliases. When 'recode' lists all\ncharsets, for example with the '-l' or '--list' option, the list\nintegrates all 'iconv' library charsets. The selection of one of the\naliases as the genuine charset name is an artifact added by 'recode', it\ndoes not come from 'iconv'. Moreover, the 'recode' library dynamically\nresolves some conflicts when it initialises itself at runtime. This\nmight explain some discrepancies in the table below, as for what is the\ngenuine charset name.\n\n* General character sets\n'US-ASCII'\n'ASCII', 'ISO646-US', 'ISO646.IRV:1991', 'ISO-IR-6',\n'ANSIX3.4-1968', 'CP367', 'IBM367', 'US', 'csASCII' and\n'ISO646.1991-IRV' are aliases for this charset.\n\n* General multi-byte encodings\n'UTF-8'\n'UTF8' is an alias for this charset.\n\n'UCS-2'\n'ISO-10646-UCS-2' and 'csUnicode' are aliases for this\ncharset.\n\n'UCS-2BE'\n'UNICODEBIG', 'UNICODE-1-1' and 'csUnicode11' are aliases for\nthis charset.\n\n'UCS-2LE'\n'UNICODELITTLE' is an alias for this charset.\n\n'UCS-4'\n'ISO-10646-UCS-4' and 'csUCS4' are aliases for this charset.\n\n'UCS-4BE'\n\n'UCS-4LE'\n\n'UTF-16'\n\n'UTF-16BE'\n\n'UTF-16LE'\n\n'UTF-7'\n'UNICODE-1-1-UTF-7' and 'csUnicode11UTF7' are aliases for this\ncharset.\n\n'UCS-2-INTERNAL'\n\n'UCS-2-SWAPPED'\n\n'UCS-4-INTERNAL'\n\n'UCS-4-SWAPPED'\n\n'JAVA'\n\n* Standard 8-bit encodings\n'ISO-8859-1'\n'ISO8859-1', 'ISO8859-1:1987', 'ISO-IR-100', 'CP819',\n'IBM819', 'LATIN1', 'L1', 'csISOLatin1', 'ISO8859-1' and\n'ISO88591' are aliases for this charset.\n\n'ISO-8859-2'\n'ISO8859-2', 'ISO8859-2:1987', 'ISO-IR-101', 'LATIN2', 'L2',\n'csISOLatin2', 'ISO8859-2' and 'ISO88592' are aliases for\nthis charset.\n\n'ISO-8859-3'\n'ISO8859-3', 'ISO8859-3:1988', 'ISO-IR-109', 'LATIN3', 'L3',\n'csISOLatin3', 'ISO8859-3' and 'ISO88593' are aliases for\nthis charset.\n\n'ISO-8859-4'\n'ISO8859-4', 'ISO8859-4:1988', 'ISO-IR-110', 'LATIN4', 'L4',\n'csISOLatin4', 'ISO8859-4' and 'ISO88594' are aliases for\nthis charset.\n\n'ISO-8859-5'\n'ISO8859-5', 'ISO8859-5:1988', 'ISO-IR-144', 'CYRILLIC',\n'csISOLatinCyrillic', 'ISO8859-5' and 'ISO88595' are aliases\nfor this charset.\n\n'ISO-8859-6'\n'ISO8859-6', 'ISO8859-6:1987', 'ISO-IR-127', 'ECMA-114',\n'ASMO-708', 'ARABIC', 'csISOLatinArabic', 'ISO8859-6' and\n'ISO88596' are aliases for this charset.\n\n'ISO-8859-7'\n'ISO8859-7', 'ISO8859-7:1987', 'ISO-IR-126', 'ECMA-118',\n'ELOT928', 'GREEK8', 'GREEK', 'csISOLatinGreek', 'ISO8859-7'\nand 'ISO88597' are aliases for this charset.\n\n'ISO-8859-8'\n'ISO8859-8', 'ISO8859-8:1988', 'ISO-IR-138', 'HEBREW',\n'csISOLatinHebrew', 'ISO8859-8' and 'ISO88598' are aliases\nfor this charset.\n\n'ISO-8859-9'\n'ISO8859-9', 'ISO8859-9:1989', 'ISO-IR-148', 'LATIN5', 'L5',\n'csISOLatin5', 'ISO8859-9' and 'ISO88599' are aliases for\nthis charset.\n\n'ISO-8859-10'\n'ISO8859-10', 'ISO8859-10:1992', 'ISO-IR-157', 'LATIN6',\n'L6', 'csISOLatin6' and 'ISO8859-10' are aliases for this\ncharset.\n\n'ISO-8859-13'\n'ISO8859-13', 'ISO-IR-179', 'LATIN7' and 'L7' are aliases for\nthis charset.\n\n'ISO-8859-14'\n'ISO8859-14', 'ISO8859-14:1998', 'ISO-IR-199', 'LATIN8' and\n'L8' are aliases for this charset.\n\n'ISO-8859-15'\n'ISO8859-15', 'ISO8859-15:1998' and 'ISO-IR-203' are aliases\nfor this charset.\n\n'ISO-8859-16'\n'ISO8859-16', 'ISO8859-16:2000' and 'ISO-IR-226' are aliases\nfor this charset.\n\n'KOI8-R'\n'csKOI8R' is an alias for this charset.\n\n'KOI8-U'\n\n'KOI8-RU'\n\n* Windows 8-bit encodings\n'CP1250'\n'WINDOWS-1250' and 'MS-EE' are aliases for this charset.\n\n'CP1251'\n'WINDOWS-1251' and 'MS-CYRL' are aliases for this charset.\n\n'CP1252'\n'WINDOWS-1252' and 'MS-ANSI' are aliases for this charset.\n\n'CP1253'\n'WINDOWS-1253' and 'MS-GREEK' are aliases for this charset.\n\n'CP1254'\n'WINDOWS-1254' and 'MS-TURK' are aliases for this charset.\n\n'CP1255'\n'WINDOWS-1255' and 'MS-HEBR' are aliases for this charset.\n\n'CP1256'\n'WINDOWS-1256' and 'MS-ARAB' are aliases for this charset.\n\n'CP1257'\n'WINDOWS-1257' and 'WINBALTRIM' are aliases for this charset.\n\n'CP1258'\n'WINDOWS-1258' is an alias for this charset.\n\n* DOS 8-bit encodings\n'CP850'\n'IBM850', '850' and 'csPC850Multilingual' are aliases for this\ncharset.\n\n'CP866'\n'IBM866', '866' and 'csIBM866' are aliases for this charset.\n\n* Macintosh 8-bit encodings\n'MacRoman'\n'Macintosh', 'MAC' and 'csMacintosh' are aliases for this\ncharset.\n\n'MacCentralEurope'\n\n'MacIceland'\n\n'MacCroatian'\n\n'MacRomania'\n\n'MacCyrillic'\n\n'MacUkraine'\n\n'MacGreek'\n\n'MacTurkish'\n\n'MacHebrew'\n\n'MacArabic'\n\n'MacThai'\n\n* Other platform specific 8-bit encodings\n'HP-ROMAN8'\n'ROMAN8', 'R8' and 'csHPRoman8' are aliases for this charset.\n\n'NEXTSTEP'\n\n* Regional 8-bit encodings used for a single language\n'ARMSCII-8'\n\n'Georgian-Academy'\n\n'Georgian-PS'\n\n'MuleLao-1'\n\n'CP1133'\n'IBM-CP1133' is an alias for this charset.\n\n'TIS-620'\n'TIS620', 'TIS620-0', 'TIS620.2529-1', 'TIS620.2533-0',\n'TIS620.2533-1' and 'ISO-IR-166' are aliases for this charset.\n\n'CP874'\n'WINDOWS-874' is an alias for this charset.\n\n'VISCII'\n'VISCII1.1-1' and 'csVISCII' are aliases for this charset.\n\n'TCVN'\n'TCVN-5712', 'TCVN5712-1' and 'TCVN5712-1:1993' are aliases\nfor this charset.\n\n* CJK character sets (not documented)\n'JISC6220-1969-RO'\n'ISO646-JP', 'ISO-IR-14', 'JP' and 'csISO14JISC6220ro' are\naliases for this charset.\n\n'JISX0201'\n'JISX0201-1976', 'X0201', 'csHalfWidthKatakana',\n'JISX0201.1976-0' and 'JIS0201' are aliases for this charset.\n\n'JISX0208'\n'JISX0208-1983', 'JISX0208-1990', 'JIS0208', 'X0208',\n'ISO-IR-87', 'csISO87JISX0208', 'JISX0208.1983-0',\n'JISX0208.1990-0' and 'JIS0208' are aliases for this charset.\n\n'JISX0212'\n'JISX0212.1990-0', 'JISX0212-1990', 'X0212', 'ISO-IR-159',\n'csISO159JISX02121990', 'JISX0212.1990-0' and 'JIS0212' are\naliases for this charset.\n\n'GB1988-80'\n'ISO646-CN', 'ISO-IR-57', 'CN' and 'csISO57GB1988' are aliases\nfor this charset.\n\n'GB2312-80'\n'ISO-IR-58', 'csISO58GB231280', 'CHINESE' and 'GB2312.1980-0'\nare aliases for this charset.\n\n'ISO-IR-165'\n'CN-GB-ISOIR165' is an alias for this charset.\n\n'KSC5601'\n'KSC5601-1987', 'KSC5601-1989', 'ISO-IR-149',\n'csKSC56011987', 'KOREAN', 'KSC5601.1987-0' and 'KSX1001:1992'\nare aliases for this charset.\n\n* CJK encodings\n'EUC-JP'\n'EUCJP', 'ExtendedUNIXCodePackedFormatforJapanese',\n'csEUCPkdFmtJapanese' and 'EUCJP' are aliases for this\ncharset.\n\n'SJIS'\n'SHIFTJIS', 'SHIFT-JIS', 'MSKANJI' and 'csShiftJIS' are\naliases for this charset.\n\n'CP932'\n\n'ISO-2022-JP'\n'csISO2022JP' and 'ISO2022JP' are aliases for this charset.\n\n'ISO-2022-JP-1'\n\n'ISO-2022-JP-2'\n'csISO2022JP2' is an alias for this charset.\n\n'EUC-CN'\n'EUCCN', 'GB2312', 'CN-GB', 'csGB2312' and 'EUCCN' are\naliases for this charset.\n\n'GBK'\n'CP936' is an alias for this charset.\n\n'GB18030'\n\n'ISO-2022-CN'\n'csISO2022CN' and 'ISO2022CN' are aliases for this charset.\n\n'ISO-2022-CN-EXT'\n\n'HZ'\n'HZ-GB-2312' is an alias for this charset.\n\n'EUC-TW'\n'EUCTW', 'csEUCTW' and 'EUCTW' are aliases for this charset.\n\n'BIG5'\n'BIG-5', 'BIG-FIVE', 'BIGFIVE', 'CN-BIG5' and 'csBig5' are\naliases for this charset.\n\n'CP950'\n\n'BIG5HKSCS'\n\n'EUC-KR'\n'EUCKR', 'csEUCKR' and 'EUCKR' are aliases for this charset.\n\n'CP949'\n'UHC' is an alias for this charset.\n\n'JOHAB'\n'CP1361' is an alias for this charset.\n\n'ISO-2022-KR'\n'csISO2022KR' and 'ISO2022KR' are aliases for this charset.\n\n'CHAR'\n\n'WCHART'\n", "subsections": [] }, "File: recode.info, Node: Tabular, Next: ASCII misc, Prev: libiconv, Up: Top": { "content": "", "subsections": [] }, "7 Tabular sources (RFC 1345)": { "content": "", "subsections": [] }, "An important part of the tabular charset knowledge in 'recode' comes": { "content": "from RFC 1345 or, alternatively, from the 'chset' tools, both maintained\nby Keld Simonsen. The RFC 1345 document:\n\n\"Character Mnemonics & Character Sets\", K. Simonsen, Request for\nComments no. 1345, Network Working Group, June 1992.\n\ndefines many character mnemonics and character sets. The 'recode'\nlibrary implements most of RFC 1345, however:\n\n* It does not recognise those charsets which overload character\npositions: 'dk-us' and 'us-dk'. However, *Note Mixed::.\n\n* It does not recognise those charsets which combine two characters\nfor representing a third: 'ANSIX3.110-1983', 'ISO6937-2-add',\n'T.101-G2', 'T.61-8bit', 'iso-ir-90' and 'videotex-suppl'.\n\n* It does not recognise 16-bits charsets: 'GB2312-80',\n'JISC6226-1978', 'JISC6226-1983', 'JISX0212-1990' and\n'KSC5601-1987'.\n\n* It interprets the charset 'isoir91' as 'NATS-DANO' (alias\n'iso-ir-9-1'), not as 'JISC6229-1984-a' (alias 'iso-ir-91'). It\nalso interprets the charset 'isoir92' as 'NATS-DANO-ADD' (alias\n'iso-ir-9-2'), not as 'JISC6229-1984-b' (alias 'iso-ir-92'). It\nmight be better just avoiding these two alias names.\n\nKeld Simonsen did most of RFC 1345 himself, with some\nfunding from Danish Standards and Nordic standards (INSTA) project. He\nalso did the character set design work, with substantial input from Olle", "subsections": [] }, "Jaernefors. Keld typed in almost all of the tables, some have been": { "content": "contributed. A number of people have checked the tables in various\nways. The RFC lists a number of people who helped.\n\nKeld and the 'recode' maintainer have an arrangement by which any new\ndiscovered information submitted by 'recode' users, about tabular\ncharsets, is forwarded to Keld, eventually merged into Keld's work, and\nonly then, reimported into 'recode'. Neither the 'recode' program nor\nits library try to compete, nor even establish themselves as an\nalternate or diverging reference: RFC 1345 and its new drafts stay the\ngenuine source for most tabular information conveyed by 'recode'. Keld\nhas been more than collaborative so far, so there is no reason that we\nact otherwise. In a word, 'recode' should be perceived as the\napplication of external references, but not as a reference in itself.\n\nInternally, RFC 1345 associates which each character an unambiguous\nmnemonic of a few characters, taken from ISO 646, which is a minimal\nASCII subset of 83 characters. The charset made up by these mnemonics\nis available in 'recode' under the name 'RFC1345'. It has 'mnemonic'\nand '1345' for aliases. As implemened, this charset exactly corresponds\nto 'mnemonic+ascii+38', using RFC 1345 nomenclature. Roughly said,\nISO 646 characters represent themselves, except for the ampersand ('&')\nwhich appears doubled. A prefix of a single ampersand introduces a\nmnemonic. For mnemonics using two characters, the prefix is immediately\nby the mnemonic. For longer mnemonics, the prefix is followed by an\nunderline (''), the mmemonic, and another underline. Conversions to\nthis charset are usually reversible.\n\nCurrently, 'recode' does not offer any of the many other possible\nvariations of this family of representations. They will likely be\nimplemented in some future version, however.\n\n'ANSIX3.4-1968'\n'367', 'ANSIX3.4-1986', 'ASCII', 'CP367', 'IBM367', 'ISO646-US',\n'ISO646.irv:1991', 'US-ASCII', 'iso-ir-6' and 'us' are aliases for\nthis charset. Source: ISO 2375 registry.\n\n'ASMO449'\n'ISO9036', 'arabic7' and 'iso-ir-89' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'BS4730'\n'ISO646-GB', 'gb', 'iso-ir-4' and 'uk' are aliases for this\ncharset. Source: ISO 2375 registry.\n\n'BSviewdata'\n'iso-ir-47' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'CP1250'\n'1250', 'ms-ee' and 'windows-1250' are aliases for this charset.\nSource: UNICODE 1.0.\n\n'CP1251'\n'1251', 'ms-cyrl' and 'windows-1251' are aliases for this charset.\nSource: UNICODE 1.0.\n\n'CP1252'\n'1252', 'ms-ansi' and 'windows-1252' are aliases for this charset.\nSource: UNICODE 1.0.\n\n'CP1253'\n'1253', 'ms-greek' and 'windows-1253' are aliases for this charset.\nSource: UNICODE 1.0.\n\n'CP1254'\n'1254', 'ms-turk' and 'windows-1254' are aliases for this charset.\nSource: UNICODE 1.0.\n\n'CP1255'\n'1255', 'ms-hebr' and 'windows-1255' are aliases for this charset.\nSource: UNICODE 1.0.\n\n'CP1256'\n'1256', 'ms-arab' and 'windows-1256' are aliases for this charset.\nSource: UNICODE 1.0.\n\n'CP1257'\n'1257', 'WinBaltRim' and 'windows-1257' are aliases for this\ncharset. Source: CEN/TC304 N283.\n\n'CSAZ243.4-1985-1'\n'ISO646-CA', 'ca', 'csa7-1' and 'iso-ir-121' are aliases for this\ncharset. Source: ISO 2375 registry.\n\n'CSAZ243.4-1985-2'\n'ISO646-CA2', 'csa7-2' and 'iso-ir-122' are aliases for this\ncharset. Source: ISO 2375 registry.\n\n'CSAZ243.4-1985-gr'\n'iso-ir-123' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'CSN369103'\n'KOI-8L2', 'iso-ir-139' and 'koi8l2' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'CWI'\n'CWI-2' and 'cp-hu' are aliases for this charset. Source:\nComputerworld Sza'mita'stechnika vol 3 issue 13 1988-06-29.\n\n'DEC-MCS'\n'dec' is an alias for this charset. VAX/VMS User's Manual, Order\nNumber: AI-Y517A-TE, April 1986.\n\n'DIN66003'\n'ISO646-DE', 'de' and 'iso-ir-21' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'DS2089'\n'DS2089', 'ISO646-DK' and 'dk' are aliases for this charset.\nSource: Danish Standard, DS 2089, February 1974.\n\n'EBCDIC-AT-DE'\nSource: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\n\n'EBCDIC-AT-DE-A'\nSource: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\n\n'EBCDIC-CA-FR'\nSource: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\n\n'EBCDIC-DK-NO'\nSource: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\n\n'EBCDIC-DK-NO-A'\nSource: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\n\n'EBCDIC-ES'\nSource: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\n\n'EBCDIC-ES-A'\nSource: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\n\n'EBCDIC-ES-S'\nSource: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\n\n'EBCDIC-FI-SE'\nSource: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\n\n'EBCDIC-FI-SE-A'\nSource: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\n\n'EBCDIC-FR'\nSource: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\n\n'EBCDIC-IS-FRISS'\n'friss' is an alias for this charset. Source: Skyrsuvelar Rikisins\nog Reykjavikurborgar, feb 1982.\n\n'EBCDIC-IT'\nSource: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\n\n'EBCDIC-PT'\nSource: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\n\n'EBCDIC-UK'\nSource: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\n\n'EBCDIC-US'\nSource: IBM 3270 Char Set Ref Ch 10, GA27-2837-9, April 1987.\n\n'ECMA-cyrillic'\n'ECMA-113', 'ECMA-113:1986' and 'iso-ir-111' are aliases for this\ncharset. Source: ISO 2375 registry.\n\n'ES'\n'ISO646-ES' and 'iso-ir-17' are aliases for this charset. Source:\nISO 2375 registry.\n\n'ES2'\n'ISO646-ES2' and 'iso-ir-85' are aliases for this charset. Source:\nISO 2375 registry.\n\n'GB1988-80'\n'ISO646-CN', 'cn' and 'iso-ir-57' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'GOST19768-87'\n'STSEV358-88' and 'iso-ir-153' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'IBM037'\n'037', 'CP037', 'ebcdic-cp-ca', 'ebcdic-cp-nl', 'ebcdic-cp-us' and\n'ebcdic-cp-wt' are aliases for this charset. Source: IBM NLS RM\nVol2 SE09-8002-01, March 1990.\n\n'IBM038'\n'038', 'CP038' and 'EBCDIC-INT' are aliases for this charset.\nSource: IBM 3174 Character Set Ref, GA27-3831-02, March 1990.\n\n'IBM1004'\n'1004', 'CP1004' and 'os2latin1' are aliases for this charset.\nSource: CEN/TC304 N283, 1994-02-04.\n\n'IBM1026'\n'1026' and 'CP1026' are aliases for this charset. Source: IBM NLS\nRM Vol2 SE09-8002-01, March 1990.\n\n'IBM1047'\n'1047' and 'CP1047' are aliases for this charset. Source: IBM\nCharacter Data Representation Architecture. Registry SC09-1391-00\np 150.\n\n'IBM256'\n'256', 'CP256' and 'EBCDIC-INT1' are aliases for this charset.\nSource: IBM Registry C-H 3-3220-050.\n\n'IBM273'\n'273' and 'CP273' are aliases for this charset. Source: IBM NLS RM\nVol2 SE09-8002-01, March 1990.\n\n'IBM274'\n'274', 'CP274' and 'EBCDIC-BE' are aliases for this charset.\nSource: IBM 3174 Character Set Ref, GA27-3831-02, March 1990.\n\n'IBM275'\n'275', 'CP275' and 'EBCDIC-BR' are aliases for this charset.\nSource: IBM NLS RM Vol2 SE09-8002-01, March 1990.\n\n'IBM277'\n'EBCDIC-CP-DK' and 'EBCDIC-CP-NO' are aliases for this charset.\nSource: IBM NLS RM Vol2 SE09-8002-01, March 1990.\n\n'IBM278'\n'278', 'CP278', 'ebcdic-cp-fi' and 'ebcdic-cp-se' are aliases for\nthis charset. Source: IBM NLS RM Vol2 SE09-8002-01, March 1990.\n\n'IBM280'\n'280', 'CP280' and 'ebcdic-cp-it' are aliases for this charset.\nSource: IBM NLS RM Vol2 SE09-8002-01, March 1990.\n\n'IBM281'\n'281', 'CP281' and 'EBCDIC-JP-E' are aliases for this charset.\nSource: IBM 3174 Character Set Ref, GA27-3831-02, March 1990.\n\n'IBM284'\n'284', 'CP284' and 'ebcdic-cp-es' are aliases for this charset.\nSource: IBM NLS RM Vol2 SE09-8002-01, March 1990.\n\n'IBM285'\n'285', 'CP285' and 'ebcdic-cp-gb' are aliases for this charset.\nSource: IBM NLS RM Vol2 SE09-8002-01, March 1990.\n\n'IBM290'\n'290', 'CP290' and 'EBCDIC-JP-kana' are aliases for this charset.\nSource: IBM 3174 Character Set Ref, GA27-3831-02, March 1990.\n\n'IBM297'\n'297', 'CP297' and 'ebcdic-cp-fr' are aliases for this charset.\nSource: IBM NLS RM Vol2 SE09-8002-01, March 1990.\n\n'IBM420'\n'420', 'CP420' and 'ebcdic-cp-ar1' are aliases for this charset.\nSource: IBM NLS RM Vol2 SE09-8002-01, March 1990. IBM NLS RM p\n11-11.\n\n'IBM423'\n'423', 'CP423' and 'ebcdic-cp-gr' are aliases for this charset.\nSource: IBM NLS RM Vol2 SE09-8002-01, March 1990.\n\n'IBM424'\n'424', 'CP424' and 'ebcdic-cp-he' are aliases for this charset.\nSource: IBM NLS RM Vol2 SE09-8002-01, March 1990.\n\n'IBM437'\n'437' and 'CP437' are aliases for this charset. Source: IBM NLS RM\nVol2 SE09-8002-01, March 1990.\n\n'IBM500'\n'500', '500V1', 'CP500', 'ebcdic-cp-be' and 'ebcdic-cp-ch' are\naliases for this charset. Source: IBM NLS RM Vol2 SE09-8002-01,\nMarch 1990.\n\n'IBM850'\n'850' and 'CP850' are aliases for this charset. Source: IBM NLS RM\nVol2 SE09-8002-01, March 1990. Source: UNICODE 1.0.\n\n'IBM851'\n'851' and 'CP851' are aliases for this charset. Source: IBM NLS RM\nVol2 SE09-8002-01, March 1990.\n\n'IBM852'\n'852', 'CP852', 'pcl2' and 'pclatin2' are aliases for this charset.\nSource: IBM NLS RM Vol2 SE09-8002-01, March 1990.\n\n'IBM855'\n'855' and 'CP855' are aliases for this charset. Source: IBM NLS RM\nVol2 SE09-8002-01, March 1990.\n\n'IBM857'\n'857' and 'CP857' are aliases for this charset. Source: IBM NLS RM\nVol2 SE09-8002-01, March 1990.\n\n'IBM860'\n'860' and 'CP860' are aliases for this charset. Source: IBM NLS RM\nVol2 SE09-8002-01, March 1990.\n\n'IBM861'\n'861', 'CP861' and 'cp-is' are aliases for this charset. Source:\nIBM NLS RM Vol2 SE09-8002-01, March 1990.\n\n'IBM862'\n'862' and 'CP862' are aliases for this charset. Source: IBM NLS RM\nVol2 SE09-8002-01, March 1990.\n\n'IBM863'\n'863' and 'CP863' are aliases for this charset. Source: IBM\nKeyboard layouts and code pages, PN 07G4586 June 1991.\n\n'IBM864'\n'864' and 'CP864' are aliases for this charset. Source: IBM\nKeyboard layouts and code pages, PN 07G4586 June 1991.\n\n'IBM865'\n'865' and 'CP865' are aliases for this charset. Source: IBM DOS\n3.3 Ref (Abridged), 94X9575 (Feb 1987).\n\n'IBM868'\n'868', 'CP868' and 'cp-ar' are aliases for this charset. Source:\nIBM NLS RM Vol2 SE09-8002-01, March 1990.\n\n'IBM869'\n'869', 'CP869' and 'cp-gr' are aliases for this charset. Source:\nIBM Keyboard layouts and code pages, PN 07G4586 June 1991.\n\n'IBM870'\n'870', 'CP870', 'ebcdic-cp-roece' and 'ebcdic-cp-yu' are aliases\nfor this charset. Source: IBM NLS RM Vol2 SE09-8002-01, March\n1990.\n\n'IBM871'\n'871', 'CP871' and 'ebcdic-cp-is' are aliases for this charset.\nSource: IBM NLS RM Vol2 SE09-8002-01, March 1990.\n\n'IBM875'\n'875', 'CP875' and 'EBCDIC-Greek' are aliases for this charset.\nSource: UNICODE 1.0.\n\n'IBM880'\n'880', 'CP880' and 'EBCDIC-Cyrillic' are aliases for this charset.\nSource: IBM NLS RM Vol2 SE09-8002-01, March 1990.\n\n'IBM891'\n'891' and 'CP891' are aliases for this charset. Source: IBM NLS RM\nVol2 SE09-8002-01, March 1990.\n\n'IBM903'\n'903' and 'CP903' are aliases for this charset. Source: IBM NLS RM\nVol2 SE09-8002-01, March 1990.\n\n'IBM904'\n'904' and 'CP904' are aliases for this charset. Source: IBM NLS RM\nVol2 SE09-8002-01, March 1990.\n\n'IBM905'\n'905', 'CP905' and 'ebcdic-cp-tr' are aliases for this charset.\nSource: IBM 3174 Character Set Ref, GA27-3831-02, March 1990.\n\n'IBM918'\n'918', 'CP918' and 'ebcdic-cp-ar2' are aliases for this charset.\nSource: IBM NLS RM Vol2 SE09-8002-01, March 1990.\n\n'IECP27-1'\n'iso-ir-143' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'INIS'\n'iso-ir-49' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'INIS-8'\n'iso-ir-50' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'INIS-cyrillic'\n'iso-ir-51' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'INVARIANT'\n'iso-ir-170' is an alias for this charset.\n\n'ISO-8859-1'\n'819', 'CP819', 'IBM819', 'ISO8859-1', 'ISO8859-1',\n'ISO8859-1:1987', 'iso-ir-100', 'l1' and 'latin1' are aliases for\nthis charset. Source: ISO 2375 registry.\n\n'ISO-8859-10'\n'ISO8859-10', 'ISO8859-10', 'ISO8859-10:1993', 'L6', 'iso-ir-157'\nand 'latin6' are aliases for this charset. Source: ISO 2375\nregistry.\n\n'ISO-8859-13'\n'ISO8859-13', 'ISO8859-13', 'ISO8859-13:1998', 'iso-baltic',\n'iso-ir-179a', 'l7' and 'latin7' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'ISO-8859-14'\n'ISO8859-14', 'ISO8859-14', 'ISO8859-14:1998', 'iso-celtic',\n'iso-ir-199', 'l8' and 'latin8' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'ISO-8859-15'\n'ISO8859-15', 'ISO8859-15', 'ISO8859-15:1998', 'iso-ir-203', 'l9'\nand 'latin9' are aliases for this charset. Source: ISO 2375\nregistry.\n\n'ISO-8859-2'\n'912', 'CP912', 'IBM912', 'ISO8859-2', 'ISO8859-2',\n'ISO8859-2:1987', 'iso-ir-101', 'l2' and 'latin2' are aliases for\nthis charset. Source: ISO 2375 registry.\n\n'ISO-8859-3'\n'ISO8859-3', 'ISO8859-3', 'ISO8859-3:1988', 'iso-ir-109', 'l3'\nand 'latin3' are aliases for this charset. Source: ISO 2375\nregistry.\n\n'ISO-8859-4'\n'ISO8859-4', 'ISO8859-4', 'ISO8859-4:1988', 'iso-ir-110', 'l4'\nand 'latin4' are aliases for this charset. Source: ISO 2375\nregistry.\n\n'ISO-8859-5'\n'ISO8859-5', 'ISO8859-5', 'ISO8859-5:1988', 'cyrillic' and\n'iso-ir-144' are aliases for this charset. Source: ISO 2375\nregistry.\n\n'ISO-8859-6'\n'ASMO-708', 'ECMA-114', 'ISO8859-6', 'ISO8859-6',\n'ISO8859-6:1987', 'arabic' and 'iso-ir-127' are aliases for this\ncharset. Source: ISO 2375 registry.\n\n'ISO-8859-7'\n'ECMA-118', 'ELOT928', 'ISO8859-7', 'ISO8859-7',\n'ISO8859-7:1987', 'greek', 'greek8' and 'iso-ir-126' are aliases\nfor this charset. Source: ISO 2375 registry.\n\n'ISO-8859-8'\n'ISO8859-8', 'ISO8859-8', 'ISO8859-8:1988', 'hebrew' and\n'iso-ir-138' are aliases for this charset. Source: ISO 2375\nregistry.\n\n'ISO-8859-9'\n'ISO8859-9', 'ISO8859-9', 'ISO8859-9:1989', 'iso-ir-148', 'l5'\nand 'latin5' are aliases for this charset. Source: ISO 2375\nregistry.\n\n'ISO10367-box'\n'iso-ir-155' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'ISO2033-1983'\n'e13b' and 'iso-ir-98' are aliases for this charset. Source: ISO\n2375 registry.\n\n'ISO5427'\n'iso-ir-37' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'ISO5427-ext'\n'ISO5427:1981' and 'iso-ir-54' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'ISO5428'\n'ISO5428:1980' and 'iso-ir-55' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'ISO646.basic'\n'ISO646.basic:1983' and 'ref' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'ISO646.irv'\n'ISO646.irv:1983', 'irv' and 'iso-ir-2' are aliases for this\ncharset. Source: ISO 2375 registry.\n\n'ISO6937-2-25'\n'iso-ir-152' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'ISO8859-supp'\n'iso-ir-154' and 'latin1-2-5' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'IT'\n'ISO646-IT' and 'iso-ir-15' are aliases for this charset. Source:\nISO 2375 registry.\n\n'JISC6220-1969-jp'\n'JISC6220-1969', 'iso-ir-13', 'katakana' and 'x0201-7' are aliases\nfor this charset. Source: ISO 2375 registry.\n\n'JISC6220-1969-ro'\n'ISO646-JP', 'iso-ir-14' and 'jp' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'JISC6229-1984-a'\n'jp-ocr-a' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'JISC6229-1984-b'\n'ISO646-JP-OCR-B' and 'jp-ocr-b' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'JISC6229-1984-b-add'\n'iso-ir-93' and 'jp-ocr-b-add' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'JISC6229-1984-hand'\n'iso-ir-94' and 'jp-ocr-hand' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'JISC6229-1984-hand-add'\n'iso-ir-95' and 'jp-ocr-hand-add' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'JISC6229-1984-kana'\n'iso-ir-96' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'JISX0201'\n'X0201' is an alias for this charset.\n\n'JUSI.B1.002'\n'ISO646-YU', 'iso-ir-141', 'js' and 'yu' are aliases for this\ncharset. Source: ISO 2375 registry.\n\n'JUSI.B1.003-mac'\n'iso-ir-147' and 'macedonian' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'JUSI.B1.003-serb'\n'iso-ir-146' and 'serbian' are aliases for this charset. Source:\nISO 2375 registry.\n\n'KOI-7'\nSource: Andrey A. Chernov .\n\n'KOI-8'\n'GOST19768-74' is an alias for this charset. Source: Andrey A.\nChernov .\n\n'KOI8-R'\nSource: RFC1489 via Gabor Kiss . And Andrey A.\nChernov .\n\n'KOI8-RU'\nSource: http://cad.ntu-kpi.kiev.ua/multiling/koi8-ru/.\n\n'KOI8-U'\nSource: RFC 2319. Mibenum: 2088. Source:\nhttp://www.net.ua/KOI8-U/.\n\n'KSC5636'\n'ISO646-KR' is an alias for this charset.\n\n'Latin-greek-1'\n'iso-ir-27' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'MSZ7795.3'\n'ISO646-HU', 'hu' and 'iso-ir-86' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'NATS-DANO'\n'iso-ir-9-1' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'NATS-DANO-ADD'\n'iso-ir-9-2' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'NATS-SEFI'\n'iso-ir-8-1' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'NATS-SEFI-ADD'\n'iso-ir-8-2' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'NCNC00-10'\n'ISO646-CU', 'NCNC00-10:81', 'cuba' and 'iso-ir-151' are aliases\nfor this charset. Source: ISO 2375 registry.\n\n'NFZ62-010'\n'ISO646-FR', 'fr' and 'iso-ir-69' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'NFZ62-010(1973)'\n'ISO646-FR1' and 'iso-ir-25' are aliases for this charset. Source:\nISO 2375 registry.\n\n'NS4551-1'\n'ISO646-NO', 'iso-ir-60' and 'no' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'NS4551-2'\n'ISO646-NO2', 'iso-ir-61' and 'no2' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'NeXTSTEP'\n'next' is an alias for this charset. Source: Peter Svanberg -\npsv@nada.kth.se.\n\n'PT'\n'ISO646-PT' and 'iso-ir-16' are aliases for this charset. Source:\nISO 2375 registry.\n\n'PT2'\n'ISO646-PT2' and 'iso-ir-84' are aliases for this charset. Source:\nISO 2375 registry.\n\n'SEN850200B'\n'FI', 'ISO646-FI', 'ISO646-SE', 'SS636127', 'iso-ir-10' and 'se'\nare aliases for this charset. Source: ISO 2375 registry.\n\n'SEN850200C'\n'ISO646-SE2', 'iso-ir-11' and 'se2' are aliases for this charset.\nSource: ISO 2375 registry.\n\n'T.61-7bit'\n'iso-ir-102' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'baltic'\n'iso-ir-179' is an alias for this charset. Source: ISO 2375\nregistry. &g1esc x2d56 &g2esc x2e56 &g3esc x2f56.\n\n'greek-ccitt'\n'iso-ir-150' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'greek7'\n'iso-ir-88' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'greek7-old'\n'iso-ir-18' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'hp-roman8'\n'r8' and 'roman8' are aliases for this charset. Source: LaserJet\nIIP Printer User's Manual,. HP part no 33471-90901,\nHewlet-Packard, June 1989.\n\n'latin-greek'\n'iso-ir-19' is an alias for this charset. Source: ISO 2375\nregistry.\n\n'mac-is'\n\n'macintosh'\n'mac' is an alias for this charset. Source: The Unicode Standard\nver 1.0, ISBN 0-201-56788-1, Oct 1991.\n\n'macintoshce'\n'macce' is an alias for this charset. Source: Macintosh CE fonts.\n\n'sami'\n'iso-ir-158', 'lap' and 'latin-lap' are aliases for this charset.\nSource: ISO 2375 registry.\n", "subsections": [] }, "File: recode.info, Node: ASCII misc, Next: IBM and MS, Prev: Tabular, Up: Top": { "content": "", "subsections": [] }, "8 ASCII and some derivatives": { "content": "* Menu:\n\n* ASCII:: Usual ASCII\n* ISO 8859:: ASCII extended by Latin Alphabets\n* ASCII-BS:: ASCII 7-bits, 'BS' to overstrike\n* flat:: ASCII without diacritics nor underline\n", "subsections": [] }, "File: recode.info, Node: ASCII, Next: ISO 8859, Prev: ASCII misc, Up: ASCII misc": { "content": "", "subsections": [ { "name": "8.1 Usual ASCII", "content": "" } ] }, "This charset is available in 'recode' under the name 'ASCII'. In fact,": { "content": "it's true name is 'ANSIX3.4-1968' as per RFC 1345, accepted aliases\nbeing 'ANSIX3.4-1986', 'ASCII', 'IBM367', 'ISO646-US',\n'ISO646.irv:1991', 'US-ASCII', 'cp367', 'iso-ir-6' and 'us'. The\nshortest way of specifying it in 'recode' is 'us'.\n\nThis documentation used to include ASCII tables. They have been\nremoved since the 'recode' program can now recreate these easily:\n\nrecode -lf us for commented ASCII\nrecode -ld us for concise decimal table\nrecode -lo us for concise octal table\nrecode -lh us for concise hexadecimal table\n", "subsections": [] }, "File: recode.info, Node: ISO 8859, Next: ASCII-BS, Prev: ASCII, Up: ASCII misc": { "content": "", "subsections": [ { "name": "8.2 ASCII extended by Latin Alphabets", "content": "" } ] }, "There are many Latin charsets. The following has been written by Tim": { "content": "Lasko , a long while ago:\n\nISO Latin-1, or more completely ISO Latin Alphabet No 1, is now an\ninternational standard as of February 1987 (IS 8859, Part 1). For\nthose American USEnet'rs that care, the 8-bit ASCII standard, which\nis essentially the same code, is going through the final\nadministrative processes prior to publication. ISO Latin-1 (IS\n8859/1) is actually one of an entire family of eight-bit one-byte\ncharacter sets, all having ASCII on the left hand side, and with\nvarying repertoires on the right hand side:\n\n* Latin Alphabet No 1 (caters to Western Europe - now approved).\n* Latin Alphabet No 2 (caters to Eastern Europe - now approved).\n* Latin Alphabet No 3 (caters to SE Europe + others - in draft\nballot).\n* Latin Alphabet No 4 (caters to Northern Europe - in draft\nballot).\n* Latin-Cyrillic alphabet (right half all Cyrillic - processing\ncurrently suspended pending USSR input).\n* Latin-Arabic alphabet (right half all Arabic - now approved).\n* Latin-Greek alphabet (right half Greek + symbols - in draft\nballot).\n* Latin-Hebrew alphabet (right half Hebrew + symbols -\nproposed).\n\nThe ISO Latin Alphabet 1 is available as a charset in 'recode' under\nthe name 'Latin-1'. In fact, it's true name is 'ISO8859-1:1987' as per\nRFC 1345, accepted aliases being 'CP819', 'IBM819', 'ISO-8859-1',\n'ISO8859-1', 'iso-ir-100', 'l1' and 'Latin-1'. The shortest way of\nspecifying it in 'recode' is 'l1'.\n\nIt is an eight-bit code which coincides with ASCII for the lower\nhalf. This documentation used to include Latin-1 tables. They have\nbeen removed since the 'recode' program can now recreate these easily:\n\nrecode -lf l1 for commented ISO Latin-1\nrecode -ld l1 for concise decimal table\nrecode -lo l1 for concise octal table\nrecode -lh l1 for concise hexadecimal table\n", "subsections": [] }, "File: recode.info, Node: ASCII-BS, Next: flat, Prev: ISO 8859, Up: ASCII misc": { "content": "", "subsections": [ { "name": "8.3 ASCII 7-bits, 'BS' to overstrike", "content": "" } ] }, "This charset is available in 'recode' under the name 'ASCII-BS', with": { "content": "'BS' as an acceptable alias.\n\nThe file is straight ASCII, seven bits only. According to the\ndefinition of ASCII, diacritics are applied by a sequence of three\ncharacters: the letter, one 'BS', the diacritic mark. We deviate\nslightly from this by exchanging the diacritic mark and the letter so,\non a screen device, the diacritic will disappear and let the letter\nalone. At recognition time, both methods are acceptable.\n\nThe French quotes are coded by the sequences: '< BS \"' or '\" BS <'\nfor the opening quote and '> BS \"' or '\" BS >' for the closing quote.", "subsections": [] }, "This artificial convention was inherited in straight 'ASCII-BS' from": { "content": "habits around 'Bang-Bang' entry, and is not well known. But we decided\nto stick to it so that 'ASCII-BS' charset will not lose French quotes.\n\nThe 'ASCII-BS' charset is independent of 'ASCII', and different. The\nfollowing examples demonstrate this, knowing at advance that '!2' is the\n'Bang-Bang' way of representing an 'e' with an acute accent. Compare:\n\n% echo \\!2 | recode -v bang..l1/d\nRequest: Bang-Bang..ISO-8859-1/Decimal-1\n233, 10\n\nwith:\n\n% echo \\!2 | recode -v bang..bs/d\nRequest: Bang-Bang..ISO-8859-1..ASCII-BS/Decimal-1\n39, 8, 101, 10\n\nIn the first case, the 'e' with an acute accent is merely transmitted\nby the 'Latin-1..ASCII' mapping, not having a special recoding rule for\nit. In the 'Latin-1..ASCII-BS' case, the acute accent is applied over\nthe 'e' with a backspace: diacriticised characters have special rules.", "subsections": [] }, "For the 'ASCII-BS' charset, reversibility is still possible, but there": { "content": "might be difficult cases.\n", "subsections": [] }, "File: recode.info, Node: flat, Prev: ASCII-BS, Up: ASCII misc": { "content": "", "subsections": [ { "name": "8.4 ASCII without diacritics nor underline", "content": "This charset is available in 'recode' under the name 'flat'.\n\nThis code is ASCII expunged of all diacritics and underlines, as long\nas they are applied using three character sequences, with 'BS' in the\nmiddle. Also, despite slightly unrelated, each control character is\nrepresented by a sequence of two or three graphic characters. The\nnewline character, however, keeps its functionality and is not\nrepresented.\n\nNote that charset 'flat' is a terminal charset. We can convert to\n'flat', but not from it.\n" } ] }, "File: recode.info, Node: IBM and MS, Next: CDC, Prev: ASCII misc, Up: Top": { "content": "", "subsections": [] }, "9 Some IBM or Microsoft charsets": { "content": "The 'recode' program provides various IBM or Microsoft code pages (*note", "subsections": [] }, "Tabular::). An easy way to find them all at once out of the 'recode'": { "content": "program itself is through the command:\n\nrecode -l | egrep -i '(CP|IBM)[0-9]'\n\nBut also, see few special charsets presented in the incoming sections.\n\n* Menu:\n\n* EBCDIC:: EBCDIC codes\n* IBM-PC:: IBM's PC code\n* Icon-QNX:: Unisys' Icon code\n", "subsections": [] }, "File: recode.info, Node: EBCDIC, Next: IBM-PC, Prev: IBM and MS, Up: IBM and MS": { "content": "", "subsections": [ { "name": "9.1 EBCDIC code", "content": "" } ] }, "This charset is the IBM's External Binary Coded Decimal for Interchange": { "content": "", "subsections": [] }, "Coding. This is an eight bits code. The following three variants were": { "content": "implemented in 'recode' independently of RFC 1345:\n\n'EBCDIC'\nIn 'recode', the 'us..ebcdic' conversion is identical to 'dd\nconv=ebcdic' conversion, and 'recode' 'ebcdic..us' conversion is\nidentical to 'dd conv=ascii' conversion. This charset also\nrepresents the way Control Data Corporation relates EBCDIC to\n8-bits ASCII.\n\n'EBCDIC-CCC'\nIn 'recode', the 'us..ebcdic-ccc' or 'ebcdic-ccc..us' conversions\nrepresent the way Concurrent Computer Corporation (formerly Perkin\nElmer) relates EBCDIC to 8-bits ASCII.\n\n'EBCDIC-IBM'\nIn 'recode', the 'us..ebcdic-ibm' conversion is almost identical\nto the GNU 'dd conv=ibm' conversion. Given the exact 'dd conv=ibm'\nconversion table, 'recode' once said:\n\nCodes 91 and 213 both recode to 173\nCodes 93 and 229 both recode to 189\nNo character recodes to 74\nNo character recodes to 106\n\nSo I arbitrarily chose to recode 213 by 74 and 229 by 106. This\nmakes the 'EBCDIC-IBM' recoding reversible, but this is not\nnecessarily the best correction. In any case, I think that GNU\n'dd' should be amended. 'dd' and 'recode' should ideally agree on\nthe same correction. So, this table might change once again.\n\nRFC 1345 brings into 'recode' 15 other EBCDIC charsets, and 21 other\ncharsets having EBCDIC in at least one of their alias names. You can\nget a list of all these by executing:\n\nrecode -l | grep -i ebcdic\n\nNote that 'recode' may convert a pure stream of EBCDIC characters,\nbut it does not know how to handle binary data between records which is\nsometimes used to delimit them and build physical blocks. If end of\nlines are not marked, fixed record size may produce something readable,\nbut 'VB' or 'VBS' blocking is likely to yield some garbage in the\nconverted results.\n", "subsections": [] }, "File: recode.info, Node: IBM-PC, Next: Icon-QNX, Prev: EBCDIC, Up: IBM and MS": { "content": "", "subsections": [ { "name": "9.2 IBM's PC code", "content": "" } ] }, "This charset is available in 'recode' under the name 'IBM-PC', with": { "content": "'dos', 'MSDOS' and 'pc' as acceptable aliases. The shortest way of\nspecifying it in 'recode' is 'pc'.\n\nThe charset is aimed towards a PC microcomputer from IBM or any\ncompatible. This is an eight-bit code. This charset is fairly old in\n'recode', its tables were produced a long while ago by mere inspection\nof a printed chart of the IBM-PC codes and glyph.\n\nIt has 'CR-LF' as its implied surface. This means that, if the\noriginal end of lines have to be preserved while going out of 'IBM-PC',\nthey should currently be added back through the usage of a surface on\nthe other charset, or better, just never removed. Here are examples for\nboth cases:\n\nrecode pc..l2/cl < INPUT > OUTPUT\nrecode pc/..l2 < INPUT > OUTPUT\n\nRFC 1345 brings into 'recode' 44 'IBM' charsets or code pages, and\nalso 8 other code pages. You can get a list of these all these by\nexecuting:(1)\n\nrecode -l | egrep -i '(CP|IBM)[0-9]'\n", "subsections": [] }, "All charset or aliases beginning with letters 'CP' or 'IBM' also have": { "content": "'CR-LF' as their implied surface. The same is true for a purely numeric\nalias in the same family. For example, all of '819', 'CP819' and\n'IBM819' imply 'CR-LF' as a surface. Note that 'ISO-8859-1' does not\nimply a surface, despite it shares the same tabular data as '819'.\n\nThere are a few discrepancies between this 'IBM-PC' charset and the\nvery similar RFC 1345 charset 'ibm437', which have not been analysed\nyet, so the charsets are being kept separate for now. This might change\nin the future, and the 'IBM-PC' charset might disappear. Wizards would\nbe interested in comparing the output of these two commands:\n\nrecode -vh IBM-PC..Latin-1\nrecode -vh IBM437..Latin-1\n", "subsections": [] }, "The first command uses the charset prior to RFC 1345 introduction. Both": { "content": "methods give different recodings. These differences are annoying, the\nfuzziness will have to be explained and settle down one day.\n\n---------- Footnotes ----------\n\n(1) On DOS/Windows, stock shells do not know that apostrophes quote\nspecial characters like '|', so one need to use double quotes instead of\napostrophes.\n", "subsections": [] }, "File: recode.info, Node: Icon-QNX, Prev: IBM-PC, Up: IBM and MS": { "content": "", "subsections": [ { "name": "9.3 Unisys' Icon code", "content": "" } ] }, "This charset is available in 'recode' under the name 'Icon-QNX', with": { "content": "'QNX' as an acceptable alias.\n\nThe file is using Unisys' Icon way to represent diacritics with code\n25 escape sequences, under the system QNX. This is a seven-bit code,\neven if eight-bit codes can flow through as part of IBM-PC charset.\n", "subsections": [] }, "File: recode.info, Node: CDC, Next: Micros, Prev: IBM and MS, Up: Top": { "content": "", "subsections": [] }, "10 Charsets for CDC machines": { "content": "", "subsections": [] }, "What is now 'recode' evolved out, through many transformations really,": { "content": "from a set of programs which were originally written in \"COMPASS\",", "subsections": [] }, "Control Data Corporation's assembler, with bits in FORTRAN, and later": { "content": "rewritten in CDC 6000 Pascal. The CDC heritage shows by the fact some\nold CDC charsets are still supported.\n\nThe 'recode' author used to be familiar with CDC Scope-NOS/BE and", "subsections": [] }, "Kronos-NOS, and many CDC formats. Reading CDC tapes directly on other": { "content": "machines is often a challenge, and 'recode' does not always solve it.", "subsections": [] }, "It helps having tapes created in coded mode instead of binary mode, and": { "content": "using 'S' (Stranger) tapes instead of 'I' (Internal) tapes. ANSI labels\nand multi-file tapes might be the source of trouble. There are ways to\nhandle a few Cyber Record Manager formats, but some of them might be\nquite difficult to decode properly after the transfer is done.\n\nThe 'recode' program is usable only for a small subset of NOS text\nformats, and surely not with binary textual formats, like 'UPDATE' or\n'MODIFY' sources, for example. 'recode' is not especially suited for\nreading 8/12 or 56/60 packing, yet this could easily arranged if there\nwas a demand for it. It does not have the ability to translate Display", "subsections": [] }, "Code directly, as the ASCII conversion implied by tape drivers or FTP": { "content": "does the initial approximation. 'recode' can decode 6/12 caret notation\nover Display Code already mapped to ASCII.\n\n* Menu:\n\n* Display Code:: Control Data's Display Code\n* CDC-NOS:: ASCII 6/12 from NOS\n* Bang-Bang:: ASCII \"bang bang\"\n", "subsections": [] }, "File: recode.info, Node: Display Code, Next: CDC-NOS, Prev: CDC, Up: CDC": { "content": "", "subsections": [ { "name": "10.1 Control Data's Display Code", "content": "This code is not available in 'recode', but repeated here for reference.\nThis is a 6-bit code used on CDC mainframes.\n\nOctal display code to graphic Octal display code to octal ASCII\n\n00 : 20 P 40 5 60 # 00 072 20 120 40 065 60 043\n01 A 21 Q 41 6 61 [ 01 101 21 121 41 066 61 133\n02 B 22 R 42 7 62 ] 02 102 22 122 42 067 62 135\n03 C 23 S 43 8 63 % 03 103 23 123 43 070 63 045\n04 D 24 T 44 9 64 \" 04 104 24 124 44 071 64 042\n05 E 25 U 45 + 65 05 105 25 125 45 053 65 137\n06 F 26 V 46 - 66 ! 06 106 26 126 46 055 66 041\n07 G 27 W 47 * 67 & 07 107 27 127 47 052 67 046\n10 H 30 X 50 / 70 ' 10 110 30 130 50 057 70 047\n11 I 31 Y 51 ( 71 ? 11 111 31 131 51 050 71 077\n12 J 32 Z 52 ) 72 < 12 112 32 132 52 051 72 074\n13 K 33 0 53 $ 73 > 13 113 33 060 53 044 73 076\n14 L 34 1 54 = 74 @ 14 114 34 061 54 075 74 100\n15 M 35 2 55 75 \\ 15 115 35 062 55 040 75 134\n16 N 36 3 56 , 76 ^ 16 116 36 063 56 054 76 136\n17 O 37 4 57 . 77 ; 17 117 37 064 57 056 77 073\n\nIn older times, ':' used octal 63, and octal 0 was not a character." } ] }, "The table above shows the ASCII glyph interpretation of codes 60 to 77,": { "content": "yet these 16 codes were once defined differently.\n\nThere is no explicit end of line in Display Code, and the Cyber", "subsections": [] }, "Record Manager introduced many new ways to represent them, the": { "content": "traditional end of lines being reachable by setting 'RT' to 'Z'. If\n6-bit bytes in a file are sequentially counted from 1, a traditional end\nof line does exist if bytes 10*N+9 and 10N+10 are both zero for a given\nN, in which case these two bytes are not to be interpreted as '::'.", "subsections": [] }, "Also, up to 9 immediately preceeding zero bytes, going backward, are to": { "content": "be considered as part of the end of line and not interpreted as ':'(1).\n\n---------- Footnotes ----------\n\n(1) This convention replaced an older one saying that up to 4\nimmediately preceeding pairs of zero bytes, going backward, are to be\nconsidered as part of the end of line and not interpreted as '::'.\n", "subsections": [] }, "File: recode.info, Node: CDC-NOS, Next: Bang-Bang, Prev: Display Code, Up: CDC": { "content": "", "subsections": [ { "name": "10.2 ASCII 6/12 from NOS", "content": "" } ] }, "This charset is available in 'recode' under the name 'CDC-NOS', with": { "content": "'NOS' as an acceptable alias.\n\nThis is one of the charsets in use on CDC Cyber NOS systems to\nrepresent ASCII, sometimes named \"NOS 6/12\" code for coding ASCII. This\ncode is also known as \"caret ASCII\". It is based on a six bits character\nset in which small letters and control characters are coded using a '^'\nescape and, sometimes, a '@' escape.\n\nThe routines given here presume that the six bits code is already\nexpressed in ASCII by the communication channel, with embedded ASCII '^'\nand '@' escapes.\n\nHere is a table showing which characters are being used to encode\neach ASCII character.\n\n000 ^5 020 ^# 040 060 0 100 @A 120 P 140 @G 160 ^P\n001 ^6 021 ^[ 041 ! 061 1 101 A 121 Q 141 ^A 161 ^Q\n002 ^7 022 ^] 042 \" 062 2 102 B 122 R 142 ^B 162 ^R\n003 ^8 023 ^% 043 # 063 3 103 C 123 S 143 ^C 163 ^S\n004 ^9 024 ^\" 044 $ 064 4 104 D 124 T 144 ^D 164 ^T\n005 ^+ 025 ^ 045 % 065 5 105 E 125 U 145 ^E 165 ^U\n006 ^- 026 ^! 046 & 066 6 106 F 126 V 146 ^F 166 ^V\n007 ^* 027 ^& 047 ' 067 7 107 G 127 W 147 ^G 167 ^W\n010 ^/ 030 ^' 050 ( 070 8 110 H 130 X 150 ^H 170 ^X\n011 ^( 031 ^? 051 ) 071 9 111 I 131 Y 151 ^I 171 ^Y\n012 ^) 032 ^< 052 * 072 @D 112 J 132 Z 152 ^J 172 ^Z\n013 ^$ 033 ^> 053 + 073 ; 113 K 133 [ 153 ^K 173 ^0\n014 ^= 034 ^@ 054 , 074 < 114 L 134 \\ 154 ^L 174 ^1\n015 ^ 035 ^\\ 055 - 075 = 115 M 135 ] 155 ^M 175 ^2\n016 ^, 036 ^^ 056 . 076 > 116 N 136 @B 156 ^N 176 ^3\n017 ^. 037 ^; 057 / 077 ? 117 O 137 157 ^O 177 ^4\n", "subsections": [] }, "File: recode.info, Node: Bang-Bang, Prev: CDC-NOS, Up: CDC": { "content": "", "subsections": [ { "name": "10.3 ASCII \"bang bang\"", "content": "This charset is available in 'recode' under the name 'Bang-Bang'.\n\nThis code, in use on Cybers at Universite' de Montre'al mainly, served\nto code a lot of French texts. The original name of this charset is\n\"ASCII code' Display\". This code is also known as \"Bang-bang\". It is\nbased on a six bits character set in which capitals, French diacritics\nand a few others are coded using an '!' escape followed by a single\ncharacter, and control characters using a double '!' escape followed by\na single character.\n\nThe routines given here presume that the six bits code is already\nexpressed in ASCII by the communication channel, with embedded ASCII '!'\nescapes.\n\nHere is a table showing which characters are being used to encode\neach ASCII character.\n\n000 !!@ 020 !!P 040 060 0 100 @ 120 !P 140 !@ 160 P\n001 !!A 021 !!Q 041 !\" 061 1 101 !A 121 !Q 141 A 161 Q\n002 !!B 022 !!R 042 \" 062 2 102 !B 122 !R 142 B 162 R\n003 !!C 023 !!S 043 # 063 3 103 !C 123 !S 143 C 163 S\n004 !!D 024 !!T 044 $ 064 4 104 !D 124 !T 144 D 164 T\n005 !!E 025 !!U 045 % 065 5 105 !E 125 !U 145 E 165 U\n006 !!F 026 !!V 046 & 066 6 106 !F 126 !V 146 F 166 V\n007 !!G 027 !!W 047 ' 067 7 107 !G 127 !W 147 G 167 W\n010 !!H 030 !!X 050 ( 070 8 110 !H 130 !X 150 H 170 X\n011 !!I 031 !!Y 051 ) 071 9 111 !I 131 !Y 151 I 171 Y\n012 !!J 032 !!Z 052 * 072 : 112 !J 132 !Z 152 J 172 Z\n013 !!K 033 !![ 053 + 073 ; 113 !K 133 [ 153 K 173 ![\n014 !!L 034 !!\\ 054 , 074 < 114 !L 134 \\ 154 L 174 !\\\n015 !!M 035 !!] 055 - 075 = 115 !M 135 ] 155 M 175 !]\n016 !!N 036 !!^ 056 . 076 > 116 !N 136 ^ 156 N 176 !^\n017 !!O 037 !! 057 / 077 ? 117 !O 137 157 O 177 !\n" } ] }, "File: recode.info, Node: Micros, Next: Miscellaneous, Prev: CDC, Up: Top": { "content": "", "subsections": [] }, "11 Other micro-computer charsets": { "content": "", "subsections": [] }, "The 'NeXT' charset, which used to be especially provided in releases of": { "content": "'recode' before 3.5, has been integrated since as one RFC 1345 table.\n\n* Menu:\n\n* Apple-Mac:: Apple's Macintosh code\n* AtariST:: Atari ST code\n", "subsections": [] }, "File: recode.info, Node: Apple-Mac, Next: AtariST, Prev: Micros, Up: Micros": { "content": "", "subsections": [ { "name": "11.1 Apple's Macintosh code", "content": "" } ] }, "This charset is available in 'recode' under the name 'Apple-Mac'. The": { "content": "shortest way of specifying it in 'recode' is 'ap'.\n\nThe charset is aimed towards a Macintosh micro-computer from Apple.", "subsections": [] }, "This is an eight bit code. The file is the data fork only. This": { "content": "charset is fairly old in 'recode', its tables were produced a long while\nago by mere inspection of a printed chart of the Macintosh codes and\nglyph.\n\nIt has 'CR' as its implied surface. This means that, if the original\nend of lines have to be preserved while going out of 'Apple-Mac', they\nshould currently be added back through the usage of a surface on the\nother charset, or better, just never removed. Here are examples for\nboth cases:\n\nrecode ap..l2/cr < INPUT > OUTPUT\nrecode ap/..l2 < INPUT > OUTPUT\n\nRFC 1345 brings into 'recode' 2 other Macintosh charsets. You can\ndiscover them by using 'grep' over the output of 'recode -l':\n\nrecode -l | grep -i mac\n", "subsections": [] }, "Charsets 'macintosh' and 'macintoshce', as well as their aliases 'mac'": { "content": "and 'macce' also have 'CR' as their implied surface.\n\nThere are a few discrepancies between the 'Apple-Mac' charset and the\nvery similar RFC 1345 charset 'macintosh', which have not been analysed\nyet, so the charsets are being kept separate for now. This might change\nin the future, and the 'Apple-Mac' charset might disappear. Wizards\nwould be interested in comparing the output of these two commands:\n\nrecode -vh Apple-Mac..Latin-1\nrecode -vh macintosh..Latin-1\n", "subsections": [] }, "The first command use the charset prior to RFC 1345 introduction. Both": { "content": "methods give different recodings. These differences are annoying, the\nfuzziness will have to be explained and settle down one day.\n\nAs a side note, some people ask if there is a Macintosh port of the\n'recode' program. I'm not aware of any. I presume that if the tool\nfills a need for Macintosh users, someone will port it one of these\ndays?\n", "subsections": [] }, "File: recode.info, Node: AtariST, Prev: Apple-Mac, Up: Micros": { "content": "", "subsections": [ { "name": "11.2 Atari ST code", "content": "This charset is available in 'recode' under the name 'AtariST'.\n\nThis is the character set used on the Atari ST/TT/Falcon. This is\nsimilar to 'IBM-PC', but differs in some details: it includes some more\naccented characters, the graphic characters are mostly replaced by" } ] }, "Hebrew characters, and there is a true German 'sharp s' different from": { "content": "Greek 'beta'.\n\nAbout the end-of-line conversions: the canonical end-of-line on the\nAtari is '\\r\\n', but unlike 'IBM-PC', the OS makes no difference between\ntext and binary input/output; it is up to the application how to\ninterpret the data. In fact, most of the libraries that come with\ncompilers can grok both '\\r\\n' and '\\n' as end of lines. Many of the\nusers who also have access to Unix systems prefer '\\n' to ease porting", "subsections": [] }, "Unix utilities. So, for easing reversibility, 'recode' tries to let": { "content": "'\\r' undisturbed through recodings.\n", "subsections": [] }, "File: recode.info, Node: Miscellaneous, Next: Surfaces, Prev: Micros, Up: Top": { "content": "", "subsections": [] }, "12 Various other charsets": { "content": "", "subsections": [] }, "Even if these charsets were originally added to 'recode' for handling": { "content": "texts written in French, they find other uses. We did use them a lot\nfor writing French diacriticised texts in the past, so 'recode' knows\nhow to handle these particularly well for French texts.\n\n* Menu:\n\n* HTML:: World Wide Web representations\n* LaTeX:: LaTeX macro calls\n* Texinfo:: GNU project documentation files\n* Vietnamese::\n* African:: African charsets\n* Others::\n* Texte:: Easy French conventions\n* Mule:: Mule as a multiplexed charset\n", "subsections": [] }, "File: recode.info, Node: HTML, Next: LaTeX, Prev: Miscellaneous, Up: Miscellaneous": { "content": "", "subsections": [ { "name": "12.1 World Wide Web representations", "content": "" } ] }, "Character entities have been introduced by SGML and made widely popular": { "content": "through HTML, the markup language in use for the World Wide Web, or Web\nor WWW for short. For representing unusual characters, HTML texts use\nspecial sequences, beginning with an ampersand '&' and ending with a\nsemicolon ';'. The sequence may itself start with a number sigh '#' and\nbe followed by digits, so forming a \"numeric character reference\", or\nelse be an alphabetic identifier, so forming a \"character entity\nreference\".\n\nThe HTML standards have been revised into different HTML levels over\ntime, and the list of allowable character entities differ in them. The\nlater XML, meant to simplify many things, has an option\n('standalone=yes') which much restricts that list. The 'recode' library\nis able to convert character references between their mnemonic form and\ntheir numeric form, depending on aimed HTML standard level. It also\ncan, of course, convert between HTML and various other charsets.\n\nHere is a list of those HTML variants which 'recode' supports. Some\nnotes have been provided by Franc,ois Yergeau .\n\n'XML-standalone'\nThis charset is available in 'recode' under the name\n'XML-standalone', with 'h0' as an acceptable alias. It is\ndocumented in section 4.1 of . It\nonly knows '&', '>', '<', '"' and '''.\n\n'HTML1.1'\nThis charset is available in 'recode' under the name 'HTML1.1',\nwith 'h1' as an acceptable alias. HTML 1.0 was never really\ndocumented.\n\n'HTML2.0'\nThis charset is available in 'recode' under the name 'HTML2.0',\nand has 'RFC1866', '1866' and 'h2' for aliases. HTML 2.0 entities\nare listed in RFC 1866. Basically, there is an entity for each\nalphabetical character in the right part of ISO 8859-1. In\naddition, there are four entities for syntax-significant ASCII\ncharacters: '&', '>', '<' and '"'.\n\n'HTML-i18n'\nThis charset is available in 'recode' under the name 'HTML-i18n',\nand has 'RFC2070' and '2070' for aliases. RFC 2070 added entities\nto cover the whole right part of ISO 8859-1. The list is\nconveniently accessible at\n. In\naddition, four i18n-related entities were added: '‌'\n('‌'), '‍' ('‍'), '‎' ('‎') and '‏'\n('‏').\n\n'HTML3.2'\nThis charset is available in 'recode' under the name 'HTML3.2',\nwith 'h3' as an acceptable alias. HTML 3.2\n(http://www.w3.org/TR/REC-html32.html) took up the full Latin-1\nlist but not the i18n-related entities from RFC 2070.\n\n'HTML4.0'\nThis charset is available in 'recode' under the name 'HTML4.0',\nand has 'h4' and 'h' for aliases. Beware that the particular alias\n'h' is not tied to HTML 4.0, but to the highest HTML level\nsupported by 'recode'; so it might later represent HTML level 5 if\nthis is ever created. HTML 4.0 (http://www.w3.org/TR/REC-html40/)\nhas the whole Latin-1 list, a set of entities for symbols,\nmathematical symbols, and Greek letters, and another set for\nmarkup-significant and internationalization characters comprising\nthe 4 ASCII entities, the 4 i18n-related from RFC 2070 plus some\nmore. See .\n\nPrintable characters from Latin-1 may be used directly in an HTML\ntext. However, partly because people have deficient keyboards, partly\nbecause people want to transmit HTML texts over non 8-bit clean channels\nwhile not using MIME, it is common (yet debatable) to use character\nentity references even for Latin-1 characters, when they fall outside\nASCII (that is, when they have the 8th bit set).\n\nWhen you recode from another charset to 'HTML', beware that all\noccurrences of double quotes, ampersands, and left or right angle\nbrackets are translated into special sequences. However, in practice,\npeople often use ampersands and angle brackets in the other charset for\nintroducing HTML commands, compromising it: it is not pure HTML, not it\nis pure other charset. These particular translations can be rather\ninconvenient, they may be specifically inhibited through the command\noption '-d' (*note Mixed::).\n\nCodes not having a mnemonic entity are output by 'recode' using the\n'&#NNN;' notation, where NNN is a decimal representation of the UCS code\nvalue. When there is an entity name for a character, it is always\npreferred over a numeric character reference. ASCII printable\ncharacters are always generated directly. So is the newline. While\nreading HTML, 'recode' supports numeric character reference as alternate\nwritings, even when written as hexadecimal numbers, as in '�'.\nThis is documented in:\n\nhttp://www.w3.org/TR/REC-html40/intro/sgmltut.html#h-3.2.3\n\nWhen 'recode' translates to HTML, the translation occurs according to\nthe HTML level as selected by the goal charset. When translating from\nHTML, 'recode' not only accepts the character entity references known at\nthat level, but also those of all other levels, as well as a few\nalternative special sequences, to be forgiving to files using other HTML\nstandards.\n\nThe 'recode' program can be used to normalise an HTML file using\noldish conventions. For example, it accepts '&AE;', as this once was a\nvalid writing, somewhere. However, it should always produce 'Æ'\ninstead of '&AE;'. Yet, this is not completely true. If one does:\n\nrecode h3..h3 < INPUT\n\nthe operation will be optimised into a mere copy, and you can get '&AE;'\nthis way, if you had some in your input file. But if you explicitly\ndefeat the optimisation, like this maybe:\n\nrecode h3..u2,u2..h3 < INPUT\n\nthen '&AE;' should be normalised into 'Æ' by the operation.\n", "subsections": [] }, "File: recode.info, Node: LaTeX, Next: Texinfo, Prev: HTML, Up: Miscellaneous": { "content": "", "subsections": [ { "name": "12.2 LaTeX macro calls", "content": "" } ] }, "This charset is available in 'recode' under the name 'LaTeX' and has": { "content": "'ltex' as an alias. It is used for ASCII files coded to be read by\nLaTeX or, in certain cases, by TeX.\n\nWhenever you recode from another charset to 'LaTeX', beware that all\noccurrences of backslashes '\\' are translated into the string\n'\\backslash{}'. However, in practice, people often use backslashes in\nthe other charset for introducing TeX commands, compromising it: it is\nnot pure TeX, nor it is pure other charset. This translation of\nbackslashes into '\\backslash{}' can be rather inconvenient, it may be\ninhibited through the command option '-d' (*note Mixed::).\n", "subsections": [] }, "File: recode.info, Node: Texinfo, Next: Vietnamese, Prev: LaTeX, Up: Miscellaneous": { "content": "", "subsections": [ { "name": "12.3 GNU project documentation files", "content": "" } ] }, "This charset is available in 'recode' under the name 'Texinfo' and has": { "content": "'texi' and 'ti' for aliases. It is used by the GNU project for its\ndocumentation. Texinfo files may be converted into Info files by the\n'makeinfo' program and into nice printed manuals by the TeX system.\n\nEven if 'recode' may transform other charsets to Texinfo, it may not\nread Texinfo files yet. In these times, usages are also changing\nbetween versions of Texinfo, and 'recode' only partially succeeds in\ncorrectly following these changes. So, for now, Texinfo support in\n'recode' should be considered as work still in progress (!).\n", "subsections": [] }, "File: recode.info, Node: Vietnamese, Next: African, Prev: Texinfo, Up: Miscellaneous": { "content": "", "subsections": [ { "name": "12.4 Vietnamese charsets", "content": "" } ] }, "We are currently experimenting the implementation, in 'recode', of a few": { "content": "character sets and transliterated forms to handle the Vietnamese\nlanguage. They are quite briefly summarised, here.\n\n'TCVN'\nThe TCVN charset has an incomplete name. It might be one of the\nthree charset 'VN1', 'VN2' or 'VN3'. Yes 'VN2' might be a second\nversion of 'VISCII'. To be clarified.\n\n'VISCII'\nThis is an 8-bit character set which seems to be rather popular for\nwriting Vietnamese.\n\n'VPS'\nThis is an 8-bit character set for Vietnamese. No much reference.\n\n'VIQR'\nThe VIQR convention is a 7-bit, 'ASCII' transliteration for\nVietnamese.\n\n'VNI'\nThe VNI convention is a 8-bit, 'Latin-1' transliteration for\nVietnamese.\n\nStill lacking for Vietnamese in 'recode', are the charsets 'CP1129'\nand 'CP1258'.\n", "subsections": [] }, "File: recode.info, Node: African, Next: Others, Prev: Vietnamese, Up: Miscellaneous": { "content": "", "subsections": [ { "name": "12.5 African charsets", "content": "" } ] }, "Some African character sets are available for a few languages, when": { "content": "these are heavily used in countries where French is also currently\nspoken.\n\nOne African charset is usable for Bambara, Ewondo and Fulfude, as\nwell as for French. This charset is available in 'recode' under the\nname 'AFRFUL-102-BPIOCIL'. Accepted aliases are 'bambara', 'bra',\n'ewondo' and 'fulfude'. Transliterated forms of the same are available\nunder the name 'AFRFUL-103-BPIOCIL'. Accepted aliases are 't-bambara',\n't-bra', 't-ewondo' and 't-fulfude'.\n\nAnother African charset is usable for Lingala, Sango and Wolof, as\nwell as for French. This charset is available in 'recode' under the\nname 'AFRLIN-104-BPIOCIL'. Accepted aliases are 'lingala', 'lin',\n'sango' and 'wolof'. Transliterated forms of the same are available\nunder the name 'AFRLIN-105-BPIOCIL'. Accepted aliases are 't-lingala',\n't-lin', 't-sango' and 't-wolof'.\n\nTo ease exchange with 'ISO-8859-1', there is a charset conveying\ntransliterated forms for Latin-1 in a way which is compatible with the\nother African charsets in this series. This charset is available in\n'recode' under the name 'AFRL1-101-BPIOCIL'. Accepted aliases are\n't-fra' and 't-francais'.\n", "subsections": [] }, "File: recode.info, Node: Others, Next: Texte, Prev: African, Up: Miscellaneous": { "content": "", "subsections": [ { "name": "12.6 Cyrillic and other charsets", "content": "" } ] }, "The following Cyrillic charsets are already available in 'recode'": { "content": "through RFC 1345 tables: 'CP1251' with aliases '1251', ' ms-cyrl' and\n'windows-1251'; 'CSN369103' with aliases 'ISO-IR-139' and 'KOI8L2';\n'ECMA-cyrillic' with aliases 'ECMA-113', 'ECMA-113:1986' and\n'iso-ir-111', 'IBM880' with aliases '880', 'CP880' and\n'EBCDIC-Cyrillic'; 'INIS-cyrillic' with alias 'iso-ir-51'; 'ISO-8859-5'\nwith aliases 'cyrillic', ' ISO-8859-5:1988' and 'iso-ir-144'; 'KOI-7';\n'KOI-8' with alias 'GOST19768-74'; 'KOI8-R'; 'KOI8-RU' and finally\n'KOI8-U'.\n\nThere seems to remain some confusion in Roman charsets for Cyrillic\nlanguages, and because a few users requested it repeatedly, 'recode' now\noffers special services in that area. Consider these charsets as\nexperimental and debatable, as the extraneous tables describing them are\nstill a bit fuzzy or non-standard. Hopefully, in the long run, these\ncharsets will be covered in Keld Simonsen's works to the satisfaction of\neverybody, and this section will merely disappear.\n\n'KEYBCS2'\nThis charset is available under the name 'KEYBCS2', with\n'Kamenicky' as an accepted alias.\n\n'CORK'\nThis charset is available under the name 'CORK', with 'T1' as an\naccepted alias.\n\n'KOI-8CS2'\nThis charset is available under the name 'KOI-8CS2'.\n", "subsections": [] }, "File: recode.info, Node: Texte, Next: Mule, Prev: Others, Up: Miscellaneous": { "content": "", "subsections": [ { "name": "12.7 Easy French conventions", "content": "" } ] }, "This charset is available in 'recode' under the name 'Texte' and has": { "content": "'txte' for an alias. It is a seven bits code, identical to 'ASCII-BS',\nsave for French diacritics which are noted using a slightly different\nconvention.\n\nAt text entry time, these conventions provide a little speed up. At\nread time, they slightly improve the readability over a few alternate\nways of coding diacritics. Of course, it would better to have a\nspecialised keyboard to make direct eight bits entries and fonts for\nimmediately displaying eight bit ISO Latin-1 characters. But not\neverybody is so fortunate. In a few mailing environments, and sadly\nenough, it still happens that the eight bit is often willing-fully\ndestroyed.\n\nEasy French has been in use in France for a while. I only slightly\nadapted it (the diaeresis option) to make it more comfortable to several\nusages in Que'bec originating from Universite' de Montre'al. In fact, the\nmain problem for me was not to necessarily to invent Easy French, but to\nrecognise the \"best\" convention to use, (best is not being defined,\nhere) and to try to solve the main pitfalls associated with the selected\nconvention. Shortly said, we have:\n\n'e''\nfor 'e' (and some other vowels) with an acute accent,\n'e`'\nfor 'e' (and some other vowels) with a grave accent,\n'e^'\nfor 'e' (and some other vowels) with a circumflex accent,\n'e\"'\nfor 'e' (and some other vowels) with a diaeresis,\n'c,'\nfor 'c' with a cedilla.\n", "subsections": [] }, "There is no attempt at expressing the 'ae' and 'oe' diphthongs. French": { "content": "also uses tildes over 'n' and 'a', but seldomly, and this is not\nrepresented either. In some countries, ':' is used instead of '\"' to\nmark diaeresis. 'recode' supports only one convention per call,\ndepending on the '-c' option of the 'recode' command. French quotes\n(sometimes called \"angle quotes\") are noted the same way English quotes\nare noted in TeX, id est by '``' and ''''. No effort has been put to\npreserve Latin ligatures ('ae', 'oe') which are representable in several\nother charsets. So, these ligatures may be lost through Easy French\nconventions.\n\nThe convention is prone to losing information, because the diacritic\nmeaning overloads some characters that already have other uses. To\nalleviate this, some knowledge of the French language is boosted into\nthe recognition routines. So, the following subtleties are\nsystematically obeyed by the various recognisers.\n\n1. A comma which follows a 'c' is interpreted as a cedilla only if it\nis followed by one of the vowels 'a', 'o' or 'u'.\n\n2. A single quote which follows a 'e' does not necessarily means an\nacute accent if it is followed by a single other one. For example:\n\n'e''\nwill give an 'e' with an acute accent.\n'e'''\nwill give a simple 'e', with a closing quotation mark.\n'e''''\nwill give an 'e' with an acute accent, followed by a closing\nquotation mark.\n\nThere is a problem induced by this convention if there are English\nquotations with a French text. In sentences like:\n\nThere's a meeting at Archie's restaurant.\n\nthe single quotes will be mistaken twice for acute accents. So\nEnglish contractions and suffix possessives could be mangled.\n\n3. A double quote or colon, depending on '-c' option, which follows a\nvowel is interpreted as diaeresis only if it is followed by another\nletter. But there are in French several words that end with a\ndiaeresis, and the 'recode' library is aware of them. There are\nwords ending in \"igue\", either feminine words without a relative\nmasculine (besaigue\" and cigue\"), or feminine words with a relative\nmasculine(1) (aigue\", ambigue\", contigue\", exigue\", subaigue\" and\nsuraigue\"). There are also words not ending in \"igue\", but instead,\neither ending by \"i\"(2) (ai\", congai\", goi\", hai\"kai\", inoui\",\nsai\", samurai\", thai\" and tokai\"), ending by \"e\" (canoe\") or ending\nby \"u\"(3) (Esau\").\n\nJust to complete this topic, note that it would be wrong to make a\nrule for all words ending in \"igue\" as needing a diaerisis, as\nthere are counter-examples (becfigue, be`sigue, bigue, bordigue,\nbourdigue, brigue, contre-digue, digue, d'intrigue, fatigue, figue,\ngarrigue, gigue, igue, intrigue, ligue, prodigue, sarigue and\nzigue).\n\n---------- Footnotes ----------\n\n(1) There are supposed to be seven words in this case. So, one is\nmissing.\n\n(2) Look at one of the following sentences (the second has to be\ninterpreted with the '-c' option):\n\n\"Ai\"e! Voici le proble`me que j'ai\"\nAi:e! Voici le proble`me que j'ai:\n\nThere is an ambiguity between an ai\", the small animal, and the\nindicative future of avoir (first person singular), when followed by\nwhat could be a diaeresis mark. Hopefully, the case is solved by the\nfact that an apostrophe always precedes the verb and almost never the\nanimal.\n\n(3) I did not pay attention to proper nouns, but this one showed up\nas being fairly evident.\n", "subsections": [] }, "File: recode.info, Node: Mule, Prev: Texte, Up: Miscellaneous": { "content": "", "subsections": [ { "name": "12.8 Mule as a multiplexed charset", "content": "" } ] }, "This version of 'recode' barely starts supporting multiplexed or": { "content": "super-charsets, that is, those encoding methods by which a single text\nstream may contain a combination of more than one constituent charset.", "subsections": [] }, "The only multiplexed charset in 'recode' is 'Mule', and even then, it is": { "content": "only very partially implemented: the only correspondence available is\nwith 'Latin-1'. The author fastly implemented this only because he\nneeded this for himself. However, it is intended that Mule support to\nbecome more real in subsequent releases of 'recode'.\n\nMultiplexed charsets are not to be confused with mixed charset texts\n(*note Mixed::). For mixed charset input, the rules allowing to\ndistinguish which charset is current, at any given place, are kind of\ninformal, and driven from the semantics of what the file contains. On\nthe other side, multiplexed charsets are designed to be interpreted\nfairly precisely, and quite independently of any informational context.\n\nThe spelling 'Mule' originally stands for 'multilingual\nenhancement to GNU Emacs', it is the result of a collective effort\norchestrated by Handa Ken'ichi since 1993. When 'Mule' got rewritten in\nthe main development stream of GNU Emacs 20, the FSF renamed it 'MULE',\nmeaning 'multilingual environment in GNU Emacs'. Even if the\ncharset 'Mule' is meant to stay internal to GNU Emacs, it sometimes\nbreaks loose in external files, and as a consequence, a recoding tool is\nsometimes needed. Within Emacs, 'Mule' comes with 'Leim', which stands\nfor 'libraries of emacs input methods'. One of these libraries\nis named 'quail'(1).\n\n---------- Footnotes ----------\n\n(1) Usually, quail means quail egg in Japanese, while egg alone is\nusually chicken egg. Both quail egg and chicken egg are popular food in", "subsections": [] }, "Japan. The 'quail' input system has been named because it is smaller": { "content": "that the previous 'EGG' system. As for 'EGG', it is the translation of\n'TAMAGO'. This word comes from the Japanese sentence 'takusan\nmatasete gomennasai', meaning 'sorry to have let you wait so long'.\nOf course, the publication of 'EGG' has been delayed many times...\n(Story by Takahashi Naoto)\n", "subsections": [] }, "File: recode.info, Node: Surfaces, Next: Internals, Prev: Miscellaneous, Up: Top": { "content": "", "subsections": [] }, "13 All about surfaces": { "content": "The \"trivial surface\" consists of using a fixed number of bits (often\neight) for each character, the bits together hold the integer value of\nthe index for the character in its charset table. There are many kinds\nof surfaces, beyond the trivial one, all having the purpose of\nincreasing selected qualities for the storage or transmission. For\nexample, surfaces might increase the resistance to channel limits\n('Base64'), the transmission speed ('gzip'), the information privacy\n('DES'), the conformance to operating system conventions ('CR-LF'), the\nblocking into records ('VB'), and surely other things as well(1). Many\nsurfaces may be applied to a stream of characters from a charset, the\norder of application of surfaces is important, and surfaces should be\nremoved in the reverse order of their application.\n\nEven if surfaces may generally be applied to various charsets, some\nsurfaces were specifically designed for a particular charset, and would\nnot make much sense if applied to other charsets. In such cases, these\nconceptual surfaces have been implemented as 'recode' charsets, instead\nof as surfaces. This choice yields to cleaner syntax and usage. *Note\nUniversal::.\n\nSurfaces are implemented within 'recode' as special charsets which\nmay only transform to or from the 'data' or 'tree' special charsets.", "subsections": [] }, "Clever users may use this knowledge for writing surface names in": { "content": "requests exactly as if they were pure charsets, when the only need is to\nchange surfaces without any kind of recoding between real charsets. In\nsuch contexts, either 'data' or 'tree' may also be used as if it were\nsome kind of generic, anonymous charset: the request 'data..SURFACE'\nmerely adds the given SURFACE, while the request 'SURFACE..data' removes\nit.\n\nThe 'recode' library distinguishes between mere data surfaces, and\nstructural surfaces, also called tree surfaces for short. Structural\nsurfaces might allow, in the long run, transformations between a few\nspecialised representations of structural information like MIME parts,", "subsections": [] }, "Perl or Python initialisers, LISP S-expressions, XML, Emacs outlines,": { "content": "etc.\n\nWe are still experimenting with surfaces in 'recode'. The concept\nopens the doors to many avenues; it is not clear yet which ones are\nworth pursuing, and which should be abandoned. In particular,\nimplementation of structural surfaces is barely starting, there is not\neven a commitment that tree surfaces will stay in 'recode', if they do\nprove to be more cumbersome than useful. This chapter presents all\nsurfaces currently available.\n\n* Menu:\n\n* Permutations:: Permuting groups of bytes\n* End lines:: Representation for end of lines\n* MIME:: MIME contents encodings\n* Dump:: Interpreted character dumps\n* Test:: Artificial data for testing\n\n---------- Footnotes ----------\n\n(1) These are mere examples to explain the concept, 'recode' only has\n'Base64' and 'CR-LF', actually.\n", "subsections": [] }, "File: recode.info, Node: Permutations, Next: End lines, Prev: Surfaces, Up: Surfaces": { "content": "", "subsections": [ { "name": "13.1 Permuting groups of bytes", "content": "A permutation is a surface transformation which reorders groups of\neight-bit bytes. A 21 permutation exchanges pairs of successive\nbytes. If the text contains an odd number of bytes, the last byte is\nmerely copied. An 4321 permutation inverts the order of quadruples of\nbytes. If the text does not contains a multiple of four bytes, the\nremaining bytes are nevertheless permuted as 321 if there are three\nbytes, 21 if there are two bytes, or merely copied otherwise.\n\n'21'\nThis surface is available in 'recode' under the name\n'21-Permutation' and has 'swabytes' for an alias.\n\n'4321'\nThis surface is available in 'recode' under the name\n'4321-Permutation'.\n" } ] }, "File: recode.info, Node: End lines, Next: MIME, Prev: Permutations, Up: Surfaces": { "content": "", "subsections": [ { "name": "13.2 Representation for end of lines", "content": "" } ] }, "The same charset might slightly differ, from one system to another, for": { "content": "the single fact that end of lines are not represented identically on all\nsystems. The representation for an end of line within 'recode' is the\n'ASCII' or 'UCS' code with value 10, or 'LF'. Other conventions for\nrepresenting end of lines are available through surfaces.\n\n'CR'\nThis convention is popular on Apple's Macintosh machines. When\nthis surface is applied, each line is terminated by 'CR', which has\n'ASCII' value 13. Unless the library is operating in strict mode,\nadding or removing the surface will in fact exchange 'CR' and\n'LF', for better reversibility. However, in strict mode, the\nexchange does not happen, any 'CR' will be copied verbatim while\napplying the surface, and any 'LF' will be copied verbatim while\nremoving it.\n\nThis surface is available in 'recode' under the name 'CR', it does\nnot have any aliases. This is the implied surface for the Apple\nMacintosh related charsets.\n\n'CR-LF'\nThis convention is popular on Microsoft systems running on IBM PCs\nand compatible. When this surface is applied, each line is\nterminated by a sequence of two characters: one 'CR' followed by\none 'LF', in that order.\n\nFor compatibility with oldish MS-DOS systems, removing a 'CR-LF'\nsurface will discard the first encountered 'C-z', which has 'ASCII'\nvalue 26, and everything following it in the text. Adding this\nsurface will not, however, append a 'C-z' to the result.\n\nThis surface is available in 'recode' under the name 'CR-LF' and\nhas 'cl' for an alias. This is the implied surface for the IBM or\nMicrosoft related charsets or code pages.\n\nSome other charsets might have their own representation for an end of\nline, which is different from 'LF'. For example, this is the case of\nvarious 'EBCDIC' charsets, or 'Icon-QNX'. The recoding of end of lines\nis intimately tied into such charsets, it is not available separately as\nsurfaces.\n", "subsections": [] }, "File: recode.info, Node: MIME, Next: Dump, Prev: End lines, Up: Surfaces": { "content": "", "subsections": [ { "name": "13.3 MIME contents encodings", "content": "RFC 2045 defines two 7-bit surfaces, meant to prepare 8-bit messages for\ntransmission. Base64 is especially usable for binary entities, while" } ] }, "Quoted-Printable is especially usable for text entities, in those case": { "content": "the lower 128 characters of the underlying charset coincide with ASCII.\n\n'Base64'\nThis surface is available in 'recode' under the name 'Base64', with\n'b64' and '64' as acceptable aliases.\n\n'Quoted-Printable'\nThis surface is available in 'recode' under the name\n'Quoted-Printable', with 'quote-printable' and 'QP' as acceptable\naliases.\n\nNote that 'UTF-7', which may be also considered as a MIME surface, is\nprovided as a genuine charset instead, as it necessary relates to\n'UCS-2' and nothing else. *Note UTF-7::.\n\nA little historical note, also showing the three levels of acceptance\nof Internet standards. MIME changed from a \"Proposed Standard\"\n(RFC 1341-1344, 1992) to a \"Draft Standard\" (RFC 1521-1523) in 1993, and\nwas recycled as a \"Draft Standard\" in 1996-11. It is not yet a \"Full\nStandard\".\n", "subsections": [] }, "File: recode.info, Node: Dump, Next: Test, Prev: MIME, Up: Surfaces": { "content": "", "subsections": [ { "name": "13.4 Interpreted character dumps", "content": "" } ] }, "Dumps are surfaces meant to express, in ways which are a bit more": { "content": "readable, the bit patterns used to represent characters. They allow the\ninspection or debugging of character streams, but also, they may assist\na bit the production of C source code which, once compiled, would hold\nin memory a copy of the original coding. However, 'recode' does not\nattempt, in any way, to produce complete C source files in dumps. User\nhand editing or 'Makefile' trickery is still needed for adding missing\nlines. Dumps may be given in decimal, hexadecimal and octal, and be\nbased over chunks of either one, two or four eight-bit bytes.", "subsections": [] }, "Formatting has been chosen to respect the C language syntax for number": { "content": "constants, with commas and newlines inserted appropriately.\n\nHowever, when dumping two or four byte chunks, the last chunk may be\nincomplete. This is observable through the usage of narrower expression\nfor that last chunk only. Such a shorter chunk would not be compiled\nproperly within a C initialiser, as all members of an array share a\nsingle type, and so, have identical sizes.\n\n'Octal-1'\nThis surface corresponds to an octal expression of each input byte.\n\nIt is available in 'recode' under the name 'Octal-1', with 'o1' and\n'o' as acceptable aliases.\n\n'Octal-2'\nThis surface corresponds to an octal expression of each pair of\ninput bytes, except for the last pair, which may be short.\n\nIt is available in 'recode' under the name 'Octal-2' and has 'o2'\nfor an alias.\n\n'Octal-4'\nThis surface corresponds to an octal expression of each quadruple\nof input bytes, except for the last quadruple, which may be short.\n\nIt is available in 'recode' under the name 'Octal-4' and has 'o4'\nfor an alias.\n\n'Decimal-1'\nThis surface corresponds to an decimal expression of each input\nbyte.\n\nIt is available in 'recode' under the name 'Decimal-1', with 'd1'\nand 'd' as acceptable aliases.\n\n'Decimal-2'\nThis surface corresponds to an decimal expression of each pair of\ninput bytes, except for the last pair, which may be short.\n\nIt is available in 'recode' under the name 'Decimal-2' and has 'd2'\nfor an alias.\n\n'Decimal-4'\nThis surface corresponds to an decimal expression of each quadruple\nof input bytes, except for the last quadruple, which may be short.\n\nIt is available in 'recode' under the name 'Decimal-4' and has 'd4'\nfor an alias.\n\n'Hexadecimal-1'\nThis surface corresponds to an hexadecimal expression of each input\nbyte.\n\nIt is available in 'recode' under the name 'Hexadecimal-1', with\n'x1' and 'x' as acceptable aliases.\n\n'Hexadecimal-2'\nThis surface corresponds to an hexadecimal expression of each pair\nof input bytes, except for the last pair, which may be short.\n\nIt is available in 'recode' under the name 'Hexadecimal-2', with\n'x2' for an alias.\n\n'Hexadecimal-4'\nThis surface corresponds to an hexadecimal expression of each\nquadruple of input bytes, except for the last quadruple, which may\nbe short.\n\nIt is available in 'recode' under the name 'Hexadecimal-4', with\n'x4' for an alias.\n\nWhen removing a dump surface, that is, when reading a dump results\nback into a sequence of bytes, the narrower expression for a short last\nchunk is recognised, so dumping is a fully reversible operation.", "subsections": [] }, "However, in case you want to produce dumps by other means than through": { "content": "'recode', beware that for decimal dumps, the library has to rely on the\nnumber of spaces to establish the original byte size of the chunk.\n\nAlthough the library might report reversibility errors, removing a\ndump surface is a rather forgiving process: one may mix bases, group a\nvariable number of data per source line, or use shorter chunks in places\nother than at the far end. Also, source lines not beginning with a\nnumber are skipped. So, 'recode' should often be able to read a whole C\nheader file, wrapping the results of a previous dump, and regenerate the\noriginal byte string.\n", "subsections": [] }, "File: recode.info, Node: Test, Prev: Dump, Up: Surfaces": { "content": "", "subsections": [ { "name": "13.5 Artificial data for testing", "content": "A few pseudo-surfaces exist to generate debugging data out of thin air." } ] }, "These surfaces are only meant for the expert 'recode' user, and are only": { "content": "useful in a few contexts, like for generating binary permutations from\nthe recoding or acting on them.\n\nDebugging surfaces, when removed, insert their generated data at\nthe beginning of the output stream, and copy all the input stream after\nthe generated data, unchanged. This strange removal constraint comes\nfrom the fact that debugging surfaces are usually specified in the\nbefore position instead of the after position within a request.", "subsections": [] }, "With debugging surfaces, one often recodes file '/dev/null' in filter": { "content": "mode. Specifying many debugging surfaces at once has an accumulation\neffect on the output, and since surfaces are removed from right to left,\neach generating its data at the beginning of previous output, the net\neffect is an impression that debugging surfaces are generated from\nleft to right, each appending to the result of the previous. In any\ncase, any real input data gets appended after what was generated.\n\n'test7'\nWhen removed, this surface produces 128 single bytes, the first\nhaving value 0, the second having value 1, and so forth until all\n128 values have been generated.\n\n'test8'\nWhen removed, this surface produces 256 single bytes, the first\nhaving value 0, the second having value 1, and so forth until all\n256 values have been generated.\n\n'test15'\nWhen removed, this surface produces 64509 double bytes, the first\nhaving value 0, the second having value 1, and so forth until all\nvalues have been generated, but excluding risky 'UCS-2' values,\nlike all codes from the surrogate 'UCS-2' area (for 'UTF-16'), the\nbyte order mark, and values known as invalid 'UCS-2'.\n\n'test16'\nWhen removed, this surface produces 65536 double bytes, the first\nhaving value 0, the second having value 1, and so forth until all\n65536 values have been generated.\n\nAs an example, the command 'recode l5/test8..dump < /dev/null' is a\nconvoluted way to produce an output similar to 'recode -lf l5'. It says\nto generate all possible 256 bytes and interpret them as 'ISO-8859-9'\ncodes, while converting them to 'UCS-2'. Resulting 'UCS-2' characters\nare dumped one per line, accompanied with their explicative name.\n", "subsections": [] }, "File: recode.info, Node: Internals, Next: Concept Index, Prev: Surfaces, Up: Top": { "content": "", "subsections": [] }, "14 Internal aspects": { "content": "", "subsections": [] }, "The incoming explanations of the internals of 'recode' should help": { "content": "people who want to dive into 'recode' sources for adding new charsets.", "subsections": [] }, "Adding new charsets does not require much knowledge about the overall": { "content": "organisation of 'recode'. You can rather concentrate of your new\ncharset, letting the remainder of the 'recode' mechanics take care of\ninterconnecting it with all others charsets.\n\nIf you intend to play seriously at modifying 'recode', beware that\nyou may need some other GNU tools which were not required when you first\ninstalling 'recode'. If you modify or create any '.l' file, then you\nneed Flex, and some better 'awk' like 'mawk', GNU 'awk', or 'nawk'. If\nyou modify the documentation (and you should!), you need 'makeinfo'. If\nyou are really audacious, you may also want Perl for modifying tabular\nprocessing, then 'm4', Autoconf, Automake and 'libtool' for adjusting\nconfiguration matters.\n\n* Menu:\n\n* Main flow:: Overall organisation\n* New charsets:: Adding new charsets\n* New surfaces:: Adding new surfaces\n* Design:: Comments on the library design\n", "subsections": [] }, "File: recode.info, Node: Main flow, Next: New charsets, Prev: Internals, Up: Internals": { "content": "", "subsections": [ { "name": "14.1 Overall organisation", "content": "" } ] }, "The 'recode' mechanics slowly evolved for many years, and it would be": { "content": "tedious to explain all problems I met and mistakes I did all along,\nyielding the current behaviour. Surely, one of the key choices was to\nstop trying to do all conversions in memory, one line or one buffer at a\ntime. It has been fruitful to use the character stream paradigm, and\nthe elementary recoding steps now convert a whole stream to another.", "subsections": [] }, "Most of the control complexity in 'recode' exists so that each": { "content": "elementary recoding step stays simple, making easier to add new ones.", "subsections": [] }, "The whole point of 'recode', as I see it, is providing a comfortable": { "content": "nest for growing new charset conversions.\n\nThe main 'recode' driver constructs, while initialising all\nconversion modules, a table giving all the conversion routines available\n(\"single step\"s) and for each, the starting charset and the ending\ncharset. If we consider these charsets as being the nodes of a directed\ngraph, each single step may be considered as oriented arc from one node\nto the other. A cost is attributed to each arc: for example, a high\npenalty is given to single steps which are prone to losing characters, a\nlower penalty is given to those which need studying more than one input\ncharacter for producing an output character, etc.\n\nGiven a starting code and a goal code, 'recode' computes the most\neconomical route through the elementary recodings, that is, the best\nsequence of conversions that will transform the input charset into the\nfinal charset. To speed up execution, 'recode' looks for subsequences\nof conversions which are simple enough to be merged, and then\ndynamically creates new single steps to represent these mergings.\n\nA \"double step\" in 'recode' is a special concept representing a\nsequence of two single steps, the output of the first single step being\nthe special charset 'UCS-2', the input of the second single step being\nalso 'UCS-2'. Special 'recode' machinery dynamically produces\nefficient, reversible, merge-able single steps out of these double\nsteps.\n\nI made some statistics about how many internal recoding steps are\nrequired between any two charsets chosen at random. The initial\nrecoding layout, before optimisation, always uses between 1 and 5 steps.", "subsections": [] }, "Optimisation could sometimes produce mere copies, which are counted as": { "content": "no steps at all. In other cases, optimisation is unable to save any\nstep. The number of steps after optimisation is currently between 0 and\n5 steps. Of course, the expected number of steps is affected by\noptimisation: it drops from 2.8 to 1.8. This means that 'recode' uses a\ntheoretical average of a bit less than one step per recoding job. This\nlooks good. This was computed using reversible recodings. In strict\nmode, optimisation might be defeated somewhat. Number of steps run\nbetween 1 and 6, both before and after optimisation, and the expected\nnumber of steps decreases by a lesser amount, going from 2.2 to 1.3.\nThis is still manageable.\n", "subsections": [] }, "File: recode.info, Node: New charsets, Next: New surfaces, Prev: Main flow, Up: Internals": { "content": "", "subsections": [ { "name": "14.2 Adding new charsets", "content": "" } ] }, "The main part of 'recode' is written in C, as are most single steps. A": { "content": "few single steps need to recognise sequences of multiple characters,\nthey are often better written in Flex. It is easy for a programmer to\nadd a new charset to 'recode'. All it requires is making a few\nfunctions kept in a single '.c' file, adjusting 'Makefile.am' and\nremaking 'recode'.\n\nOne of the function should convert from any previous charset to the\nnew one. Any previous charset will do, but try to select it so you will\nnot lose too much information while converting. The other function\nshould convert from the new charset to any older one. You do not have\nto select the same old charset than what you selected for the previous\nroutine. Once again, select any charset for which you will not lose too\nmuch information while converting.\n\nIf, for any of these two functions, you have to read multiple bytes\nof the old charset before recognising the character to produce, you\nmight prefer programming it in Flex in a separate '.l' file. Prototype\nyour C or Flex files after one of those which exist already, so to keep\nthe sources uniform. Besides, at 'make' time, all '.l' files are\nautomatically merged into a single big one by the script 'mergelex.awk'.\n\nThere are a few hidden rules about how to write new 'recode' modules,\nfor allowing the automatic creation of 'decsteps.h' and 'initsteps.h' at\n'make' time, or the proper merging of all Flex files. Mimetism is a\nsimple approach which relieves me of explaining all these rules! Start\nwith a module closely resembling what you intend to do. Here is some\nadvice for picking up a model. First decide if your new charset module\nis to be be driven by algorithms rather than by tables. For algorithmic\nrecodings, see 'iconqnx.c' for C code, or 'txtelat1.l' for Flex code.", "subsections": [] }, "For table driven recodings, see 'ebcdic.c' for one-to-one style": { "content": "recodings, 'lat1html.c' for one-to-many style recodings, or 'atarist.c'\nfor double-step style recodings. Just select an example from the style\nthat better fits your application.\n\nEach of your source files should have its own initialisation\nfunction, named 'moduleCHARSET', which is meant to be executed\nquickly once, prior to any recoding. It should declare the name of\nyour charsets and the single steps (or elementary recodings) you\nprovide, by calling 'declarestep' one or more times. Besides the\ncharset names, 'declarestep' expects a description of the recoding\nquality (see 'recodext.h') and two functions you also provide.\n\nThe first such function has the purpose of allocating structures,\npre-conditioning conversion tables, etc. It is also the way of further\nmodifying the 'STEP' structure. This function is executed if and only\nif the single step is retained in an actual recoding sequence. If you\ndo not need such delayed initialisation, merely use 'NULL' for the\nfunction argument.\n\nThe second function executes the elementary recoding on a whole file.\nThere are a few cases when you can spare writing this function:\n\n* Some single steps do nothing else than a pure copy of the input\nonto the output, in this case, you can use the predefined function\n'fileonetoone', while having a delayed initialisation for\npresetting the 'STEP' field 'onetoone' to the predefined value\n'onetosame'.\n\n* Some single steps are driven by a table which recodes one character\ninto another; if the recoding does nothing else, you can use the\npredefined function 'fileonetoone', while having a delayed\ninitialisation for presetting the 'STEP' field 'onetoone' with\nyour table.\n\n* Some single steps are driven by a table which recodes one character\ninto a string; if the recoding does nothing else, you can use the\npredefined function 'fileonetomany', while having a delayed\ninitialisation for presetting the 'STEP' field 'onetomany' with\nyour table.\n\nIf you have a recoding table handy in a suitable format but do not\nuse one of the predefined recoding functions, it is still a good idea to\nuse a delayed initialisation to save it anyway, because 'recode' option\n'-h' will take advantage of this information when available.\n\nFinally, edit 'Makefile.am' to add the source file name of your\nroutines to the 'CSTEPS' or 'LSTEPS' macro definition, depending on\nthe fact your routines is written in C or in Flex.\n", "subsections": [] }, "File: recode.info, Node: New surfaces, Next: Design, Prev: New charsets, Up: Internals": { "content": "", "subsections": [ { "name": "14.3 Adding new surfaces", "content": "" } ] }, "Adding a new surface is technically quite similar to adding a new": { "content": "charset. *Note New charsets::. A surface is provided as a set of two\ntransformations: one from the predefined special charset 'data' or\n'tree' to the new surface, meant to apply the surface, the other from\nthe new surface to the predefined special charset 'data' or 'tree',\nmeant to remove the surface.\n\nInternally in 'recode', function 'declarestep' especially recognises\nwhen a charset is so related to 'data' or 'tree', and then takes\nappropriate actions so that charset gets indeed installed as a surface.\n", "subsections": [] }, "File: recode.info, Node: Design, Prev: New surfaces, Up: Internals": { "content": "", "subsections": [ { "name": "14.4 Comments on the library design", "content": "* Why a shared library?\n\nThere are many different approaches to reduce system requirements\nto handle all tables needed in the 'recode' library. One of them\nis to have the tables in an external format and only read them in\non demand. After having pondered this for a while, I finally\ndecided against it, mainly because it involves its own kind of\ninstallation complexity, and it is not clear to me that it would be\nas interesting as I first imagined.\n\nIt looks more efficient to see all tables and algorithms already\nmapped into virtual memory from the start of the execution, yet not\nloaded in actual memory, than to go through many disk accesses for\nopening various data files once the program is already started, as\nthis would be needed with other solutions. Using a shared library\nalso has the indirect effect of making various algorithms handily\navailable, right in the same modules providing the tables. This\nalleviates much the burden of the maintenance.\n\nOf course, I would like to later make an exception for only a few\ntables, built locally by users for their own particular needs once\n'recode' is installed. 'recode' should just go and fetch them.\nBut I do not perceive this as very urgent, yet useful enough to be\nworth implementing.\n\nCurrently, all tables needed for recoding are precompiled into\nbinaries, and all these binaries are then made into a shared\nlibrary. As an initial step, I turned 'recode' into a main program\nand a non-shared library, this allowed me to tidy up the API, get\nrid of all global variables, etc. It required a surprising amount\nof program source massaging. But once this cleaned enough, it was\neasy to use Gordon Matzigkeit's 'libtool' package, and take\nadvantage of the Automake interface to neatly turn the non-shared\nlibrary into a shared one.\n\nSites linking with the 'recode' library, whose system does not\nsupport any form of shared libraries, might end up with bulky\nexecutables. Surely, the 'recode' library will have to be used\nstatically, and might not very nicely usable on such systems. It\nseems that progress has a price for those being slow at it.\n\nThere is a locality problem I did not address yet. Currently, the\n'recode' library takes many cycles to initialise itself, calling\neach module in turn for it to set up associated knowledge about\ncharsets, aliases, elementary steps, recoding weights, etc.\nThen, the recoding sequence is decided out of the command given.\nI would not be surprised if initialisation was taking a perceivable\nfraction of a second on slower machines. One thing to do, most\nprobably not right in version 3.5, but the version after, would\nhave 'recode' to pre-load all tables and dump them at installation\ntime. The result would then be compiled and added to the library.\nThis would spare many initialisation cycles, but more importantly,\nwould avoid calling all library modules, scattered through the\nvirtual memory, and so, possibly causing many spurious page\nexceptions each time the initialisation is requested, at least once\nper program execution.\n\n* Why not a central charset?\n\nIt would be simpler, and I would like, if something like ISO 10646\nwas used as a turning template for all charsets in 'recode'. Even\nif I think it could help to a certain extent, I'm still not fully\nsure it would be sufficient in all cases. Moreover, some people\ndisagree about using ISO 10646 as the central charset, to the point\nI cannot totally ignore them, and surely, 'recode' is not a mean\nfor me to force my own opinions on people. I would like that\n'recode' be practical more than dogmatic, and reflect usage more\nthan religions.\n\nCurrently, if you ask 'recode' to go from CHARSET1 to CHARSET2\nchosen at random, it is highly probable that the best path will be\nquickly found as:\n\nCHARSET1..UCS-2..CHARSET2\n\nThat is, it will almost always use the 'UCS' as a trampoline\nbetween charsets. However, 'UCS-2' will be immediately be\noptimised out, and CHARSET1..CHARSET2 will often be performed in a\nsingle step through a permutation table generated on the fly for\nthe circumstance (1).\n\nIn those few cases where 'UCS-2' is not selected as a conceptual\nintermediate, I plan to study if it could be made so. But I guess\nsome cases will remain where 'UCS-2' is not a proper choice. Even\nif 'UCS' is often the good choice, I do not intend to forcefully\nrestrain 'recode' around 'UCS-2' (nor 'UCS-4') for now. We might\ncome to that one day, but it will come out of the natural evolution\nof 'recode'. It will then reflect a fact, rather than a preset\ndogma.\n\n* Why not 'iconv'?\n\nThe 'iconv' routine and library allows for converting characters\nfrom an input buffer to an input buffer, synchronously advancing\nboth buffer cursors. If the output buffer is not big enough to\nreceive all of the conversion, the routine returns with the input\ncursor set at the position where the conversion could later be\nresumed, and the output cursor set to indicate until where the\noutput buffer has been filled. Despite this scheme is simple and\nnice, the 'recode' library does not offer it currently. Why not?\n\nWhen long sequences of decodings, stepwise recodings, and\nre-encodings are involved, as it happens in true life,\nsynchronising the input buffer back to where it should have\nstopped, when the output buffer becomes full, is a difficult\nproblem. Oh, we could make it simpler at the expense of loosing\nspace or speed: by inserting markers between each input character\nand counting them at the output end; by processing only one\ncharacter in a time through the whole sequence; by repeatedly\nattempting to recode various subsets of the input buffer, binary\nsearching on their length until the output just fits. The overhead\nof such solutions looks fully prohibitive to me, and the gain very\nminimal. I do not see a real advantage, nowadays, imposing a fixed\nlength to an output buffer. It makes things so much simpler and\nefficient to just let the output buffer size float a bit.\n\nOf course, if the above problem was solved, the 'iconv' library\nshould be easily emulated, given that 'recode' has similar\nknowledge about charsets, of course. This either solved or not,\nthe 'iconv' program remains trivial (given similar knowledge about\ncharsets). I also presume that the 'genxlt' program would be easy\ntoo, but I do not have enough detailed specifications of it to be\nsure.\n\nA lot of years ago, 'recode' was using a similar scheme, and I\nfound it rather hard to manage for some cases. I rethought the\noverall structure of 'recode' for getting away from that scheme,\nand never regretted it. I perceive 'iconv' as an artificial\nsolution which surely has some elegances and virtues, but I do not\nfind it really useful as it stands: one always has to wrap 'iconv'\ninto something more refined, extending it for real cases. From\npast experience, I think it is unduly hard to fully implement this\nscheme. It would be awkward that we do contortions for the sole\npurpose of implementing exactly its specification, without real,\nfairly sounded reasons (other then the fact some people once\nthought it was worth standardising). It is much better to\nimmediately aim for the refinement we need, without uselessly\nforcing us into the dubious detour 'iconv' represents.\n\nSome may argue that if 'recode' was using a comprehensive charset\nas a turning template, as discussed in a previous point, this would\nmake 'iconv' easier to implement. Some may be tempted to say that\nthe cases which are hard to handle are not really needed, nor\ninteresting, anyway. I feel and fear a bit some pressure wanting\nthat 'recode' be split into the part that well fits the 'iconv'\nmodel, and the part that does not fit, considering this second part\nless important, with the idea of dropping it one of these days,\nmaybe. My guess is that users of the 'recode' library, whatever\nits form, would not like to have such arbitrary limitations. In\nthe long run, we should not have to explain to our users that some\nrecodings may not be made available just because they do not fit\nthe simple model we had in mind when we did it. Instead, we should\ntry to stay opened to the difficulties of real life. There is\nstill a lot of complex needs for Asian people, say, that 'recode'\ndoes not currently address, while it should. Not only the doors\nshould stay open, but we should force them wider!\n\n---------- Footnotes ----------\n\n(1) If strict mapping is requested, another efficient device will be\nused instead of a permutation.\n" } ] }, "File: recode.info, Node: Concept Index, Next: Option Index, Prev: Internals, Up: Top": { "content": "", "subsections": [] }, "Concept Index": { "content": "* Menu:\n\n* abbreviated names for charsets and surfaces: Requests. (line 89)\n* adding new charsets: New charsets. (line 6)\n* adding new surfaces: New surfaces. (line 6)\n* African charsets: African. (line 6)\n* aliases: Requests. (line 81)\n* alternate names for charsets and surfaces: Requests. (line 81)\n* ambiguous output, error message: Errors. (line 33)\n* ASCII table, recreating with recode: ASCII. (line 12)\n* average number of recoding steps: Main flow. (line 41)\n* bool data type: Outer level. (line 31)\n* box-drawing characters: Recoding. (line 16)\n* bug reports, where to send: Contributing. (line 38)\n* byte order mark: UCS-2. (line 12)\n* byte order swapping: Permutations. (line 6)\n* caret ASCII code: CDC-NOS. (line 9)\n* CDC charsets: CDC. (line 6)\n* CDC Display Code, a table: Display Code. (line 6)\n* chaining of charsets in a request: Requests. (line 23)\n* character entities: HTML. (line 6)\n* character entity references: HTML. (line 6)\n* character mnemonics, documentation: Tabular. (line 6)\n* character streams, description: dump-with-names. (line 6)\n* charset level functions: Charset level. (line 6)\n* charset names, valid characters: Requests. (line 10)\n* charset, default: Requests. (line 105)\n* charset, pure: Surface overview. (line 17)\n* charset, what it is: Introduction. (line 15)\n* charsets for CDC machines: CDC. (line 6)\n* charsets, aliases: Requests. (line 81)\n* charsets, chaining in a request: Requests. (line 23)\n* charsets, guessing: Listings. (line 52)\n* charsets, overview: Charset overview. (line 6)\n* chset tools: Tabular. (line 6)\n* codepages: IBM and MS. (line 6)\n* combining characters: UCS-2. (line 33)\n* commutativity of surfaces: Requests. (line 57)\n* contributing charsets: Contributing. (line 6)\n* conversions, unavailable: Charset overview. (line 32)\n* convert a subset of characters: Mixed. (line 20)\n* convert strings and comments: Mixed. (line 39)\n* copyright conditions, printing: Listings. (line 19)\n* counting characters: count-characters. (line 6)\n* CR surface, in Macintosh charsets: Apple-Mac. (line 15)\n* CR-LF surface, in IBM-PC charsets: IBM-PC. (line 30)\n* Ctrl-Z, discarding: End lines. (line 32)\n* Cyrillic charsets: Others. (line 6)\n* debugging surfaces: Test. (line 11)\n* default charset: Requests. (line 105)\n* description of individual characters: dump-with-names. (line 6)\n* details about recoding: Recoding. (line 35)\n* deviations from RFC 1345: Tabular. (line 13)\n* diacritics and underlines, removing: flat. (line 8)\n* diacritics, with ASCII-BS charset: ASCII-BS. (line 9)\n* diaeresis: Recoding. (line 11)\n* disable map filling: Reversibility. (line 49)\n* double step: Main flow. (line 34)\n* dumping characters: Dump. (line 6)\n* dumping characters, with description: dump-with-names. (line 6)\n* Easy French: Texte. (line 20)\n* EBCDIC charsets: EBCDIC. (line 6)\n* end of line format: End lines. (line 6)\n* endiannes, changing: Permutations. (line 6)\n* entities: HTML. (line 6)\n* error handling: Errors. (line 6)\n* error level threshold: Errors. (line 100)\n* error messages: Errors. (line 6)\n* error messages, suppressing: Reversibility. (line 37)\n* exceptions to available conversions: Charset overview. (line 32)\n* file sequencing: Sequencing. (line 28)\n* file time stamps: Recoding. (line 26)\n* filter operation: Synopsis. (line 27)\n* force recoding: Reversibility. (line 10)\n* French description of charsets: Listings. (line 186)\n* guessing charsets: Listings. (line 52)\n* Haible, Bruno: libiconv. (line 6)\n* handling errors: Errors. (line 6)\n* help page, printing: Listings. (line 14)\n* HTML: HTML. (line 6)\n* HTML normalization: HTML. (line 107)\n* IBM codepages: IBM and MS. (line 6)\n* IBM graphics characters: Recoding. (line 16)\n* iconv: Design. (line 98)\n* iconv library: libiconv. (line 6)\n* identifying subsets in charsets: Listings. (line 198)\n* ignore charsets: Recoding. (line 60)\n* implied surfaces: Requests. (line 70)\n* impossible conversions: Charset overview. (line 32)\n* information about charsets: Listings. (line 141)\n* initialisation functions, outer: Outer level. (line 84)\n* initialisation functions, request: Request level. (line 42)\n* initialisation functions, task: Task level. (line 52)\n* interface, with iconv library: libiconv. (line 6)\n* intermediate charsets: Requests. (line 23)\n* internal functions: Charset level. (line 6)\n* internal recoding bug, error message: Errors. (line 88)\n* internals: Internals. (line 6)\n* invalid input, error message: Errors. (line 65)\n* invocation of recode, synopsis: Synopsis. (line 6)\n* irreversible recoding: Reversibility. (line 10)\n* ISO 10646: Universal. (line 6)\n* languages, programming: Listings. (line 26)\n* LaTeX files: LaTeX. (line 6)\n* Latin charsets: ISO 8859. (line 6)\n* Latin-1 table, recreating with recode: ISO 8859. (line 38)\n* letter case, in charset and surface names: Requests. (line 94)\n* libiconv: libiconv. (line 6)\n* library, iconv: libiconv. (line 6)\n* listing charsets: Listings. (line 141)\n* Macintosh charset: Apple-Mac. (line 6)\n* map filling: Reversibility. (line 98)\n* map filling, disable: Reversibility. (line 49)\n* markup language: HTML. (line 6)\n* memory sequencing: Sequencing. (line 22)\n* MIME encodings: MIME. (line 6)\n* misuse of recoding library, error message: Errors. (line 82)\n* MS-DOS charsets: IBM-PC. (line 6)\n* MULE, in Emacs: Mule. (line 22)\n* multiplexed charsets: Mule. (line 6)\n* names of charsets and surfaces, abbreviation: Requests. (line 89)\n* new charsets, how to add: New charsets. (line 6)\n* new surfaces, how to add: New surfaces. (line 6)\n* NeXT charsets: Micros. (line 6)\n* non canonical input, error message: Errors. (line 20)\n* normilise an HTML file: HTML. (line 107)\n* NOS 6/12 code: CDC-NOS. (line 9)\n* numeric character references: HTML. (line 6)\n* outer level functions: Outer level. (line 6)\n* partial conversion: Mixed. (line 20)\n* permutations of groups of bytes: Permutations. (line 6)\n* pipe sequencing: Sequencing. (line 38)\n* programname variable: Outer level. (line 122)\n* programming language support: Listings. (line 26)\n* pseudo-charsets: Charset overview. (line 32)\n* pure charset: Surface overview. (line 17)\n* quality of recoding: Recoding. (line 35)\n* recode internals: Internals. (line 6)\n* recode request syntax: Requests. (line 16)\n* recode use, a tutorial: Tutorial. (line 6)\n* recode version, printing: Listings. (line 10)\n* recode, a Macintosh port: Apple-Mac. (line 45)\n* recode, and RFC 1345: Tabular. (line 40)\n* recode, main flow of operation: Main flow. (line 6)\n* recode, operation as filter: Synopsis. (line 27)\n* recode, synopsis of invocation: Synopsis. (line 6)\n* recoding details: Recoding. (line 35)\n* recoding library: Library. (line 6)\n* recoding path, rejection: Recoding. (line 60)\n* recoding steps, statistics: Main flow. (line 41)\n* removing diacritics and underlines: flat. (line 8)\n* reporting bugs: Contributing. (line 38)\n* request level functions: Request level. (line 6)\n* request, syntax: Requests. (line 16)\n* reversibility of recoding: Reversibility. (line 61)\n* RFC 1345: Tabular. (line 6)\n* RFC 2045: MIME. (line 6)\n* sequencing: Sequencing. (line 6)\n* SGML: HTML. (line 6)\n* shared library implementation: Design. (line 6)\n* silent operation: Reversibility. (line 37)\n* single step: Main flow. (line 17)\n* source file generation: Listings. (line 26)\n* stdbool.h header: Outer level. (line 31)\n* strict operation: Reversibility. (line 49)\n* string and comments conversion: Mixed. (line 39)\n* structural surfaces: Surfaces. (line 36)\n* subsets in charsets: Listings. (line 198)\n* super-charsets: Mule. (line 6)\n* supported programming languages: Listings. (line 26)\n* suppressing diagnostic messages: Reversibility. (line 37)\n* surface, what it is: Introduction. (line 31)\n* surface, what it is <1>: Surfaces. (line 6)\n* surfaces, aliases: Requests. (line 81)\n* surfaces, commutativity: Requests. (line 57)\n* surfaces, implementation in recode: Surfaces. (line 26)\n* surfaces, implied: Requests. (line 70)\n* surfaces, overview: Surface overview. (line 6)\n* surfaces, structural: Surfaces. (line 36)\n* surfaces, syntax: Requests. (line 52)\n* surfaces, trees: Surfaces. (line 36)\n* system detected problem, error message: Errors. (line 76)\n* task execution: Task level. (line 218)\n* task level functions: Task level. (line 6)\n* TeX files: LaTeX. (line 6)\n* Texinfo files: Texinfo. (line 6)\n* threshold for error reporting: Errors. (line 100)\n* time stamps of files: Recoding. (line 26)\n* trivial surface: Surfaces. (line 6)\n* tutorial: Tutorial. (line 6)\n* unavailable conversions: Charset overview. (line 32)\n* Unicode: UCS-2. (line 6)\n* unknown charsets: Listings. (line 52)\n* unreachable charsets: Charset overview. (line 32)\n* untranslatable input, error message: Errors. (line 53)\n* valid characters in charset names: Requests. (line 10)\n* verbose operation: Recoding. (line 35)\n* Vietnamese charsets: Vietnamese. (line 6)\n* Web: HTML. (line 6)\n* World Wide Web: HTML. (line 6)\n* WWW: HTML. (line 6)\n* XML: HTML. (line 6)\n", "subsections": [] }, "File: recode.info, Node: Option Index, Next: Library Index, Prev: Concept Index, Up: Top": { "content": "", "subsections": [] }, "Option Index": { "content": "", "subsections": [] }, "This is an alphabetical list of all command-line options accepted by": { "content": "'recode'.\n\n\n* Menu:\n\n* --colons: Recoding. (line 11)\n* --copyright: Listings. (line 19)\n* --diacritics: Mixed. (line 20)\n* --find-subsets: Listings. (line 198)\n* --force: Reversibility. (line 10)\n* --graphics: Recoding. (line 16)\n* --header: Listings. (line 26)\n* --help: Listings. (line 14)\n* --ignore: Recoding. (line 60)\n* --known=: Listings. (line 52)\n* --list: Listings. (line 141)\n* --quiet: Reversibility. (line 37)\n* --sequence: Sequencing. (line 22)\n* --silent: Reversibility. (line 37)\n* --source: Mixed. (line 39)\n* --strict: Reversibility. (line 49)\n* --touch: Recoding. (line 26)\n* --verbose: Recoding. (line 35)\n* --version: Listings. (line 10)\n* -C: Listings. (line 19)\n* -c: Recoding. (line 11)\n* -d: Mixed. (line 20)\n* -f: Reversibility. (line 10)\n* -g: Recoding. (line 16)\n* -h: Listings. (line 26)\n* -i: Sequencing. (line 28)\n* -k: Listings. (line 52)\n* -l: Listings. (line 141)\n* -p: Sequencing. (line 38)\n* -q: Reversibility. (line 37)\n* -s: Reversibility. (line 49)\n* -S: Mixed. (line 39)\n* -T: Listings. (line 198)\n* -t: Recoding. (line 26)\n* -v: Recoding. (line 35)\n* -x: Recoding. (line 60)\n", "subsections": [] }, "File: recode.info, Node: Library Index, Next: Charset and Surface Index, Prev: Option Index, Up: Top": { "content": "", "subsections": [] }, "Library Index": { "content": "", "subsections": [] }, "This is an alphabetical index of important functions, data structures,": { "content": "and variables in the 'recode' library.\n\n\n* Menu:\n\n* abortlevel: Task level. (line 201)\n* asciigraphics: Request level. (line 101)\n* byteordermark: Task level. (line 185)\n* declarestep: New surfaces. (line 13)\n* DEFAULTCHARSET: Requests. (line 105)\n* diacriticsonly: Request level. (line 92)\n* diaeresischar: Request level. (line 76)\n* errorsofar: Task level. (line 213)\n* faillevel: Task level. (line 191)\n* fileonetomany: New charsets. (line 71)\n* fileonetoone: New charsets. (line 59)\n* findcharset: Charset level. (line 15)\n* LANG, when listing charsets: Listings. (line 186)\n* LANGUAGE, when listing charsets: Listings. (line 186)\n* listallcharsets: Charset level. (line 15)\n* listconcisecharset: Charset level. (line 15)\n* listfullcharset: Charset level. (line 15)\n* makeheaderflag: Request level. (line 83)\n* RECODEAMBIGUOUSOUTPUT: Errors. (line 33)\n* recodebuffertobuffer: Request level. (line 147)\n* recodebuffertofile: Request level. (line 147)\n* recodedeleteouter: Outer level. (line 89)\n* recodedeleterequest: Request level. (line 47)\n* recodedeletetask: Task level. (line 54)\n* recodefiletobuffer: Request level. (line 147)\n* recodefiletofile: Request level. (line 147)\n* recodefilterclose: Task level. (line 220)\n* recodefilterclose, not available: Request level. (line 213)\n* recodefilteropen: Task level. (line 220)\n* recodefilteropen, not available: Request level. (line 213)\n* recodeformattable: Request level. (line 228)\n* RECODEINTERNALERROR: Errors. (line 88)\n* RECODEINVALIDINPUT: Errors. (line 65)\n* RECODEMAXIMUMERROR: Task level. (line 201)\n* RECODEMAXIMUMERROR <1>: Errors. (line 96)\n* recodenewouter: Outer level. (line 89)\n* recodenewrequest: Request level. (line 47)\n* recodenewtask: Task level. (line 54)\n* RECODENOERROR: Errors. (line 16)\n* RECODENOTCANONICAL: Errors. (line 20)\n* RECODEOUTER structure: Outer level. (line 25)\n* recodeperformtask: Task level. (line 220)\n* RECODEREQUEST structure: Request level. (line 37)\n* recoderequest structure: Request level. (line 59)\n* recodescanrequest: Request level. (line 111)\n* RECODESEQUENCEINMEMORY: Task level. (line 174)\n* RECODESEQUENCEWITHFILES: Task level. (line 176)\n* RECODESEQUENCEWITHPIPE: Task level. (line 178)\n* RECODESTRATEGYUNDECIDED: Task level. (line 167)\n* recodestring: Request level. (line 140)\n* recodestringtobuffer: Request level. (line 147)\n* recodestringtofile: Request level. (line 147)\n* RECODESYSTEMERROR: Errors. (line 76)\n* RECODETASK structure: Task level. (line 46)\n* RECODEUNTRANSLATABLE: Errors. (line 53)\n* RECODEUSERERROR: Errors. (line 82)\n* strategy: Task level. (line 167)\n* taskrequest structure: Task level. (line 81)\n* verboseflag: Request level. (line 71)\n", "subsections": [] }, "File: recode.info, Node: Charset and Surface Index, Prev: Library Index, Up: Top": { "content": "", "subsections": [] }, "Charset and Surface Index": { "content": "", "subsections": [] }, "This is an alphabetical list of all the charsets and surfaces supported": { "content": "by 'recode', and their aliases.\n\n\n* Menu:\n\n* 037: Tabular. (line 219)\n* 038: Tabular. (line 224)\n* 1004: Tabular. (line 228)\n* 1026: Tabular. (line 232)\n* 1047: Tabular. (line 236)\n* 10646: UCS-4. (line 10)\n* 1129, not available: Vietnamese. (line 30)\n* 1250: Tabular. (line 86)\n* 1251: Tabular. (line 90)\n* 1252: Tabular. (line 94)\n* 1253: Tabular. (line 98)\n* 1254: Tabular. (line 102)\n* 1255: Tabular. (line 106)\n* 1256: Tabular. (line 110)\n* 1257: Tabular. (line 114)\n* 1258, not available: Vietnamese. (line 30)\n* 1345: Tabular. (line 51)\n* 1866: HTML. (line 38)\n* 2070: HTML. (line 46)\n* 21-Permutation: Permutations. (line 15)\n* 256: Tabular. (line 241)\n* 273: Tabular. (line 245)\n* 274: Tabular. (line 249)\n* 275: Tabular. (line 253)\n* 278: Tabular. (line 261)\n* 280: Tabular. (line 265)\n* 281: Tabular. (line 269)\n* 284: Tabular. (line 273)\n* 285: Tabular. (line 277)\n* 290: Tabular. (line 281)\n* 297: Tabular. (line 285)\n* 367: Tabular. (line 69)\n* 420: Tabular. (line 289)\n* 423: Tabular. (line 294)\n* 424: Tabular. (line 298)\n* 4321-Permutation: Permutations. (line 19)\n* 437: Tabular. (line 302)\n* 500: Tabular. (line 306)\n* 500V1: Tabular. (line 306)\n* 64: MIME. (line 11)\n* 819: Tabular. (line 419)\n* 850: libiconv. (line 215)\n* 850 <1>: Tabular. (line 311)\n* 851: Tabular. (line 315)\n* 852: Tabular. (line 319)\n* 855: Tabular. (line 323)\n* 857: Tabular. (line 327)\n* 860: Tabular. (line 331)\n* 861: Tabular. (line 335)\n* 862: Tabular. (line 339)\n* 863: Tabular. (line 343)\n* 864: Tabular. (line 347)\n* 865: Tabular. (line 351)\n* 866: libiconv. (line 219)\n* 868: Tabular. (line 355)\n* 869: Tabular. (line 359)\n* 870: Tabular. (line 363)\n* 871: Tabular. (line 368)\n* 875: Tabular. (line 372)\n* 880: Tabular. (line 376)\n* 891: Tabular. (line 380)\n* 903: Tabular. (line 384)\n* 904: Tabular. (line 388)\n* 905: Tabular. (line 392)\n* 912: Tabular. (line 444)\n* 918: Tabular. (line 396)\n* AFRFUL-102-BPIOCIL, and aliases: African. (line 10)\n* AFRFUL-103-BPIOCIL, and aliases: African. (line 10)\n* AFRL1-101-BPIOCIL: African. (line 24)\n* AFRLIN-104-BPIOCIL: African. (line 17)\n* AFRLIN-105-BPIOCIL: African. (line 17)\n* ANSIX3.110-1983, not recognised by recode: Tabular. (line 19)\n* ANSIX3.4-1968: libiconv. (line 65)\n* ANSIX3.4-1968, aliases and source: Tabular. (line 69)\n* ANSIX3.4-1968, and its aliases: ASCII. (line 6)\n* ANSIX3.4-1986: Tabular. (line 69)\n* Apple-Mac: Apple-Mac. (line 6)\n* ARABIC: libiconv. (line 138)\n* arabic: Tabular. (line 464)\n* arabic7: Tabular. (line 74)\n* ARMSCII-8: libiconv. (line 256)\n* ASCII: libiconv. (line 65)\n* ASCII <1>: Tabular. (line 69)\n* ASCII-BS, and its aliases: ASCII-BS. (line 6)\n* ASCII, an alias for the ANSIX3.4-1968 charset: ASCII. (line 6)\n* ASMO449, aliases and source: Tabular. (line 74)\n* ASMO-708: libiconv. (line 138)\n* ASMO-708 <1>: Tabular. (line 464)\n* AtariST: AtariST. (line 6)\n* b64: MIME. (line 11)\n* baltic, aliases and source: Tabular. (line 660)\n* bambara: African. (line 10)\n* Bang-Bang: Bang-Bang. (line 6)\n* Base64: MIME. (line 11)\n* BIG-5: libiconv. (line 356)\n* BIG-FIVE: libiconv. (line 356)\n* BIG5, aliases: libiconv. (line 356)\n* BIG5HKSCS: libiconv. (line 362)\n* BIGFIVE: libiconv. (line 356)\n* BMP: UCS-2. (line 29)\n* bra: African. (line 10)\n* BS4730, aliases and source: Tabular. (line 78)\n* BSviewdata, aliases and source: Tabular. (line 82)\n* BS, an alias for ASCII-BS charset: ASCII-BS. (line 6)\n* ca: Tabular. (line 118)\n* CDC-NOS, and its aliases: CDC-NOS. (line 6)\n* CHAR: libiconv. (line 376)\n* CHINESE: libiconv. (line 304)\n* cl: End lines. (line 37)\n* CN: libiconv. (line 300)\n* cn: Tabular. (line 211)\n* CN-BIG5: libiconv. (line 356)\n* CN-GB: libiconv. (line 336)\n* CN-GB-ISOIR165: libiconv. (line 308)\n* combined-UCS-2: UCS-2. (line 33)\n* CORK: Others. (line 29)\n* count-characters: count-characters. (line 6)\n* count-characters, not as before charset: Charset overview. (line 32)\n* cp-ar: Tabular. (line 355)\n* cp-gr: Tabular. (line 359)\n* cp-hu: Tabular. (line 134)\n* cp-is: Tabular. (line 335)\n* CP037: Tabular. (line 219)\n* CP038: Tabular. (line 224)\n* CP1004: Tabular. (line 228)\n* CP1026: Tabular. (line 232)\n* CP1047: Tabular. (line 236)\n* CP1129, not available: Vietnamese. (line 30)\n* CP1133, aliases: libiconv. (line 264)\n* CP1250, aliases: libiconv. (line 187)\n* CP1250, aliases and source: Tabular. (line 86)\n* CP1251, aliases: libiconv. (line 190)\n* CP1251, aliases and source: Tabular. (line 90)\n* CP1252, aliases: libiconv. (line 193)\n* CP1252, aliases and source: Tabular. (line 94)\n* CP1253, aliases: libiconv. (line 196)\n* CP1253, aliases and source: Tabular. (line 98)\n* CP1254, aliases: libiconv. (line 199)\n* CP1254, aliases and source: Tabular. (line 102)\n* CP1255, aliases: libiconv. (line 202)\n* CP1255, aliases and source: Tabular. (line 106)\n* CP1256, aliases: libiconv. (line 205)\n* CP1256, aliases and source: Tabular. (line 110)\n* CP1257, aliases: libiconv. (line 208)\n* CP1257, aliases and source: Tabular. (line 114)\n* CP1258, aliases: libiconv. (line 211)\n* CP1258, not available: Vietnamese. (line 30)\n* CP1361: libiconv. (line 370)\n* CP256: Tabular. (line 241)\n* CP273: Tabular. (line 245)\n* CP274: Tabular. (line 249)\n* CP275: Tabular. (line 253)\n* CP278: Tabular. (line 261)\n* CP280: Tabular. (line 265)\n* CP281: Tabular. (line 269)\n* CP284: Tabular. (line 273)\n* CP285: Tabular. (line 277)\n* CP290: Tabular. (line 281)\n* CP297: Tabular. (line 285)\n* CP367: libiconv. (line 65)\n* CP367 <1>: Tabular. (line 69)\n* cp367: ASCII. (line 6)\n* CP420: Tabular. (line 289)\n* CP423: Tabular. (line 294)\n* CP424: Tabular. (line 298)\n* CP437: Tabular. (line 302)\n* CP500: Tabular. (line 306)\n* CP819: libiconv. (line 113)\n* CP819 <1>: Tabular. (line 419)\n* CP850: Tabular. (line 311)\n* CP850, aliases: libiconv. (line 215)\n* CP851: Tabular. (line 315)\n* CP852: Tabular. (line 319)\n* CP855: Tabular. (line 323)\n* CP857: Tabular. (line 327)\n* CP860: Tabular. (line 331)\n* CP861: Tabular. (line 335)\n* CP862: Tabular. (line 339)\n* CP863: Tabular. (line 343)\n* CP864: Tabular. (line 347)\n* CP865: Tabular. (line 351)\n* CP866, aliases: libiconv. (line 219)\n* CP868: Tabular. (line 355)\n* CP869: Tabular. (line 359)\n* CP870: Tabular. (line 363)\n* CP871: Tabular. (line 368)\n* CP874, aliases: libiconv. (line 271)\n* CP875: Tabular. (line 372)\n* CP880: Tabular. (line 376)\n* CP891: Tabular. (line 380)\n* CP903: Tabular. (line 384)\n* CP904: Tabular. (line 388)\n* CP905: Tabular. (line 392)\n* CP912: Tabular. (line 444)\n* CP918: Tabular. (line 396)\n* CP932: libiconv. (line 326)\n* CP936: libiconv. (line 340)\n* CP949, aliases: libiconv. (line 367)\n* CP950: libiconv. (line 360)\n* CR-LF, a surface: End lines. (line 27)\n* CR, a surface: End lines. (line 13)\n* CSAZ243.4-1985-1, aliases and source: Tabular. (line 118)\n* CSAZ243.4-1985-2, aliases and source: Tabular. (line 122)\n* CSAZ243.4-1985-gr, aliases and source: Tabular. (line 126)\n* csa7-1: Tabular. (line 118)\n* csa7-2: Tabular. (line 122)\n* csASCII: libiconv. (line 65)\n* csBig5: libiconv. (line 356)\n* csEUCKR: libiconv. (line 364)\n* csEUCPkdFmtJapanese: libiconv. (line 317)\n* csEUCTW: libiconv. (line 353)\n* csGB2312: libiconv. (line 336)\n* csHalfWidthKatakana: libiconv. (line 286)\n* csHPRoman8: libiconv. (line 250)\n* csIBM866: libiconv. (line 219)\n* csISO14JISC6220ro: libiconv. (line 282)\n* csISO159JISX02121990: libiconv. (line 295)\n* csISO2022CN: libiconv. (line 345)\n* csISO2022JP: libiconv. (line 328)\n* csISO2022JP2: libiconv. (line 333)\n* csISO2022KR: libiconv. (line 373)\n* csISO57GB1988: libiconv. (line 300)\n* csISO58GB231280: libiconv. (line 304)\n* csISO87JISX0208: libiconv. (line 290)\n* csISOLatin1: libiconv. (line 113)\n* csISOLatin2: libiconv. (line 118)\n* csISOLatin3: libiconv. (line 123)\n* csISOLatin4: libiconv. (line 128)\n* csISOLatin5: libiconv. (line 153)\n* csISOLatin6: libiconv. (line 158)\n* csISOLatinArabic: libiconv. (line 138)\n* csISOLatinCyrillic: libiconv. (line 133)\n* csISOLatinGreek: libiconv. (line 143)\n* csISOLatinHebrew: libiconv. (line 148)\n* csKOI8R: libiconv. (line 179)\n* csKSC56011987: libiconv. (line 311)\n* csMacintosh: libiconv. (line 223)\n* CSN369103, aliases and source: Tabular. (line 130)\n* csPC850Multilingual: libiconv. (line 215)\n* csShiftJIS: libiconv. (line 322)\n* csUCS4: libiconv. (line 85)\n* csUnicode: libiconv. (line 74)\n* csUnicode11: libiconv. (line 78)\n* csUnicode11UTF7: libiconv. (line 98)\n* csVISCII: libiconv. (line 274)\n* cuba: Tabular. (line 616)\n* CWI-2: Tabular. (line 134)\n* CWI, aliases and source: Tabular. (line 134)\n* CYRILLIC: libiconv. (line 133)\n* cyrillic: Tabular. (line 459)\n* d1: Dump. (line 45)\n* d2: Dump. (line 52)\n* d4: Dump. (line 59)\n* data, a special charset: Surfaces. (line 26)\n* data, not with charsets: Charset overview. (line 32)\n* de: Tabular. (line 142)\n* dec: Tabular. (line 138)\n* DEC-MCS, aliases and source: Tabular. (line 138)\n* Decimal-1: Dump. (line 45)\n* Decimal-2: Dump. (line 52)\n* Decimal-4: Dump. (line 59)\n* DIN66003, aliases and source: Tabular. (line 142)\n* dk: Tabular. (line 146)\n* dk-us, not recognised by recode: Tabular. (line 16)\n* dos: IBM-PC. (line 6)\n* DS2089, aliases and source: Tabular. (line 146)\n* DS2089: Tabular. (line 146)\n* dump-with-names: dump-with-names. (line 6)\n* dump-with-names, not as before charset: Charset overview. (line 32)\n* e13b: Tabular. (line 488)\n* EBCDIC-AT-DE-A, aliases and source: Tabular. (line 153)\n* EBCDIC-AT-DE, aliases and source: Tabular. (line 150)\n* EBCDIC-BE: Tabular. (line 249)\n* EBCDIC-BR: Tabular. (line 253)\n* EBCDIC-CA-FR, aliases and source: Tabular. (line 156)\n* EBCDIC-CCC: EBCDIC. (line 18)\n* ebcdic-cp-ar1: Tabular. (line 289)\n* ebcdic-cp-ar2: Tabular. (line 396)\n* ebcdic-cp-be: Tabular. (line 306)\n* ebcdic-cp-ca: Tabular. (line 219)\n* ebcdic-cp-ch: Tabular. (line 306)\n* EBCDIC-CP-DK: Tabular. (line 257)\n* ebcdic-cp-es: Tabular. (line 273)\n* ebcdic-cp-fi: Tabular. (line 261)\n* ebcdic-cp-fr: Tabular. (line 285)\n* ebcdic-cp-gb: Tabular. (line 277)\n* ebcdic-cp-gr: Tabular. (line 294)\n* ebcdic-cp-he: Tabular. (line 298)\n* ebcdic-cp-is: Tabular. (line 368)\n* ebcdic-cp-it: Tabular. (line 265)\n* ebcdic-cp-nl: Tabular. (line 219)\n* EBCDIC-CP-NO: Tabular. (line 257)\n* ebcdic-cp-roece: Tabular. (line 363)\n* ebcdic-cp-se: Tabular. (line 261)\n* ebcdic-cp-tr: Tabular. (line 392)\n* ebcdic-cp-us: Tabular. (line 219)\n* ebcdic-cp-wt: Tabular. (line 219)\n* ebcdic-cp-yu: Tabular. (line 363)\n* EBCDIC-Cyrillic: Tabular. (line 376)\n* EBCDIC-DK-NO-A, aliases and source: Tabular. (line 162)\n* EBCDIC-DK-NO, aliases and source: Tabular. (line 159)\n* EBCDIC-ES-A, aliases and source: Tabular. (line 168)\n* EBCDIC-ES-S, aliases and source: Tabular. (line 171)\n* EBCDIC-ES, aliases and source: Tabular. (line 165)\n* EBCDIC-FI-SE-A, aliases and source: Tabular. (line 177)\n* EBCDIC-FI-SE, aliases and source: Tabular. (line 174)\n* EBCDIC-FR, aliases and source: Tabular. (line 180)\n* EBCDIC-Greek: Tabular. (line 372)\n* EBCDIC-IBM: EBCDIC. (line 23)\n* EBCDIC-INT: Tabular. (line 224)\n* EBCDIC-INT1: Tabular. (line 241)\n* EBCDIC-IS-FRISS, aliases and source: Tabular. (line 183)\n* EBCDIC-IT, aliases and source: Tabular. (line 187)\n* EBCDIC-JP-E: Tabular. (line 269)\n* EBCDIC-JP-kana: Tabular. (line 281)\n* EBCDIC-PT, aliases and source: Tabular. (line 190)\n* EBCDIC-UK, aliases and source: Tabular. (line 193)\n* EBCDIC-US, aliases and source: Tabular. (line 196)\n* EBCDIC, a charset: EBCDIC. (line 11)\n* ECMA-113: Tabular. (line 199)\n* ECMA-113(1986): Tabular. (line 199)\n* ECMA-114: libiconv. (line 138)\n* ECMA-114 <1>: Tabular. (line 464)\n* ECMA-118: libiconv. (line 143)\n* ECMA-118 <1>: Tabular. (line 469)\n* ECMA-cyrillic, aliases and source: Tabular. (line 199)\n* ELOT928: libiconv. (line 143)\n* ELOT928 <1>: Tabular. (line 469)\n* ES, aliases and source: Tabular. (line 203)\n* ES2, aliases and source: Tabular. (line 207)\n* EUCCN: libiconv. (line 336)\n* EUCJP: libiconv. (line 317)\n* EUCKR: libiconv. (line 364)\n* EUCTW: libiconv. (line 353)\n* EUC-CN, aliases: libiconv. (line 336)\n* EUC-JP, aliases: libiconv. (line 317)\n* EUC-KR, aliases: libiconv. (line 364)\n* EUC-TW, aliases: libiconv. (line 353)\n* EUCCN: libiconv. (line 336)\n* EUCJP: libiconv. (line 317)\n* EUCKR: libiconv. (line 364)\n* EUCTW: libiconv. (line 353)\n* ewondo: African. (line 10)\n* ExtendedUNIXCodePackedFormatforJapanese: libiconv. (line 317)\n* FI: Tabular. (line 648)\n* flat, a charset: flat. (line 6)\n* flat, not as before charset: Charset overview. (line 32)\n* fr: Tabular. (line 620)\n* friss: Tabular. (line 183)\n* FSSUTF: UTF-8. (line 81)\n* fulfude: African. (line 10)\n* gb: Tabular. (line 78)\n* GB1988-80, aliases: libiconv. (line 300)\n* GB1988-80, aliases and source: Tabular. (line 211)\n* GB2312-80, aliases: libiconv. (line 304)\n* GB2312-80, not recognised by recode: Tabular. (line 23)\n* GB18030: libiconv. (line 343)\n* GB2312: libiconv. (line 336)\n* GB2312.1980-0: libiconv. (line 304)\n* GBK, aliases: libiconv. (line 340)\n* Georgian-Academy: libiconv. (line 258)\n* Georgian-PS: libiconv. (line 260)\n* GOST19768-74: Tabular. (line 574)\n* GOST19768-87, aliases and source: Tabular. (line 215)\n* GREEK: libiconv. (line 143)\n* greek: Tabular. (line 469)\n* greek-ccitt, aliases and source: Tabular. (line 664)\n* greek7-old, aliases and source: Tabular. (line 672)\n* greek7, aliases and source: Tabular. (line 668)\n* GREEK8: libiconv. (line 143)\n* greek8: Tabular. (line 469)\n* h: HTML. (line 62)\n* h0: HTML. (line 27)\n* h1: HTML. (line 33)\n* h2: HTML. (line 38)\n* h3: HTML. (line 56)\n* h4: HTML. (line 62)\n* HEBREW: libiconv. (line 148)\n* hebrew: Tabular. (line 474)\n* Hexadecimal-1: Dump. (line 66)\n* Hexadecimal-2: Dump. (line 73)\n* Hexadecimal-4: Dump. (line 80)\n* HP-ROMAN8, aliases: libiconv. (line 250)\n* hp-roman8, aliases and source: Tabular. (line 676)\n* HTML1.1: HTML. (line 33)\n* HTML2.0: HTML. (line 38)\n* HTML3.2: HTML. (line 56)\n* HTML-i18n: HTML. (line 46)\n* hu: Tabular. (line 596)\n* HZ-GB-2312: libiconv. (line 350)\n* HZ, aliases: libiconv. (line 350)\n* IBM-CP1133: libiconv. (line 264)\n* IBM-PC: IBM-PC. (line 6)\n* IBM-PC charset, and CR-LF surface: Requests. (line 70)\n* IBM037, aliases and source: Tabular. (line 219)\n* IBM038, aliases and source: Tabular. (line 224)\n* IBM1004, aliases and source: Tabular. (line 228)\n* IBM1026, aliases and source: Tabular. (line 232)\n* IBM1047, aliases and source: Tabular. (line 236)\n* IBM256, aliases and source: Tabular. (line 241)\n* IBM273, aliases and source: Tabular. (line 245)\n* IBM274, aliases and source: Tabular. (line 249)\n* IBM275, aliases and source: Tabular. (line 253)\n* IBM277, aliases and source: Tabular. (line 257)\n* IBM278, aliases and source: Tabular. (line 261)\n* IBM280, aliases and source: Tabular. (line 265)\n* IBM281, aliases and source: Tabular. (line 269)\n* IBM284, aliases and source: Tabular. (line 273)\n* IBM285, aliases and source: Tabular. (line 277)\n* IBM290, aliases and source: Tabular. (line 281)\n* IBM297, aliases and source: Tabular. (line 285)\n* IBM367: libiconv. (line 65)\n* IBM367 <1>: Tabular. (line 69)\n* IBM367 <2>: ASCII. (line 6)\n* IBM420, aliases and source: Tabular. (line 289)\n* IBM423, aliases and source: Tabular. (line 294)\n* IBM424, aliases and source: Tabular. (line 298)\n* ibm437: IBM-PC. (line 36)\n* IBM437, aliases and source: Tabular. (line 302)\n* IBM500, aliases and source: Tabular. (line 306)\n* IBM819: libiconv. (line 113)\n* IBM819 <1>: Tabular. (line 419)\n* IBM819, and CR-LF surface: IBM-PC. (line 30)\n* IBM850: libiconv. (line 215)\n* IBM850, aliases and source: Tabular. (line 311)\n* IBM851, aliases and source: Tabular. (line 315)\n* IBM852, aliases and source: Tabular. (line 319)\n* IBM855, aliases and source: Tabular. (line 323)\n* IBM857, aliases and source: Tabular. (line 327)\n* IBM860, aliases and source: Tabular. (line 331)\n* IBM861, aliases and source: Tabular. (line 335)\n* IBM862, aliases and source: Tabular. (line 339)\n* IBM863, aliases and source: Tabular. (line 343)\n* IBM864, aliases and source: Tabular. (line 347)\n* IBM865, aliases and source: Tabular. (line 351)\n* IBM866: libiconv. (line 219)\n* IBM868, aliases and source: Tabular. (line 355)\n* IBM869, aliases and source: Tabular. (line 359)\n* IBM870, aliases and source: Tabular. (line 363)\n* IBM871, aliases and source: Tabular. (line 368)\n* IBM875, aliases and source: Tabular. (line 372)\n* IBM880, aliases and source: Tabular. (line 376)\n* IBM891, aliases and source: Tabular. (line 380)\n* IBM903, aliases and source: Tabular. (line 384)\n* IBM904, aliases and source: Tabular. (line 388)\n* IBM905, aliases and source: Tabular. (line 392)\n* IBM912: Tabular. (line 444)\n* IBM918, aliases and source: Tabular. (line 396)\n* Icon-QNX, and aliases: Icon-QNX. (line 6)\n* IECP27-1, aliases and source: Tabular. (line 400)\n* INIS-8, aliases and source: Tabular. (line 408)\n* INIS-cyrillic, aliases and source: Tabular. (line 412)\n* INIS, aliases and source: Tabular. (line 404)\n* INVARIANT, aliases and source: Tabular. (line 416)\n* irv: Tabular. (line 508)\n* ISO10367-box, aliases and source: Tabular. (line 484)\n* ISO10646: UCS-4. (line 10)\n* ISO2033-1983, aliases and source: Tabular. (line 488)\n* ISO5427-ext, aliases and source: Tabular. (line 496)\n* ISO5427, aliases and source: Tabular. (line 492)\n* ISO5427(1981): Tabular. (line 496)\n* ISO5428, aliases and source: Tabular. (line 500)\n* ISO5428(1980): Tabular. (line 500)\n* ISO646.basic, aliases and source: Tabular. (line 504)\n* ISO646.basic(1983): Tabular. (line 504)\n* ISO646.irv, aliases and source: Tabular. (line 508)\n* ISO646.irv(1983): Tabular. (line 508)\n* ISO646.IRV(1991): libiconv. (line 65)\n* ISO646.irv(1991): Tabular. (line 69)\n* ISO6937-2-25, aliases and source: Tabular. (line 512)\n* ISO6937-2-add, not recognised by recode: Tabular. (line 19)\n* ISO8859-1: libiconv. (line 113)\n* ISO8859-1 <1>: Tabular. (line 419)\n* ISO8859-1(1987): libiconv. (line 113)\n* ISO8859-1(1987) <1>: Tabular. (line 419)\n* ISO8859-10: libiconv. (line 158)\n* ISO8859-10 <1>: Tabular. (line 424)\n* ISO8859-10(1992): libiconv. (line 158)\n* ISO8859-10(1993): Tabular. (line 424)\n* ISO8859-13: libiconv. (line 163)\n* ISO8859-13 <1>: Tabular. (line 429)\n* ISO8859-13(1998): Tabular. (line 429)\n* ISO8859-14: libiconv. (line 167)\n* ISO8859-14 <1>: Tabular. (line 434)\n* ISO8859-14(1998): libiconv. (line 167)\n* ISO8859-14(1998) <1>: Tabular. (line 434)\n* ISO8859-15: libiconv. (line 171)\n* ISO8859-15 <1>: Tabular. (line 439)\n* ISO8859-15(1998): libiconv. (line 171)\n* ISO8859-15(1998) <1>: Tabular. (line 439)\n* ISO8859-16: libiconv. (line 175)\n* ISO8859-16(2000): libiconv. (line 175)\n* ISO8859-2: libiconv. (line 118)\n* ISO8859-2 <1>: Tabular. (line 444)\n* ISO8859-2(1987): libiconv. (line 118)\n* ISO8859-2(1987) <1>: Tabular. (line 444)\n* ISO8859-3: libiconv. (line 123)\n* ISO8859-3 <1>: Tabular. (line 449)\n* ISO8859-3(1988): libiconv. (line 123)\n* ISO8859-3(1988) <1>: Tabular. (line 449)\n* ISO8859-4: libiconv. (line 128)\n* ISO8859-4 <1>: Tabular. (line 454)\n* ISO8859-4(1988): libiconv. (line 128)\n* ISO8859-4(1988) <1>: Tabular. (line 454)\n* ISO8859-5: libiconv. (line 133)\n* ISO8859-5 <1>: Tabular. (line 459)\n* ISO8859-5(1988): libiconv. (line 133)\n* ISO8859-5(1988) <1>: Tabular. (line 459)\n* ISO8859-6: libiconv. (line 138)\n* ISO8859-6 <1>: Tabular. (line 464)\n* ISO8859-6(1987): libiconv. (line 138)\n* ISO8859-6(1987) <1>: Tabular. (line 464)\n* ISO8859-7: libiconv. (line 143)\n* ISO8859-7 <1>: Tabular. (line 469)\n* ISO8859-7(1987): libiconv. (line 143)\n* ISO8859-7(1987) <1>: Tabular. (line 469)\n* ISO8859-8: libiconv. (line 148)\n* ISO8859-8 <1>: Tabular. (line 474)\n* ISO8859-8(1988): libiconv. (line 148)\n* ISO8859-8(1988) <1>: Tabular. (line 474)\n* ISO8859-9: libiconv. (line 153)\n* ISO8859-9 <1>: Tabular. (line 479)\n* ISO8859-9(1989): libiconv. (line 153)\n* ISO8859-9(1989) <1>: Tabular. (line 479)\n* ISO8859-supp, aliases and source: Tabular. (line 516)\n* ISO9036: Tabular. (line 74)\n* ISO-10646-UCS-2: libiconv. (line 74)\n* ISO-10646-UCS-2, and aliases: UCS-2. (line 29)\n* ISO-10646-UCS-4: libiconv. (line 85)\n* ISO-10646-UCS-4, and aliases: UCS-4. (line 10)\n* ISO-2022-CN-EXT: libiconv. (line 348)\n* ISO-2022-CN, aliases: libiconv. (line 345)\n* ISO-2022-JP-1: libiconv. (line 331)\n* ISO-2022-JP-2, aliases: libiconv. (line 333)\n* ISO-2022-JP, aliases: libiconv. (line 328)\n* ISO-2022-KR, aliases: libiconv. (line 373)\n* ISO-8859-1, aliases: libiconv. (line 113)\n* ISO-8859-1, aliases and source: Tabular. (line 419)\n* ISO-8859-10, aliases: libiconv. (line 158)\n* ISO-8859-10, aliases and source: Tabular. (line 424)\n* ISO-8859-13, aliases: libiconv. (line 163)\n* ISO-8859-13, aliases and source: Tabular. (line 429)\n* ISO-8859-14, aliases: libiconv. (line 167)\n* ISO-8859-14, aliases and source: Tabular. (line 434)\n* ISO-8859-15, aliases: libiconv. (line 171)\n* ISO-8859-15, aliases and source: Tabular. (line 439)\n* ISO-8859-16, aliases: libiconv. (line 175)\n* ISO-8859-2, aliases: libiconv. (line 118)\n* ISO-8859-2, aliases and source: Tabular. (line 444)\n* ISO-8859-3, aliases: libiconv. (line 123)\n* ISO-8859-3, aliases and source: Tabular. (line 449)\n* ISO-8859-4, aliases: libiconv. (line 128)\n* ISO-8859-4, aliases and source: Tabular. (line 454)\n* ISO-8859-5, aliases: libiconv. (line 133)\n* ISO-8859-5, aliases and source: Tabular. (line 459)\n* ISO-8859-6, aliases: libiconv. (line 138)\n* ISO-8859-6, aliases and source: Tabular. (line 464)\n* ISO-8859-7, aliases: libiconv. (line 143)\n* ISO-8859-7, aliases and source: Tabular. (line 469)\n* ISO-8859-8, aliases: libiconv. (line 148)\n* ISO-8859-8, aliases and source: Tabular. (line 474)\n* ISO-8859-9, aliases: libiconv. (line 153)\n* ISO-8859-9, aliases and source: Tabular. (line 479)\n* iso-baltic: Tabular. (line 429)\n* iso-celtic: Tabular. (line 434)\n* iso-ir-10: Tabular. (line 648)\n* ISO-IR-100: libiconv. (line 113)\n* iso-ir-100: Tabular. (line 419)\n* ISO-IR-101: libiconv. (line 118)\n* iso-ir-101: Tabular. (line 444)\n* iso-ir-102: Tabular. (line 656)\n* ISO-IR-109: libiconv. (line 123)\n* iso-ir-109: Tabular. (line 449)\n* iso-ir-11: Tabular. (line 652)\n* ISO-IR-110: libiconv. (line 128)\n* iso-ir-110: Tabular. (line 454)\n* iso-ir-111: Tabular. (line 199)\n* iso-ir-121: Tabular. (line 118)\n* iso-ir-122: Tabular. (line 122)\n* iso-ir-123: Tabular. (line 126)\n* ISO-IR-126: libiconv. (line 143)\n* iso-ir-126: Tabular. (line 469)\n* ISO-IR-127: libiconv. (line 138)\n* iso-ir-127: Tabular. (line 464)\n* iso-ir-13: Tabular. (line 524)\n* ISO-IR-138: libiconv. (line 148)\n* iso-ir-138: Tabular. (line 474)\n* iso-ir-139: Tabular. (line 130)\n* ISO-IR-14: libiconv. (line 282)\n* iso-ir-14: Tabular. (line 528)\n* iso-ir-141: Tabular. (line 559)\n* iso-ir-143: Tabular. (line 400)\n* ISO-IR-144: libiconv. (line 133)\n* iso-ir-144: Tabular. (line 459)\n* iso-ir-146: Tabular. (line 567)\n* iso-ir-147: Tabular. (line 563)\n* ISO-IR-148: libiconv. (line 153)\n* iso-ir-148: Tabular. (line 479)\n* ISO-IR-149: libiconv. (line 311)\n* iso-ir-15: Tabular. (line 520)\n* iso-ir-150: Tabular. (line 664)\n* iso-ir-151: Tabular. (line 616)\n* iso-ir-152: Tabular. (line 512)\n* iso-ir-153: Tabular. (line 215)\n* iso-ir-154: Tabular. (line 516)\n* iso-ir-155: Tabular. (line 484)\n* ISO-IR-157: libiconv. (line 158)\n* iso-ir-157: Tabular. (line 424)\n* iso-ir-158: Tabular. (line 694)\n* ISO-IR-159: libiconv. (line 295)\n* iso-ir-16: Tabular. (line 640)\n* ISO-IR-165, aliases: libiconv. (line 308)\n* ISO-IR-166: libiconv. (line 267)\n* iso-ir-17: Tabular. (line 203)\n* iso-ir-170: Tabular. (line 416)\n* ISO-IR-179: libiconv. (line 163)\n* iso-ir-179: Tabular. (line 660)\n* iso-ir-179a: Tabular. (line 429)\n* iso-ir-18: Tabular. (line 672)\n* iso-ir-19: Tabular. (line 681)\n* ISO-IR-199: libiconv. (line 167)\n* iso-ir-199: Tabular. (line 434)\n* iso-ir-2: Tabular. (line 508)\n* ISO-IR-203: libiconv. (line 171)\n* iso-ir-203: Tabular. (line 439)\n* iso-ir-21: Tabular. (line 142)\n* ISO-IR-226: libiconv. (line 175)\n* iso-ir-25: Tabular. (line 624)\n* iso-ir-27: Tabular. (line 592)\n* iso-ir-37: Tabular. (line 492)\n* iso-ir-4: Tabular. (line 78)\n* iso-ir-47: Tabular. (line 82)\n* iso-ir-49: Tabular. (line 404)\n* iso-ir-50: Tabular. (line 408)\n* iso-ir-51: Tabular. (line 412)\n* iso-ir-54: Tabular. (line 496)\n* iso-ir-55: Tabular. (line 500)\n* ISO-IR-57: libiconv. (line 300)\n* iso-ir-57: Tabular. (line 211)\n* ISO-IR-58: libiconv. (line 304)\n* ISO-IR-6: libiconv. (line 65)\n* iso-ir-6: Tabular. (line 69)\n* iso-ir-6 <1>: ASCII. (line 6)\n* iso-ir-60: Tabular. (line 628)\n* iso-ir-61: Tabular. (line 632)\n* iso-ir-69: Tabular. (line 620)\n* iso-ir-8-1: Tabular. (line 608)\n* iso-ir-8-2: Tabular. (line 612)\n* iso-ir-84: Tabular. (line 644)\n* iso-ir-85: Tabular. (line 207)\n* iso-ir-86: Tabular. (line 596)\n* ISO-IR-87: libiconv. (line 290)\n* iso-ir-88: Tabular. (line 668)\n* iso-ir-89: Tabular. (line 74)\n* iso-ir-9-1: Tabular. (line 600)\n* iso-ir-9-2: Tabular. (line 604)\n* iso-ir-90, not recognised by recode: Tabular. (line 19)\n* iso-ir-93: Tabular. (line 540)\n* iso-ir-94: Tabular. (line 544)\n* iso-ir-95: Tabular. (line 548)\n* iso-ir-96: Tabular. (line 552)\n* iso-ir-98: Tabular. (line 488)\n* ISO2022CN: libiconv. (line 345)\n* ISO2022JP: libiconv. (line 328)\n* ISO2022KR: libiconv. (line 373)\n* ISO646-CA: Tabular. (line 118)\n* ISO646-CA2: Tabular. (line 122)\n* ISO646-CN: libiconv. (line 300)\n* ISO646-CN <1>: Tabular. (line 211)\n* ISO646-CU: Tabular. (line 616)\n* ISO646-DE: Tabular. (line 142)\n* ISO646-DK: Tabular. (line 146)\n* ISO646-ES: Tabular. (line 203)\n* ISO646-ES2: Tabular. (line 207)\n* ISO646-FI: Tabular. (line 648)\n* ISO646-FR: Tabular. (line 620)\n* ISO646-FR1: Tabular. (line 624)\n* ISO646-GB: Tabular. (line 78)\n* ISO646-HU: Tabular. (line 596)\n* ISO646-IT: Tabular. (line 520)\n* ISO646-JP: libiconv. (line 282)\n* ISO646-JP <1>: Tabular. (line 528)\n* ISO646-JP-OCR-B: Tabular. (line 536)\n* ISO646-KR: Tabular. (line 589)\n* ISO646-NO: Tabular. (line 628)\n* ISO646-NO2: Tabular. (line 632)\n* ISO646-PT: Tabular. (line 640)\n* ISO646-PT2: Tabular. (line 644)\n* ISO646-SE: Tabular. (line 648)\n* ISO646-SE2: Tabular. (line 652)\n* ISO646-US: libiconv. (line 65)\n* ISO646-US <1>: Tabular. (line 69)\n* ISO646-YU: Tabular. (line 559)\n* ISO646.1991-IRV: libiconv. (line 65)\n* ISO88591: libiconv. (line 113)\n* ISO88592: libiconv. (line 118)\n* ISO88593: libiconv. (line 123)\n* ISO88594: libiconv. (line 128)\n* ISO88595: libiconv. (line 133)\n* ISO88596: libiconv. (line 138)\n* ISO88597: libiconv. (line 143)\n* ISO88598: libiconv. (line 148)\n* ISO88599: libiconv. (line 153)\n* ISO8859-1: libiconv. (line 113)\n* ISO8859-1 <1>: Tabular. (line 419)\n* ISO8859-10: libiconv. (line 158)\n* ISO8859-10 <1>: Tabular. (line 424)\n* ISO8859-13: Tabular. (line 429)\n* ISO8859-14: Tabular. (line 434)\n* ISO8859-15: Tabular. (line 439)\n* ISO8859-2: libiconv. (line 118)\n* ISO8859-2 <1>: Tabular. (line 444)\n* ISO8859-3: libiconv. (line 123)\n* ISO8859-3 <1>: Tabular. (line 449)\n* ISO8859-4: libiconv. (line 128)\n* ISO8859-4 <1>: Tabular. (line 454)\n* ISO8859-5: libiconv. (line 133)\n* ISO8859-5 <1>: Tabular. (line 459)\n* ISO8859-6: libiconv. (line 138)\n* ISO8859-6 <1>: Tabular. (line 464)\n* ISO8859-7: libiconv. (line 143)\n* ISO8859-7 <1>: Tabular. (line 469)\n* ISO8859-8: libiconv. (line 148)\n* ISO8859-8 <1>: Tabular. (line 474)\n* ISO8859-9: libiconv. (line 153)\n* ISO8859-9 <1>: Tabular. (line 479)\n* isoir91: Tabular. (line 27)\n* isoir92: Tabular. (line 27)\n* IT, aliases and source: Tabular. (line 520)\n* JAVA: libiconv. (line 110)\n* JISC6220-1969: Tabular. (line 524)\n* JISC6220-1969-jp, aliases and source: Tabular. (line 524)\n* JISC6220-1969-RO, aliases: libiconv. (line 282)\n* JISC6220-1969-ro, aliases and source: Tabular. (line 528)\n* JISC6226-1978, not recognised by recode: Tabular. (line 23)\n* JISC6229-1984-a, aliases and source: Tabular. (line 532)\n* JISC6229-1984-b-add, aliases and source: Tabular. (line 540)\n* JISC6229-1984-b, aliases and source: Tabular. (line 536)\n* JISC6229-1984-hand-add, aliases and source: Tabular. (line 548)\n* JISC6229-1984-hand, aliases and source: Tabular. (line 544)\n* JISC6229-1984-kana, aliases and source: Tabular. (line 552)\n* JISX0201, aliases: libiconv. (line 286)\n* JISX0201, aliases and source: Tabular. (line 556)\n* JISX0208-1983: libiconv. (line 290)\n* JISX0208-1990: libiconv. (line 290)\n* JISX0208, aliases: libiconv. (line 290)\n* JISX0212-1990: libiconv. (line 295)\n* JISX0212-1990, not recognised by recode: Tabular. (line 23)\n* JISX0212, aliases: libiconv. (line 295)\n* JISX0212.1990-0: libiconv. (line 295)\n* JIS0201: libiconv. (line 286)\n* JIS0208: libiconv. (line 290)\n* JIS0208 <1>: libiconv. (line 290)\n* JIS0212: libiconv. (line 295)\n* JISX0201-1976: libiconv. (line 286)\n* JISX0201.1976-0: libiconv. (line 286)\n* JISX0208.1983-0: libiconv. (line 290)\n* JISX0208.1990-0: libiconv. (line 290)\n* JISX0212.1990-0: libiconv. (line 295)\n* JOHAB, aliases: libiconv. (line 370)\n* JP: libiconv. (line 282)\n* jp: Tabular. (line 528)\n* jp-ocr-a: Tabular. (line 532)\n* jp-ocr-b: Tabular. (line 536)\n* jp-ocr-b-add: Tabular. (line 540)\n* jp-ocr-hand: Tabular. (line 544)\n* jp-ocr-hand-add: Tabular. (line 548)\n* js: Tabular. (line 559)\n* JUSI.B1.002, aliases and source: Tabular. (line 559)\n* JUSI.B1.003-mac, aliases and source: Tabular. (line 563)\n* JUSI.B1.003-serb, aliases and source: Tabular. (line 567)\n* Kamenicky: Others. (line 25)\n* katakana: Tabular. (line 524)\n* KEYBCS2: Others. (line 25)\n* KOI-7, aliases and source: Tabular. (line 571)\n* KOI-8CS2: Others. (line 33)\n* KOI-8L2: Tabular. (line 130)\n* KOI-8, aliases and source: Tabular. (line 574)\n* KOI8-R, aliases: libiconv. (line 179)\n* KOI8-R, aliases and source: Tabular. (line 578)\n* KOI8-RU: libiconv. (line 184)\n* KOI8-RU, aliases and source: Tabular. (line 582)\n* KOI8-U: libiconv. (line 182)\n* KOI8-U, aliases and source: Tabular. (line 585)\n* koi8l2: Tabular. (line 130)\n* KOREAN: libiconv. (line 311)\n* KSC5601-1987: libiconv. (line 311)\n* KSC5601-1987, not recognised by recode: Tabular. (line 23)\n* KSC5601-1989: libiconv. (line 311)\n* KSC5601, aliases: libiconv. (line 311)\n* KSC5601.1987-0: libiconv. (line 311)\n* KSC5636, aliases and source: Tabular. (line 589)\n* KSX1001(1992): libiconv. (line 311)\n* L1: libiconv. (line 113)\n* l1: Tabular. (line 419)\n* L2: libiconv. (line 118)\n* l2: Tabular. (line 444)\n* L3: libiconv. (line 123)\n* l3: Tabular. (line 449)\n* L4: libiconv. (line 128)\n* l4: Tabular. (line 454)\n* L5: libiconv. (line 153)\n* l5: Tabular. (line 479)\n* L6: libiconv. (line 158)\n* L6 <1>: Tabular. (line 424)\n* L7: libiconv. (line 163)\n* l7: Tabular. (line 429)\n* L8: libiconv. (line 167)\n* l8: Tabular. (line 434)\n* l9: Tabular. (line 439)\n* lap: Tabular. (line 694)\n* LaTeX, a charset: LaTeX. (line 6)\n* Latin-1: ISO 8859. (line 32)\n* Latin-greek-1, aliases and source: Tabular. (line 592)\n* latin-greek, aliases and source: Tabular. (line 681)\n* latin-lap: Tabular. (line 694)\n* LATIN1: libiconv. (line 113)\n* latin1: Tabular. (line 419)\n* latin1-2-5: Tabular. (line 516)\n* LATIN2: libiconv. (line 118)\n* latin2: Tabular. (line 444)\n* LATIN3: libiconv. (line 123)\n* latin3: Tabular. (line 449)\n* LATIN4: libiconv. (line 128)\n* latin4: Tabular. (line 454)\n* LATIN5: libiconv. (line 153)\n* latin5: Tabular. (line 479)\n* LATIN6: libiconv. (line 158)\n* latin6: Tabular. (line 424)\n* LATIN7: libiconv. (line 163)\n* latin7: Tabular. (line 429)\n* LATIN8: libiconv. (line 167)\n* latin8: Tabular. (line 434)\n* latin9: Tabular. (line 439)\n* libiconv: libiconv. (line 20)\n* libiconv, not in requests: Charset overview. (line 32)\n* lin: African. (line 17)\n* lingala: African. (line 17)\n* ltex: LaTeX. (line 6)\n* MAC: libiconv. (line 223)\n* mac: Tabular. (line 687)\n* mac <1>: Apple-Mac. (line 29)\n* mac-is, aliases and source: Tabular. (line 685)\n* MacArabic: libiconv. (line 245)\n* macce: Tabular. (line 691)\n* macce <1>: Apple-Mac. (line 29)\n* MacCentralEurope: libiconv. (line 227)\n* MacCroatian: libiconv. (line 231)\n* MacCyrillic: libiconv. (line 235)\n* macedonian: Tabular. (line 563)\n* MacGreek: libiconv. (line 239)\n* MacHebrew: libiconv. (line 243)\n* MacIceland: libiconv. (line 229)\n* Macintosh: libiconv. (line 223)\n* macintoshce, aliases and source: Tabular. (line 691)\n* macintoshce, and its aliases: Apple-Mac. (line 29)\n* macintosh, a charset, and its aliases: Apple-Mac. (line 29)\n* macintosh, aliases and source: Tabular. (line 687)\n* MacRoman, aliases: libiconv. (line 223)\n* MacRomania: libiconv. (line 233)\n* MacThai: libiconv. (line 247)\n* MacTurkish: libiconv. (line 241)\n* MacUkraine: libiconv. (line 237)\n* mnemonic, an alias for RFC1345 charset: Tabular. (line 51)\n* MSKANJI: libiconv. (line 322)\n* MS-ANSI: libiconv. (line 193)\n* ms-ansi: Tabular. (line 94)\n* MS-ARAB: libiconv. (line 205)\n* ms-arab: Tabular. (line 110)\n* MS-CYRL: libiconv. (line 190)\n* ms-cyrl: Tabular. (line 90)\n* MS-EE: libiconv. (line 187)\n* ms-ee: Tabular. (line 86)\n* MS-GREEK: libiconv. (line 196)\n* ms-greek: Tabular. (line 98)\n* MS-HEBR: libiconv. (line 202)\n* ms-hebr: Tabular. (line 106)\n* MS-TURK: libiconv. (line 199)\n* ms-turk: Tabular. (line 102)\n* MSDOS: IBM-PC. (line 6)\n* MSZ7795.3, aliases and source: Tabular. (line 596)\n* Mule, a charset: Mule. (line 6)\n* MuleLao-1: libiconv. (line 262)\n* NATS-DANO-ADD, aliases and source: Tabular. (line 604)\n* NATS-DANO, aliases and source: Tabular. (line 600)\n* NATS-SEFI-ADD, aliases and source: Tabular. (line 612)\n* NATS-SEFI, aliases and source: Tabular. (line 608)\n* NCNC00-10, aliases and source: Tabular. (line 616)\n* NCNC00-10(81): Tabular. (line 616)\n* next: Tabular. (line 636)\n* NEXTSTEP: libiconv. (line 253)\n* NeXTSTEP, aliases and source: Tabular. (line 636)\n* NFZ62-010(1973), aliases and source: Tabular. (line 624)\n* NFZ62-010, aliases and source: Tabular. (line 620)\n* no: Tabular. (line 628)\n* no2: Tabular. (line 632)\n* NOS: CDC-NOS. (line 6)\n* NS4551-1, aliases and source: Tabular. (line 628)\n* NS4551-2, aliases and source: Tabular. (line 632)\n* o1: Dump. (line 25)\n* o2: Dump. (line 31)\n* o4: Dump. (line 38)\n* Octal-1: Dump. (line 25)\n* Octal-2: Dump. (line 31)\n* Octal-4: Dump. (line 38)\n* os2latin1: Tabular. (line 228)\n* pc: IBM-PC. (line 6)\n* pcl2: Tabular. (line 319)\n* pclatin2: Tabular. (line 319)\n* PT, aliases and source: Tabular. (line 640)\n* PT2, aliases and source: Tabular. (line 644)\n* QNX, an alias for a charset: Icon-QNX. (line 6)\n* QP: MIME. (line 16)\n* quote-printable: MIME. (line 16)\n* Quoted-Printable: MIME. (line 16)\n* R8: libiconv. (line 250)\n* r8: Tabular. (line 676)\n* ref: Tabular. (line 504)\n* RFC1345, a charset, and its aliases: Tabular. (line 51)\n* RFC1866: HTML. (line 38)\n* RFC2070: HTML. (line 46)\n* ROMAN8: libiconv. (line 250)\n* roman8: Tabular. (line 676)\n* rune: UCS-2. (line 29)\n* sami, aliases and source: Tabular. (line 694)\n* sango: African. (line 17)\n* se: Tabular. (line 648)\n* se2: Tabular. (line 652)\n* SEN850200B, aliases and source: Tabular. (line 648)\n* SEN850200C, aliases and source: Tabular. (line 652)\n* serbian: Tabular. (line 567)\n* SHIFTJIS: libiconv. (line 322)\n* SHIFT-JIS: libiconv. (line 322)\n* SJIS, aliases: libiconv. (line 322)\n* SS636127: Tabular. (line 648)\n* STSEV358-88: Tabular. (line 215)\n* swabytes: Permutations. (line 15)\n* t-bambara: African. (line 10)\n* t-bra: African. (line 10)\n* t-ewondo: African. (line 10)\n* t-fra: African. (line 24)\n* t-francais: African. (line 24)\n* t-fulfude: African. (line 10)\n* t-lin: African. (line 17)\n* t-lingala: African. (line 17)\n* t-sango: African. (line 17)\n* t-wolof: African. (line 17)\n* T.101-G2, not recognised by recode: Tabular. (line 19)\n* T.61-7bit, aliases and source: Tabular. (line 656)\n* T.61-8bit, not recognised by recode: Tabular. (line 19)\n* T1: Others. (line 29)\n* TCVN-5712: libiconv. (line 277)\n* TCVN, aliases: libiconv. (line 277)\n* TCVN, for Vienamese: Vietnamese. (line 11)\n* TCVN5712-1: libiconv. (line 277)\n* TCVN5712-1(1993): libiconv. (line 277)\n* test15: Test. (line 35)\n* test16: Test. (line 42)\n* test7: Test. (line 25)\n* test8: Test. (line 30)\n* texi: Texinfo. (line 6)\n* Texinfo, a charset: Texinfo. (line 6)\n* Texte: Texte. (line 6)\n* TF-16: UTF-16. (line 22)\n* TF-7: UTF-7. (line 12)\n* TF-8: UTF-8. (line 81)\n* ti: Texinfo. (line 6)\n* TIS-620, aliases: libiconv. (line 267)\n* TIS620: libiconv. (line 267)\n* TIS620-0: libiconv. (line 267)\n* TIS620.2529-1: libiconv. (line 267)\n* TIS620.2533-0: libiconv. (line 267)\n* TIS620.2533-1: libiconv. (line 267)\n* tree, a special charset: Surfaces. (line 26)\n* txte: Texte. (line 6)\n* u2: UCS-2. (line 29)\n* u4: UCS-4. (line 10)\n* u6: UTF-16. (line 22)\n* u7: UTF-7. (line 12)\n* u8: UTF-8. (line 81)\n* UCS: Universal. (line 11)\n* UCS-2: UCS-2. (line 6)\n* UCS-2-INTERNAL: libiconv. (line 102)\n* UCS-2-SWAPPED: libiconv. (line 104)\n* UCS-2, aliases: libiconv. (line 74)\n* UCS-2BE, aliases: libiconv. (line 78)\n* UCS-2LE, aliases: libiconv. (line 82)\n* UCS-4: UCS-4. (line 6)\n* UCS-4-INTERNAL: libiconv. (line 106)\n* UCS-4-SWAPPED: libiconv. (line 108)\n* UCS-4, aliases: libiconv. (line 85)\n* UCS-4BE: libiconv. (line 88)\n* UCS-4LE: libiconv. (line 90)\n* UHC: libiconv. (line 367)\n* uk: Tabular. (line 78)\n* UNICODE-1-1: libiconv. (line 78)\n* UNICODE-1-1-UTF-7: libiconv. (line 98)\n* UNICODE-1-1-UTF-7, and aliases: UTF-7. (line 12)\n* Unicode, an alias for UTF-16: UTF-16. (line 22)\n* UNICODEBIG: libiconv. (line 78)\n* UNICODELITTLE: libiconv. (line 82)\n* US: libiconv. (line 65)\n* us: Tabular. (line 69)\n* us <1>: ASCII. (line 6)\n* US-ASCII: Tabular. (line 69)\n* US-ASCII <1>: ASCII. (line 6)\n* US-ASCII, aliases: libiconv. (line 65)\n* us-dk, not recognised by recode: Tabular. (line 16)\n* UTF-1: Universal. (line 18)\n* UTF-16: libiconv. (line 92)\n* UTF-16, and aliases: UTF-16. (line 6)\n* UTF-16BE: libiconv. (line 94)\n* UTF-16LE: libiconv. (line 96)\n* UTF-7: UTF-7. (line 6)\n* UTF-7, aliases: libiconv. (line 98)\n* UTF-8: UTF-8. (line 6)\n* UTF-8, aliases: UTF-8. (line 81)\n* UTF-8, aliases <1>: libiconv. (line 71)\n* UTF-FSS: UTF-8. (line 81)\n* UTF8: libiconv. (line 71)\n* VIQR: Vietnamese. (line 23)\n* VISCII: Vietnamese. (line 16)\n* VISCII, aliases: libiconv. (line 274)\n* VISCII1.1-1: libiconv. (line 274)\n* VN1, maybe not available: Vietnamese. (line 11)\n* VN2, maybe not available: Vietnamese. (line 11)\n* VN3, maybe not available: Vietnamese. (line 11)\n* VNI: Vietnamese. (line 27)\n* VPS: Vietnamese. (line 20)\n* WCHART: libiconv. (line 378)\n* WINBALTRIM: libiconv. (line 208)\n* WinBaltRim: Tabular. (line 114)\n* WINDOWS-1250: libiconv. (line 187)\n* windows-1250: Tabular. (line 86)\n* WINDOWS-1251: libiconv. (line 190)\n* windows-1251: Tabular. (line 90)\n* WINDOWS-1252: libiconv. (line 193)\n* windows-1252: Tabular. (line 94)\n* WINDOWS-1253: libiconv. (line 196)\n* windows-1253: Tabular. (line 98)\n* WINDOWS-1254: libiconv. (line 199)\n* windows-1254: Tabular. (line 102)\n* WINDOWS-1255: libiconv. (line 202)\n* windows-1255: Tabular. (line 106)\n* WINDOWS-1256: libiconv. (line 205)\n* windows-1256: Tabular. (line 110)\n* WINDOWS-1257: libiconv. (line 208)\n* windows-1257: Tabular. (line 114)\n* WINDOWS-1258: libiconv. (line 211)\n* WINDOWS-874: libiconv. (line 271)\n* wolof: African. (line 17)\n* X0201: libiconv. (line 286)\n* X0201 <1>: Tabular. (line 556)\n* x0201-7: Tabular. (line 524)\n* X0208: libiconv. (line 290)\n* X0212: libiconv. (line 295)\n* x1: Dump. (line 66)\n* x2: Dump. (line 73)\n* x4: Dump. (line 80)\n* XML-standalone: HTML. (line 27)\n* yu: Tabular. (line 559)\n\n", "subsections": [] } }, "flags": [], "examples": [], "see_also": [] }