{ "content": [ { "type": "text", "text": "# pcrepattern (info)\n\n## NAME\n\nPCRE - Perl-compatible regular expressions\n\n## Sections\n\n- **NAME**\n- **PCRE REGULAR EXPRESSION DETAILS**\n- **SPECIAL START-OF-PATTERN ITEMS**\n- **EBCDIC CHARACTER CODES**\n- **CHARACTERS AND METACHARACTERS**\n- **BACKSLASH**\n- **CIRCUMFLEX AND DOLLAR**\n- **MATCHING A SINGLE DATA UNIT**\n- **SQUARE BRACKETS AND CHARACTER CLASSES**\n- **POSIX CHARACTER CLASSES**\n- **COMPATIBILITY FEATURE FOR WORD BOUNDARIES**\n- **VERTICAL BAR**\n- **INTERNAL OPTION SETTING**\n- **SUBPATTERNS**\n- **DUPLICATE SUBPATTERN NUMBERS**\n- **NAMED SUBPATTERNS**\n- **REPETITION**\n- **ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS**\n- **BACK REFERENCES**\n- **ASSERTIONS**\n- **CONDITIONAL SUBPATTERNS**\n- **COMMENTS**\n- **RECURSIVE PATTERNS**\n- **SUBPATTERNS AS SUBROUTINES**\n- **ONIGURUMA SUBROUTINE SYNTAX**\n- **CALLOUTS**\n- **BACKTRACKING CONTROL**\n- **SEE ALSO**\n- **AUTHOR**\n- **REVISION**\n\nUse structuredContent.sections for detailed options, examples, and full documentation.\n" } ], "structuredContent": { "command": "pcrepattern", "section": "", "mode": "info", "summary": "PCRE - Perl-compatible regular expressions", "synopsis": null, "tldr_summary": null, "tldr_examples": [], "tldr_source": null, "flags": [], "examples": [], "see_also": [ { "name": "pcreapi", "section": "3", "url": "https://www.chedong.com/phpMan.php/man/pcreapi/3/json" }, { "name": "pcrecallout", "section": "3", "url": "https://www.chedong.com/phpMan.php/man/pcrecallout/3/json" }, { "name": "pcrematching", "section": "3", "url": "https://www.chedong.com/phpMan.php/man/pcrematching/3/json" }, { "name": "pcresyntax", "section": "3", "url": "https://www.chedong.com/phpMan.php/man/pcresyntax/3/json" }, { "name": "pcre", "section": "3", "url": "https://www.chedong.com/phpMan.php/man/pcre/3/json" }, { "name": "pcre16", "section": "3", "url": "https://www.chedong.com/phpMan.php/man/pcre16/3/json" }, { "name": "pcre32", "section": "3", "url": "https://www.chedong.com/phpMan.php/man/pcre32/3/json" } ], "section_outline": [ { "name": "NAME", "lines": 2, "subsections": [] }, { "name": "PCRE REGULAR EXPRESSION DETAILS", "lines": 25, "subsections": [] }, { "name": "SPECIAL START-OF-PATTERN ITEMS", "lines": 116, "subsections": [] }, { "name": "EBCDIC CHARACTER CODES", "lines": 6, "subsections": [] }, { "name": "CHARACTERS AND METACHARACTERS", "lines": 55, "subsections": [] }, { "name": "BACKSLASH", "lines": 670, "subsections": [] }, { "name": "CIRCUMFLEX AND DOLLAR", "lines": 80, "subsections": [] }, { "name": "MATCHING A SINGLE DATA UNIT", "lines": 36, "subsections": [] }, { "name": "SQUARE BRACKETS AND CHARACTER CLASSES", "lines": 109, "subsections": [] }, { "name": "POSIX CHARACTER CLASSES", "lines": 79, "subsections": [] }, { "name": "COMPATIBILITY FEATURE FOR WORD BOUNDARIES", "lines": 17, "subsections": [] }, { "name": "VERTICAL BAR", "lines": 12, "subsections": [] }, { "name": "INTERNAL OPTION SETTING", "lines": 59, "subsections": [] }, { "name": "SUBPATTERNS", "lines": 52, "subsections": [] }, { "name": "DUPLICATE SUBPATTERN NUMBERS", "lines": 41, "subsections": [] }, { "name": "NAMED SUBPATTERNS", "lines": 76, "subsections": [] }, { "name": "REPETITION", "lines": 160, "subsections": [] }, { "name": "ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS", "lines": 101, "subsections": [] }, { "name": "BACK REFERENCES", "lines": 115, "subsections": [] }, { "name": "ASSERTIONS", "lines": 174, "subsections": [] }, { "name": "CONDITIONAL SUBPATTERNS", "lines": 135, "subsections": [] }, { "name": "COMMENTS", "lines": 26, "subsections": [] }, { "name": "RECURSIVE PATTERNS", "lines": 206, "subsections": [] }, { "name": "SUBPATTERNS AS SUBROUTINES", "lines": 39, "subsections": [] }, { "name": "ONIGURUMA SUBROUTINE SYNTAX", "lines": 18, "subsections": [] }, { "name": "CALLOUTS", "lines": 46, "subsections": [] }, { "name": "BACKTRACKING CONTROL", "lines": 383, "subsections": [] }, { "name": "SEE ALSO", "lines": 3, "subsections": [] }, { "name": "AUTHOR", "lines": 4, "subsections": [] }, { "name": "REVISION", "lines": 4, "subsections": [] } ], "sections": { "NAME": { "content": "PCRE - Perl-compatible regular expressions\n", "subsections": [] }, "PCRE REGULAR EXPRESSION DETAILS": { "content": "The syntax and semantics of the regular expressions that are supported\nby PCRE are described in detail below. There is a quick-reference syn-\ntax summary in the pcresyntax page. PCRE tries to match Perl syntax and\nsemantics as closely as it can. PCRE also supports some alternative\nregular expression syntax (which does not conflict with the Perl syn-\ntax) in order to provide some compatibility with regular expressions in\nPython, .NET, and Oniguruma.\n\nPerl's regular expressions are described in its own documentation, and\nregular expressions in general are covered in a number of books, some\nof which have copious examples. Jeffrey Friedl's \"Mastering Regular Ex-\npressions\", published by O'Reilly, covers regular expressions in great\ndetail. This description of PCRE's regular expressions is intended as\nreference material.\n\nThis document discusses the patterns that are supported by PCRE when\none its main matching functions, pcreexec() (8-bit) or\npcre[16|32]exec() (16- or 32-bit), is used. PCRE also has alternative\nmatching functions, pcredfaexec() and pcre[16|32dfaexec(), which\nmatch using a different algorithm that is not Perl-compatible. Some of\nthe features discussed below are not available when DFA matching is\nused. The advantages and disadvantages of the alternative functions,\nand how they differ from the normal functions, are discussed in the\npcrematching page.\n", "subsections": [] }, "SPECIAL START-OF-PATTERN ITEMS": { "content": "A number of options that can be passed to pcrecompile() can also be\nset by special items at the start of a pattern. These are not Perl-com-\npatible, but are provided to make these options accessible to pattern\nwriters who are not able to change the program that processes the pat-\ntern. Any number of these items may appear, but they must all be to-\ngether right at the start of the pattern string, and the letters must\nbe in upper case.\n\nUTF support\n\nThe original operation of PCRE was on strings of one-byte characters.\nHowever, there is now also support for UTF-8 strings in the original\nlibrary, an extra library that supports 16-bit and UTF-16 character\nstrings, and a third library that supports 32-bit and UTF-32 character\nstrings. To use these features, PCRE must be built to include appropri-\nate support. When using UTF strings you must either call the compiling\nfunction with the PCREUTF8, PCREUTF16, or PCREUTF32 option, or the\npattern must start with one of these special sequences:\n\n(*UTF8)\n(*UTF16)\n(*UTF32)\n(*UTF)\n\n(*UTF) is a generic sequence that can be used with any of the li-\nbraries. Starting a pattern with such a sequence is equivalent to set-\nting the relevant option. How setting a UTF mode affects pattern match-\ning is mentioned in several places below. There is also a summary of\nfeatures in the pcreunicode page.\n\nSome applications that allow their users to supply patterns may wish to\nrestrict them to non-UTF data for security reasons. If the\nPCRENEVERUTF option is set at compile time, (*UTF) etc. are not al-\nlowed, and their appearance causes an error.\n\nUnicode property support\n\nAnother special sequence that may appear at the start of a pattern is\n(*UCP). This has the same effect as setting the PCREUCP option: it\ncauses sequences such as \\d and \\w to use Unicode properties to deter-\nmine character types, instead of recognizing only characters with codes\nless than 128 via a lookup table.\n\nDisabling auto-possessification\n\nIf a pattern starts with (*NOAUTOPOSSESS), it has the same effect as\nsetting the PCRENOAUTOPOSSESS option at compile time. This stops\nPCRE from making quantifiers possessive when what follows cannot match\nthe repeated item. For example, by default a+b is treated as a++b. For\nmore details, see the pcreapi documentation.\n\nDisabling start-up optimizations\n\nIf a pattern starts with (*NOSTARTOPT), it has the same effect as\nsetting the PCRENOSTARTOPTIMIZE option either at compile or matching\ntime. This disables several optimizations for quickly reaching \"no\nmatch\" results. For more details, see the pcreapi documentation.\n\nNewline conventions\n\nPCRE supports five different conventions for indicating line breaks in\nstrings: a single CR (carriage return) character, a single LF (line-\nfeed) character, the two-character sequence CRLF, any of the three pre-\nceding, or any Unicode newline sequence. The pcreapi page has further\ndiscussion about newlines, and shows how to set the newline convention\nin the options arguments for the compiling and matching functions.\n\nIt is also possible to specify a newline convention by starting a pat-\ntern string with one of the following five sequences:\n\n(*CR) carriage return\n(*LF) linefeed\n(*CRLF) carriage return, followed by linefeed\n(*ANYCRLF) any of the three above\n(*ANY) all Unicode newline sequences\n\nThese override the default and the options given to the compiling func-\ntion. For example, on a Unix system where LF is the default newline se-\nquence, the pattern\n\n(*CR)a.b\n\nchanges the convention to CR. That pattern matches \"a\\nb\" because LF is\nno longer a newline. If more than one of these settings is present, the\nlast one is used.\n\nThe newline convention affects where the circumflex and dollar asser-\ntions are true. It also affects the interpretation of the dot metachar-\nacter when PCREDOTALL is not set, and the behaviour of \\N. However, it\ndoes not affect what the \\R escape sequence matches. By default, this\nis any Unicode newline sequence, for Perl compatibility. However, this\ncan be changed; see the description of \\R in the section entitled \"New-\nline sequences\" below. A change of \\R setting can be combined with a\nchange of newline convention.\n\nSetting match and recursion limits\n\nThe caller of pcreexec() can set a limit on the number of times the\ninternal match() function is called and on the maximum depth of recur-\nsive calls. These facilities are provided to catch runaway matches that\nare provoked by patterns with huge matching trees (a typical example is\na pattern with nested unlimited repeats) and to avoid running out of\nsystem stack by too much recursion. When one of these limits is\nreached, pcreexec() gives an error return. The limits can also be set\nby items at the start of the pattern of the form\n\n(*LIMITMATCH=d)\n(*LIMITRECURSION=d)\n\nwhere d is any number of decimal digits. However, the value of the set-\nting must be less than the value set (or defaulted) by the caller of\npcreexec() for it to have any effect. In other words, the pattern\nwriter can lower the limits set by the programmer, but not raise them.\nIf there is more than one setting of one of these limits, the lower\nvalue is used.\n", "subsections": [] }, "EBCDIC CHARACTER CODES": { "content": "PCRE can be compiled to run in an environment that uses EBCDIC as its\ncharacter code rather than ASCII or Unicode (typically a mainframe sys-\ntem). In the sections below, character code values are ASCII or Uni-\ncode; in an EBCDIC environment these characters may have different code\nvalues, and there are no code points greater than 255.\n", "subsections": [] }, "CHARACTERS AND METACHARACTERS": { "content": "A regular expression is a pattern that is matched against a subject\nstring from left to right. Most characters stand for themselves in a\npattern, and match the corresponding characters in the subject. As a\ntrivial example, the pattern\n\nThe quick brown fox\n\nmatches a portion of a subject string that is identical to itself. When\ncaseless matching is specified (the PCRECASELESS option), letters are\nmatched independently of case. In a UTF mode, PCRE always understands\nthe concept of case for characters whose values are less than 128, so\ncaseless matching is always possible. For characters with higher val-\nues, the concept of case is supported if PCRE is compiled with Unicode\nproperty support, but not otherwise. If you want to use caseless\nmatching for characters 128 and above, you must ensure that PCRE is\ncompiled with Unicode property support as well as with UTF support.\n\nThe power of regular expressions comes from the ability to include al-\nternatives and repetitions in the pattern. These are encoded in the\npattern by the use of metacharacters, which do not stand for themselves\nbut instead are interpreted in some special way.\n\nThere are two different sets of metacharacters: those that are recog-\nnized anywhere in the pattern except within square brackets, and those\nthat are recognized within square brackets. Outside square brackets,\nthe metacharacters are as follows:\n\n\\ general escape character with several uses\n^ assert start of string (or line, in multiline mode)\n$ assert end of string (or line, in multiline mode)\n. match any character except newline (by default)\n[ start character class definition\n| start of alternative branch\n( start subpattern\n) end subpattern\n? extends the meaning of (\nalso 0 or 1 quantifier\nalso quantifier minimizer\n* 0 or more quantifier\n+ 1 or more quantifier\nalso \"possessive quantifier\"\n{ start min/max quantifier\n\nPart of a pattern that is in square brackets is called a \"character\nclass\". In a character class the only metacharacters are:\n\n\\ general escape character\n^ negate the class, but only if the first character\n- indicates character range\n[ POSIX character class (only if followed by POSIX\nsyntax)\n] terminates the character class\n\nThe following sections describe the use of each of the metacharacters.\n", "subsections": [] }, "BACKSLASH": { "content": "The backslash character has several uses. Firstly, if it is followed by\na character that is not a number or a letter, it takes away any special\nmeaning that character may have. This use of backslash as an escape\ncharacter applies both inside and outside character classes.\n\nFor example, if you want to match a * character, you write \\* in the\npattern. This escaping action applies whether or not the following\ncharacter would otherwise be interpreted as a metacharacter, so it is\nalways safe to precede a non-alphanumeric with backslash to specify\nthat it stands for itself. In particular, if you want to match a back-\nslash, you write \\\\.\n\nIn a UTF mode, only ASCII numbers and letters have any special meaning\nafter a backslash. All other characters (in particular, those whose\ncodepoints are greater than 127) are treated as literals.\n\nIf a pattern is compiled with the PCREEXTENDED option, most white\nspace in the pattern (other than in a character class), and characters\nbetween a # outside a character class and the next newline, inclusive,\nare ignored. An escaping backslash can be used to include a white space\nor # character as part of the pattern.\n\nIf you want to remove the special meaning from a sequence of charac-\nters, you can do so by putting them between \\Q and \\E. This is differ-\nent from Perl in that $ and @ are handled as literals in \\Q...\\E se-\nquences in PCRE, whereas in Perl, $ and @ cause variable interpolation.\nNote the following examples:\n\nPattern PCRE matches Perl matches\n\n\\Qabc$xyz\\E abc$xyz abc followed by the\ncontents of $xyz\n\\Qabc\\$xyz\\E abc\\$xyz abc\\$xyz\n\\Qabc\\E\\$\\Qxyz\\E abc$xyz abc$xyz\n\nThe \\Q...\\E sequence is recognized both inside and outside character\nclasses. An isolated \\E that is not preceded by \\Q is ignored. If \\Q\nis not followed by \\E later in the pattern, the literal interpretation\ncontinues to the end of the pattern (that is, \\E is assumed at the\nend). If the isolated \\Q is inside a character class, this causes an\nerror, because the character class is not terminated.\n\nNon-printing characters\n\nA second use of backslash provides a way of encoding non-printing char-\nacters in patterns in a visible manner. There is no restriction on the\nappearance of non-printing characters, apart from the binary zero that\nterminates a pattern, but when a pattern is being prepared by text\nediting, it is often easier to use one of the following escape se-\nquences than the binary character it represents. In an ASCII or Uni-\ncode environment, these escapes are as follows:\n\n\\a alarm, that is, the BEL character (hex 07)\n\\cx \"control-x\", where x is any ASCII character\n\\e escape (hex 1B)\n\\f form feed (hex 0C)\n\\n linefeed (hex 0A)\n\\r carriage return (hex 0D)\n\\t tab (hex 09)\n\\0dd character with octal code 0dd\n\\ddd character with octal code ddd, or back reference\n\\o{ddd..} character with octal code ddd..\n\\xhh character with hex code hh\n\\x{hhh..} character with hex code hhh.. (non-JavaScript mode)\n\\uhhhh character with hex code hhhh (JavaScript mode only)\n\nThe precise effect of \\cx on ASCII characters is as follows: if x is a\nlower case letter, it is converted to upper case. Then bit 6 of the\ncharacter (hex 40) is inverted. Thus \\cA to \\cZ become hex 01 to hex 1A\n(A is 41, Z is 5A), but \\c{ becomes hex 3B ({ is 7B), and \\c; becomes\nhex 7B (; is 3B). If the data item (byte or 16-bit value) following \\c\nhas a value greater than 127, a compile-time error occurs. This locks\nout non-ASCII characters in all modes.\n\nWhen PCRE is compiled in EBCDIC mode, \\a, \\e, \\f, \\n, \\r, and \\t gener-\nate the appropriate EBCDIC code values. The \\c escape is processed as\nspecified for Perl in the perlebcdic document. The only characters that\nare allowed after \\c are A-Z, a-z, or one of @, [, \\, ], ^, , or ?.\nAny other character provokes a compile-time error. The sequence \\@ en-\ncodes character code 0; the letters (in either case) encode characters\n1-26 (hex 01 to hex 1A); [, \\, ], ^, and encode characters 27-31 (hex\n1B to hex 1F), and \\? becomes either 255 (hex FF) or 95 (hex 5F).\n\nThus, apart from \\?, these escapes generate the same character code\nvalues as they do in an ASCII environment, though the meanings of the\nvalues mostly differ. For example, \\G always generates code value 7,\nwhich is BEL in ASCII but DEL in EBCDIC.\n\nThe sequence \\? generates DEL (127, hex 7F) in an ASCII environment,\nbut because 127 is not a control character in EBCDIC, Perl makes it\ngenerate the APC character. Unfortunately, there are several variants\nof EBCDIC. In most of them the APC character has the value 255 (hex\nFF), but in the one Perl calls POSIX-BC its value is 95 (hex 5F). If\ncertain other characters have POSIX-BC values, PCRE makes \\? generate\n95; otherwise it generates 255.\n\nAfter \\0 up to two further octal digits are read. If there are fewer\nthan two digits, just those that are present are used. Thus the se-\nquence \\0\\x\\015 specifies two binary zeros followed by a CR character\n(code value 13). Make sure you supply two digits after the initial zero\nif the pattern character that follows is itself an octal digit.\n\nThe escape \\o must be followed by a sequence of octal digits, enclosed\nin braces. An error occurs if this is not the case. This escape is a\nrecent addition to Perl; it provides way of specifying character code\npoints as octal numbers greater than 0777, and it also allows octal\nnumbers and back references to be unambiguously specified.\n\nFor greater clarity and unambiguity, it is best to avoid following \\ by\na digit greater than zero. Instead, use \\o{} or \\x{} to specify charac-\nter numbers, and \\g{} to specify back references. The following para-\ngraphs describe the old, ambiguous syntax.\n\nThe handling of a backslash followed by a digit other than 0 is compli-\ncated, and Perl has changed in recent releases, causing PCRE also to\nchange. Outside a character class, PCRE reads the digit and any follow-\ning digits as a decimal number. If the number is less than 8, or if\nthere have been at least that many previous capturing left parentheses\nin the expression, the entire sequence is taken as a back reference. A\ndescription of how this works is given later, following the discussion\nof parenthesized subpatterns.\n\nInside a character class, or if the decimal number following \\ is\ngreater than 7 and there have not been that many capturing subpatterns,\nPCRE handles \\8 and \\9 as the literal characters \"8\" and \"9\", and oth-\nerwise re-reads up to three octal digits following the backslash, using\nthem to generate a data character. Any subsequent digits stand for\nthemselves. For example:\n\n\\040 is another way of writing an ASCII space\n\\40 is the same, provided there are fewer than 40\nprevious capturing subpatterns\n\\7 is always a back reference\n\\11 might be a back reference, or another way of\nwriting a tab\n\\011 is always a tab\n\\0113 is a tab followed by the character \"3\"\n\\113 might be a back reference, otherwise the\ncharacter with octal code 113\n\\377 might be a back reference, otherwise\nthe value 255 (decimal)\n\\81 is either a back reference, or the two\ncharacters \"8\" and \"1\"\n\nNote that octal values of 100 or greater that are specified using this\nsyntax must not be introduced by a leading zero, because no more than\nthree octal digits are ever read.\n\nBy default, after \\x that is not followed by {, from zero to two hexa-\ndecimal digits are read (letters can be in upper or lower case). Any\nnumber of hexadecimal digits may appear between \\x{ and }. If a charac-\nter other than a hexadecimal digit appears between \\x{ and }, or if\nthere is no terminating }, an error occurs.\n\nIf the PCREJAVASCRIPTCOMPAT option is set, the interpretation of \\x\nis as just described only when it is followed by two hexadecimal dig-\nits. Otherwise, it matches a literal \"x\" character. In JavaScript\nmode, support for code points greater than 256 is provided by \\u, which\nmust be followed by four hexadecimal digits; otherwise it matches a\nliteral \"u\" character.\n\nCharacters whose value is less than 256 can be defined by either of the\ntwo syntaxes for \\x (or by \\u in JavaScript mode). There is no differ-\nence in the way they are handled. For example, \\xdc is exactly the same\nas \\x{dc} (or \\u00dc in JavaScript mode).\n\nConstraints on character values\n\nCharacters that are specified using octal or hexadecimal numbers are\nlimited to certain values, as follows:\n\n8-bit non-UTF mode less than 0x100\n8-bit UTF-8 mode less than 0x10ffff and a valid codepoint\n16-bit non-UTF mode less than 0x10000\n16-bit UTF-16 mode less than 0x10ffff and a valid codepoint\n32-bit non-UTF mode less than 0x100000000\n32-bit UTF-32 mode less than 0x10ffff and a valid codepoint\n\nInvalid Unicode codepoints are the range 0xd800 to 0xdfff (the so-\ncalled \"surrogate\" codepoints), and 0xffef.\n\nEscape sequences in character classes\n\nAll the sequences that define a single character value can be used both\ninside and outside character classes. In addition, inside a character\nclass, \\b is interpreted as the backspace character (hex 08).\n\n\\N is not allowed in a character class. \\B, \\R, and \\X are not special\ninside a character class. Like other unrecognized escape sequences,\nthey are treated as the literal characters \"B\", \"R\", and \"X\" by de-\nfault, but cause an error if the PCREEXTRA option is set. Outside a\ncharacter class, these sequences have different meanings.\n\nUnsupported escape sequences\n\nIn Perl, the sequences \\l, \\L, \\u, and \\U are recognized by its string\nhandler and used to modify the case of following characters. By de-\nfault, PCRE does not support these escape sequences. However, if the\nPCREJAVASCRIPTCOMPAT option is set, \\U matches a \"U\" character, and\n\\u can be used to define a character by code point, as described in the\nprevious section.\n\nAbsolute and relative back references\n\nThe sequence \\g followed by an unsigned or a negative number, option-\nally enclosed in braces, is an absolute or relative back reference. A\nnamed back reference can be coded as \\g{name}. Back references are dis-\ncussed later, following the discussion of parenthesized subpatterns.\n\nAbsolute and relative subroutine calls\n\nFor compatibility with Oniguruma, the non-Perl syntax \\g followed by a\nname or a number enclosed either in angle brackets or single quotes, is\nan alternative syntax for referencing a subpattern as a \"subroutine\".\nDetails are discussed later. Note that \\g{...} (Perl syntax) and\n\\g<...> (Oniguruma syntax) are not synonymous. The former is a back\nreference; the latter is a subroutine call.\n\nGeneric character types\n\nAnother use of backslash is for specifying generic character types:\n\n\\d any decimal digit\n\\D any character that is not a decimal digit\n\\h any horizontal white space character\n\\H any character that is not a horizontal white space character\n\\s any white space character\n\\S any character that is not a white space character\n\\v any vertical white space character\n\\V any character that is not a vertical white space character\n\\w any \"word\" character\n\\W any \"non-word\" character\n\nThere is also the single sequence \\N, which matches a non-newline char-\nacter. This is the same as the \".\" metacharacter when PCREDOTALL is\nnot set. Perl also uses \\N to match characters by name; PCRE does not\nsupport this.\n\nEach pair of lower and upper case escape sequences partitions the com-\nplete set of characters into two disjoint sets. Any given character\nmatches one, and only one, of each pair. The sequences can appear both\ninside and outside character classes. They each match one character of\nthe appropriate type. If the current matching point is at the end of\nthe subject string, all of them fail, because there is no character to\nmatch.\n\nFor compatibility with Perl, \\s did not used to match the VT character\n(code 11), which made it different from the the POSIX \"space\" class.\nHowever, Perl added VT at release 5.18, and PCRE followed suit at re-\nlease 8.34. The default \\s characters are now HT (9), LF (10), VT (11),\nFF (12), CR (13), and space (32), which are defined as white space in\nthe \"C\" locale. This list may vary if locale-specific matching is tak-\ning place. For example, in some locales the \"non-breaking space\" char-\nacter (\\xA0) is recognized as white space, and in others the VT charac-\nter is not.\n\nA \"word\" character is an underscore or any character that is a letter\nor digit. By default, the definition of letters and digits is con-\ntrolled by PCRE's low-valued character tables, and may vary if locale-\nspecific matching is taking place (see \"Locale support\" in the pcreapi\npage). For example, in a French locale such as \"frFR\" in Unix-like\nsystems, or \"french\" in Windows, some character codes greater than 127\nare used for accented letters, and these are then matched by \\w. The\nuse of locales with Unicode is discouraged.\n\nBy default, characters whose code points are greater than 127 never\nmatch \\d, \\s, or \\w, and always match \\D, \\S, and \\W, although this may\nvary for characters in the range 128-255 when locale-specific matching\nis happening. These escape sequences retain their original meanings\nfrom before Unicode support was available, mainly for efficiency rea-\nsons. If PCRE is compiled with Unicode property support, and the\nPCREUCP option is set, the behaviour is changed so that Unicode prop-\nerties are used to determine character types, as follows:\n\n\\d any character that matches \\p{Nd} (decimal digit)\n\\s any character that matches \\p{Z} or \\h or \\v\n\\w any character that matches \\p{L} or \\p{N}, plus underscore\n\nThe upper case escapes match the inverse sets of characters. Note that\n\\d matches only decimal digits, whereas \\w matches any Unicode digit,\nas well as any Unicode letter, and underscore. Note also that PCREUCP\naffects \\b, and \\B because they are defined in terms of \\w and \\W.\nMatching these sequences is noticeably slower when PCREUCP is set.\n\nThe sequences \\h, \\H, \\v, and \\V are features that were added to Perl\nat release 5.10. In contrast to the other sequences, which match only\nASCII characters by default, these always match certain high-valued\ncode points, whether or not PCREUCP is set. The horizontal space char-\nacters are:\n\nU+0009 Horizontal tab (HT)\nU+0020 Space\nU+00A0 Non-break space\nU+1680 Ogham space mark\nU+180E Mongolian vowel separator\nU+2000 En quad\nU+2001 Em quad\nU+2002 En space\nU+2003 Em space\nU+2004 Three-per-em space\nU+2005 Four-per-em space\nU+2006 Six-per-em space\nU+2007 Figure space\nU+2008 Punctuation space\nU+2009 Thin space\nU+200A Hair space\nU+202F Narrow no-break space\nU+205F Medium mathematical space\nU+3000 Ideographic space\n\nThe vertical space characters are:\n\nU+000A Linefeed (LF)\nU+000B Vertical tab (VT)\nU+000C Form feed (FF)\nU+000D Carriage return (CR)\nU+0085 Next line (NEL)\nU+2028 Line separator\nU+2029 Paragraph separator\n\nIn 8-bit, non-UTF-8 mode, only the characters with codepoints less than\n256 are relevant.\n\nNewline sequences\n\nOutside a character class, by default, the escape sequence \\R matches\nany Unicode newline sequence. In 8-bit non-UTF-8 mode \\R is equivalent\nto the following:\n\n(?>\\r\\n|\\n|\\x0b|\\f|\\r|\\x85)\n\nThis is an example of an \"atomic group\", details of which are given be-\nlow. This particular group matches either the two-character sequence\nCR followed by LF, or one of the single characters LF (linefeed,\nU+000A), VT (vertical tab, U+000B), FF (form feed, U+000C), CR (car-\nriage return, U+000D), or NEL (next line, U+0085). The two-character\nsequence is treated as a single unit that cannot be split.\n\nIn other modes, two additional characters whose codepoints are greater\nthan 255 are added: LS (line separator, U+2028) and PS (paragraph sepa-\nrator, U+2029). Unicode character property support is not needed for\nthese characters to be recognized.\n\nIt is possible to restrict \\R to match only CR, LF, or CRLF (instead of\nthe complete set of Unicode line endings) by setting the option\nPCREBSRANYCRLF either at compile time or when the pattern is matched.\n(BSR is an abbrevation for \"backslash R\".) This can be made the default\nwhen PCRE is built; if this is the case, the other behaviour can be re-\nquested via the PCREBSRUNICODE option. It is also possible to spec-\nify these settings by starting a pattern string with one of the follow-\ning sequences:\n\n(*BSRANYCRLF) CR, LF, or CRLF only\n(*BSRUNICODE) any Unicode newline sequence\n\nThese override the default and the options given to the compiling func-\ntion, but they can themselves be overridden by options given to a\nmatching function. Note that these special settings, which are not\nPerl-compatible, are recognized only at the very start of a pattern,\nand that they must be in upper case. If more than one of them is\npresent, the last one is used. They can be combined with a change of\nnewline convention; for example, a pattern can start with:\n\n(*ANY)(*BSRANYCRLF)\n\nThey can also be combined with the (*UTF8), (*UTF16), (*UTF32), (*UTF)\nor (*UCP) special sequences. Inside a character class, \\R is treated as\nan unrecognized escape sequence, and so matches the letter \"R\" by de-\nfault, but causes an error if PCREEXTRA is set.\n\nUnicode character properties\n\nWhen PCRE is built with Unicode character property support, three addi-\ntional escape sequences that match characters with specific properties\nare available. When in 8-bit non-UTF-8 mode, these sequences are of\ncourse limited to testing characters whose codepoints are less than\n256, but they do work in this mode. The extra escape sequences are:\n\n\\p{xx} a character with the xx property\n\\P{xx} a character without the xx property\n\\X a Unicode extended grapheme cluster\n\nThe property names represented by xx above are limited to the Unicode\nscript names, the general category properties, \"Any\", which matches any\ncharacter (including newline), and some special PCRE properties (de-\nscribed in the next section). Other Perl properties such as \"InMusi-\ncalSymbols\" are not currently supported by PCRE. Note that \\P{Any} does\nnot match any characters, so always causes a match failure.\n\nSets of Unicode characters are defined as belonging to certain scripts.\nA character from one of these sets can be matched using a script name.\nFor example:\n\n\\p{Greek}\n\\P{Han}\n\nThose that are not part of an identified script are lumped together as\n\"Common\". The current list of scripts is:\n\nArabic, Armenian, Avestan, Balinese, Bamum, BassaVah, Batak, Bengali,\nBopomofo, Brahmi, Braille, Buginese, Buhid, CanadianAboriginal, Car-\nian, CaucasianAlbanian, Chakma, Cham, Cherokee, Common, Coptic, Cunei-\nform, Cypriot, Cyrillic, Deseret, Devanagari, Duployan, EgyptianHiero-\nglyphs, Elbasan, Ethiopic, Georgian, Glagolitic, Gothic, Grantha,\nGreek, Gujarati, Gurmukhi, Han, Hangul, Hanunoo, Hebrew, Hiragana, Im-\nperialAramaic, Inherited, InscriptionalPahlavi, Inscrip-\ntionalParthian, Javanese, Kaithi, Kannada, Katakana, KayahLi,\nKharoshthi, Khmer, Khojki, Khudawadi, Lao, Latin, Lepcha, Limbu, Lin-\nearA, LinearB, Lisu, Lycian, Lydian, Mahajani, Malayalam, Mandaic,\nManichaean, MeeteiMayek, MendeKikakui, MeroiticCursive, MeroiticHi-\neroglyphs, Miao, Modi, Mongolian, Mro, Myanmar, Nabataean, NewTaiLue,\nNko, Ogham, OlChiki, OldItalic, OldNorthArabian, OldPermic,\nOldPersian, OldSouthArabian, OldTurkic, Oriya, Osmanya, Pa-\nhawhHmong, Palmyrene, PauCinHau, PhagsPa, Phoenician,\nPsalterPahlavi, Rejang, Runic, Samaritan, Saurashtra, Sharada, Sha-\nvian, Siddham, Sinhala, SoraSompeng, Sundanese, SylotiNagri, Syriac,\nTagalog, Tagbanwa, TaiLe, TaiTham, TaiViet, Takri, Tamil, Telugu,\nThaana, Thai, Tibetan, Tifinagh, Tirhuta, Ugaritic, Vai, WarangCiti,\nYi.\n\nEach character has exactly one Unicode general category property, spec-\nified by a two-letter abbreviation. For compatibility with Perl, nega-\ntion can be specified by including a circumflex between the opening\nbrace and the property name. For example, \\p{^Lu} is the same as\n\\P{Lu}.\n\nIf only one letter is specified with \\p or \\P, it includes all the gen-\neral category properties that start with that letter. In this case, in\nthe absence of negation, the curly brackets in the escape sequence are\noptional; these two examples have the same effect:\n\n\\p{L}\n\\pL\n\nThe following general category property codes are supported:\n\nC Other\nCc Control\nCf Format\nCn Unassigned\nCo Private use\nCs Surrogate\n\nL Letter\nLl Lower case letter\nLm Modifier letter\nLo Other letter\nLt Title case letter\nLu Upper case letter\n\nM Mark\nMc Spacing mark\nMe Enclosing mark\nMn Non-spacing mark\n\nN Number\nNd Decimal number\nNl Letter number\nNo Other number\n\nP Punctuation\nPc Connector punctuation\nPd Dash punctuation\nPe Close punctuation\nPf Final punctuation\nPi Initial punctuation\nPo Other punctuation\nPs Open punctuation\n\nS Symbol\nSc Currency symbol\nSk Modifier symbol\nSm Mathematical symbol\nSo Other symbol\n\nZ Separator\nZl Line separator\nZp Paragraph separator\nZs Space separator\n\nThe special property L& is also supported: it matches a character that\nhas the Lu, Ll, or Lt property, in other words, a letter that is not\nclassified as a modifier or \"other\".\n\nThe Cs (Surrogate) property applies only to characters in the range\nU+D800 to U+DFFF. Such characters are not valid in Unicode strings and\nso cannot be tested by PCRE, unless UTF validity checking has been\nturned off (see the discussion of PCRENOUTF8CHECK,\nPCRENOUTF16CHECK and PCRENOUTF32CHECK in the pcreapi page). Perl\ndoes not support the Cs property.\n\nThe long synonyms for property names that Perl supports (such as\n\\p{Letter}) are not supported by PCRE, nor is it permitted to prefix\nany of these properties with \"Is\".\n\nNo character that is in the Unicode table has the Cn (unassigned) prop-\nerty. Instead, this property is assumed for any code point that is not\nin the Unicode table.\n\nSpecifying caseless matching does not affect these escape sequences.\nFor example, \\p{Lu} always matches only upper case letters. This is\ndifferent from the behaviour of current versions of Perl.\n\nMatching characters by Unicode property is not fast, because PCRE has\nto do a multistage table lookup in order to find a character's prop-\nerty. That is why the traditional escape sequences such as \\d and \\w do\nnot use Unicode properties in PCRE by default, though you can make them\ndo so by setting the PCREUCP option or by starting the pattern with\n(*UCP).\n\nExtended grapheme clusters\n\nThe \\X escape matches any number of Unicode characters that form an\n\"extended grapheme cluster\", and treats the sequence as an atomic group\n(see below). Up to and including release 8.31, PCRE matched an ear-\nlier, simpler definition that was equivalent to\n\n(?>\\PM\\pM*)\n\nThat is, it matched a character without the \"mark\" property, followed\nby zero or more characters with the \"mark\" property. Characters with\nthe \"mark\" property are typically non-spacing accents that affect the\npreceding character.\n\nThis simple definition was extended in Unicode to include more compli-\ncated kinds of composite character by giving each character a grapheme\nbreaking property, and creating rules that use these properties to de-\nfine the boundaries of extended grapheme clusters. In releases of PCRE\nlater than 8.31, \\X matches one of these clusters.\n\n\\X always matches at least one character. Then it decides whether to\nadd additional characters according to the following rules for ending a\ncluster:\n\n1. End at the end of the subject string.\n\n2. Do not end between CR and LF; otherwise end after any control char-\nacter.\n\n3. Do not break Hangul (a Korean script) syllable sequences. Hangul\ncharacters are of five types: L, V, T, LV, and LVT. An L character may\nbe followed by an L, V, LV, or LVT character; an LV or V character may\nbe followed by a V or T character; an LVT or T character may be follwed\nonly by a T character.\n\n4. Do not end before extending characters or spacing marks. Characters\nwith the \"mark\" property always have the \"extend\" grapheme breaking\nproperty.\n\n5. Do not end after prepend characters.\n\n6. Otherwise, end the cluster.\n\nPCRE's additional properties\n\nAs well as the standard Unicode properties described above, PCRE sup-\nports four more that make it possible to convert traditional escape se-\nquences such as \\w and \\s to use Unicode properties. PCRE uses these\nnon-standard, non-Perl properties internally when PCREUCP is set. How-\never, they may also be used explicitly. These properties are:\n\nXan Any alphanumeric character\nXps Any POSIX space character\nXsp Any Perl space character\nXwd Any Perl \"word\" character\n\nXan matches characters that have either the L (letter) or the N (num-\nber) property. Xps matches the characters tab, linefeed, vertical tab,\nform feed, or carriage return, and any other character that has the Z\n(separator) property. Xsp is the same as Xps; it used to exclude ver-\ntical tab, for Perl compatibility, but Perl changed, and so PCRE fol-\nlowed at release 8.34. Xwd matches the same characters as Xan, plus un-\nderscore.\n\nThere is another non-standard property, Xuc, which matches any charac-\nter that can be represented by a Universal Character Name in C++ and\nother programming languages. These are the characters $, @, ` (grave\naccent), and all characters with Unicode code points greater than or\nequal to U+00A0, except for the surrogates U+D800 to U+DFFF. Note that\nmost base (ASCII) characters are excluded. (Universal Character Names\nare of the form \\uHHHH or \\UHHHHHHHH where H is a hexadecimal digit.\nNote that the Xuc property does not match these sequences but the char-\nacters that they represent.)\n\nResetting the match start\n\nThe escape sequence \\K causes any previously matched characters not to\nbe included in the final matched sequence. For example, the pattern:\n\nfoo\\Kbar\n\nmatches \"foobar\", but reports that it has matched \"bar\". This feature\nis similar to a lookbehind assertion (described below). However, in\nthis case, the part of the subject before the real match does not have\nto be of fixed length, as lookbehind assertions do. The use of \\K does\nnot interfere with the setting of captured substrings. For example,\nwhen the pattern\n\n(foo)\\Kbar\n\nmatches \"foobar\", the first substring is still set to \"foo\".\n\nPerl documents that the use of \\K within assertions is \"not well de-\nfined\". In PCRE, \\K is acted upon when it occurs inside positive asser-\ntions, but is ignored in negative assertions. Note that when a pattern\nsuch as (?=ab\\K) matches, the reported start of the match can be\ngreater than the end of the match.\n\nSimple assertions\n\nThe final use of backslash is for certain simple assertions. An asser-\ntion specifies a condition that has to be met at a particular point in\na match, without consuming any characters from the subject string. The\nuse of subpatterns for more complicated assertions is described below.\nThe backslashed assertions are:\n\n\\b matches at a word boundary\n\\B matches when not at a word boundary\n\\A matches at the start of the subject\n\\Z matches at the end of the subject\nalso matches before a newline at the end of the subject\n\\z matches only at the end of the subject\n\\G matches at the first matching position in the subject\n\nInside a character class, \\b has a different meaning; it matches the\nbackspace character. If any other of these assertions appears in a\ncharacter class, by default it matches the corresponding literal char-\nacter (for example, \\B matches the letter B). However, if the PCREEX-\nTRA option is set, an \"invalid escape sequence\" error is generated in-\nstead.\n\nA word boundary is a position in the subject string where the current\ncharacter and the previous character do not both match \\w or \\W (i.e.\none matches \\w and the other matches \\W), or the start or end of the\nstring if the first or last character matches \\w, respectively. In a\nUTF mode, the meanings of \\w and \\W can be changed by setting the\nPCREUCP option. When this is done, it also affects \\b and \\B. Neither\nPCRE nor Perl has a separate \"start of word\" or \"end of word\" metase-\nquence. However, whatever follows \\b normally determines which it is.\nFor example, the fragment \\ba matches \"a\" at the start of a word.\n\nThe \\A, \\Z, and \\z assertions differ from the traditional circumflex\nand dollar (described in the next section) in that they only ever match\nat the very start and end of the subject string, whatever options are\nset. Thus, they are independent of multiline mode. These three asser-\ntions are not affected by the PCRENOTBOL or PCRENOTEOL options, which\naffect only the behaviour of the circumflex and dollar metacharacters.\nHowever, if the startoffset argument of pcreexec() is non-zero, indi-\ncating that matching is to start at a point other than the beginning of\nthe subject, \\A can never match. The difference between \\Z and \\z is\nthat \\Z matches before a newline at the end of the string as well as at\nthe very end, whereas \\z matches only at the end.\n\nThe \\G assertion is true only when the current matching position is at\nthe start point of the match, as specified by the startoffset argument\nof pcreexec(). It differs from \\A when the value of startoffset is\nnon-zero. By calling pcreexec() multiple times with appropriate argu-\nments, you can mimic Perl's /g option, and it is in this kind of imple-\nmentation where \\G can be useful.\n\nNote, however, that PCRE's interpretation of \\G, as the start of the\ncurrent match, is subtly different from Perl's, which defines it as the\nend of the previous match. In Perl, these can be different when the\npreviously matched string was empty. Because PCRE does just one match\nat a time, it cannot reproduce this behaviour.\n\nIf all the alternatives of a pattern begin with \\G, the expression is\nanchored to the starting match position, and the \"anchored\" flag is set\nin the compiled regular expression.\n", "subsections": [] }, "CIRCUMFLEX AND DOLLAR": { "content": "The circumflex and dollar metacharacters are zero-width assertions.\nThat is, they test for a particular condition being true without con-\nsuming any characters from the subject string.\n\nOutside a character class, in the default matching mode, the circumflex\ncharacter is an assertion that is true only if the current matching\npoint is at the start of the subject string. If the startoffset argu-\nment of pcreexec() is non-zero, circumflex can never match if the\nPCREMULTILINE option is unset. Inside a character class, circumflex\nhas an entirely different meaning (see below).\n\nCircumflex need not be the first character of the pattern if a number\nof alternatives are involved, but it should be the first thing in each\nalternative in which it appears if the pattern is ever to match that\nbranch. If all possible alternatives start with a circumflex, that is,\nif the pattern is constrained to match only at the start of the sub-\nject, it is said to be an \"anchored\" pattern. (There are also other\nconstructs that can cause a pattern to be anchored.)\n\nThe dollar character is an assertion that is true only if the current\nmatching point is at the end of the subject string, or immediately be-\nfore a newline at the end of the string (by default). Note, however,\nthat it does not actually match the newline. Dollar need not be the\nlast character of the pattern if a number of alternatives are involved,\nbut it should be the last item in any branch in which it appears. Dol-\nlar has no special meaning in a character class.\n\nThe meaning of dollar can be changed so that it matches only at the\nvery end of the string, by setting the PCREDOLLARENDONLY option at\ncompile time. This does not affect the \\Z assertion.\n\nThe meanings of the circumflex and dollar characters are changed if the\nPCREMULTILINE option is set. When this is the case, a circumflex\nmatches immediately after internal newlines as well as at the start of\nthe subject string. It does not match after a newline that ends the\nstring. A dollar matches before any newlines in the string, as well as\nat the very end, when PCREMULTILINE is set. When newline is specified\nas the two-character sequence CRLF, isolated CR and LF characters do\nnot indicate newlines.\n\nFor example, the pattern /^abc$/ matches the subject string \"def\\nabc\"\n(where \\n represents a newline) in multiline mode, but not otherwise.\nConsequently, patterns that are anchored in single line mode because\nall branches start with ^ are not anchored in multiline mode, and a\nmatch for circumflex is possible when the startoffset argument of\npcreexec() is non-zero. The PCREDOLLARENDONLY option is ignored if\nPCREMULTILINE is set.\n\nNote that the sequences \\A, \\Z, and \\z can be used to match the start\nand end of the subject in both modes, and if all branches of a pattern\nstart with \\A it is always anchored, whether or not PCREMULTILINE is\nset.\n\nFULL STOP (PERIOD, DOT) AND \\N\n\nOutside a character class, a dot in the pattern matches any one charac-\nter in the subject string except (by default) a character that signi-\nfies the end of a line.\n\nWhen a line ending is defined as a single character, dot never matches\nthat character; when the two-character sequence CRLF is used, dot does\nnot match CR if it is immediately followed by LF, but otherwise it\nmatches all characters (including isolated CRs and LFs). When any Uni-\ncode line endings are being recognized, dot does not match CR or LF or\nany of the other line ending characters.\n\nThe behaviour of dot with regard to newlines can be changed. If the\nPCREDOTALL option is set, a dot matches any one character, without ex-\nception. If the two-character sequence CRLF is present in the subject\nstring, it takes two dots to match it.\n\nThe handling of dot is entirely independent of the handling of circum-\nflex and dollar, the only relationship being that they both involve\nnewlines. Dot has no special meaning in a character class.\n\nThe escape sequence \\N behaves like a dot, except that it is not af-\nfected by the PCREDOTALL option. In other words, it matches any char-\nacter except one that signifies the end of a line. Perl also uses \\N to\nmatch characters by name; PCRE does not support this.\n", "subsections": [] }, "MATCHING A SINGLE DATA UNIT": { "content": "Outside a character class, the escape sequence \\C matches any one data\nunit, whether or not a UTF mode is set. In the 8-bit library, one data\nunit is one byte; in the 16-bit library it is a 16-bit unit; in the\n32-bit library it is a 32-bit unit. Unlike a dot, \\C always matches\nline-ending characters. The feature is provided in Perl in order to\nmatch individual bytes in UTF-8 mode, but it is unclear how it can use-\nfully be used. Because \\C breaks up characters into individual data\nunits, matching one unit with \\C in a UTF mode means that the rest of\nthe string may start with a malformed UTF character. This has undefined\nresults, because PCRE assumes that it is dealing with valid UTF strings\n(and by default it checks this at the start of processing unless the\nPCRENOUTF8CHECK, PCRENOUTF16CHECK or PCRENOUTF32CHECK option\nis used).\n\nPCRE does not allow \\C to appear in lookbehind assertions (described\nbelow) in a UTF mode, because this would make it impossible to calcu-\nlate the length of the lookbehind.\n\nIn general, the \\C escape sequence is best avoided. However, one way of\nusing it that avoids the problem of malformed UTF characters is to use\na lookahead to check the length of the next character, as in this pat-\ntern, which could be used with a UTF-8 string (ignore white space and\nline breaks):\n\n(?| (?=[\\x00-\\x7f])(\\C) |\n(?=[\\x80-\\x{7ff}])(\\C)(\\C) |\n(?=[\\x{800}-\\x{ffff}])(\\C)(\\C)(\\C) |\n(?=[\\x{10000}-\\x{1fffff}])(\\C)(\\C)(\\C)(\\C))\n\nA group that starts with (?| resets the capturing parentheses numbers\nin each alternative (see \"Duplicate Subpattern Numbers\" below). The as-\nsertions at the start of each branch check the next UTF-8 character for\nvalues whose encoding uses 1, 2, 3, or 4 bytes, respectively. The char-\nacter's individual bytes are then captured by the appropriate number of\ngroups.\n", "subsections": [] }, "SQUARE BRACKETS AND CHARACTER CLASSES": { "content": "An opening square bracket introduces a character class, terminated by a\nclosing square bracket. A closing square bracket on its own is not spe-\ncial by default. However, if the PCREJAVASCRIPTCOMPAT option is set,\na lone closing square bracket causes a compile-time error. If a closing\nsquare bracket is required as a member of the class, it should be the\nfirst data character in the class (after an initial circumflex, if\npresent) or escaped with a backslash.\n\nA character class matches a single character in the subject. In a UTF\nmode, the character may be more than one data unit long. A matched\ncharacter must be in the set of characters defined by the class, unless\nthe first character in the class definition is a circumflex, in which\ncase the subject character must not be in the set defined by the class.\nIf a circumflex is actually required as a member of the class, ensure\nit is not the first character, or escape it with a backslash.\n\nFor example, the character class [aeiou] matches any lower case vowel,\nwhile [^aeiou] matches any character that is not a lower case vowel.\nNote that a circumflex is just a convenient notation for specifying the\ncharacters that are in the class by enumerating those that are not. A\nclass that starts with a circumflex is not an assertion; it still con-\nsumes a character from the subject string, and therefore it fails if\nthe current pointer is at the end of the string.\n\nIn UTF-8 (UTF-16, UTF-32) mode, characters with values greater than 255\n(0xffff) can be included in a class as a literal string of data units,\nor by using the \\x{ escaping mechanism.\n\nWhen caseless matching is set, any letters in a class represent both\ntheir upper case and lower case versions, so for example, a caseless\n[aeiou] matches \"A\" as well as \"a\", and a caseless [^aeiou] does not\nmatch \"A\", whereas a caseful version would. In a UTF mode, PCRE always\nunderstands the concept of case for characters whose values are less\nthan 128, so caseless matching is always possible. For characters with\nhigher values, the concept of case is supported if PCRE is compiled\nwith Unicode property support, but not otherwise. If you want to use\ncaseless matching in a UTF mode for characters 128 and above, you must\nensure that PCRE is compiled with Unicode property support as well as\nwith UTF support.\n\nCharacters that might indicate line breaks are never treated in any\nspecial way when matching character classes, whatever line-ending se-\nquence is in use, and whatever setting of the PCREDOTALL and PCREMUL-\nTILINE options is used. A class such as [^a] always matches one of\nthese characters.\n\nThe minus (hyphen) character can be used to specify a range of charac-\nters in a character class. For example, [d-m] matches any letter be-\ntween d and m, inclusive. If a minus character is required in a class,\nit must be escaped with a backslash or appear in a position where it\ncannot be interpreted as indicating a range, typically as the first or\nlast character in the class, or immediately after a range. For example,\n[b-d-z] matches letters in the range b to d, a hyphen character, or z.\n\nIt is not possible to have the literal character \"]\" as the end charac-\nter of a range. A pattern such as [W-]46] is interpreted as a class of\ntwo characters (\"W\" and \"-\") followed by a literal string \"46]\", so it\nwould match \"W46]\" or \"-46]\". However, if the \"]\" is escaped with a\nbackslash it is interpreted as the end of range, so [W-\\]46] is inter-\npreted as a class containing a range followed by two other characters.\nThe octal or hexadecimal representation of \"]\" can also be used to end\na range.\n\nAn error is generated if a POSIX character class (see below) or an es-\ncape sequence other than one that defines a single character appears at\na point where a range ending character is expected. For example,\n[z-\\xff] is valid, but [A-\\d] and [A-[:digit:]] are not.\n\nRanges operate in the collating sequence of character values. They can\nalso be used for characters specified numerically, for example\n[\\000-\\037]. Ranges can include any characters that are valid for the\ncurrent mode.\n\nIf a range that includes letters is used when caseless matching is set,\nit matches the letters in either case. For example, [W-c] is equivalent\nto [][\\\\^`wxyzabc], matched caselessly, and in a non-UTF mode, if\ncharacter tables for a French locale are in use, [\\xc8-\\xcb] matches\naccented E characters in both cases. In UTF modes, PCRE supports the\nconcept of case for characters with values greater than 128 only when\nit is compiled with Unicode property support.\n\nThe character escape sequences \\d, \\D, \\h, \\H, \\p, \\P, \\s, \\S, \\v, \\V,\n\\w, and \\W may appear in a character class, and add the characters that\nthey match to the class. For example, [\\dABCDEF] matches any hexadeci-\nmal digit. In UTF modes, the PCREUCP option affects the meanings of\n\\d, \\s, \\w and their upper case partners, just as it does when they ap-\npear outside a character class, as described in the section entitled\n\"Generic character types\" above. The escape sequence \\b has a different\nmeaning inside a character class; it matches the backspace character.\nThe sequences \\B, \\N, \\R, and \\X are not special inside a character\nclass. Like any other unrecognized escape sequences, they are treated\nas the literal characters \"B\", \"N\", \"R\", and \"X\" by default, but cause\nan error if the PCREEXTRA option is set.\n\nA circumflex can conveniently be used with the upper case character\ntypes to specify a more restricted set of characters than the matching\nlower case type. For example, the class [^\\W] matches any letter or\ndigit, but not underscore, whereas [\\w] includes underscore. A positive\ncharacter class should be read as \"something OR something OR ...\" and a\nnegative class as \"NOT something AND NOT something AND NOT ...\".\n\nThe only metacharacters that are recognized in character classes are\nbackslash, hyphen (only where it can be interpreted as specifying a\nrange), circumflex (only at the start), opening square bracket (only\nwhen it can be interpreted as introducing a POSIX class name, or for a\nspecial compatibility feature - see the next two sections), and the\nterminating closing square bracket. However, escaping other non-al-\nphanumeric characters does no harm.\n", "subsections": [] }, "POSIX CHARACTER CLASSES": { "content": "Perl supports the POSIX notation for character classes. This uses names\nenclosed by [: and :] within the enclosing square brackets. PCRE also\nsupports this notation. For example,\n\n[01[:alpha:]%]\n\nmatches \"0\", \"1\", any alphabetic character, or \"%\". The supported class\nnames are:\n\nalnum letters and digits\nalpha letters\nascii character codes 0 - 127\nblank space or tab only\ncntrl control characters\ndigit decimal digits (same as \\d)\ngraph printing characters, excluding space\nlower lower case letters\nprint printing characters, including space\npunct printing characters, excluding letters and digits and space\nspace white space (the same as \\s from PCRE 8.34)\nupper upper case letters\nword \"word\" characters (same as \\w)\nxdigit hexadecimal digits\n\nThe default \"space\" characters are HT (9), LF (10), VT (11), FF (12),\nCR (13), and space (32). If locale-specific matching is taking place,\nthe list of space characters may be different; there may be fewer or\nmore of them. \"Space\" used to be different to \\s, which did not include\nVT, for Perl compatibility. However, Perl changed at release 5.18, and\nPCRE followed at release 8.34. \"Space\" and \\s now match the same set\nof characters.\n\nThe name \"word\" is a Perl extension, and \"blank\" is a GNU extension\nfrom Perl 5.8. Another Perl extension is negation, which is indicated\nby a ^ character after the colon. For example,\n\n[12[:^digit:]]\n\nmatches \"1\", \"2\", or any non-digit. PCRE (and Perl) also recognize the\nPOSIX syntax [.ch.] and [=ch=] where \"ch\" is a \"collating element\", but\nthese are not supported, and an error is given if they are encountered.\n\nBy default, characters with values greater than 128 do not match any of\nthe POSIX character classes. However, if the PCREUCP option is passed\nto pcrecompile(), some of the classes are changed so that Unicode\ncharacter properties are used. This is achieved by replacing certain\nPOSIX classes by other sequences, as follows:\n\n[:alnum:] becomes \\p{Xan}\n[:alpha:] becomes \\p{L}\n[:blank:] becomes \\h\n[:digit:] becomes \\p{Nd}\n[:lower:] becomes \\p{Ll}\n[:space:] becomes \\p{Xps}\n[:upper:] becomes \\p{Lu}\n[:word:] becomes \\p{Xwd}\n\nNegated versions, such as [:^alpha:] use \\P instead of \\p. Three other\nPOSIX classes are handled specially in UCP mode:\n\n[:graph:] This matches characters that have glyphs that mark the page\nwhen printed. In Unicode property terms, it matches all char-\nacters with the L, M, N, P, S, or Cf properties, except for:\n\nU+061C Arabic Letter Mark\nU+180E Mongolian Vowel Separator\nU+2066 - U+2069 Various \"isolate\"s\n\n[:print:] This matches the same characters as [:graph:] plus space\ncharacters that are not controls, that is, characters with\nthe Zs property.\n\n[:punct:] This matches all characters that have the Unicode P (punctua-\ntion) property, plus those characters whose code points are\nless than 128 that have the S (Symbol) property.\n\nThe other POSIX classes are unchanged, and match only characters with\ncode points less than 128.\n", "subsections": [] }, "COMPATIBILITY FEATURE FOR WORD BOUNDARIES": { "content": "In the POSIX.2 compliant library that was included in 4.4BSD Unix, the\nugly syntax [[:<:]] and [[:>:]] is used for matching \"start of word\"\nand \"end of word\". PCRE treats these items as follows:\n\n[[:<:]] is converted to \\b(?=\\w)\n[[:>:]] is converted to \\b(?<=\\w)\n\nOnly these exact character sequences are recognized. A sequence such as\n[a[:<:]b] provokes error for an unrecognized POSIX class name. This\nsupport is not compatible with Perl. It is provided to help migrations\nfrom other environments, and is best not used in any new patterns. Note\nthat \\b matches at the start and the end of a word (see \"Simple asser-\ntions\" above), and in a Perl-style pattern the preceding or following\ncharacter normally shows which is wanted, without the need for the as-\nsertions that are used above in order to give exactly the POSIX behav-\niour.\n", "subsections": [] }, "VERTICAL BAR": { "content": "Vertical bar characters are used to separate alternative patterns. For\nexample, the pattern\n\ngilbert|sullivan\n\nmatches either \"gilbert\" or \"sullivan\". Any number of alternatives may\nappear, and an empty alternative is permitted (matching the empty\nstring). The matching process tries each alternative in turn, from left\nto right, and the first one that succeeds is used. If the alternatives\nare within a subpattern (defined below), \"succeeds\" means matching the\nrest of the main pattern as well as the alternative in the subpattern.\n", "subsections": [] }, "INTERNAL OPTION SETTING": { "content": "The settings of the PCRECASELESS, PCREMULTILINE, PCREDOTALL, and\nPCREEXTENDED options (which are Perl-compatible) can be changed from\nwithin the pattern by a sequence of Perl option letters enclosed be-\ntween \"(?\" and \")\". The option letters are\n\ni for PCRECASELESS\nm for PCREMULTILINE\ns for PCREDOTALL\nx for PCREEXTENDED\n\nFor example, (?im) sets caseless, multiline matching. It is also possi-\nble to unset these options by preceding the letter with a hyphen, and a\ncombined setting and unsetting such as (?im-sx), which sets PCRECASE-\nLESS and PCREMULTILINE while unsetting PCREDOTALL and PCREEXTENDED,\nis also permitted. If a letter appears both before and after the hy-\nphen, the option is unset.\n\nThe PCRE-specific options PCREDUPNAMES, PCREUNGREEDY, and PCREEXTRA\ncan be changed in the same way as the Perl-compatible options by using\nthe characters J, U and X respectively.\n\nWhen one of these option changes occurs at top level (that is, not in-\nside subpattern parentheses), the change applies to the remainder of\nthe pattern that follows. If the change is placed right at the start of\na pattern, PCRE extracts it into the global options (and it will there-\nfore show up in data extracted by the pcrefullinfo() function).\n\nAn option change within a subpattern (see below for a description of\nsubpatterns) affects only that part of the subpattern that follows it,\nso\n\n(a(?i)b)c\n\nmatches abc and aBc and no other strings (assuming PCRECASELESS is not\nused). By this means, options can be made to have different settings\nin different parts of the pattern. Any changes made in one alternative\ndo carry on into subsequent branches within the same subpattern. For\nexample,\n\n(a(?i)b|c)\n\nmatches \"ab\", \"aB\", \"c\", and \"C\", even though when matching \"C\" the\nfirst branch is abandoned before the option setting. This is because\nthe effects of option settings happen at compile time. There would be\nsome very weird behaviour otherwise.\n\nNote: There are other PCRE-specific options that can be set by the ap-\nplication when the compiling or matching functions are called. In some\ncases the pattern can contain special leading sequences such as (*CRLF)\nto override what the application has set or what has been defaulted.\nDetails are given in the section entitled \"Newline sequences\" above.\nThere are also the (*UTF8), (*UTF16),(*UTF32), and (*UCP) leading se-\nquences that can be used to set UTF and Unicode property modes; they\nare equivalent to setting the PCREUTF8, PCREUTF16, PCREUTF32 and the\nPCREUCP options, respectively. The (*UTF) sequence is a generic ver-\nsion that can be used with any of the libraries. However, the applica-\ntion can set the PCRENEVERUTF option, which locks out the use of the\n(*UTF) sequences.\n", "subsections": [] }, "SUBPATTERNS": { "content": "Subpatterns are delimited by parentheses (round brackets), which can be\nnested. Turning part of a pattern into a subpattern does two things:\n\n1. It localizes a set of alternatives. For example, the pattern\n\ncat(aract|erpillar|)\n\nmatches \"cataract\", \"caterpillar\", or \"cat\". Without the parentheses,\nit would match \"cataract\", \"erpillar\" or an empty string.\n\n2. It sets up the subpattern as a capturing subpattern. This means\nthat, when the whole pattern matches, that portion of the subject\nstring that matched the subpattern is passed back to the caller via the\novector argument of the matching function. (This applies only to the\ntraditional matching functions; the DFA matching functions do not sup-\nport capturing.)\n\nOpening parentheses are counted from left to right (starting from 1) to\nobtain numbers for the capturing subpatterns. For example, if the\nstring \"the red king\" is matched against the pattern\n\nthe ((red|white) (king|queen))\n\nthe captured substrings are \"red king\", \"red\", and \"king\", and are num-\nbered 1, 2, and 3, respectively.\n\nThe fact that plain parentheses fulfil two functions is not always\nhelpful. There are often times when a grouping subpattern is required\nwithout a capturing requirement. If an opening parenthesis is followed\nby a question mark and a colon, the subpattern does not do any captur-\ning, and is not counted when computing the number of any subsequent\ncapturing subpatterns. For example, if the string \"the white queen\" is\nmatched against the pattern\n\nthe ((?:red|white) (king|queen))\n\nthe captured substrings are \"white queen\" and \"queen\", and are numbered\n1 and 2. The maximum number of capturing subpatterns is 65535.\n\nAs a convenient shorthand, if any option settings are required at the\nstart of a non-capturing subpattern, the option letters may appear be-\ntween the \"?\" and the \":\". Thus the two patterns\n\n(?i:saturday|sunday)\n(?:(?i)saturday|sunday)\n\nmatch exactly the same set of strings. Because alternative branches are\ntried from left to right, and options are not reset until the end of\nthe subpattern is reached, an option setting in one branch does affect\nsubsequent branches, so the above patterns match \"SUNDAY\" as well as\n\"Saturday\".\n", "subsections": [] }, "DUPLICATE SUBPATTERN NUMBERS": { "content": "Perl 5.10 introduced a feature whereby each alternative in a subpattern\nuses the same numbers for its capturing parentheses. Such a subpattern\nstarts with (?| and is itself a non-capturing subpattern. For example,\nconsider this pattern:\n\n(?|(Sat)ur|(Sun))day\n\nBecause the two alternatives are inside a (?| group, both sets of cap-\nturing parentheses are numbered one. Thus, when the pattern matches,\nyou can look at captured substring number one, whichever alternative\nmatched. This construct is useful when you want to capture part, but\nnot all, of one of a number of alternatives. Inside a (?| group, paren-\ntheses are numbered as usual, but the number is reset at the start of\neach branch. The numbers of any capturing parentheses that follow the\nsubpattern start after the highest number used in any branch. The fol-\nlowing example is taken from the Perl documentation. The numbers under-\nneath show in which buffer the captured content will be stored.\n\n# before ---------------branch-reset----------- after\n/ ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x\n# 1 2 2 3 2 3 4\n\nA back reference to a numbered subpattern uses the most recent value\nthat is set for that number by any subpattern. The following pattern\nmatches \"abcabc\" or \"defdef\":\n\n/(?|(abc)|(def))\\1/\n\nIn contrast, a subroutine call to a numbered subpattern always refers\nto the first one in the pattern with the given number. The following\npattern matches \"abcabc\" or \"defabc\":\n\n/(?|(abc)|(def))(?1)/\n\nIf a condition test for a subpattern's having matched refers to a non-\nunique number, the test is true if any of the subpatterns of that num-\nber have matched.\n\nAn alternative approach to using this \"branch reset\" feature is to use\nduplicate named subpatterns, as described in the next section.\n", "subsections": [] }, "NAMED SUBPATTERNS": { "content": "Identifying capturing parentheses by number is simple, but it can be\nvery hard to keep track of the numbers in complicated regular expres-\nsions. Furthermore, if an expression is modified, the numbers may\nchange. To help with this difficulty, PCRE supports the naming of sub-\npatterns. This feature was not added to Perl until release 5.10. Python\nhad the feature earlier, and PCRE introduced it at release 4.0, using\nthe Python syntax. PCRE now supports both the Perl and the Python syn-\ntax. Perl allows identically numbered subpatterns to have different\nnames, but PCRE does not.\n\nIn PCRE, a subpattern can be named in one of three ways: (?...)\nor (?'name'...) as in Perl, or (?P...) as in Python. References\nto capturing parentheses from other parts of the pattern, such as back\nreferences, recursion, and conditions, can be made by name as well as\nby number.\n\nNames consist of up to 32 alphanumeric characters and underscores, but\nmust start with a non-digit. Named capturing parentheses are still al-\nlocated numbers as well as names, exactly as if the names were not\npresent. The PCRE API provides function calls for extracting the name-\nto-number translation table from a compiled pattern. There is also a\nconvenience function for extracting a captured substring by name.\n\nBy default, a name must be unique within a pattern, but it is possible\nto relax this constraint by setting the PCREDUPNAMES option at compile\ntime. (Duplicate names are also always permitted for subpatterns with\nthe same number, set up as described in the previous section.) Dupli-\ncate names can be useful for patterns where only one instance of the\nnamed parentheses can match. Suppose you want to match the name of a\nweekday, either as a 3-letter abbreviation or as the full name, and in\nboth cases you want to extract the abbreviation. This pattern (ignoring\nthe line breaks) does the job:\n\n(?Mon|Fri|Sun)(?:day)?|\n(?Tue)(?:sday)?|\n(?Wed)(?:nesday)?|\n(?Thu)(?:rsday)?|\n(?Sat)(?:urday)?\n\nThere are five capturing substrings, but only one is ever set after a\nmatch. (An alternative way of solving this problem is to use a \"branch\nreset\" subpattern, as described in the previous section.)\n\nThe convenience function for extracting the data by name returns the\nsubstring for the first (and in this example, the only) subpattern of\nthat name that matched. This saves searching to find which numbered\nsubpattern it was.\n\nIf you make a back reference to a non-unique named subpattern from\nelsewhere in the pattern, the subpatterns to which the name refers are\nchecked in the order in which they appear in the overall pattern. The\nfirst one that is set is used for the reference. For example, this pat-\ntern matches both \"foofoo\" and \"barbar\" but not \"foobar\" or \"barfoo\":\n\n(?:(?foo)|(?bar))\\k\n\nIf you make a subroutine call to a non-unique named subpattern, the one\nthat corresponds to the first occurrence of the name is used. In the\nabsence of duplicate numbers (see the previous section) this is the one\nwith the lowest number.\n\nIf you use a named reference in a condition test (see the section about\nconditions below), either to check whether a subpattern has matched, or\nto check for recursion, all subpatterns with the same name are tested.\nIf the condition is true for any one of them, the overall condition is\ntrue. This is the same behaviour as testing by number. For further de-\ntails of the interfaces for handling named subpatterns, see the pcreapi\ndocumentation.\n\nWarning: You cannot use different names to distinguish between two sub-\npatterns with the same number because PCRE uses only the numbers when\nmatching. For this reason, an error is given at compile time if differ-\nent names are given to subpatterns with the same number. However, you\ncan always give the same name to subpatterns with the same number, even\nwhen PCREDUPNAMES is not set.\n", "subsections": [] }, "REPETITION": { "content": "Repetition is specified by quantifiers, which can follow any of the\nfollowing items:\n\na literal data character\nthe dot metacharacter\nthe \\C escape sequence\nthe \\X escape sequence\nthe \\R escape sequence\nan escape such as \\d or \\pL that matches a single character\na character class\na back reference (see next section)\na parenthesized subpattern (including assertions)\na subroutine call to a subpattern (recursive or otherwise)\n\nThe general repetition quantifier specifies a minimum and maximum num-\nber of permitted matches, by giving the two numbers in curly brackets\n(braces), separated by a comma. The numbers must be less than 65536,\nand the first must be less than or equal to the second. For example:\n\nz{2,4}\n\nmatches \"zz\", \"zzz\", or \"zzzz\". A closing brace on its own is not a\nspecial character. If the second number is omitted, but the comma is\npresent, there is no upper limit; if the second number and the comma\nare both omitted, the quantifier specifies an exact number of required\nmatches. Thus\n\n[aeiou]{3,}\n\nmatches at least 3 successive vowels, but may match many more, while\n\n\\d{8}\n\nmatches exactly 8 digits. An opening curly bracket that appears in a\nposition where a quantifier is not allowed, or one that does not match\nthe syntax of a quantifier, is taken as a literal character. For exam-\nple, {,6} is not a quantifier, but a literal string of four characters.\n\nIn UTF modes, quantifiers apply to characters rather than to individual\ndata units. Thus, for example, \\x{100}{2} matches two characters, each\nof which is represented by a two-byte sequence in a UTF-8 string. Simi-\nlarly, \\X{3} matches three Unicode extended grapheme clusters, each of\nwhich may be several data units long (and they may be of different\nlengths).\n\nThe quantifier {0} is permitted, causing the expression to behave as if\nthe previous item and the quantifier were not present. This may be use-\nful for subpatterns that are referenced as subroutines from elsewhere\nin the pattern (but see also the section entitled \"Defining subpatterns\nfor use by reference only\" below). Items other than subpatterns that\nhave a {0} quantifier are omitted from the compiled pattern.\n\nFor convenience, the three most common quantifiers have single-charac-\nter abbreviations:\n\n* is equivalent to {0,}\n+ is equivalent to {1,}\n? is equivalent to {0,1}\n\nIt is possible to construct infinite loops by following a subpattern\nthat can match no characters with a quantifier that has no upper limit,\nfor example:\n\n(a?)*\n\nEarlier versions of Perl and PCRE used to give an error at compile time\nfor such patterns. However, because there are cases where this can be\nuseful, such patterns are now accepted, but if any repetition of the\nsubpattern does in fact match no characters, the loop is forcibly bro-\nken.\n\nBy default, the quantifiers are \"greedy\", that is, they match as much\nas possible (up to the maximum number of permitted times), without\ncausing the rest of the pattern to fail. The classic example of where\nthis gives problems is in trying to match comments in C programs. These\nappear between /* and */ and within the comment, individual * and /\ncharacters may appear. An attempt to match C comments by applying the\npattern\n\n/\\*.*\\*/\n\nto the string\n\n/* first comment */ not comment /* second comment */\n\nfails, because it matches the entire string owing to the greediness of\nthe .* item.\n\nHowever, if a quantifier is followed by a question mark, it ceases to\nbe greedy, and instead matches the minimum number of times possible, so\nthe pattern\n\n/\\*.*?\\*/\n\ndoes the right thing with the C comments. The meaning of the various\nquantifiers is not otherwise changed, just the preferred number of\nmatches. Do not confuse this use of question mark with its use as a\nquantifier in its own right. Because it has two uses, it can sometimes\nappear doubled, as in\n\n\\d??\\d\n\nwhich matches one digit by preference, but can match two if that is the\nonly way the rest of the pattern matches.\n\nIf the PCREUNGREEDY option is set (an option that is not available in\nPerl), the quantifiers are not greedy by default, but individual ones\ncan be made greedy by following them with a question mark. In other\nwords, it inverts the default behaviour.\n\nWhen a parenthesized subpattern is quantified with a minimum repeat\ncount that is greater than 1 or with a limited maximum, more memory is\nrequired for the compiled pattern, in proportion to the size of the\nminimum or maximum.\n\nIf a pattern starts with .* or .{0,} and the PCREDOTALL option (equiv-\nalent to Perl's /s) is set, thus allowing the dot to match newlines,\nthe pattern is implicitly anchored, because whatever follows will be\ntried against every character position in the subject string, so there\nis no point in retrying the overall match at any position after the\nfirst. PCRE normally treats such a pattern as though it were preceded\nby \\A.\n\nIn cases where it is known that the subject string contains no new-\nlines, it is worth setting PCREDOTALL in order to obtain this opti-\nmization, or alternatively using ^ to indicate anchoring explicitly.\n\nHowever, there are some cases where the optimization cannot be used.\nWhen .* is inside capturing parentheses that are the subject of a back\nreference elsewhere in the pattern, a match at the start may fail where\na later one succeeds. Consider, for example:\n\n(.*)abc\\1\n\nIf the subject is \"xyz123abc123\" the match point is the fourth charac-\nter. For this reason, such a pattern is not implicitly anchored.\n\nAnother case where implicit anchoring is not applied is when the lead-\ning .* is inside an atomic group. Once again, a match at the start may\nfail where a later one succeeds. Consider this pattern:\n\n(?>.*?a)b\n\nIt matches \"ab\" in the subject \"aab\". The use of the backtracking con-\ntrol verbs (*PRUNE) and (*SKIP) also disable this optimization.\n\nWhen a capturing subpattern is repeated, the value captured is the sub-\nstring that matched the final iteration. For example, after\n\n(tweedle[dume]{3}\\s*)+\n\nhas matched \"tweedledum tweedledee\" the value of the captured substring\nis \"tweedledee\". However, if there are nested capturing subpatterns,\nthe corresponding captured values may have been set in previous itera-\ntions. For example, after\n\n/(a|(b))+/\n\nmatches \"aba\" the value of the second captured substring is \"b\".\n", "subsections": [] }, "ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS": { "content": "With both maximizing (\"greedy\") and minimizing (\"ungreedy\" or \"lazy\")\nrepetition, failure of what follows normally causes the repeated item\nto be re-evaluated to see if a different number of repeats allows the\nrest of the pattern to match. Sometimes it is useful to prevent this,\neither to change the nature of the match, or to cause it fail earlier\nthan it otherwise might, when the author of the pattern knows there is\nno point in carrying on.\n\nConsider, for example, the pattern \\d+foo when applied to the subject\nline\n\n123456bar\n\nAfter matching all 6 digits and then failing to match \"foo\", the normal\naction of the matcher is to try again with only 5 digits matching the\n\\d+ item, and then with 4, and so on, before ultimately failing.\n\"Atomic grouping\" (a term taken from Jeffrey Friedl's book) provides\nthe means for specifying that once a subpattern has matched, it is not\nto be re-evaluated in this way.\n\nIf we use atomic grouping for the previous example, the matcher gives\nup immediately on failing to match \"foo\" the first time. The notation\nis a kind of special parenthesis, starting with (?> as in this example:\n\n(?>\\d+)foo\n\nThis kind of parenthesis \"locks up\" the part of the pattern it con-\ntains once it has matched, and a failure further into the pattern is\nprevented from backtracking into it. Backtracking past it to previous\nitems, however, works as normal.\n\nAn alternative description is that a subpattern of this type matches\nthe string of characters that an identical standalone pattern would\nmatch, if anchored at the current point in the subject string.\n\nAtomic grouping subpatterns are not capturing subpatterns. Simple cases\nsuch as the above example can be thought of as a maximizing repeat that\nmust swallow everything it can. So, while both \\d+ and \\d+? are pre-\npared to adjust the number of digits they match in order to make the\nrest of the pattern match, (?>\\d+) can only match an entire sequence of\ndigits.\n\nAtomic groups in general can of course contain arbitrarily complicated\nsubpatterns, and can be nested. However, when the subpattern for an\natomic group is just a single repeated item, as in the example above, a\nsimpler notation, called a \"possessive quantifier\" can be used. This\nconsists of an additional + character following a quantifier. Using\nthis notation, the previous example can be rewritten as\n\n\\d++foo\n\nNote that a possessive quantifier can be used with an entire group, for\nexample:\n\n(abc|xyz){2,3}+\n\nPossessive quantifiers are always greedy; the setting of the PCREUN-\nGREEDY option is ignored. They are a convenient notation for the sim-\npler forms of atomic group. However, there is no difference in the\nmeaning of a possessive quantifier and the equivalent atomic group,\nthough there may be a performance difference; possessive quantifiers\nshould be slightly faster.\n\nThe possessive quantifier syntax is an extension to the Perl 5.8 syn-\ntax. Jeffrey Friedl originated the idea (and the name) in the first\nedition of his book. Mike McCloskey liked it, so implemented it when he\nbuilt Sun's Java package, and PCRE copied it from there. It ultimately\nfound its way into Perl at release 5.10.\n\nPCRE has an optimization that automatically \"possessifies\" certain sim-\nple pattern constructs. For example, the sequence A+B is treated as\nA++B because there is no point in backtracking into a sequence of A's\nwhen B must follow.\n\nWhen a pattern contains an unlimited repeat inside a subpattern that\ncan itself be repeated an unlimited number of times, the use of an\natomic group is the only way to avoid some failing matches taking a\nvery long time indeed. The pattern\n\n(\\D+|<\\d+>)*[!?]\n\nmatches an unlimited number of substrings that either consist of non-\ndigits, or digits enclosed in <>, followed by either ! or ?. When it\nmatches, it runs quickly. However, if it is applied to\n\naaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n\nit takes a long time before reporting failure. This is because the\nstring can be divided between the internal \\D+ repeat and the external\n* repeat in a large number of ways, and all have to be tried. (The ex-\nample uses [!?] rather than a single character at the end, because both\nPCRE and Perl have an optimization that allows for fast failure when a\nsingle character is used. They remember the last single character that\nis required for a match, and fail early if it is not present in the\nstring.) If the pattern is changed so that it uses an atomic group,\nlike this:\n\n((?>\\D+)|<\\d+>)*[!?]\n\nsequences of non-digits cannot be broken, and failure happens quickly.\n", "subsections": [] }, "BACK REFERENCES": { "content": "Outside a character class, a backslash followed by a digit greater than\n0 (and possibly further digits) is a back reference to a capturing sub-\npattern earlier (that is, to its left) in the pattern, provided there\nhave been that many previous capturing left parentheses.\n\nHowever, if the decimal number following the backslash is less than 10,\nit is always taken as a back reference, and causes an error only if\nthere are not that many capturing left parentheses in the entire pat-\ntern. In other words, the parentheses that are referenced need not be\nto the left of the reference for numbers less than 10. A \"forward back\nreference\" of this type can make sense when a repetition is involved\nand the subpattern to the right has participated in an earlier itera-\ntion.\n\nIt is not possible to have a numerical \"forward back reference\" to a\nsubpattern whose number is 10 or more using this syntax because a se-\nquence such as \\50 is interpreted as a character defined in octal. See\nthe subsection entitled \"Non-printing characters\" above for further de-\ntails of the handling of digits following a backslash. There is no such\nproblem when named parentheses are used. A back reference to any sub-\npattern is possible using named parentheses (see below).\n\nAnother way of avoiding the ambiguity inherent in the use of digits\nfollowing a backslash is to use the \\g escape sequence. This escape\nmust be followed by an unsigned number or a negative number, optionally\nenclosed in braces. These examples are all identical:\n\n(ring), \\1\n(ring), \\g1\n(ring), \\g{1}\n\nAn unsigned number specifies an absolute reference without the ambigu-\nity that is present in the older syntax. It is also useful when literal\ndigits follow the reference. A negative number is a relative reference.\nConsider this example:\n\n(abc(def)ghi)\\g{-1}\n\nThe sequence \\g{-1} is a reference to the most recently started captur-\ning subpattern before \\g, that is, is it equivalent to \\2 in this exam-\nple. Similarly, \\g{-2} would be equivalent to \\1. The use of relative\nreferences can be helpful in long patterns, and also in patterns that\nare created by joining together fragments that contain references\nwithin themselves.\n\nA back reference matches whatever actually matched the capturing sub-\npattern in the current subject string, rather than anything matching\nthe subpattern itself (see \"Subpatterns as subroutines\" below for a way\nof doing that). So the pattern\n\n(sens|respons)e and \\1ibility\n\nmatches \"sense and sensibility\" and \"response and responsibility\", but\nnot \"sense and responsibility\". If caseful matching is in force at the\ntime of the back reference, the case of letters is relevant. For exam-\nple,\n\n((?i)rah)\\s+\\1\n\nmatches \"rah rah\" and \"RAH RAH\", but not \"RAH rah\", even though the\noriginal capturing subpattern is matched caselessly.\n\nThere are several different ways of writing back references to named\nsubpatterns. The .NET syntax \\k{name} and the Perl syntax \\k or\n\\k'name' are supported, as is the Python syntax (?P=name). Perl 5.10's\nunified back reference syntax, in which \\g can be used for both numeric\nand named references, is also supported. We could rewrite the above ex-\nample in any of the following ways:\n\n(?(?i)rah)\\s+\\k\n(?'p1'(?i)rah)\\s+\\k{p1}\n(?P(?i)rah)\\s+(?P=p1)\n(?(?i)rah)\\s+\\g{p1}\n\nA subpattern that is referenced by name may appear in the pattern be-\nfore or after the reference.\n\nThere may be more than one back reference to the same subpattern. If a\nsubpattern has not actually been used in a particular match, any back\nreferences to it always fail by default. For example, the pattern\n\n(a|(bc))\\2\n\nalways fails if it starts to match \"a\" rather than \"bc\". However, if\nthe PCREJAVASCRIPTCOMPAT option is set at compile time, a back refer-\nence to an unset value matches an empty string.\n\nBecause there may be many capturing parentheses in a pattern, all dig-\nits following a backslash are taken as part of a potential back refer-\nence number. If the pattern continues with a digit character, some de-\nlimiter must be used to terminate the back reference. If the PCREEX-\nTENDED option is set, this can be white space. Otherwise, the \\g{ syn-\ntax or an empty comment (see \"Comments\" below) can be used.\n\nRecursive back references\n\nA back reference that occurs inside the parentheses to which it refers\nfails when the subpattern is first used, so, for example, (a\\1) never\nmatches. However, such references can be useful inside repeated sub-\npatterns. For example, the pattern\n\n(a|b\\1)+\n\nmatches any number of \"a\"s and also \"aba\", \"ababbaa\" etc. At each iter-\nation of the subpattern, the back reference matches the character\nstring corresponding to the previous iteration. In order for this to\nwork, the pattern must be such that the first iteration does not need\nto match the back reference. This can be done using alternation, as in\nthe example above, or by a quantifier with a minimum of zero.\n\nBack references of this type cause the group that they reference to be\ntreated as an atomic group. Once the whole group has been matched, a\nsubsequent matching failure cannot cause backtracking into the middle\nof the group.\n", "subsections": [] }, "ASSERTIONS": { "content": "An assertion is a test on the characters following or preceding the\ncurrent matching point that does not actually consume any characters.\nThe simple assertions coded as \\b, \\B, \\A, \\G, \\Z, \\z, ^ and $ are de-\nscribed above.\n\nMore complicated assertions are coded as subpatterns. There are two\nkinds: those that look ahead of the current position in the subject\nstring, and those that look behind it. An assertion subpattern is\nmatched in the normal way, except that it does not cause the current\nmatching position to be changed.\n\nAssertion subpatterns are not capturing subpatterns. If such an asser-\ntion contains capturing subpatterns within it, these are counted for\nthe purposes of numbering the capturing subpatterns in the whole pat-\ntern. However, substring capturing is carried out only for positive as-\nsertions. (Perl sometimes, but not always, does do capturing in nega-\ntive assertions.)\n\nFor compatibility with Perl, assertion subpatterns may be repeated;\nthough it makes no sense to assert the same thing several times, the\nside effect of capturing parentheses may occasionally be useful. In\npractice, there only three cases:\n\n(1) If the quantifier is {0}, the assertion is never obeyed during\nmatching. However, it may contain internal capturing parenthesized\ngroups that are called from elsewhere via the subroutine mechanism.\n\n(2) If quantifier is {0,n} where n is greater than zero, it is treated\nas if it were {0,1}. At run time, the rest of the pattern match is\ntried with and without the assertion, the order depending on the greed-\niness of the quantifier.\n\n(3) If the minimum repetition is greater than zero, the quantifier is\nignored. The assertion is obeyed just once when encountered during\nmatching.\n\nLookahead assertions\n\nLookahead assertions start with (?= for positive assertions and (?! for\nnegative assertions. For example,\n\n\\w+(?=;)\n\nmatches a word followed by a semicolon, but does not include the semi-\ncolon in the match, and\n\nfoo(?!bar)\n\nmatches any occurrence of \"foo\" that is not followed by \"bar\". Note\nthat the apparently similar pattern\n\n(?!foo)bar\n\ndoes not find an occurrence of \"bar\" that is preceded by something\nother than \"foo\"; it finds any occurrence of \"bar\" whatsoever, because\nthe assertion (?!foo) is always true when the next three characters are\n\"bar\". A lookbehind assertion is needed to achieve the other effect.\n\nIf you want to force a matching failure at some point in a pattern, the\nmost convenient way to do it is with (?!) because an empty string al-\nways matches, so an assertion that requires there not to be an empty\nstring must always fail. The backtracking control verb (*FAIL) or (*F)\nis a synonym for (?!).\n\nLookbehind assertions\n\nLookbehind assertions start with (?<= for positive assertions and (?)...) or (?('name')...) to test for a\nused subpattern by name. For compatibility with earlier versions of\nPCRE, which had this facility before Perl, the syntax (?(name)...) is\nalso recognized.\n\nRewriting the above example to use a named subpattern gives this:\n\n(? \\( )? [^()]+ (?() \\) )\n\nIf the name used in a condition of this kind is a duplicate, the test\nis applied to all subpatterns of the same name, and is true if any one\nof them has matched.\n\nChecking for pattern recursion\n\nIf the condition is the string (R), and there is no subpattern with the\nname R, the condition is true if a recursive call to the whole pattern\nor any subpattern has been made. If digits or a name preceded by amper-\nsand follow the letter R, for example:\n\n(?(R3)...) or (?(R&name)...)\n\nthe condition is true if the most recent recursion is into a subpattern\nwhose number or name is given. This condition does not check the entire\nrecursion stack. If the name used in a condition of this kind is a du-\nplicate, the test is applied to all subpatterns of the same name, and\nis true if any one of them is the most recent recursion.\n\nAt \"top level\", all these recursion test conditions are false. The\nsyntax for recursive patterns is described below.\n\nDefining subpatterns for use by reference only\n\nIf the condition is the string (DEFINE), and there is no subpattern\nwith the name DEFINE, the condition is always false. In this case,\nthere may be only one alternative in the subpattern. It is always\nskipped if control reaches this point in the pattern; the idea of DE-\nFINE is that it can be used to define subroutines that can be refer-\nenced from elsewhere. (The use of subroutines is described below.) For\nexample, a pattern to match an IPv4 address such as \"192.168.23.245\"\ncould be written like this (ignore white space and line breaks):\n\n(?(DEFINE) (? 2[0-4]\\d | 25[0-5] | 1\\d\\d | [1-9]?\\d) )\n\\b (?&byte) (\\.(?&byte)){3} \\b\n\nThe first part of the pattern is a DEFINE group inside which a another\ngroup named \"byte\" is defined. This matches an individual component of\nan IPv4 address (a number less than 256). When matching takes place,\nthis part of the pattern is skipped because DEFINE acts like a false\ncondition. The rest of the pattern uses references to the named group\nto match the four dot-separated components of an IPv4 address, insist-\ning on a word boundary at each end.\n\nAssertion conditions\n\nIf the condition is not in any of the above formats, it must be an as-\nsertion. This may be a positive or negative lookahead or lookbehind\nassertion. Consider this pattern, again containing non-significant\nwhite space, and with the two alternatives on the second line:\n\n(?(?=[^a-z]*[a-z])\n\\d{2}-[a-z]{3}-\\d{2} | \\d{2}-\\d{2}-\\d{2} )\n\nThe condition is a positive lookahead assertion that matches an op-\ntional sequence of non-letters followed by a letter. In other words, it\ntests for the presence of at least one letter in the subject. If a let-\nter is found, the subject is matched against the first alternative;\notherwise it is matched against the second. This pattern matches\nstrings in one of the two forms dd-aaa-dd or dd-dd-dd, where aaa are\nletters and dd are digits.\n", "subsections": [] }, "COMMENTS": { "content": "There are two ways of including comments in patterns that are processed\nby PCRE. In both cases, the start of the comment must not be in a char-\nacter class, nor in the middle of any other sequence of related charac-\nters such as (?: or a subpattern name or number. The characters that\nmake up a comment play no part in the pattern matching.\n\nThe sequence (?# marks the start of a comment that continues up to the\nnext closing parenthesis. Nested parentheses are not permitted. If the\nPCREEXTENDED option is set, an unescaped # character also introduces a\ncomment, which in this case continues to immediately after the next\nnewline character or character sequence in the pattern. Which charac-\nters are interpreted as newlines is controlled by the options passed to\na compiling function or by a special sequence at the start of the pat-\ntern, as described in the section entitled \"Newline conventions\" above.\nNote that the end of this type of comment is a literal newline sequence\nin the pattern; escape sequences that happen to represent a newline do\nnot count. For example, consider this pattern when PCREEXTENDED is\nset, and the default newline convention is in force:\n\nabc #comment \\n still comment\n\nOn encountering the # character, pcrecompile() skips along, looking\nfor a newline in the pattern. The sequence \\n is still literal at this\nstage, so it does not terminate the comment. Only an actual character\nwith the code value 0x0a (the default newline) does so.\n", "subsections": [] }, "RECURSIVE PATTERNS": { "content": "Consider the problem of matching a string in parentheses, allowing for\nunlimited nested parentheses. Without the use of recursion, the best\nthat can be done is to use a pattern that matches up to some fixed\ndepth of nesting. It is not possible to handle an arbitrary nesting\ndepth.\n\nFor some time, Perl has provided a facility that allows regular expres-\nsions to recurse (amongst other things). It does this by interpolating\nPerl code in the expression at run time, and the code can refer to the\nexpression itself. A Perl pattern using code interpolation to solve the\nparentheses problem can be created like this:\n\n$re = qr{\\( (?: (?>[^()]+) | (?p{$re}) )* \\)}x;\n\nThe (?p{...}) item interpolates Perl code at run time, and in this case\nrefers recursively to the pattern in which it appears.\n\nObviously, PCRE cannot support the interpolation of Perl code. Instead,\nit supports special syntax for recursion of the entire pattern, and\nalso for individual subpattern recursion. After its introduction in\nPCRE and Python, this kind of recursion was subsequently introduced\ninto Perl at release 5.10.\n\nA special item that consists of (? followed by a number greater than\nzero and a closing parenthesis is a recursive subroutine call of the\nsubpattern of the given number, provided that it occurs inside that\nsubpattern. (If not, it is a non-recursive subroutine call, which is\ndescribed in the next section.) The special item (?R) or (?0) is a re-\ncursive call of the entire regular expression.\n\nThis PCRE pattern solves the nested parentheses problem (assume the\nPCREEXTENDED option is set so that white space is ignored):\n\n\\( ( [^()]++ | (?R) )* \\)\n\nFirst it matches an opening parenthesis. Then it matches any number of\nsubstrings which can either be a sequence of non-parentheses, or a re-\ncursive match of the pattern itself (that is, a correctly parenthesized\nsubstring). Finally there is a closing parenthesis. Note the use of a\npossessive quantifier to avoid backtracking into sequences of non-\nparentheses.\n\nIf this were part of a larger pattern, you would not want to recurse\nthe entire pattern, so instead you could use this:\n\n( \\( ( [^()]++ | (?1) )* \\) )\n\nWe have put the pattern into parentheses, and caused the recursion to\nrefer to them instead of the whole pattern.\n\nIn a larger pattern, keeping track of parenthesis numbers can be\ntricky. This is made easier by the use of relative references. Instead\nof (?1) in the pattern above you can write (?-2) to refer to the second\nmost recently opened parentheses preceding the recursion. In other\nwords, a negative number counts capturing parentheses leftwards from\nthe point at which it is encountered.\n\nIt is also possible to refer to subsequently opened parentheses, by\nwriting references such as (?+2). However, these cannot be recursive\nbecause the reference is not inside the parentheses that are refer-\nenced. They are always non-recursive subroutine calls, as described in\nthe next section.\n\nAn alternative approach is to use named parentheses instead. The Perl\nsyntax for this is (?&name); PCRE's earlier syntax (?P>name) is also\nsupported. We could rewrite the above example as follows:\n\n(? \\( ( [^()]++ | (?&pn) )* \\) )\n\nIf there is more than one subpattern with the same name, the earliest\none is used.\n\nThis particular example pattern that we have been looking at contains\nnested unlimited repeats, and so the use of a possessive quantifier for\nmatching strings of non-parentheses is important when applying the pat-\ntern to strings that do not match. For example, when this pattern is\napplied to\n\n(aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa()\n\nit yields \"no match\" quickly. However, if a possessive quantifier is\nnot used, the match runs for a very long time indeed because there are\nso many different ways the + and * repeats can carve up the subject,\nand all have to be tested before failure can be reported.\n\nAt the end of a match, the values of capturing parentheses are those\nfrom the outermost level. If you want to obtain intermediate values, a\ncallout function can be used (see below and the pcrecallout documenta-\ntion). If the pattern above is matched against\n\n(ab(cd)ef)\n\nthe value for the inner capturing parentheses (numbered 2) is \"ef\",\nwhich is the last value taken on at the top level. If a capturing sub-\npattern is not matched at the top level, its final captured value is\nunset, even if it was (temporarily) set at a deeper level during the\nmatching process.\n\nIf there are more than 15 capturing parentheses in a pattern, PCRE has\nto obtain extra memory to store data during a recursion, which it does\nby using pcremalloc, freeing it via pcrefree afterwards. If no memory\ncan be obtained, the match fails with the PCREERRORNOMEMORY error.\n\nDo not confuse the (?R) item with the condition (R), which tests for\nrecursion. Consider this pattern, which matches text in angle brack-\nets, allowing for arbitrary nesting. Only digits are allowed in nested\nbrackets (that is, when recursing), whereas any characters are permit-\nted at the outer level.\n\n< (?: (?(R) \\d++ | [^<>]*+) | (?R)) * >\n\nIn this pattern, (?(R) is the start of a conditional subpattern, with\ntwo different alternatives for the recursive and non-recursive cases.\nThe (?R) item is the actual recursive call.\n\nDifferences in recursion processing between PCRE and Perl\n\nRecursion processing in PCRE differs from Perl in two important ways.\nIn PCRE (like Python, but unlike Perl), a recursive subpattern call is\nalways treated as an atomic group. That is, once it has matched some of\nthe subject string, it is never re-entered, even if it contains untried\nalternatives and there is a subsequent matching failure. This can be\nillustrated by the following pattern, which purports to match a palin-\ndromic string that contains an odd number of characters (for example,\n\"a\", \"aba\", \"abcba\", \"abcdcba\"):\n\n^(.|(.)(?1)\\2)$\n\nThe idea is that it either matches a single character, or two identical\ncharacters surrounding a sub-palindrome. In Perl, this pattern works;\nin PCRE it does not if the pattern is longer than three characters.\nConsider the subject string \"abcba\":\n\nAt the top level, the first character is matched, but as it is not at\nthe end of the string, the first alternative fails; the second alterna-\ntive is taken and the recursion kicks in. The recursive call to subpat-\ntern 1 successfully matches the next character (\"b\"). (Note that the\nbeginning and end of line tests are not part of the recursion).\n\nBack at the top level, the next character (\"c\") is compared with what\nsubpattern 2 matched, which was \"a\". This fails. Because the recursion\nis treated as an atomic group, there are now no backtracking points,\nand so the entire match fails. (Perl is able, at this point, to re-en-\nter the recursion and try the second alternative.) However, if the pat-\ntern is written with the alternatives in the other order, things are\ndifferent:\n\n^((.)(?1)\\2|.)$\n\nThis time, the recursing alternative is tried first, and continues to\nrecurse until it runs out of characters, at which point the recursion\nfails. But this time we do have another alternative to try at the\nhigher level. That is the big difference: in the previous case the re-\nmaining alternative is at a deeper recursion level, which PCRE cannot\nuse.\n\nTo change the pattern so that it matches all palindromic strings, not\njust those with an odd number of characters, it is tempting to change\nthe pattern to this:\n\n^((.)(?1)\\2|.?)$\n\nAgain, this works in Perl, but not in PCRE, and for the same reason.\nWhen a deeper recursion has matched a single character, it cannot be\nentered again in order to match an empty string. The solution is to\nseparate the two cases, and write out the odd and even cases as alter-\nnatives at the higher level:\n\n^(?:((.)(?1)\\2|)|((.)(?3)\\4|.))\n\nIf you want to match typical palindromic phrases, the pattern has to\nignore all non-word characters, which can be done like this:\n\n^\\W*+(?:((.)\\W*+(?1)\\W*+\\2|)|((.)\\W*+(?3)\\W*+\\4|\\W*+.\\W*+))\\W*+$\n\nIf run with the PCRECASELESS option, this pattern matches phrases such\nas \"A man, a plan, a canal: Panama!\" and it works well in both PCRE and\nPerl. Note the use of the possessive quantifier *+ to avoid backtrack-\ning into sequences of non-word characters. Without this, PCRE takes a\ngreat deal longer (ten times or more) to match typical phrases, and\nPerl takes so long that you think it has gone into a loop.\n\nWARNING: The palindrome-matching patterns above work only if the sub-\nject string does not start with a palindrome that is shorter than the\nentire string. For example, although \"abcba\" is correctly matched, if\nthe subject is \"ababa\", PCRE finds the palindrome \"aba\" at the start,\nthen fails at top level because the end of the string does not follow.\nOnce again, it cannot jump back into the recursion to try other alter-\nnatives, so the entire match fails.\n\nThe second way in which PCRE and Perl differ in their recursion pro-\ncessing is in the handling of captured values. In Perl, when a subpat-\ntern is called recursively or as a subpattern (see the next section),\nit has no access to any values that were captured outside the recur-\nsion, whereas in PCRE these values can be referenced. Consider this\npattern:\n\n^(.)(\\1|a(?2))\n\nIn PCRE, this pattern matches \"bab\". The first capturing parentheses\nmatch \"b\", then in the second group, when the back reference \\1 fails\nto match \"b\", the second alternative matches \"a\" and then recurses. In\nthe recursion, \\1 does now match \"b\" and so the whole match succeeds.\nIn Perl, the pattern fails to match because inside the recursive call\n\\1 cannot access the externally set value.\n", "subsections": [] }, "SUBPATTERNS AS SUBROUTINES": { "content": "If the syntax for a recursive subpattern call (either by number or by\nname) is used outside the parentheses to which it refers, it operates\nlike a subroutine in a programming language. The called subpattern may\nbe defined before or after the reference. A numbered reference can be\nabsolute or relative, as in these examples:\n\n(...(absolute)...)...(?2)...\n(...(relative)...)...(?-1)...\n(...(?+1)...(relative)...\n\nAn earlier example pointed out that the pattern\n\n(sens|respons)e and \\1ibility\n\nmatches \"sense and sensibility\" and \"response and responsibility\", but\nnot \"sense and responsibility\". If instead the pattern\n\n(sens|respons)e and (?1)ibility\n\nis used, it does match \"sense and responsibility\" as well as the other\ntwo strings. Another example is given in the discussion of DEFINE\nabove.\n\nAll subroutine calls, whether recursive or not, are always treated as\natomic groups. That is, once a subroutine has matched some of the sub-\nject string, it is never re-entered, even if it contains untried alter-\nnatives and there is a subsequent matching failure. Any capturing\nparentheses that are set during the subroutine call revert to their\nprevious values afterwards.\n\nProcessing options such as case-independence are fixed when a subpat-\ntern is defined, so if it is used as a subroutine, such options cannot\nbe changed for different calls. For example, consider this pattern:\n\n(abc)(?i:(?-1))\n\nIt matches \"abcabc\". It does not match \"abcABC\" because the change of\nprocessing option does not affect the called subpattern.\n", "subsections": [] }, "ONIGURUMA SUBROUTINE SYNTAX": { "content": "For compatibility with Oniguruma, the non-Perl syntax \\g followed by a\nname or a number enclosed either in angle brackets or single quotes, is\nan alternative syntax for referencing a subpattern as a subroutine,\npossibly recursively. Here are two of the examples used above, rewrit-\nten using this syntax:\n\n(? \\( ( (?>[^()]+) | \\g )* \\) )\n(sens|respons)e and \\g'1'ibility\n\nPCRE supports an extension to Oniguruma: if a number is preceded by a\nplus or a minus sign it is taken as a relative reference. For example:\n\n(abc)(?i:\\g<-1>)\n\nNote that \\g{...} (Perl syntax) and \\g<...> (Oniguruma syntax) are not\nsynonymous. The former is a back reference; the latter is a subroutine\ncall.\n", "subsections": [] }, "CALLOUTS": { "content": "Perl has a feature whereby using the sequence (?{...}) causes arbitrary\nPerl code to be obeyed in the middle of matching a regular expression.\nThis makes it possible, amongst other things, to extract different sub-\nstrings that match the same pair of parentheses when there is a repeti-\ntion.\n\nPCRE provides a similar feature, but of course it cannot obey arbitrary\nPerl code. The feature is called \"callout\". The caller of PCRE provides\nan external function by putting its entry point in the global variable\npcrecallout (8-bit library) or pcre[16|32]callout (16-bit or 32-bit\nlibrary). By default, this variable contains NULL, which disables all\ncalling out.\n\nWithin a regular expression, (?C) indicates the points at which the ex-\nternal function is to be called. If you want to identify different\ncallout points, you can put a number less than 256 after the letter C.\nThe default value is zero. For example, this pattern has two callout\npoints:\n\n(?C1)abc(?C2)def\n\nIf the PCREAUTOCALLOUT flag is passed to a compiling function, call-\nouts are automatically installed before each item in the pattern. They\nare all numbered 255. If there is a conditional group in the pattern\nwhose condition is an assertion, an additional callout is inserted just\nbefore the condition. An explicit callout may also be set at this posi-\ntion, as in this example:\n\n(?(?C9)(?=a)abc|def)\n\nNote that this applies only to assertion conditions, not to other types\nof condition.\n\nDuring matching, when PCRE reaches a callout point, the external func-\ntion is called. It is provided with the number of the callout, the po-\nsition in the pattern, and, optionally, one item of data originally\nsupplied by the caller of the matching function. The callout function\nmay cause matching to proceed, to backtrack, or to fail altogether.\n\nBy default, PCRE implements a number of optimizations at compile time\nand matching time, and one side-effect is that sometimes callouts are\nskipped. If you need all possible callouts to happen, you need to set\noptions that disable the relevant optimizations. More details, and a\ncomplete description of the interface to the callout function, are\ngiven in the pcrecallout documentation.\n", "subsections": [] }, "BACKTRACKING CONTROL": { "content": "Perl 5.10 introduced a number of \"Special Backtracking Control Verbs\",\nwhich are still described in the Perl documentation as \"experimental\nand subject to change or removal in a future version of Perl\". It goes\non to say: \"Their usage in production code should be noted to avoid\nproblems during upgrades.\" The same remarks apply to the PCRE features\ndescribed in this section.\n\nThe new verbs make use of what was previously invalid syntax: an open-\ning parenthesis followed by an asterisk. They are generally of the form\n(*VERB) or (*VERB:NAME). Some may take either form, possibly behaving\ndifferently depending on whether or not a name is present. A name is\nany sequence of characters that does not include a closing parenthesis.\nThe maximum length of name is 255 in the 8-bit library and 65535 in the\n16-bit and 32-bit libraries. If the name is empty, that is, if the\nclosing parenthesis immediately follows the colon, the effect is as if\nthe colon were not there. Any number of these verbs may occur in a\npattern.\n\nSince these verbs are specifically related to backtracking, most of\nthem can be used only when the pattern is to be matched using one of\nthe traditional matching functions, because these use a backtracking\nalgorithm. With the exception of (*FAIL), which behaves like a failing\nnegative assertion, the backtracking control verbs cause an error if\nencountered by a DFA matching function.\n\nThe behaviour of these verbs in repeated groups, assertions, and in\nsubpatterns called as subroutines (whether or not recursively) is docu-\nmented below.\n\nOptimizations that affect backtracking verbs\n\nPCRE contains some optimizations that are used to speed up matching by\nrunning some checks at the start of each match attempt. For example, it\nmay know the minimum length of matching subject, or that a particular\ncharacter must be present. When one of these optimizations bypasses the\nrunning of a match, any included backtracking verbs will not, of\ncourse, be processed. You can suppress the start-of-match optimizations\nby setting the PCRENOSTARTOPTIMIZE option when calling pcrecom-\npile() or pcreexec(), or by starting the pattern with (*NOSTARTOPT).\nThere is more discussion of this option in the section entitled \"Option\nbits for pcreexec()\" in the pcreapi documentation.\n\nExperiments with Perl suggest that it too has similar optimizations,\nsometimes leading to anomalous results.\n\nVerbs that act immediately\n\nThe following verbs act as soon as they are encountered. They may not\nbe followed by a name.\n\n(*ACCEPT)\n\nThis verb causes the match to end successfully, skipping the remainder\nof the pattern. However, when it is inside a subpattern that is called\nas a subroutine, only that subpattern is ended successfully. Matching\nthen continues at the outer level. If (*ACCEPT) in triggered in a posi-\ntive assertion, the assertion succeeds; in a negative assertion, the\nassertion fails.\n\nIf (*ACCEPT) is inside capturing parentheses, the data so far is cap-\ntured. For example:\n\nA((?:A|B(*ACCEPT)|C)D)\n\nThis matches \"AB\", \"AAD\", or \"ACD\"; when it matches \"AB\", \"B\" is cap-\ntured by the outer parentheses.\n\n(*FAIL) or (*F)\n\nThis verb causes a matching failure, forcing backtracking to occur. It\nis equivalent to (?!) but easier to read. The Perl documentation notes\nthat it is probably useful only when combined with (?{}) or (??{}).\nThose are, of course, Perl features that are not present in PCRE. The\nnearest equivalent is the callout feature, as for example in this pat-\ntern:\n\na+(?C)(*FAIL)\n\nA match with the string \"aaaa\" always fails, but the callout is taken\nbefore each backtrack happens (in this example, 10 times).\n\nRecording which path was taken\n\nThere is one verb whose main purpose is to track how a match was ar-\nrived at, though it also has a secondary use in conjunction with ad-\nvancing the match starting point (see (*SKIP) below).\n\n(*MARK:NAME) or (*:NAME)\n\nA name is always required with this verb. There may be as many in-\nstances of (*MARK) as you like in a pattern, and their names do not\nhave to be unique.\n\nWhen a match succeeds, the name of the last-encountered (*MARK:NAME),\n(*PRUNE:NAME), or (*THEN:NAME) on the matching path is passed back to\nthe caller as described in the section entitled \"Extra data for\npcreexec()\" in the pcreapi documentation. Here is an example of\npcretest output, where the /K modifier requests the retrieval and out-\nputting of (*MARK) data:\n\nre> /X(*MARK:A)Y|X(*MARK:B)Z/K\ndata> XY\n0: XY\nMK: A\nXZ\n0: XZ\nMK: B\n\nThe (*MARK) name is tagged with \"MK:\" in this output, and in this exam-\nple it indicates which of the two alternatives matched. This is a more\nefficient way of obtaining this information than putting each alterna-\ntive in its own capturing parentheses.\n\nIf a verb with a name is encountered in a positive assertion that is\ntrue, the name is recorded and passed back if it is the last-encoun-\ntered. This does not happen for negative assertions or failing positive\nassertions.\n\nAfter a partial match or a failed match, the last encountered name in\nthe entire match process is returned. For example:\n\nre> /X(*MARK:A)Y|X(*MARK:B)Z/K\ndata> XP\nNo match, mark = B\n\nNote that in this unanchored example the mark is retained from the\nmatch attempt that started at the letter \"X\" in the subject. Subsequent\nmatch attempts starting at \"P\" and then with an empty string do not get\nas far as the (*MARK) item, but nevertheless do not reset it.\n\nIf you are interested in (*MARK) values after failed matches, you\nshould probably set the PCRENOSTARTOPTIMIZE option (see above) to\nensure that the match is always attempted.\n\nVerbs that act after backtracking\n\nThe following verbs do nothing when they are encountered. Matching con-\ntinues with what follows, but if there is no subsequent match, causing\na backtrack to the verb, a failure is forced. That is, backtracking\ncannot pass to the left of the verb. However, when one of these verbs\nappears inside an atomic group or an assertion that is true, its effect\nis confined to that group, because once the group has been matched,\nthere is never any backtracking into it. In this situation, backtrack-\ning can \"jump back\" to the left of the entire atomic group or asser-\ntion. (Remember also, as stated above, that this localization also ap-\nplies in subroutine calls.)\n\nThese verbs differ in exactly what kind of failure occurs when back-\ntracking reaches them. The behaviour described below is what happens\nwhen the verb is not in a subroutine or an assertion. Subsequent sec-\ntions cover these special cases.\n\n(*COMMIT)\n\nThis verb, which may not be followed by a name, causes the whole match\nto fail outright if there is a later matching failure that causes back-\ntracking to reach it. Even if the pattern is unanchored, no further at-\ntempts to find a match by advancing the starting point take place. If\n(*COMMIT) is the only backtracking verb that is encountered, once it\nhas been passed pcreexec() is committed to finding a match at the cur-\nrent starting point, or not at all. For example:\n\na+(*COMMIT)b\n\nThis matches \"xxaab\" but not \"aacaab\". It can be thought of as a kind\nof dynamic anchor, or \"I've started, so I must finish.\" The name of the\nmost recently passed (*MARK) in the path is passed back when (*COMMIT)\nforces a match failure.\n\nIf there is more than one backtracking verb in a pattern, a different\none that follows (*COMMIT) may be triggered first, so merely passing\n(*COMMIT) during a match does not always guarantee that a match must be\nat this starting point.\n\nNote that (*COMMIT) at the start of a pattern is not the same as an an-\nchor, unless PCRE's start-of-match optimizations are turned off, as\nshown in this output from pcretest:\n\nre> /(*COMMIT)abc/\ndata> xyzabc\n0: abc\ndata> xyzabc\\Y\nNo match\n\nFor this pattern, PCRE knows that any match must start with \"a\", so the\noptimization skips along the subject to \"a\" before applying the pattern\nto the first set of data. The match attempt then succeeds. In the sec-\nond set of data, the escape sequence \\Y is interpreted by the pcretest\nprogram. It causes the PCRENOSTARTOPTIMIZE option to be set when\npcreexec() is called. This disables the optimization that skips along\nto the first character. The pattern is now applied starting at \"x\", and\nso the (*COMMIT) causes the match to fail without trying any other\nstarting points.\n\n(*PRUNE) or (*PRUNE:NAME)\n\nThis verb causes the match to fail at the current starting position in\nthe subject if there is a later matching failure that causes backtrack-\ning to reach it. If the pattern is unanchored, the normal \"bumpalong\"\nadvance to the next starting character then happens. Backtracking can\noccur as usual to the left of (*PRUNE), before it is reached, or when\nmatching to the right of (*PRUNE), but if there is no match to the\nright, backtracking cannot cross (*PRUNE). In simple cases, the use of\n(*PRUNE) is just an alternative to an atomic group or possessive quan-\ntifier, but there are some uses of (*PRUNE) that cannot be expressed in\nany other way. In an anchored pattern (*PRUNE) has the same effect as\n(*COMMIT).\n\nThe behaviour of (*PRUNE:NAME) is the not the same as\n(*MARK:NAME)(*PRUNE). It is like (*MARK:NAME) in that the name is re-\nmembered for passing back to the caller. However, (*SKIP:NAME) searches\nonly for names set with (*MARK).\n\n(*SKIP)\n\nThis verb, when given without a name, is like (*PRUNE), except that if\nthe pattern is unanchored, the \"bumpalong\" advance is not to the next\ncharacter, but to the position in the subject where (*SKIP) was encoun-\ntered. (*SKIP) signifies that whatever text was matched leading up to\nit cannot be part of a successful match. Consider:\n\na+(*SKIP)b\n\nIf the subject is \"aaaac...\", after the first match attempt fails\n(starting at the first character in the string), the starting point\nskips on to start the next attempt at \"c\". Note that a possessive quan-\ntifer does not have the same effect as this example; although it would\nsuppress backtracking during the first match attempt, the second at-\ntempt would start at the second character instead of skipping on to\n\"c\".\n\n(*SKIP:NAME)\n\nWhen (*SKIP) has an associated name, its behaviour is modified. When it\nis triggered, the previous path through the pattern is searched for the\nmost recent (*MARK) that has the same name. If one is found, the\n\"bumpalong\" advance is to the subject position that corresponds to that\n(*MARK) instead of to where (*SKIP) was encountered. If no (*MARK) with\na matching name is found, the (*SKIP) is ignored.\n\nNote that (*SKIP:NAME) searches only for names set by (*MARK:NAME). It\nignores names that are set by (*PRUNE:NAME) or (*THEN:NAME).\n\n(*THEN) or (*THEN:NAME)\n\nThis verb causes a skip to the next innermost alternative when back-\ntracking reaches it. That is, it cancels any further backtracking\nwithin the current alternative. Its name comes from the observation\nthat it can be used for a pattern-based if-then-else block:\n\n( COND1 (*THEN) FOO | COND2 (*THEN) BAR | COND3 (*THEN) BAZ ) ...\n\nIf the COND1 pattern matches, FOO is tried (and possibly further items\nafter the end of the group if FOO succeeds); on failure, the matcher\nskips to the second alternative and tries COND2, without backtracking\ninto COND1. If that succeeds and BAR fails, COND3 is tried. If subse-\nquently BAZ fails, there are no more alternatives, so there is a back-\ntrack to whatever came before the entire group. If (*THEN) is not in-\nside an alternation, it acts like (*PRUNE).\n\nThe behaviour of (*THEN:NAME) is the not the same as\n(*MARK:NAME)(*THEN). It is like (*MARK:NAME) in that the name is re-\nmembered for passing back to the caller. However, (*SKIP:NAME) searches\nonly for names set with (*MARK).\n\nA subpattern that does not contain a | character is just a part of the\nenclosing alternative; it is not a nested alternation with only one al-\nternative. The effect of (*THEN) extends beyond such a subpattern to\nthe enclosing alternative. Consider this pattern, where A, B, etc. are\ncomplex pattern fragments that do not contain any | characters at this\nlevel:\n\nA (B(*THEN)C) | D\n\nIf A and B are matched, but there is a failure in C, matching does not\nbacktrack into A; instead it moves to the next alternative, that is, D.\nHowever, if the subpattern containing (*THEN) is given an alternative,\nit behaves differently:\n\nA (B(*THEN)C | (*FAIL)) | D\n\nThe effect of (*THEN) is now confined to the inner subpattern. After a\nfailure in C, matching moves to (*FAIL), which causes the whole subpat-\ntern to fail because there are no more alternatives to try. In this\ncase, matching does now backtrack into A.\n\nNote that a conditional subpattern is not considered as having two al-\nternatives, because only one is ever used. In other words, the | char-\nacter in a conditional subpattern has a different meaning. Ignoring\nwhite space, consider:\n\n^.*? (?(?=a) a | b(*THEN)c )\n\nIf the subject is \"ba\", this pattern does not match. Because .*? is un-\ngreedy, it initially matches zero characters. The condition (?=a) then\nfails, the character \"b\" is matched, but \"c\" is not. At this point,\nmatching does not backtrack to .*? as might perhaps be expected from\nthe presence of the | character. The conditional subpattern is part of\nthe single alternative that comprises the whole pattern, and so the\nmatch fails. (If there was a backtrack into .*?, allowing it to match\n\"b\", the match would succeed.)\n\nThe verbs just described provide four different \"strengths\" of control\nwhen subsequent matching fails. (*THEN) is the weakest, carrying on the\nmatch at the next alternative. (*PRUNE) comes next, failing the match\nat the current starting position, but allowing an advance to the next\ncharacter (for an unanchored pattern). (*SKIP) is similar, except that\nthe advance may be more than one character. (*COMMIT) is the strongest,\ncausing the entire match to fail.\n\nMore than one backtracking verb\n\nIf more than one backtracking verb is present in a pattern, the one\nthat is backtracked onto first acts. For example, consider this pat-\ntern, where A, B, etc. are complex pattern fragments:\n\n(A(*COMMIT)B(*THEN)C|ABD)\n\nIf A matches but B fails, the backtrack to (*COMMIT) causes the entire\nmatch to fail. However, if A and B match, but C fails, the backtrack to\n(*THEN) causes the next alternative (ABD) to be tried. This behaviour\nis consistent, but is not always the same as Perl's. It means that if\ntwo or more backtracking verbs appear in succession, all the the last\nof them has no effect. Consider this example:\n\n...(*COMMIT)(*PRUNE)...\n\nIf there is a matching failure to the right, backtracking onto (*PRUNE)\ncauses it to be triggered, and its action is taken. There can never be\na backtrack onto (*COMMIT).\n\nBacktracking verbs in repeated groups\n\nPCRE differs from Perl in its handling of backtracking verbs in re-\npeated groups. For example, consider:\n\n/(a(*COMMIT)b)+ac/\n\nIf the subject is \"abac\", Perl matches, but PCRE fails because the\n(*COMMIT) in the second repeat of the group acts.\n\nBacktracking verbs in assertions\n\n(*FAIL) in an assertion has its normal effect: it forces an immediate\nbacktrack.\n\n(*ACCEPT) in a positive assertion causes the assertion to succeed with-\nout any further processing. In a negative assertion, (*ACCEPT) causes\nthe assertion to fail without any further processing.\n\nThe other backtracking verbs are not treated specially if they appear\nin a positive assertion. In particular, (*THEN) skips to the next al-\nternative in the innermost enclosing group that has alternations,\nwhether or not this is within the assertion.\n\nNegative assertions are, however, different, in order to ensure that\nchanging a positive assertion into a negative assertion changes its re-\nsult. Backtracking into (*COMMIT), (*SKIP), or (*PRUNE) causes a nega-\ntive assertion to be true, without considering any further alternative\nbranches in the assertion. Backtracking into (*THEN) causes it to skip\nto the next enclosing alternative within the assertion (the normal be-\nhaviour), but if the assertion does not have such an alternative,\n(*THEN) behaves like (*PRUNE).\n\nBacktracking verbs in subroutines\n\nThese behaviours occur whether or not the subpattern is called recur-\nsively. Perl's treatment of subroutines is different in some cases.\n\n(*FAIL) in a subpattern called as a subroutine has its normal effect:\nit forces an immediate backtrack.\n\n(*ACCEPT) in a subpattern called as a subroutine causes the subroutine\nmatch to succeed without any further processing. Matching then contin-\nues after the subroutine call.\n\n(*COMMIT), (*SKIP), and (*PRUNE) in a subpattern called as a subroutine\ncause the subroutine match to fail.\n\n(*THEN) skips to the next alternative in the innermost enclosing group\nwithin the subpattern that has alternatives. If there is no such group\nwithin the subpattern, (*THEN) causes the subroutine match to fail.\n", "subsections": [] }, "SEE ALSO": { "content": "pcreapi(3), pcrecallout(3), pcrematching(3), pcresyntax(3), pcre(3),\npcre16(3), pcre32(3).\n", "subsections": [] }, "AUTHOR": { "content": "Philip Hazel\nUniversity Computing Service\nCambridge CB2 3QH, England.\n", "subsections": [] }, "REVISION": { "content": "Last updated: 14 June 2015\nCopyright (c) 1997-2015 University of Cambridge.\n\nPCRE 8.38 14 June 2015 PCREPATTERN(3)", "subsections": [] } } } }