{ "mode": "man", "parameter": "PCREPATTERN", "section": "3", "url": "https://www.chedong.com/phpMan.php/man/PCREPATTERN/3/json", "generated": "2026-06-13T20:00:54Z", "sections": { "NAME": { "content": "PCRE - Perl-compatible regular expressions\n", "subsections": [] }, "PCRE REGULAR EXPRESSION DETAILS": { "content": "The syntax and semantics of the regular expressions that are supported by PCRE are described\nin detail below. There is a quick-reference syntax summary in the pcresyntax page. PCRE tries\nto match Perl syntax and semantics as closely as it can. PCRE also supports some alternative\nregular expression syntax (which does not conflict with the Perl syntax) in order to provide\nsome compatibility with regular expressions in Python, .NET, and Oniguruma.\n\nPerl's regular expressions are described in its own documentation, and regular expressions in\ngeneral are covered in a number of books, some of which have copious examples. Jeffrey\nFriedl's \"Mastering Regular Expressions\", published by O'Reilly, covers regular expressions\nin great detail. This description of PCRE's regular expressions is intended as reference ma‐\nterial.\n\nThis document discusses the patterns that are supported by PCRE when one its main matching\nfunctions, pcreexec() (8-bit) or pcre[16|32]exec() (16- or 32-bit), is used. PCRE also has\nalternative matching functions, pcredfaexec() and pcre[16|32dfaexec(), which match using\na different algorithm that is not Perl-compatible. Some of the features discussed below are\nnot available when DFA matching is used. The advantages and disadvantages of the alternative\nfunctions, and how they differ from the normal functions, are discussed in the pcrematching\npage.\n", "subsections": [] }, "SPECIAL START-OF-PATTERN ITEMS": { "content": "A number of options that can be passed to pcrecompile() can also be set by special items at\nthe start of a pattern. These are not Perl-compatible, but are provided to make these options\naccessible to pattern writers who are not able to change the program that processes the pat‐\ntern. Any number of these items may appear, but they must all be together right at the start\nof the pattern string, and the letters must be in upper case.\n", "subsections": [ { "name": "UTF support", "content": "The original operation of PCRE was on strings of one-byte characters. However, there is now\nalso support for UTF-8 strings in the original library, an extra library that supports 16-bit\nand UTF-16 character strings, and a third library that supports 32-bit and UTF-32 character\nstrings. To use these features, PCRE must be built to include appropriate support. When using\nUTF strings you must either call the compiling function with the PCREUTF8, PCREUTF16, or\nPCREUTF32 option, or the pattern must start with one of these special sequences:\n\n(*UTF8)\n(*UTF16)\n(*UTF32)\n(*UTF)\n\n(*UTF) is a generic sequence that can be used with any of the libraries. Starting a pattern\nwith such a sequence is equivalent to setting the relevant option. How setting a UTF mode af‐\nfects pattern matching is mentioned in several places below. There is also a summary of fea‐\ntures in the pcreunicode page.\n\nSome applications that allow their users to supply patterns may wish to restrict them to non-\nUTF data for security reasons. If the PCRENEVERUTF option is set at compile time, (*UTF)\netc. are not allowed, and their appearance causes an error.\n" }, { "name": "Unicode property support", "content": "Another special sequence that may appear at the start of a pattern is (*UCP). This has the\nsame effect as setting the PCREUCP option: it causes sequences such as \\d and \\w to use Uni‐\ncode properties to determine character types, instead of recognizing only characters with\ncodes less than 128 via a lookup table.\n" }, { "name": "Disabling auto-possessification", "content": "If a pattern starts with (*NOAUTOPOSSESS), it has the same effect as setting the\nPCRENOAUTOPOSSESS option at compile time. This stops PCRE from making quantifiers posses‐\nsive when what follows cannot match the repeated item. For example, by default a+b is treated\nas a++b. For more details, see the pcreapi documentation.\n" }, { "name": "Disabling start-up optimizations", "content": "If a pattern starts with (*NOSTARTOPT), it has the same effect as setting the\nPCRENOSTARTOPTIMIZE option either at compile or matching time. This disables several opti‐\nmizations for quickly reaching \"no match\" results. For more details, see the pcreapi documen‐\ntation.\n" }, { "name": "Newline conventions", "content": "PCRE supports five different conventions for indicating line breaks in strings: a single CR\n(carriage return) character, a single LF (linefeed) character, the two-character sequence\nCRLF, any of the three preceding, or any Unicode newline sequence. The pcreapi page has fur‐\nther discussion about newlines, and shows how to set the newline convention in the options\narguments for the compiling and matching functions.\n\nIt is also possible to specify a newline convention by starting a pattern string with one of\nthe following five sequences:\n\n(*CR) carriage return\n(*LF) linefeed\n(*CRLF) carriage return, followed by linefeed\n(*ANYCRLF) any of the three above\n(*ANY) all Unicode newline sequences\n\nThese override the default and the options given to the compiling function. For example, on a\nUnix system where LF is the default newline sequence, the pattern\n\n(*CR)a.b\n\nchanges the convention to CR. That pattern matches \"a\\nb\" because LF is no longer a newline.\nIf more than one of these settings is present, the last one is used.\n\nThe newline convention affects where the circumflex and dollar assertions are true. It also\naffects the interpretation of the dot metacharacter when PCREDOTALL is not set, and the be‐\nhaviour of \\N. However, it does not affect what the \\R escape sequence matches. By default,\nthis is any Unicode newline sequence, for Perl compatibility. However, this can be changed;\nsee the description of \\R in the section entitled \"Newline sequences\" below. A change of \\R\nsetting can be combined with a change of newline convention.\n" }, { "name": "Setting match and recursion limits", "content": "The caller of pcreexec() can set a limit on the number of times the internal match() func‐\ntion is called and on the maximum depth of recursive calls. These facilities are provided to\ncatch runaway matches that are provoked by patterns with huge matching trees (a typical exam‐\nple is a pattern with nested unlimited repeats) and to avoid running out of system stack by\ntoo much recursion. When one of these limits is reached, pcreexec() gives an error return.\nThe limits can also be set by items at the start of the pattern of the form\n\n(*LIMITMATCH=d)\n(*LIMITRECURSION=d)\n\nwhere d is any number of decimal digits. However, the value of the setting must be less than\nthe value set (or defaulted) by the caller of pcreexec() for it to have any effect. In other\nwords, the pattern writer can lower the limits set by the programmer, but not raise them. If\nthere is more than one setting of one of these limits, the lower value is used.\n" } ] }, "EBCDIC CHARACTER CODES": { "content": "PCRE can be compiled to run in an environment that uses EBCDIC as its character code rather\nthan ASCII or Unicode (typically a mainframe system). In the sections below, character code\nvalues are ASCII or Unicode; in an EBCDIC environment these characters may have different\ncode values, and there are no code points greater than 255.\n", "subsections": [] }, "CHARACTERS AND METACHARACTERS": { "content": "A regular expression is a pattern that is matched against a subject string from left to\nright. Most characters stand for themselves in a pattern, and match the corresponding charac‐\nters in the subject. As a trivial example, the pattern\n\nThe quick brown fox\n\nmatches a portion of a subject string that is identical to itself. When caseless matching is\nspecified (the PCRECASELESS option), letters are matched independently of case. In a UTF\nmode, PCRE always understands the concept of case for characters whose values are less than\n128, so caseless matching is always possible. For characters with higher values, the concept\nof case is supported if PCRE is compiled with Unicode property support, but not otherwise.\nIf you want to use caseless matching for characters 128 and above, you must ensure that PCRE\nis compiled with Unicode property support as well as with UTF support.\n\nThe power of regular expressions comes from the ability to include alternatives and repeti‐\ntions in the pattern. These are encoded in the pattern by the use of metacharacters, which do\nnot stand for themselves but instead are interpreted in some special way.\n\nThere are two different sets of metacharacters: those that are recognized anywhere in the\npattern except within square brackets, and those that are recognized within square brackets.\nOutside square brackets, the metacharacters are as follows:\n\n\\ general escape character with several uses\n^ assert start of string (or line, in multiline mode)\n$ assert end of string (or line, in multiline mode)\n. match any character except newline (by default)\n[ start character class definition\n| start of alternative branch\n( start subpattern\n) end subpattern\n? extends the meaning of (\nalso 0 or 1 quantifier\nalso quantifier minimizer\n* 0 or more quantifier\n+ 1 or more quantifier\nalso \"possessive quantifier\"\n{ start min/max quantifier\n\nPart of a pattern that is in square brackets is called a \"character class\". In a character\nclass the only metacharacters are:\n\n\\ general escape character\n^ negate the class, but only if the first character\n- indicates character range\n[ POSIX character class (only if followed by POSIX\nsyntax)\n] terminates the character class\n\nThe following sections describe the use of each of the metacharacters.\n", "subsections": [] }, "BACKSLASH": { "content": "The backslash character has several uses. Firstly, if it is followed by a character that is\nnot a number or a letter, it takes away any special meaning that character may have. This use\nof backslash as an escape character applies both inside and outside character classes.\n\nFor example, if you want to match a * character, you write \\* in the pattern. This escaping\naction applies whether or not the following character would otherwise be interpreted as a\nmetacharacter, so it is always safe to precede a non-alphanumeric with backslash to specify\nthat it stands for itself. In particular, if you want to match a backslash, you write \\\\.\n\nIn a UTF mode, only ASCII numbers and letters have any special meaning after a backslash. All\nother characters (in particular, those whose codepoints are greater than 127) are treated as\nliterals.\n\nIf a pattern is compiled with the PCREEXTENDED option, most white space in the pattern\n(other than in a character class), and characters between a # outside a character class and\nthe next newline, inclusive, are ignored. An escaping backslash can be used to include a\nwhite space or # character as part of the pattern.\n\nIf you want to remove the special meaning from a sequence of characters, you can do so by\nputting them between \\Q and \\E. This is different from Perl in that $ and @ are handled as\nliterals in \\Q...\\E sequences in PCRE, whereas in Perl, $ and @ cause variable interpolation.\nNote the following examples:\n\nPattern PCRE matches Perl matches\n\n\\Qabc$xyz\\E abc$xyz abc followed by the\ncontents of $xyz\n\\Qabc\\$xyz\\E abc\\$xyz abc\\$xyz\n\\Qabc\\E\\$\\Qxyz\\E abc$xyz abc$xyz\n\nThe \\Q...\\E sequence is recognized both inside and outside character classes. An isolated \\E\nthat is not preceded by \\Q is ignored. If \\Q is not followed by \\E later in the pattern, the\nliteral interpretation continues to the end of the pattern (that is, \\E is assumed at the\nend). If the isolated \\Q is inside a character class, this causes an error, because the char‐\nacter class is not terminated.\n", "subsections": [ { "name": "Non-printing characters", "content": "A second use of backslash provides a way of encoding non-printing characters in patterns in a\nvisible manner. There is no restriction on the appearance of non-printing characters, apart\nfrom the binary zero that terminates a pattern, but when a pattern is being prepared by text\nediting, it is often easier to use one of the following escape sequences than the binary\ncharacter it represents. In an ASCII or Unicode environment, these escapes are as follows:\n\n\\a alarm, that is, the BEL character (hex 07)\n\\cx \"control-x\", where x is any ASCII character\n\\e escape (hex 1B)\n\\f form feed (hex 0C)\n\\n linefeed (hex 0A)\n\\r carriage return (hex 0D)\n\\t tab (hex 09)\n\\0dd character with octal code 0dd\n\\ddd character with octal code ddd, or back reference\n\\o{ddd..} character with octal code ddd..\n\\xhh character with hex code hh\n\\x{hhh..} character with hex code hhh.. (non-JavaScript mode)\n\\uhhhh character with hex code hhhh (JavaScript mode only)\n\nThe precise effect of \\cx on ASCII characters is as follows: if x is a lower case letter, it\nis converted to upper case. Then bit 6 of the character (hex 40) is inverted. Thus \\cA to \\cZ\nbecome hex 01 to hex 1A (A is 41, Z is 5A), but \\c{ becomes hex 3B ({ is 7B), and \\c; becomes\nhex 7B (; is 3B). If the data item (byte or 16-bit value) following \\c has a value greater\nthan 127, a compile-time error occurs. This locks out non-ASCII characters in all modes.\n\nWhen PCRE is compiled in EBCDIC mode, \\a, \\e, \\f, \\n, \\r, and \\t generate the appropriate\nEBCDIC code values. The \\c escape is processed as specified for Perl in the perlebcdic docu‐\nment. The only characters that are allowed after \\c are A-Z, a-z, or one of @, [, \\, ], ^, ,\nor ?. Any other character provokes a compile-time error. The sequence \\@ encodes character\ncode 0; the letters (in either case) encode characters 1-26 (hex 01 to hex 1A); [, \\, ], ^,\nand encode characters 27-31 (hex 1B to hex 1F), and \\? becomes either 255 (hex FF) or 95\n(hex 5F).\n\nThus, apart from \\?, these escapes generate the same character code values as they do in an\nASCII environment, though the meanings of the values mostly differ. For example, \\G always\ngenerates code value 7, which is BEL in ASCII but DEL in EBCDIC.\n\nThe sequence \\? generates DEL (127, hex 7F) in an ASCII environment, but because 127 is not a\ncontrol character in EBCDIC, Perl makes it generate the APC character. Unfortunately, there\nare several variants of EBCDIC. In most of them the APC character has the value 255 (hex FF),\nbut in the one Perl calls POSIX-BC its value is 95 (hex 5F). If certain other characters have\nPOSIX-BC values, PCRE makes \\? generate 95; otherwise it generates 255.\n\nAfter \\0 up to two further octal digits are read. If there are fewer than two digits, just\nthose that are present are used. Thus the sequence \\0\\x\\015 specifies two binary zeros fol‐\nlowed by a CR character (code value 13). Make sure you supply two digits after the initial\nzero if the pattern character that follows is itself an octal digit.\n\nThe escape \\o must be followed by a sequence of octal digits, enclosed in braces. An error\noccurs if this is not the case. This escape is a recent addition to Perl; it provides way of\nspecifying character code points as octal numbers greater than 0777, and it also allows octal\nnumbers and back references to be unambiguously specified.\n\nFor greater clarity and unambiguity, it is best to avoid following \\ by a digit greater than\nzero. Instead, use \\o{} or \\x{} to specify character numbers, and \\g{} to specify back refer‐\nences. The following paragraphs describe the old, ambiguous syntax.\n\nThe handling of a backslash followed by a digit other than 0 is complicated, and Perl has\nchanged in recent releases, causing PCRE also to change. Outside a character class, PCRE\nreads the digit and any following digits as a decimal number. If the number is less than 8,\nor if there have been at least that many previous capturing left parentheses in the expres‐\nsion, the entire sequence is taken as a back reference. A description of how this works is\ngiven later, following the discussion of parenthesized subpatterns.\n\nInside a character class, or if the decimal number following \\ is greater than 7 and there\nhave not been that many capturing subpatterns, PCRE handles \\8 and \\9 as the literal charac‐\nters \"8\" and \"9\", and otherwise re-reads up to three octal digits following the backslash,\nusing them to generate a data character. Any subsequent digits stand for themselves. For ex‐\nample:\n\n\\040 is another way of writing an ASCII space\n\\40 is the same, provided there are fewer than 40\nprevious capturing subpatterns\n\\7 is always a back reference\n\\11 might be a back reference, or another way of\nwriting a tab\n\\011 is always a tab\n\\0113 is a tab followed by the character \"3\"\n\\113 might be a back reference, otherwise the\ncharacter with octal code 113\n\\377 might be a back reference, otherwise\nthe value 255 (decimal)\n\\81 is either a back reference, or the two\ncharacters \"8\" and \"1\"\n\nNote that octal values of 100 or greater that are specified using this syntax must not be in‐\ntroduced by a leading zero, because no more than three octal digits are ever read.\n\nBy default, after \\x that is not followed by {, from zero to two hexadecimal digits are read\n(letters can be in upper or lower case). Any number of hexadecimal digits may appear between\n\\x{ and }. If a character other than a hexadecimal digit appears between \\x{ and }, or if\nthere is no terminating }, an error occurs.\n\nIf the PCREJAVASCRIPTCOMPAT option is set, the interpretation of \\x is as just described\nonly when it is followed by two hexadecimal digits. Otherwise, it matches a literal \"x\"\ncharacter. In JavaScript mode, support for code points greater than 256 is provided by \\u,\nwhich must be followed by four hexadecimal digits; otherwise it matches a literal \"u\" charac‐\nter.\n\nCharacters whose value is less than 256 can be defined by either of the two syntaxes for \\x\n(or by \\u in JavaScript mode). There is no difference in the way they are handled. For exam‐\nple, \\xdc is exactly the same as \\x{dc} (or \\u00dc in JavaScript mode).\n" }, { "name": "Constraints on character values", "content": "Characters that are specified using octal or hexadecimal numbers are limited to certain val‐\nues, as follows:\n\n8-bit non-UTF mode less than 0x100\n8-bit UTF-8 mode less than 0x10ffff and a valid codepoint\n16-bit non-UTF mode less than 0x10000\n16-bit UTF-16 mode less than 0x10ffff and a valid codepoint\n32-bit non-UTF mode less than 0x100000000\n32-bit UTF-32 mode less than 0x10ffff and a valid codepoint\n\nInvalid Unicode codepoints are the range 0xd800 to 0xdfff (the so-called \"surrogate\" code‐\npoints), and 0xffef.\n" }, { "name": "Escape sequences in character classes", "content": "All the sequences that define a single character value can be used both inside and outside\ncharacter classes. In addition, inside a character class, \\b is interpreted as the backspace\ncharacter (hex 08).\n\n\\N is not allowed in a character class. \\B, \\R, and \\X are not special inside a character\nclass. Like other unrecognized escape sequences, they are treated as the literal characters\n\"B\", \"R\", and \"X\" by default, but cause an error if the PCREEXTRA option is set. Outside a\ncharacter class, these sequences have different meanings.\n" }, { "name": "Unsupported escape sequences", "content": "In Perl, the sequences \\l, \\L, \\u, and \\U are recognized by its string handler and used to\nmodify the case of following characters. By default, PCRE does not support these escape se‐\nquences. However, if the PCREJAVASCRIPTCOMPAT option is set, \\U matches a \"U\" character,\nand \\u can be used to define a character by code point, as described in the previous section.\n" }, { "name": "Absolute and relative back references", "content": "The sequence \\g followed by an unsigned or a negative number, optionally enclosed in braces,\nis an absolute or relative back reference. A named back reference can be coded as \\g{name}.\nBack references are discussed later, following the discussion of parenthesized subpatterns.\n" }, { "name": "Absolute and relative subroutine calls", "content": "For compatibility with Oniguruma, the non-Perl syntax \\g followed by a name or a number en‐\nclosed either in angle brackets or single quotes, is an alternative syntax for referencing a\nsubpattern as a \"subroutine\". Details are discussed later. Note that \\g{...} (Perl syntax)\nand \\g<...> (Oniguruma syntax) are not synonymous. The former is a back reference; the latter\nis a subroutine call.\n" }, { "name": "Generic character types", "content": "Another use of backslash is for specifying generic character types:\n\n\\d any decimal digit\n\\D any character that is not a decimal digit\n\\h any horizontal white space character\n\\H any character that is not a horizontal white space character\n\\s any white space character\n\\S any character that is not a white space character\n\\v any vertical white space character\n\\V any character that is not a vertical white space character\n\\w any \"word\" character\n\\W any \"non-word\" character\n\nThere is also the single sequence \\N, which matches a non-newline character. This is the\nsame as the \".\" metacharacter when PCREDOTALL is not set. Perl also uses \\N to match charac‐\nters by name; PCRE does not support this.\n\nEach pair of lower and upper case escape sequences partitions the complete set of characters\ninto two disjoint sets. Any given character matches one, and only one, of each pair. The se‐\nquences can appear both inside and outside character classes. They each match one character\nof the appropriate type. If the current matching point is at the end of the subject string,\nall of them fail, because there is no character to match.\n\nFor compatibility with Perl, \\s did not used to match the VT character (code 11), which made\nit different from the the POSIX \"space\" class. However, Perl added VT at release 5.18, and\nPCRE followed suit at release 8.34. The default \\s characters are now HT (9), LF (10), VT\n(11), FF (12), CR (13), and space (32), which are defined as white space in the \"C\" locale.\nThis list may vary if locale-specific matching is taking place. For example, in some locales\nthe \"non-breaking space\" character (\\xA0) is recognized as white space, and in others the VT\ncharacter is not.\n\nA \"word\" character is an underscore or any character that is a letter or digit. By default,\nthe definition of letters and digits is controlled by PCRE's low-valued character tables, and\nmay vary if locale-specific matching is taking place (see \"Locale support\" in the pcreapi\npage). For example, in a French locale such as \"frFR\" in Unix-like systems, or \"french\" in\nWindows, some character codes greater than 127 are used for accented letters, and these are\nthen matched by \\w. The use of locales with Unicode is discouraged.\n\nBy default, characters whose code points are greater than 127 never match \\d, \\s, or \\w, and\nalways match \\D, \\S, and \\W, although this may vary for characters in the range 128-255 when\nlocale-specific matching is happening. These escape sequences retain their original meanings\nfrom before Unicode support was available, mainly for efficiency reasons. If PCRE is compiled\nwith Unicode property support, and the PCREUCP option is set, the behaviour is changed so\nthat Unicode properties are used to determine character types, as follows:\n\n\\d any character that matches \\p{Nd} (decimal digit)\n\\s any character that matches \\p{Z} or \\h or \\v\n\\w any character that matches \\p{L} or \\p{N}, plus underscore\n\nThe upper case escapes match the inverse sets of characters. Note that \\d matches only deci‐\nmal digits, whereas \\w matches any Unicode digit, as well as any Unicode letter, and under‐\nscore. Note also that PCREUCP affects \\b, and \\B because they are defined in terms of \\w and\n\\W. Matching these sequences is noticeably slower when PCREUCP is set.\n\nThe sequences \\h, \\H, \\v, and \\V are features that were added to Perl at release 5.10. In\ncontrast to the other sequences, which match only ASCII characters by default, these always\nmatch certain high-valued code points, whether or not PCREUCP is set. The horizontal space\ncharacters are:\n\nU+0009 Horizontal tab (HT)\nU+0020 Space\nU+00A0 Non-break space\nU+1680 Ogham space mark\nU+180E Mongolian vowel separator\nU+2000 En quad\nU+2001 Em quad\nU+2002 En space\nU+2003 Em space\nU+2004 Three-per-em space\nU+2005 Four-per-em space\nU+2006 Six-per-em space\nU+2007 Figure space\nU+2008 Punctuation space\nU+2009 Thin space\nU+200A Hair space\nU+202F Narrow no-break space\nU+205F Medium mathematical space\nU+3000 Ideographic space\n\nThe vertical space characters are:\n\nU+000A Linefeed (LF)\nU+000B Vertical tab (VT)\nU+000C Form feed (FF)\nU+000D Carriage return (CR)\nU+0085 Next line (NEL)\nU+2028 Line separator\nU+2029 Paragraph separator\n\nIn 8-bit, non-UTF-8 mode, only the characters with codepoints less than 256 are relevant.\n" }, { "name": "Newline sequences", "content": "Outside a character class, by default, the escape sequence \\R matches any Unicode newline se‐\nquence. In 8-bit non-UTF-8 mode \\R is equivalent to the following:\n\n(?>\\r\\n|\\n|\\x0b|\\f|\\r|\\x85)\n\nThis is an example of an \"atomic group\", details of which are given below. This particular\ngroup matches either the two-character sequence CR followed by LF, or one of the single char‐\nacters LF (linefeed, U+000A), VT (vertical tab, U+000B), FF (form feed, U+000C), CR (carriage\nreturn, U+000D), or NEL (next line, U+0085). The two-character sequence is treated as a sin‐\ngle unit that cannot be split.\n\nIn other modes, two additional characters whose codepoints are greater than 255 are added: LS\n(line separator, U+2028) and PS (paragraph separator, U+2029). Unicode character property\nsupport is not needed for these characters to be recognized.\n\nIt is possible to restrict \\R to match only CR, LF, or CRLF (instead of the complete set of\nUnicode line endings) by setting the option PCREBSRANYCRLF either at compile time or when\nthe pattern is matched. (BSR is an abbrevation for \"backslash R\".) This can be made the de‐\nfault when PCRE is built; if this is the case, the other behaviour can be requested via the\nPCREBSRUNICODE option. It is also possible to specify these settings by starting a pattern\nstring with one of the following sequences:\n\n(*BSRANYCRLF) CR, LF, or CRLF only\n(*BSRUNICODE) any Unicode newline sequence\n\nThese override the default and the options given to the compiling function, but they can\nthemselves be overridden by options given to a matching function. Note that these special\nsettings, which are not Perl-compatible, are recognized only at the very start of a pattern,\nand that they must be in upper case. If more than one of them is present, the last one is\nused. They can be combined with a change of newline convention; for example, a pattern can\nstart with:\n\n(*ANY)(*BSRANYCRLF)\n\nThey can also be combined with the (*UTF8), (*UTF16), (*UTF32), (*UTF) or (*UCP) special se‐\nquences. Inside a character class, \\R is treated as an unrecognized escape sequence, and so\nmatches the letter \"R\" by default, but causes an error if PCREEXTRA is set.\n" }, { "name": "Unicode character properties", "content": "When PCRE is built with Unicode character property support, three additional escape sequences\nthat match characters with specific properties are available. When in 8-bit non-UTF-8 mode,\nthese sequences are of course limited to testing characters whose codepoints are less than\n256, but they do work in this mode. The extra escape sequences are:\n\n\\p{xx} a character with the xx property\n\\P{xx} a character without the xx property\n\\X a Unicode extended grapheme cluster\n\nThe property names represented by xx above are limited to the Unicode script names, the gen‐\neral category properties, \"Any\", which matches any character (including newline), and some\nspecial PCRE properties (described in the next section). Other Perl properties such as \"In‐\nMusicalSymbols\" are not currently supported by PCRE. Note that \\P{Any} does not match any\ncharacters, so always causes a match failure.\n\nSets of Unicode characters are defined as belonging to certain scripts. A character from one\nof these sets can be matched using a script name. For example:\n\n\\p{Greek}\n\\P{Han}\n\nThose that are not part of an identified script are lumped together as \"Common\". The current\nlist of scripts is:\n\nArabic, Armenian, Avestan, Balinese, Bamum, BassaVah, Batak, Bengali, Bopomofo, Brahmi,\nBraille, Buginese, Buhid, CanadianAboriginal, Carian, CaucasianAlbanian, Chakma, Cham,\nCherokee, Common, Coptic, Cuneiform, Cypriot, Cyrillic, Deseret, Devanagari, Duployan, Egyp‐\ntianHieroglyphs, Elbasan, Ethiopic, Georgian, Glagolitic, Gothic, Grantha, Greek, Gujarati,\nGurmukhi, Han, Hangul, Hanunoo, Hebrew, Hiragana, ImperialAramaic, Inherited, Inscrip‐\ntionalPahlavi, InscriptionalParthian, Javanese, Kaithi, Kannada, Katakana, KayahLi,\nKharoshthi, Khmer, Khojki, Khudawadi, Lao, Latin, Lepcha, Limbu, LinearA, LinearB, Lisu,\nLycian, Lydian, Mahajani, Malayalam, Mandaic, Manichaean, MeeteiMayek, MendeKikakui,\nMeroiticCursive, MeroiticHieroglyphs, Miao, Modi, Mongolian, Mro, Myanmar, Nabataean,\nNewTaiLue, Nko, Ogham, OlChiki, OldItalic, OldNorthArabian, OldPermic, OldPersian,\nOldSouthArabian, OldTurkic, Oriya, Osmanya, PahawhHmong, Palmyrene, PauCinHau,\nPhagsPa, Phoenician, PsalterPahlavi, Rejang, Runic, Samaritan, Saurashtra, Sharada, Sha‐\nvian, Siddham, Sinhala, SoraSompeng, Sundanese, SylotiNagri, Syriac, Tagalog, Tagbanwa,\nTaiLe, TaiTham, TaiViet, Takri, Tamil, Telugu, Thaana, Thai, Tibetan, Tifinagh, Tirhuta,\nUgaritic, Vai, WarangCiti, Yi.\n\nEach character has exactly one Unicode general category property, specified by a two-letter\nabbreviation. For compatibility with Perl, negation can be specified by including a circum‐\nflex between the opening brace and the property name. For example, \\p{^Lu} is the same as\n\\P{Lu}.\n\nIf only one letter is specified with \\p or \\P, it includes all the general category proper‐\nties that start with that letter. In this case, in the absence of negation, the curly brack‐\nets in the escape sequence are optional; these two examples have the same effect:\n\n\\p{L}\n\\pL\n\nThe following general category property codes are supported:\n\nC Other\nCc Control\nCf Format\nCn Unassigned\nCo Private use\nCs Surrogate\n\nL Letter\nLl Lower case letter\nLm Modifier letter\nLo Other letter\nLt Title case letter\nLu Upper case letter\n\nM Mark\nMc Spacing mark\nMe Enclosing mark\nMn Non-spacing mark\n\nN Number\nNd Decimal number\nNl Letter number\nNo Other number\n\nP Punctuation\nPc Connector punctuation\nPd Dash punctuation\nPe Close punctuation\nPf Final punctuation\nPi Initial punctuation\nPo Other punctuation\nPs Open punctuation\n\nS Symbol\nSc Currency symbol\nSk Modifier symbol\nSm Mathematical symbol\nSo Other symbol\n\nZ Separator\nZl Line separator\nZp Paragraph separator\nZs Space separator\n\nThe special property L& is also supported: it matches a character that has the Lu, Ll, or Lt\nproperty, in other words, a letter that is not classified as a modifier or \"other\".\n\nThe Cs (Surrogate) property applies only to characters in the range U+D800 to U+DFFF. Such\ncharacters are not valid in Unicode strings and so cannot be tested by PCRE, unless UTF va‐\nlidity checking has been turned off (see the discussion of PCRENOUTF8CHECK,\nPCRENOUTF16CHECK and PCRENOUTF32CHECK in the pcreapi page). Perl does not support the\nCs property.\n\nThe long synonyms for property names that Perl supports (such as \\p{Letter}) are not sup‐\nported by PCRE, nor is it permitted to prefix any of these properties with \"Is\".\n\nNo character that is in the Unicode table has the Cn (unassigned) property. Instead, this\nproperty is assumed for any code point that is not in the Unicode table.\n\nSpecifying caseless matching does not affect these escape sequences. For example, \\p{Lu} al‐\nways matches only upper case letters. This is different from the behaviour of current ver‐\nsions of Perl.\n\nMatching characters by Unicode property is not fast, because PCRE has to do a multistage ta‐\nble lookup in order to find a character's property. That is why the traditional escape se‐\nquences such as \\d and \\w do not use Unicode properties in PCRE by default, though you can\nmake them do so by setting the PCREUCP option or by starting the pattern with (*UCP).\n" }, { "name": "Extended grapheme clusters", "content": "The \\X escape matches any number of Unicode characters that form an \"extended grapheme clus‐\nter\", and treats the sequence as an atomic group (see below). Up to and including release\n8.31, PCRE matched an earlier, simpler definition that was equivalent to\n\n(?>\\PM\\pM*)\n\nThat is, it matched a character without the \"mark\" property, followed by zero or more charac‐\nters with the \"mark\" property. Characters with the \"mark\" property are typically non-spacing\naccents that affect the preceding character.\n\nThis simple definition was extended in Unicode to include more complicated kinds of composite\ncharacter by giving each character a grapheme breaking property, and creating rules that use\nthese properties to define the boundaries of extended grapheme clusters. In releases of PCRE\nlater than 8.31, \\X matches one of these clusters.\n\n\\X always matches at least one character. Then it decides whether to add additional charac‐\nters according to the following rules for ending a cluster:\n\n1. End at the end of the subject string.\n\n2. Do not end between CR and LF; otherwise end after any control character.\n\n3. Do not break Hangul (a Korean script) syllable sequences. Hangul characters are of five\ntypes: L, V, T, LV, and LVT. An L character may be followed by an L, V, LV, or LVT character;\nan LV or V character may be followed by a V or T character; an LVT or T character may be\nfollwed only by a T character.\n\n4. Do not end before extending characters or spacing marks. Characters with the \"mark\" prop‐\nerty always have the \"extend\" grapheme breaking property.\n\n5. Do not end after prepend characters.\n\n6. Otherwise, end the cluster.\n" }, { "name": "PCRE's additional properties", "content": "As well as the standard Unicode properties described above, PCRE supports four more that make\nit possible to convert traditional escape sequences such as \\w and \\s to use Unicode proper‐\nties. PCRE uses these non-standard, non-Perl properties internally when PCREUCP is set. How‐\never, they may also be used explicitly. These properties are:\n\nXan Any alphanumeric character\nXps Any POSIX space character\nXsp Any Perl space character\nXwd Any Perl \"word\" character\n\nXan matches characters that have either the L (letter) or the N (number) property. Xps\nmatches the characters tab, linefeed, vertical tab, form feed, or carriage return, and any\nother character that has the Z (separator) property. Xsp is the same as Xps; it used to ex‐\nclude vertical tab, for Perl compatibility, but Perl changed, and so PCRE followed at release\n8.34. Xwd matches the same characters as Xan, plus underscore.\n\nThere is another non-standard property, Xuc, which matches any character that can be repre‐\nsented by a Universal Character Name in C++ and other programming languages. These are the\ncharacters $, @, ` (grave accent), and all characters with Unicode code points greater than\nor equal to U+00A0, except for the surrogates U+D800 to U+DFFF. Note that most base (ASCII)\ncharacters are excluded. (Universal Character Names are of the form \\uHHHH or \\UHHHHHHHH\nwhere H is a hexadecimal digit. Note that the Xuc property does not match these sequences but\nthe characters that they represent.)\n" }, { "name": "Resetting the match start", "content": "The escape sequence \\K causes any previously matched characters not to be included in the fi‐\nnal matched sequence. For example, the pattern:\n\nfoo\\Kbar\n\nmatches \"foobar\", but reports that it has matched \"bar\". This feature is similar to a lookbe‐\nhind assertion (described below). However, in this case, the part of the subject before the\nreal match does not have to be of fixed length, as lookbehind assertions do. The use of \\K\ndoes not interfere with the setting of captured substrings. For example, when the pattern\n\n(foo)\\Kbar\n\nmatches \"foobar\", the first substring is still set to \"foo\".\n\nPerl documents that the use of \\K within assertions is \"not well defined\". In PCRE, \\K is\nacted upon when it occurs inside positive assertions, but is ignored in negative assertions.\nNote that when a pattern such as (?=ab\\K) matches, the reported start of the match can be\ngreater than the end of the match.\n" }, { "name": "Simple assertions", "content": "The final use of backslash is for certain simple assertions. An assertion specifies a condi‐\ntion that has to be met at a particular point in a match, without consuming any characters\nfrom the subject string. The use of subpatterns for more complicated assertions is described\nbelow. The backslashed assertions are:\n\n\\b matches at a word boundary\n\\B matches when not at a word boundary\n\\A matches at the start of the subject\n\\Z matches at the end of the subject\nalso matches before a newline at the end of the subject\n\\z matches only at the end of the subject\n\\G matches at the first matching position in the subject\n\nInside a character class, \\b has a different meaning; it matches the backspace character. If\nany other of these assertions appears in a character class, by default it matches the corre‐\nsponding literal character (for example, \\B matches the letter B). However, if the PCREEXTRA\noption is set, an \"invalid escape sequence\" error is generated instead.\n\nA word boundary is a position in the subject string where the current character and the pre‐\nvious character do not both match \\w or \\W (i.e. one matches \\w and the other matches \\W), or\nthe start or end of the string if the first or last character matches \\w, respectively. In a\nUTF mode, the meanings of \\w and \\W can be changed by setting the PCREUCP option. When this\nis done, it also affects \\b and \\B. Neither PCRE nor Perl has a separate \"start of word\" or\n\"end of word\" metasequence. However, whatever follows \\b normally determines which it is. For\nexample, the fragment \\ba matches \"a\" at the start of a word.\n\nThe \\A, \\Z, and \\z assertions differ from the traditional circumflex and dollar (described in\nthe next section) in that they only ever match at the very start and end of the subject\nstring, whatever options are set. Thus, they are independent of multiline mode. These three\nassertions are not affected by the PCRENOTBOL or PCRENOTEOL options, which affect only the\nbehaviour of the circumflex and dollar metacharacters. However, if the startoffset argument\nof pcreexec() is non-zero, indicating that matching is to start at a point other than the\nbeginning of the subject, \\A can never match. The difference between \\Z and \\z is that \\Z\nmatches before a newline at the end of the string as well as at the very end, whereas \\z\nmatches only at the end.\n\nThe \\G assertion is true only when the current matching position is at the start point of the\nmatch, as specified by the startoffset argument of pcreexec(). It differs from \\A when the\nvalue of startoffset is non-zero. By calling pcreexec() multiple times with appropriate ar‐\nguments, you can mimic Perl's /g option, and it is in this kind of implementation where \\G\ncan be useful.\n\nNote, however, that PCRE's interpretation of \\G, as the start of the current match, is subtly\ndifferent from Perl's, which defines it as the end of the previous match. In Perl, these can\nbe different when the previously matched string was empty. Because PCRE does just one match\nat a time, it cannot reproduce this behaviour.\n\nIf all the alternatives of a pattern begin with \\G, the expression is anchored to the start‐\ning match position, and the \"anchored\" flag is set in the compiled regular expression.\n" } ] }, "CIRCUMFLEX AND DOLLAR": { "content": "The circumflex and dollar metacharacters are zero-width assertions. That is, they test for a\nparticular condition being true without consuming any characters from the subject string.\n\nOutside a character class, in the default matching mode, the circumflex character is an as‐\nsertion that is true only if the current matching point is at the start of the subject\nstring. If the startoffset argument of pcreexec() is non-zero, circumflex can never match if\nthe PCREMULTILINE option is unset. Inside a character class, circumflex has an entirely dif‐\nferent meaning (see below).\n\nCircumflex need not be the first character of the pattern if a number of alternatives are in‐\nvolved, but it should be the first thing in each alternative in which it appears if the pat‐\ntern is ever to match that branch. If all possible alternatives start with a circumflex, that\nis, if the pattern is constrained to match only at the start of the subject, it is said to be\nan \"anchored\" pattern. (There are also other constructs that can cause a pattern to be an‐\nchored.)\n\nThe dollar character is an assertion that is true only if the current matching point is at\nthe end of the subject string, or immediately before a newline at the end of the string (by\ndefault). Note, however, that it does not actually match the newline. Dollar need not be the\nlast character of the pattern if a number of alternatives are involved, but it should be the\nlast item in any branch in which it appears. Dollar has no special meaning in a character\nclass.\n\nThe meaning of dollar can be changed so that it matches only at the very end of the string,\nby setting the PCREDOLLARENDONLY option at compile time. This does not affect the \\Z asser‐\ntion.\n\nThe meanings of the circumflex and dollar characters are changed if the PCREMULTILINE option\nis set. When this is the case, a circumflex matches immediately after internal newlines as\nwell as at the start of the subject string. It does not match after a newline that ends the\nstring. A dollar matches before any newlines in the string, as well as at the very end, when\nPCREMULTILINE is set. When newline is specified as the two-character sequence CRLF, isolated\nCR and LF characters do not indicate newlines.\n\nFor example, the pattern /^abc$/ matches the subject string \"def\\nabc\" (where \\n represents a\nnewline) in multiline mode, but not otherwise. Consequently, patterns that are anchored in\nsingle line mode because all branches start with ^ are not anchored in multiline mode, and a\nmatch for circumflex is possible when the startoffset argument of pcreexec() is non-zero.\nThe PCREDOLLARENDONLY option is ignored if PCREMULTILINE is set.\n\nNote that the sequences \\A, \\Z, and \\z can be used to match the start and end of the subject\nin both modes, and if all branches of a pattern start with \\A it is always anchored, whether\nor not PCREMULTILINE is set.\n", "subsections": [ { "name": "FULL STOP (PERIOD, DOT) AND \\N", "content": "Outside a character class, a dot in the pattern matches any one character in the subject\nstring except (by default) a character that signifies the end of a line.\n\nWhen a line ending is defined as a single character, dot never matches that character; when\nthe two-character sequence CRLF is used, dot does not match CR if it is immediately followed\nby LF, but otherwise it matches all characters (including isolated CRs and LFs). When any\nUnicode line endings are being recognized, dot does not match CR or LF or any of the other\nline ending characters.\n\nThe behaviour of dot with regard to newlines can be changed. If the PCREDOTALL option is\nset, a dot matches any one character, without exception. If the two-character sequence CRLF\nis present in the subject string, it takes two dots to match it.\n\nThe handling of dot is entirely independent of the handling of circumflex and dollar, the\nonly relationship being that they both involve newlines. Dot has no special meaning in a\ncharacter class.\n\nThe escape sequence \\N behaves like a dot, except that it is not affected by the PCREDOTALL\noption. In other words, it matches any character except one that signifies the end of a line.\nPerl also uses \\N to match characters by name; PCRE does not support this.\n" } ] }, "MATCHING A SINGLE DATA UNIT": { "content": "Outside a character class, the escape sequence \\C matches any one data unit, whether or not a\nUTF mode is set. In the 8-bit library, one data unit is one byte; in the 16-bit library it is\na 16-bit unit; in the 32-bit library it is a 32-bit unit. Unlike a dot, \\C always matches\nline-ending characters. The feature is provided in Perl in order to match individual bytes in\nUTF-8 mode, but it is unclear how it can usefully be used. Because \\C breaks up characters\ninto individual data units, matching one unit with \\C in a UTF mode means that the rest of\nthe string may start with a malformed UTF character. This has undefined results, because PCRE\nassumes that it is dealing with valid UTF strings (and by default it checks this at the start\nof processing unless the PCRENOUTF8CHECK, PCRENOUTF16CHECK or PCRENOUTF32CHECK op‐\ntion is used).\n\nPCRE does not allow \\C to appear in lookbehind assertions (described below) in a UTF mode,\nbecause this would make it impossible to calculate the length of the lookbehind.\n\nIn general, the \\C escape sequence is best avoided. However, one way of using it that avoids\nthe problem of malformed UTF characters is to use a lookahead to check the length of the next\ncharacter, as in this pattern, which could be used with a UTF-8 string (ignore white space\nand line breaks):\n\n(?| (?=[\\x00-\\x7f])(\\C) |\n(?=[\\x80-\\x{7ff}])(\\C)(\\C) |\n(?=[\\x{800}-\\x{ffff}])(\\C)(\\C)(\\C) |\n(?=[\\x{10000}-\\x{1fffff}])(\\C)(\\C)(\\C)(\\C))\n\nA group that starts with (?| resets the capturing parentheses numbers in each alternative\n(see \"Duplicate Subpattern Numbers\" below). The assertions at the start of each branch check\nthe next UTF-8 character for values whose encoding uses 1, 2, 3, or 4 bytes, respectively.\nThe character's individual bytes are then captured by the appropriate number of groups.\n", "subsections": [] }, "SQUARE BRACKETS AND CHARACTER CLASSES": { "content": "An opening square bracket introduces a character class, terminated by a closing square\nbracket. A closing square bracket on its own is not special by default. However, if the\nPCREJAVASCRIPTCOMPAT option is set, a lone closing square bracket causes a compile-time er‐\nror. If a closing square bracket is required as a member of the class, it should be the first\ndata character in the class (after an initial circumflex, if present) or escaped with a back‐\nslash.\n\nA character class matches a single character in the subject. In a UTF mode, the character may\nbe more than one data unit long. A matched character must be in the set of characters defined\nby the class, unless the first character in the class definition is a circumflex, in which\ncase the subject character must not be in the set defined by the class. If a circumflex is\nactually required as a member of the class, ensure it is not the first character, or escape\nit with a backslash.\n\nFor example, the character class [aeiou] matches any lower case vowel, while [^aeiou] matches\nany character that is not a lower case vowel. Note that a circumflex is just a convenient no‐\ntation for specifying the characters that are in the class by enumerating those that are not.\nA class that starts with a circumflex is not an assertion; it still consumes a character from\nthe subject string, and therefore it fails if the current pointer is at the end of the\nstring.\n\nIn UTF-8 (UTF-16, UTF-32) mode, characters with values greater than 255 (0xffff) can be in‐\ncluded in a class as a literal string of data units, or by using the \\x{ escaping mechanism.\n\nWhen caseless matching is set, any letters in a class represent both their upper case and\nlower case versions, so for example, a caseless [aeiou] matches \"A\" as well as \"a\", and a\ncaseless [^aeiou] does not match \"A\", whereas a caseful version would. In a UTF mode, PCRE\nalways understands the concept of case for characters whose values are less than 128, so\ncaseless matching is always possible. For characters with higher values, the concept of case\nis supported if PCRE is compiled with Unicode property support, but not otherwise. If you\nwant to use caseless matching in a UTF mode for characters 128 and above, you must ensure\nthat PCRE is compiled with Unicode property support as well as with UTF support.\n\nCharacters that might indicate line breaks are never treated in any special way when matching\ncharacter classes, whatever line-ending sequence is in use, and whatever setting of the\nPCREDOTALL and PCREMULTILINE options is used. A class such as [^a] always matches one of\nthese characters.\n\nThe minus (hyphen) character can be used to specify a range of characters in a character\nclass. For example, [d-m] matches any letter between d and m, inclusive. If a minus character\nis required in a class, it must be escaped with a backslash or appear in a position where it\ncannot be interpreted as indicating a range, typically as the first or last character in the\nclass, or immediately after a range. For example, [b-d-z] matches letters in the range b to\nd, a hyphen character, or z.\n\nIt is not possible to have the literal character \"]\" as the end character of a range. A pat‐\ntern such as [W-]46] is interpreted as a class of two characters (\"W\" and \"-\") followed by a\nliteral string \"46]\", so it would match \"W46]\" or \"-46]\". However, if the \"]\" is escaped with\na backslash it is interpreted as the end of range, so [W-\\]46] is interpreted as a class con‐\ntaining a range followed by two other characters. The octal or hexadecimal representation of\n\"]\" can also be used to end a range.\n\nAn error is generated if a POSIX character class (see below) or an escape sequence other than\none that defines a single character appears at a point where a range ending character is ex‐\npected. For example, [z-\\xff] is valid, but [A-\\d] and [A-[:digit:]] are not.\n\nRanges operate in the collating sequence of character values. They can also be used for char‐\nacters specified numerically, for example [\\000-\\037]. Ranges can include any characters that\nare valid for the current mode.\n\nIf a range that includes letters is used when caseless matching is set, it matches the let‐\nters in either case. For example, [W-c] is equivalent to [][\\\\^`wxyzabc], matched case‐\nlessly, and in a non-UTF mode, if character tables for a French locale are in use,\n[\\xc8-\\xcb] matches accented E characters in both cases. In UTF modes, PCRE supports the con‐\ncept of case for characters with values greater than 128 only when it is compiled with Uni‐\ncode property support.\n\nThe character escape sequences \\d, \\D, \\h, \\H, \\p, \\P, \\s, \\S, \\v, \\V, \\w, and \\W may appear\nin a character class, and add the characters that they match to the class. For example,\n[\\dABCDEF] matches any hexadecimal digit. In UTF modes, the PCREUCP option affects the mean‐\nings of \\d, \\s, \\w and their upper case partners, just as it does when they appear outside a\ncharacter class, as described in the section entitled \"Generic character types\" above. The\nescape sequence \\b has a different meaning inside a character class; it matches the backspace\ncharacter. The sequences \\B, \\N, \\R, and \\X are not special inside a character class. Like\nany other unrecognized escape sequences, they are treated as the literal characters \"B\", \"N\",\n\"R\", and \"X\" by default, but cause an error if the PCREEXTRA option is set.\n\nA circumflex can conveniently be used with the upper case character types to specify a more\nrestricted set of characters than the matching lower case type. For example, the class\n[^\\W] matches any letter or digit, but not underscore, whereas [\\w] includes underscore. A\npositive character class should be read as \"something OR something OR ...\" and a negative\nclass as \"NOT something AND NOT something AND NOT ...\".\n\nThe only metacharacters that are recognized in character classes are backslash, hyphen (only\nwhere it can be interpreted as specifying a range), circumflex (only at the start), opening\nsquare bracket (only when it can be interpreted as introducing a POSIX class name, or for a\nspecial compatibility feature - see the next two sections), and the terminating closing\nsquare bracket. However, escaping other non-alphanumeric characters does no harm.\n", "subsections": [] }, "POSIX CHARACTER CLASSES": { "content": "Perl supports the POSIX notation for character classes. This uses names enclosed by [: and :]\nwithin the enclosing square brackets. PCRE also supports this notation. For example,\n\n[01[:alpha:]%]\n\nmatches \"0\", \"1\", any alphabetic character, or \"%\". The supported class names are:\n\nalnum letters and digits\nalpha letters\nascii character codes 0 - 127\nblank space or tab only\ncntrl control characters\ndigit decimal digits (same as \\d)\ngraph printing characters, excluding space\nlower lower case letters\nprint printing characters, including space\npunct printing characters, excluding letters and digits and space\nspace white space (the same as \\s from PCRE 8.34)\nupper upper case letters\nword \"word\" characters (same as \\w)\nxdigit hexadecimal digits\n\nThe default \"space\" characters are HT (9), LF (10), VT (11), FF (12), CR (13), and space\n(32). If locale-specific matching is taking place, the list of space characters may be dif‐\nferent; there may be fewer or more of them. \"Space\" used to be different to \\s, which did not\ninclude VT, for Perl compatibility. However, Perl changed at release 5.18, and PCRE followed\nat release 8.34. \"Space\" and \\s now match the same set of characters.\n\nThe name \"word\" is a Perl extension, and \"blank\" is a GNU extension from Perl 5.8. Another\nPerl extension is negation, which is indicated by a ^ character after the colon. For example,\n\n[12[:^digit:]]\n\nmatches \"1\", \"2\", or any non-digit. PCRE (and Perl) also recognize the POSIX syntax [.ch.]\nand [=ch=] where \"ch\" is a \"collating element\", but these are not supported, and an error is\ngiven if they are encountered.\n\nBy default, characters with values greater than 128 do not match any of the POSIX character\nclasses. However, if the PCREUCP option is passed to pcrecompile(), some of the classes are\nchanged so that Unicode character properties are used. This is achieved by replacing certain\nPOSIX classes by other sequences, as follows:\n\n[:alnum:] becomes \\p{Xan}\n[:alpha:] becomes \\p{L}\n[:blank:] becomes \\h\n[:digit:] becomes \\p{Nd}\n[:lower:] becomes \\p{Ll}\n[:space:] becomes \\p{Xps}\n[:upper:] becomes \\p{Lu}\n[:word:] becomes \\p{Xwd}\n\nNegated versions, such as [:^alpha:] use \\P instead of \\p. Three other POSIX classes are han‐\ndled specially in UCP mode:\n\n[:graph:] This matches characters that have glyphs that mark the page when printed. In Uni‐\ncode property terms, it matches all characters with the L, M, N, P, S, or Cf prop‐\nerties, except for:\n\nU+061C Arabic Letter Mark\nU+180E Mongolian Vowel Separator\nU+2066 - U+2069 Various \"isolate\"s\n\n\n[:print:] This matches the same characters as [:graph:] plus space characters that are not\ncontrols, that is, characters with the Zs property.\n\n[:punct:] This matches all characters that have the Unicode P (punctuation) property, plus\nthose characters whose code points are less than 128 that have the S (Symbol) prop‐\nerty.\n\nThe other POSIX classes are unchanged, and match only characters with code points less than\n128.\n", "subsections": [] }, "COMPATIBILITY FEATURE FOR WORD BOUNDARIES": { "content": "In the POSIX.2 compliant library that was included in 4.4BSD Unix, the ugly syntax [[:<:]]\nand [[:>:]] is used for matching \"start of word\" and \"end of word\". PCRE treats these items\nas follows:\n\n[[:<:]] is converted to \\b(?=\\w)\n[[:>:]] is converted to \\b(?<=\\w)\n\nOnly these exact character sequences are recognized. A sequence such as [a[:<:]b] provokes\nerror for an unrecognized POSIX class name. This support is not compatible with Perl. It is\nprovided to help migrations from other environments, and is best not used in any new pat‐\nterns. Note that \\b matches at the start and the end of a word (see \"Simple assertions\"\nabove), and in a Perl-style pattern the preceding or following character normally shows which\nis wanted, without the need for the assertions that are used above in order to give exactly\nthe POSIX behaviour.\n", "subsections": [] }, "VERTICAL BAR": { "content": "Vertical bar characters are used to separate alternative patterns. For example, the pattern\n\ngilbert|sullivan\n\nmatches either \"gilbert\" or \"sullivan\". Any number of alternatives may appear, and an empty\nalternative is permitted (matching the empty string). The matching process tries each alter‐\nnative in turn, from left to right, and the first one that succeeds is used. If the alterna‐\ntives are within a subpattern (defined below), \"succeeds\" means matching the rest of the main\npattern as well as the alternative in the subpattern.\n", "subsections": [] }, "INTERNAL OPTION SETTING": { "content": "The settings of the PCRECASELESS, PCREMULTILINE, PCREDOTALL, and PCREEXTENDED options\n(which are Perl-compatible) can be changed from within the pattern by a sequence of Perl op‐\ntion letters enclosed between \"(?\" and \")\". The option letters are\n\ni for PCRECASELESS\nm for PCREMULTILINE\ns for PCREDOTALL\nx for PCREEXTENDED\n\nFor example, (?im) sets caseless, multiline matching. It is also possible to unset these op‐\ntions by preceding the letter with a hyphen, and a combined setting and unsetting such as\n(?im-sx), which sets PCRECASELESS and PCREMULTILINE while unsetting PCREDOTALL and\nPCREEXTENDED, is also permitted. If a letter appears both before and after the hyphen, the\noption is unset.\n\nThe PCRE-specific options PCREDUPNAMES, PCREUNGREEDY, and PCREEXTRA can be changed in the\nsame way as the Perl-compatible options by using the characters J, U and X respectively.\n\nWhen one of these option changes occurs at top level (that is, not inside subpattern paren‐\ntheses), the change applies to the remainder of the pattern that follows. If the change is\nplaced right at the start of a pattern, PCRE extracts it into the global options (and it will\ntherefore show up in data extracted by the pcrefullinfo() function).\n\nAn option change within a subpattern (see below for a description of subpatterns) affects\nonly that part of the subpattern that follows it, so\n\n(a(?i)b)c\n\nmatches abc and aBc and no other strings (assuming PCRECASELESS is not used). By this\nmeans, options can be made to have different settings in different parts of the pattern. Any\nchanges made in one alternative do carry on into subsequent branches within the same subpat‐\ntern. For example,\n\n(a(?i)b|c)\n\nmatches \"ab\", \"aB\", \"c\", and \"C\", even though when matching \"C\" the first branch is abandoned\nbefore the option setting. This is because the effects of option settings happen at compile\ntime. There would be some very weird behaviour otherwise.\n\nNote: There are other PCRE-specific options that can be set by the application when the com‐\npiling or matching functions are called. In some cases the pattern can contain special lead‐\ning sequences such as (*CRLF) to override what the application has set or what has been de‐\nfaulted. Details are given in the section entitled \"Newline sequences\" above. There are also\nthe (*UTF8), (*UTF16),(*UTF32), and (*UCP) leading sequences that can be used to set UTF and\nUnicode property modes; they are equivalent to setting the PCREUTF8, PCREUTF16, PCREUTF32\nand the PCREUCP options, respectively. The (*UTF) sequence is a generic version that can be\nused with any of the libraries. However, the application can set the PCRENEVERUTF option,\nwhich locks out the use of the (*UTF) sequences.\n", "subsections": [] }, "SUBPATTERNS": { "content": "Subpatterns are delimited by parentheses (round brackets), which can be nested. Turning part\nof a pattern into a subpattern does two things:\n\n1. It localizes a set of alternatives. For example, the pattern\n\ncat(aract|erpillar|)\n\nmatches \"cataract\", \"caterpillar\", or \"cat\". Without the parentheses, it would match\n\"cataract\", \"erpillar\" or an empty string.\n\n2. It sets up the subpattern as a capturing subpattern. This means that, when the whole pat‐\ntern matches, that portion of the subject string that matched the subpattern is passed back\nto the caller via the ovector argument of the matching function. (This applies only to the\ntraditional matching functions; the DFA matching functions do not support capturing.)\n\nOpening parentheses are counted from left to right (starting from 1) to obtain numbers for\nthe capturing subpatterns. For example, if the string \"the red king\" is matched against the\npattern\n\nthe ((red|white) (king|queen))\n\nthe captured substrings are \"red king\", \"red\", and \"king\", and are numbered 1, 2, and 3, re‐\nspectively.\n\nThe fact that plain parentheses fulfil two functions is not always helpful. There are often\ntimes when a grouping subpattern is required without a capturing requirement. If an opening\nparenthesis is followed by a question mark and a colon, the subpattern does not do any cap‐\nturing, and is not counted when computing the number of any subsequent capturing subpatterns.\nFor example, if the string \"the white queen\" is matched against the pattern\n\nthe ((?:red|white) (king|queen))\n\nthe captured substrings are \"white queen\" and \"queen\", and are numbered 1 and 2. The maximum\nnumber of capturing subpatterns is 65535.\n\nAs a convenient shorthand, if any option settings are required at the start of a non-captur‐\ning subpattern, the option letters may appear between the \"?\" and the \":\". Thus the two pat‐\nterns\n\n(?i:saturday|sunday)\n(?:(?i)saturday|sunday)\n\nmatch exactly the same set of strings. Because alternative branches are tried from left to\nright, and options are not reset until the end of the subpattern is reached, an option set‐\nting in one branch does affect subsequent branches, so the above patterns match \"SUNDAY\" as\nwell as \"Saturday\".\n", "subsections": [] }, "DUPLICATE SUBPATTERN NUMBERS": { "content": "Perl 5.10 introduced a feature whereby each alternative in a subpattern uses the same numbers\nfor its capturing parentheses. Such a subpattern starts with (?| and is itself a non-captur‐\ning subpattern. For example, consider this pattern:\n\n(?|(Sat)ur|(Sun))day\n\nBecause the two alternatives are inside a (?| group, both sets of capturing parentheses are\nnumbered one. Thus, when the pattern matches, you can look at captured substring number one,\nwhichever alternative matched. This construct is useful when you want to capture part, but\nnot all, of one of a number of alternatives. Inside a (?| group, parentheses are numbered as\nusual, but the number is reset at the start of each branch. The numbers of any capturing\nparentheses that follow the subpattern start after the highest number used in any branch. The\nfollowing example is taken from the Perl documentation. The numbers underneath show in which\nbuffer the captured content will be stored.\n\n# before ---------------branch-reset----------- after\n/ ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x\n# 1 2 2 3 2 3 4\n\nA back reference to a numbered subpattern uses the most recent value that is set for that\nnumber by any subpattern. The following pattern matches \"abcabc\" or \"defdef\":\n\n/(?|(abc)|(def))\\1/\n\nIn contrast, a subroutine call to a numbered subpattern always refers to the first one in the\npattern with the given number. The following pattern matches \"abcabc\" or \"defabc\":\n\n/(?|(abc)|(def))(?1)/\n\nIf a condition test for a subpattern's having matched refers to a non-unique number, the test\nis true if any of the subpatterns of that number have matched.\n\nAn alternative approach to using this \"branch reset\" feature is to use duplicate named sub‐\npatterns, as described in the next section.\n", "subsections": [] }, "NAMED SUBPATTERNS": { "content": "Identifying capturing parentheses by number is simple, but it can be very hard to keep track\nof the numbers in complicated regular expressions. Furthermore, if an expression is modified,\nthe numbers may change. To help with this difficulty, PCRE supports the naming of subpat‐\nterns. This feature was not added to Perl until release 5.10. Python had the feature earlier,\nand PCRE introduced it at release 4.0, using the Python syntax. PCRE now supports both the\nPerl and the Python syntax. Perl allows identically numbered subpatterns to have different\nnames, but PCRE does not.\n\nIn PCRE, a subpattern can be named in one of three ways: (?...) or (?'name'...) as in\nPerl, or (?P...) as in Python. References to capturing parentheses from other parts of\nthe pattern, such as back references, recursion, and conditions, can be made by name as well\nas by number.\n\nNames consist of up to 32 alphanumeric characters and underscores, but must start with a non-\ndigit. Named capturing parentheses are still allocated numbers as well as names, exactly as\nif the names were not present. The PCRE API provides function calls for extracting the name-\nto-number translation table from a compiled pattern. There is also a convenience function for\nextracting a captured substring by name.\n\nBy default, a name must be unique within a pattern, but it is possible to relax this con‐\nstraint by setting the PCREDUPNAMES option at compile time. (Duplicate names are also always\npermitted for subpatterns with the same number, set up as described in the previous section.)\nDuplicate names can be useful for patterns where only one instance of the named parentheses\ncan match. Suppose you want to match the name of a weekday, either as a 3-letter abbreviation\nor as the full name, and in both cases you want to extract the abbreviation. This pattern\n(ignoring the line breaks) does the job:\n\n(?Mon|Fri|Sun)(?:day)?|\n(?Tue)(?:sday)?|\n(?Wed)(?:nesday)?|\n(?Thu)(?:rsday)?|\n(?Sat)(?:urday)?\n\nThere are five capturing substrings, but only one is ever set after a match. (An alternative\nway of solving this problem is to use a \"branch reset\" subpattern, as described in the previ‐\nous section.)\n\nThe convenience function for extracting the data by name returns the substring for the first\n(and in this example, the only) subpattern of that name that matched. This saves searching to\nfind which numbered subpattern it was.\n\nIf you make a back reference to a non-unique named subpattern from elsewhere in the pattern,\nthe subpatterns to which the name refers are checked in the order in which they appear in the\noverall pattern. The first one that is set is used for the reference. For example, this pat‐\ntern matches both \"foofoo\" and \"barbar\" but not \"foobar\" or \"barfoo\":\n\n(?:(?foo)|(?bar))\\k\n\n\nIf you make a subroutine call to a non-unique named subpattern, the one that corresponds to\nthe first occurrence of the name is used. In the absence of duplicate numbers (see the previ‐\nous section) this is the one with the lowest number.\n\nIf you use a named reference in a condition test (see the section about conditions below),\neither to check whether a subpattern has matched, or to check for recursion, all subpatterns\nwith the same name are tested. If the condition is true for any one of them, the overall con‐\ndition is true. This is the same behaviour as testing by number. For further details of the\ninterfaces for handling named subpatterns, see the pcreapi documentation.\n\nWarning: You cannot use different names to distinguish between two subpatterns with the same\nnumber because PCRE uses only the numbers when matching. For this reason, an error is given\nat compile time if different names are given to subpatterns with the same number. However,\nyou can always give the same name to subpatterns with the same number, even when PCREDUP‐\nNAMES is not set.\n", "subsections": [] }, "REPETITION": { "content": "Repetition is specified by quantifiers, which can follow any of the following items:\n\na literal data character\nthe dot metacharacter\nthe \\C escape sequence\nthe \\X escape sequence\nthe \\R escape sequence\nan escape such as \\d or \\pL that matches a single character\na character class\na back reference (see next section)\na parenthesized subpattern (including assertions)\na subroutine call to a subpattern (recursive or otherwise)\n\nThe general repetition quantifier specifies a minimum and maximum number of permitted\nmatches, by giving the two numbers in curly brackets (braces), separated by a comma. The num‐\nbers must be less than 65536, and the first must be less than or equal to the second. For ex‐\nample:\n\nz{2,4}\n\nmatches \"zz\", \"zzz\", or \"zzzz\". A closing brace on its own is not a special character. If the\nsecond number is omitted, but the comma is present, there is no upper limit; if the second\nnumber and the comma are both omitted, the quantifier specifies an exact number of required\nmatches. Thus\n\n[aeiou]{3,}\n\nmatches at least 3 successive vowels, but may match many more, while\n\n\\d{8}\n\nmatches exactly 8 digits. An opening curly bracket that appears in a position where a quanti‐\nfier is not allowed, or one that does not match the syntax of a quantifier, is taken as a\nliteral character. For example, {,6} is not a quantifier, but a literal string of four char‐\nacters.\n\nIn UTF modes, quantifiers apply to characters rather than to individual data units. Thus, for\nexample, \\x{100}{2} matches two characters, each of which is represented by a two-byte se‐\nquence in a UTF-8 string. Similarly, \\X{3} matches three Unicode extended grapheme clusters,\neach of which may be several data units long (and they may be of different lengths).\n\nThe quantifier {0} is permitted, causing the expression to behave as if the previous item and\nthe quantifier were not present. This may be useful for subpatterns that are referenced as\nsubroutines from elsewhere in the pattern (but see also the section entitled \"Defining sub‐\npatterns for use by reference only\" below). Items other than subpatterns that have a {0}\nquantifier are omitted from the compiled pattern.\n\nFor convenience, the three most common quantifiers have single-character abbreviations:\n\n* is equivalent to {0,}\n+ is equivalent to {1,}\n? is equivalent to {0,1}\n\nIt is possible to construct infinite loops by following a subpattern that can match no char‐\nacters with a quantifier that has no upper limit, for example:\n\n(a?)*\n\nEarlier versions of Perl and PCRE used to give an error at compile time for such patterns.\nHowever, because there are cases where this can be useful, such patterns are now accepted,\nbut if any repetition of the subpattern does in fact match no characters, the loop is\nforcibly broken.\n\nBy default, the quantifiers are \"greedy\", that is, they match as much as possible (up to the\nmaximum number of permitted times), without causing the rest of the pattern to fail. The\nclassic example of where this gives problems is in trying to match comments in C programs.\nThese appear between /* and */ and within the comment, individual * and / characters may ap‐\npear. An attempt to match C comments by applying the pattern\n\n/\\*.*\\*/\n\nto the string\n\n/* first comment */ not comment /* second comment */\n\nfails, because it matches the entire string owing to the greediness of the .* item.\n\nHowever, if a quantifier is followed by a question mark, it ceases to be greedy, and instead\nmatches the minimum number of times possible, so the pattern\n\n/\\*.*?\\*/\n\ndoes the right thing with the C comments. The meaning of the various quantifiers is not oth‐\nerwise changed, just the preferred number of matches. Do not confuse this use of question\nmark with its use as a quantifier in its own right. Because it has two uses, it can sometimes\nappear doubled, as in\n\n\\d??\\d\n\nwhich matches one digit by preference, but can match two if that is the only way the rest of\nthe pattern matches.\n\nIf the PCREUNGREEDY option is set (an option that is not available in Perl), the quantifiers\nare not greedy by default, but individual ones can be made greedy by following them with a\nquestion mark. In other words, it inverts the default behaviour.\n\nWhen a parenthesized subpattern is quantified with a minimum repeat count that is greater\nthan 1 or with a limited maximum, more memory is required for the compiled pattern, in pro‐\nportion to the size of the minimum or maximum.\n\nIf a pattern starts with .* or .{0,} and the PCREDOTALL option (equivalent to Perl's /s) is\nset, thus allowing the dot to match newlines, the pattern is implicitly anchored, because\nwhatever follows will be tried against every character position in the subject string, so\nthere is no point in retrying the overall match at any position after the first. PCRE nor‐\nmally treats such a pattern as though it were preceded by \\A.\n\nIn cases where it is known that the subject string contains no newlines, it is worth setting\nPCREDOTALL in order to obtain this optimization, or alternatively using ^ to indicate an‐\nchoring explicitly.\n\nHowever, there are some cases where the optimization cannot be used. When .* is inside cap‐\nturing parentheses that are the subject of a back reference elsewhere in the pattern, a match\nat the start may fail where a later one succeeds. Consider, for example:\n\n(.*)abc\\1\n\nIf the subject is \"xyz123abc123\" the match point is the fourth character. For this reason,\nsuch a pattern is not implicitly anchored.\n\nAnother case where implicit anchoring is not applied is when the leading .* is inside an\natomic group. Once again, a match at the start may fail where a later one succeeds. Consider\nthis pattern:\n\n(?>.*?a)b\n\nIt matches \"ab\" in the subject \"aab\". The use of the backtracking control verbs (*PRUNE) and\n(*SKIP) also disable this optimization.\n\nWhen a capturing subpattern is repeated, the value captured is the substring that matched the\nfinal iteration. For example, after\n\n(tweedle[dume]{3}\\s*)+\n\nhas matched \"tweedledum tweedledee\" the value of the captured substring is \"tweedledee\". How‐\never, if there are nested capturing subpatterns, the corresponding captured values may have\nbeen set in previous iterations. For example, after\n\n/(a|(b))+/\n\nmatches \"aba\" the value of the second captured substring is \"b\".\n", "subsections": [] }, "ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS": { "content": "With both maximizing (\"greedy\") and minimizing (\"ungreedy\" or \"lazy\") repetition, failure of\nwhat follows normally causes the repeated item to be re-evaluated to see if a different num‐\nber of repeats allows the rest of the pattern to match. Sometimes it is useful to prevent\nthis, either to change the nature of the match, or to cause it fail earlier than it otherwise\nmight, when the author of the pattern knows there is no point in carrying on.\n\nConsider, for example, the pattern \\d+foo when applied to the subject line\n\n123456bar\n\nAfter matching all 6 digits and then failing to match \"foo\", the normal action of the matcher\nis to try again with only 5 digits matching the \\d+ item, and then with 4, and so on, before\nultimately failing. \"Atomic grouping\" (a term taken from Jeffrey Friedl's book) provides the\nmeans for specifying that once a subpattern has matched, it is not to be re-evaluated in this\nway.\n\nIf we use atomic grouping for the previous example, the matcher gives up immediately on fail‐\ning to match \"foo\" the first time. The notation is a kind of special parenthesis, starting\nwith (?> as in this example:\n\n(?>\\d+)foo\n\nThis kind of parenthesis \"locks up\" the part of the pattern it contains once it has matched,\nand a failure further into the pattern is prevented from backtracking into it. Backtracking\npast it to previous items, however, works as normal.\n\nAn alternative description is that a subpattern of this type matches the string of characters\nthat an identical standalone pattern would match, if anchored at the current point in the\nsubject string.\n\nAtomic grouping subpatterns are not capturing subpatterns. Simple cases such as the above ex‐\nample can be thought of as a maximizing repeat that must swallow everything it can. So, while\nboth \\d+ and \\d+? are prepared to adjust the number of digits they match in order to make the\nrest of the pattern match, (?>\\d+) can only match an entire sequence of digits.\n\nAtomic groups in general can of course contain arbitrarily complicated subpatterns, and can\nbe nested. However, when the subpattern for an atomic group is just a single repeated item,\nas in the example above, a simpler notation, called a \"possessive quantifier\" can be used.\nThis consists of an additional + character following a quantifier. Using this notation, the\nprevious example can be rewritten as\n\n\\d++foo\n\nNote that a possessive quantifier can be used with an entire group, for example:\n\n(abc|xyz){2,3}+\n\nPossessive quantifiers are always greedy; the setting of the PCREUNGREEDY option is ignored.\nThey are a convenient notation for the simpler forms of atomic group. However, there is no\ndifference in the meaning of a possessive quantifier and the equivalent atomic group, though\nthere may be a performance difference; possessive quantifiers should be slightly faster.\n\nThe possessive quantifier syntax is an extension to the Perl 5.8 syntax. Jeffrey Friedl\noriginated the idea (and the name) in the first edition of his book. Mike McCloskey liked it,\nso implemented it when he built Sun's Java package, and PCRE copied it from there. It ulti‐\nmately found its way into Perl at release 5.10.\n\nPCRE has an optimization that automatically \"possessifies\" certain simple pattern constructs.\nFor example, the sequence A+B is treated as A++B because there is no point in backtracking\ninto a sequence of A's when B must follow.\n\nWhen a pattern contains an unlimited repeat inside a subpattern that can itself be repeated\nan unlimited number of times, the use of an atomic group is the only way to avoid some fail‐\ning matches taking a very long time indeed. The pattern\n\n(\\D+|<\\d+>)*[!?]\n\nmatches an unlimited number of substrings that either consist of non-digits, or digits en‐\nclosed in <>, followed by either ! or ?. When it matches, it runs quickly. However, if it is\napplied to\n\naaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n\nit takes a long time before reporting failure. This is because the string can be divided be‐\ntween the internal \\D+ repeat and the external * repeat in a large number of ways, and all\nhave to be tried. (The example uses [!?] rather than a single character at the end, because\nboth PCRE and Perl have an optimization that allows for fast failure when a single character\nis used. They remember the last single character that is required for a match, and fail early\nif it is not present in the string.) If the pattern is changed so that it uses an atomic\ngroup, like this:\n\n((?>\\D+)|<\\d+>)*[!?]\n\nsequences of non-digits cannot be broken, and failure happens quickly.\n", "subsections": [] }, "BACK REFERENCES": { "content": "Outside a character class, a backslash followed by a digit greater than 0 (and possibly fur‐\nther digits) is a back reference to a capturing subpattern earlier (that is, to its left) in\nthe pattern, provided there have been that many previous capturing left parentheses.\n\nHowever, if the decimal number following the backslash is less than 10, it is always taken as\na back reference, and causes an error only if there are not that many capturing left paren‐\ntheses in the entire pattern. In other words, the parentheses that are referenced need not be\nto the left of the reference for numbers less than 10. A \"forward back reference\" of this\ntype can make sense when a repetition is involved and the subpattern to the right has partic‐\nipated in an earlier iteration.\n\nIt is not possible to have a numerical \"forward back reference\" to a subpattern whose number\nis 10 or more using this syntax because a sequence such as \\50 is interpreted as a character\ndefined in octal. See the subsection entitled \"Non-printing characters\" above for further de‐\ntails of the handling of digits following a backslash. There is no such problem when named\nparentheses are used. A back reference to any subpattern is possible using named parentheses\n(see below).\n\nAnother way of avoiding the ambiguity inherent in the use of digits following a backslash is\nto use the \\g escape sequence. This escape must be followed by an unsigned number or a nega‐\ntive number, optionally enclosed in braces. These examples are all identical:\n\n(ring), \\1\n(ring), \\g1\n(ring), \\g{1}\n\nAn unsigned number specifies an absolute reference without the ambiguity that is present in\nthe older syntax. It is also useful when literal digits follow the reference. A negative num‐\nber is a relative reference. Consider this example:\n\n(abc(def)ghi)\\g{-1}\n\nThe sequence \\g{-1} is a reference to the most recently started capturing subpattern before\n\\g, that is, is it equivalent to \\2 in this example. Similarly, \\g{-2} would be equivalent\nto \\1. The use of relative references can be helpful in long patterns, and also in patterns\nthat are created by joining together fragments that contain references within themselves.\n\nA back reference matches whatever actually matched the capturing subpattern in the current\nsubject string, rather than anything matching the subpattern itself (see \"Subpatterns as sub‐\nroutines\" below for a way of doing that). So the pattern\n\n(sens|respons)e and \\1ibility\n\nmatches \"sense and sensibility\" and \"response and responsibility\", but not \"sense and respon‐\nsibility\". If caseful matching is in force at the time of the back reference, the case of\nletters is relevant. For example,\n\n((?i)rah)\\s+\\1\n\nmatches \"rah rah\" and \"RAH RAH\", but not \"RAH rah\", even though the original capturing sub‐\npattern is matched caselessly.\n\nThere are several different ways of writing back references to named subpatterns. The .NET\nsyntax \\k{name} and the Perl syntax \\k or \\k'name' are supported, as is the Python syn‐\ntax (?P=name). Perl 5.10's unified back reference syntax, in which \\g can be used for both\nnumeric and named references, is also supported. We could rewrite the above example in any of\nthe following ways:\n\n(?(?i)rah)\\s+\\k\n(?'p1'(?i)rah)\\s+\\k{p1}\n(?P(?i)rah)\\s+(?P=p1)\n(?(?i)rah)\\s+\\g{p1}\n\nA subpattern that is referenced by name may appear in the pattern before or after the refer‐\nence.\n\nThere may be more than one back reference to the same subpattern. If a subpattern has not ac‐\ntually been used in a particular match, any back references to it always fail by default. For\nexample, the pattern\n\n(a|(bc))\\2\n\nalways fails if it starts to match \"a\" rather than \"bc\". However, if the PCREJAVASCRIPTCOM‐\nPAT option is set at compile time, a back reference to an unset value matches an empty\nstring.\n\nBecause there may be many capturing parentheses in a pattern, all digits following a back‐\nslash are taken as part of a potential back reference number. If the pattern continues with\na digit character, some delimiter must be used to terminate the back reference. If the\nPCREEXTENDED option is set, this can be white space. Otherwise, the \\g{ syntax or an empty\ncomment (see \"Comments\" below) can be used.\n", "subsections": [ { "name": "Recursive back references", "content": "A back reference that occurs inside the parentheses to which it refers fails when the subpat‐\ntern is first used, so, for example, (a\\1) never matches. However, such references can be\nuseful inside repeated subpatterns. For example, the pattern\n\n(a|b\\1)+\n\nmatches any number of \"a\"s and also \"aba\", \"ababbaa\" etc. At each iteration of the subpat‐\ntern, the back reference matches the character string corresponding to the previous itera‐\ntion. In order for this to work, the pattern must be such that the first iteration does not\nneed to match the back reference. This can be done using alternation, as in the example\nabove, or by a quantifier with a minimum of zero.\n\nBack references of this type cause the group that they reference to be treated as an atomic\ngroup. Once the whole group has been matched, a subsequent matching failure cannot cause\nbacktracking into the middle of the group.\n" } ] }, "ASSERTIONS": { "content": "An assertion is a test on the characters following or preceding the current matching point\nthat does not actually consume any characters. The simple assertions coded as \\b, \\B, \\A, \\G,\n\\Z, \\z, ^ and $ are described above.\n\nMore complicated assertions are coded as subpatterns. There are two kinds: those that look\nahead of the current position in the subject string, and those that look behind it. An asser‐\ntion subpattern is matched in the normal way, except that it does not cause the current\nmatching position to be changed.\n\nAssertion subpatterns are not capturing subpatterns. If such an assertion contains capturing\nsubpatterns within it, these are counted for the purposes of numbering the capturing subpat‐\nterns in the whole pattern. However, substring capturing is carried out only for positive as‐\nsertions. (Perl sometimes, but not always, does do capturing in negative assertions.)\n\nFor compatibility with Perl, assertion subpatterns may be repeated; though it makes no sense\nto assert the same thing several times, the side effect of capturing parentheses may occa‐\nsionally be useful. In practice, there only three cases:\n\n(1) If the quantifier is {0}, the assertion is never obeyed during matching. However, it may\ncontain internal capturing parenthesized groups that are called from elsewhere via the sub‐\nroutine mechanism.\n\n(2) If quantifier is {0,n} where n is greater than zero, it is treated as if it were {0,1}.\nAt run time, the rest of the pattern match is tried with and without the assertion, the order\ndepending on the greediness of the quantifier.\n\n(3) If the minimum repetition is greater than zero, the quantifier is ignored. The assertion\nis obeyed just once when encountered during matching.\n", "subsections": [ { "name": "Lookahead assertions", "content": "Lookahead assertions start with (?= for positive assertions and (?! for negative assertions.\nFor example,\n\n\\w+(?=;)\n\nmatches a word followed by a semicolon, but does not include the semicolon in the match, and\n\nfoo(?!bar)\n\nmatches any occurrence of \"foo\" that is not followed by \"bar\". Note that the apparently simi‐\nlar pattern\n\n(?!foo)bar\n\ndoes not find an occurrence of \"bar\" that is preceded by something other than \"foo\"; it finds\nany occurrence of \"bar\" whatsoever, because the assertion (?!foo) is always true when the\nnext three characters are \"bar\". A lookbehind assertion is needed to achieve the other ef‐\nfect.\n\nIf you want to force a matching failure at some point in a pattern, the most convenient way\nto do it is with (?!) because an empty string always matches, so an assertion that requires\nthere not to be an empty string must always fail. The backtracking control verb (*FAIL) or\n(*F) is a synonym for (?!).\n" }, { "name": "Lookbehind assertions", "content": "Lookbehind assertions start with (?<= for positive assertions and (?)...) or (?('name')...) to test for a used subpattern by name.\nFor compatibility with earlier versions of PCRE, which had this facility before Perl, the\nsyntax (?(name)...) is also recognized.\n\nRewriting the above example to use a named subpattern gives this:\n\n(? \\( )? [^()]+ (?() \\) )\n\nIf the name used in a condition of this kind is a duplicate, the test is applied to all sub‐\npatterns of the same name, and is true if any one of them has matched.\n" }, { "name": "Checking for pattern recursion", "content": "If the condition is the string (R), and there is no subpattern with the name R, the condition\nis true if a recursive call to the whole pattern or any subpattern has been made. If digits\nor a name preceded by ampersand follow the letter R, for example:\n\n(?(R3)...) or (?(R&name)...)\n\nthe condition is true if the most recent recursion is into a subpattern whose number or name\nis given. This condition does not check the entire recursion stack. If the name used in a\ncondition of this kind is a duplicate, the test is applied to all subpatterns of the same\nname, and is true if any one of them is the most recent recursion.\n\nAt \"top level\", all these recursion test conditions are false. The syntax for recursive pat‐\nterns is described below.\n" }, { "name": "Defining subpatterns for use by reference only", "content": "If the condition is the string (DEFINE), and there is no subpattern with the name DEFINE, the\ncondition is always false. In this case, there may be only one alternative in the subpattern.\nIt is always skipped if control reaches this point in the pattern; the idea of DEFINE is that\nit can be used to define subroutines that can be referenced from elsewhere. (The use of sub‐\nroutines is described below.) For example, a pattern to match an IPv4 address such as\n\"192.168.23.245\" could be written like this (ignore white space and line breaks):\n\n(?(DEFINE) (? 2[0-4]\\d | 25[0-5] | 1\\d\\d | [1-9]?\\d) )\n\\b (?&byte) (\\.(?&byte)){3} \\b\n\nThe first part of the pattern is a DEFINE group inside which a another group named \"byte\" is\ndefined. This matches an individual component of an IPv4 address (a number less than 256).\nWhen matching takes place, this part of the pattern is skipped because DEFINE acts like a\nfalse condition. The rest of the pattern uses references to the named group to match the four\ndot-separated components of an IPv4 address, insisting on a word boundary at each end.\n" }, { "name": "Assertion conditions", "content": "If the condition is not in any of the above formats, it must be an assertion. This may be a\npositive or negative lookahead or lookbehind assertion. Consider this pattern, again contain‐\ning non-significant white space, and with the two alternatives on the second line:\n\n(?(?=[^a-z]*[a-z])\n\\d{2}-[a-z]{3}-\\d{2} | \\d{2}-\\d{2}-\\d{2} )\n\nThe condition is a positive lookahead assertion that matches an optional sequence of non-let‐\nters followed by a letter. In other words, it tests for the presence of at least one letter\nin the subject. If a letter is found, the subject is matched against the first alternative;\notherwise it is matched against the second. This pattern matches strings in one of the two\nforms dd-aaa-dd or dd-dd-dd, where aaa are letters and dd are digits.\n" } ] }, "COMMENTS": { "content": "There are two ways of including comments in patterns that are processed by PCRE. In both\ncases, the start of the comment must not be in a character class, nor in the middle of any\nother sequence of related characters such as (?: or a subpattern name or number. The charac‐\nters that make up a comment play no part in the pattern matching.\n\nThe sequence (?# marks the start of a comment that continues up to the next closing parenthe‐\nsis. Nested parentheses are not permitted. If the PCREEXTENDED option is set, an unescaped #\ncharacter also introduces a comment, which in this case continues to immediately after the\nnext newline character or character sequence in the pattern. Which characters are interpreted\nas newlines is controlled by the options passed to a compiling function or by a special se‐\nquence at the start of the pattern, as described in the section entitled \"Newline conven‐\ntions\" above. Note that the end of this type of comment is a literal newline sequence in the\npattern; escape sequences that happen to represent a newline do not count. For example, con‐\nsider this pattern when PCREEXTENDED is set, and the default newline convention is in force:\n\nabc #comment \\n still comment\n\nOn encountering the # character, pcrecompile() skips along, looking for a newline in the\npattern. The sequence \\n is still literal at this stage, so it does not terminate the com‐\nment. Only an actual character with the code value 0x0a (the default newline) does so.\n", "subsections": [] }, "RECURSIVE PATTERNS": { "content": "Consider the problem of matching a string in parentheses, allowing for unlimited nested\nparentheses. Without the use of recursion, the best that can be done is to use a pattern that\nmatches up to some fixed depth of nesting. It is not possible to handle an arbitrary nesting\ndepth.\n\nFor some time, Perl has provided a facility that allows regular expressions to recurse\n(amongst other things). It does this by interpolating Perl code in the expression at run\ntime, and the code can refer to the expression itself. A Perl pattern using code interpola‐\ntion to solve the parentheses problem can be created like this:\n\n$re = qr{\\( (?: (?>[^()]+) | (?p{$re}) )* \\)}x;\n\nThe (?p{...}) item interpolates Perl code at run time, and in this case refers recursively to\nthe pattern in which it appears.\n\nObviously, PCRE cannot support the interpolation of Perl code. Instead, it supports special\nsyntax for recursion of the entire pattern, and also for individual subpattern recursion. Af‐\nter its introduction in PCRE and Python, this kind of recursion was subsequently introduced\ninto Perl at release 5.10.\n\nA special item that consists of (? followed by a number greater than zero and a closing\nparenthesis is a recursive subroutine call of the subpattern of the given number, provided\nthat it occurs inside that subpattern. (If not, it is a non-recursive subroutine call, which\nis described in the next section.) The special item (?R) or (?0) is a recursive call of the\nentire regular expression.\n\nThis PCRE pattern solves the nested parentheses problem (assume the PCREEXTENDED option is\nset so that white space is ignored):\n\n\\( ( [^()]++ | (?R) )* \\)\n\nFirst it matches an opening parenthesis. Then it matches any number of substrings which can\neither be a sequence of non-parentheses, or a recursive match of the pattern itself (that is,\na correctly parenthesized substring). Finally there is a closing parenthesis. Note the use\nof a possessive quantifier to avoid backtracking into sequences of non-parentheses.\n\nIf this were part of a larger pattern, you would not want to recurse the entire pattern, so\ninstead you could use this:\n\n( \\( ( [^()]++ | (?1) )* \\) )\n\nWe have put the pattern into parentheses, and caused the recursion to refer to them instead\nof the whole pattern.\n\nIn a larger pattern, keeping track of parenthesis numbers can be tricky. This is made easier\nby the use of relative references. Instead of (?1) in the pattern above you can write (?-2)\nto refer to the second most recently opened parentheses preceding the recursion. In other\nwords, a negative number counts capturing parentheses leftwards from the point at which it is\nencountered.\n\nIt is also possible to refer to subsequently opened parentheses, by writing references such\nas (?+2). However, these cannot be recursive because the reference is not inside the paren‐\ntheses that are referenced. They are always non-recursive subroutine calls, as described in\nthe next section.\n\nAn alternative approach is to use named parentheses instead. The Perl syntax for this is\n(?&name); PCRE's earlier syntax (?P>name) is also supported. We could rewrite the above exam‐\nple as follows:\n\n(? \\( ( [^()]++ | (?&pn) )* \\) )\n\nIf there is more than one subpattern with the same name, the earliest one is used.\n\nThis particular example pattern that we have been looking at contains nested unlimited re‐\npeats, and so the use of a possessive quantifier for matching strings of non-parentheses is\nimportant when applying the pattern to strings that do not match. For example, when this pat‐\ntern is applied to\n\n(aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa()\n\nit yields \"no match\" quickly. However, if a possessive quantifier is not used, the match runs\nfor a very long time indeed because there are so many different ways the + and * repeats can\ncarve up the subject, and all have to be tested before failure can be reported.\n\nAt the end of a match, the values of capturing parentheses are those from the outermost\nlevel. If you want to obtain intermediate values, a callout function can be used (see below\nand the pcrecallout documentation). If the pattern above is matched against\n\n(ab(cd)ef)\n\nthe value for the inner capturing parentheses (numbered 2) is \"ef\", which is the last value\ntaken on at the top level. If a capturing subpattern is not matched at the top level, its fi‐\nnal captured value is unset, even if it was (temporarily) set at a deeper level during the\nmatching process.\n\nIf there are more than 15 capturing parentheses in a pattern, PCRE has to obtain extra memory\nto store data during a recursion, which it does by using pcremalloc, freeing it via\npcrefree afterwards. If no memory can be obtained, the match fails with the PCREER‐\nRORNOMEMORY error.\n\nDo not confuse the (?R) item with the condition (R), which tests for recursion. Consider\nthis pattern, which matches text in angle brackets, allowing for arbitrary nesting. Only dig‐\nits are allowed in nested brackets (that is, when recursing), whereas any characters are per‐\nmitted at the outer level.\n\n< (?: (?(R) \\d++ | [^<>]*+) | (?R)) * >\n\nIn this pattern, (?(R) is the start of a conditional subpattern, with two different alterna‐\ntives for the recursive and non-recursive cases. The (?R) item is the actual recursive call.\n", "subsections": [ { "name": "Differences in recursion processing between PCRE and Perl", "content": "Recursion processing in PCRE differs from Perl in two important ways. In PCRE (like Python,\nbut unlike Perl), a recursive subpattern call is always treated as an atomic group. That is,\nonce it has matched some of the subject string, it is never re-entered, even if it contains\nuntried alternatives and there is a subsequent matching failure. This can be illustrated by\nthe following pattern, which purports to match a palindromic string that contains an odd num‐\nber of characters (for example, \"a\", \"aba\", \"abcba\", \"abcdcba\"):\n\n^(.|(.)(?1)\\2)$\n\nThe idea is that it either matches a single character, or two identical characters surround‐\ning a sub-palindrome. In Perl, this pattern works; in PCRE it does not if the pattern is\nlonger than three characters. Consider the subject string \"abcba\":\n\nAt the top level, the first character is matched, but as it is not at the end of the string,\nthe first alternative fails; the second alternative is taken and the recursion kicks in. The\nrecursive call to subpattern 1 successfully matches the next character (\"b\"). (Note that the\nbeginning and end of line tests are not part of the recursion).\n\nBack at the top level, the next character (\"c\") is compared with what subpattern 2 matched,\nwhich was \"a\". This fails. Because the recursion is treated as an atomic group, there are now\nno backtracking points, and so the entire match fails. (Perl is able, at this point, to re-\nenter the recursion and try the second alternative.) However, if the pattern is written with\nthe alternatives in the other order, things are different:\n\n^((.)(?1)\\2|.)$\n\nThis time, the recursing alternative is tried first, and continues to recurse until it runs\nout of characters, at which point the recursion fails. But this time we do have another al‐\nternative to try at the higher level. That is the big difference: in the previous case the\nremaining alternative is at a deeper recursion level, which PCRE cannot use.\n\nTo change the pattern so that it matches all palindromic strings, not just those with an odd\nnumber of characters, it is tempting to change the pattern to this:\n\n^((.)(?1)\\2|.?)$\n\nAgain, this works in Perl, but not in PCRE, and for the same reason. When a deeper recursion\nhas matched a single character, it cannot be entered again in order to match an empty string.\nThe solution is to separate the two cases, and write out the odd and even cases as alterna‐\ntives at the higher level:\n\n^(?:((.)(?1)\\2|)|((.)(?3)\\4|.))\n\nIf you want to match typical palindromic phrases, the pattern has to ignore all non-word\ncharacters, which can be done like this:\n\n^\\W*+(?:((.)\\W*+(?1)\\W*+\\2|)|((.)\\W*+(?3)\\W*+\\4|\\W*+.\\W*+))\\W*+$\n\nIf run with the PCRECASELESS option, this pattern matches phrases such as \"A man, a plan, a\ncanal: Panama!\" and it works well in both PCRE and Perl. Note the use of the possessive quan‐\ntifier *+ to avoid backtracking into sequences of non-word characters. Without this, PCRE\ntakes a great deal longer (ten times or more) to match typical phrases, and Perl takes so\nlong that you think it has gone into a loop.\n\nWARNING: The palindrome-matching patterns above work only if the subject string does not\nstart with a palindrome that is shorter than the entire string. For example, although\n\"abcba\" is correctly matched, if the subject is \"ababa\", PCRE finds the palindrome \"aba\" at\nthe start, then fails at top level because the end of the string does not follow. Once again,\nit cannot jump back into the recursion to try other alternatives, so the entire match fails.\n\nThe second way in which PCRE and Perl differ in their recursion processing is in the handling\nof captured values. In Perl, when a subpattern is called recursively or as a subpattern (see\nthe next section), it has no access to any values that were captured outside the recursion,\nwhereas in PCRE these values can be referenced. Consider this pattern:\n\n^(.)(\\1|a(?2))\n\nIn PCRE, this pattern matches \"bab\". The first capturing parentheses match \"b\", then in the\nsecond group, when the back reference \\1 fails to match \"b\", the second alternative matches\n\"a\" and then recurses. In the recursion, \\1 does now match \"b\" and so the whole match suc‐\nceeds. In Perl, the pattern fails to match because inside the recursive call \\1 cannot access\nthe externally set value.\n" } ] }, "SUBPATTERNS AS SUBROUTINES": { "content": "If the syntax for a recursive subpattern call (either by number or by name) is used outside\nthe parentheses to which it refers, it operates like a subroutine in a programming language.\nThe called subpattern may be defined before or after the reference. A numbered reference can\nbe absolute or relative, as in these examples:\n\n(...(absolute)...)...(?2)...\n(...(relative)...)...(?-1)...\n(...(?+1)...(relative)...\n\nAn earlier example pointed out that the pattern\n\n(sens|respons)e and \\1ibility\n\nmatches \"sense and sensibility\" and \"response and responsibility\", but not \"sense and respon‐\nsibility\". If instead the pattern\n\n(sens|respons)e and (?1)ibility\n\nis used, it does match \"sense and responsibility\" as well as the other two strings. Another\nexample is given in the discussion of DEFINE above.\n\nAll subroutine calls, whether recursive or not, are always treated as atomic groups. That is,\nonce a subroutine has matched some of the subject string, it is never re-entered, even if it\ncontains untried alternatives and there is a subsequent matching failure. Any capturing\nparentheses that are set during the subroutine call revert to their previous values after‐\nwards.\n\nProcessing options such as case-independence are fixed when a subpattern is defined, so if it\nis used as a subroutine, such options cannot be changed for different calls. For example,\nconsider this pattern:\n\n(abc)(?i:(?-1))\n\nIt matches \"abcabc\". It does not match \"abcABC\" because the change of processing option does\nnot affect the called subpattern.\n", "subsections": [] }, "ONIGURUMA SUBROUTINE SYNTAX": { "content": "For compatibility with Oniguruma, the non-Perl syntax \\g followed by a name or a number en‐\nclosed either in angle brackets or single quotes, is an alternative syntax for referencing a\nsubpattern as a subroutine, possibly recursively. Here are two of the examples used above,\nrewritten using this syntax:\n\n(? \\( ( (?>[^()]+) | \\g )* \\) )\n(sens|respons)e and \\g'1'ibility\n\nPCRE supports an extension to Oniguruma: if a number is preceded by a plus or a minus sign it\nis taken as a relative reference. For example:\n\n(abc)(?i:\\g<-1>)\n\nNote that \\g{...} (Perl syntax) and \\g<...> (Oniguruma syntax) are not synonymous. The former\nis a back reference; the latter is a subroutine call.\n", "subsections": [] }, "CALLOUTS": { "content": "Perl has a feature whereby using the sequence (?{...}) causes arbitrary Perl code to be\nobeyed in the middle of matching a regular expression. This makes it possible, amongst other\nthings, to extract different substrings that match the same pair of parentheses when there is\na repetition.\n\nPCRE provides a similar feature, but of course it cannot obey arbitrary Perl code. The fea‐\nture is called \"callout\". The caller of PCRE provides an external function by putting its en‐\ntry point in the global variable pcrecallout (8-bit library) or pcre[16|32]callout (16-bit\nor 32-bit library). By default, this variable contains NULL, which disables all calling out.\n\nWithin a regular expression, (?C) indicates the points at which the external function is to\nbe called. If you want to identify different callout points, you can put a number less than\n256 after the letter C. The default value is zero. For example, this pattern has two callout\npoints:\n\n(?C1)abc(?C2)def\n\nIf the PCREAUTOCALLOUT flag is passed to a compiling function, callouts are automatically\ninstalled before each item in the pattern. They are all numbered 255. If there is a condi‐\ntional group in the pattern whose condition is an assertion, an additional callout is in‐\nserted just before the condition. An explicit callout may also be set at this position, as in\nthis example:\n\n(?(?C9)(?=a)abc|def)\n\nNote that this applies only to assertion conditions, not to other types of condition.\n\nDuring matching, when PCRE reaches a callout point, the external function is called. It is\nprovided with the number of the callout, the position in the pattern, and, optionally, one\nitem of data originally supplied by the caller of the matching function. The callout function\nmay cause matching to proceed, to backtrack, or to fail altogether.\n\nBy default, PCRE implements a number of optimizations at compile time and matching time, and\none side-effect is that sometimes callouts are skipped. If you need all possible callouts to\nhappen, you need to set options that disable the relevant optimizations. More details, and a\ncomplete description of the interface to the callout function, are given in the pcrecallout\ndocumentation.\n", "subsections": [] }, "BACKTRACKING CONTROL": { "content": "Perl 5.10 introduced a number of \"Special Backtracking Control Verbs\", which are still de‐\nscribed in the Perl documentation as \"experimental and subject to change or removal in a fu‐\nture version of Perl\". It goes on to say: \"Their usage in production code should be noted to\navoid problems during upgrades.\" The same remarks apply to the PCRE features described in\nthis section.\n\nThe new verbs make use of what was previously invalid syntax: an opening parenthesis followed\nby an asterisk. They are generally of the form (*VERB) or (*VERB:NAME). Some may take either\nform, possibly behaving differently depending on whether or not a name is present. A name is\nany sequence of characters that does not include a closing parenthesis. The maximum length of\nname is 255 in the 8-bit library and 65535 in the 16-bit and 32-bit libraries. If the name is\nempty, that is, if the closing parenthesis immediately follows the colon, the effect is as if\nthe colon were not there. Any number of these verbs may occur in a pattern.\n\nSince these verbs are specifically related to backtracking, most of them can be used only\nwhen the pattern is to be matched using one of the traditional matching functions, because\nthese use a backtracking algorithm. With the exception of (*FAIL), which behaves like a fail‐\ning negative assertion, the backtracking control verbs cause an error if encountered by a DFA\nmatching function.\n\nThe behaviour of these verbs in repeated groups, assertions, and in subpatterns called as\nsubroutines (whether or not recursively) is documented below.\n", "subsections": [ { "name": "Optimizations that affect backtracking verbs", "content": "PCRE contains some optimizations that are used to speed up matching by running some checks at\nthe start of each match attempt. For example, it may know the minimum length of matching sub‐\nject, or that a particular character must be present. When one of these optimizations by‐\npasses the running of a match, any included backtracking verbs will not, of course, be pro‐\ncessed. You can suppress the start-of-match optimizations by setting the PCRENOSTARTOPTI‐\nMIZE option when calling pcrecompile() or pcreexec(), or by starting the pattern with\n(*NOSTARTOPT). There is more discussion of this option in the section entitled \"Option bits\nfor pcreexec()\" in the pcreapi documentation.\n\nExperiments with Perl suggest that it too has similar optimizations, sometimes leading to\nanomalous results.\n" }, { "name": "Verbs that act immediately", "content": "The following verbs act as soon as they are encountered. They may not be followed by a name.\n\n(*ACCEPT)\n\nThis verb causes the match to end successfully, skipping the remainder of the pattern. How‐\never, when it is inside a subpattern that is called as a subroutine, only that subpattern is\nended successfully. Matching then continues at the outer level. If (*ACCEPT) in triggered in\na positive assertion, the assertion succeeds; in a negative assertion, the assertion fails.\n\nIf (*ACCEPT) is inside capturing parentheses, the data so far is captured. For example:\n\nA((?:A|B(*ACCEPT)|C)D)\n\nThis matches \"AB\", \"AAD\", or \"ACD\"; when it matches \"AB\", \"B\" is captured by the outer paren‐\ntheses.\n\n(*FAIL) or (*F)\n\nThis verb causes a matching failure, forcing backtracking to occur. It is equivalent to (?!)\nbut easier to read. The Perl documentation notes that it is probably useful only when com‐\nbined with (?{}) or (??{}). Those are, of course, Perl features that are not present in PCRE.\nThe nearest equivalent is the callout feature, as for example in this pattern:\n\na+(?C)(*FAIL)\n\nA match with the string \"aaaa\" always fails, but the callout is taken before each backtrack\nhappens (in this example, 10 times).\n" }, { "name": "Recording which path was taken", "content": "There is one verb whose main purpose is to track how a match was arrived at, though it also\nhas a secondary use in conjunction with advancing the match starting point (see (*SKIP) be‐\nlow).\n\n(*MARK:NAME) or (*:NAME)\n\nA name is always required with this verb. There may be as many instances of (*MARK) as you\nlike in a pattern, and their names do not have to be unique.\n\nWhen a match succeeds, the name of the last-encountered (*MARK:NAME), (*PRUNE:NAME), or\n(*THEN:NAME) on the matching path is passed back to the caller as described in the section\nentitled \"Extra data for pcreexec()\" in the pcreapi documentation. Here is an example of\npcretest output, where the /K modifier requests the retrieval and outputting of (*MARK) data:\n\nre> /X(*MARK:A)Y|X(*MARK:B)Z/K\ndata> XY\n0: XY\nMK: A\nXZ\n0: XZ\nMK: B\n\nThe (*MARK) name is tagged with \"MK:\" in this output, and in this example it indicates which\nof the two alternatives matched. This is a more efficient way of obtaining this information\nthan putting each alternative in its own capturing parentheses.\n\nIf a verb with a name is encountered in a positive assertion that is true, the name is\nrecorded and passed back if it is the last-encountered. This does not happen for negative as‐\nsertions or failing positive assertions.\n\nAfter a partial match or a failed match, the last encountered name in the entire match\nprocess is returned. For example:\n\nre> /X(*MARK:A)Y|X(*MARK:B)Z/K\ndata> XP\nNo match, mark = B\n\nNote that in this unanchored example the mark is retained from the match attempt that started\nat the letter \"X\" in the subject. Subsequent match attempts starting at \"P\" and then with an\nempty string do not get as far as the (*MARK) item, but nevertheless do not reset it.\n\nIf you are interested in (*MARK) values after failed matches, you should probably set the\nPCRENOSTARTOPTIMIZE option (see above) to ensure that the match is always attempted.\n" }, { "name": "Verbs that act after backtracking", "content": "The following verbs do nothing when they are encountered. Matching continues with what fol‐\nlows, but if there is no subsequent match, causing a backtrack to the verb, a failure is\nforced. That is, backtracking cannot pass to the left of the verb. However, when one of these\nverbs appears inside an atomic group or an assertion that is true, its effect is confined to\nthat group, because once the group has been matched, there is never any backtracking into it.\nIn this situation, backtracking can \"jump back\" to the left of the entire atomic group or as‐\nsertion. (Remember also, as stated above, that this localization also applies in subroutine\ncalls.)\n\nThese verbs differ in exactly what kind of failure occurs when backtracking reaches them. The\nbehaviour described below is what happens when the verb is not in a subroutine or an asser‐\ntion. Subsequent sections cover these special cases.\n\n(*COMMIT)\n\nThis verb, which may not be followed by a name, causes the whole match to fail outright if\nthere is a later matching failure that causes backtracking to reach it. Even if the pattern\nis unanchored, no further attempts to find a match by advancing the starting point take\nplace. If (*COMMIT) is the only backtracking verb that is encountered, once it has been\npassed pcreexec() is committed to finding a match at the current starting point, or not at\nall. For example:\n\na+(*COMMIT)b\n\nThis matches \"xxaab\" but not \"aacaab\". It can be thought of as a kind of dynamic anchor, or\n\"I've started, so I must finish.\" The name of the most recently passed (*MARK) in the path is\npassed back when (*COMMIT) forces a match failure.\n\nIf there is more than one backtracking verb in a pattern, a different one that follows (*COM‐\nMIT) may be triggered first, so merely passing (*COMMIT) during a match does not always guar‐\nantee that a match must be at this starting point.\n\nNote that (*COMMIT) at the start of a pattern is not the same as an anchor, unless PCRE's\nstart-of-match optimizations are turned off, as shown in this output from pcretest:\n\nre> /(*COMMIT)abc/\ndata> xyzabc\n0: abc\ndata> xyzabc\\Y\nNo match\n\nFor this pattern, PCRE knows that any match must start with \"a\", so the optimization skips\nalong the subject to \"a\" before applying the pattern to the first set of data. The match at‐\ntempt then succeeds. In the second set of data, the escape sequence \\Y is interpreted by the\npcretest program. It causes the PCRENOSTARTOPTIMIZE option to be set when pcreexec() is\ncalled. This disables the optimization that skips along to the first character. The pattern\nis now applied starting at \"x\", and so the (*COMMIT) causes the match to fail without trying\nany other starting points.\n\n(*PRUNE) or (*PRUNE:NAME)\n\nThis verb causes the match to fail at the current starting position in the subject if there\nis a later matching failure that causes backtracking to reach it. If the pattern is unan‐\nchored, the normal \"bumpalong\" advance to the next starting character then happens. Back‐\ntracking can occur as usual to the left of (*PRUNE), before it is reached, or when matching\nto the right of (*PRUNE), but if there is no match to the right, backtracking cannot cross\n(*PRUNE). In simple cases, the use of (*PRUNE) is just an alternative to an atomic group or\npossessive quantifier, but there are some uses of (*PRUNE) that cannot be expressed in any\nother way. In an anchored pattern (*PRUNE) has the same effect as (*COMMIT).\n\nThe behaviour of (*PRUNE:NAME) is the not the same as (*MARK:NAME)(*PRUNE). It is like\n(*MARK:NAME) in that the name is remembered for passing back to the caller. However,\n(*SKIP:NAME) searches only for names set with (*MARK).\n\n(*SKIP)\n\nThis verb, when given without a name, is like (*PRUNE), except that if the pattern is unan‐\nchored, the \"bumpalong\" advance is not to the next character, but to the position in the sub‐\nject where (*SKIP) was encountered. (*SKIP) signifies that whatever text was matched leading\nup to it cannot be part of a successful match. Consider:\n\na+(*SKIP)b\n\nIf the subject is \"aaaac...\", after the first match attempt fails (starting at the first\ncharacter in the string), the starting point skips on to start the next attempt at \"c\". Note\nthat a possessive quantifer does not have the same effect as this example; although it would\nsuppress backtracking during the first match attempt, the second attempt would start at the\nsecond character instead of skipping on to \"c\".\n\n(*SKIP:NAME)\n\nWhen (*SKIP) has an associated name, its behaviour is modified. When it is triggered, the\nprevious path through the pattern is searched for the most recent (*MARK) that has the same\nname. If one is found, the \"bumpalong\" advance is to the subject position that corresponds to\nthat (*MARK) instead of to where (*SKIP) was encountered. If no (*MARK) with a matching name\nis found, the (*SKIP) is ignored.\n\nNote that (*SKIP:NAME) searches only for names set by (*MARK:NAME). It ignores names that are\nset by (*PRUNE:NAME) or (*THEN:NAME).\n\n(*THEN) or (*THEN:NAME)\n\nThis verb causes a skip to the next innermost alternative when backtracking reaches it. That\nis, it cancels any further backtracking within the current alternative. Its name comes from\nthe observation that it can be used for a pattern-based if-then-else block:\n\n( COND1 (*THEN) FOO | COND2 (*THEN) BAR | COND3 (*THEN) BAZ ) ...\n\nIf the COND1 pattern matches, FOO is tried (and possibly further items after the end of the\ngroup if FOO succeeds); on failure, the matcher skips to the second alternative and tries\nCOND2, without backtracking into COND1. If that succeeds and BAR fails, COND3 is tried. If\nsubsequently BAZ fails, there are no more alternatives, so there is a backtrack to whatever\ncame before the entire group. If (*THEN) is not inside an alternation, it acts like (*PRUNE).\n\nThe behaviour of (*THEN:NAME) is the not the same as (*MARK:NAME)(*THEN). It is like\n(*MARK:NAME) in that the name is remembered for passing back to the caller. However,\n(*SKIP:NAME) searches only for names set with (*MARK).\n\nA subpattern that does not contain a | character is just a part of the enclosing alternative;\nit is not a nested alternation with only one alternative. The effect of (*THEN) extends be‐\nyond such a subpattern to the enclosing alternative. Consider this pattern, where A, B, etc.\nare complex pattern fragments that do not contain any | characters at this level:\n\nA (B(*THEN)C) | D\n\nIf A and B are matched, but there is a failure in C, matching does not backtrack into A; in‐\nstead it moves to the next alternative, that is, D. However, if the subpattern containing\n(*THEN) is given an alternative, it behaves differently:\n\nA (B(*THEN)C | (*FAIL)) | D\n\nThe effect of (*THEN) is now confined to the inner subpattern. After a failure in C, matching\nmoves to (*FAIL), which causes the whole subpattern to fail because there are no more alter‐\nnatives to try. In this case, matching does now backtrack into A.\n\nNote that a conditional subpattern is not considered as having two alternatives, because only\none is ever used. In other words, the | character in a conditional subpattern has a different\nmeaning. Ignoring white space, consider:\n\n^.*? (?(?=a) a | b(*THEN)c )\n\nIf the subject is \"ba\", this pattern does not match. Because .*? is ungreedy, it initially\nmatches zero characters. The condition (?=a) then fails, the character \"b\" is matched, but\n\"c\" is not. At this point, matching does not backtrack to .*? as might perhaps be expected\nfrom the presence of the | character. The conditional subpattern is part of the single alter‐\nnative that comprises the whole pattern, and so the match fails. (If there was a backtrack\ninto .*?, allowing it to match \"b\", the match would succeed.)\n\nThe verbs just described provide four different \"strengths\" of control when subsequent match‐\ning fails. (*THEN) is the weakest, carrying on the match at the next alternative. (*PRUNE)\ncomes next, failing the match at the current starting position, but allowing an advance to\nthe next character (for an unanchored pattern). (*SKIP) is similar, except that the advance\nmay be more than one character. (*COMMIT) is the strongest, causing the entire match to fail.\n" }, { "name": "More than one backtracking verb", "content": "If more than one backtracking verb is present in a pattern, the one that is backtracked onto\nfirst acts. For example, consider this pattern, where A, B, etc. are complex pattern frag‐\nments:\n\n(A(*COMMIT)B(*THEN)C|ABD)\n\nIf A matches but B fails, the backtrack to (*COMMIT) causes the entire match to fail. How‐\never, if A and B match, but C fails, the backtrack to (*THEN) causes the next alternative\n(ABD) to be tried. This behaviour is consistent, but is not always the same as Perl's. It\nmeans that if two or more backtracking verbs appear in succession, all the the last of them\nhas no effect. Consider this example:\n\n...(*COMMIT)(*PRUNE)...\n\nIf there is a matching failure to the right, backtracking onto (*PRUNE) causes it to be trig‐\ngered, and its action is taken. There can never be a backtrack onto (*COMMIT).\n" }, { "name": "Backtracking verbs in repeated groups", "content": "PCRE differs from Perl in its handling of backtracking verbs in repeated groups. For example,\nconsider:\n\n/(a(*COMMIT)b)+ac/\n\nIf the subject is \"abac\", Perl matches, but PCRE fails because the (*COMMIT) in the second\nrepeat of the group acts.\n" }, { "name": "Backtracking verbs in assertions", "content": "(*FAIL) in an assertion has its normal effect: it forces an immediate backtrack.\n\n(*ACCEPT) in a positive assertion causes the assertion to succeed without any further pro‐\ncessing. In a negative assertion, (*ACCEPT) causes the assertion to fail without any further\nprocessing.\n\nThe other backtracking verbs are not treated specially if they appear in a positive asser‐\ntion. In particular, (*THEN) skips to the next alternative in the innermost enclosing group\nthat has alternations, whether or not this is within the assertion.\n\nNegative assertions are, however, different, in order to ensure that changing a positive as‐\nsertion into a negative assertion changes its result. Backtracking into (*COMMIT), (*SKIP),\nor (*PRUNE) causes a negative assertion to be true, without considering any further alterna‐\ntive branches in the assertion. Backtracking into (*THEN) causes it to skip to the next en‐\nclosing alternative within the assertion (the normal behaviour), but if the assertion does\nnot have such an alternative, (*THEN) behaves like (*PRUNE).\n" }, { "name": "Backtracking verbs in subroutines", "content": "These behaviours occur whether or not the subpattern is called recursively. Perl's treatment\nof subroutines is different in some cases.\n\n(*FAIL) in a subpattern called as a subroutine has its normal effect: it forces an immediate\nbacktrack.\n\n(*ACCEPT) in a subpattern called as a subroutine causes the subroutine match to succeed with‐\nout any further processing. Matching then continues after the subroutine call.\n\n(*COMMIT), (*SKIP), and (*PRUNE) in a subpattern called as a subroutine cause the subroutine\nmatch to fail.\n\n(*THEN) skips to the next alternative in the innermost enclosing group within the subpattern\nthat has alternatives. If there is no such group within the subpattern, (*THEN) causes the\nsubroutine match to fail.\n" } ] }, "SEE ALSO": { "content": "pcreapi(3), pcrecallout(3), pcrematching(3), pcresyntax(3), pcre(3), pcre16(3), pcre32(3).\n", "subsections": [] }, "AUTHOR": { "content": "Philip Hazel\nUniversity Computing Service\nCambridge CB2 3QH, England.\n", "subsections": [] }, "REVISION": { "content": "Last updated: 14 June 2015\nCopyright (c) 1997-2015 University of Cambridge.\n\n\n\nPCRE 8.38 14 June 2015 PCREPATTERN(3)", "subsections": [] } }, "summary": "PCRE - Perl-compatible regular expressions", "flags": [], "examples": [], "see_also": [ { "name": "pcreapi", "section": "3", "url": "https://www.chedong.com/phpMan.php/man/pcreapi/3/json" }, { "name": "pcrecallout", "section": "3", "url": "https://www.chedong.com/phpMan.php/man/pcrecallout/3/json" }, { "name": "pcrematching", "section": "3", "url": "https://www.chedong.com/phpMan.php/man/pcrematching/3/json" }, { "name": "pcresyntax", "section": "3", "url": "https://www.chedong.com/phpMan.php/man/pcresyntax/3/json" }, { "name": "pcre", "section": "3", "url": "https://www.chedong.com/phpMan.php/man/pcre/3/json" }, { "name": "pcre16", "section": "3", "url": "https://www.chedong.com/phpMan.php/man/pcre16/3/json" }, { "name": "pcre32", "section": "3", "url": "https://www.chedong.com/phpMan.php/man/pcre32/3/json" } ] }