{ "content": [ { "type": "text", "text": "# Filter::Simple (perldoc)\n\n## NAME\n\nFilter::Simple - Simplified source filtering\n\n## SYNOPSIS\n\n# in MyFilter.pm:\npackage MyFilter;\nuse Filter::Simple;\nFILTER { ... };\n# or just:\n#\n# use Filter::Simple sub { ... };\n# in user's code:\nuse MyFilter;\n# this code is filtered\nno MyFilter;\n# this code is not\n\n## Sections\n\n- **NAME**\n- **SYNOPSIS**\n- **DESCRIPTION** (6 subsections)\n- **AUTHOR**\n- **CONTACT**\n- **COPYRIGHT AND LICENSE**\n\nUse structuredContent.sections for detailed options, examples, and full documentation.\n" } ], "structuredContent": { "command": "Filter::Simple", "section": "", "mode": "perldoc", "summary": "Filter::Simple - Simplified source filtering", "synopsis": "# in MyFilter.pm:\npackage MyFilter;\nuse Filter::Simple;\nFILTER { ... };\n# or just:\n#\n# use Filter::Simple sub { ... };\n# in user's code:\nuse MyFilter;\n# this code is filtered\nno MyFilter;\n# this code is not", "tldr_summary": null, "tldr_examples": [], "tldr_source": null, "flags": [], "examples": [], "see_also": [], "section_outline": [ { "name": "NAME", "lines": 2, "subsections": [] }, { "name": "SYNOPSIS", "lines": 22, "subsections": [] }, { "name": "DESCRIPTION", "lines": 1, "subsections": [ { "name": "The Problem", "lines": 163 }, { "name": "All-in-one interface", "lines": 29 }, { "name": "Filtering only specific components of source code", "lines": 79 }, { "name": "Filtering only the code parts of source code", "lines": 88 }, { "name": "Using Filter::Simple and Exporter together", "lines": 19 }, { "name": "How it works", "lines": 25 } ] }, { "name": "AUTHOR", "lines": 2, "subsections": [] }, { "name": "CONTACT", "lines": 10, "subsections": [] }, { "name": "COPYRIGHT AND LICENSE", "lines": 4, "subsections": [] } ], "sections": { "NAME": { "content": "Filter::Simple - Simplified source filtering\n", "subsections": [] }, "SYNOPSIS": { "content": "# in MyFilter.pm:\n\npackage MyFilter;\n\nuse Filter::Simple;\n\nFILTER { ... };\n\n# or just:\n#\n# use Filter::Simple sub { ... };\n\n# in user's code:\n\nuse MyFilter;\n\n# this code is filtered\n\nno MyFilter;\n\n# this code is not\n", "subsections": [] }, "DESCRIPTION": { "content": "", "subsections": [ { "name": "The Problem", "content": "Source filtering is an immensely powerful feature of recent versions of Perl. It allows one to\nextend the language itself (e.g. the Switch module), to simplify the language (e.g.\nLanguage::Pythonesque), or to completely recast the language (e.g. Lingua::Romana::Perligata).\nEffectively, it allows one to use the full power of Perl as its own, recursively applied, macro\nlanguage.\n\nThe excellent Filter::Util::Call module (by Paul Marquess) provides a usable Perl interface to\nsource filtering, but it is often too powerful and not nearly as simple as it could be.\n\nTo use the module it is necessary to do the following:\n\n1. Download, build, and install the Filter::Util::Call module. (If you have Perl 5.7.1 or\nlater, this is already done for you.)\n\n2. Set up a module that does a \"use Filter::Util::Call\".\n\n3. Within that module, create an \"import\" subroutine.\n\n4. Within the \"import\" subroutine do a call to \"filteradd\", passing it either a subroutine\nreference.\n\n5. Within the subroutine reference, call \"filterread\" or \"filterreadexact\" to \"prime\" $\nwith source code data from the source file that will \"use\" your module. Check the status\nvalue returned to see if any source code was actually read in.\n\n6. Process the contents of $ to change the source code in the desired manner.\n\n7. Return the status value.\n\n8. If the act of unimporting your module (via a \"no\") should cause source code filtering to\ncease, create an \"unimport\" subroutine, and have it call \"filterdel\". Make sure that the\ncall to \"filterread\" or \"filterreadexact\" in step 5 will not accidentally read past the\n\"no\". Effectively this limits source code filters to line-by-line operation, unless the\n\"import\" subroutine does some fancy pre-pre-parsing of the source code it's filtering.\n\nFor example, here is a minimal source code filter in a module named BANG.pm. It simply converts\nevery occurrence of the sequence \"BANG\\s+BANG\" to the sequence \"die 'BANG' if $BANG\" in any\npiece of code following a \"use BANG;\" statement (until the next \"no BANG;\" statement, if any):\n\npackage BANG;\n\nuse Filter::Util::Call ;\n\nsub import {\nfilteradd( sub {\nmy $caller = caller;\nmy ($status, $noseen, $data);\nwhile ($status = filterread()) {\nif (/^\\s*no\\s+$caller\\s*;\\s*?$/) {\n$noseen=1;\nlast;\n}\n$data .= $;\n$ = \"\";\n}\n$ = $data;\ns/BANG\\s+BANG/die 'BANG' if \\$BANG/g\nunless $status < 0;\n$ .= \"no $class;\\n\" if $noseen;\nreturn 1;\n})\n}\n\nsub unimport {\nfilterdel();\n}\n\n1 ;\n\nThis level of sophistication puts filtering out of the reach of many programmers.\n\nA Solution\nThe Filter::Simple module provides a simplified interface to Filter::Util::Call; one that is\nsufficient for most common cases.\n\nInstead of the above process, with Filter::Simple the task of setting up a source code filter is\nreduced to:\n\n1. Download and install the Filter::Simple module. (If you have Perl 5.7.1 or later, this is\nalready done for you.)\n\n2. Set up a module that does a \"use Filter::Simple\" and then calls \"FILTER { ... }\".\n\n3. Within the anonymous subroutine or block that is passed to \"FILTER\", process the contents of\n$ to change the source code in the desired manner.\n\nIn other words, the previous example, would become:\n\npackage BANG;\nuse Filter::Simple;\n\nFILTER {\ns/BANG\\s+BANG/die 'BANG' if \\$BANG/g;\n};\n\n1 ;\n\nNote that the source code is passed as a single string, so any regex that uses \"^\" or \"$\" to\ndetect line boundaries will need the \"/m\" flag.\n\nDisabling or changing behaviour\nBy default, the installed filter only filters up to a line consisting of one of the three\nstandard source \"terminators\":\n\nno ModuleName; # optional comment\n\nor:\n\nEND\n\nor:\n\nDATA\n\nbut this can be altered by passing a second argument to \"use Filter::Simple\" or \"FILTER\" (just\nremember: there's *no* comma after the initial block when you use \"FILTER\").\n\nThat second argument may be either a \"qr\"'d regular expression (which is then used to match the\nterminator line), or a defined false value (which indicates that no terminator line should be\nlooked for), or a reference to a hash (in which case the terminator is the value associated with\nthe key 'terminator'.\n\nFor example, to cause the previous filter to filter only up to a line of the form:\n\nGNAB esu;\n\nyou would write:\n\npackage BANG;\nuse Filter::Simple;\n\nFILTER {\ns/BANG\\s+BANG/die 'BANG' if \\$BANG/g;\n}\nqr/^\\s*GNAB\\s+esu\\s*;\\s*?$/;\n\nor:\n\nFILTER {\ns/BANG\\s+BANG/die 'BANG' if \\$BANG/g;\n}\n{ terminator => qr/^\\s*GNAB\\s+esu\\s*;\\s*?$/ };\n\nand to prevent the filter's being turned off in any way:\n\npackage BANG;\nuse Filter::Simple;\n\nFILTER {\ns/BANG\\s+BANG/die 'BANG' if \\$BANG/g;\n}\n\"\"; # or: 0\n\nor:\n\nFILTER {\ns/BANG\\s+BANG/die 'BANG' if \\$BANG/g;\n}\n{ terminator => \"\" };\n\nNote that, no matter what you set the terminator pattern to, the actual terminator itself *must*\nbe contained on a single source line.\n" }, { "name": "All-in-one interface", "content": "Separating the loading of Filter::Simple:\n\nuse Filter::Simple;\n\nfrom the setting up of the filtering:\n\nFILTER { ... };\n\nis useful because it allows other code (typically parser support code or caching variables) to\nbe defined before the filter is invoked. However, there is often no need for such a separation.\n\nIn those cases, it is easier to just append the filtering subroutine and any terminator\nspecification directly to the \"use\" statement that loads Filter::Simple, like so:\n\nuse Filter::Simple sub {\ns/BANG\\s+BANG/die 'BANG' if \\$BANG/g;\n};\n\nThis is exactly the same as:\n\nuse Filter::Simple;\nBEGIN {\nFilter::Simple::FILTER {\ns/BANG\\s+BANG/die 'BANG' if \\$BANG/g;\n};\n}\n\nexcept that the \"FILTER\" subroutine is not exported by Filter::Simple.\n" }, { "name": "Filtering only specific components of source code", "content": "One of the problems with a filter like:\n\nuse Filter::Simple;\n\nFILTER { s/BANG\\s+BANG/die 'BANG' if \\$BANG/g };\n\nis that it indiscriminately applies the specified transformation to the entire text of your\nsource program. So something like:\n\nwarn 'BANG BANG, YOU'RE DEAD';\nBANG BANG;\n\nwill become:\n\nwarn 'die 'BANG' if $BANG, YOU'RE DEAD';\ndie 'BANG' if $BANG;\n\nIt is very common when filtering source to only want to apply the filter to the\nnon-character-string parts of the code, or alternatively to *only* the character strings.\n\nFilter::Simple supports this type of filtering by automatically exporting the \"FILTERONLY\"\nsubroutine.\n\n\"FILTERONLY\" takes a sequence of specifiers that install separate (and possibly multiple)\nfilters that act on only parts of the source code. For example:\n\nuse Filter::Simple;\n\nFILTERONLY\ncode => sub { s/BANG\\s+BANG/die 'BANG' if \\$BANG/g },\nquotelike => sub { s/BANG\\s+BANG/CHITTY CHITTY/g };\n\nThe \"code\" subroutine will only be used to filter parts of the source code that are not\nquotelikes, POD, or \"DATA\". The \"quotelike\" subroutine only filters Perl quotelikes\n(including here documents).\n\nThe full list of alternatives is:\n\n\"code\"\nFilters only those sections of the source code that are not quotelikes, POD, or \"DATA\".\n\n\"codenocomments\"\nFilters only those sections of the source code that are not quotelikes, POD, comments, or\n\"DATA\".\n\n\"executable\"\nFilters only those sections of the source code that are not POD or \"DATA\".\n\n\"executablenocomments\"\nFilters only those sections of the source code that are not POD, comments, or \"DATA\".\n\n\"quotelike\"\nFilters only Perl quotelikes (as interpreted by &Text::Balanced::extractquotelike).\n\n\"string\"\nFilters only the string literal parts of a Perl quotelike (i.e. the contents of a string\nliteral, either half of a \"tr///\", the second half of an \"s///\").\n\n\"regex\"\nFilters only the pattern literal parts of a Perl quotelike (i.e. the contents of a \"qr//\" or\nan \"m//\", the first half of an \"s///\").\n\n\"all\"\nFilters everything. Identical in effect to \"FILTER\".\n\nExcept for \"FILTERONLY code => sub {...}\", each of the component filters is called repeatedly,\nonce for each component found in the source code.\n\nNote that you can also apply two or more of the same type of filter in a single \"FILTERONLY\".\nFor example, here's a simple macro-preprocessor that is only applied within regexes, with a\nfinal debugging pass that prints the resulting source code:\n\nuse Regexp::Common;\nFILTERONLY\nregex => sub { s/!\\[/[^/g },\nregex => sub { s/%d/$RE{num}{int}/g },\nregex => sub { s/%f/$RE{num}{real}/g },\nall => sub { print if $::DEBUG };\n" }, { "name": "Filtering only the code parts of source code", "content": "Most source code ceases to be grammatically correct when it is broken up into the pieces between\nstring literals and regexes. So the 'code' and 'codenocomments' component filter behave\nslightly differently from the other partial filters described in the previous section.\n\nRather than calling the specified processor on each individual piece of code (i.e. on the bits\nbetween quotelikes), the 'code...' partial filters operate on the entire source code, but with\nthe quotelike bits (and, in the case of 'codenocomments', the comments) \"blanked out\".\n\nThat is, a 'code...' filter *replaces* each quoted string, quotelike, regex, POD, and DATA\nsection with a placeholder. The delimiters of this placeholder are the contents of the $;\nvariable at the time the filter is applied (normally \"\\034\"). The remaining four bytes are a\nunique identifier for the component being replaced.\n\nThis approach makes it comparatively easy to write code preprocessors without worrying about the\nform or contents of strings, regexes, etc.\n\nFor convenience, during a 'code...' filtering operation, Filter::Simple provides a package\nvariable ($Filter::Simple::placeholder) that contains a pre-compiled regex that matches any\nplaceholder...and captures the identifier within the placeholder. Placeholders can be moved and\nre-ordered within the source code as needed.\n\nIn addition, a second package variable (@Filter::Simple::components) contains a list of the\nvarious pieces of $, as they were originally split up to allow placeholders to be inserted.\n\nOnce the filtering has been applied, the original strings, regexes, POD, etc. are re-inserted\ninto the code, by replacing each placeholder with the corresponding original component (from\n@components). Note that this means that the @components variable must be treated with extreme\ncare within the filter. The @components array stores the \"back- translations\" of each\nplaceholder inserted into $, as well as the interstitial source code between placeholders. If\nthe placeholder backtranslations are altered in @components, they will be similarly changed when\nthe placeholders are removed from $ after the filter is complete.\n\nFor example, the following filter detects concatenated pairs of strings/quotelikes and reverses\nthe order in which they are concatenated:\n\npackage DemoRevCat;\nuse Filter::Simple;\n\nFILTERONLY code => sub {\nmy $ph = $Filter::Simple::placeholder;\ns{ ($ph) \\s* [.] \\s* ($ph) }{ $2.$1 }gx\n};\n\nThus, the following code:\n\nuse DemoRevCat;\n\nmy $str = \"abc\" . q(def);\n\nprint \"$str\\n\";\n\nwould become:\n\nmy $str = q(def).\"abc\";\n\nprint \"$str\\n\";\n\nand hence print:\n\ndefabc\n\nUsing Filter::Simple with an explicit \"import\" subroutine\nFilter::Simple generates a special \"import\" subroutine for your module (see \"How it works\")\nwhich would normally replace any \"import\" subroutine you might have explicitly declared.\n\nHowever, Filter::Simple is smart enough to notice your existing \"import\" and Do The Right Thing\nwith it. That is, if you explicitly define an \"import\" subroutine in a package that's using\nFilter::Simple, that \"import\" subroutine will still be invoked immediately after any filter you\ninstall.\n\nThe only thing you have to remember is that the \"import\" subroutine *must* be declared *before*\nthe filter is installed. If you use \"FILTER\" to install the filter:\n\npackage Filter::TurnItUpTo11;\n\nuse Filter::Simple;\n\nFILTER { s/(\\w+)/\\U$1/ };\n\nthat will almost never be a problem, but if you install a filtering subroutine by passing it\ndirectly to the \"use Filter::Simple\" statement:\n\npackage Filter::TurnItUpTo11;\n\nuse Filter::Simple sub{ s/(\\w+)/\\U$1/ };\n\nthen you must make sure that your \"import\" subroutine appears before that \"use\" statement.\n" }, { "name": "Using Filter::Simple and Exporter together", "content": "Likewise, Filter::Simple is also smart enough to Do The Right Thing if you use Exporter:\n\npackage Switch;\nuse base Exporter;\nuse Filter::Simple;\n\n@EXPORT = qw(switch case);\n@EXPORTOK = qw(given when);\n\nFILTER { $ = magicPerlfilter($) }\n\nImmediately after the filter has been applied to the source, Filter::Simple will pass control to\nExporter, so it can do its magic too.\n\nOf course, here too, Filter::Simple has to know you're using Exporter before it applies the\nfilter. That's almost never a problem, but if you're nervous about it, you can guarantee that\nthings will work correctly by ensuring that your \"use base Exporter\" always precedes your \"use\nFilter::Simple\".\n" }, { "name": "How it works", "content": "The Filter::Simple module exports into the package that calls \"FILTER\" (or \"use\"s it directly)\n-- such as package \"BANG\" in the above example -- two automagically constructed subroutines --\n\"import\" and \"unimport\" -- which take care of all the nasty details.\n\nIn addition, the generated \"import\" subroutine passes its own argument list to the filtering\nsubroutine, so the BANG.pm filter could easily be made parametric:\n\npackage BANG;\n\nuse Filter::Simple;\n\nFILTER {\nmy ($diemsg, $varname) = @;\ns/BANG\\s+BANG/die '$diemsg' if \\${$varname}/g;\n};\n\n# and in some user code:\n\nuse BANG \"BOOM\", \"BAM\"; # \"BANG BANG\" becomes: die 'BOOM' if $BAM\n\nThe specified filtering subroutine is called every time a \"use BANG\" is encountered, and passed\nall the source code following that call, up to either the next \"no BANG;\" (or whatever\nterminator you've set) or the end of the source file, whichever occurs first. By default, any\n\"no BANG;\" call must appear by itself on a separate line, or it is ignored.\n" } ] }, "AUTHOR": { "content": "Damian Conway\n", "subsections": [] }, "CONTACT": { "content": "Filter::Simple is now maintained by the Perl5-Porters. Please submit bug via the \"perlbug\" tool\nthat comes with your perl. For usage instructions, read \"perldoc perlbug\" or possibly \"man\nperlbug\". For mostly anything else, please contact .\n\nMaintainer of the CPAN release is Steffen Mueller . Contact him with\ntechnical difficulties with respect to the packaging of the CPAN module.\n\nPraise of the module, flowers, and presents still go to the author, Damian Conway\n.\n", "subsections": [] }, "COPYRIGHT AND LICENSE": { "content": "Copyright (c) 2000-2014, Damian Conway. All Rights Reserved.\nThis module is free software. It may be used, redistributed\nand/or modified under the same terms as Perl itself.\n", "subsections": [] } } } }