{
    "mode": "perldoc",
    "parameter": "HTML::TokeParser::Simple",
    "section": "",
    "url": "https://www.chedong.com/phpMan.php/perldoc/HTML%3A%3ATokeParser%3A%3ASimple/json",
    "generated": "2026-06-11T10:32:32Z",
    "synopsis": "use HTML::TokeParser::Simple;\nmy $p = HTML::TokeParser::Simple->new( $somefile );\nwhile ( my $token = $p->gettoken ) {\n# This prints all text in an HTML doc (i.e., it strips the HTML)\nnext unless $token->istext;\nprint $token->asis;\n}",
    "sections": {
        "NAME": {
            "content": "HTML::TokeParser::Simple - Easy to use \"HTML::TokeParser\" interface\n",
            "subsections": []
        },
        "SYNOPSIS": {
            "content": "use HTML::TokeParser::Simple;\nmy $p = HTML::TokeParser::Simple->new( $somefile );\n\nwhile ( my $token = $p->gettoken ) {\n# This prints all text in an HTML doc (i.e., it strips the HTML)\nnext unless $token->istext;\nprint $token->asis;\n}\n",
            "subsections": []
        },
        "DESCRIPTION": {
            "content": "\"HTML::TokeParser\" is an excellent module that's often used for parsing HTML. However, the\ntokens returned are not exactly intuitive to parse:\n\n[\"S\",  $tag, $attr, $attrseq, $text]\n[\"E\",  $tag, $text]\n[\"T\",  $text, $isdata]\n[\"C\",  $text]\n[\"D\",  $text]\n[\"PI\", $token0, $text]\n\nTo simplify this, \"HTML::TokeParser::Simple\" allows the user ask more intuitive (read: more\nself-documenting) questions about the tokens returned.\n\nYou can also rebuild some tags on the fly. Frequently, the attributes associated with start tags\nneed to be altered, added to, or deleted. This functionality is built in.\n\nSince this is a subclass of \"HTML::TokeParser\", all \"HTML::TokeParser\" methods are available. To\ntruly appreciate the power of this module, please read the documentation for \"HTML::TokeParser\"\nand \"HTML::Parser\".\n",
            "subsections": []
        },
        "CONTRUCTORS": {
            "content": "\"new($source)\"\nThe constructor for \"HTML::TokeParser::Simple\" can be used just like \"HTML::TokeParser\"'s\nconstructor:\n\nmy $parser = HTML::TokeParser::Simple->new($filename);\n# or\nmy $parser = HTML::TokeParser::Simple->new($filehandle);\n# or\nmy $parser = HTML::TokeParser::Simple->new(\\$htmlstring);\n\n\"new($sourcetype, $source)\"\nIf you wish to be more explicit, there is a new style of constructor available.\n\nmy $parser = HTML::TokeParser::Simple->new(file   => $filename);\n# or\nmy $parser = HTML::TokeParser::Simple->new(handle => $filehandle);\n# or\nmy $parser = HTML::TokeParser::Simple->new(string => $htmlstring);\n\nNote that you do not have to provide a reference for the string if using the string constructor.\n\nAs a convenience, you can also attempt to fetch the HTML directly from a URL.\n\nmy $parser = HTML::TokeParser::Simple->new(url => 'http://some.url');\n\nThis method relies on \"LWP::Simple\". If this module is not found or the page cannot be fetched,\nthe constructor will \"croak()\".\n",
            "subsections": []
        },
        "PARSER METHODS": {
            "content": "gettoken\nThis method will return the next token that \"HTML::TokeParser::gettoken()\" method would return.\nHowever, it will be blessed into a class appropriate which represents the token type.\n\ngettag\nThis method will return the next token that \"HTML::TokeParser::gettag()\" method would return.\nHowever, it will be blessed into either the HTML::TokeParser::Simple::Token::Tag::Start or\nHTML::TokeParser::Simple::Token::Tag::End class.\n\npeek\nAs of version 3.14, you can now \"peek()\" at the upcomings tokens without affecting the state of\nthe parser. By default, \"peek()\" will return the text of the next token, but specifying an\ninteger $count will return the text of the next $count tokens.\n\nThis is useful when you're trying to debug where you are in a document.\n\nwarn $parser->peek(3); # show the next 3 tokens\n",
            "subsections": []
        },
        "ACCESSORS": {
            "content": "The following methods may be called on the token object which is returned, not on the parser\nobject.\n",
            "subsections": [
                {
                    "name": "Boolean Accessors",
                    "content": "These accessors return true or false.\n\n*   \"istag([$tag])\"\n\nUse this to determine if you have any tag. An optional \"tag type\" may be passed. This will\nallow you to match if it's a *particular* tag. The supplied tag is case-insensitive.\n\nif ( $token->istag ) { ... }\n\nOptionally, you may pass a regular expression as an argument.\n\n*   \"isstarttag([$tag])\"\n\nUse this to determine if you have a start tag. An optional \"tag type\" may be passed. This\nwill allow you to match if it's a *particular* start tag. The supplied tag is\ncase-insensitive.\n\nif ( $token->isstarttag ) { ... }\nif ( $token->isstarttag( 'font' ) ) { ... }\n\nOptionally, you may pass a regular expression as an argument. To match all header (h1, h2,\n... h6) tags:\n\nif ( $token->isstarttag( qr/^h[123456]$/ ) ) { ... }\n\n*   \"isendtag([$tag])\"\n\nUse this to determine if you have an end tag. An optional \"tag type\" may be passed. This\nwill allow you to match if it's a *particular* end tag. The supplied tag is\ncase-insensitive.\n\nWhen testing for an end tag, the forward slash on the tag is optional.\n\nwhile ( $token = $p->gettoken ) {\nif ( $token->isendtag( 'form' ) ) { ... }\n}\n\nOr:\n\nwhile ( $token = $p->gettoken ) {\nif ( $token->isendtag( '/form' ) ) { ... }\n}\n\nOptionally, you may pass a regular expression as an argument.\n\n*   \"istext()\"\n\nUse this to determine if you have text. Note that this is *not* to be confused with the\n\"returntext\" (*deprecated*) method described below! \"istext\" will identify text that the\nuser typically sees display in the Web browser.\n\n*   \"iscomment()\"\n\nAre you still reading this? Nobody reads POD. Don't you know you're supposed to go to CLPM,\nask a question that's answered in the POD and get flamed? It's a rite of passage.\n\nReally.\n\n\"iscomment\" is used to identify comments. See the HTML::Parser documentation for more\ninformation about comments. There's more than you might think.\n\n*   \"isdeclaration()\"\n\nThis will match the DTD at the top of your HTML. (You *do* use DTD's, don't you?)\n\n*   \"isprocessinstruction()\"\n\nProcess Instructions are from XML. This is very handy if you need to parse out PHP and\nsimilar things with a parser.\n\nCurrently, there appear to be some problems with process instructions. You can override\n\"HTML::TokeParser::Simple::Token::ProcessInstruction\" if you need to.\n\n*   \"ispi()\"\n\nThis is a shorthand for \"isprocessinstruction()\".\n"
                },
                {
                    "name": "Data Accessors",
                    "content": "Some of these were originally \"return\" methods, but that name was not only unwieldy, but also\nwent against reasonable conventions. The \"get\" methods listed below still have \"return\"\nmethods available for backwards compatibility reasons, but they merely call their \"get\"\ncounterpart. For example, calling \"returntag()\" actually calls \"gettag()\" internally.\n\n*   \"gettag()\"\n\nDo you have a start tag or end tag? This will return the type (lower case). Note that this\nis *not* the same as the \"gettag()\" method on the actual parser object.\n\n*   \"getattr([$attribute])\"\n\nIf you have a start tag, this will return a hash ref with the attribute names as keys and\nthe values as the values.\n\nIf you pass in an attribute name, it will return the value for just that attribute.\n\nReturns false if the token is not a start tag.\n\n*   \"getattrseq()\"\n\nFor a start tag, this is an array reference with the sequence of the attributes, if any.\n\nReturns false if the token is not a start tag.\n\n*   \"returntext()\"\n\nThis method has been heavily deprecated (for a couple of years) in favor of \"asis\".\nProgrammers were getting confused over the difference between \"istext\", \"returntext\", and\nsome parser methods such as \"HTML::TokeParser::gettext\" and friends.\n\nUsing this method still succeeds, but will now carp and will be removed in the next major\nrelease of this module.\n\n*   \"asis()\"\n\nThis is the exact text of whatever the token is representing.\n\n*   \"gettoken0()\"\n\nFor processing instructions, this will return the token found immediately after the opening\ntag. Example: For <?php, \"php\" will be the start of the returned string.\n\nNote that process instruction handling appears to be incomplete in \"HTML::TokeParser\".\n\nReturns false if the token is not a process instruction.\n"
                }
            ]
        },
        "MUTATORS": {
            "content": "The \"deleteattr()\" and \"setattr()\" methods allow the programmer to rewrite start tag\nattributes on the fly. It should be noted that bad HTML will be \"corrected\" by this.\nSpecifically, the new tag will have all attributes lower-cased with the values properly quoted.\n\nSelf-closing tags (e.g. <hr />) are also handled correctly. Some older browsers require a space\nprior to the final slash in a self-closed tag. If such a space is detected in the original HTML,\nit will be preserved.\n\nCalling a mutator on an token type that does not support that property is a no-op. For example:\n\nif ($token->iscomment) {\n$token->setattr(foo => 'bar'); # does nothing\n}\n\n*   \"deleteattr($name)\"\n\nThis method attempts to delete the attribute specified. It will silently fail if called on\nanything other than a start tag. The argument is case-insensitive, but must otherwise be an\nexact match of the attribute you are attempting to delete. If the attribute is not found,\nthe method will return without changing the tag.\n\n# <body bgcolor=\"#FFFFFF\">\n$token->deleteattr('bgcolor');\nprint $token->asis;\n# <body>\n\nAfter this method is called, if successful, the \"asis()\", \"getattr()\" and \"getattrseq()\"\nmethods will all return updated results.\n\n*   \"setattr($name,$value)\"\n\nThis method will set the value of an attribute. If the attribute is not found, then\n\"getattrseq()\" will have the new attribute listed at the end.\n\n# <p>\n$token->setattr(class => 'someclass');\nprint $token->asis;\n# <p class=\"someclass\">\n\n# <body bgcolor=\"#FFFFFF\">\n$token->setattr('bgcolor','red');\nprint $token->asis;\n# <body bgcolor=\"red\">\n\nAfter this method is called, if successful, the \"asis()\", \"getattr()\" and \"getattrseq()\"\nmethods will all return updated results.\n\n*   \"setattr($hashref)\"\n\nUnder the premise that \"set\" methods should accept what their corresponding \"get\" methods\nemit, the following works:\n\n$tag->setattr($tag->getattr);\n\nTheoretically that's a no-op and for purposes of rendering HTML, it should be. However,\ninternally this calls \"$tag->rewritetag\", so see that method to understand how this may\naffect you.\n\nOf course, this is useless if you want to actually change the attributes, so you can do\nthis:\n\nmy $attrs = {\nclass  => 'headline',\nvalign => 'top'\n};\n$token->setattr($attrs)\nif $token->isstarttag('td') &&  $token->getattr('class') eq 'stories';\n\n*   \"rewritetag()\"\n\nThis method rewrites the tag. The tag name and the name of all attributes will be\nlower-cased. Values that are not quoted with double quotes will be. This may be called on\nboth start or end tags. Note that both \"setattr()\" and \"deleteattr()\" call this method\nprior to returning.\n\nIf called on a token that is not a tag, it simply returns. Regardless of how it is called,\nit returns the token.\n\n# <body alink=#0000ff BGCOLOR=#ffffff class='none'>\n$token->rewritetag;\nprint $token->asis;\n# <body alink=\"#0000ff\" bgcolor=\"#ffffff\" class=\"none\">\n\nA quick cleanup of sloppy HTML is now the following:\n\nmy $parser = HTML::TokeParser::Simple->new( string => $uglyhtml );\nwhile (my $token = $parser->gettoken) {\n$token->rewritetag;\nprint $token->asis;\n}\n",
            "subsections": []
        },
        "PARSER VERSUS TOKENS": {
            "content": "The parser returns tokens that are blessed into appropriate classes. Some people get confused\nand try to call parser methods on tokens and token methods on the parser. To prevent this,\n\"HTML::TokeParser::Simple\" versions 1.4 and above now bless all tokens into appropriate token\nclasses. Please keep this in mind while using this module (and many thanks to PodMaster\n<http://www.perlmonks.org/index.pl?nodeid=107642> for pointing out this issue to me.)\n",
            "subsections": []
        },
        "EXAMPLES": {
            "content": "",
            "subsections": [
                {
                    "name": "Finding comments",
                    "content": "For some strange reason, your Pointy-Haired Boss (PHB) is convinced that the graphics department\nis making fun of him by embedding rude things about him in HTML comments. You need to get all\nHTML comments from the HTML.\n\nuse strict;\nuse HTML::TokeParser::Simple;\n\nmy @htmldocs = glob( \"*.html\" );\n\nopen PHB, \"> phbreport.txt\" or die \"Cannot open phbreport for writing: $!\";\n\nforeach my $doc ( @htmldocs ) {\nprint \"Processing $doc\\n\";\nmy $p = HTML::TokeParser::Simple->new( file => $doc );\nwhile ( my $token = $p->gettoken ) {\nnext unless $token->iscomment;\nprint PHB $token->asis, \"\\n\";\n}\n}\n\nclose PHB;\n"
                },
                {
                    "name": "Stripping Comments",
                    "content": "Uh oh. Turns out that your PHB was right for a change. Many of the comments in the HTML weren't\nvery polite. Since your entire graphics department was just fired, it falls on you need to strip\nthose comments from the HTML.\n\nuse strict;\nuse HTML::TokeParser::Simple;\n\nmy $newfolder = 'nocomment/';\nmy @htmldocs  = glob( \"*.html\" );\n\nforeach my $doc ( @htmldocs ) {\nprint \"Processing $doc\\n\";\nmy $newfile = \"$newfolder$doc\";\n\nopen PHB, \"> $newfile\" or die \"Cannot open $newfile for writing: $!\";\n\nmy $p = HTML::TokeParser::Simple->new( $file => doc );\nwhile ( my $token = $p->gettoken ) {\nnext if $token->iscomment;\nprint PHB $token->asis;\n}\nclose PHB;\n}\n"
                },
                {
                    "name": "Changing form tags",
                    "content": "Your company was foo.com and now is bar.com. Unfortunately, whoever wrote your HTML decided to\nhardcode \"http://www.foo.com/\" into the \"action\" attribute of the form tags. You need to change\nit to \"http://www.bar.com/\".\n\nuse strict;\nuse HTML::TokeParser::Simple;\n\nmy $newfolder = 'newhtml/';\nmy @htmldocs  = glob( \"*.html\" );\n\nforeach my $doc ( @htmldocs ) {\nprint \"Processing $doc\\n\";\nmy $newfile = \"$newfolder$doc\";\n\nopen FILE, \"> $newfile\" or die \"Cannot open $newfile for writing: $!\";\n\nmy $p = HTML::TokeParser::Simple->new( file => $doc );\nwhile ( my $token = $p->gettoken ) {\nif ( $token->isstarttag('form') ) {\nmy $action = $token->getattr(action);\n$action =~ s/www\\.foo\\.com/www.bar.com/;\n$token->setattr('action', $action);\n}\nprint FILE $token->asis;\n}\nclose FILE;\n}\n"
                }
            ]
        },
        "CAVEATS": {
            "content": "For compatibility reasons with \"HTML::TokeParser\", methods that return references are violating\nencapsulation and altering the references directly will alter the state of the object.\nSubsequent calls to \"rewritetag()\" can thus have unexpected results. Do not alter these\nreferences directly unless you are following behavior described in these docs. In the future,\ncertain methods such as \"getattr\", \"getattrseq\" and others may return a copy of the reference\nrather than the original reference. This behavior has not yet been changed in order to maintain\ncompatibility with previous versions of this module. At the present time, your author is not\naware of anyone taking advantage of this \"feature,\" but it's better to be safe than sorry.\n\nUse of $HTML::Parser::VERSION which is less than 3.25 may result in incorrect behavior as older\nversions do not always handle XHTML correctly. It is the programmer's responsibility to verify\nthat the behavior of this code matches the programmer's needs.\n\nNote that \"HTML::Parser\" processes text in 512 byte chunks. This sometimes will cause strange\nbehavior and cause text to be broken into more than one token. You can suppress this behavior\nwith the following command:\n\n$p->unbrokentext( [$bool] );\n\nSee the \"HTML::Parser\" documentation and http://www.perlmonks.org/index.pl?nodeid=230667 for\nmore information.\n",
            "subsections": []
        },
        "BUGS": {
            "content": "There are no known bugs, but that's no guarantee.\n\nAddress bug reports and comments to: <eopdivositruc@yahoo.com>. When sending bug reports,\nplease provide the version of \"HTML::Parser\", \"HTML::TokeParser\", \"HTML::TokeParser::Simple\",\nthe version of Perl, and the version of the operating system you are using.\n\nReverse the name to email the author.\n",
            "subsections": []
        },
        "SUBCLASSING": {
            "content": "You may wish to change the behavior of this module. You probably do not want to subclass\n\"HTML::TokeParser::Simple\". Instead, you'll want to subclass one of the token classes.\n\"HTML::TokeParser::Simple::Token\" is the base class for all tokens. Global behavioral changes\nshould go there. Otherwise, see the appropriate token class for the behavior you wish to alter.\n",
            "subsections": []
        },
        "SEE ALSO": {
            "content": "HTML::TokeParser::Simple::Token\n\nHTML::TokeParser::Simple::Token::Tag\n\nHTML::TokeParser::Simple::Token::Text\n\nHTML::TokeParser::Simple::Token::Comment\n\nHTML::TokeParser::Simple::Token::Declaration\n\nHTML::TokeParser::Simple::Token::ProcessInstruction\n",
            "subsections": []
        },
        "COPYRIGHT": {
            "content": "Copyright (c) 2004 by Curtis \"Ovid\" Poe. All rights reserved. This program is free software; you\nmay redistribute it and/or modify it under the same terms as Perl itself\n",
            "subsections": []
        },
        "AUTHOR": {
            "content": "Curtis \"Ovid\" Poe <eopdivositruc@yahoo.com>\n\nReverse the name to email the author.\n",
            "subsections": []
        }
    },
    "summary": "HTML::TokeParser::Simple - Easy to use \"HTML::TokeParser\" interface",
    "flags": [],
    "examples": [
        "For some strange reason, your Pointy-Haired Boss (PHB) is convinced that the graphics department",
        "is making fun of him by embedding rude things about him in HTML comments. You need to get all",
        "HTML comments from the HTML.",
        "use strict;",
        "use HTML::TokeParser::Simple;",
        "my @htmldocs = glob( \"*.html\" );",
        "open PHB, \"> phbreport.txt\" or die \"Cannot open phbreport for writing: $!\";",
        "foreach my $doc ( @htmldocs ) {",
        "print \"Processing $doc\\n\";",
        "my $p = HTML::TokeParser::Simple->new( file => $doc );",
        "while ( my $token = $p->gettoken ) {",
        "next unless $token->iscomment;",
        "print PHB $token->asis, \"\\n\";",
        "close PHB;",
        "Uh oh. Turns out that your PHB was right for a change. Many of the comments in the HTML weren't",
        "very polite. Since your entire graphics department was just fired, it falls on you need to strip",
        "those comments from the HTML.",
        "use strict;",
        "use HTML::TokeParser::Simple;",
        "my $newfolder = 'nocomment/';",
        "my @htmldocs  = glob( \"*.html\" );",
        "foreach my $doc ( @htmldocs ) {",
        "print \"Processing $doc\\n\";",
        "my $newfile = \"$newfolder$doc\";",
        "open PHB, \"> $newfile\" or die \"Cannot open $newfile for writing: $!\";",
        "my $p = HTML::TokeParser::Simple->new( $file => doc );",
        "while ( my $token = $p->gettoken ) {",
        "next if $token->iscomment;",
        "print PHB $token->asis;",
        "close PHB;",
        "Your company was foo.com and now is bar.com. Unfortunately, whoever wrote your HTML decided to",
        "hardcode \"http://www.foo.com/\" into the \"action\" attribute of the form tags. You need to change",
        "it to \"http://www.bar.com/\".",
        "use strict;",
        "use HTML::TokeParser::Simple;",
        "my $newfolder = 'newhtml/';",
        "my @htmldocs  = glob( \"*.html\" );",
        "foreach my $doc ( @htmldocs ) {",
        "print \"Processing $doc\\n\";",
        "my $newfile = \"$newfolder$doc\";",
        "open FILE, \"> $newfile\" or die \"Cannot open $newfile for writing: $!\";",
        "my $p = HTML::TokeParser::Simple->new( file => $doc );",
        "while ( my $token = $p->gettoken ) {",
        "if ( $token->isstarttag('form') ) {",
        "my $action = $token->getattr(action);",
        "$action =~ s/www\\.foo\\.com/www.bar.com/;",
        "$token->setattr('action', $action);",
        "print FILE $token->asis;",
        "close FILE;"
    ],
    "see_also": []
}