Plucene::Analysis::CharTokenizer - phpMan

Plucene::Analysis::CharTokenizeUsermContributed Perl DocumenPlucene::Analysis::CharTokenizer(3pm)

NAME
       Plucene::Analysis::CharTokenizer - base class for character tokenisers

SYNOPSIS
               # isa Plucene::Analysis::Tokenizer

               my $next = $chartokenizer->next;

DESCRIPTION
       This is an abstract base class for simple, character-oriented tokenizers.

METHODS
   token_re
       This should be defined in subclasses.

   normalize
       This will normalise the character before it is added to the token.

   next
               my $next = $chartokenizer->next;

       This will return the next token in the string, or undef at the end of the string.

perl v5.26.1                                2018-04-02      Plucene::Analysis::CharTokenizer(3pm)

Generated by $Id: phpMan.php,v 4.55 2007/09/05 04:42:51 chedong Exp $ Author: Che Dong
On Apache
Under GNU General Public License
2025-11-21 17:45 @216.73.216.164 CrawledBy Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)