# RDoc::TokenStream - ri - phpman

= [**RDoc::TokenStream**](https://www.chedong.com/phpMan.php/perldoc/RDoc%3A%3ATokenStream/markdown)

(from gem rdoc-7.2.0)
------------------------------------------------------------------------
A TokenStream is a list of tokens, gathered during the parse of some
entity (say a method). Entities populate these streams by being
registered with the lexer. Any class can collect tokens by including
## TokenStream. From the outside, you use such an object by calling the
start_collecting_tokens method, followed by calls to add_token and
pop_token.
------------------------------------------------------------------------
= **Class methods:**

  to_html

= **Instance methods:**

  add_token
  add_tokens
  collect_tokens
  pop_token
  source_language
  start_collecting_tokens
  token_stream
  tokens_to_s