2007-04-21

Writing a parser: basic terminology

Programming code (or any code that should be parsed) is usually plain text. It's just a sequence of characters. A tokenizer converts this sequence of characters into a sequence of tokens. A token is a group of characters that form a meaningful unit.

Converting characters into tokens.

The parser analyzes this sequence of tokens and transforms it into a tree of operators (primitive functions) and operands (the arguments of the operators).

Transforming tokens into a tree of operators and operands.

Technorati tags: , .
Comments:
Great job..! Kudos to you for blogging on this very important topic. Am sure it would have very helpful to many.
Thanks to you.
 
Post a Comment

Links to this post:

Create a Link