Lexers (also called tokenizers) can be used to prepare a string or stream of character data into a form suitable as an input for parsers generated by parser generators.

  • cl-lex - cl-lex is a set of Common Lisp macros for generating lexical analyzers automatically
  • DEFLEXER - The LEXER package implements a lexical-analyzer-generator called DEFLEXER, which is built on top of both REGEX and CLAWK
  • graylex - graylex offers a means to do string operations on input streams without slurping all input at once by using Common Lisp Gray Streams, fixed-sized and flexible buffers
  • Zebu - A Tool for Specifying Reversible LALR(1) Parsers