tokenize
Creates a lexer for the given CF source text.
This is a convenience function that constructs a Lexer range.
Parameters
input | The CF source text to tokenize |
filename | Optional filename for error reporting |
Returns
A Lexer input range that yields tokens.