tokenizeAll

fnToken[] tokenizeAll(string input, string filename = "") @safe pure

Tokenizes the entire input and returns all tokens as an array.

Parameters

inputThe CF source text to tokenize
filenameOptional filename for error reporting

Returns

Array of all tokens (excluding EOF).