dyaml.scanner

YAML scanner. Code based on PyYAML: http://www.pyyaml.org

alias isBChar isBreak isBreakOrSpace isFlowScalarBreakSpace isNonLinebreakWhitespace isNonScalarStartCharacter isNSAnchorName isNSChar isURIChar isWhiteSpace
struct Scanner
fn insert

Types 11

aliasisBreak = among!('\0', '\n', '\r', '\u0085', '\u2028', '\u2029')

Scanner produces tokens of the following types: STREAM-START STREAM-END DIRECTIVE(name, value) DOCUMENT-START DOCUMENT-END BLOCK-SEQUENCE-START BLOCK-MAPPING-START BLOCK-END FLOW-SEQUENCE-START FLOW-MAPPING-START FLOW-SEQUENCE-END FLOW-MAPPING-END BLOCK-ENTRY FLOW-ENTRY KEY VALUE ALIAS(value) ANCHOR(value) TAG(value) SCALAR(value, plain, style)

aliasisBreakOrSpace = among!(' ', '\0', '\n', '\r', '\u0085', '\u2028', '\u2029')
aliasisWhiteSpace = among!(' ', '\t', '\0', '\n', '\r', '\u0085', '\u2028', '\u2029')
aliasisNonLinebreakWhitespace = among!(' ', '\t')
aliasisNonScalarStartCharacter = among!('-', '?', ':', ',', '[', ']', '{', '}', '#', '&', '*', '!', '|', '>', '\'', '"', '%', '@', '`', ' ', '\t', '\0', '\n', '\r', '\u0085', '\u2028', '\u2029')
aliasisURIChar = among!('-', ';', '/', '?', ':', '@', '&', '=', '+', '$', ',', '_', '.', '!', '~', '*', '\'', '(', ')', '[', ']', '%')
aliasisNSChar = among!(' ', '\n', '\r', '\u0085', '\u2028', '\u2029')
aliasisBChar = among!('\n', '\r', '\u0085', '\u2028', '\u2029')
aliasisFlowScalarBreakSpace = among!(' ', '\t', '\0', '\n', '\r', '\u0085', '\u2028', '\u2029', '\'', '"', '\\')
aliasisNSAnchorName = c => !c.isWhiteSpace && !c.among!('[', ']', '{', '}', ',', '\uFEFF')
structScanner

Generates tokens from data provided by a Reader.

Fields
private Reader reader_Reader used to read from a file/stream.
private bool done_Are we done scanning?
private uint flowLevel_Level of nesting in flow context. If 0, we're in block context.
private int indent_Current indentation level.
private Appender!(int[]) indents_Past indentation levels. Used as a stack.
private Queue!Token tokens_Processed tokens not yet emitted. Used as a queue.
private uint tokensTaken_Number of tokens emitted through the getToken method.
private bool allowSimpleKey_Can a simple key start at the current position? A simple key may start: - at the beginning of the line, not counting indentation spaces (in block context), - after '{', '[', ',' (in the flow contex...
private SimpleKey[] possibleSimpleKeys_Possible simple keys indexed by flow levels.
Methods
void popFront() @safeAdvance to the next token
const(Token) front() @safeReturn the current token
bool empty() @safeReturn whether there are any more tokens left.
inout(string) name() ref inout @safe return pure nothrow @nogcSet file name.
Mark mark() const @safe pure nothrow @nogcGet a mark from the current reader position
private string expected(T)(string expected, T found)Most scanning error messages have the same format; so build them with this function.
private bool needMoreTokens() @safe pureDetermine whether or not we need to fetch more tokens before peeking/getting a token.
private void fetchToken() @safeFetch at token, adding it to tokens_.
private uint nextPossibleSimpleKey() @safe pure nothrow @nogcReturn the token number of the nearest possible simple key.
private void stalePossibleSimpleKeys() @safe pureRemove entries that are no longer possible simple keys.
private void savePossibleSimpleKey() @safe pureCheck if the next token starts a possible simple key and if so, save its position.
private void removePossibleSimpleKey() @safe pureRemove the saved possible key position at the current flow level.
private void unwindIndent(const int column) @safeDecrease indentation, removing entries in indents_.
private bool addIndent(int column) @safeIncrease indentation if needed.
private void fetchStreamStart() @safe nothrowAdd STREAM-START token.
private void fetchStreamEnd() @safeAdd STREAM-END token.
private void fetchDirective() @safeAdd DIRECTIVE token.
private void fetchDocumentIndicator(TokenID id)() if(id == TokenID.documentStart || id == TokenID.documentEnd)Add DOCUMENT-START or DOCUMENT-END token.
private void fetchFlowCollectionStart(TokenID id)() @safeAdd FLOW-SEQUENCE-START or FLOW-MAPPING-START token.
private void fetchFlowCollectionEnd(TokenID id)()Add FLOW-SEQUENCE-START or FLOW-MAPPING-START token.
private void fetchFlowEntry() @safeAdd FLOW-ENTRY token;
private void blockChecks(string type, TokenID id)()Additional checks used in block context in fetchBlockEntry and fetchKey.
private void fetchBlockEntry() @safeAdd BLOCK-ENTRY token. Might add BLOCK-SEQUENCE-START in the process.
private void fetchKey() @safeAdd KEY token. Might add BLOCK-MAPPING-START in the process.
private void fetchValue() @safeAdd VALUE token. Might add KEY and/or BLOCK-MAPPING-START in the process.
private void fetchAnchor_(TokenID id)() if(id == TokenID.alias_ || id == TokenID.anchor) @safeAdd ALIAS or ANCHOR token.
private void fetchTag() @safeAdd TAG token.
private void fetchBlockScalar(ScalarStyle style)() if(style == ScalarStyle.literal || style == ScalarStyle.folded) @safeAdd block SCALAR token.
private void fetchFlowScalar(ScalarStyle quotes)()Add quoted flow SCALAR token.
private void fetchPlain() @safeAdd plain SCALAR token.
private bool checkDirective() pure @safeCheck if the next token is DIRECTIVE: ^ '%' ...
private bool checkDocumentStart() pure @safeCheck if the next token is DOCUMENT-START: ^ '---' (' '|'\n')
private bool checkDocumentEnd() pure @safeCheck if the next token is DOCUMENT-END: ^ '...' (' '|'\n')
private bool checkBlockEntry() pure @safeCheck if the next token is BLOCK-ENTRY: '-' (' '|'\n')
private bool checkKey() pure @safeCheck if the next token is KEY(flow context): '?'
private bool checkValue() pure @safeCheck if the next token is VALUE(flow context): ':'
private bool checkPlain() pure @safeCheck if the next token is a plain scalar.
private void findNextNonSpace() pure @safeMove to the next non-space character.
private void scanAlphaNumericToSlice(string name)(ref char[] slice, const Mark startMark) pureScan a string of alphanumeric or "-_" characters.
private char[] readAnchorAlias(const Mark startMark) pure @safeScan a string.
private void scanToNextBreak() pure @safeScan and throw away all characters until next line break.
private void scanToNextBreakToSlice(ref char[] slice) pure @safeScan all characters until next line break.
private void scanToNextToken() pure @safeMove to next token in the file/stream.
private Token scanDirective() pure @safeScan directive token.
private void scanDirectiveNameToSlice(ref char[] slice, const Mark startMark) pure @safeScan name of a directive token.
private void scanYAMLDirectiveValueToSlice(ref char[] slice, const Mark startMark) pure @safeScan value of a YAML directive token. Returns major, minor version separated by '.'.
private void scanYAMLDirectiveNumberToSlice(ref char[] slice, const Mark startMark) pure @safeScan a number from a YAML directive.
private uint scanTagDirectiveValueToSlice(ref char[] slice, const Mark startMark) pure @safeScan value of a tag directive.
private void scanTagDirectiveHandleToSlice(ref char[] slice, const Mark startMark) pure @safeScan handle of a tag directive.
private void scanTagDirectivePrefixToSlice(ref char[] slice, const Mark startMark) pure @safeScan prefix of a tag directive.
private void scanDirectiveIgnoredLine(const Mark startMark) pure @safeScan (and ignore) ignored line after a directive.
private Token scanAnchor(const TokenID id) pure @safeScan an alias or an anchor.
private Token scanTag() pure @safeScan a tag token.
private Token scanBlockScalar(const ScalarStyle style) pure @safeScan a block scalar token with specified style.
private Tuple!(Chomping, int) scanBlockScalarIndicators(const Mark startMark) pure @safeScan chomping and indentation indicators of a scalar token.
private bool getChomping(ref dchar c, ref Chomping chomping) pure @safeGet chomping indicator, if detected. Return false otherwise.
private bool getIncrement(ref dchar c, ref int increment, const Mark startMark) pure @safeGet increment indicator, if detected. Return false otherwise.
private void scanBlockScalarIgnoredLine(const Mark startMark) pure @safeScan (and ignore) ignored line in a block scalar.
private Tuple!(uint, Mark) scanBlockScalarIndentationToSlice(ref char[] slice) pure @safeScan indentation in a block scalar, returning line breaks, max indent and end mark.
private Mark scanBlockScalarBreaksToSlice(ref char[] slice, const uint indent) pure @safeScan line breaks at lower or specified indentation in a block scalar.
private Token scanFlowScalar(const ScalarStyle quotes) pure @safeScan a qouted flow scalar token with specified quotes.
private void scanFlowScalarNonSpacesToSlice(ref char[] slice, const ScalarStyle quotes, const Mark startMark) pure @safeScan nonspace characters in a flow scalar.
private void scanFlowScalarSpacesToSlice(ref char[] slice, const Mark startMark) pure @safeScan space characters in a flow scalar.
private bool scanFlowScalarBreaksToSlice(ref char[] slice, const Mark startMark) pure @safeScan line breaks in a flow scalar.
private Token scanPlain() pure @safeScan plain scalar token (no block, no quotes).
private void scanPlainSpacesToSlice(ref char[] slice) pure @trustedScan spaces in a plain scalar.
private void scanTagHandleToSlice(string name)(ref char[] slice, const Mark startMark) pureScan handle of a tag token.
private void scanTagURIToSlice(string name)(ref char[] slice, const Mark startMark) pureScan URI in a tag token.
private void scanURIEscapesToSlice(string name)(ref char[] slice, const Mark startMark) pureScan URI escape sequences.
private dchar scanLineBreak() pure @safeScan a line break, if any.
Constructors
this(Reader reader)Construct a Scanner using specified Reader.
Nested Templates
SimpleKeyA simple key is a key that is not denoted by the '?' indicator. For example:
ChompingBlock chomping types.

Functions 1

private fnvoid insert(ref char[] slice, const dchar c, const size_t position) @safe pure