dyaml.scanner

YAML scanner. Code based on PyYAML: http://www.pyyaml.org

alias isBChar isBreak isBreakOrSpace isFlowScalarBreakSpace isNonLinebreakWhitespace isNonScalarStartCharacter isNSAnchorName isNSChar isURIChar isWhiteSpace

struct Scanner

fn insert

Types 11

aliasisBreak = among!('\0', '\n', '\r', '\u0085', '\u2028', '\u2029')

Scanner produces tokens of the following types: STREAM-START STREAM-END DIRECTIVE(name, value) DOCUMENT-START DOCUMENT-END BLOCK-SEQUENCE-START BLOCK-MAPPING-START BLOCK-END FLOW-SEQUENCE-START FLOW-MAPPING-START FLOW-SEQUENCE-END FLOW-MAPPING-END BLOCK-ENTRY FLOW-ENTRY KEY VALUE ALIAS(value) ANCHOR(value) TAG(value) SCALAR(value, plain, style)

aliasisBreakOrSpace = among!(' ', '\0', '\n', '\r', '\u0085', '\u2028', '\u2029')

aliasisWhiteSpace = among!(' ', '\t', '\0', '\n', '\r', '\u0085', '\u2028', '\u2029')

aliasisNonLinebreakWhitespace = among!(' ', '\t')

aliasisNonScalarStartCharacter = among!('-', '?', ':', ',', '[', ']', '{', '}', '#', '&', '*', '!', '|', '>', '\'', '"', '%', '@', '`', ' ', '\t', '\0', '\n', '\r', '\u0085', '\u2028', '\u2029')

aliasisURIChar = among!('-', ';', '/', '?', ':', '@', '&', '=', '+', '$', ',', '_', '.', '!', '~', '*', '\'', '(', ')', '[', ']', '%')

aliasisNSChar = among!(' ', '\n', '\r', '\u0085', '\u2028', '\u2029')

aliasisBChar = among!('\n', '\r', '\u0085', '\u2028', '\u2029')

aliasisFlowScalarBreakSpace = among!(' ', '\t', '\0', '\n', '\r', '\u0085', '\u2028', '\u2029', '\'', '"', '\\')

aliasisNSAnchorName = c => !c.isWhiteSpace && !c.among!('[', ']', '{', '}', ',', '\uFEFF')

structScanner

Generates tokens from data provided by a Reader.

Fields

Reader reader_Reader used to read from a file/stream.

bool done_Are we done scanning?

uint flowLevel_Level of nesting in flow context. If 0, we're in block context.

int indent_Current indentation level.

Appender!(int[]) indents_Past indentation levels. Used as a stack.

Queue!Token tokens_Processed tokens not yet emitted. Used as a queue.

uint tokensTaken_Number of tokens emitted through the getToken method.

bool allowSimpleKey_Can a simple key start at the current position? A simple key may start: - at the beginning of the line, not counting indentation spaces (in block context), - after '{', '[', ',' (in the flow contex...

SimpleKey[] possibleSimpleKeys_Possible simple keys indexed by flow levels.

Methods

void popFront() @safeAdvance to the next token

const(Token) front() @safeReturn the current token

bool empty() @safeReturn whether there are any more tokens left.

inout(string) name() ref inout @safe return pure nothrow @nogcSet file name.

Mark mark() const @safe pure nothrow @nogcGet a mark from the current reader position

string expected(T)(string expected, T found)Most scanning error messages have the same format; so build them with this function.

bool needMoreTokens() @safe pureDetermine whether or not we need to fetch more tokens before peeking/getting a token.

void fetchToken() @safeFetch at token, adding it to tokens_.

uint nextPossibleSimpleKey() @safe pure nothrow @nogcReturn the token number of the nearest possible simple key.

void stalePossibleSimpleKeys() @safe pureRemove entries that are no longer possible simple keys.

void savePossibleSimpleKey() @safe pureCheck if the next token starts a possible simple key and if so, save its position.

void removePossibleSimpleKey() @safe pureRemove the saved possible key position at the current flow level.

void unwindIndent(const int column) @safeDecrease indentation, removing entries in indents_.

bool addIndent(int column) @safeIncrease indentation if needed.

void fetchStreamStart() @safe nothrowAdd STREAM-START token.

void fetchStreamEnd() @safeAdd STREAM-END token.

void fetchDirective() @safeAdd DIRECTIVE token.

void fetchDocumentIndicator(TokenID id)() if(id == TokenID.documentStart || id == TokenID.documentEnd)Add DOCUMENT-START or DOCUMENT-END token.

void fetchFlowCollectionStart(TokenID id)() @safeAdd FLOW-SEQUENCE-START or FLOW-MAPPING-START token.

void fetchFlowCollectionEnd(TokenID id)()Add FLOW-SEQUENCE-START or FLOW-MAPPING-START token.

void fetchFlowEntry() @safeAdd FLOW-ENTRY token;

void blockChecks(string type, TokenID id)()Additional checks used in block context in fetchBlockEntry and fetchKey.

void fetchBlockEntry() @safeAdd BLOCK-ENTRY token. Might add BLOCK-SEQUENCE-START in the process.

void fetchKey() @safeAdd KEY token. Might add BLOCK-MAPPING-START in the process.

void fetchValue() @safeAdd VALUE token. Might add KEY and/or BLOCK-MAPPING-START in the process.

void fetchAnchor_(TokenID id)() if(id == TokenID.alias_ || id == TokenID.anchor) @safeAdd ALIAS or ANCHOR token.

void fetchTag() @safeAdd TAG token.

void fetchBlockScalar(ScalarStyle style)() if(style == ScalarStyle.literal || style == ScalarStyle.folded) @safeAdd block SCALAR token.

void fetchFlowScalar(ScalarStyle quotes)()Add quoted flow SCALAR token.

void fetchPlain() @safeAdd plain SCALAR token.

bool checkDirective() @safeCheck if the next token is DIRECTIVE: ^ '%' ...

bool checkDocumentStart() @safeCheck if the next token is DOCUMENT-START: ^ '---' (' '|'\n')

bool checkDocumentEnd() @safeCheck if the next token is DOCUMENT-END: ^ '...' (' '|'\n')

bool checkBlockEntry() @safeCheck if the next token is BLOCK-ENTRY: '-' (' '|'\n')

bool checkKey() @safeCheck if the next token is KEY(flow context): '?'

bool checkValue() @safeCheck if the next token is VALUE(flow context): ':'

bool checkPlain() @safeCheck if the next token is a plain scalar.

void findNextNonSpace() @safeMove to the next non-space character.

void scanAlphaNumericToSlice(string name)(ref char[] slice, const Mark startMark)Scan a string of alphanumeric or "-_" characters.

char[] readAnchorAlias(const Mark startMark) @safeScan a string.

void scanToNextBreak() @safeScan and throw away all characters until next line break.

void scanToNextBreakToSlice(ref char[] slice) @safeScan all characters until next line break.

void scanToNextToken() @safeMove to next token in the file/stream.

Token scanDirective() @safeScan directive token.

void scanDirectiveNameToSlice(ref char[] slice, const Mark startMark) @safeScan name of a directive token.

void scanYAMLDirectiveValueToSlice(ref char[] slice, const Mark startMark) @safeScan value of a YAML directive token. Returns major, minor version separated by '.'.

void scanYAMLDirectiveNumberToSlice(ref char[] slice, const Mark startMark) @safeScan a number from a YAML directive.

uint scanTagDirectiveValueToSlice(ref char[] slice, const Mark startMark) @safeScan value of a tag directive.

void scanTagDirectiveHandleToSlice(ref char[] slice, const Mark startMark) @safeScan handle of a tag directive.

void scanTagDirectivePrefixToSlice(ref char[] slice, const Mark startMark) @safeScan prefix of a tag directive.

void scanDirectiveIgnoredLine(const Mark startMark) @safeScan (and ignore) ignored line after a directive.

Token scanAnchor(const TokenID id) @safeScan an alias or an anchor.

Token scanTag() @safeScan a tag token.

Token scanBlockScalar(const ScalarStyle style) @safeScan a block scalar token with specified style.

Tuple!(Chomping, int) scanBlockScalarIndicators(const Mark startMark) @safeScan chomping and indentation indicators of a scalar token.

bool getChomping(ref dchar c, ref Chomping chomping) @safeGet chomping indicator, if detected. Return false otherwise.

bool getIncrement(ref dchar c, ref int increment, const Mark startMark) @safeGet increment indicator, if detected. Return false otherwise.

void scanBlockScalarIgnoredLine(const Mark startMark) @safeScan (and ignore) ignored line in a block scalar.

Tuple!(uint, Mark) scanBlockScalarIndentationToSlice(ref char[] slice) @safeScan indentation in a block scalar, returning line breaks, max indent and end mark.

Mark scanBlockScalarBreaksToSlice(ref char[] slice, const uint indent) @safeScan line breaks at lower or specified indentation in a block scalar.

Token scanFlowScalar(const ScalarStyle quotes) @safeScan a qouted flow scalar token with specified quotes.

void scanFlowScalarNonSpacesToSlice(ref char[] slice, const ScalarStyle quotes, const Mark startMark) @safeScan nonspace characters in a flow scalar.

void scanFlowScalarSpacesToSlice(ref char[] slice, const Mark startMark) @safeScan space characters in a flow scalar.

bool scanFlowScalarBreaksToSlice(ref char[] slice, const Mark startMark) @safeScan line breaks in a flow scalar.

Token scanPlain() @safeScan plain scalar token (no block, no quotes).

void scanPlainSpacesToSlice(ref char[] slice) @trustedScan spaces in a plain scalar.

void scanTagHandleToSlice(string name)(ref char[] slice, const Mark startMark)Scan handle of a tag token.

void scanTagURIToSlice(string name)(ref char[] slice, const Mark startMark)Scan URI in a tag token.

void scanURIEscapesToSlice(string name)(ref char[] slice, const Mark startMark)Scan URI escape sequences.

dchar scanLineBreak() @safeScan a line break, if any.

Constructors

this(Reader reader)Construct a Scanner using specified Reader.

Nested Templates

SimpleKeyA simple key is a key that is not denoted by the '?' indicator. For example: --- block simple key: value ? not a simple key: : { flow simple key: value } We emit the KEY token before all keys, so w...

ChompingBlock chomping types.

Functions 1

private fnvoid insert(ref char[] slice, const dchar c, const size_t position) @safe pure