import ddn.data.json5 : parseJSON5, toJSON5, toJSON, minify, Json5Error;
import ddn.var : var;
Json5Error err;
var v;
assert(parseJSON5(v, "{a:1, b:'x',}", err));
assert(v["a"].as!long == 1);
// Minify JSON5 (strip comments/whitespace).
// Note: avoid block-comment markers in DDoc examples (they can terminate DDoc blocks).
auto compact = minify("{ a: 1, //c\n b: 2, }");
assert(compact == "{a:1,b:2,}");
// Write JSON5 vs strict JSON.
auto json5Text = toJSON5(v); // e.g. "{a:1,b:'x'}"
auto jsonText = toJSON(v); // e.g. "{\"a\":1,\"b\":\"x\"}"ddn.data.json5
JSON5 support for DDN — Design and Policy
This module documents the specification scope and type mapping between JSON5 values and ddn.var. The actual parser/writer implementation is introduced in subsequent tasks.
Mapping rules to var:
- Objects: map to
var.Type.OBJECTwithstringkeys andvarvalues. - Arrays: map to
var.Type.ARRAYwith elements converted tovarrecursively. - Strings: map to
var.Type.STRING(both single-quoted and double-quoted forms are accepted). - Booleans: map to
var.Type.BOOL. - null: map to
var.Type.NULL. - Numbers:
- If the numeric value is integral and fits a 64-bit signed integer, use
Type.LONG. - Else if the numeric value is integral and non-negative and fits 64-bit unsigned, use
Type.ULONG. - Otherwise, use
Type.DOUBLEand preserve special cases:NaN,Infinity,-Infinity,-0.0.
Writer rules (summary):
- Keys are unquoted when they are valid JSON5 identifiers; otherwise quoted with minimal escaping.
- Values are emitted compactly by default; pretty printing is configurable.
- Deterministic output is achieved by lexicographically sorting object keys by default.
Feature coverage (JSON5):
- Comments: `//` line and C-style block comments.
- Trailing commas in arrays/objects.
- Unquoted keys when identifier-compatible.
- Single-quoted strings and escape sequences, including line continuations.
- Numbers with leading `+`, hex (
0x...), leading/trailing decimal points, exponents, and special numbers. - BOM (UTF‑8) accepted at the beginning of input.
Examples
Types 23
Feature switches for JSON5 support.
Policy controlling writer defaults and parser behavior.
Note on Policy Naming:This struct is named Json5Policy following the convention of format-specific policy structs (JsonPolicy, SdlPolicy). The CfParserConfig struct uses a different naming pattern for historical reasons (it predates the policy naming convention and has additional configuration beyond parsing behavior).
lastKeyWins:
Unlike JsonPolicy which uses a JsonDuplicateKeyPolicy enum with three options (LAST_WINS, FIRST_WINS, ERROR), this struct uses a simple boolean. This is intentional: JSON5 is designed as a more lenient format for human-authored configuration files, where strict duplicate detection (ERROR) is rarely desired. The boolean provides a simpler API for the common case of choosing between first-wins and last-wins semantics. When useStrictJsonParser is enabled, the boolean maps to the corresponding enum value in the underlying JsonPolicy.
bool preferSignedIntegersPrefer signed integers when a value fits both signed and unsigned.bool lastKeyWinsDuplicate keys policy: if true, last occurrence wins; if false, first wins. Unlike `JsonPolicy.duplicateKeyPolicy`, this is a boolean because JSON5's lenient nature makes the strict ERROR option ra...size_t maxDepthMaximum nesting depth allowed during parsing to prevent runaway recursion.bool sortKeysOnWriteWriter: sort object keys for deterministic output.bool asciiOnlyWriteWriter: emit only ASCII by escaping non-ASCII code points.bool useStrictJsonParserWhen true, delegate parsing to `ddn.data.json` for potentially faster strict JSON parsing.A non-throwing error report produced by the JSON5 parser/writer.
size_t line1-based line number where the error occurred (0 if unknown).size_t column1-based column number where the error occurred (0 if unknown).size_t index0-based byte offset into the original input (0 if unknown).Json5ErrorCode codeMachine-readable error category.string messageHuman-readable message.string contextExcerpt of the source around `index` with a caret line, if available.this(size_t line, size_t column, string message,
Json5ErrorCode code = Json5ErrorCode.UNKNOWN,
size_t index = 0,
string context = "")Construct an error report.Quote style used by the JSON5 writer.
Output mode for the writer.
Writer formatting options for JSON5 emission.
Fields:
- pretty: enable multi-line pretty printing with indentation.
- indentWidth: number of spaces per indentation level (used only when
pretty). - quoteStyle: whether strings are emitted with single or double quotes.
- asciiOnly: when true, escape non-ASCII as
\uXXXXor\u{...}. - sortKeys: when true, object keys are emitted sorted for deterministic output.
- maxDepth: maximum nesting depth allowed during writing; when exceeded, the writer emits
null. - trailingCommas: when true, pretty mode emits a trailing comma after the last array element
and the last object property; ignored in compact mode.
Json5WriteMode writeModeSelect whether to emit JSON5 or strict JSON.bool prettyPretty-print output with indentation.uint indentWidthNumber of spaces per indentation level when `pretty` is true.Json5QuoteStyle quoteStyleString quoting style used by the writer.Json5FloatFormat floatFormatFloating-point number formatting style.uint floatPrecisionPrecision parameter for float formatting.uint scientificExponentThresholdIf `floatFormat == AUTO`, switch to scientific when `abs(exponent) >= threshold`.bool hexIntegersEmit integer values in hexadecimal form (`0x...`) when true.Json5NegativeZeroStyle negativeZeroStyleControls how `-0.0` is emitted.bool asciiOnlyEscape non-ASCII code points as \uXXXX.bool sortKeysSort object keys for deterministic output.bool trailingCommasEmit trailing commas in pretty mode for arrays/objects.size_t maxDepthMaximum nesting depth allowed during writing to prevent runaway recursion.Reusable scratch buffers for JSON5/JSON writing.
This structure allows caller-controlled reuse of temporary allocations across repeated serializations.
Currently used for:
- Deterministic object serialization (
sortKeys == true) via per-depth key buffers.
string[][] keyBuffersPer-depth buffers used to collect and sort object keys.Floating-point formatting mode used by the JSON5 writer.
Controls how floating-point negative zero is emitted.
Reviver callback used to transform parsed values.
The callback is invoked in post-order (children first), similarly to JavaScript JSON.parse(text, reviver).
Arguments:
key: Object key / array index (as decimal string). Root is passed as `""`.value: Current value.
The return value is the replacement value.
Kind of a JSON5 patch operation.
A single JSON5 patch operation.
The path is a JSON Pointer-like path using `/` separators.
- Root is `""`.
~1decodes to `/`.~0decodes to `~`.
Json5PatchOpKind kindOperation kind.string pathTarget path.var valueValue for `ADD`/`REPLACE`.Patch application error.
size_t opIndexIndex of the failing operation.string pathPath associated with the failure.string messageHuman-readable message.Result of a streaming JSON5 parse attempt.
Incremental/streaming JSON5 parser that accepts input in chunks.
This API is intended for large inputs or network streams where the complete document may not be available at once.
Notes:
- The parser produces at most one root JSON5 value.
- After a successful parse (
Json5StreamResult.OK), the instance becomes done;
call reset() to parse another document.
- When
finish()has not been called yet, parse failures that look like
“unexpected end of input” are reported as NEED_MORE.
Examples
import ddn.data.json5 : Json5StreamParser, Json5StreamResult, Json5Error;
import ddn.var : var;
Json5StreamParser sp;
sp.push("{a:1,");
var v; Json5Error err;
assert(sp.tryParse(v, err) == Json5StreamResult.NEED_MORE);
sp.push("b:2}");
assert(sp.tryParse(v, err) == Json5StreamResult.OK);
assert(v["b"].as!long == 2);Json5StreamResult tryParse(out var value, out Json5Error err) @safeAttempt to parse the buffered data as a single JSON5 root value.bool _looksIncomplete(ref Json5Parser p) const @safeHeuristic: detect failures likely caused by an incomplete final chunk.this(const Json5Policy policy)Construct a streaming parser using `policy`.Callbacks for SAX-style JSON5 parsing.
Any callback may be left null to ignore that event.
Event semantics:
onObjectStart/onObjectEndare emitted for `{` / `}`.onArrayStart/onArrayEndare emitted for `[` / `]`.onKeyis emitted for each object property key, before its value.onValueis emitted for scalar values only (null/bool/number/string).
void delegate() @safe onObjectStartCalled when an object begins (`{`).void delegate() @safe onObjectEndCalled when an object ends (`}`).void delegate() @safe onArrayStartCalled when an array begins (`[`).void delegate() @safe onArrayEndCalled when an array ends (`]`).void delegate(const string key) @safe onKeyCalled for each object key.void delegate(const var value) @safe onValueCalled for each scalar value.Kinds of JSON5 tokens.
A single JSON5 token.
Json5TokenKind kindToken kind.size_t line1-based line on which the token starts.size_t column1-based column on which the token starts.const(char)[] lexemeRaw slice of the input covering the token text (for STRING/IDENTIFIER/NUMBER).size_t startIndexStart index in the source buffer (for reconstruction when `lexeme` is empty).size_t endIndexEnd index (one past last) in the source buffer.High-performance JSON5 lexer over a UTF‑8 buffer.
- Skips BOM, whitespace, and comments.
- Returns slices for identifiers, strings, and numbers without allocating.
- Tracks line/column for diagnostics.
private const(char)[] _srcprivate size_t _iprivate size_t _lineprivate size_t _colJson5Token nextTokenWithComments() @safeReturn the next token, emitting COMMENT tokens instead of skipping them.this(const(char)[] input)Construct a lexer over `input`.A small, non-throwing JSON5 recursive-descent parser that builds ddn.var directly.
Create an instance with input and policy, then call parseRoot(out var) to parse a value. On failure, parseRoot returns false and populates the provided Json5Error with line/column and a brief message.
Json5Lexer lxUnderlying lexer (exposed for diagnostics in tests).Json5Token curCurrent token (public for testing and the outer wrapper).size_t _prevEndIndexEnd index of the previously returned token (used for boundary recovery).Json5Policy policyParser policy (depth limits, preferences, etc.).private Json5Error * _perrError sink pointer (non-owning).bool parseArray(out var outv, size_t depth) @safeParse an array value; assumes `cur` is `[` on entry.bool parseObject(out var outv, size_t depth) @safeParse an object value; assumes `cur` is `{` on entry.string _nearSnippet(size_t context = 30) @safeBuild a context window for diagnostics based on current token.bool fail(const(char)[] msg, const Json5ErrorCode code = Json5ErrorCode.UNEXPECTED_TOKEN) @safeRecord an error at the current position and return `false`.this(const(char)[] input, Json5Policy policy, ref Json5Error err)Construct a parser for `input` with `policy`, using `err` as the error sink.Internal CDM-producing parser structure that reuses the existing lexer.
This struct is used by parseJson5Cdm to produce CdmDocument instances while preserving comments, source locations, and formatting metadata.
Json5Lexer lxJson5Policy policyJson5Error * errstring sourceJson5Token curCdmComment[] pendingCommentsBuilder builderbool fail(const(char)[] msg, Json5ErrorCode code = Json5ErrorCode.UNEXPECTED_TOKEN) @safeRecord an error and return false.this(const(char)[] input, Json5Policy policy, Json5Error * err, string source)Construct a CDM parser over `input`.Parse JSON5 text into a CDM document.
Preserves comments, source locations, and formatting metadata for roundtrip fidelity.
Parameters
input | JSON5 text to parse. |
source | Optional source identifier (file path, URI). |
Returns
CdmDocument containing the parsed structure.Throws
Json5Exception on parse error.
Example:
import ddn.data.json5;
import ddn.data.cdm;
auto doc = parseJson5Cdm(`{
// Server configuration
host: "localhost",
port: 8080
}`);
assert(doc.root.isObject);
assert(doc.root["host"].as!string == "localhost");Exception thrown when JSON5 parsing fails.
This exception includes source location information for error reporting.
size_t lineLine number where the error occurred (1-based)size_t columnColumn number where the error occurred (1-based)this(string msg, size_t line, size_t column)Construct a Json5Exception.Functions 71
Json5Error _makeError(const(char)[] src,
const size_t line,
const size_t column,
const size_t index,
const Json5ErrorCode code,
const string msg) @safeBuild a structured error with a context window.string _json5ContextWindow(const(char)[] src, size_t index, size_t context = 30) @safeBuild a context window for diagnostics.string[] _scratchKeys(ref Json5WriteScratch scratch, const size_t depth) ref @safeObtain a cleared scratch key buffer for a given writer recursion `depth`.bool parseJSON5(out var value, const(char)[] input, out Json5Error err) @safeParse JSON5 text from `input` into `value`.bool parseJSON5(out var value, const(char)[] input, out Json5Error err, Json5Policy policy) @safeParse JSON5 text into a `var` value with custom policy.bool parseJSON5(out var value, const(char)[] input, Json5Reviver reviver, out Json5Error err) @safeParse JSON5 text and apply `reviver` to each value.void _applyReviver(ref var value, const string key, scope Json5Reviver reviver) @safeApply `reviver` to `value` in post-order.var parseJSON5(const(char)[] input) @safeConvenience overload that parses JSON5 text and returns the resulting value. Note: returns `var.init` on failure.bool minify(out string output, const(char)[] input, out Json5Error err) @safeMinify JSON5 text by removing comments and optional whitespace.bool json5Patch(ref var base, scope const Json5PatchOp[] ops, out Json5PatchError err) @safeApply a patch to `base`.string[] _jsonPointerSplit(const string path) @safeSplit a JSON Pointer path into decoded segments.bool _applyPatchOp(ref var base, ref const Json5PatchOp op, ref Json5PatchError err) @trustedvoid _ensureContainerForPath(ref var container, const string seg) @safeEnsure `container` is initialized as an object or array to accommodate `seg`.bool _looksLikeArrayIndex(const string seg) @safe pure nothrow @nogcReturns true if `seg` is a plausible JSON Patch array index.bool _parseArrayIndex(const string seg, out size_t index, const size_t length, const bool allowEnd = false) @safe pure nothrow @nogcbool parseJSON5(out var value, ref File file, out Json5Error err,
const Json5Policy policy = Json5Policy.init,
size_t chunkSize = 16_384) @safeParse JSON5 data from an open `std.stdio.File`.bool parseJSON5(R)(out var value, R input, out Json5Error err,
const Json5Policy policy = Json5Policy.init,
size_t chunkSize = 16_384) if (
isInputRange!R &&
(is(ElementType!R == ubyte) || isSomeChar!(ElementType!R))
) @safeParse JSON5 data from an input range of `char` or `ubyte`.bool parseJSON5Sax(const(char)[] input, ref Json5SaxHandler handler, out Json5Error err,
const Json5Policy policy = Json5Policy.init) @safeParse JSON5 text and emit SAX-style events into `handler`.string toJSON5(const var value, const Json5WriteOptions opts = Json5WriteOptions.init) @safeSerialize a `var` to JSON5 text using the provided `opts`.string toJSON5(const var value, const Json5WriteOptions opts, ref Json5WriteScratch scratch) @safeSerialize a `var` to JSON5 text using the provided `opts` and reusable `scratch` buffers.string toJSON(const var value, const Json5WriteOptions opts = Json5WriteOptions.init) @safeSerialize a `var` to strict JSON text.string toJSON(const var value, const Json5WriteOptions opts, ref Json5WriteScratch scratch) @safeSerialize a `var` to strict JSON text using the provided `opts` and reusable `scratch` buffers.void _writeJSON5Impl(const var value, ref Appender!string buf, const Json5WriteOptions opts,
uint depth, ref Json5WriteScratch scratch) @safevoid _writeInteger(T)(T v, ref Appender!string buf, const Json5WriteOptions opts) @safeWrite a numeric integer value according to `opts`.void _putHexUnsigned(const ulong v, ref Appender!string buf) @safeEmit a hexadecimal unsigned integer with `0x` prefix.void _writeDouble(T)(T d, ref Appender!string buf, const Json5WriteOptions opts) @safeWrite a floating-point value according to `opts`.void _writeJSONString(scope const(char)[] s, ref Appender!string buf, const bool asciiOnly,
const Json5QuoteStyle quoteStyle = Json5QuoteStyle.DOUBLE) @safevoid _writeKey(const string key, ref Appender!string buf, const Json5WriteOptions opts) @safevoid _writePrettyArray(const var[] arr, ref Appender!string buf, const Json5WriteOptions opts,
uint depth, ref Json5WriteScratch scratch) @safevoid _writePrettyObject(const var[string] obj, const string[] keys, ref Appender!string buf,
const Json5WriteOptions opts, uint depth, ref Json5WriteScratch scratch) @safebool load(out var obj, const string fileName, out Json5Error err) @safeLoad a JSON5 file from disk into `obj`.bool save(const var obj, const string fileName, const Json5WriteOptions opts = Json5WriteOptions.init, out Json5Error err) @safeSave a `var` as JSON5 to disk.void _removeNoThrow(const string fileName) @safeRemove a file, ignoring any exception that may occur.bool _isJSON5UnicodeWhitespace(const dchar dc) @safe pure nothrow @nogcReturns true if `dc` is a non-ASCII whitespace character recognized by JSON5.bool _isJSON5UnicodeLineTerminator(const dchar dc) @safe pure nothrow @nogcReturns true if `dc` is a JSON5 Unicode line terminator.bool parseJSON5NumericLiteral(out var value, const(char)[] text, const Json5Policy policy = Json5Policy.init) @safeParse a JSON5 numeric literal from `text` into `out value`.bool parseJSON5StringLiteral(out string result, const(char)[] text, out Json5ErrorCode errCode) @safeDecode a JSON5 string literal (single or double quoted) into a D `string`.bool parseJSON5StringLiteral(out string result, const(char)[] text) @safeDecode a JSON5 string literal.uint _hexVal(const char h) @safe @nogc pure nothrowConvert a hex digit char to its numeric value (assumes valid hex digit).bool _isHex(const char h) @safe @nogc pure nothrowCheck if a character is a hexadecimal digit (0-9, a-f, A-F).void _putCodePoint(ref Appender!string buf, uint code) @safeAppend a Unicode code point as UTF‑8 to buffer.string decodeIdentifierEscapes(const(char)[] text) @safeDecode Unicode escape sequences (\uXXXX) in an identifier lexeme. Identifiers may contain \uXXXX escapes per JSON5 spec. Returns the decoded string, or the original if no escapes present.bool isValidUnquotedKey(const(char)[] key) @safeCheck whether `key` can be emitted without quotes in JSON5 (valid identifier).string extractCommentText(const(char)[] raw) @safeExtract comment text from raw token (strip delimiters)CdmFormat.QuoteStyle detectQuoteStyle(const(char)[] token) @safe pure nothrow @nogcDetect quote style from string tokenCdmFormat.NumberFormat detectNumberFormat(const(char)[] token) @safe pure nothrow @nogcDetect number format from literalbool parseCdmValue(Builder)(ref Json5CdmParser!Builder p, size_t depth) @safeParse a value via the builderbool parseCdmNumber(Builder)(ref Builder builder, const(char)[] lexeme, Json5Policy policy) @safeParse a number literal and set it on the builderbool parseCdmString(Builder)(ref Builder builder, const(char)[] lexeme) @safeParse a string literal and set it on the builderbool parseCdmObject(Builder)(ref Json5CdmParser!Builder p,
CdmComment[] leadingComments, CdmLocation loc, size_t depth) @safeParse an object via the builderbool parseCdmArray(Builder)(ref Json5CdmParser!Builder p,
CdmComment[] leadingComments, CdmLocation loc, size_t depth) @safeParse an array via the builderauto parseJson5Cdm(Builder = CdmBuilder)(const(char)[] input, string source = "") @trustedParse JSON5 text.bool parseJson5Cdm(out CdmDocument doc, const(char)[] input, Json5Error * err,
string source = "", Json5Policy policy = Json5Policy.init) @safeNon-throwing parse variant that produces a `CdmDocument`.bool parseJson5CdmWithBuilder(Builder)(ref Builder.Result result,
const(char)[] input, Json5Error * err,
string source = "", Json5Policy policy = Json5Policy.init) @safeInternal templated parse function that works with any builder.string toJson5(const ref CdmDocument doc, Json5WriteOptions opts = Json5WriteOptions.init) @safeSerialize a CDM document to JSON5 text.void writeCdmNode(Out)(ref Out buf, const ref CdmNode node,
uint depth, const ref Json5WriteOptions opts) @safeWrite a CDM node to the output buffervoid writeCdmComment(Out)(ref Out buf, const CdmComment c,
uint depth, const ref Json5WriteOptions opts) @safeWrite a comment to the output buffervoid writeInlineCdmComment(Out)(ref Out buf, const CdmComment c) @safeWrite an inline commentvoid writeCdmString(Out)(ref Out buf, const(char)[] s,
CdmFormat.QuoteStyle style, const ref Json5WriteOptions opts) @safeWrite a string with proper quotingvoid writeCdmObject(Out)(ref Out buf, const ref CdmNode node,
uint depth, const ref Json5WriteOptions opts) @safeWrite an object to the output buffervoid writeCdmArray(Out)(ref Out buf, const ref CdmNode node,
uint depth, const ref Json5WriteOptions opts) @safeWrite an array to the output buffervoid writeCdmValueOnly(Out)(ref Out buf, const ref CdmNode node,
uint depth, const ref Json5WriteOptions opts) @safeWrite only the value (without leading comments)void writeCdmKey(Out)(ref Out buf, const(char)[] key,
CdmFormat.QuoteStyle style, const ref Json5WriteOptions opts) @safeWrite a key with proper quotingVariables 1
JSON5_DEFAULT_FEATURES = cast(uint)(
Json5Feature.COMMENTS
| Json5Feature.TRAILING_COMMAS
| Json5Feature.SINGLE_QUOTED_STRINGS
| Json5Feature.UNQUOTED_KEYS
| Json5Feature.HEX_NUMBERS
| Json5Feature.SPECIAL_NUMBERS
| Json5Feature.LEADING_PLUS
| Json5Feature.LEAD_TAIL_DECIMAL
| Json5Feature.LINE_CONTINUATIONS
| Json5Feature.BOM_UTF8)Default feature set enabled by the JSON5 reader/writer.