ddn.data.csv

Newline policy controls how record boundaries are detected and, for writer, how newlines are emitted.

Planned values match RFC 4180 defaults while allowing practical overrides.

import ddn.data.csv;

// Reader detects CRLF/LF by default
auto d1 = CsvDialect.init; // newlinePolicy = DETECT

// Force writer to emit LF newlines
auto d2 = CsvDialect.init; d2.newlinePolicy = NewlinePolicy.FORCE_LF;

Escape style for non‑RFC datasets (optional extension). none — only RFC 4180 double‑quote escaping inside quoted fields. backslash — treat backslash as escape for delimiter/quote/newline in unquoted fields.

Example:

import ddn.data.csv;
auto d = CsvDialect.init;
d.escapeStyle = EscapeStyle.NONE; // RFC 4180 behavior (default)

Reader error handling mode.

• PERMISSIVE (default): malformed rows are skipped and errors are

counted; iteration continues. Optionally collect diagnostics.

• FAIL_FAST: stop iteration at the first error and surface it via

CsvReader.stats.

Convenience aliases for common public types.

These aliases have no runtime cost and exist purely for readability in user code and documentation.

import ddn.data.csv;

CsvField f;          // alias for FieldView
CsvRow row;          // alias for RowView
CsvResultT!int ri;   // alias for CsvResult!int

// Parametric reader/writer helpers
alias Reader = CsvReaderOf!(const(char)[]);
alias Writer = CsvWriterTo!(typeof((const(char)[] s){}) ); // any sink with put()

Shorthand for a parsed CSV row view.

Shorthand for CsvResult!T.

Shorthand for CsvReader!R.

Shorthand for CsvWriter!S.

Lightweight non‑owning view over a CSV field.

Header index providing fast name -> column index lookup.

Built once from a set of header fields (typically the first CSV record when CsvDialect.header == true) and attached to subsequent RowView instances via RowView.attachHeader to enable row.byName("col") without per-row overhead.

Policy for duplicates: the first occurrence wins; hasDuplicates is set to true when the header contains duplicate (normalized) names.

Lightweight non‑owning view over a parsed CSV row.

Error codes describing common CSV parsing and configuration issues.

Structured error information returned in CsvResult!T.

Exception type used by optional throwing helpers.

This exception wraps a CsvError and is thrown by convenience APIs such as CsvResult!T.valueOrThrow() for users who prefer exception‑driven control flow instead of explicit result checking. Hot paths should prefer CsvResult without throwing for performance.

Result container for error‑aware APIs without throwing.

Holds either a value of type T (when isOk is true) or an error. For compile‑time friendliness we store both value and err; users should consult isOk before accessing value.

CSV dialect configuration.

Represents the set of options that control how CSV is parsed and written. Defaults follow RFC 4180: comma delimiter, double‑quote for quoting, CRLF/LF detection on read, RFC‑only escaping, and no header row by default.

Fields (all public):

• delimiter: Single‑byte field separator (default: `,`).
• quote: Quote character (default: `"`).
• doubleQuote: When true (default), two consecutive quotes inside

quoted fields represent a literal quote character.

• trimWhitespace: When true, trim leading/trailing whitespace for

unquoted fields. Default: false.

• newlinePolicy: Controls record boundary handling and writer emission

policy. Default: NewlinePolicy.detect.

• escapeStyle: Optional extension for non‑RFC datasets. Default: none.
• header: When true, the first record is interpreted as a header row.

Construction:

• The type can be used with its .init value for RFC defaults, e.g.

auto d = CsvDialect.init; and then mutate fields.

• A convenience constructor allows specifying any subset (positional with

defaults) while other options fall back to defaults, e.g.: auto d = CsvDialect(';'); // semicolon delimiter

Validation:

• isValid() returns true if options are self‑consistent. At minimum it

ensures delimiter != quote.

import ddn.data.csv;

// Customize only the delimiter (semicolon-separated values)
auto d1 = CsvDialect(';');

// Fully configure a dialect
auto d2 = CsvDialect(',', '\'', true, false, NewlinePolicy.forceLF);
assert(d1.isValid && d2.isValid);

High‑throughput CSV reader over an input range of bytes.

Models a forward input range over RowView. Instances created via the provided constructors yield an empty range.

import ddn.data.csv;
// Parse two rows, sum first column lazily (no allocations on hot path)
const csv = "x,y\n10,20\n30,40\n";
long sum = 0;
auto r = byRows(csv);
// Skip header
r.popFront();
while (!r.empty)
{
   auto row = r.front;
   auto a = fromCsv!long(row[0]);
   auto b = fromCsv!long(row[1]);
   if (a.isOk && b.isOk) sum += a.value + b.value;
   r.popFront();
}
assert(sum == 100);

Efficient CSV writer to an output range of bytes.

Implements a buffered writer core that minimizes calls to the underlying sink by batching output into a large internal buffer. The buffer size is configurable via the constructor; a sensible default is used when not specified.

Notes:

• Fields are written as provided. Quoting rules per RFC 4180.
• Newline emission follows dialect.newlinePolicy: detect and

forceCRLF emit CRLF; forceLF emits LF.

import ddn.data.csv;
static class Sink { string data; void put(const(char)[] s) { data ~= s; } }

auto s = new Sink();
auto w = CsvWriter!(Sink)(s); // default RFC dialect
assert(w.writeRow(["id", "name"]).isOk);
assert(w.writeRow(["1", "Doe, Jane"]).isOk); // quoted due to comma
w.flush();
assert(s.data.startsWith("id,name"));

Package-internal buffer manager that batches read() calls into large contiguous buffers to reduce syscall overhead and improve throughput.

Usage pattern (illustrative):

auto bm = BufferManager!MyReader(reader, 64 * 1024);
for (;;)
{
   auto chunk = bm.nextChunk();
   if (chunk.length == 0) break; // EOF
   // Consume data from `chunk` (do not hold past next call).
   bm.popFront(chunk.length);
}

Notes:

• Buffer size is configurable via the constructor; defaults to 64 KiB.
• nextChunk returns a slice of the internal buffer; its content becomes

invalid after the next nextChunk call.

• Metrics readCalls and bytesRead are available for tests/diagnostics.

Package‑internal scan result for a single CSV row.

• row: A RowView referencing the provided buffer slice; valid until the

next scan or buffer reuse.

• consumed: Number of bytes consumed from the start of the buffer,

including the record terminator (CRLF/LF) when found.

• hasRow: true when a complete row was found; false otherwise (no

terminator encountered in the provided chunk per newlinePolicy).

Zero‑copy row scanner that splits a single row into fields without allocations and without quote/escape handling.

Notes:

• Handles CRLF and/or LF per CsvDialect.NEWLINE_POLICY.
• Delimiter is taken from CsvDialect.DELIMITER.
• trimWhitespace is applied to all fields (unquoted fields only are

relevant at this stage).

• Produces FieldView slices that reference the input buffer; callers must

ensure the buffer remains valid until they finish using the row.

• The scanner maintains an internal fixed array to avoid GC allocations in

hot paths. If the number of fields exceeds the fixed capacity, a dynamic array fallback is used for that row (rare); @nogc tests should keep field counts below the fixed capacity to avoid allocations.

Reading statistics and optional diagnostics collected by CsvReader.

Fields:

• rows: number of successfully yielded rows.
• badRows: number of malformed rows encountered.
• errors: total error count (equals badRows in current implementation).
• lastError: the most recent error encountered.
• diagnostics: per-error list (populated only when collectDiagnostics is enabled).

Mode for opening a CsvFile.

Convenience wrapper for reading/writing CSV files.

import ddn.data.csv;
import std.file : remove, exists;

// Write a small CSV file
CsvFile out = CsvFile("./t23_sample.csv", CsvOpenMode.write);
string[][] rows = [["a","b"],["1","x,y"],["2","z"]];
assert(out.writeRows(rows).isOk);
assert(out.close().isOk);

// Read it back (buffered path)
CsvFile f = CsvFile("./t23_sample.csv", CsvOpenMode.read);
auto rr = f.reader();
size_t cnt = 0; while (!rr.empty) { ++cnt; rr.popFront(); }
assert(cnt == rows.length);
// Cleanup
if (exists("./t23_sample.csv")) remove("./t23_sample.csv");

Maximum file size to attempt memory mapping on 32‑bit platforms (heuristic).

Mapping very large files on 32‑bit OSes can fail due to limited virtual address space. We apply a conservative cap (≈1.5 GB) and fall back to buffered reading beyond this size.

Default delimiter (RFC 4180).

Default quote character (RFC 4180).

Default: interpret doubled quotes inside quoted fields.

Default: do not trim whitespace in unquoted fields.

Default newline policy: detect CRLF and LF on read.

Default escape style: RFC only.

Default: header row absent.

Trait that evaluates to true when R provides a read(ubyte[]) method returning the number of bytes read (size_t). This matches common D I/O patterns (e.g., file descriptors, custom sources) and enables our buffered reader to minimize read calls.