License
BSD-3-Clause
ddn.data.arrow.c_data
The Arrow interchange layer of ddn.df (ADR-2026-10-24-15): the C Data Interface (the ArrowArray/ArrowSchema ABI structs with release callbacks), implemented natively in D with no external dependency. Export is zero-copy where the sealed-buffer layout matches Arrow's (validity, fixed-width values, offsets); import copies into fresh sealed chunks (the immutable-buffer invariant; foreign memory is never held after the import returns).
Copyright
Copyright (c) 2026, DDN (D Developer Network) Members
Types 4
structArrowSchema
Fields
const(char) * formatconst(char) * nameconst(char) * metadatalong flagslong n_childrenArrowSchema * * childrenArrowSchema * dictionaryvoid function(ArrowSchema *) releasevoid * private_datastructArrowArray
Fields
long lengthlong null_countlong offsetlong n_bufferslong n_childrenconst(void) * * buffersArrowArray * * childrenArrowArray * dictionaryvoid function(ArrowArray *) releasevoid * private_dataFields
const(void) *[] bufsArrowArray[] childStorageArrowArray * dictStorageubyte[] wideBytesulong[] validityWordsFunctions 17
fn
string arrowFormatOf(DataType dtype) @safeMaps a flat `DataType` to its Arrow C Data Interface format string (verified against pyarrow 21.0.0's `export_to_c`): STRING maps to `"U"` (large_string — our offsets are int64), DATETIME to `"ts...fn
DataType dtypeOfArrowFormat(scope const(char)[] format) @safeMaps an Arrow format string to a `DataType`. The admitted flat set: the fixed-width kinds, `"u"`/`"U"` (utf8 and large_utf8; the 32-bit-offset form imports by widening), `"tsn:"` (nanosecond timest...fn
ArrowSchema exportSchema(DataType dtype, string name = null) @trustedBuilds the `ArrowSchema` for one dtype: the verified format string, an optional name, null metadata, and the rich kinds' wiring - LIST carries its element child ("item"), CATEGORICAL carries its la...private fn
string cstr(const(char) * p) @trusted pure nothrowReads one NUL-terminated C string into a D slice (null pointers become the empty slice).fn
DataType dtypeOfSchema(const ArrowSchema * s) @safeDerives the dtype an `ArrowSchema` describes. The flat tier maps through `dtypeOfArrowFormat`; the rich tier parses the tree: `+L`/`+l` with one flat element child becomes LIST (the 32-bit `+l` off...fn
ArrowArray exportArray(const ArrayData chunk) @trustedExports one sealed chunk as an `ArrowArray`. The buffers reference the chunk's immutable storage (zero-copy); the release callback frees only the C-side bookkeeping, not the data.fn
ArrayData importArray(const ArrowArray arr, DataType dtype,
bool largeOffsets = true, const ArrowSchema * schema = null) @trustedImports an `ArrowArray` into a fresh sealed chunk (copy-on-import: the foreign buffers are copied, the source's release fires normally, and no foreign memory is held after the return). The ABI's `o...fn
ArrayData importArray(const ArrowArray arr, const ArrowSchema * schema) @trustedDitto, schema-driven: the dtype and the offsets width come from the paired `ArrowSchema` (the ABI's intended entry).fn
ArrowArray[] exportColumn(Series column) @trustedExports a whole column: one `ArrowArray` per chunk, in chunk order. A chunked column is several C Data arrays (one array per contiguous buffer set); the chunk boundaries are the column's own and ar...fn
Series importColumn(string name, scope const(ArrowArray)[] arrays,
DataType dtype) @trustedImports a column from its exported chunk arrays: each array seals into one chunk, in order (the counterpart of `exportColumn`).private fn
void copyBits(const(ulong) * src, size_t srcBit, ubyte[] dst,
size_t nBits) @trusted pure nothrow