ddn.columnar.array

The chunk layer of ddn.df: the untyped, Arrow-style ArrayData plus the cheap typed views built on kernel entry (D1, D6, D8; ADR-2026-10-18-03, ADR-2026-10-18-06).

A chunk is an immutable unit of column storage: an optional validity bitmap (the sentinel means "no nulls"), a dense values buffer, an extra byte buffer for STRING payloads, and length, offset and nullCount. Slicing re-bases the validity bitmap and recomputes nullCount; the value buffers stay whole (visibility comes from offset and length), which keeps every buffer 64-byte aligned for typedView (D2) and lets views take element-aligned subslices of the typed array.

Copyright

Copyright (c) 2026, DDN (D Developer Network) Members

License

BSD-3-Clause
tmpl viewOf
struct ArrayData BooleanArray ElementRange PrimitiveArray StringArray ValueRange

Types 6

structArrayData

The untyped storage of one chunk, what a Series stores an array of. The dtype selects the interpretation of values: the physical element type for primitives, long offsets for STRING (with the UTF-8 bytes in extra), packed bits for BOOLEAN. Immutable once constructed (builders seal it, D3); operations produce new chunks.

Invariants, checkable with validate in debug builds: the buffers hold at least offset + length elements; a non-sentinel validity bitmap has exactly length bits; nullCount is the count of unset validity bits over the visible range (recomputed on every slice — a count inherited from the parent would be wrong after head, tail or slice).

Fields
DataType dtypeThe logical type of the column this chunk belongs to.
size_t lengthNumber of visible elements.
size_t offsetSlot index of the first visible element (zero-copy slicing).
size_t nullCountNulls in the visible range; recomputed on every slice.
Bitmap validityValidity, one bit per visible slot; the sentinel means "no nulls".
immutable(ubyte)[] valuesValues buffer: physical elements, STRING offsets or BOOLEAN bits.
immutable(ubyte)[] extraExtra buffer: the UTF-8 bytes of a STRING chunk.
ArrayData[] childrenThe richer kinds' children (ADR-2026-10-23-13): LIST - the element chunk; STRUCT - the field chunks in order; CATEGORICAL - the labels chunk. `null` for the flat kinds.
Methods
bool isValid(size_t i) const @safe pure nothrowParams: i = visible slot index (`0 .. length`)
ArrayData slice(size_t start, size_t len) const @trusted pure nothrowZero-copy slice: advances `offset`, sets `length`, re-bases the validity bitmap over the visible window and recomputes `nullCount`. The value buffers are shared unchanged.
private void assertPrimitives(size_t elemSize)() const @safe pure nothrow
structPrimitiveArray(T)

A typed view over the values of a primitive (or temporal, physically long) chunk: an element-aligned slice of the sealed buffer plus the validity bitmap. Cheap to build on kernel entry; borrows the chunk.

Fields
immutable(T)[] valuesThe visible values.
Bitmap validityOne bit per visible slot; the sentinel means "no nulls".
Methods
size_t length() const @safe pure nothrowReturns: the number of visible slots.
Nullable!T opIndex(size_t i) const @safe pure nothrowParams: i = visible slot index (`0 .. length`)
T value(size_t i) const @safe pure nothrowParams: i = visible slot index
auto byElement() const @safe pure nothrowReturns: an input range over `Nullable!T` elements.
auto byValue() const @safe pure nothrowReturns: an input range over the raw values (null slots included).
structBooleanArray

A typed view over a BOOLEAN chunk: the value bits plus the validity.

Fields
Bitmap bitsThe visible value bits; the sentinel only for an empty chunk.
Bitmap validityOne bit per visible slot; the sentinel means "no nulls".
Methods
size_t length() const @safe pure nothrowReturns: the number of visible slots.
Nullable!bool opIndex(size_t i) const @safe pure nothrowParams: i = visible slot index (`0 .. length`)
bool value(size_t i) const @safe pure nothrowReturns: the raw bit in slot `i` without a validity check.
auto byElement() const @safe pure nothrowReturns: an input range over `Nullable!bool` elements.
auto byValue() const @safe pure nothrowReturns: an input range over the raw bits (null slots included).
structStringArray

A typed view over a STRING chunk: visible offsets, bytes and validity.

Fields
immutable(long)[] offsetsThe visible offsets, `length + 1` entries into `bytes`.
immutable(ubyte)[] bytesThe whole UTF-8 payload buffer.
Bitmap validityOne bit per visible slot; the sentinel means "no nulls".
Methods
size_t length() const @safe pure nothrowReturns: the number of visible slots.
Nullable!string opIndex(size_t i) const @safe pure nothrowParams: i = visible slot index (`0 .. length`)
string value(size_t i) const @safe pure nothrowReturns: the raw string in slot `i` without a validity check.
auto byElement() const @safe pure nothrowReturns: an input range over `Nullable!string` elements.
auto byValue() const @safe pure nothrowReturns: an input range over the raw strings (null slots included).
private structElementRange(View)
Fields
View _view
size_t _i
Methods
bool empty() @property const @safe pure nothrow
@property auto front() const @safe pure nothrow
void popFront() @safe pure nothrow
private structValueRange(View)
Fields
View _view
size_t _i
Methods
bool empty() @property const @safe pure nothrow
@property auto front() const @safe pure nothrow
void popFront() @safe pure nothrow

Templates 1

tmplviewOf(Kind K)

Builds the typed view of a chunk for the given compile-time kind: a PrimitiveArray!(PhysicalOf!K) for the numeric and temporal kinds, a BooleanArray for BOOLEAN and a StringArray for STRING. The view applies the chunk's offset and length; the value buffers stay whole and 64-byte aligned.

Parameters

Kthe kind to view as
datathe chunk to view

Returns

the typed view borrowing data's storage.