ddn.columnar.categorical

The categorical-column layer of ddn.df (ADR-2026-10-23-13/-14): int32 codes in values, the labels chunk in children[0]. Nulls are validity-0 slots (no code); codes never compare directly - equality, hashing and reconciliation go through the labels (dictionary merge by first-appearance label order, deterministic across threads).

Copyright

License

BSD-3-Clause

struct CategoricalBuilder Reconciled ReconciledCategorical

fn categoricalAppend categoricalLabels categoricalRechunk categoricalSeries categoricalTake categoricalWhen countBits reconcile reconcileCategorical remapCodes validateCategoricalChunk

Types 3

structCategoricalBuilder

Builds one CATEGORICAL chunk: int32 codes validated against the dictionary (or -1 for nulls), the labels chunk as the child.

Fields

private int[] _codes

private ulong[] _validity

private size_t _length

private bool _anyNull

private bool _sealed

Methods

void append(int code) @safe pure nothrowAppends one valid slot with the dictionary code (validated at finish against the dictionary's length).

void appendNull() @safe pure nothrowAppends a null slot (no code).

private void setValid(size_t i, bool v) @safe pure nothrow

ArrayData finish(ArrayData dictionary) @safeSeals the chunk.

private size_t countValid() @safe pure nothrow

structReconciled

Reconciles two dictionaries into a merged first-appearance dictionary plus code remappings. Deterministic: the merged label order is the left's labels then the right's unseen labels.

Parameters

`left`	the left labels chunk
`right`	the right labels chunk

Returns

the merged labels chunk, the left codes' remap and the

right codes' remap (both indexable by the original code).

Fields

ArrayData merged

int[] leftRemap

int[] rightRemap

structReconciledCategorical

The reconciled form of a categorical Series' chunks: one shared dictionary every code chunk indexes, in force after every producing operation (the Series-level dictionary invariant, ADR-2026-10-23-14's 2026-10-28 amendment).

Fields

ArrayData dictionary

ArrayData[] chunks

bool alreadyUniformTrue when the input already satisfied the invariant (nothing was rebuilt; the chunks may alias the input).

Functions 11

fnvoid validateCategoricalChunk(const ArrayData chunk) @safeValidates a CATEGORICAL chunk: codes within the dictionary or -1.

fnSeries categoricalSeries(string name, Series labels, int[] codes)Builds a CATEGORICAL Series from a labels Series and per-slot codes.

fnSeries categoricalLabels(Series s) @trustedThe labels of a CATEGORICAL Series as a STRING Series (slot `i` reads the dictionary label of code `i`); nulls stay null.

fnReconciled reconcile(ArrayData left, ArrayData right) @trusted

ReconciledCategorical reconcileCategorical(
       const  ArrayData[]  chunks,  DataType dtype)

@trustedBrings a chunk list to the single-dictionary invariant. Fast paths: zero/one chunks, and identical dictionaries (the same payload buffer - slices share it). Otherwise the dictionaries fold through ...

private fn

ArrayData remapCodes(const  ArrayData chunk,  ArrayData dict,
       const(int)[]  remap)

@trusted

fnSeries categoricalWhen(Series mask, Series a, Series b) @trustedThe elementwise select for categorical branches (the `when` contract of ADR-2026-10-23-08: a null mask slot reads null; the selected branch's null stays null; the output carries one shared dictiona...

ArrayData[] categoricalAppend(const  ArrayData[]  left,
       const  ArrayData[]  right,  DataType dtype)

@trustedAppends two categorical chunk lists under the invariant: the dictionaries reconcile once, both sides' codes remap onto the merged dictionary, and the result chunks share it.

ArrayData categoricalTake(const  ArrayData[]  chunks,
       const  size_t[]  chunkStarts,  const(size_t)[]  indices,
       DataType dtype)

@trustedGathers categorical slots at global indices: codes and validity gather onto the reconciled dictionary (the invariant's output holds by construction). An index of `size_t.max` is the null marker (th...

private fnsize_t countBits(const ulong[] words, size_t n) @safe pure nothrow

ArrayData categoricalRechunk(const  ArrayData[]  chunks,
       DataType dtype)

@trustedRebuilds multi-chunk categorical storage as one chunk: codes concatenate onto the reconciled dictionary, validity merges in row order.