ddn.columnar.hash

The hashing layer of ddn.df: per-row hashing, typed column equality and Series.unique/nUnique (D16; ADR-2026-10-18-15). The GroupTable itself lives in ddn.df.groupby, shaped by its two consumers (groupBy and join).

One 64-bit hash per row per key column: hashOf on the physical value (which coalesces -0.0/0.0 and all NaN payloads, fact 7), the byte slice for strings, a fixed sentinel for nulls with nulls tracked separately (they never match in joins and form their own group in groupBy). Equality is typed, per key column, through a function-pointer table selected once by kind — per-row delegate comparers measured 3.3 times slower (fact 12) — and total on floats.

Copyright

License

BSD-3-Clause

var NULL_HASH

tmpl isInKernel

alias ColumnEq

struct Entry

fn childOf coeffOf columnEqCategorical columnEqDecimal columnEqFor columnEqList columnEqOf columnEqStruct combineHash hashCategoricalRow hashDecimalRow hashListRow hashRow hashStructRow isInRange labelOf listOffsetsOf locateGlobal nUniqueOf rehash rowHashChunkOf rowHashColumn rowHashColumnOf rowHashColumnParallel rowHashColumnRich rowHashCombine rowHashOf totalEq uniqueOf

Types 2

aliasColumnEq = bool function(const ArrayData, size_t, const ArrayData, size_t) @safe pure nothrow

The per-key-column equality function pointer: does row ia of chunk a equal row ib of chunk b? Both chunks are at the same dtype. Nulls equal nulls (they form one group); floats compare totally.

private structEntry

The distinct values of a Series in first-appearance order; nulls form one group and floats group under the total order (NaN with NaN, -0.0 with 0.0).

Parameters

`s`	the column

Returns

the deduplicated single-chunk Series.

Fields

size_t hash

size_t row

Functions 29

private fn

bool columnEqFor(Kind K)(const  ArrayData a,  size_t ia,
       const  ArrayData b,  size_t ib)

@safe pure nothrow

private fnbool totalEq(T)(T a, T b) @safe pure nothrow

private fnArrayData childOf(const ArrayData chunk, size_t i) @trusted pure nothrowSelects the typed equality function pointer for one key column's dtype — once per column, never per row (D16).

private fnconst(long)[] listOffsetsOf(const ArrayData chunk) @trusted pure nothrowDitto.

private fn

bool columnEqList(const  ArrayData a,  size_t ia,  const  ArrayData b,
       size_t ib)

@safe pure nothrow

private fn

bool columnEqStruct(const  ArrayData a,  size_t ia,  const  ArrayData b,
       size_t ib)

@safe pure nothrow

private fn

bool columnEqCategorical(const  ArrayData a,  size_t ia,
       const  ArrayData b,  size_t ib)

@safe pure nothrow

private fnstring labelOf(const ArrayData chunk, size_t i) @trusted pure nothrow

private fnsize_t hashCategoricalRow(const ArrayData chunk, size_t i) @safe pure nothrow

private fn

bool columnEqDecimal(const  ArrayData a,  size_t ia,  const  ArrayData b,
       size_t ib)

@safe pure nothrow

private fnlong coeffOf(const ArrayData chunk, size_t i) @trusted pure nothrow

private fnsize_t hashDecimalRow(const ArrayData chunk, size_t i) @safe pure nothrow

fnColumnEq columnEqOf(DataType dtype) @safe pure nothrow

private fnsize_t hashRow(Kind K)(const ArrayData chunk, size_t i) @safe pure nothrow

private fnsize_t combineHash(size_t h, size_t next) @safe pure nothrow

private fnsize_t hashListRow(const ArrayData chunk, size_t i) @trusted pure nothrow

private fnsize_t hashStructRow(const ArrayData chunk, size_t i) @trusted pure nothrow

fnsize_t rowHashOf(Kind k, const ArrayData chunk, size_t i) @safe pure nothrow

fnsize_t rowHashCombine(size_t h, size_t next) @safe pure nothrow

void rowHashColumn(Kind k,  const  ArrayData[]  chunks,
       size_t[]  hashes)

@safe pure nothrowCombines one more key column's per-row hashes into the running `hashes` array, densely: one typed view per chunk and one validity test per row - no per-row view rebuilds, no `Nullable` traffic. Nul...

private fn

void rowHashColumnRich(Kind k,  const  ArrayData[]  chunks,
       size_t[]  hashes)

@safe pure nothrow

private fn

void rowHashColumnOf(Kind K)(const  ArrayData[]  chunks,
       size_t[]  hashes)

@safe pure nothrow

private fn

void rowHashChunkOf(Kind K)(const  ArrayData chunk,  size_t at,
       size_t[]  hashes)

@safe pure nothrow

void rowHashColumnParallel(Kind k,
       const  ArrayData[]  chunks,  size_t[]  hashes)

@safeThe chunk-parallel form of `rowHashColumn`: every chunk writes its own disjoint range of `hashes`, so the result is identical to the serial pass whatever the scheduling (ADR-2026-10-19-02).

ArrayData[] isInRange(ArrayData[]  input,  ArrayData[]  set_,
       DataType dtype)

@safeThe membership kernel for the rich kinds: `isInKernel`'s table and probe with the runtime-kind structural hash and equality (the column equality and row hash above). Null input slots stay null; nul...

fnSeries uniqueOf(Series s)

fnsize_t nUniqueOf(Series s)Params: s = the column

private fnvoid rehash(ref Entry[] table, ref bool[] used)

auto locateGlobal(const(ArrayData)[]  chunks,
       size_t global)

Variables 1

enumvarNULL_HASH = 0x9E37_79B9_7F4A_7C15UL

The null sentinel of the row hash (fact 7).

Templates 1

tmplisInKernel(Kind K)

The membership kernel of Series.isIn (ADR-2026-10-23-08): the set side is hashed into an open-addressed (hash, row) table by linear probing (null set slots never enter it - they cannot match), and every non-null input slot probes its hash chain, re-verifying each candidate through the typed total equality - a hash collision can never become a false match, and NaN/-0.0 coalescing in hashOf guarantees equal-under-total values share a chain. Null input slots stay null; the BOOLEAN result follows the input chunking.

Parameters

`K`	the kind of both sides
`input`	the probed side's chunks
`set_`	the set side's chunks

Returns

the BOOLEAN membership chunks, one per input chunk.

Functions

ArrayData[] isInKernel(ArrayData[] input, ArrayData[] set_)