ddn.df.groupby

The grouping layer of ddn.df: DataFrame.groupBy returning a GroupBy with groups, len and agg (T27; D12; ADR-2026-10-18-05, ADR-2026-10-18-15).

The GroupTable maps (row hash, representative row) to a group id in first-appearance order — free with open addressing, where Polars needs maintain_order=True. The aggregation kernels of ddn.df.compute (one-slot ArrayData producers, ADR-2026-10-18-14) produce the per-group output columns directly, one slot per group.

Implements ADR-2026-10-18-20 (amending ADR-2026-10-18-14): the grouped aggregations walk the group's row list through cached typed views - same one-slot results, same semantics - instead of gathering per group, and the group rows live in one flat index array.

Copyright

License

BSD-3-Clause

tmpl AggBuild AggElem aggRowsBatched aggRowsKernel

enum AggKind

struct AggSpec GroupBy GroupTable

fn aggCollect aggColumn aggColumnBatched aggDtypeOf aggFinish aggInto aggStat chunkIndexOf collect count first groupByDynamic iotaLabels iotaLongsLocal isFastWordKind last max mean median min overAgg product quantile scatterBack std sum sumDtypeOf variance

Types 4

enumAggKind

The aggregation kinds GroupBy.agg accepts.

SUMThe sum in its promoted dtype.

COUNTThe non-null count.

MEANThe FLOAT64 (FLOAT32 for FLOAT32) arithmetic mean.

MINThe dtype-preserving minimum (total order on floats).

MAXThe dtype-preserving maximum (total order on floats).

FIRSTThe first element of the group.

LASTThe last element of the group.

COLLECTThe group's rows as one LIST element (row order within the group; null members stay null; ADR-2026-10-23-14).

PRODUCTThe product in the sum's promoted dtype (identity 1; integers wrap; ADR-2026-10-28-01).

QUANTILEThe linear-interpolation quantile at `param` (FLOAT64; ADR-2026-10-28-01).

STDThe sample standard deviation at `param` ddof (FLOAT64; ddof >= group count reads null; ADR-2026-10-28-01).

VARIANCEThe sample variance at `param` ddof (FLOAT64; ddof >= group count reads null; ADR-2026-10-28-01).

structAggSpec

One aggregation specification for GroupBy.agg.

Fields

AggKind kindThe aggregation kind.

string columnThe source column name.

string outputNameThe output column name (defaults to the source name).

double paramThe statistical parameter: `q` for QUANTILE, `ddof` for STD/VARIANCE (ignored by the parameterless kinds; ADR-2026-10-28-01).

Constructors

this(AggKind kind,  string column,  string outputName =  "",
          double  param =  0)

Constructs a specification.

private structGroupTable

Fields

private size_t[] _hashes

private size_t[] _rows

private size_t _count

private const(ArrayData)[][] _keyChunks

private size_t[][] _keyStarts

private const(ColumnEq)[] _keyEqs

private const(ArrayData)[] _fastCol

private size_t[] _fastStarts

private Kind _fastKind

private ulong[] _fastWords

Methods

private size_t intern(size_t h, size_t globalRow)

private void grow()

private ulong fastWord(size_t row) @safe pure nothrow

private bool rowEq(size_t a, size_t b)

private const(ArrayData) chunkOf(size_t c, size_t row) @safe pure nothrow

private size_t localOf(size_t c, size_t row) @safe pure nothrow

Constructors

this(const(ArrayData)[][]  keyChunks,  const  size_t[][]  keyStarts,
          const(ColumnEq)[]  keyEqs)

structGroupBy

The grouped view of a frame: the key columns plus per-group row lists.

Fields

private DataFrame _frame

private string[] _keys

private size_t[][] _groupRows

private size_t[] _order

Methods

size_t[] representativeRows() @safe pure nothrowThe first-appearance representative row of every distinct key tuple, in discovery order - the frame-level `unique` keep-first semantics (ADR-2026-10-23-08). Key equality is the grouping's total mul...

size_t count()Returns: the number of groups.

size_t[][] groups()The per-group row index lists, in first-appearance order.

DataFrame len()The keys plus a UINT32 row count per group (nulls included, the `GroupBy.len` of the plan).

DataFrame agg(AggSpec[] specs...)Aggregates: one output column per specification (keys first, then aggregations in order). The kernels are the one-slot `ArrayData` producers, applied per group's row list.

Constructors

this(DataFrame frame, string[] keys)Groups a frame by its key columns. Prefer `DataFrame.groupBy`.

Functions 29

fnAggSpec sum(string column, string as = "")Factory for the sum of `column`.

fnAggSpec collect(string column, string as = "")Factory for the count of `column`.

fnAggSpec count(string column, string as = "")

fnAggSpec mean(string column, string as = "")Factory for the mean of `column`.

fnAggSpec min(string column, string as = "")Factory for the minimum of `column`.

fnAggSpec max(string column, string as = "")Factory for the maximum of `column`.

fnAggSpec first(string column, string as = "")Factory for the first element of `column`.

fnAggSpec last(string column, string as = "")Factory for the last element of `column`.

fnAggSpec product(string column, string as = "")Factory for the product of `column` (the sum's promotion; identity 1; integers wrap; nulls skipped - ADR-2026-10-28-01).

fnAggSpec median(string column, string as = "")Factory for the median of `column` (the 0.5 quantile; FLOAT64).

fnAggSpec quantile(string column, double q, string as = "")Factory for the `q` quantile of `column` (linear interpolation on the total order; FLOAT64; `q` must lie in [0, 1]).

fnAggSpec std(string column, size_t ddof = 1, string as = "")Factory for the sample standard deviation of `column` (two-pass; FLOAT64; `ddof` >= the group's valid count reads null).

fnAggSpec variance(string column, size_t ddof = 1, string as = "")Factory for the sample variance of `column` (two-pass; FLOAT64; `ddof` >= the group's valid count reads null).

private fnbool isFastWordKind(Kind k) @safe pure nothrow

private fnsize_t chunkIndexOf(const size_t[] starts, size_t row) @safe pure nothrowThe binary search of the chunk holding a global row over its start table (logarithmic where `locateGlobal` walks the chunk list from the front every call).

private fn

Series aggCollect(string outName,  Series col,
       size_t[][]  groupRows)

Aggregates one column over every group's row list, producing the single-chunk output Series of one slot per group.

private fn

Series aggStat(AggKind kind,  double  param,  string outName,
       Series col,  size_t[][]  groupRows)

Series aggColumn(string agg)(string outName,  Series col,
       size_t[][]  groupRows)

Series aggColumnBatched(string agg)(string outName,
       Series col,  size_t[][]  groupRows)

The batched sibling of `aggColumn` (ADR-2026-10-24-20): the same slot arithmetic - one `aggInto` per group, the very statements the one-slot kernels run - appended into ONE builder, so the output i...

private fn

void aggInto(string agg,  Kind K,  Build)(ref  GatherCursor!K cursor,
       const  size_t[]  rows,  ref  Build b)

The aggregation arithmetic itself: appends exactly one slot (value or null) into `b` - the same statements the one-slot kernel has always executed, moved verbatim so the one-slot and batched paths ...

private fnArrayData aggFinish(string agg, Kind K, Build)(ref Build b)

fnDataType aggDtypeOf(AggKind agg, DataType d) @safe pure nothrowThe promoted sum dtype of one kind, mirroring the one-slot sum kernel of `ddn.df.compute`: narrow integers widen to INT64, BOOLEAN sums into UINT32, everything else keeps its own kind.

private fnDataType sumDtypeOf(Kind K) @safe pure nothrow

private fnstring[] iotaLabels(size_t n) @safe pure nothrow

private fnlong[] iotaLongsLocal(size_t n) @safe pure nothrow

Series overAgg(DataFrame df,  string column,  string[]  keys,
       string agg)

@trustedComputes a partitioned aggregation and scatters each partition's result back to every input row of that partition (the `over` semantics of ADR-2026-10-24-17: the output has the input's length and r...

Series overAgg(DataFrame df,  string column,  string[]  keys,  string agg,
       double  param)

@trustedDitto, with the statistical parameter: `q` for "quantile" (default 0.5 - "median" is its alias), `ddof` for "std" and "variance" (default 1; ADR-2026-10-28-01).

private fn

Series scatterBack(Series values,  size_t[]  rowToGroup,
       string name)

@trusted

DataFrame groupByDynamic(DataFrame df,  string timeColumn,  long  every,
       long  period,  long  offset =  0,  Closed closed =  Closed.LEFT,
       size_t minPeriods =  1,  AggSpec[]  specs...)

@trustedThe dynamic time buckets (ADR-2026-10-30-03): windows anchored at `epoch + offset`, advancing by `every`, each row joining every bucket whose period contains it - `Closed.LEFT` the `[start, start +...

Templates 4

tmplAggElem(string agg, Kind K)

tmplAggBuild(string agg, Kind K)

tmplaggRowsKernel(string agg)

One aggregation over one group's row list, at the column's own kind: the one-slot semantics of the ddn.df.compute kernels (the sum promotion, the double mean accumulator with one rounding, the total order on floats, null-skip and null-propagation) applied through one cached view per chunk - no gather materialised.

Parameters

`agg`	the aggregation kernel name
`K`	the aggregated kind
`chunks`	the aggregated column's chunks
`chunkStarts`	the global start index of every chunk
`rows`	the group's rows, in original order

Returns

the one-slot result chunk.

Functions

ArrayData aggRowsKernel(Kind K)(const ArrayData[] chunks, const size_t[] chunkStarts, const size_t[] rows)

tmplaggRowsBatched(string agg)

Functions

ArrayData aggRowsBatched(Kind K)(const ArrayData[] chunks, const size_t[] chunkStarts, const size_t[][] groupRows)