License
BSD-3-Clause
ddn.columnar.column
The Series layer of ddn.df: a named, typed column of chunks (D11).
A Series is a value type holding an array of immutable ArrayData chunks. Copying a Series copies the chunk reference array, not the data; append on a copy does not affect the original, consistent with D slice semantics. Chunk boundaries are tracked in chunkStarts so element access locates a slot's chunk by binary search. Logical equality compares dtype, length and values regardless of chunking.
Copyright
Copyright (c) 2026, DDN (D Developer Network) Members
alias Series
fn allNullChunk allNullIndices bitmapWords blitWords carriedBackward carriedForward dropNullsKernel elementsEqual elementsEqualKernel elementTextOf fillDirectionKernel gatherBits gatherStrings gatherValues locateIn nullPredKernel rechunkKernel sealWords takeKernel takeNullsBits takeNullsKernel takeNullsStrings takeWithNulls
Types 4
A named column: one dtype, zero or more immutable chunks. Constructed from D arrays (plain, with a validity array, or of Nullable values) or from ready chunks via fromChunks. The historical name of Column (one canonical type); the dataframe package re-exports this alias.
structColumn
Fields
private string _nameprivate DataType _dtypeprivate ArrayData[] _chunksprivate size_t _lengthprivate size_t _nullCountprivate size_t[] _chunkStartsMethods
Series fromChunks(string name, DataType dtype, ArrayData[] chunks) staticConstructs from ready chunks, validating uniform dtype.ArrayData[] chunks() @safe pure nothrowReturns: the chunks (immutable data; the array itself is a copy).private
size_t locate(size_t i) const @safe pure nothrowLocates the chunk holding the global index `i` by binary search over the chunk-start table.Nullable!T get(T)(size_t i)The typed element accessor: `T` must be the logical element type of the dtype (`Timestamp` on DATETIME, `TimeOfDay` on TIME, the physical type on the numeric kinds, `bool`, `string`). `get!long` on...T physical(T)(size_t i)The raw accessor: `T` must be the physical storage type of the dtype (`long` on the temporal kinds; BOOLEAN and STRING have no scalar physical type - use `get!bool` and `get!string`). Null slots re...ChunkedArray!T as(T)()Returns this column as a `ChunkedArray!T`: an indexable range of `Nullable!T` that also exposes the typed chunk views.T[] toArray(T)()Materialises the column into a plain D array.Nullable!T[] toNullableArray(T)()Materialises the column into an array of optional values.Series slice(size_t start, size_t len)Zero-copy slice: each result chunk is a slice of the overlapping input chunk, so only the chunk arrays are allocated, never the data. Null counts are recomputed per sliced chunk.Series append(Series other)Appends another Series of the same dtype: O(1), the result references both chunk arrays.Series isNull()A BOOLEAN Series that is `true` at null slots. The sentinel validity (no nulls) yields all `false`.private
Series mapPredChunks(bool invertNull)private
Series nullMaskRich(bool invertNull)bool equals(Series other, bool nullEqual = true)Logical equality: same dtype, same length, and element-wise equal values, independent of chunking and offsets. Floats compare with IEEE `==` (sorting and grouping use a total order instead). With `...Series shift(long n)Keeps the elements whose mask entry is `true`; a null mask entry counts as `false`.private
Series positionalArith(string op)(long n)Series diff(long n = 1)The positional difference (ADR-2026-10-23-08): `x[i] - x[i - n]`, null for the first `n` slots and wherever either operand slot is null. Numeric kinds only (temporal arithmetic stays rejected); int...Nullable!double variance(size_t ddof = 1)The percentage change (ADR-2026-10-23-08): `(x[i] - x[i - n]) / x[i - n]` as FLOAT64 - null for the first `n` slots and *wherever the denominator slot is null or zero* (the guarded-division rule; n...Nullable!double std(size_t ddof = 1)Ditto, the standard deviation (`sqrt(variance)` - non-negative by construction), sample by default.Series rank(RankMethod method, bool descending = false)Ranks the valid values under the total order (ADR-2026-10-23-11): 1-based; ties receive the method's position; null slots stay null; `descending` reverses the value order, not the tie semantics. `A...Series cumMin()Ditto, the cumulative minimum (the extrema's total order; NaN sticks as the greatest value once met).Series floor()Rounds down (toward negative infinity) - FLOAT32/64 only (ADR-2026-10-23-08); integers are rejected, users cast deliberately.Series round()Rounds to the nearest whole value, *half away from zero* (D's `std.math.round` and Polars agree; ADR-2026-10-23-08) - FLOAT32/64 only.Series sqrt()The unary mathematical transforms (ADR-2026-10-30-01): `sqrt`, `cbrt`, `exp`, `log`, `log2`, `log10`, `log1p`, `expm1`, `sin`, `cos`, `tan`, `asin`, `acos` and `atan` share one contract. FLOAT32 an...Series pow(Series exponent)Raises the column to the `exponent` column, elementwise (ADR-2026-10-30-01): float pairs compute at their supertype (FLOAT32 x FLOAT32 stays FLOAT32); integer pairs stay integer with wrap-like-D mu...Series atan2(Series x)The angle whose tangent is `this / x`, elementwise (`atan2(y, x)`; ADR-2026-10-30-01): float pairs compute at their supertype; integer pairs promote to FLOAT64; nulls propagate.Series clip(T)(T lower, T upper)Clips to the inclusive `[lower, upper]` bounds (ADR-2026-10-23-08): numeric kinds; bounds cast to the dtype (out-of-range bounds fail fast); *reversed bounds throw before any work*; nulls and NaN p...size_t[] maskIndices(Series mask) staticThe indices whose mask entry is `true`; a null mask entry counts as `false`. The walk is word-wise over the mask's packed bits: a counting pass per chunk runs as independent parallel units, an orde...Series castTo(DataType target, bool strict = true)Converts the column to the target dtype, chunk by chunk. See `ddn.df.cast.castChunk` for the full source/target behaviour table; nulls are always preserved, `strict = false` maps convertible failur...string toString()Renders the Series Polars-style:string elementText(size_t i)Renders one element for display: `null`, quoted strings, temporals through the `ddn.datetime` formatters, `to!string` otherwise (the documented precision for floats is D's shortest round-trip). Thi...size_t[] argSort(bool descending = false, bool nullsLast = false)The permutation that sorts this column: stable `(key, index)` pair sort with the index tie-break always applied (D17), nulls first by default, floats under the total order (NaN greatest). See `ddn....Series sort(bool descending = false, bool nullsLast = false)The sorted column: `take(argSort(...))`.Series opBinary(string op)(Series other) if (op == "+" || op == "-" || op == "*" || op == "/" || op == "%")The arithmetic operators with another Series: the supertype rules of `ddn.columnar.compute.binaryArith` apply (one dispatch, nulls propagate, integer guards, `/` promoting to FLOAT64).Series opBinary(string op, T)(T scalar) if ((op == "+" || op == "-" || op == "*" || op == "/" || op == "%")
&& isSupportedScalar!T && !is(T == Series))Ditto, with a broadcast scalar on the right.Series opBinaryRight(string op, T)(T scalar) if ((op == "+" || op == "-" || op == "*" || op == "/" || op == "%")
&& isSupportedScalar!T)Ditto, with a broadcast scalar on the left.Series opBinary(string op)(Series other) if (op == "&" || op == "|" || op == "^" || op == "<<"
|| op == ">>")The integer bitwise operations (ADR-2026-10-30-01): `&`, `|`, `^`, `<<` and `>>` between two columns over integer kinds paired at the arithmetic supertype; nulls propagate; shift counts must satisf...Series opBinary(string op, T)(T scalar) if ((op == "&" || op == "|" || op == "^" || op == "<<"
|| op == ">>") && isSupportedScalar!T && !is(T == Series))Ditto, with a broadcast scalar on the right.Series opBinaryRight(string op, T)(T scalar) if ((op == "&" || op == "|" || op == "^" || op == "<<"
|| op == ">>") && isSupportedScalar!T)Ditto, with a broadcast scalar on the left.Series eq(Series other)The comparison with another Series (D cannot overload `==` to return a Series): same-kind rule, numeric pairs at the supertype, null operands yielding null, IEEE float semantics.bool any()The sum in the aggregation result dtype, read and strictly converted to `T` (`sum!byte` of an INT64 column holding 300 throws). Sum of an empty or all-null column is 0; BOOLEAN sums into UINT32; in...bool all()Ditto, the all-reduction: `true` iff no non-null slot is `false` (empty or all-null: `true`).Nullable!size_t argMin()The lowest-index winning slot under the minimum's total order (ADR-2026-10-23-08): the earliest occurrence of the minimum value; nulls never win; null when the column is empty or all-null.Nullable!T product(T)()The product in the sum's promoted dtype: mirrors the sum exactly (ADR-2026-10-28-01) - the same promotion, per-chunk partials in chunk order, identity 1 for the empty/all-null column, wrapping inte...Nullable!double mean()The arithmetic mean: FLOAT64 accumulation (FLOAT32 columns round once at the end); null for an empty or all-null column.Series fillNull(T)(T value)Replaces null slots with a scalar of a compatible type, under the strict cast rules (the value must be representable in the column's dtype).Series fillNull(FillStrategy strategy)Fills nulls by strategy. `FORWARD` copies the previous valid row (leading nulls stay null), `BACKWARD` copies the next valid row (trailing nulls stay null); both run across chunk boundaries. `MEAN`...private
Series fillStatistic(FillStrategy strategy)Series interpolate()Interpolates null gaps linearly over the implicit row positions (ADR-2026-10-30-01): a null slot strictly between two valid observations at positions `i0` and `i1` takes `lo + (hi - lo) * (i - i0) ...private
Series interpolateWalk(Kind K, T)()private
Series fillScalar(T)(T value)private
Series fillNumeric(T)(T value)private
Series fillDirection(bool forward)Series unique()The distinct values in first-appearance order (nulls once, floats under the total order). See `ddn.columnar.hash.uniqueOf`.size_t nUnique()Returns: the number of distinct values (nulls count once).private
void recomputeStarts() @safe pureRecomputes the chunk-start table and the totals from the chunks.Constructors
this(string name, const(T)[] values)Constructs from plain values, with no nulls. The constraint resolves the ambiguity the unconstrained pair has for `Nullable!T[]` (fact 9): optional input goes to the `Nullable` overload.this(string name, const(Nullable!T)[] values)Constructs from optional values; `null` entries become null slots.this(string name, const(T)[] values, const(bool)[] valid)Constructs from plain values plus a validity array (`true` = valid).structChunkedArray(T)
The typed view of a whole Series: indexable element access returning Nullable!T, an input range over the elements, and the typed chunk views of every chunk.
Methods
structGatherCursor(Kind K)
The chunk cursor of the gather kernels: every chunk's typed view is built once, so locating a global element index never rebuilds a view - ascending index lists (filter, joins) walk forward without the binary search while random lists (sort permutations, group row lists) only pay the search on chunk changes.
Parameters
K | the kind of the chunks being walked |
Fields
private ArrayData[] _chunksprivate const(size_t)[] _startsprivate View[] _viewsprivate size_t _ciMethods
size_t locate(size_t i, ref View view) @safe pure nothrowLocates the global index `i`, refreshing the cached view on chunk changes.Functions 23
fn
Series takeWithNulls(Series col, const size_t[] indices)Gathers the elements at `indices`, in order, with `size_t.max` as the null marker (the left-join right side): the same packed-bit gather `take` uses, with a null slot instead of an out-of-range ele...private fn
ArrayData takeNullsKernel(Kind K)(ArrayData[] chunks,
const size_t[] chunkStarts, const size_t[] indices)The primitive null-marker gather kernel of `takeWithNulls`.private fn
ArrayData takeNullsBits(ArrayData[] chunks,
const size_t[] chunkStarts, const size_t[] indices)The BOOLEAN null-marker gather of `takeWithNulls`, word-wise.private fn
ArrayData takeNullsStrings(ArrayData[] chunks,
const size_t[] chunkStarts, const size_t[] indices)The STRING null-marker gather of `takeWithNulls`.private fn
bool elementsEqualKernel(Kind K)(const ArrayData a, size_t ia,
const ArrayData b, size_t ib)private fn
ArrayData takeKernel(Kind K)(ArrayData[] chunks,
const size_t[] chunkStarts, const(size_t)[] indices)private fn
bool gatherValues(Kind K)(ArrayData[] chunks,
const size_t[] chunkStarts, const(size_t)[] indices,
PhysicalOf!K[] values, ulong[] valid) @safeGathers raw primitive values and their packed validity bits at the given global indices (null slots hold arbitrary values, D14).private fn
void gatherBits(ArrayData[] chunks, const size_t[] chunkStarts,
const(size_t)[] indices, ref BooleanBuilder b) @safeGathers BOOLEAN value and validity bits at the given global indices, packing 64 output slots per builder word.private fn
ArrayData allNullChunk(DataType dtype, size_t n) @safe pure nothrowBuilds a single all-null chunk of `n` slots for `dtype` without any cursor walk: the null-marker gather's zero-chunk fast path (a zero-row column gathered under left joins).private fn
immutable(ulong)[] sealWords(ref ulong[] buffer) @trusted pure nothrowSeals a bitmap word buffer into the immutable storage `Bitmap` reads.private fn
ArrayData gatherStrings(ArrayData[] chunks,
const size_t[] chunkStarts, const(size_t)[] indices) @safeGathers STRING slots at the given global indices into one sealed chunk, in parallel above the row threshold: word-aligned output segments size their payload with their own cursor (disjoint `segByte...