ddn.df.ipc_io

Frame and query adapters over the Arrow IPC codec: the eager file readers, the frame writer with the approved temp-then-rename publication policy, the lazy scan source and the incremental sink. Everything frame-, plan- or query-shaped lives here; the codec tier itself (IpcBatchReader, IpcBatchWriter, buildIpcBytes over ColumnBatch) stays in ddn.df.ipc and never imports this module.

Copyright

Copyright (c) 2026, DDN (D Developer Network) Members

License

BSD-3-Clause
class IpcIncrementalWriter IpcScanSource
fn buildIpcBytes readIpc readIpcFile scanIpc writeIpcFile

Types 2

private classIpcIncrementalWriter
Fields
private string path
private string tmp
private IpcBatchWriter w
Methods
void addBatch(DataFrame batch) @trusted
void finish() @trusted
void disposeTemp() @trusted
Constructors
this(string path, ColumnSchema[] schema, IpcCompression compression)
private classIpcScanSource : ScanSource
Fields
private string path
private IpcBatchReader reader
Methods
ColumnSchema[] schema() @safe
string description() const @safe
Nullable!DataFrame nextBatch() @trusted
void close() @trusted
ScanSource acquire() @safe
bool supportsProjection() @safe
ScanSource project(scope string[] columns) @safe
Constructors
this(string path)

Functions 6

fnDataFrame readIpcFile(string path) @trustedReads an Arrow IPC file (`.arrows`, Feather V2) from disk.
fnDataFrame readIpc(const(ubyte)[] bytes) @trustedReads an Arrow IPC file from bytes (`.arrows`, Feather V2): the footer's schema names the columns, the dictionary batches arrive before the record batches, and each record batch becomes one chunk p...
fnubyte[] buildIpcBytes(DataFrame df, IpcCompression compression = IpcCompression.NONE) @trustedBuilds the file bytes of a frame (the codec tier owns the chunk-alignment policy; this adapter hands it the frame's columns).
fnvoid writeIpcFile(string path, DataFrame df, IpcCompression compression = IpcCompression.NONE) @trustedWrites a frame to an Arrow IPC file (`.arrows`, Feather V2) under the approved failure policy: the bytes stage into a sibling temp file that renames into place on success and is removed on any fail...
fnLazyFrame scanIpc(string path) @trustedOpens an Arrow IPC file (`.arrows`, Feather V2) as a lazy scan source: the header and footer are read eagerly, then each record batch is sought and decoded on demand - one batch per pull, no whole-...
fnvoid writeIpcFile(string path, LazyFrame df, IpcCompression compression = IpcCompression.NONE) @trustedWrites a deferred frame to an Arrow IPC file incrementally: the schema derives from metadata alone, batches pull through the row-wise chain one at a time (the input never materialises), and each pu...