ddn.compressor.snappy

ddn.compressor.snappy

Snappy compression provider for ddn.api.compressor.

This module provides a complete Snappy compression and decompression implementation conforming to the Snappy raw format specification. It includes:

  • Streaming Compressor and Decompressor classes implementing the ddn.api.compressor interface.
  • Hash-based match finding for LZ77-style compression.
  • Full support for all Snappy element types (literals, 1/2/4-byte copies).
  • Interoperability with the standard Snappy library format.

Snappy is designed for speed rather than maximum compression ratio. It achieves reasonable compression at very high speeds, making it suitable for real-time applications, database storage, and RPC systems.

Format specification: https://github.com/google/snappy/blob/main/format_description.txt

Copyright

BSD-3-Clause
var SNAPPY_HASH_BITS SNAPPY_HASH_SIZE SNAPPY_MAX_LEN_1 SNAPPY_MAX_LEN_2 SNAPPY_MAX_OFFSET_1 SNAPPY_MAX_OFFSET_2 SNAPPY_MIN_MATCH
class SnappyCompressor SnappyDecompressor
fn compressBlock compressBlockFast decodeVarint decompressBlock emitCopy emitLiteral encodeVarint equal4 makeSnappyCompressor makeSnappyDecompressor readLE32 snappyHashFast

Module Initializers 1

shared static this()

Types 2

classSnappyCompressor : Compressor

Snappy compressor implementing the Compressor interface.

Buffers all input until finish() is called, then compresses and emits the complete Snappy stream.

Fields
private CompressionOptions _opts
private OutputSink _sink
private ubyte[] _buffer
private ulong _bytesIn
private ulong _bytesOut
private bool _finished
Methods
CompressionOptions options() @property constReturns the compression options.
void setOutputSink(OutputSink sink)Set the output sink for compressed data.
void setProgressCallback(ProgressCallback callback)Set an optional progress callback.
ulong bytesInTotal() @property constReturns total bytes written to the compressor.
ulong bytesOutTotal() @property constReturns total bytes emitted by the compressor.
void write(const(ubyte)[] data)Write data to be compressed.
void flush(FlushMode mode = FlushMode.SYNC)Flush pending data.
void finish()Finish compression and emit all output.
void reset()Reset the compressor for reuse.
bool setDictionary(const(ubyte)[] dict)Set a compression dictionary (not supported by Snappy).
bool isFinished() @property constReturns true if finish() has been called and the stream is closed for further writes.
Constructors
this(CompressionOptions opts)Construct a new Snappy compressor.
classSnappyDecompressor : Decompressor

Snappy decompressor implementing the Decompressor interface.

Buffers all input until finish() is called, then decompresses and emits the complete uncompressed data.

Fields
private DecompressionOptions _opts
private OutputSink _sink
private ubyte[] _buffer
private ulong _bytesIn
private ulong _bytesOut
private bool _finished
Methods
DecompressionOptions options() @property constReturns the decompression options.
void setOutputSink(OutputSink sink)Set the output sink for decompressed data.
void setProgressCallback(ProgressCallback callback)Set an optional progress callback.
ulong bytesInTotal() @property constReturns total bytes written to the decompressor.
ulong bytesOutTotal() @property constReturns total bytes emitted by the decompressor.
void write(const(ubyte)[] data)Write compressed data to be decompressed.
void finish()Finish decompression and emit all output.
void reset()Reset the decompressor for reuse.
bool setDictionary(const(ubyte)[] dict)Set a decompression dictionary (not supported by Snappy).
bool isFinished() @property constReturns true if finish() has been called and the stream is closed for further writes.
Constructors
this(DecompressionOptions opts)Construct a new Snappy decompressor.

Functions 12

private fnsize_t encodeVarint(uint value, ubyte[] output)Encode a value as a little-endian varint.
private fnsize_t decodeVarint(const(ubyte)[] input, out uint value)Decode a little-endian varint from input.
private fnuint snappyHashFast(const(ubyte) * p) pure nothrow @nogcFast inline hash using pointer cast for 32-bit LE read.
private fnuint readLE32(const(ubyte) * p) pure nothrow @nogcRead a 32-bit little-endian word from an unaligned pointer.
private fnbool equal4(const(ubyte) * a, const(ubyte) * b) pure nothrow @nogcCompare two unaligned 4-byte sequences for equality.
private fnvoid emitLiteral(ref ubyte * op, const(ubyte) * src, size_t len) pure nothrow @nogcEmit a literal element using raw pointers.
private fnvoid emitCopy(ref ubyte * op, size_t offset, size_t length) pure nothrow @nogcEmit a copy (backreference) element using raw pointers.
private fnsize_t compressBlockFast(const(ubyte)[] src, ubyte[] outBuf) @trustedFast single-pass compression core using raw pointers.
private fnsize_t compressBlock(const(ubyte)[] input, ubyte[] output)Compress a block of data using Snappy format.
private fnsize_t decompressBlock(const(ubyte)[] input, size_t expectedLen, ubyte[] output)Decompress a Snappy-compressed block.
fnCompressor makeSnappyCompressor(CompressionOptions opts)Create a Snappy compressor.
fnDecompressor makeSnappyDecompressor(DecompressionOptions opts)Create a Snappy decompressor.

Variables 7

private enumvarSNAPPY_MIN_MATCH = 4

Minimum match length for Snappy (4 bytes).

private enumvarSNAPPY_MAX_OFFSET_1 = 2047

Maximum offset for 1-byte copy (11 bits = 2047).

private enumvarSNAPPY_MAX_OFFSET_2 = 65535

Maximum offset for 2-byte copy (16 bits = 65535).

private enumvarSNAPPY_MAX_LEN_1 = 11

Maximum length for 1-byte copy encoding.

private enumvarSNAPPY_MAX_LEN_2 = 64

Maximum length for 2-byte and 4-byte copy encoding.

private enumvarSNAPPY_HASH_BITS = 14

Hash table size (64K entries for good match finding).

private enumvarSNAPPY_HASH_SIZE = 1 << SNAPPY_HASH_BITS