ddn.compressor.zlib

ddn.compressor.zlib

A pure D implementation of the zlib compression library interface. This module provides a drop-in replacement for etc.c.zlib with identical API, implementing the DEFLATE compression algorithm (RFC 1951), zlib format (RFC 1950), and GZIP format (RFC 1952) entirely in D.

Features:

  • Complete DEFLATE compression and decompression
  • Support for raw DEFLATE, zlib, and GZIP container formats
  • Dictionary support for improved compression
  • Streaming API compatible with zlib's z_stream interface
  • Adler-32 and CRC-32 checksum implementations

The implementation follows the zlib API conventions for maximum compatibility with existing code that uses etc.c.zlib.

Copyright

BSD-3-Clause
var codeLengthOrder crcTable distanceBase distanceExtraBits fixedCodes fixedDistanceLengths fixedLiteralLengths lengthBase lengthExtraBits MAX_BITS MAX_CODES MAX_DIST_CODES MAX_MEM_LEVEL MAX_WBITS SEEK_CUR SEEK_END SEEK_SET Z_BEST_COMPRESSION Z_BEST_SPEED Z_BLOCK Z_BUF_ERROR Z_DATA_ERROR Z_DEFAULT_COMPRESSION Z_DEFAULT_STRATEGY Z_DEFLATED Z_ERRNO Z_FILTERED Z_FINISH Z_FIXED Z_FULL_FLUSH Z_HUFFMAN_ONLY Z_MEM_ERROR Z_NEED_DICT Z_NO_COMPRESSION Z_NO_FLUSH Z_OK Z_PARTIAL_FLUSH Z_RLE Z_STREAM_END Z_STREAM_ERROR Z_SYNC_FLUSH Z_TREES Z_VERSION_ERROR ZLIB_VERSION
alias alloc_func c_ulong free_func z_streamp
enum InflateDecodeState
struct BitReader BitWriter FixedCodes HashEntry HuffmanCode HuffmanTable LZ77State Match z_stream
class DeflateState InflateState
fn adler32 buildHuffmanTable compress compress2 compressBound crc32 deflate deflateBlockFixed deflateBlockFixedToBits deflateBlockStored deflateBlockStoredToBits deflateEnd deflateInit deflateInit2 deflateInit2_ deflateReset deflateSetDictionary generateFixedCodes getDistanceCode getLengthCode inflate inflateEnd inflateInit inflateInit2 inflateInit2_ inflateReset inflateSetDictionary reverseBits uncompress

Module Initializers 1

shared static this()

Types 16

aliasalloc_func = void * function(void * opaque, uint items, uint size)

Callback type for custom memory allocation

aliasfree_func = void function(void * opaque, void * address)

Callback type for custom memory deallocation

structz_stream

The z_stream structure is used to pass information between the application and the compression/decompression library functions.

This structure mirrors the zlib z_stream_s struct for API compatibility.

Fields
ubyte * next_inInput buffer pointer
uint avail_inNumber of bytes available at next_in
c_ulong total_inTotal number of input bytes read so far
ubyte * next_outOutput buffer pointer
uint avail_outRemaining free space at next_out
c_ulong total_outTotal number of bytes output so far
const(char) * msgLast error message, null if no error
void * internalInternal state (opaque to application)
alloc_func zallocCustom allocator function (not used in pure D implementation)
free_func zfreeCustom deallocator function (not used in pure D implementation)
void * opaquePrivate data for custom allocator
int data_typeBest guess about the data type: ZBINARY or ZTEXT
c_ulong adlerAdler-32 or CRC-32 checksum value
c_ulong reservedReserved for future use
aliasz_streamp = z_stream *

Pointer to z_stream

aliasc_ulong = size_t

C unsigned long type alias for cross-platform compatibility

structHuffmanCode

Huffman code table entry

Fields
ushort symbol
ubyte bits
structHuffmanTable

Huffman decode table

Fields
HuffmanCode[1 << MAX_BITS] table
int maxBits
structBitReader

Bit reader for input stream

Fields
const(ubyte)[] data
size_t pos
ulong bits
int numBits
Methods
void init(const(ubyte)[] input) nothrow @safeInitialize the bit reader.
void setInput(const(ubyte)[] input) nothrow @safeSet new input data without resetting the bit buffer. Used for streaming decompression where leftover bits from the previous call must be preserved.
bool ensure(int n) nothrow @safeEnsure at least n bits are available in the buffer.
uint peek(int n) nothrow @safePeek at n bits without consuming them.
void drop(int n) nothrow @safeConsume n bits.
uint read(int n) nothrow @safeRead n bits and consume them.
void alignToByte() nothrow @safeAlign to byte boundary by discarding partial bits.
int readByte() nothrow @safeRead a byte directly (after byte alignment).
size_t bytesRemaining() const nothrow @safeGet remaining bytes available (not yet loaded into bit buffer).
size_t totalBytesRemaining() const nothrow @safeGet total unprocessed bytes (including those in bit buffer).
void discardBits() nothrow @safeDiscard any bits in the buffer and reset to byte-aligned reading. This effectively "unreads" any partial bytes in the bit buffer.
structBitWriter

Bit writer for output stream

Fields
ubyte[] output
size_t pos
uint bits
int numBits
Methods
void init(ubyte[] buf) nothrow @safeInitialize the bit writer.
bool write(uint value, int n) nothrow @safeWrite n bits to the output.
bool writeByte(ubyte b) nothrow @safeWrite a byte directly.
bool flush() nothrow @safeFlush remaining bits (pad with zeros to byte boundary).
void alignToByte() nothrow @safeAlign to byte boundary.
size_t bytesWritten() const nothrow @safeGet number of bytes written.
structHashEntry

Hash chain entry for LZ77 matching

Fields
ushort pos
ushort prev
structMatch

LZ77 match result

Fields
int length
int distance
structLZ77State

LZ77 compressor state

Fields
ubyte[] window
int windowSize
int windowMask
int windowPos
int lookahead
ushort[] head
ushort[] prev
int level
int strategy
int maxChainLength
int goodMatchLength
int niceMatchLength
int maxLazyMatch
Methods
void init(int windowBits, int compLevel, int strat)Initialize LZ77 state.
void setLevelParams(int compLevel)Set level-dependent compression parameters.
ushort hash3(const(ubyte) * p) nothrow @trustedCompute hash of 3 bytes.
void insertString(int pos) nothrow @trustedInsert string at current position into hash chain.
Match findMatch(int pos, int prevLength) nothrow @trustedFind longest match at current position.
void reset() nothrow @safeReset the LZ77 state.
classDeflateState

Deflate stream state

Fields
int level
int method
int windowBits
int memLevel
int strategy
int status
bool wrapHeader
int wrapTrailer
bool finished
LZ77State lz77
ubyte[] pending
int pendingPos
int pendingLen
c_ulong checksum
c_ulong crc
ubyte[] dictionary
c_ulong dictChecksum
c_ulong totalIn
c_ulong totalOut
ubyte[] compressedData
ubyte[] blockLiterals
ushort[] blockLengths
ushort[] blockDistances
int blockSize
ubyte[] bitBuffer
size_t bitBufferPos
uint bitAccum
int bitCount
16384 MAX_BLOCK_SIZE
Methods
void init(int lvl, int mthd, int wBits, int mLvl, int strat)Initialize the deflate state.
void writeBits(uint value, int n) nothrow @safeWrite bits to the bit buffer (maintains state across calls)
void flushBits() nothrow @safeFlush remaining bits (pad with zeros to byte boundary)
private void growBitBuffer() nothrow @safeGrow the bit buffer when it becomes full
ubyte[] getBitBufferContents() nothrow @safeGet the current bit buffer contents and reset
void reset()Reset the deflate state.
classInflateState

Inflate stream state

Fields
int windowBits
int status
bool wrapHeader
int wrapTrailer
bool finished
ubyte[] window
int windowSize
int windowPos
BitReader bits
bool lastBlock
int blockType
HuffmanTable litTable
HuffmanTable distTable
ubyte[320] codeLengths
int numLitCodes
int numDistCodes
int numCodeLenCodes
c_ulong checksum
c_ulong expectedChecksum
c_ulong crc
uint expectedCrc
uint expectedSize
ubyte[] dictionary
c_ulong dictChecksum
bool needDict
c_ulong totalIn
c_ulong totalOut
int decodeState
int copyLength
int copyDistance
int pendingLength
int pendingLengthExtra
int pendingLengthBase
int pendingDistanceExtra
int pendingDistanceBase
int headerState
int gzipFlags
int gzipExtra
Methods
void init(int wBits)Initialize the inflate state.
void reset()Reset the inflate state.
structFixedCodes

Fixed Huffman codes for encoding

Fields
ushort[288] litCodes
ubyte[288] litLens
ushort[32] distCodes
ubyte[32] distLens
private enumInflateDecodeState

Inflate decode states

HEADERReading header
BLOCK_HEADERReading block header
STORED_LENReading stored block length
STORED_DATACopying stored block data
DECODE_HUFFMANDecoding Huffman-compressed data
DECODE_LENGTH_EXTRAReading length extra bits (after length code decoded)
DECODE_DISTANCEDecoding distance code (after length decoded)
DECODE_DISTANCE_EXTRAReading distance extra bits (after distance code decoded)
DYNAMIC_HEADERReading dynamic Huffman header
DYNAMIC_CODESReading dynamic Huffman code lengths
COPY_MATCHCopying match data
TRAILERReading trailer
DONEStream complete

Functions 31

fnc_ulong adler32(c_ulong adler, const(ubyte) * buf, uint len) nothrow @trustedCompute the Adler-32 checksum of a data buffer.
fnc_ulong adler32(c_ulong adler, const(ubyte)[] buf) nothrow @trustedditto
fnc_ulong crc32(c_ulong crc, const(ubyte) * buf, uint len) nothrow @trustedCompute the CRC-32 checksum of a data buffer.
fnc_ulong crc32(c_ulong crc, const(ubyte)[] buf) nothrow @trustedditto
fnbool buildHuffmanTable(const(ubyte)[] codeLengths, int numCodes, ref HuffmanTable table) nothrow @safeBuild a Huffman decode table from code lengths.
fnint getLengthCode(int length) nothrow @safeGet the length code for a given match length.
fnint getDistanceCode(int distance) nothrow @safeGet the distance code for a given distance.
fnuint reverseBits(uint code, int len) nothrow @safeReverse bits in a code.
fnFixedCodes generateFixedCodes() nothrow @safeGenerate fixed Huffman codes
fnvoid deflateBlockFixedToBits(DeflateState state, const(ubyte)[] input, bool last) nothrow @safeCompress a block of data using fixed Huffman codes.
fnvoid deflateBlockStoredToBits(DeflateState state, const(ubyte)[] input, bool last) nothrow @safeEncode a stored block directly into the state's bit buffer.
fnubyte[] deflateBlockFixed(DeflateState state, const(ubyte)[] input, bool last)
fnubyte[] deflateBlockStored(const(ubyte)[] input, bool last)Compress data as a stored (uncompressed) block.
fnint deflateInit2_(z_stream * strm, int level, int method, int windowBits, int memLevel, int strategy, const(char) * ver, int streamSize) nothrowInitialize a deflate stream with full parameters.
fnint deflateInit2(z_stream * strm, int level, int method, int windowBits, int memLevel, int strategy) nothrowSimplified deflate initialization.
fnint deflateInit(z_stream * strm, int level) nothrowBasic deflate initialization with default parameters.
fnint deflate(z_stream * strm, int flush) nothrowCompress data.
fnint deflateEnd(z_stream * strm) nothrowFree deflate state and resources.
fnint deflateReset(z_stream * strm) nothrowReset deflate stream to initial state.
fnint deflateSetDictionary(z_stream * strm, const(ubyte) * dictionary, uint dictLength) nothrowSet compression dictionary.
fnint inflateInit2_(z_stream * strm, int windowBits, const(char) * ver, int streamSize) nothrowInitialize an inflate stream with full parameters.
fnint inflateInit2(z_stream * strm, int windowBits) nothrowSimplified inflate initialization.
fnint inflateInit(z_stream * strm) nothrowBasic inflate initialization with default parameters.
fnint inflate(z_stream * strm, int flush) nothrowDecompress data.
fnint inflateEnd(z_stream * strm) nothrowFree inflate state and resources.
fnint inflateReset(z_stream * strm) nothrowReset inflate stream to initial state.
fnint inflateSetDictionary(z_stream * strm, const(ubyte) * dictionary, uint dictLength) nothrowSet decompression dictionary.
fnint compress(ubyte * dest, c_ulong * destLen, const(ubyte) * source, c_ulong sourceLen) nothrowCompress source buffer to destination buffer.
fnint compress2(ubyte * dest, c_ulong * destLen, const(ubyte) * source, c_ulong sourceLen, int level) nothrowCompress source buffer to destination buffer with specified level.
fnint uncompress(ubyte * dest, c_ulong * destLen, const(ubyte) * source, c_ulong sourceLen) nothrowDecompress source buffer to destination buffer.
fnc_ulong compressBound(c_ulong sourceLen) nothrow @safeCalculate upper bound on compressed size.

Variables 44

enumvarZLIB_VERSION = "1.3.1-ddn"

zlib version string for compatibility

enumvarZ_NO_FLUSH = 0

Flush modes for compression/decompression No flush, accumulate output

enumvarZ_PARTIAL_FLUSH = 1

Partial flush (obsolete, use Z_SYNC_FLUSH)

enumvarZ_SYNC_FLUSH = 2

Flush to byte boundary, sync point

enumvarZ_FULL_FLUSH = 3

Flush and reset compression state

enumvarZ_FINISH = 4

Finish the stream

enumvarZ_BLOCK = 5

Stop at next block boundary

enumvarZ_TREES = 6

Request deflate header/trees

enumvarZ_OK = 0

Return codes Success

enumvarZ_STREAM_END = 1

End of stream reached

enumvarZ_NEED_DICT = 2

Dictionary required for decompression

enumvarZ_ERRNO = - 1

File system error

enumvarZ_STREAM_ERROR = - 2

Stream state error

enumvarZ_DATA_ERROR = - 3

Data corrupted

enumvarZ_MEM_ERROR = - 4

Memory allocation failed

enumvarZ_BUF_ERROR = - 5

Buffer too small or no progress possible

enumvarZ_VERSION_ERROR = - 6

zlib version mismatch

enumvarZ_NO_COMPRESSION = 0

Compression levels Store only, no compression

enumvarZ_BEST_SPEED = 1

Fastest compression

enumvarZ_BEST_COMPRESSION = 9

Best compression ratio

enumvarZ_DEFAULT_COMPRESSION = - 1

Default compression level (typically 6)

enumvarZ_DEFAULT_STRATEGY = 0

Compression strategies Normal compression for general data

enumvarZ_FILTERED = 1

Data with random distribution

enumvarZ_HUFFMAN_ONLY = 2

Huffman coding only (no LZ77)

enumvarZ_RLE = 3

Run-length encoding

enumvarZ_FIXED = 4

Use fixed Huffman codes

enumvarZ_DEFLATED = 8

Compression method The deflate compression method

enumvarMAX_WBITS = 15

Window bits range Maximum window bits (32K window)

enumvarMAX_MEM_LEVEL = 9

Memory level range Maximum memory level

enumvarSEEK_SET = 0

Seek modes (for gzip)

enumvarSEEK_CUR = 1
enumvarSEEK_END = 2
private varuint[256] crcTable
enumvarMAX_BITS = 15

Maximum number of bits in a Huffman code

enumvarMAX_CODES = 286

Maximum number of literal/length codes

enumvarMAX_DIST_CODES = 30

Maximum number of distance codes

varubyte[19] codeLengthOrder

Bit lengths for the code length alphabet (RFC 1951 order)

varubyte[29] lengthExtraBits

Extra bits for length codes (257-285)

varushort[29] lengthBase

Base lengths for length codes (257-285)

varubyte[30] distanceExtraBits

Extra bits for distance codes (0-29)

varushort[30] distanceBase

Base distances for distance codes (0-29)

varubyte[288] fixedLiteralLengths

Static Huffman code lengths for literals/lengths (fixed codes from RFC 1951)

varubyte[32] fixedDistanceLengths

Static Huffman code lengths for distances (fixed codes from RFC 1951)

varFixedCodes fixedCodes

Cached fixed codes