ddn.api.archive

ddn.api.archive

A robust, modern, idiomatic D archive API for reading and writing archive formats such as TAR, ZIP, 7ZIP, RAR, CPIO, AR, and similar.

Design goals (inspired by popular APIs and following patterns from ddn.api.compressor):

  • Format-agnostic core interfaces (TAR, ZIP, 7ZIP, RAR, CPIO, AR, etc.).
  • Provider-based extensibility via a registration system.
  • Streaming push model with output sinks (delegates) to avoid forcing a specific IO abstraction.
  • Clear error model with portable error codes.
  • Progress callback support.
  • Cross-platform support (POSIX and Windows).
  • Compression integration with ddn.api.compressor for compound formats (.tar.gz, .tar.bz2, etc.).
  • Graceful degradation: unsupported operations throw UNSUPPORTED_FEATURE, not compile errors.

This module defines the API only. Concrete implementations (e.g., TAR, ZIP) should live in their own packages/modules and register themselves via the provider registration functions herein.

Types 34

enumArchiveFormat : ushort

Supported archive formats.

AUTO_DETECT = 0Detect from magic bytes or extension
USER = 1Custom format provided by user
TAR = 2POSIX TAR (ustar, pax, gnu)
ZIP = 3ZIP archive (PKZIP)
SEVEN_ZIP = 47-Zip archive
RAR = 5RAR archive (READ-ONLY - proprietary format)
CPIO = 6CPIO archive (various sub-formats)
AR = 7Unix AR archive (used by .deb, .a)
ISO9660 = 8ISO 9660 CD-ROM image (read-only)
CAB = 9Microsoft Cabinet
XAR = 10eXtensible ARchive
SHAR = 11Shell archive (read-only)
enumTarFormat : ubyte

TAR sub-format variants.

AUTO = 0Choose best format based on content
V7 = 1Original Unix V7 tar (limited)
USTAR = 2POSIX.1-1988 ustar format
PAX = 3POSIX.1-2001 pax extended format (recommended)
GNU = 4GNU tar extensions
enumCpioFormat : ubyte

CPIO sub-format variants.

AUTO = 0Detect on read, use NEWC on write
BIN = 1Legacy binary format (obsolete)
ODC = 2POSIX.1 portable ASCII format
NEWC = 3SVR4 "new" portable format (recommended)
CRC = 4SVR4 with CRC checksum
enumEntryType : ubyte

Entry type within an archive.

FILE = 0Regular file
DIRECTORY = 1Directory
SYMLINK = 2Symbolic link
HARDLINK = 3Hard link
FIFO = 4Named pipe (POSIX only)
CHAR_DEVICE = 5Character device (POSIX only)
BLOCK_DEVICE = 6Block device (POSIX only)
SOCKET = 7Unix socket (POSIX only)
UNKNOWN = 255

Platform-specific extraction behavior for unsupported entry types.

SKIPSkip the entry silently
WARNSkip with warning via progress callback
ERRORThrow ArchiveError
EXTRACT_AS_FILEExtract as regular file (symlinks become copies)

Archive error codes.

OK = 0
INVALID_FORMAT = 1Not a valid archive or corrupted header
UNSUPPORTED_FORMAT = 2Format not supported by any provider
UNSUPPORTED_FEATURE = 3Feature not supported by this format/provider
ENTRY_NOT_FOUND = 4Requested entry does not exist
IO_ERROR = 5Read/write error
CHECKSUM_ERROR = 6CRC or hash mismatch
ENCRYPTION_ERROR = 7Decryption failed (wrong password, etc.)
COMPRESSION_ERROR = 8Decompression/compression failed
PATH_ERROR = 9Invalid or dangerous path (e.g., path traversal)
PERMISSION_ERROR = 10Permission denied
TRUNCATED = 11Unexpected end of archive
MEMORY_ERROR = 12Out of memory
INTERNAL_ERROR = 13Internal error
CANCELLED = 14Operation cancelled by callback
enumEncryptionMethod : ubyte

Encryption method for encrypted archives.

NONE = 0
ZIP_CRYPTO = 1Traditional ZIP encryption (weak)
AES_128 = 2AES-128
AES_192 = 3AES-192
AES_256 = 4AES-256
RAR_CRYPTO = 5RAR proprietary encryption
SEVEN_ZIP_AES = 67-Zip AES encryption
classArchiveError : Exception

Archive-related exception with a portable error code.

Fields
Constructors
this(string msg, ArchiveErrorCode code = ArchiveErrorCode.INTERNAL_ERROR, string file = __FILE__, size_t line = __LINE__, Throwable next = null)Construct an ArchiveError.

Capabilities declared by an archive provider.

These flags describe what operations and features a provider supports, allowing callers to make informed decisions before invoking optional features.

Fields
bool canRead
bool canWrite
bool canAppend
bool canUpdate
bool randomAccess
bool streaming
bool encryption
bool compression
bool preservesOwnership
bool preservesPermissions
bool preservesTimes
bool supportsLargeFiles
bool supportsUnicode
bool canStoreDirectories
bool canStoreSymlinks
bool canStoreHardlinks
bool canStoreSpecialFiles
bool canStoreComments

File permissions (POSIX-style, portable representation).

Provides a platform-independent way to represent the 12 POSIX permission bits (rwxrwxrwx + setuid/setgid/sticky) without relying on platform-specific types.

Fields
bool ownerRead
bool ownerWrite
bool ownerExecute
bool groupRead
bool groupWrite
bool groupExecute
bool otherRead
bool otherWrite
bool otherExecute
bool setuid
bool setgid
bool sticky
Methods
uint toOctal() const @safe pure nothrow @nogcConvert to octal mode (e.g., 0o755).
FilePermissions fromOctal(uint mode) @safe pure nothrow @nogcCreate from octal mode.
FilePermissions defaultFile() @safe pure nothrow @nogcCreate default permissions for a regular file (0o644).
FilePermissions defaultDirectory() @safe pure nothrow @nogcCreate default permissions for a directory (0o755).
FilePermissions defaultExecutable() @safe pure nothrow @nogcCreate default permissions for an executable (0o755).

Windows-specific file attributes (stored in archives that support them).

Maps to the Windows FILE_ATTRIBUTE_* flags for round-trip preservation of Windows metadata in archives that support extended attributes.

Fields
bool readonly
bool hidden
bool system
bool archive
bool directory
bool encrypted
bool compressed
bool sparse
bool reparsePoint
bool offline
Methods
uint toDword() const @safe pure nothrow @nogcConvert to Windows DWORD attribute flags.
WindowsAttributes fromDword(uint attrs) @safe pure nothrow @nogcCreate from Windows DWORD attribute flags.

Timestamp with optional nanosecond precision.

Stores timestamps as a Unix epoch offset (seconds + nanoseconds) to enable round-trip preservation of high-resolution timestamps across archive formats.

Fields
long unixSeconds
uint nanoseconds
Methods
auto toSysTime() constConvert to std.datetime.SysTime.
ArchiveTime fromSysTime(T)(T st)Create from a SysTime-like value. The template parameter T must provide toUnixTime() and fracSecs members (e.g., std.datetime.SysTime).
structEntryInfo

Metadata for an archive entry.

Contains all available information about a single item in an archive, including its type, size, timestamps, ownership, permissions, and format-specific data.

Fields
string path
ulong size
ulong compressedSize
uint uid
uint gid
string owner
string group
FilePermissions permissions
WindowsAttributes windowsAttrs
bool hasWindowsAttrs
string linkTarget
CompressionFormat compression
EncryptionMethod encryption
bool isEncrypted
uint crc32
string comment
ubyte[] extra
Methods
bool isDirectory() const @safe pure nothrow @nogcReturns true if this is a directory.
bool isFile() const @safe pure nothrow @nogcReturns true if this is a regular file.
bool isSymlink() const @safe pure nothrow @nogcReturns true if this is a symbolic link.
string baseName() const @safe pureReturns the base name (filename without directory).
string dirName() const @safe pureReturns the directory part of the path.
aliasContentSink = void delegate(const(ubyte)[] chunk)

Output sink for entry content. Called repeatedly with data chunks.

aliasPasswordCallback = string delegate(string archivePath, string entryPath)

Password callback for interactive password prompting. Called when an encrypted entry is encountered and no password was provided in options. Return the password, or null to skip/abort.

Progress information for archive operations.

Fields
string currentEntry
ulong entriesProcessed
ulong entriesTotal
ulong bytesProcessed
ulong bytesTotal
float ratio
aliasArchiveProgressCallback = bool delegate(const ArchiveProgress)

Progress callback. Return false to cancel operation.

Options for reading archives.

Fields
string password
PasswordCallback passwordCallback
bool verifyChecksums
bool allowPathTraversal
size_t maxEntrySize
size_t maxEntries
bool metadataOnly
UnsupportedEntryBehavior unsupportedBehavior
bool convertPathSeparators
bool detectReservedNames
bool caseSensitive

Options for writing archives.

Fields
CompressionFormat compression
string password
EncryptionMethod encryption
bool preserveOwnership
bool preservePermissions
bool preserveTimes
bool preserveWindowsAttrs
string comment
bool normalizePathSeparators
bool storeWindowsAttrs
TarFormat tarFormat
CpioFormat cpioFormat
ulong volumeSize
string volumeNaming

Options for a single entry being written.

Fields
CompressionFormat compression
EncryptionMethod encryption
string comment

Options for reading multi-volume archives.

Fields
string delegate(string currentVolume, int nextVolumeNumber) requestNextVolumeCallback to request next volume. Return path to next volume, or null to abort.
bool autoDetectVolumesWhether to auto-detect volume sequences.
interfaceArchiveReader

Interface for reading archives.

Implementations are provided by archive format providers registered via registerProvider(). Use openArchive() to obtain a reader instance.

Methods that may throw ArchiveError with UNSUPPORTED_FEATURE:

  • getEntry() if !capabilities.randomAccess
  • extractTo() / extractAll() for format-specific limitations
Methods
ArchiveFormat format() @property const;Returns the archive format.
ArchiveCapabilities capabilities() @property const;Returns the provider capabilities.
void setProgressCallback(ArchiveProgressCallback callback)Set progress callback. Return false from callback to cancel.
size_t entryCount() @property const;Get total number of entries (if known). Returns 0 if count is unknown (streaming archives).
string comment() @property const;Get archive-level comment (ZIP).
final auto entries()Iterate over all entries. Returns an input range of EntryInfo.
bool nextEntry(out EntryInfo info)Advance to the next entry in the archive.
bool contains(string path)Check if archive contains an entry at path. May be O(n) for sequential (non-random-access) formats.
EntryInfo getEntry(string path)Get entry info by path (random access archives only).
void readContent(ref const EntryInfo entry, ContentSink sink)Read entry content, calling sink with chunks of decompressed data.
ubyte[] readContent(ref const EntryInfo entry)Read entry content into a buffer. Returns the content as a newly allocated array.
void extractTo(ref const EntryInfo entry, string destPath)Extract entry to filesystem path.
void extractAll(string destDir)Extract all entries to a directory.
void close()Close the reader and release resources.
bool isClosed() @property const;Returns true if reader has been closed.

Input range for iterating over archive entries.

Primed on construction (the first entry is loaded immediately). Use with foreach or range algorithms.

Example:

auto reader = openArchive("archive.zip");
foreach (entry; reader.entries()) {
   writeln(entry.path);
}
reader.close();

Fields
private ArchiveReader _reader
private EntryInfo _current
private bool _empty
Methods
bool empty() @property const
const(EntryInfo) front() @property ref const
void popFront()
Constructors
interfaceArchiveWriter

Interface for writing archives.

Implementations are provided by archive format providers registered via registerProvider(). Use createArchive() to obtain a writer instance.

Call finish() to finalize the archive and produce valid output. Call close() to release all resources.

Methods that may throw ArchiveError with UNSUPPORTED_FEATURE:

  • addDirectory() if !capabilities.canStoreDirectories
  • addSymlink() if !capabilities.canStoreSymlinks
  • addHardlink() if !capabilities.canStoreHardlinks
  • setComment() if !capabilities.canStoreComments
  • removeEntry() if !capabilities.canUpdate
  • updateEntry() if !capabilities.canUpdate
Methods
ArchiveFormat format() @property const;Returns the archive format.
ArchiveCapabilities capabilities() @property const;Returns the provider capabilities.
void setOutputSink(ContentSink sink)Set the output sink for archive bytes.
void setProgressCallback(ArchiveProgressCallback callback)Set progress callback. Return false from callback to cancel.
void addDirectory(string path, FilePermissions perms = FilePermissions.fromOctal(octal!"755"))Add a directory entry.
void addFile(string path, const(ubyte)[] content, EntryWriteOptions opts = EntryWriteOptions.init)Add a file entry with content from a byte slice.
void addFile(string path, scope const(ubyte)[] delegate() source, ulong size = 0, EntryWriteOptions opts = EntryWriteOptions.init)Add a file entry with content from a delegate.
void addFileFrom(string archivePath, string filesystemPath, EntryWriteOptions opts = EntryWriteOptions.init)Add a file entry from the filesystem.
void addSymlink(string path, string target)Add a symbolic link entry.
void addHardlink(string path, string target)Add a hard link entry.
void addEntry(ref const EntryInfo info, scope const(ubyte)[] delegate() source)Add an entry with full metadata control.
void removeEntry(string path)Remove an entry from the archive (if supported).
void updateEntry(string path, const(ubyte)[] content, EntryWriteOptions opts = EntryWriteOptions.init)Update/replace an existing entry.
void setComment(string comment)Set archive-level comment.
void finish()Finalize the archive. Must be called to produce valid output.
void close()Close the writer and release resources.
bool isFinished() @property const;Returns true if finish() has been called.
bool isClosed() @property const;Returns true if writer has been closed.
ulong bytesWritten() @property const;Total bytes written so far.
size_t entriesWritten() @property const;Number of entries written so far.
aliasReaderFactoryFn = ArchiveReader function(const(ubyte)[] data, ReadOptions opts)

Factory function type for creating archive readers.

Factory function type for creating archive writers.

aliasFormatDetectorFn = ArchiveFormat function(const(ubyte)[] header)

Factory function type for detecting archive format from header bytes.

Provider descriptor for archive implementations.

Each provider is uniquely identified by the combination of vendor and format. The full provider name is constructed as vendor-format (e.g., "ddn-zip").

Register providers via registerProvider(). Factory functions may be null for formats that only support reading or only support writing (e.g., RAR).

Fields
string vendorVendor identifier for this provider implementation. Combined with format to form the unique provider name.
int priority
ReaderFactoryFn makeReader
WriterFactoryFn makeWriter
FormatDetectorFn detectFormat
Methods
string fullName() constReturns the full provider name in the format "vendor-format".

RAII wrapper for automatic ArchiveReader cleanup.

Automatically closes the reader when the wrapper goes out of scope. Supports transparent usage via alias this.

Example:

auto reader = scopedOpenArchive("archive.zip");
foreach (entry; reader.entries()) {
   writeln(entry.path);
}
// reader.close() called automatically

Fields
private ArchiveReader _reader
Methods
ArchiveReader get()Get the underlying reader.
Constructors
Destructors

RAII wrapper for automatic ArchiveWriter cleanup.

Automatically calls finish() (if not already called) and close() when the wrapper goes out of scope.

Example:

{
   auto writer = scopedCreateArchive("output.zip", WriteOptions(ArchiveFormat.ZIP));
   writer.addFile("hello.txt", cast(ubyte[])"Hello, World!");
}
// writer.finish() and writer.close() called automatically

Fields
private ArchiveWriter _writer
Methods
ArchiveWriter get()Get the underlying writer.
Constructors
Destructors

Information about a compressed archive (e.g., .tar.gz).

Describes a compound format where an inner archive is wrapped in an outer compression layer.

Fields
CompressionFormat outerCompression
ArchiveFormat innerArchive
bool isCompressedArchive
private classArchiveBuffer
Fields
ubyte[] data
Fields
private ArchiveWriter _inner
private File _file
private CompressionFormat _compressionFormat
private ArchiveBuffer _buffer
private bool _finished
private bool _closed
Methods
ArchiveFormat format() @property const
void addDirectory(string path, FilePermissions perms = FilePermissions.fromOctal(octal!"755"))
void addFile(string path, const(ubyte)[] content, EntryWriteOptions opts = EntryWriteOptions.init)
void addFile(string path, scope const(ubyte)[] delegate() source, ulong size = 0, EntryWriteOptions opts = EntryWriteOptions.init)
void addFileFrom(string archivePath, string filesystemPath, EntryWriteOptions opts = EntryWriteOptions.init)
void addSymlink(string path, string target)
void addHardlink(string path, string target)
void addEntry(ref const EntryInfo info, scope const(ubyte)[] delegate() source)
void removeEntry(string path)
void updateEntry(string path, const(ubyte)[] content, EntryWriteOptions opts = EntryWriteOptions.init)
void setComment(string comment)
void finish()
void close()
bool isFinished() @property const
bool isClosed() @property const
ulong bytesWritten() @property const
size_t entriesWritten() @property const
Constructors
this(ArchiveWriter inner, File file, CompressionFormat compressionFormat, ArchiveBuffer buffer)

Sink-based compressed archive writer implementation.

Buffers all archive output in memory, then compresses and writes to the configured ContentSink when finish() is called.

Fields
private ArchiveWriter _inner
private ContentSink _sink
private CompressionFormat _compressionFormat
private ArchiveBuffer _buffer
private bool _finished
private bool _closed
Methods
ArchiveFormat format() @property const
void addDirectory(string path, FilePermissions perms = FilePermissions.fromOctal(octal!"755"))
void addFile(string path, const(ubyte)[] content, EntryWriteOptions opts = EntryWriteOptions.init)
void addFile(string path, scope const(ubyte)[] delegate() source, ulong size = 0, EntryWriteOptions opts = EntryWriteOptions.init)
void addFileFrom(string archivePath, string filesystemPath, EntryWriteOptions opts = EntryWriteOptions.init)
void addSymlink(string path, string target)
void addHardlink(string path, string target)
void addEntry(ref const EntryInfo info, scope const(ubyte)[] delegate() source)
void removeEntry(string path)
void updateEntry(string path, const(ubyte)[] content, EntryWriteOptions opts = EntryWriteOptions.init)
void setComment(string comment)
void finish()
void close()
bool isFinished() @property const
bool isClosed() @property const
ulong bytesWritten() @property const
size_t entriesWritten() @property const
Constructors
this(ArchiveWriter inner, ContentSink sink, CompressionFormat compressionFormat, ArchiveBuffer buffer)

Functions 25

fnauto archiveEntries(ArchiveReader reader)Returns an input range for iterating over all entries in the archive.
fnubyte[] readContentBuffer(ArchiveReader reader, ref const EntryInfo entry)Read entry content into a newly allocated buffer.
fnvoid registerProvider(ArchiveProvider provider)Register an archive provider.
fnArchiveProvider[] providersFor(ArchiveFormat fmt)Get all registered providers for a format.
fnArchiveProvider selectProvider(ArchiveFormat fmt, string preferredVendor = null)Get the best provider for a format.
fnArchiveFormat detectFormat(const(ubyte)[] headerBytes)Detect archive format from header bytes.
fnArchiveReader openArchive(string path, ReadOptions opts = ReadOptions.init)Create a reader for an archive file.
fnArchiveReader openArchive(const(ubyte)[] data, ReadOptions opts = ReadOptions.init)Create a reader from a byte slice.
fnArchiveReader openArchive(Range)(Range source, ReadOptions opts = ReadOptions.init) if (isInputRange!Range && is(ElementType!Range : const(ubyte)[]))Create a reader from an input range.
fnArchiveWriter createArchive(string path, WriteOptions opts = WriteOptions.init)Create a writer for a new archive file.
fnArchiveWriter createArchive(ContentSink sink, WriteOptions opts = WriteOptions.init)Create a writer with an output sink.
fnScopedArchiveReader scopedOpenArchive(string path, ReadOptions opts = ReadOptions.init)Open archive with automatic cleanup.
fnScopedArchiveReader scopedOpenArchive(const(ubyte)[] data, ReadOptions opts = ReadOptions.init)Open archive from data with automatic cleanup.
fnScopedArchiveWriter scopedCreateArchive(string path, WriteOptions opts)Create archive with automatic cleanup.
fnScopedArchiveWriter scopedCreateArchive(ContentSink sink, WriteOptions opts)Create archive with sink and automatic cleanup.
fnCompressedArchiveInfo detectCompressedArchive(const(ubyte)[] header)Detect if data is a compressed archive and return info.
fnArchiveReader openCompressedArchive(string path, ReadOptions opts = ReadOptions.init)Open a potentially compressed archive (handles .tar.gz, .tar.bz2, etc.).
fnArchiveWriter createCompressedArchive(string path, ArchiveFormat archiveFormat, CompressionFormat compressionFormat, WriteOptions opts = WriteOptions.init)Create a compressed archive (e.g., .tar.gz).
fnArchiveReader openCompressedArchive(const(ubyte)[] data, ReadOptions opts = ReadOptions.init)Open a possibly compressed archive from an in-memory byte slice.
fnArchiveWriter createCompressedArchive(ContentSink sink, ArchiveFormat archiveFormat, CompressionFormat compressionFormat, WriteOptions opts = WriteOptions.init)Create a compressed archive writing to a ContentSink.
fnstring normalizePathSeparators(string path) @safe pureNormalize path separators to forward slash.
fnbool isReservedName(string name) @safe pureCheck if a filename is a Windows reserved device name.
fnbool isPathSafe(string path, bool allowTraversal = false) @safe pureCheck if a path is safe for archive extraction.
private fnstring[] pathSplit(string path) @safe pure
fnstring sanitizePath(string path, bool allowTraversal = false)Sanitize a path for archive use.

Variables 1

private varArchiveProvider[][ArchiveFormat] gProviders