ddn.api.archive
ddn.api.archive
A robust, modern, idiomatic D archive API for reading and writing archive formats such as TAR, ZIP, 7ZIP, RAR, CPIO, AR, and similar.
Design goals (inspired by popular APIs and following patterns from ddn.api.compressor):
- Format-agnostic core interfaces (TAR, ZIP, 7ZIP, RAR, CPIO, AR, etc.).
- Provider-based extensibility via a registration system.
- Streaming push model with output sinks (delegates) to avoid forcing a specific IO abstraction.
- Clear error model with portable error codes.
- Progress callback support.
- Cross-platform support (POSIX and Windows).
- Compression integration with ddn.api.compressor for compound formats (.tar.gz, .tar.bz2, etc.).
- Graceful degradation: unsupported operations throw UNSUPPORTED_FEATURE, not compile errors.
This module defines the API only. Concrete implementations (e.g., TAR, ZIP) should live in their own packages/modules and register themselves via the provider registration functions herein.
Module Initializers 1
()Types 40
Supported archive formats.
TAR sub-format variants.
CPIO sub-format variants.
Entry type within an archive.
Platform-specific extraction behavior for unsupported entry types.
Archive error codes.
Encryption method for encrypted archives.
Archive-related exception with a portable error code.
ArchiveErrorCode codethis(string msg, ArchiveErrorCode code = ArchiveErrorCode.INTERNAL_ERROR,
string file = __FILE__, size_t line = __LINE__, Throwable next = null)Construct an ArchiveError.Capabilities declared by an archive provider.
These flags describe what operations and features a provider supports, allowing callers to make informed decisions before invoking optional features.
bool canReadbool canWritebool canAppendbool canUpdatebool randomAccessbool streamingbool encryptionbool compressionbool preservesOwnershipbool preservesPermissionsbool preservesTimesbool supportsLargeFilesbool supportsUnicodebool canStoreDirectoriesbool canStoreSymlinksbool canStoreHardlinksbool canStoreSpecialFilesbool canStoreCommentsFile permissions (POSIX-style, portable representation).
Provides a platform-independent way to represent the 12 POSIX permission bits (rwxrwxrwx + setuid/setgid/sticky) without relying on platform-specific types.
bool ownerReadbool ownerWritebool ownerExecutebool groupReadbool groupWritebool groupExecutebool otherReadbool otherWritebool otherExecutebool setuidbool setgidbool stickyFilePermissions defaultFile() static @safe pure nothrow @nogcCreate default permissions for a regular file (0o644).FilePermissions defaultDirectory() static @safe pure nothrow @nogcCreate default permissions for a directory (0o755).FilePermissions defaultExecutable() static @safe pure nothrow @nogcCreate default permissions for an executable (0o755).Windows-specific file attributes (stored in archives that support them).
Maps to the Windows FILE_ATTRIBUTE_* flags for round-trip preservation of Windows metadata in archives that support extended attributes.
bool readonlybool hiddenbool systembool archivebool directorybool encryptedbool compressedbool sparsebool reparsePointbool offlineWindowsAttributes fromDword(uint attrs) static @safe pure nothrow @nogcCreate from Windows DWORD attribute flags.Timestamp with optional nanosecond precision.
Stores timestamps as a Unix epoch offset (seconds + nanoseconds) to enable round-trip preservation of high-resolution timestamps across archive formats.
long unixSecondsuint nanosecondsArchiveTime fromSysTime(T)(T st) staticCreate from a SysTime-like value. The template parameter T must provide toUnixTime() and fracSecs members (e.g., std.datetime.SysTime).Metadata for an archive entry.
Contains all available information about a single item in an archive, including its type, size, timestamps, ownership, permissions, and format-specific data.
string pathEntryType typeulong sizeulong compressedSizeArchiveTime mtimeArchiveTime atimeArchiveTime ctimeuint uiduint gidstring ownerstring groupFilePermissions permissionsWindowsAttributes windowsAttrsbool hasWindowsAttrsstring linkTargetCompressionFormat compressionEncryptionMethod encryptionbool isEncrypteduint crc32string commentubyte[] extraOutput sink for entry content. Called repeatedly with data chunks.
Password callback for interactive password prompting. Called when an encrypted entry is encountered and no password was provided in options. Return the password, or null to skip/abort.
Progress information for archive operations.
string currentEntryulong entriesProcessedulong entriesTotalulong bytesProcessedulong bytesTotalfloat ratioProgress callback. Return false to cancel operation.
Options for reading archives.
ArchiveFormat formatstring passwordPasswordCallback passwordCallbackbool verifyChecksumsbool allowPathTraversalsize_t maxEntrySizesize_t maxEntriessize_t maxArchiveSizebool metadataOnlyUnsupportedEntryBehavior unsupportedBehaviorbool convertPathSeparatorsbool detectReservedNamesbool caseSensitiveOptions for writing archives.
ArchiveFormat formatCompressionFormat compressionCompressionLevel levelstring passwordEncryptionMethod encryptionbool preserveOwnershipbool preservePermissionsbool preserveTimesbool preserveWindowsAttrsstring commentbool normalizePathSeparatorsbool storeWindowsAttrsTarFormat tarFormatCpioFormat cpioFormatulong volumeSizestring volumeNamingOptions for a single entry being written.
Interface for reading archives.
Implementations are provided by archive format providers registered via registerProvider(). Use openArchive() to obtain a reader instance.
Methods that may throw ArchiveError with UNSUPPORTED_FEATURE:
- getEntry() if !capabilities.randomAccess
- extractTo() / extractAll() for format-specific limitations
void setProgressCallback(ArchiveProgressCallback callback)Set progress callback. Return false from callback to cancel.size_t entryCount() @property const;Get total number of entries (if known). Returns 0 if count is unknown (streaming archives).final auto entries()Iterate over all entries. Returns an input range of EntryInfo.bool contains(string path)Check if archive contains an entry at path. May be O(n) for sequential (non-random-access) formats.void readContent(const EntryInfo entry, ContentSink sink)Read entry content, calling sink with chunks of decompressed data.ubyte[] readContent(const EntryInfo entry)Read entry content into a buffer. Returns the content as a newly allocated array.void extractAll(string destDir)Extract all entries to a directory.void close()Close the reader and release resources.Input range for iterating over archive entries.
Primed on construction (the first entry is loaded immediately). Use with foreach or range algorithms.
Example:
auto reader = openArchive("archive.zip");
foreach (entry; reader.entries()) {
writeln(entry.path);
}
reader.close();Input range over an archive's entries; .init is a valid empty range.
void popFront()Advance to the next entry, exhausting the range at the archive's end.this(ArchiveReader reader)Construct a range primed with the reader's first entry.Interface for writing archives.
Implementations are provided by archive format providers registered via registerProvider(). Use createArchive() to obtain a writer instance.
Call finish() to finalize the archive and produce valid output. Call close() to release all resources.
Methods that may throw ArchiveError with UNSUPPORTED_FEATURE:
- addDirectory() if !capabilities.canStoreDirectories
- addSymlink() if !capabilities.canStoreSymlinks
- addHardlink() if !capabilities.canStoreHardlinks
- setComment() if !capabilities.canStoreComments
- removeEntry() if !capabilities.canUpdate
- updateEntry() if !capabilities.canUpdate
void setOutputSink(ContentSink sink)Set the output sink for archive bytes.void setProgressCallback(ArchiveProgressCallback callback)Set progress callback. Return false from callback to cancel.void addDirectory(string path, FilePermissions perms = FilePermissions.fromOctal(octal!"755"))Add a directory entry.void addFile(string path, const(ubyte)[] content, EntryWriteOptions opts = EntryWriteOptions.init)Add a file entry with content from a byte slice.void addFile(string path, scope const(ubyte)[] delegate() source,
ulong size = 0, EntryWriteOptions opts = EntryWriteOptions.init)Add a file entry with content from a delegate.void addFileFrom(string archivePath, string filesystemPath,
EntryWriteOptions opts = EntryWriteOptions.init)Add a file entry from the filesystem.void addSymlink(string path, string target)Add a symbolic link entry.void addHardlink(string path, string target)Add a hard link entry.void addEntry(ref const EntryInfo info, scope const(ubyte)[] delegate() source)Add an entry with full metadata control.void removeEntry(string path)Remove an entry from the archive (if supported).void updateEntry(string path, const(ubyte)[] content,
EntryWriteOptions opts = EntryWriteOptions.init)Update/replace an existing entry.void setComment(string comment)Set archive-level comment.void finish()Finalize the archive. Must be called to produce valid output.void close()Close the writer and release resources.Pull-based source of raw archive bytes.
Streaming readers consume archive data through this interface instead of a fully-buffered array, so archives larger than memory can be read by providers that implement ArchiveProvider.makeStreamingReader.
Source replaying a buffered prefix followed by the remainder of another source.
this(const(ubyte)[] prefix, ArchiveByteSource rest)Factory function type for creating archive readers.
Factory function type for creating streaming readers from a byte source.
Factory function type for creating archive writers.
Factory function type for detecting archive format from header bytes.
Provider descriptor for archive implementations.
Each provider is uniquely identified by the combination of vendor and format. The full provider name is constructed as vendor-format (e.g., "ddn-zip").
Register providers via registerProvider(). Factory functions may be null for formats that only support reading or only support writing (e.g., RAR).
ArchiveFormat formatstring vendorVendor identifier for this provider implementation. Combined with format to form the unique provider name.int priorityArchiveCapabilities capsReaderFactoryFn makeReaderStreamingReaderFactoryFn makeStreamingReaderWriterFactoryFn makeWriterFormatDetectorFn detectFormatInternal wrapper that owns a File and closes it when the writer is closed. This ensures proper cleanup when using createArchive(string path, ...).
void setOutputSink(ContentSink sink)void setProgressCallback(ArchiveProgressCallback callback)void addDirectory(string path, FilePermissions perms = FilePermissions.fromOctal(octal!"755"))void addFile(string path, const(ubyte)[] content,
EntryWriteOptions opts = EntryWriteOptions.init)void addFile(string path, scope const(ubyte)[] delegate() source,
ulong size = 0, EntryWriteOptions opts = EntryWriteOptions.init)void addFileFrom(string archivePath, string filesystemPath,
EntryWriteOptions opts = EntryWriteOptions.init)void addSymlink(string path, string target)void addHardlink(string path, string target)void removeEntry(string path)void updateEntry(string path, const(ubyte)[] content,
EntryWriteOptions opts = EntryWriteOptions.init)void setComment(string comment)void finish()void close()this(File file, WriteOptions opts)RAII wrapper for automatic ArchiveReader cleanup.
Automatically closes the reader when the wrapper goes out of scope. Supports transparent usage via alias this.
Example:
auto reader = scopedOpenArchive("archive.zip");
foreach (entry; reader.entries()) {
writeln(entry.path);
}
// reader.close() called automaticallyprivate ArchiveReader _readerArchiveReader get()Get the underlying reader.RAII wrapper for automatic ArchiveWriter cleanup.
Automatically calls finish() (if not already called) and close() when the wrapper goes out of scope.
Example:
{
auto writer = scopedCreateArchive("output.zip", WriteOptions(ArchiveFormat.ZIP));
writer.addFile("hello.txt", cast(ubyte[])"Hello, World!");
}
// writer.finish() and writer.close() called automaticallyprivate ArchiveWriter _writerArchiveWriter get()Get the underlying writer.Information about a compressed archive (e.g., .tar.gz).
Describes a compound format where an inner archive is wrapped in an outer compression layer.
Accumulates inner-archive bytes until the compressed writer finishes.
ubyte[] dataBytes accumulated from the inner writer so far.Writer wrapping an inner archive into a compressed file: the inner bytes are buffered, compressed on finish, and written to the owned file.
private ArchiveWriter _innerprivate File _fileprivate CompressionFormat _compressionFormatprivate ArchiveBuffer _bufferprivate bool _finishedprivate bool _closedvoid setOutputSink(ContentSink sink)void setProgressCallback(ArchiveProgressCallback callback)void addDirectory(string path, FilePermissions perms = FilePermissions.fromOctal(octal!"755"))void addFile(string path, const(ubyte)[] content,
EntryWriteOptions opts = EntryWriteOptions.init)void addFile(string path, scope const(ubyte)[] delegate() source,
ulong size = 0, EntryWriteOptions opts = EntryWriteOptions.init)void addFileFrom(string archivePath, string filesystemPath,
EntryWriteOptions opts = EntryWriteOptions.init)void addSymlink(string path, string target)void addHardlink(string path, string target)void removeEntry(string path)void updateEntry(string path, const(ubyte)[] content,
EntryWriteOptions opts = EntryWriteOptions.init)void setComment(string comment)void finish()void close()this(ArchiveWriter inner, File file, CompressionFormat compressionFormat,
ArchiveBuffer buffer)Sink-based compressed archive writer implementation.
Buffers all archive output in memory, then compresses and writes to the configured ContentSink when finish() is called. Writer wrapping an inner archive into a compressed stream: the inner bytes are buffered, compressed on finish, and delivered to a sink.
private ArchiveWriter _innerprivate ContentSink _sinkprivate CompressionFormat _compressionFormatprivate ArchiveBuffer _bufferprivate bool _finishedprivate bool _closedvoid setOutputSink(ContentSink sink)void setProgressCallback(ArchiveProgressCallback callback)void addDirectory(string path, FilePermissions perms = FilePermissions.fromOctal(octal!"755"))void addFile(string path, const(ubyte)[] content,
EntryWriteOptions opts = EntryWriteOptions.init)void addFile(string path, scope const(ubyte)[] delegate() source,
ulong size = 0, EntryWriteOptions opts = EntryWriteOptions.init)void addFileFrom(string archivePath, string filesystemPath,
EntryWriteOptions opts = EntryWriteOptions.init)void addSymlink(string path, string target)void addHardlink(string path, string target)void removeEntry(string path)void updateEntry(string path, const(ubyte)[] content,
EntryWriteOptions opts = EntryWriteOptions.init)void setComment(string comment)void finish()void close()this(ArchiveWriter inner, ContentSink sink, CompressionFormat compressionFormat,
ArchiveBuffer buffer)A mock reader serving a fixed entry list and byte contents.
private EntryInfo[] _entriesprivate const(ubyte)[][] _contentsprivate size_t _indexprivate bool _closedvoid setProgressCallback(ArchiveProgressCallback callback)bool contains(string path)void readContent(const EntryInfo entry, ContentSink sink)void extractAll(string destDir)void close()A mock writer emitting a fixed payload on finish.
void setOutputSink(ContentSink sink)void setProgressCallback(ArchiveProgressCallback callback)void addDirectory(string path, FilePermissions perms = FilePermissions.fromOctal(octal!"755"))void addFile(string path, const(ubyte)[] content, EntryWriteOptions opts = EntryWriteOptions.init)void addFile(string path, scope const(ubyte)[] delegate() source,
ulong size = 0, EntryWriteOptions opts = EntryWriteOptions.init)void addFileFrom(string archivePath, string filesystemPath,
EntryWriteOptions opts = EntryWriteOptions.init)void addSymlink(string path, string target)void addHardlink(string path, string target)void removeEntry(string path)void updateEntry(string path, const(ubyte)[] content,
EntryWriteOptions opts = EntryWriteOptions.init)void setComment(string comment)void finish()void close()this(ContentSink sink, WriteOptions opts)Functions 28
auto archiveEntries(ArchiveReader reader)Returns an input range for iterating over all entries in the archive.ubyte[] readContentBuffer(ArchiveReader reader, const EntryInfo entry)Read entry content into a newly allocated buffer.ArchiveProvider selectProvider(ArchiveFormat fmt, string preferredVendor = null)Get the best provider for a format.ArchiveReader openArchive(string path, ReadOptions opts = ReadOptions.init)Create a reader for an archive file.ArchiveReader openArchive(const(ubyte)[] data, ReadOptions opts = ReadOptions.init)Create a reader from a byte slice.ArchiveReader openArchive(ArchiveByteSource source, ReadOptions opts = ReadOptions.init)Create a reader from a streaming byte source.ArchiveReader openArchive(Range)(Range source, ReadOptions opts = ReadOptions.init) if (isInputRange!Range && is(ElementType!Range : const(ubyte)[]))Create a reader from an input range.ArchiveWriter createArchive(string path, WriteOptions opts = WriteOptions.init)Create a writer for a new archive file.ArchiveWriter createArchive(ContentSink sink, WriteOptions opts = WriteOptions.init)Create a writer with an output sink.ScopedArchiveReader scopedOpenArchive(string path, ReadOptions opts = ReadOptions.init)Open archive with automatic cleanup.ScopedArchiveReader scopedOpenArchive(const(ubyte)[] data, ReadOptions opts = ReadOptions.init)Open archive from data with automatic cleanup.ScopedArchiveWriter scopedCreateArchive(string path, WriteOptions opts)Create archive with automatic cleanup.ScopedArchiveWriter scopedCreateArchive(ContentSink sink, WriteOptions opts)Create archive with sink and automatic cleanup.CompressedArchiveInfo detectCompressedArchive(const(ubyte)[] header)Detect if data is a compressed archive and return info.ArchiveReader openCompressedArchive(string path, ReadOptions opts = ReadOptions.init)Open a potentially compressed archive (handles .tar.gz, .tar.bz2, etc.).ArchiveWriter createCompressedArchive(string path, ArchiveFormat archiveFormat,
CompressionFormat compressionFormat, WriteOptions opts = WriteOptions.init)Create a compressed archive (e.g., .tar.gz).ArchiveReader openCompressedArchive(const(ubyte)[] data, ReadOptions opts = ReadOptions.init)Open a possibly compressed archive from an in-memory byte slice.ArchiveWriter createCompressedArchive(ContentSink sink, ArchiveFormat archiveFormat,
CompressionFormat compressionFormat, WriteOptions opts = WriteOptions.init)Create a compressed archive writing to a ContentSink.bool isPathSafe(string path, bool allowTraversal = false) @safe pureCheck if a path is safe for archive extraction.string[] pathSplit(string path) @safe pureSplit a normalized path into its `/`-separated components.Variables 3
Object gProvidersLockGuards every access to gProviders so registration and lookups are race-free.
STREAM_DETECT_PEEK = 32774Bytes buffered from a streaming source for format detection; covers the largest built-in magic (ISO9660 "CD001" at offset 32769).