Tarballer

A utility to create tarballs with a specific directory structure and built-in MD5 integrity verification.

Features

• Creates compressed tar archives (.tar.gz) with files under a specified prefix directory
• Preserves file permissions and directory structure
• Handles symbolic links correctly
• Cross-platform compatibility (FreeBSD, macOS, Linux)
• Compatible with standard tar tools
• Built-in MD5 hash verification
• Automatic file integrity checks during extraction
• Pattern-based file exclusion for creating targeted archives

Building

docker compose up --build

This builds the project and creates binaries in the ./bin directory:

• tarballer-freebsd - FreeBSD AMD64
• tarballer-darwin - macOS ARM64
• tarballer-linux - Linux AMD64

Testing

# Run all tests
docker compose up --build

# Run specific test type
docker compose run tarballer /bin/test.sh basic
docker compose run tarballer /bin/test.sh tar

All tests run in the container's /tmp directory with no files written to the host filesystem.

Tests verify:

1. Creating and extracting tarballs with nested directories and symlinks
2. File content preservation and structure integrity
3. Symlink functionality after extraction
4. Compatibility with standard tar tools
5. MD5 hash verification

See test/README.md for detailed test information.

Usage

Create Mode

./bin/tarballer-<platform> -source /path/to/directory -output myarchive.tar.gz -prefix myprefix

Options:

• -source: Source directory to compress (required)
• -output: Output tarball filename (default: "output.tar.gz")
• -prefix: Directory name that will contain all files in the tarball (default: "myapp")
• -exclude: Comma-separated list of patterns to exclude (e.g. ".log,.tmp,temp/")
• -verbose: Enable detailed output during operation

Extract Mode

./bin/tarballer-<platform> -extract -output myarchive.tar.gz -extractdir /path/to/extract

Options:

• -extract: Enables extraction mode
• -output: Tarball to extract (required)
• -extractdir: Extraction directory (default: current directory)
• -verify: Only verify hash integrity without extraction
• -verbose: Enable detailed output during operation

Examples

# Create a tarball (macOS)
./bin/tarballer-darwin -source ./myproject -output release.tar.gz -prefix app

# Create a tarball excluding specific files
./bin/tarballer-darwin -source ./myproject -output release.tar.gz -prefix app -exclude "*.log,bin/,temp/"

# Extract and verify (Linux)
./bin/tarballer-linux -extract -output release.tar.gz -extractdir /path/to/extract

# Extract with verbose output
./bin/tarballer-linux -extract -output release.tar.gz -extractdir /path/to/extract -verbose

# Only verify integrity
./bin/tarballer-linux -extract -verify -output release.tar.gz

MD5 Hash Verification

1. During creation, MD5 hashes are calculated for all files and stored in .md5-manifest.txt
2. During extraction, file hashes are verified against the manifest
3. The manifest file is removed after successful extraction
4. Extraction aborts with an error if verification fails

Exclude Patterns

The -exclude flag accepts a comma-separated list of patterns to exclude from the tarball:

• Simple wildcards using * (matches any sequence of characters) and ? (matches any single character)
• Directory patterns (ending with /) exclude entire directory trees
• File patterns can match by extension (e.g., *.log) or name

Examples:

• *.log - Excludes all files with the .log extension
• bin/ - Excludes the bin directory and all its contents
• temp/,*.tmp - Excludes the temp directory and all .tmp files
• cache/*,*.bak - Excludes all contents of the cache directory and all .bak files