tarballer/README.md

Tarballer

A simple utility to create tarballs with a specific directory structure.

Features

• Creates compressed tar archives (.tar.gz) from a source directory
• Places all files under a specified prefix directory in the tarball
• Preserves file permissions and directory structure
• Handles symbolic links correctly
• Cross-platform compatibility (FreeBSD, macOS, Linux)
• Produces tarballs compatible with standard tar tools
• Verified data integrity with MD5 hash comparison
• Automatically generates and verifies MD5 hashes of all files during extraction

Building

This project includes Docker support to build binaries for different platforms:

docker compose up --build build

This will create these binaries in the ./bin directory:

• tarballer-freebsd: FreeBSD AMD64 compatible binary
• tarballer-darwin: macOS ARM64 compatible binary
• tarballer-linux: Linux AMD64 compatible binary

Testing

You can run the included tests to verify functionality:

# Run the basic functionality test
docker compose up --build test

# Run comparison test with standard tar tools
docker compose up --build tar-test

The tests will:

1. Create test directory structures with nested directories and symlinks
2. Create tarballs from them
3. Extract the tarballs using standard tools
4. Verify the contents and file structure, including symlinks
5. Check that symlinks remain functional after extraction
6. Compare output with standard tar tools to ensure compatibility
7. Verify data integrity with MD5 hashing (original vs. extracted files)

Usage

The usage is the same for all binaries:

# Create a tarball
./bin/tarballer-<platform> -source /path/to/directory -output myarchive.tar.gz -prefix myprefix

# Extract a tarball with integrity verification
./bin/tarballer-<platform> -extract -output myarchive.tar.gz -extractdir /path/to/extract

Create Mode Options

• -source: The directory you want to compress (required)
• -output: The name of the output tarball (defaults to "output.tar.gz")
• -prefix: The directory name that will contain all files in the tarball (defaults to "myapp")

Extract Mode Options

• -extract: Enables extraction mode
• -output: The tarball to extract (required)
• -extractdir: Directory to extract to (defaults to current directory)
• -verify: Only verify hash integrity without extraction
• -keepmanifest: Keep the MD5 manifest file after extraction (defaults to removing it)

Examples

# Create a tarball (on macOS):
./bin/tarballer-darwin -source ./myproject -output release.tar.gz -prefix app

# Extract and verify (on Linux):
./bin/tarballer-linux -extract -output release.tar.gz -extractdir /path/to/extract

# Only verify hash integrity without extraction:
./bin/tarballer-linux -extract -verify -output release.tar.gz

MD5 Hash Verification

Tarballer includes built-in file integrity protection:

1. When creating a tarball, MD5 hashes are calculated for all files and stored in a hidden manifest file (.md5-manifest.txt) at the root of the extraction directory
2. During extraction, hashes are verified to ensure files haven't been corrupted or tampered with
3. The manifest file is automatically removed after extraction unless -keepmanifest is specified
4. If any file fails verification, the extraction is aborted with an error

This provides an extra layer of security and data integrity validation compared to standard tar tools.