tarballer/README.md

Tarballer

A simple utility to create tarballs with a specific directory structure.

Features

• Creates compressed tar archives (.tar.gz) from a source directory
• Places all files under a specified prefix directory in the tarball
• Preserves file permissions and directory structure
• Handles symbolic links correctly
• Cross-platform compatibility (FreeBSD, macOS, Linux)
• Produces tarballs compatible with standard tar tools
• Verified data integrity with MD5 hash comparison
• Automatically generates and verifies MD5 hashes of all files during extraction

Building

This project includes Docker support to build binaries for different platforms:

docker compose up --build

This will create these binaries in the ./bin directory:

• tarballer-freebsd: FreeBSD AMD64 compatible binary
• tarballer-darwin: macOS ARM64 compatible binary
• tarballer-linux: Linux AMD64 compatible binary

Testing

You can run the included tests to verify functionality:

# Run all tests
docker compose up --build

# Run specific test types
docker compose run tarballer /bin/test.sh basic
docker compose run tarballer /bin/test.sh tar

All tests run inside the container using its /tmp directory, ensuring no temporary files are written to the host filesystem.

The tests verify:

1. Creating test directory structures with nested directories and symlinks
2. Creating tarballs from test directories
3. Extracting the tarballs
4. Verifying the contents and file structure, including symlinks
5. Checking that symlinks remain functional after extraction
6. Comparing output with standard tar tools to ensure compatibility
7. Verifying data integrity with MD5 hashing (original vs. extracted files)

See the test/README.md for more details on the test process.

Usage

The usage is the same for all binaries:

# Create a tarball
./bin/tarballer-<platform> -source /path/to/directory -output myarchive.tar.gz -prefix myprefix

# Extract a tarball with integrity verification
./bin/tarballer-<platform> -extract -output myarchive.tar.gz -extractdir /path/to/extract

Create Mode Options

• -source: The directory you want to compress (required)
• -output: The name of the output tarball (defaults to "output.tar.gz")
• -prefix: The directory name that will contain all files in the tarball (defaults to "myapp")

Extract Mode Options

• -extract: Enables extraction mode
• -output: The tarball to extract (required)
• -extractdir: Directory to extract to (defaults to current directory)
• -verify: Only verify hash integrity without extraction

Examples

# Create a tarball (on macOS):
./bin/tarballer-darwin -source ./myproject -output release.tar.gz -prefix app

# Extract and verify (on Linux):
./bin/tarballer-linux -extract -output release.tar.gz -extractdir /path/to/extract

# Only verify hash integrity without extraction:
./bin/tarballer-linux -extract -verify -output release.tar.gz

MD5 Hash Verification

Tarballer includes built-in file integrity protection:

1. When creating a tarball, MD5 hashes are calculated for all files and stored in a hidden manifest file (.md5-manifest.txt) at the root of the extraction directory
2. During extraction, hashes are verified to ensure files haven't been corrupted or tampered with
3. The manifest file is automatically removed after extraction
4. If any file fails verification, the extraction is aborted with an error

This provides an extra layer of security and data integrity validation compared to standard tar tools.