Use container's /tmp directory for test files instead of mounted volume

This commit is contained in:
Leopere 2025-03-20 19:14:22 -04:00
parent 267131960a
commit 195142f080
4 changed files with 105 additions and 101 deletions

View File

@ -18,7 +18,7 @@ A simple utility to create tarballs with a specific directory structure.
This project includes Docker support to build binaries for different platforms:
```bash
docker compose up --build build
docker compose up --build
```
This will create these binaries in the `./bin` directory:
@ -31,21 +31,26 @@ This will create these binaries in the `./bin` directory:
You can run the included tests to verify functionality:
```bash
# Run the basic functionality test
docker compose up --build test
# Run all tests
docker compose up --build
# Run comparison test with standard tar tools
docker compose up --build tar-test
# Run specific test types
docker compose run tarballer /bin/test.sh basic
docker compose run tarballer /bin/test.sh tar
```
The tests will:
1. Create test directory structures with nested directories and symlinks
2. Create tarballs from them
3. Extract the tarballs using standard tools
4. Verify the contents and file structure, including symlinks
5. Check that symlinks remain functional after extraction
6. Compare output with standard tar tools to ensure compatibility
7. Verify data integrity with MD5 hashing (original vs. extracted files)
All tests run inside the container using its `/tmp` directory, ensuring no temporary files are written to the host filesystem.
The tests verify:
1. Creating test directory structures with nested directories and symlinks
2. Creating tarballs from test directories
3. Extracting the tarballs
4. Verifying the contents and file structure, including symlinks
5. Checking that symlinks remain functional after extraction
6. Comparing output with standard tar tools to ensure compatibility
7. Verifying data integrity with MD5 hashing (original vs. extracted files)
See the `test/README.md` for more details on the test process.
## Usage

View File

@ -5,7 +5,6 @@ services:
volumes:
- ./bin:/output
- ./test:/test
- ./test-tmp:/test-tmp
- .:/workdir
working_dir: /workdir
command: sh -c "cp /app/tarballer-freebsd /output/ && cp /app/tarballer-darwin /output/ && cp /app/tarballer-linux /output/ && chmod +x /output/tarballer-* && mkdir -p /test-tmp && /bin/test.sh all"
command: sh -c "cp /app/tarballer-freebsd /output/ && cp /app/tarballer-darwin /output/ && cp /app/tarballer-linux /output/ && chmod +x /output/tarballer-* && /bin/test.sh all"

View File

@ -4,11 +4,11 @@ This directory contains test scripts for validating the functionality of the Tar
## Test Environment
All tests run in a Docker container with temporary files stored in the `/test-tmp` directory. This directory is:
All tests run in a Docker container with temporary files stored in the container's `/tmp` directory. This directory is:
1. Git-ignored to prevent cluttering the repository
2. Automatically created by the Docker container
3. Cleaned up after tests complete (unless you specify otherwise)
1. Automatically created and managed by the container
2. Cleaned up after tests complete (unless you specify otherwise)
3. Internal to the container, so no files are written to the host filesystem
## Running Tests
@ -39,13 +39,13 @@ The test script (`test.sh`) supports several test types:
## Test Directory Structure
The tests create the following structure in the `/test-tmp` directory:
The tests create the following structure in the container's `/tmp` directory:
- `/test-tmp/complex/` - Directory with complex nested structure for basic test
- `/test-tmp/complex-extracted/` - Where the complex test tarball is extracted
- `/test-tmp/standard-test/` - Directory with sample app structure for tar comparison
- `/test-tmp/standard-extracted/` - Where our utility's tarball is extracted
- `/test-tmp/reference-extracted/` - Where standard tar's tarball is extracted
- `/tmp/complex/` - Directory with complex nested structure for basic test
- `/tmp/complex-extracted/` - Where the complex test tarball is extracted
- `/tmp/standard-test/` - Directory with sample app structure for tar comparison
- `/tmp/standard-extracted/` - Where our utility's tarball is extracted
- `/tmp/reference-extracted/` - Where standard tar's tarball is extracted
## Manual Testing
@ -63,4 +63,4 @@ If you need to manually test the tarballer utility, you can:
## Modifying Tests
When modifying tests, ensure any temporary files are created within the `/test-tmp` directory structure to keep the repository clean.
When modifying tests, keep in mind that the test script uses the container's `/tmp` directory for all temporary files. This keeps the test process self-contained within the container.

View File

@ -9,10 +9,10 @@ KEEP_TEMP_FILES=${2:-0}
cleanup_files() {
if [ "$KEEP_TEMP_FILES" -eq 0 ]; then
echo "=== CLEANING UP TEMPORARY FILES ==="
rm -rf /test-tmp/complex /test-tmp/complex-extracted
rm -f /test-tmp/complex-original-md5.txt /test-tmp/complex-extracted-md5.txt
rm -rf /test-tmp/standard-test /test-tmp/standard-extracted /test-tmp/reference-extracted
rm -f /test-tmp/original-checksums.txt /test-tmp/standard-checksums.txt /test-tmp/reference-checksums.txt
rm -rf /tmp/complex /tmp/complex-extracted
rm -f /tmp/complex-original-md5.txt /tmp/complex-extracted-md5.txt
rm -rf /tmp/standard-test /tmp/standard-extracted /tmp/reference-extracted
rm -f /tmp/original-checksums.txt /tmp/standard-checksums.txt /tmp/reference-checksums.txt
rm -f /workdir/complex.tar.gz /workdir/standard.tar.gz /workdir/reference.tar.gz
echo "Temporary files cleaned up"
else
@ -24,75 +24,75 @@ run_basic_test() {
echo "=== RUNNING BASIC TEST ==="
# Clean up existing test directories
rm -rf /test-tmp/complex /test-tmp/complex-extracted
rm -f /test-tmp/complex-original-md5.txt /test-tmp/complex-extracted-md5.txt
rm -rf /tmp/complex /tmp/complex-extracted
rm -f /tmp/complex-original-md5.txt /tmp/complex-extracted-md5.txt
# Create complex directory structure
mkdir -p /test-tmp/complex/dir1/subdir1/subsubdir1
mkdir -p /test-tmp/complex/dir1/subdir2
mkdir -p /test-tmp/complex/dir2/subdir1
mkdir -p /tmp/complex/dir1/subdir1/subsubdir1
mkdir -p /tmp/complex/dir1/subdir2
mkdir -p /tmp/complex/dir2/subdir1
# Create files at different levels
echo 'root level file' > /test-tmp/complex/rootfile.txt
echo 'level 1 file in dir1' > /test-tmp/complex/dir1/file1.txt
echo 'level 1 file in dir2' > /test-tmp/complex/dir2/file2.txt
echo 'level 2 file in subdir1' > /test-tmp/complex/dir1/subdir1/file3.txt
echo 'level 2 file in subdir2' > /test-tmp/complex/dir1/subdir2/file4.txt
echo 'level 3 file in subsubdir1' > /test-tmp/complex/dir1/subdir1/subsubdir1/file5.txt
echo 'root level file' > /tmp/complex/rootfile.txt
echo 'level 1 file in dir1' > /tmp/complex/dir1/file1.txt
echo 'level 1 file in dir2' > /tmp/complex/dir2/file2.txt
echo 'level 2 file in subdir1' > /tmp/complex/dir1/subdir1/file3.txt
echo 'level 2 file in subdir2' > /tmp/complex/dir1/subdir2/file4.txt
echo 'level 3 file in subsubdir1' > /tmp/complex/dir1/subdir1/subsubdir1/file5.txt
# Create a symbolic link with a relative path instead of absolute
cd /test-tmp/complex/dir2 && ln -s ../rootfile.txt symlink.txt && cd /workdir
cd /tmp/complex/dir2 && ln -s ../rootfile.txt symlink.txt && cd /workdir
# Calculate MD5 hashes of original files for verification
find /test-tmp/complex -type f | sort | xargs md5sum > /test-tmp/complex-original-md5.txt
find /tmp/complex -type f | sort | xargs md5sum > /tmp/complex-original-md5.txt
# Print the original structure for reference
echo '=== ORIGINAL DIRECTORY STRUCTURE ==='
find /test-tmp/complex -type f -o -type l | sort
find /tmp/complex -type f -o -type l | sort
# Create the tarball
/bin/tarballer -source /test-tmp/complex -output /workdir/complex.tar.gz -prefix complex-app
/bin/tarballer -source /tmp/complex -output /workdir/complex.tar.gz -prefix complex-app
# Extract the tarball
mkdir -p /test-tmp/complex-extracted
tar -xzf /workdir/complex.tar.gz -C /test-tmp/complex-extracted
mkdir -p /tmp/complex-extracted
tar -xzf /workdir/complex.tar.gz -C /tmp/complex-extracted
# Verify the extracted structure
echo '=== EXTRACTED DIRECTORY STRUCTURE ==='
find /test-tmp/complex-extracted -type f -o -type l | sort
find /tmp/complex-extracted -type f -o -type l | sort
# Calculate MD5 hashes of extracted files
find /test-tmp/complex-extracted -type f | sort | xargs md5sum > /test-tmp/complex-extracted-md5.txt
find /tmp/complex-extracted -type f | sort | xargs md5sum > /tmp/complex-extracted-md5.txt
# Compare file content
echo '=== VERIFYING FILE CONTENTS ==='
cat /test-tmp/complex/rootfile.txt
cat /tmp/complex/rootfile.txt
echo ' <-- Original: rootfile.txt'
cat /test-tmp/complex-extracted/complex-app/rootfile.txt
cat /tmp/complex-extracted/complex-app/rootfile.txt
echo ' <-- Extracted: rootfile.txt'
cat /test-tmp/complex/dir1/subdir1/subsubdir1/file5.txt
cat /tmp/complex/dir1/subdir1/subsubdir1/file5.txt
echo ' <-- Original: deep nested file5.txt'
cat /test-tmp/complex-extracted/complex-app/dir1/subdir1/subsubdir1/file5.txt
cat /tmp/complex-extracted/complex-app/dir1/subdir1/subsubdir1/file5.txt
echo ' <-- Extracted: deep nested file5.txt'
# Test symlink
echo '=== TESTING SYMLINK ==='
ls -la /test-tmp/complex/dir2/symlink.txt
ls -la /test-tmp/complex-extracted/complex-app/dir2/symlink.txt
ls -la /tmp/complex/dir2/symlink.txt
ls -la /tmp/complex-extracted/complex-app/dir2/symlink.txt
# Verify MD5 hashes
echo '=== MD5 HASH VERIFICATION ==='
echo 'Original file hashes:'
cat /test-tmp/complex-original-md5.txt
cat /tmp/complex-original-md5.txt
echo 'Extracted file hashes:'
cat /test-tmp/complex-extracted-md5.txt
cat /tmp/complex-extracted-md5.txt
# Verify hash comparison
echo '=== COMPARING FILE HASHES ==='
# Extract just file paths from original hashes
ORIG_FILES=$(cat /test-tmp/complex-original-md5.txt | awk '{print $2}' | sort)
ORIG_FILES=$(cat /tmp/complex-original-md5.txt | awk '{print $2}' | sort)
# For each original file, check if its corresponding extracted file has the same hash
ALL_MATCH=1
@ -101,10 +101,10 @@ run_basic_test() {
FILENAME=$(basename "$SOURCE_FILE")
# Find the corresponding hash from original file
ORIG_HASH=$(grep "$SOURCE_FILE" /test-tmp/complex-original-md5.txt | awk '{print $1}')
ORIG_HASH=$(grep "$SOURCE_FILE" /tmp/complex-original-md5.txt | awk '{print $1}')
# Find the corresponding file in the extracted directory and get its hash
EXTRACTED_FILE=$(find /test-tmp/complex-extracted -name "$FILENAME" | head -1)
EXTRACTED_FILE=$(find /tmp/complex-extracted -name "$FILENAME" | head -1)
if [ -z "$EXTRACTED_FILE" ]; then
echo "ERROR: File $FILENAME not found in extracted directory"
@ -112,7 +112,7 @@ run_basic_test() {
continue
fi
EXTRACTED_HASH=$(grep "$EXTRACTED_FILE" /test-tmp/complex-extracted-md5.txt | awk '{print $1}')
EXTRACTED_HASH=$(grep "$EXTRACTED_FILE" /tmp/complex-extracted-md5.txt | awk '{print $1}')
if [ "$ORIG_HASH" != "$EXTRACTED_HASH" ]; then
echo "ERROR: Hash mismatch for $FILENAME: original=$ORIG_HASH extracted=$EXTRACTED_HASH"
@ -135,66 +135,66 @@ run_tar_comparison_test() {
echo "=== RUNNING TAR COMPARISON TEST ==="
# Clean up test directories
rm -rf /test-tmp/standard-test /test-tmp/standard-extracted /test-tmp/reference-extracted
rm -f /test-tmp/original-checksums.txt /test-tmp/standard-checksums.txt /test-tmp/reference-checksums.txt
rm -rf /tmp/standard-test /tmp/standard-extracted /tmp/reference-extracted
rm -f /tmp/original-checksums.txt /tmp/standard-checksums.txt /tmp/reference-checksums.txt
# Create a diverse test directory structure
mkdir -p /test-tmp/standard-test/config/settings
mkdir -p /test-tmp/standard-test/data/user/documents
mkdir -p /test-tmp/standard-test/data/user/pictures
mkdir -p /test-tmp/standard-test/logs
mkdir -p /tmp/standard-test/config/settings
mkdir -p /tmp/standard-test/data/user/documents
mkdir -p /tmp/standard-test/data/user/pictures
mkdir -p /tmp/standard-test/logs
# Create various file types
echo '{"app": "tarballer", "version": "1.0"}' > /test-tmp/standard-test/config/settings/app.json
echo 'debug=true' > /test-tmp/standard-test/config/settings/debug.conf
dd if=/dev/urandom bs=1K count=10 of=/test-tmp/standard-test/data/user/documents/binary.dat 2>/dev/null
echo 'Test log entry 1' > /test-tmp/standard-test/logs/app.log
echo 'Test log entry 2' >> /test-tmp/standard-test/logs/app.log
echo '{"app": "tarballer", "version": "1.0"}' > /tmp/standard-test/config/settings/app.json
echo 'debug=true' > /tmp/standard-test/config/settings/debug.conf
dd if=/dev/urandom bs=1K count=10 of=/tmp/standard-test/data/user/documents/binary.dat 2>/dev/null
echo 'Test log entry 1' > /tmp/standard-test/logs/app.log
echo 'Test log entry 2' >> /tmp/standard-test/logs/app.log
# Create symlinks
ln -s ../config/settings/app.json /test-tmp/standard-test/data/config-link.json
ln -s ../../logs/app.log /test-tmp/standard-test/data/user/log-link.txt
ln -s ../config/settings/app.json /tmp/standard-test/data/config-link.json
ln -s ../../logs/app.log /tmp/standard-test/data/user/log-link.txt
# Store MD5 hashes of original files for comparison
find /test-tmp/standard-test -type f | sort | xargs md5sum > /test-tmp/original-checksums.txt
find /tmp/standard-test -type f | sort | xargs md5sum > /tmp/original-checksums.txt
# Create tarball using our utility
/bin/tarballer -source /test-tmp/standard-test -output /workdir/standard.tar.gz -prefix app
/bin/tarballer -source /tmp/standard-test -output /workdir/standard.tar.gz -prefix app
# Create a reference tarball using standard tar for comparison
tar -czf /workdir/reference.tar.gz -C /test-tmp/standard-test --transform 's,^./,app/,' .
tar -czf /workdir/reference.tar.gz -C /tmp/standard-test --transform 's,^./,app/,' .
echo '=== TARBALLER OUTPUT ==='
mkdir -p /test-tmp/standard-extracted
tar -xzf /workdir/standard.tar.gz -C /test-tmp/standard-extracted
find /test-tmp/standard-extracted -type f -o -type l | sort
mkdir -p /tmp/standard-extracted
tar -xzf /workdir/standard.tar.gz -C /tmp/standard-extracted
find /tmp/standard-extracted -type f -o -type l | sort
# Extract reference tarball
echo '=== REFERENCE TAR OUTPUT ==='
mkdir -p /test-tmp/reference-extracted
tar -xzf /workdir/reference.tar.gz -C /test-tmp/reference-extracted
find /test-tmp/reference-extracted -type f -o -type l | sort
mkdir -p /tmp/reference-extracted
tar -xzf /workdir/reference.tar.gz -C /tmp/reference-extracted
find /tmp/reference-extracted -type f -o -type l | sort
# Verify checksums match for all extracted files
echo '=== CHECKSUMS OF EXTRACTED FILES ==='
find /test-tmp/standard-extracted -type f | sort | xargs md5sum > /test-tmp/standard-checksums.txt
find /test-tmp/reference-extracted -type f | sort | xargs md5sum > /test-tmp/reference-checksums.txt
find /tmp/standard-extracted -type f | sort | xargs md5sum > /tmp/standard-checksums.txt
find /tmp/reference-extracted -type f | sort | xargs md5sum > /tmp/reference-checksums.txt
echo 'ORIGINAL FILE CHECKSUMS:'
cat /test-tmp/original-checksums.txt
cat /tmp/original-checksums.txt
echo 'TARBALLER EXTRACTED CHECKSUMS:'
cat /test-tmp/standard-checksums.txt
cat /tmp/standard-checksums.txt
echo 'REFERENCE TAR EXTRACTED CHECKSUMS:'
cat /test-tmp/reference-checksums.txt
cat /tmp/reference-checksums.txt
# Compare MD5 checksums systematically
echo '=== SYSTEMATIC MD5 COMPARISON ==='
# Compare original files to tarballer extraction
MATCH_COUNT=0
EXPECTED_COUNT=$(cat /test-tmp/original-checksums.txt | wc -l)
EXPECTED_COUNT=$(cat /tmp/original-checksums.txt | wc -l)
# For each original file, find its corresponding extracted file and compare hashes
while read -r line; do
@ -203,11 +203,11 @@ run_tar_comparison_test() {
FILENAME=$(basename "$ORIG_FILE")
# Find the corresponding file in the extracted directory
EXTRACTED_FILE=$(find /test-tmp/standard-extracted -name "$FILENAME" | grep -v ".md5-manifest.txt" | head -1)
EXTRACTED_FILE=$(find /tmp/standard-extracted -name "$FILENAME" | grep -v ".md5-manifest.txt" | head -1)
if [ -n "$EXTRACTED_FILE" ]; then
# Get the hash of the extracted file
EXTRACTED_HASH=$(grep "$EXTRACTED_FILE" /test-tmp/standard-checksums.txt | awk '{print $1}')
EXTRACTED_HASH=$(grep "$EXTRACTED_FILE" /tmp/standard-checksums.txt | awk '{print $1}')
if [ "$ORIG_HASH" = "$EXTRACTED_HASH" ]; then
MATCH_COUNT=$((MATCH_COUNT + 1))
@ -215,7 +215,7 @@ run_tar_comparison_test() {
echo "Hash mismatch for $FILENAME: original=$ORIG_HASH extracted=$EXTRACTED_HASH"
fi
fi
done < /test-tmp/original-checksums.txt
done < /tmp/original-checksums.txt
if [ "$MATCH_COUNT" -eq "$EXPECTED_COUNT" ]; then
echo 'SUCCESS: Tarballer extraction hashes match original files!'
@ -233,11 +233,11 @@ run_tar_comparison_test() {
FILENAME=$(basename "$ORIG_FILE")
# Find the corresponding file in the extracted directory
EXTRACTED_FILE=$(find /test-tmp/reference-extracted -name "$FILENAME" | head -1)
EXTRACTED_FILE=$(find /tmp/reference-extracted -name "$FILENAME" | head -1)
if [ -n "$EXTRACTED_FILE" ]; then
# Get the hash of the extracted file
EXTRACTED_HASH=$(grep "$EXTRACTED_FILE" /test-tmp/reference-checksums.txt | awk '{print $1}')
EXTRACTED_HASH=$(grep "$EXTRACTED_FILE" /tmp/reference-checksums.txt | awk '{print $1}')
if [ "$ORIG_HASH" = "$EXTRACTED_HASH" ]; then
MATCH_COUNT=$((MATCH_COUNT + 1))
@ -245,7 +245,7 @@ run_tar_comparison_test() {
echo "Hash mismatch for $FILENAME: original=$ORIG_HASH reference=$EXTRACTED_HASH"
fi
fi
done < /test-tmp/original-checksums.txt
done < /tmp/original-checksums.txt
if [ "$MATCH_COUNT" -eq "$EXPECTED_COUNT" ]; then
echo 'SUCCESS: Reference tar extraction hashes match original files!'
@ -256,21 +256,21 @@ run_tar_comparison_test() {
echo '=== VERIFYING SYMLINKS ==='
echo 'ORIGINAL SYMLINKS:'
find /test-tmp/standard-test -type l -exec ls -la {} \;
find /tmp/standard-test -type l -exec ls -la {} \;
echo 'EXTRACTED SYMLINKS:'
find /test-tmp/standard-extracted -type l -exec ls -la {} \;
find /tmp/standard-extracted -type l -exec ls -la {} \;
# Compare file counts to ensure all files were extracted
echo '=== FILE COUNT COMPARISON ==='
echo -n 'Original files: ' && find /test-tmp/standard-test -type f | wc -l
echo -n 'Extracted files: ' && find /test-tmp/standard-extracted -type f | wc -l
echo -n 'Original files: ' && find /tmp/standard-test -type f | wc -l
echo -n 'Extracted files: ' && find /tmp/standard-extracted -type f | wc -l
# Test symlink functionality
echo '=== TESTING SYMLINK CONTENT ==='
echo 'Original linked content:'
cat /test-tmp/standard-test/data/config-link.json
cat /tmp/standard-test/data/config-link.json
echo 'Extracted linked content:'
cat /test-tmp/standard-extracted/app/data/config-link.json
cat /tmp/standard-extracted/app/data/config-link.json
echo 'Tar comparison test completed successfully!'
return 0