Docker Test Environment

This directory contains a complete Docker-based test environment for the Airflow plugin.

Files

• Dockerfile.test - Docker image that runs tests via tox
• docker-compose.test.yml - Docker Compose config with automatic volume mounts
• run-tests.sh - Wrapper script for the easiest test experience
• DOCKER_TEST_GUIDE.md - Complete documentation with examples

Quick Start

Easiest: Use the Wrapper Script

# Run all tests
./run-tests.sh

# Run specific test
./run-tests.sh py311-airflow31 -- tests/integration/test_plugin.py::test_v2_basic_dag -v

# Update golden files (automatically saved to your local filesystem!)
./run-tests.sh py311-airflow31 -- --update-golden-files

Benefits:

• ✅ Automatic volume mounts (no manual -v flags)
• ✅ Correct file permissions (no root-owned files)
• ✅ Auto-builds image if needed
• ✅ Simple to use

Alternative: Docker Compose

# Run all tests
docker compose -f docker-compose.test.yml run --rm airflow-plugin-test

# Update golden files
docker compose -f docker-compose.test.yml run --rm airflow-plugin-test

Features

Automatic Volume Mounts

Both the wrapper script and Docker Compose automatically mount:

• Source code (metadata-ingestion and airflow-plugin)
• Golden files (tests/integration/goldens)
• Tox cache (.tox directory - Docker Compose only)

This means:

• ✅ No manual volume mount commands
• ✅ Golden files are automatically updated on your local filesystem
• ✅ Source code changes are immediately available
• ✅ Tox environments are cached between runs (Docker Compose)

Tox-Based Dependency Management

The Docker image uses tox to install all dependencies:

• Airflow versions (2.7, 2.8, 2.9, 2.10, 3.1)
• Airflow constraints files (for reproducible builds)
• Provider packages (Airflow 3.x)
• All test dependencies

This ensures:

• ✅ No duplicate dependency logic
• ✅ Same dependencies as local development
• ✅ Automatic constraint file handling

Multiple Airflow Versions

Test against any Airflow version in one image:

./run-tests.sh py310-airflow27  # Airflow 2.7
./run-tests.sh py311-airflow29  # Airflow 2.9
./run-tests.sh py311-airflow31  # Airflow 3.1

Why Use Docker?

Use Case	Recommended Approach
Local development	Local tox (faster iteration)
CI/CD pipelines	Docker (complete isolation)
Cross-platform testing	Docker (consistent environment)
Sharing test environment	Docker (works everywhere)
Testing on different OS	Docker (same Linux base)

Documentation

See DOCKER_TEST_GUIDE.md for:

• Detailed usage examples
• All three methods (wrapper, compose, CLI)
• Advanced configuration
• CI/CD integration examples
• Troubleshooting guide

Quick Reference

# Wrapper Script (easiest)
./run-tests.sh                                              # All tests
./run-tests.sh py311-airflow29                              # Airflow 2.9
./run-tests.sh py311-airflow31 -- -k snowflake -v           # Specific tests
./run-tests.sh py311-airflow31 -- --update-golden-files     # Update golden files
REBUILD=true ./run-tests.sh                                 # Force rebuild

# Docker Compose (recommended for CI/CD)
docker compose -f docker-compose.test.yml run --rm airflow-plugin-test
docker compose -f docker-compose.test.yml run --rm airflow-plugin-test py311-airflow29
docker compose -f docker-compose.test.yml build

# Docker CLI (most control)
docker build -f Dockerfile.test -t airflow-plugin-test ../../
docker run --rm airflow-plugin-test
docker run --rm airflow-plugin-test py311-airflow31 -- -v

Support

• Issues with tests: See tox.ini and test files
• Issues with Docker: See DOCKER_TEST_GUIDE.md
• Issues with dependencies: See setup.py and tox configuration