Installation

IPSL-AID uses uv for fast, portable, and reproducible environment management.

Prerequisites

  • Python 3.8 or higher

  • CUDA-compatible GPU (for training and inference)

  • Git (for cloning the repository)

Quick Install

  1. Clone the repository:

git clone https://github.com/kardaneh/IPSL-AID.git
cd IPSL-AID

  1. Create and activate a virtual environment:

uv venv --python=python3.11
source .venv/bin/activate

  1. Install the package in development mode:

uv pip install -e .

This will install all dependencies defined in pyproject.toml and make the ipsl-aid command available in your environment.

Verification

After installation, verify that the package is correctly installed:

# Check version (may take a few minutes on first run - loading dependencies)
ipsl-aid --version

# Check help (may also take a few minutes on first run)
ipsl-aid --help

# Faster alternative for version check
python -c "import IPSL_AID; print(IPSL_AID.__version__)"

# Verify CUDA availability
python -c "import torch; print(f'CUDA available: {torch.cuda.is_available()}')"
if torch.cuda.is_available():
    print(f'CUDA version: {torch.version.cuda}')
    print(f'GPU count: {torch.cuda.device_count()}')

# Test import
python -c "import IPSL_AID; print('IPSL-AID imported successfully')"

Note

The first execution of ipsl-aid --version or ipsl-aid --help can take up to couple of minutes. This is due to the current CLI implementation: heavy dependencies (such as PyTorch, Torchvision, NumPy, etc.) are imported at startup, even for lightweight commands.

On HPC machines, many of these libraries are already available via module load, which significantly reduce startup time. Users are encouraged to rely on these system-provided modules rather than reinstalling them in their virtual environment. In this case, the dependencies already available on the system need be commented out in pyproject.toml.

HPC Configuration

On HPC systems (like Leonardo), additional configuration may be necessary:

  1. Load CUDA modules (if not automatically loaded):

module load cuda/12.1

  1. Set up Cartopy data directory (to avoid downloading during runtime):

export CARTOPY_DATA_DIR="/path/to/cartopy_data"

  1. Configure environment variables for data paths:

export IPSL_AID_DATA_DIR="/path/to/your/data"
export IPSL_AID_OUTPUT_DIR="/path/to/your/outputs"

  1. SLURM job submission:

The package includes a bash script generator that creates SLURM submission scripts. Example usage:

# Generate SBATCH script with your configuration
./your_setup_script.sh

# Submit the job
sbatch slurm/sbatch_diffusion_*.sh

Hardware Requirements

  • Training: Multi-GPU systems (tested with 4× NVIDIA A100 64GB)

  • Inference: Single GPU or multi-GPU for parallel generation

  • Storage: Sufficient disk space for climate datasets (ERA5 ~TB scale)

  • Memory: Adequate RAM for data loading and preprocessing

Dependencies

IPSL-AID requires the following key dependencies (automatically installed):

  • PyTorch >= 2.5.1

  • NumPy, SciPy, Pandas, Xarray

  • Matplotlib, Cartopy, Seaborn

  • tqdm, Rich, TensorBoard

  • CDS API client (for data download)

Optional Dependencies

For development and documentation:

# Install development dependencies
uv pip install -e ".[dev]"

# Install documentation dependencies
uv pip install -e ".[docs]"

Development Installation

For developers working on the codebase:

  1. Install in editable mode with development dependencies:

uv pip install -e ".[dev]"

  1. Set up pre-commit hooks:

pre-commit install
pre-commit run --all-files

  1. Run tests:

python tests/test_all.py

Troubleshooting

Issue: ipsl-aid: command not found

Solution: Ensure the virtual environment is activated: source .venv/bin/activate

Issue: CUDA not available

Solution: Check PyTorch installation: python -c "import torch; print(torch.cuda.is_available())" If False, reinstall PyTorch with CUDA support.

Issue: Import errors

Solution: Verify package installation: uv pip list | grep ipsl If not found, reinstall: uv pip install -e .

Issue: Slow startup with --version or --help

Solution: This is expected on first run as PyTorch and other heavy modules load. We are actively working on implementing a fast CLI handler. For quick verification, use the alternative commands shown in the Verification section.

Issue: ModuleNotFoundError: No module named 'IPSL_AID'

Solution: Ensure the package is installed and the virtual environment is activated. Run: uv pip install -e . from the project root directory.

Issue: Cartopy data download fails

Solution: Set the Cartopy data directory to a shared location: export CARTOPY_DATA_DIR="/path/to/cartopy_data"

Getting Help