Winner of ML4Earth - Foundation Models for Earth Observation Hackathon 2024
September 18-23, 2024 | €1,800 in prizes | 139 participants
Prithvi4QR is an advanced Earth Observation foundation model designed for quick-response (QR) teams to support decision makers during climate-related disasters. By fine-tuning the NASA-IBM Prithvi-ViT-100 foundation model, our solution provides rapid, accurate spatial information for effective resource allocation during large-scale disasters like floods, wildfires, and other climate emergencies.
- Input: Optical images of size 512×512 pixels
- Output: High-resolution segmentation masks with 5 critical land cover classes
- Purpose: Supporting quick-response teams with automated spatial analysis
- Impact: Reducing manual interpretation time for emergency decision-making
- Foundation Model: NASA-IBM Prithvi-ViT-100 (100M parameters)
- Decoder: UperNet for semantic segmentation
- Fine-tuned Parameters: 12.1 million parameters
- Training Strategy: Frozen backbone with trainable decoder
| Class | Description | Use Case |
|---|---|---|
| 🌫️ Background | Non-classified areas | General terrain |
| 🏢 Buildings | Urban structures | Infrastructure assessment |
| 🌲 Woodland | Forest areas | Environmental monitoring |
| 💧 Water | Water bodies | Flood extent mapping |
| 🛣️ Roads | Transportation networks | Accessibility analysis |
- Training Duration: 50 epochs maximum
- Overall Accuracy: 90.43% on test data
- Input Resolution: 512×512 pixels
- GPU Acceleration: Single device training
- Class Balancing: Advanced weighting for imbalanced datasets
model_args = {
"backbone": "prithvi_vit_100",
"decoder": "UperNetDecoder",
"in_channels": 3,
"num_classes": 5,
"bands": [HLSBands.RED, HLSBands.GREEN, HLSBands.BLUE],
"pretrained": True,
"decoder_channels": 256,
"head_dropout": 0.1,
"freeze_backbone": True
}# Key Training Parameters
batch_size: 32
learning_rate: 5e-4
optimizer: AdamW
weight_decay: 0.05
max_epochs: 50
precision: 16-mixed
accelerator: gpuclass_weights = [0.02, 0.65, 0.04, 0.1, 0.4]
# Background, Buildings, Woodland, Water, Roads- Python 3.10+
- CUDA-capable GPU (recommended)
- 16GB+ RAM
# Clone the repository
git clone https://github.com/ro-hit81/ML4EARTH.git
cd ML4EARTH
# Create conda environment
conda env create -f environment.yml
conda activate terratorch
# Download LandCover.ai dataset (automatic via torchgeo)terratorch: Foundation model frameworkpytorch-lightning: Training orchestrationtorchgeo: Geospatial datasets and transformstensorboard: Experiment trackingtimm: Model architectures
# Fine-tune Prithvi model on LandCover.ai dataset
jupyter notebook prithvi_finetuning.ipynb# Generate predictions and visualizations
jupyter notebook prediction.ipynb# Analyze metrics and create reports
jupyter notebook analysis.ipynb# Alternative: Use YAML configuration
python -m lightning.pytorch.cli fit --config landcoverai_config.yamlML4EARTH/
│
├── 📄 README.md # Project documentation
├── 📓 prithvi_finetuning.ipynb # Main training notebook
├── 📓 prediction.ipynb # Inference and visualization
├── 📓 analysis.ipynb # Performance analysis
├── ⚙️ landcoverai_config.yaml # Training configuration
├── 🐍 environment.yml # Conda environment
│
├── 📂 predicted/ # Model outputs
├── 📂 data/landcoverai/ # Dataset (auto-downloaded)
├── 📂 models/logs/ # Training logs
└── 🎥 ML4EARTH Hackathon Introduction.mp4 # Project demo
- Overall Accuracy: 90.43%
- F1 Score: 90.43%
- Test Loss: 0.247
| Class | Accuracy | Jaccard Index |
|---|---|---|
| Woodland 🌲 | 96.12% | 85.21% |
| Water 💧 | 95.32% | 78.85% |
| Building 🏢 | 92.03% | 43.35% |
| Background 🌫️ | 86.82% | 85.02% |
| Road 🛣️ | 79.74% | 41.00% |
- Best performing classes: Woodland and Water detection
- Challenging classes: Roads and Buildings (lower IoU despite good accuracy)
- Overall strong performance: 90%+ overall accuracy with room for improvement in segmentation precision
Climate change disasters like European floods require rapid spatial intelligence for effective resource allocation. Our solution bridges the gap between raw satellite imagery and actionable insights for emergency response teams.
- First-time EO Foundation Model Fine-tuning: Successfully adapted cutting-edge models
- Class Imbalance Solutions: Advanced weighting strategies for real-world datasets
- Real-time Capability: Optimized for emergency response scenarios
- Multi-framework Integration: Seamless combination of terratorch, PyTorch Lightning, and TorchGeo
- Successfully fine-tuned two different Earth Observation foundation models
- Achieved notable results under strict time constraints
- Created a production-ready solution for disaster response
- Implemented comprehensive evaluation framework
| Model | Architecture | Implementation | Selection |
|---|---|---|---|
| Prithvi-ViT-100 ✅ | Vision Transformer | TeraTorch framework | Selected |
| Satlas Aerial SwinB | Swin Transformer | Alternative considered | Evaluated |
Decision: Prithvi model selected based on framework compatibility and foundation model capabilities.
- Flood Mapping: Rapid water extent assessment
- Infrastructure Damage: Building and road network analysis
- Evacuation Planning: Accessibility route identification
- Resource Allocation: Priority area determination
- Deforestation Detection: Woodland area changes
- Urban Expansion: Building development tracking
- Water Resource Management: Surface water monitoring
- Disaster Impact Assessment: Before/after comparisons
- Infrastructure Development: Smart city planning
- Transportation Networks: Road connectivity analysis
- Green Space Management: Urban forest monitoring
- Population Distribution: Settlement pattern analysis
# Freeze backbone for stable fine-tuning
freeze_backbone: True
# Dynamic learning rate scheduling
lr: 5e-4
# Mixed precision for memory efficiency
precision: "16-mixed"
# Class weights for imbalanced data
class_weights: [0.02, 0.55, 0.04, 0.14, 0.25]- Geometric: Rotation, flipping, scaling
- Photometric: Brightness, contrast adjustments
- Spatial: Random cropping and resizing
- Domain-specific: Atmospheric effects simulation
- Primary: Weighted Cross-Entropy Loss
- Focus: Class imbalance handling
- Monitoring: Jaccard Index for critical classes
- Early Stopping: Patience-based convergence
- Model Integration: Adapting foundation models to specific frameworks
- Class Imbalance: Handling unequal distribution in land cover data
- Framework Compatibility: Ensuring smooth integration with TeraTorch
- Evaluation Metrics: Implementing comprehensive performance tracking
- Foundation Model Selection: Chose Prithvi for framework compatibility
- Class Balancing: Dynamic weight adjustment in loss function
- Training Optimization: Mixed precision and efficient data loading
- Comprehensive Monitoring: TensorBoard integration for metric tracking
We welcome contributions to improve Prithvi4QR! Here's how you can help:
- Use GitHub Issues for bug reports
- Include environment details and error logs
- Provide sample data and reproduction steps
- Additional land cover classes
- New foundation model backends
- Performance optimizations
- Deployment improvements
- Fork the repository
- Create a feature branch
- Implement changes with tests
- Submit pull request with documentation
- Clay et al. (2023): "Foundation Models for Earth Observation"
- NASA-IBM (2023): "Prithvi: A Foundation Model for Earth Observation"
- Bastani et al. (2023): "SatLas: A Large-Scale Multi-Modal Dataset"
- ML4Earth Hackathon 2024: Official Page
- TUM Data Science in EO: Competition organizers
- GitHub Starter Pack: ML4Earth-Hackathon-2024
- Terratorch: Foundation model training framework
- PyTorch Lightning: Scalable deep learning
- TorchGeo: Geospatial machine learning datasets
- TensorBoard: Experiment tracking and visualization
This project is open source and available under the MIT License.
- GitHub: ro-hit81/ML4EARTH
- Email: [email protected]
- Competition: ML4Earth DevPost
- Demo Video: ML4EARTH Hackathon Introduction.mp4
- Live Demo: Coming soon
- Documentation: This README and Jupyter notebooks
Making satellite imagery analysis as fast as decision-making demands.
