Phase 2 rules and procedure

• Phase 2 uses an entirely new dataset which is hosted separately on EOTDL here: https://www.eotdl.com/datasets/charter-eo4ai-etq-challenge-testing
• Please note that only data from this new dataset is viable to be used for your submissions for Phase 2!
• If you have access to the Phase 1 dataset your access to the Phase 2 dataset will be transferred automatically. If you have trouble accessing it, please shoot us a message.

Two Phases, One Mission: Real-World Disaster Response

In this ESA Phi-Lab AI Challenge, we’re not just testing models, we’re testing readiness.

This challenge is designed to simulate the real-world conditions of a satellite-based disaster response mission. That’s why we’ve structured it into two distinct phases that push participants to go beyond model accuracy and demonstrate true operational capability

Phase 1: Training and Live Scoring - Fill in the Gaps

During this first stage, participants will be provided with satellite imagery from past earthquake-affected areas. Some images will include complete annotations (damaged and undamaged buildings) to be used for training. Other images will only be partially annotated, and you will be asked to return these building files completely annotated. The annotations you fill in will be used for the Live Scoring and will be visible to all on the leaderboard. So, your mission in this phase is to train your models as best as possible using the available annotations, then annotate the missing buildings in the partially labeled scenes to track the progress of your model’s accuracy.

This phase will assess your model's ability to generalize from incomplete information, a critical skill in humanitarian crisis scenarios.

Phase 2: The Stress Test – A Real Case Simulation

Now the pressure is on.

In the final days of the challenge, participants will be given completely unannotated images from new, undisclosed locations affected by earthquakes.

Just like in a real International Charter activation, you won’t know the location, you’ll work with no pre-annotations, and you’ll have a tight window to deliver the associated building polygon files completely annotated as damaged or undamaged.

In a real activation, the first EO-based map products are usually sent to the end user within 72 hours, however we are granting a bit of extra time for this test to allow participants with previous time commitments adequate time to receive and understand the given dataset. This phase will therefore last 9 days, giving participants the opportunity to review their results and make any final adjustments before submission, as the previously unseen data may unlock new training possibilities. In the end, your model should be robust, ready, and scalable.

This is the ultimate test: Can your model deliver actionable results in unfamiliar, high-pressure conditions?

What You’ll Work With

• Satellite imagery from recent Charter earthquake activations (e.g., Türkiye, Syria, Morocco)
• Pre- and post-event scenes, some containing panchromatic assets for optional pan-sharpening
• Hand-curated building polygon files
• Object-level annotations (damaged/undamaged), where available
• Realistic scenes with:
  • Partial annotations
  • Non-co-registered imagery
  • Mixed sensor types and angles
  • Some images contain Multispectral and Panchromatic assests (to be pan-sharpened) others contain only pre-sharpened multispectral assets

Participant recommendations

Your models can choose to:

• Use both pre- and post-event imagery or post-event alone
• Perform pan-sharpening using provided panchromatic bands, or take as input pre-sharpened images
• Implement image co-registration to correct for differences in viewing angles, spatial alignment, or sensor types

There are multiple valid approaches. Your task is to create a model capable of efficiently annotating large urban areas given a building polygon file and satellite imagery, just like in a real-time emergency where responders need to locate the worst-hit zones as fast as possible. In a real scenario, EO experts involved in a charter earthquake activation would retrieve the satellite data from the Charter portal, retrieve the available building polygon file from OpenStreetMap, and begin the manual annotation process in a GIS system in order to create a map product. This map product would then be delivered to the Charter End User, which is typically the civil protection of the affected country. With an effective AI model, the time spent on the annotation process can be greatly reduced and therefore the damage maps can be delivered at a faster rate.

You’ll also get access to community support channels and best practice guides to help you along the way.

This phase simulates the operational urgency of a Charter activation, where trained models could make an impact in the delivery of actionable information in unfamiliar contexts.

Please see the “rules” tab for more information on the evaluation of this challenge!

Two Versions of the Phase 1 Dataset Available

To support participants with varying technical and storage capacities, two versions of the Phase 1 dataset are available for download:

• Slim Dataset (138 GB) – This version includes pre- and post-event imagery from Pleiades, WorldView, and GeoEye sensors, which were used to create the building annotations. It’s ideal for getting a first look at the challenge data without having to download the full set, understanding the structure, and training a model using the most consistent and high-quality sources.
• Full Dataset (475 GB) – In addition to the slim dataset, this version includes additional imagery from other sensors commonly used in Charter activations. This richer dataset enables you to train more robust and generalizable models capable of handling variations in sensor type, image quality, and acquisition conditions.

While the challenge can be completed using only the slim dataset, the full dataset better reflects the spirit of the International Charter, which promotes collaborative, multi-agency data use. In real activations, sensor availability varies by region and time, so tuning your model to work across diverse data sources is part of simulating an operational scenario. The full dataset will be available on the EOTDL on July 18th. In the meantime, the slim dataset contains all the required data to begin training your model.

Need Help?

Join the discussion forum, ask questions, or reach out through the official support channels provided with the challenge. We’re here to help.