Pelvic Bone Fragments with Injuries Segmentation Challenge Banner
Join

Rules

  1. The PENGWIN Challenge consists of two independent tasks. Participant are allowed to choose one of them or both. Each task has a Preliminary Development Phase and a Final Test Phase. Training sets are now accessible on Zenodo for all participants.
  2. The test data will not be opened to the participants. For final submissions, participants must use a Docker container. A template for the Docker container will be made available prior to the opening of submissions.
  3. During the Preliminary Development Phase, participants may make up to five submissions to perform sanity checks using a small validation set. In the Final Test Phase, only one submission per participant is permitted.
  4. Participants are required to use their real names and affiliations on their Grand Challenge profiles. Entries from individuals with multiple or duplicate profiles will result in disqualification.
  5. Participants may develop their methods using pre-trained AI models or any publicly available datasets, provided these are under a permissive license. The use of such resources must be explicitly stated in their submissions.
  6. Top three performing teams are required to share their code on a public repository.
  7. Participants are allowed to publish their results separately after the challenge. The publication embargo period will be 12 months after the announcement of the challenge results.

Evaluation

  • The primary metrics for evaluation are Intersection over Union (IoU), Hausdorff Distance at 95% (HD95), and Average Symmetric Surface Distance (ASSD). 
  • The predicted fragment instances and the ground-truth labels are matched based on the highest IoU.
  • Evaluation is perfromed on both fracture segmentation result (comparing matched fragment instance and ground truth) and anatomical segmentation result (labels for each bone are merged and compared to the ground-truth anatomical labels of sacrum, left hipbone, and right hipbone). Therefore, there are six final metrics: fracture_iou, fracture_hd95, fracture_assd, anatomical_iou, anatomical_hd95, and anatomical_assd.
  • The metrics are ranked indepentedly. The final rank for each submission is determined by averaging the ranks on single metrics.