Manual reference standard

Training data is accompanied by the manual reference standard, containing the following three labels:

  1. Background
  2. White matter hyperintensities
  3. Other pathology

The objective of this challenge is to automatically segment WMH (label 0 vs 1). Because we do not require methods to identify all other types of pathology, we provide a rough mask (label 2) for them that is ignored during evaluation.

Rating criteria

WMH and other pathology were manually segmented according to the STandards for ReportIng Vascular changes on nEuroimaging (STRIVE). One expert observer (O1) segmented all images using a contour drawing technique, delineating the outline of all WMH. This observer O1 has extensive prior experience with the manual segmentation of WMH and has segmented 500+ cases. A second expert observer (O2) performed extensive peer review on all manual delineations. In case of mistakes, errors, or delineations that were not according to the STRIVE standards, O1 corrected the segmentation. Hence, the provided reference standard is the corrected segmentation of O1, after peer review by O2.

Binary masks

The contours were converted to binary masks, including all voxels whose volume >50% was included within the manually drawn contour. Background received label 0 and WMH label 1.

Other pathology was converted to binary masks as well, receiving label 2. These masks were dilated by 1 pixel in-plane (with a 3x3x1 kernel). In case of overlap between labels 1 and 2, label 1 was assigned.

Inter observer agreement

All training cases were manually segmented by two additional observers (O3 and O4), who were also subjected to peer review. The final inter observer agreement on the training cases will be announced as soon as possible.