Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge
Authors:
Hugo J. Kuijf,
J. Matthijs Biesbroek,
Jeroen de Bresser,
Rutger Heinen,
Simon Andermatt,
Mariana Bento,
Matt Berseth,
Mikhail Belyaev,
M. Jorge Cardoso,
Adrià Casamitjana,
D. Louis Collins,
Mahsa Dadar,
Achilleas Georgiou,
Mohsen Ghafoorian,
Dakai **,
April Khademi,
Jesse Knight,
Hongwei Li,
Xavier Lladó,
Miguel Luna,
Qaiser Mahmood,
Richard McKinley,
Alireza Mehrtash,
Sébastien Ourselin,
Bo-yong Park
, et al. (19 additional authors not shown)
Abstract:
Quantification of cerebral white matter hyperintensities (WMH) of presumed vascular origin is of key importance in many neurological research studies. Currently, measurements are often still obtained from manual segmentations on brain MR images, which is a laborious procedure. Automatic WMH segmentation methods exist, but a standardized comparison of the performance of such methods is lacking. We…
▽ More
Quantification of cerebral white matter hyperintensities (WMH) of presumed vascular origin is of key importance in many neurological research studies. Currently, measurements are often still obtained from manual segmentations on brain MR images, which is a laborious procedure. Automatic WMH segmentation methods exist, but a standardized comparison of the performance of such methods is lacking. We organized a scientific challenge, in which developers could evaluate their method on a standardized multi-center/-scanner image dataset, giving an objective comparison: the WMH Segmentation Challenge (https://wmh.isi.uu.nl/).
Sixty T1+FLAIR images from three MR scanners were released with manual WMH segmentations for training. A test set of 110 images from five MR scanners was used for evaluation. Segmentation methods had to be containerized and submitted to the challenge organizers. Five evaluation metrics were used to rank the methods: (1) Dice similarity coefficient, (2) modified Hausdorff distance (95th percentile), (3) absolute log-transformed volume difference, (4) sensitivity for detecting individual lesions, and (5) F1-score for individual lesions. Additionally, methods were ranked on their inter-scanner robustness.
Twenty participants submitted their method for evaluation. This paper provides a detailed analysis of the results. In brief, there is a cluster of four methods that rank significantly better than the other methods, with one clear winner. The inter-scanner robustness ranking shows that not all methods generalize to unseen scanners.
The challenge remains open for future submissions and provides a public platform for method evaluation.
△ Less
Submitted 1 April, 2019;
originally announced April 2019.
BioInfoBase : A Bioinformatics Resourceome
Authors:
Saeid Kadkhodaei,
Fatemeh Barantalab,
Sima Taheri,
Majid Foroughi,
Farahnaz Golestan Hashemi,
Mahmood Reza Shabanimofrad,
Hossein Hosseinimonfared,
Morvarid Akhavan Rezaei,
Ali Ranjbarfard,
Mahbod Sahebi,
Parisa Azizi,
Maryam Dadar,
Rambod Abiri,
Mohammad Fazel Harighi,
Nahid Kalhori,
Mohammad Reza Etemadi,
Ali Baradaran,
Mahmoud Danaee,
Iman Zare,
Ahmad Ghafarpour,
Zahra Azhdari,
Hamid Rajabi Memari,
Vajiheh Safavi,
Naser Tajabadi,
Faruku Bande
Abstract:
Over the past decade there has been a significant growth in bioinformatics databases, tools and resources. Although, bioinformatics is becoming more specific, increasing the number of bioinformatics-wares has made it difficult for researchers to find the most appropriate databases, tools or methods which match their needs. Our coordinated effort has been planned to establish a reference website in…
▽ More
Over the past decade there has been a significant growth in bioinformatics databases, tools and resources. Although, bioinformatics is becoming more specific, increasing the number of bioinformatics-wares has made it difficult for researchers to find the most appropriate databases, tools or methods which match their needs. Our coordinated effort has been planned to establish a reference website in Bioinformatics as a public repository of tools, databases, directories and resources annotated with contextual information and organized by functional relevance. Within the first phase of BioInfoBase development, 22 experts in different fields of molecular biology contributed and more than 2500 records were registered, which are increasing daily. For each record submitted to the database of website almost all related data (40 features) has been extracted. These include information from the biological category and subcategory to the scientific article and developer information. Searching the query keyword(s) returns links containing the entered keyword(s) found within the different features of the records with more weights on the title, abstract and application fields. The search results simply provide the users with the most informative features of the records to select the most suitable ones. The usefulness of the returned results is ranked according to the matching score based on the Term Frequency-Inverse Document Frequency (TF-IDF) methods. Therefore, this search engine will screen a comprehensive index of bioinformatics tools, databases and resources and provide the best suited records (links) to the researchers need. The BioInfoBase resource is available at www.bioinfobase.info.
△ Less
Submitted 20 November, 2016; v1 submitted 11 July, 2016;
originally announced July 2016.
