DeepGestalt - Identifying Rare Genetic Syndromes Using Deep Learning.

Yaron Gurovich4
Estimated H-index: 4
Yair Hanani4
Estimated H-index: 4
+ 8 AuthorsKaren W. Gripp33
Estimated H-index: 33
Facial analysis technologies have recently measured up to the capabilities of expert clinicians in syndrome identification. To date, these technologies could only identify phenotypes of a few diseases, limiting their role in clinical settings where hundreds of diagnoses must be considered. We developed a facial analysis framework, DeepGestalt, using computer vision and deep learning algorithms, that quantifies similarities to hundreds of genetic syndromes based on unconstrained 2D images. DeepGestalt is currently trained with over 26,000 patient cases from a rapidly growing phenotype-genotype database, consisting of tens of thousands of validated clinical cases, curated through a community-driven platform. DeepGestalt currently achieves 91% top-10-accuracy in identifying over 215 different genetic syndromes and has outperformed clinical experts in three separate experiments. We suggest that this form of artificial intelligence is ready to support medical genetics in clinical and laboratory practices and will play a key role in the future of precision medicine.
Figures & Tables
  • References (36)
  • Citations (9)
📖 Papers frequently viewed together
5 Citations
11 Authors (Shahad Kudama, ..., Nicholas W. Wood)
2014ICCS: International Conference on Conceptual Structures
1 Citations
78% of Scinapse members use related papers. After signing in, all features are FREE.
#1Jean Tori Pantel (Charité)H-Index: 3
#2Max Zhao (Charité)H-Index: 3
Last. Peter KrawitzH-Index: 27
view all 11 authors...
Significant improvements in automated image analysis have been achieved over the recent years and tools are now increasingly being used in computer-assisted syndromology. However, the recognizability of the facial gestalt might depend on the syndrome and may also be confounded by severity of phenotype, size of available training sets, ethnicity, age, and sex. Therefore, benchmarking and comparing the performance of deep-learned classification processes is inherently difficult. For a systematic a...
4 CitationsSource
#1Smail Hadj-Rabia (Paris V: Paris Descartes University)H-Index: 29
#2Holm SchneiderH-Index: 17
Last. Dorothy K. Grange (WashU: Washington University in St. Louis)H-Index: 32
view all 11 authors...
X-linked hypohidrotic ectodermal dysplasia (XLHED) is a genetic disorder that affects ectodermal structures and presents with a characteristic facial appearance. The ability of automated facial recognition technology to detect the phenotype from images was assessed . In Phase 1 of this study we examined if the age of male patients affected the technology's recognition. In Phase 2 we investigated how well the technology discriminated affected males cases from female carriers and from individuals ...
9 CitationsSource
#1Hui DingH-Index: 5
#2Shaohua Kevin ZhouH-Index: 5
Last. Rama Chellappa (UMD: University of Maryland, College Park)H-Index: 93
view all 3 authors...
Relatively small data sets available for expression recognition research make the training of deep networks very challenging. Although fine-tuning can partially alleviate the issue, the performance is still below acceptable levels as the deep features probably contain redundant information from the pretrained domain. In this paper, we present FaceNet2ExpNet, a novel idea to train an expression recognition network based on static images. We first propose a new distribution function to model the h...
96 CitationsSource
#1Paul Kruszka (NIH: National Institutes of Health)H-Index: 12
#2Yonit A. Addissie (NIH: National Institutes of Health)H-Index: 7
Last. Maximilian Muenke (NIH: National Institutes of Health)H-Index: 66
view all 39 authors...
22q11.2 deletion syndrome (22q11.2 DS) is the most common microdeletion syndrome and is underdiagnosed in diverse populations. This syndrome has a variable phenotype and affects multiple systems, making early recognition imperative. In this study, individuals from diverse populations with 22q11.2 DS were evaluated clinically and by facial analysis technology. Clinical information from 106 individuals and images from 101 were collected from individuals with 22q11.2 DS from 11 countries; average a...
25 CitationsSource
Mar 1, 2017 in WACV (Workshop on Applications of Computer Vision)
#1Pushkar ShuklaH-Index: 1
#2Tanu Gupta (IITR: Indian Institute of Technology Roorkee)H-Index: 2
Last. Raman BalasubramanianH-Index: 1
view all 5 authors...
Developmental Disorders are chronic disabilities that have a severe impact on the day to day functioning of a large section of the human population. Recognizing developmental disorders from facial images is an important but a relatively unexplored challenge in the field of computer vision. This paper proposes a novel framework to detect developmental disorders from facial images. A spectrum of disorders constituting of Autism Spectrum Disorder, Cerebral Palsy, Fetal Alcohol Syndrome, Down syndro...
10 CitationsSource
#1Paul Kruszka (NIH: National Institutes of Health)H-Index: 12
#2Antonio R. PorrasH-Index: 7
Last. Maximilian Muenke (NIH: National Institutes of Health)H-Index: 66
view all 44 authors...
Down syndrome is the most common cause of cognitive impairment and presents clinically with universally recognizable signs and symptoms. In this study, we focus on exam findings and digital facial analysis technology in individuals with Down syndrome in diverse populations. Photos and clinical information were collected on 65 individuals from 13 countries, 56.9% were male and the average age was 6.6 years (range 1 month to 26 years; SD = 6.6 years). Subjective findings showed that clinical featu...
20 CitationsSource
#1Karen W. Gripp (DuPont)H-Index: 18
#2Laura Baker (DuPont)H-Index: 2
Last. Kristin G. Monaghan (GeneDx)H-Index: 13
view all 4 authors...
The genetic basis of numerous intellectual disability (ID) syndromes has recently been identified by applying exome analysis on a research or clinical basis. There is significant clinical overlap of biologically related syndromes, as exemplified by Nicolaides-Baraitser (NCBRS) and Coffin-Siris (CSS) syndrome. Both result from mutations affecting the BAF (mSWI/SNF) complex and belong to the growing category of BAFopathies. In addition to the notable clinical overlap between these BAFopathies, het...
16 CitationsSource
#2Lior Wolf (TAU: Tel Aviv University)H-Index: 46
Last. Matthew A. Deardoff (UPenn: University of Pennsylvania)H-Index: 1
view all 7 authors...
Facial analysis systems are becoming available to healthcare providers to aid in the recognition of dysmorphic phenotypes associated with a multitude of genetic syndromes. These technologies automatically detect facial points and extract various measurements from images to recognize dysmorphic features and evaluate similarities to known facial patterns (gestalts). To evaluate such systems' usefulness for supporting the clinical practice of healthcare professionals, the recognition accuracy of th...
29 CitationsSource
#1Karim KouzH-Index: 1
#2Christina LissewskiH-Index: 10
Last. Martin ZenkerH-Index: 46
view all 26 authors...
Purpose: Noonan syndrome (NS) is an autosomal-dominant disorder characterized by craniofacial dysmorphism, growth retardation, cardiac abnormalities, and learning difficulties. It belongs to the RASopathies, which are caused by germ-line mutations in genes encoding components of the RAS mitogen-activated protein kinase (MAPK) pathway. RIT1 was recently reported as a disease gene for NS, but the number of published cases is still limited. Methods: We sequenced RIT1 in 310 mutation-negative indivi...
24 CitationsSource
Dec 1, 2015 in ICCV (International Conference on Computer Vision)
#1Ziwei Liu (CUHK: The Chinese University of Hong Kong)H-Index: 16
#2Ping LuoH-Index: 33
Last. Xiaoou Tang (CUHK: The Chinese University of Hong Kong)H-Index: 99
view all 4 authors...
Predicting face attributes in the wild is challenging due to complex face variations. We propose a novel deep learning framework for attribute prediction in the wild. It cascades two CNNs, LNet and ANet, which are fine-tuned jointly with attribute tags, but pre-trained differently. LNet is pre-trained by massive general object categories for face localization, while ANet is pre-trained by massive face identities for attribute prediction. This framework not only outperforms the state-of-the-art w...
1,460 CitationsSource
Cited By9
This work describes an automatic methodology to discriminate between individuals with the genetic disorder Pitt-Hopkins syndrome (PTHS), and healthy individuals. As input data, the methodology accepts unconstrained frontal facial photographs, from which faces are located with Histograms of Oriented Gradients features descriptors. Pre-processing steps of the methodology consist of colour normalisation, scaling down, rotation, and cropping in order to produce a series of images of faces with consi...
#1Lynnea Myers (KI: Karolinska Institutet)
#1Lynnea Myers (KI: Karolinska Institutet)H-Index: 1
Last. Sven Bölte (KI: Karolinska Institutet)H-Index: 42
view all 7 authors...
Physical examinations are recommended as part of a comprehensive evaluation for individuals with neurodevelopmental disorders (NDDs), such as autism spectrum disorder (ASD) and attention-deficit/hyperactivity disorder. These examinations should include assessment for morphological variants. Previous studies have shown an increase in morphological variants in individuals with NDDs, particularly ASD, and that these variants may be present in greater amounts in individuals with genetic alterations....
Genetic diseases are the most common next-generation diseases because of the improper mutation of the genes and DNA. These genetic diseases are failed to predict with an accurate manner in the beginning stage by using the particular genes and related information. So, the genetic diseases are identified in the medical systems by utilizing the hybridization of multimedia techniques such as big data and related soft computing techniques.Initially, the genetic disease-related medical images are coll...
#1Dhanya Lakshmi Narayanan (Nizam's Institute of Medical Sciences)H-Index: 2
Last. Kaushik MandalH-Index: 1
view all 6 authors...
OBJECTIVE: To assess the utility of computer-aided facial analysis in identifying dysmorphic syndromes in Indian children. METHODS: Fifty-one patients with a definite molecular or cytogenetic diagnosis and recognizable facial dysmorphism were enrolled in the study and their facial photographs were uploaded in the Face2Gene software. The results provided by the software were compared with the molecular diagnosis. RESULTS: Of the 51 patients, the software predicted the correct diagnosis in 37 pati...
1 CitationsSource
#1Tzung-Chien Hsieh (Humboldt University of Berlin)H-Index: 2
#2Martin A. Mensah (Humboldt University of Berlin)H-Index: 3
Last. Peter M. Krawitz (University of Bonn)H-Index: 2
view all 94 authors...
Phenotype information is crucial for the interpretation of genomic variants. So far it has only been accessible for bioinformatics workflows after encoding into clinical terms by expert dysmorphologists. Here, we introduce an approach driven by artificial intelligence that uses portrait photographs for the interpretation of clinical exome data. We measured the value added by computer-assisted image analysis to the diagnostic yield on a cohort consisting of 679 individuals with 105 different mono...
3 CitationsSource
#1Raymond Saa-Eru Maalman (University of Health and Allied Sciences)H-Index: 1
#2Chrissie Stansie Abaidoo (KNUST: Kwame Nkrumah University of Science and Technology)H-Index: 3
Last. Joshua Tetteh (KNUST: Kwame Nkrumah University of Science and Technology)H-Index: 1
view all 4 authors...
Abstract Craniofacial measurements can be considered to be one of the important tools for determination of the inter-racial and intra-racial morphological characteristics of the head and face. As such, facial indices serve as prominent identification tools in combination with fingerprint patterns for biometric and forensic purposes in the developed world. However in Ghana, although emphasis is placed on the face in the photographic recognition systems used in the issuance of passports, very litt...
#1Allan BayatH-Index: 7
#2Alexej Knaus (University Hospital Bonn)H-Index: 5
Last. Rikke S. Møller (University of Southern Denmark)H-Index: 36
view all 27 authors...
To provide a detailed electroclinical description and expand the phenotype of PIGT-CDG, to perform genotype–phenotype correlation, and to investigate the onset and severity of the epilepsy associated with the different genetic subtypes of this rare disorder. Furthermore, to use computer-assisted facial gestalt analysis in PIGT-CDG and to the compare findings with other glycosylphosphatidylinositol (GPI) anchor deficiencies. We evaluated 13 children from eight unrelated families with homozygous o...
3 CitationsSource
Introduction Phosphomannomutase-2 deficiency (PMM2-CDG) is associated with a recognisable facial pattern. There are no early severity predictors for this disorder and no phenotype–genotype correlation. We performed a detailed dysmorphology evaluation to describe facial gestalt and its changes over time, to train digital recognition facial analysis tools and to identify early severity predictors. Methods Paediatric PMM2-CDG patients were evaluated and compared with controls. A computer-assisted r...
4 CitationsSource
Hintergrund und Methoden Durch den Einsatz von Hochdurchsatz-Sequenziertechnologie kann bei der Mehrheit von Patienten mit Intelligenzminderung (ID) eine molekulare Ursache gefunden werden. Fur die Integration der Ganzgenomsequenzierung in die Regelversorgung ist der Einsatz effektiver Filter- und Priorisierungsverfahren unerlasslich, um die Datenmengen effizient sichten zu konnen. Entscheidend ist dabei die Kommunikation zwischen Klinik und Labor, die die Kombination von phanotypischer und mole...
1 CitationsSource
#1Tzung-Chien Hsieh (University of Bonn)H-Index: 1
#2Martin A. Mensah (Charité)H-Index: 3
Last. Peter M. Krawitz (University of Bonn)H-Index: 2
view all 4 authors...
Phenotype information is crucial for the interpretation of genomic variants. So far it has only been accessible for bioinformatics workflows after encoding into clinical terms by expert dysmorphologists. Here, we introduce an approach, driven by artificial intelligence that uses portrait photographs for the interpretation of clinical exome data. We measured the value added by computer-assisted image analysis to the diagnostic yield on a cohort consisting of 679 individuals with 105 different mon...
1 CitationsSource