Advertisement

Recommendations for next generation sequencing data reanalysis of unsolved cases with suspected Mendelian disorders: A systematic review and meta-analysis

  • Pei Dai
    Affiliations
    Precision Immunology Program, Garvan Institute of Medical Research, Sydney, New South Wales, Australia

    St Vincent’s Clinical School, Faculty of Medicine, UNSW Sydney, Sydney, New South Wales, Australia

    Clinical Immunogenomics Research Consortium Australasia (CIRCA), Garvan Institute of Medical Research, Sydney, New South Wales, Australia
    Search for articles by this author
  • Andrew Honda
    Affiliations
    The Kinghorn Centre for Clinical Genomics, Garvan Institute of Medical Research, Sydney, New South Wales, Australia
    Search for articles by this author
  • Lisa Ewans
    Affiliations
    Department of Clinical Genetics, Royal Prince Alfred Hospital, Sydney, New South Wales, Australia
    Search for articles by this author
  • Julie McGaughran
    Affiliations
    Genetic Health Queensland, Royal Brisbane and Women's Hospital, Brisbane, Queensland, Australia
    Search for articles by this author
  • Leslie Burnett
    Affiliations
    Precision Immunology Program, Garvan Institute of Medical Research, Sydney, New South Wales, Australia

    St Vincent’s Clinical School, Faculty of Medicine, UNSW Sydney, Sydney, New South Wales, Australia

    Clinical Immunogenomics Research Consortium Australasia (CIRCA), Garvan Institute of Medical Research, Sydney, New South Wales, Australia

    The Kinghorn Centre for Clinical Genomics, Garvan Institute of Medical Research, Sydney, New South Wales, Australia

    Genetic Medicine Program, Northern Clinical School, Faculty of Medicine and Health, University of Sydney, Sydney, New South Wales, Australia
    Search for articles by this author
  • Matthew Law
    Affiliations
    Biostatistics and Databases Program, The Kirby Institute, UNSW Sydney, Sydney, New South Wales, Australia
    Search for articles by this author
  • Tri Giang Phan
    Correspondence
    Correspondence and requests for materials should be addressed to Tri Giang Phan, Precision Immunology Program, Garvan Institute of Medical Research, 384 Victoria St, Darlinghurst, New South Wales, 2010, Australia
    Affiliations
    Precision Immunology Program, Garvan Institute of Medical Research, Sydney, New South Wales, Australia

    St Vincent’s Clinical School, Faculty of Medicine, UNSW Sydney, Sydney, New South Wales, Australia

    Clinical Immunogenomics Research Consortium Australasia (CIRCA), Garvan Institute of Medical Research, Sydney, New South Wales, Australia
    Search for articles by this author
Open AccessPublished:May 13, 2022DOI:https://doi.org/10.1016/j.gim.2022.04.021

      Abstract

      Purpose

      The study aimed to determine the diagnostic yield, optimal timing, and methodology of next generation sequencing data reanalysis in suspected Mendelian disorders.

      Methods

      We conducted a systematic review and meta-analysis of studies that conducted data reanalysis in patients with suspected Mendelian disorders. Random effects model was used to pool the estimated outcome with subgroup analysis stratified by timing, sequencing methodology, sample size, segregation, use of research validation, and artificial intelligence (AI) variant curation tools.

      Results

      A search of PubMed, Embase, Scopus, and Web of Science between 2007 and 2021 yielded 9327 articles, of which 29 were selected. Significant heterogeneity was noted between studies. Reanalysis had an overall diagnostic yield of 0.10 (95% CI = 0.06-0.13). Literature updates accounted for most new diagnoses. Diagnostic yield was higher after 24 months, although this was not statistically significant. Increased diagnoses were obtained with research validation and data sharing. AI-based tools did not adversely affect reanalysis diagnostic rate.

      Conclusion

      Next generation sequencing data reanalysis can improve diagnostic yield. Owing to the heterogeneity of the studies, the optimal time to reanalysis and the impact of AI-based tools could not be determined with confidence. We propose standardized guidelines for future studies to reduce heterogeneity and improve the quality of the conclusions.

      Keywords

      GenePod

      July 6, 2022

      July 2022: Recommendations for next generation sequencing data reanalysis of unsolved cases with suspected Mendelian disorders: A systematic review and meta-analysis

      Diagnostic yield, optimal timing, and methodology of next generation sequencing data reanalysis.

      Next generation sequencing has becoming increasingly powerful in diagnosing Mendelian disorders, yet typically more than 50 percent of cases remain unsolved after an initial clinical exome or clinical genome sequencing. 

      Loading ...

      Introduction

      Next generation sequencing (NGS) technologies, such as clinical exome sequencing (cES) and clinical genome sequencing (cGS) has revolutionized the diagnosis of Mendelian disorders at single-nucleotide resolution.
      • Katsanis S.H.
      • Katsanis N.
      Molecular genetic testing and the future of clinical genomics.
      It allows the efficient testing for known disease-causing genes and the identification of potentially novel genes and gene variants, leading to significant improvements in cost, diagnostic yield, and time to diagnosis.
      • Wang J.
      • Gotway G.
      • Pascual J.M.
      • Park J.Y.
      Diagnostic yield of clinical next-generation sequencing panels for epilepsy.
      However, >50% of cases typically remain unsolved after initial NGS testing.
      • Lee H.
      • Deignan J.L.
      • Dorrani N.
      • et al.
      Clinical exome sequencing for genetic identification of rare Mendelian disorders.
      ,
      • Phan T.G.
      • Gray P.E.
      • Wong M.
      • et al.
      The Clinical Immunogenomics Research Consortium Australasia (CIRCA): a distributed network model for genomic healthcare delivery.
      In these undiagnosed patients, there is a need to re-examine the genomic data; as new disease-causing genes are discovered, new data analysis tools and bioinformatic pipelines are developed and new clinical features emerge that require review.
      • Sarmady M.
      • Abou Tayoun A.
      Need for automated interactive genomic interpretation and ongoing reanalysis.
      This distinguishes NGS from other pathology tests in that genomic interpretation, and consequently the results of the test, will vary as new knowledge is acquired.
      The periodic reanalysis of unsolved NGS data has been proposed as a solution to meet this need and improve the diagnostic yield.
      • Sarmady M.
      • Abou Tayoun A.
      Need for automated interactive genomic interpretation and ongoing reanalysis.
      The provision of a genetic diagnosis in a previously unsolved case can have significant effects on patient management and outcomes.
      • Tan T.Y.
      • Dillon O.J.
      • Stark Z.
      • et al.
      Diagnostic impact and cost-effectiveness of whole-exome sequencing for ambulant children with suspected monogenic conditions.
      However, variant curation is a labor-intensive, time-consuming process, and genomic reanalysis and clinical interpretation represents a significant time and financial commitment from each laboratory and clinical team.
      • Sarmady M.
      • Abou Tayoun A.
      Need for automated interactive genomic interpretation and ongoing reanalysis.
      Although the American College of Medical Genetics and Genomics (ACMG) have published points to consider for data reanalysis,
      • Deignan J.L.
      • Chung W.K.
      • Kearney H.M.
      • et al.
      Points to consider in the reevaluation and reanalysis of genomic test results: a statement of the American College of Medical Genetics and Genomics (ACMG).
      there remain multiple unanswered questions including the likely diagnostic yield, clinical utility, optimal timing, and optimal method for reanalysis. The effect of future technologies, such as artificial intelligence (AI) and automation have also not been systematically evaluated in this regard.
      • Sarmady M.
      • Abou Tayoun A.
      Need for automated interactive genomic interpretation and ongoing reanalysis.
      ,
      • Ji J.
      • Leung M.L.
      • Baker S.
      • Deignan J.L.
      • Santani A.
      Clinical exome reanalysis: current practice and beyond.
      There have been numerous studies on genomic reanalysis. These studies were diverse with inconsistent adherence to standard published variant interpretation guidelines such as the consensus recommendations from the ACMG and Association for Molecular Pathology (AMP) for variant classification.
      • Richards S.
      • Aziz N.
      • Bale S.
      • et al.
      Standards and guidelines for the interpretation of sequence variants: a joint consensus recommendation of the American College of Medical Genetics and Genomics and the Association for Molecular Pathology.
      The studies also frequently differed in the definition of what constitutes a positive diagnosis. In addition, there has been 1 notable systematic review of 27 published studies on NGS data reanalysis.
      • Tan N.B.
      • Stapleton R.
      • Stark Z.
      • et al.
      Evaluating systematic reanalysis of clinical genomic data in rare disease from single center experience and literature review.
      With this foundation, we undertook a systematic review and meta-analysis of the literature on genomic reanalysis. We identified 29 published studies in which NGS data reanalysis had been performed on 9419 undiagnosed patients with suspected Mendelian diseases. We identified an overall yield for reanalysis of 10% in previously undiagnosed cases with an approximate median time between initial analysis and subsequent reanalysis of approximately 2 years among all studies. Increased diagnostic yield was associated with external data sharing and/or functional validation of novel candidate genes and variants and reanalysis performed 36 or more months after original analysis. We also identified a role for AI-based tools in variant filtration and curation in improving the speed of analysis.

      Materials and Methods

      We conducted a systematic review and meta-analysis in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.
      • Page M.J.
      • McKenzie J.E.
      • Bossuyt P.M.
      • et al.
      The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.
      Two authors (P.D. and A.H.) independently conducted the search for eligible studies, performance of data extraction and certainty assessment with a third author (L.B.) acting as adjudicator for any disagreements that could not be resolved through discussion.

      Search strategy and study selection

      We searched MEDLINE, Embase, PubMed, Scopus, and Web of Science from the period January 1, 2007 to October 14, 2021, restricting the search to publications in the English language. The complete search strategy is shown in Supplemental Table 1. We further manually examined reference lists from short-listed full text articles and searched for any publications resulting from conference abstracts for relevant articles.

      Eligibility criteria

      Studies were eligible for inclusion if they met the following criteria: (1) is a cohort study that included patients with suspected Mendelian disorders who had previously undergone cES or cGS without a molecular diagnosis being reached, (2) has performed reanalysis of the NGS data from these patients, and (3) has reported the yield of new molecular diagnoses after reanalysis. We define reanalysis as bioinformatic examination of the original sequencing data, ie, FASTQ, BAM, and/or variant call format files. This ranges from repeated bioinformatic processing (eg, repeat alignment and variant calling from FASTQ files) to application of novel bioinformatic tools, repeat variant curation, and examination for additional classes of variants (eg, structural variant calling). Studies were excluded if (1) the patient cohort underwent resequencing of the DNA before reanalysis (eg, cES to cGS), (2) they did not use the standard clinical 100× cES or 30× cGS backbone in the original analysis (eg, used selective gene panel-based library preparation or low-depth cGS), (3) they are case reports or include a cohort with fewer than 20 unsolved patients, (4) they did not provide sufficient information to determine diagnostic yield, and (5) they are conference abstracts.

      Outcome measures

      The primary outcome is the proportion of cases without a molecular diagnosis after initial sequencing that subsequently reached a diagnosis upon reanalysis. A case is defined as an affected proband, either sequenced and analyzed as a singleton, or together with family members who may or may not be affected. A molecular diagnosis is defined as either the discovery of a genetic variant, such as single-nucleotide variant, indel variant, or copy number variation (CNV) in a known disease-causing gene that explains the patient’s phenotype in whom the variant is in the correct zygosity (if available) and either can be classified as pathogenic/likely pathogenic (class 4/5) by the ACMG/AMP criteria, or in cases of variants of uncertain significance (VUS) or variants in novel genes, the pathogenicity has been determined by further functional and/or segregation studies leading to peer-reviewed publications or publications that are in progress. In cases of CNVs, the change occurs in gene(s) associated with the patient’s phenotype in whom gene dosage sensitivity is a known mechanism of disease.
      In studies that reported yield as the sum of both ACMG/AMP class 4/5 variants and unvalidated candidate variants, we have selected the former to calculate our diagnostic rate for the purpose of analysis. Similarly, in studies that reported diagnostic rates resulting from both pure data reanalysis and other laboratory methods such as repeat sequencing and microarrays, we have extracted only the yield from reanalysis for data synthesis. In studies that did not report the ACMG/AMP classification for their diagnostic variants in known disease-causing genes, we used the yield as reported by the study for the purpose of our data synthesis. Because some studies reanalyzed both diagnosed and undiagnosed cases after initial sequencing and analysis, we have excluded cases that were initially diagnosed and subsequently either were reclassified or reached additional diagnoses after reanalysis. In 1 study that reanalyzed the same cohort of patients at both 12 and 24 months post original analysis,
      • Nambot S.
      • Thevenon J.
      • Kuentz P.
      • et al.
      Clinical whole-exome sequencing for the diagnosis of rare disorders with congenital anomalies and/or intellectual disability: substantial interest of prospective annual reanalysis.
      we took as the diagnostic yield at 24 month time point the original yield at 12 month time point plus the additional diagnoses made at the 24 month time point, because we would have expected the latter analysis to have picked up the diagnoses made at the 12 month time point if reanalysis had not been performed 1 year earlier.

      Data extraction

      The following data were extracted from each eligible study into a Microsoft Excel spreadsheet: study characteristics (name of authors, year of publication, country, name of journal), demographics of population for reanalysis (age, sex), clinical indication for sequencing, timing between original analysis and reanalysis, sequencing methodology (proband vs family sequencing, exome sequencing [ES] vs genome sequencing [GS], sequencing platform, reference genome build, bioinformatic tools for alignment, variant calling, annotation), variant curation strategies, yield from initial sequencing in the original cohort (if available), total number of undiagnosed patients for reanalysis, number of patients with positive diagnosis on reanalysis, number of cases with new VUS and/or variants in unvalidated candidate genes, primary reason for each case of positive molecular diagnosis, and impact on management when available.
      Variant curation was defined in this case as the analytical process involved in determining whether a variant is believed to be associated with the patient’s clinical presentation. This includes initial variant filtration on the basis of population allele frequency and analysis of other characteristics such as variant type, eg, missense vs indel, in silico predictors of pathogenicity, and genotype–phenotype correlation.

      Quality assessment

      Most existing tools for appraising diagnostic studies eg, QUADAS-2
      • Whiting P.F.
      • Rutjes A.W.
      • Westwood M.E.
      • et al.
      QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies.
      were designed for studies that compare the index test to a reference standard, which is not applicable in this study. Therefore, we adopted a check list derived from the 2015 Standards for Reporting of Diagnostic Accuracy criteria
      • Bossuyt P.M.
      • Reitsma J.B.
      • Bruns D.E.
      • et al.
      STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies.
      to assess for possible bias and completeness of reporting (Supplemental Table 2A). Articles were scored on 19 items covering patient eligibility and selection, test protocols, reporting of results, and study limitations. Funnel plots and Egger’s method
      • Egger M.
      • Davey Smith G.
      • Schneider M.
      • Minder C.
      Bias in meta-analysis detected by a simple, graphical test.
      were used to assess for potential publication bias.

      Data synthesis

      The primary measure of efficacy of reanalysis is the proportion of undiagnosed patients reaching a positive diagnosis on reanalysis after first round sequencing and analysis. A random-effects model with the Clopper-Pearson confidence limits for binomial proportions and Freeman-Tukey double arcsine transformation was used to obtained pooled estimate and 95% CI of the primary outcome measure across all studies. Results were summarized using forest plots. Heterogeneity was interrogated with the Cochran Q test and the I2 statistic (values of 25%, 50%, and 75% are considered to represent low, medium, and high heterogeneity, respectively).
      To examine potential factors influencing the outcome of reanalysis and heterogeneity, we performed further subgroup analyses on the basis of time interval between the original analysis and reanalysis (<24 months compared with ≥24 months), sequencing methodology (ES vs GS), study sample size (<50, 50-100, >100 patients), sequencing of family members for segregation analysis, whether research validation of novel variants/genes were conducted, and whether any AI-based tools (eg, Diploid Moon) were used in variant curation. In studies that did not provide individual patient information on the timing of reanalysis, we used as a surrogate the time interval for 50% or more of patients in the cohort.
      Studies that did not provide enough information for data extraction were excluded from subsequent subgroup analysis. Studies in which the patient data spanned >1 subgroup, their results were split as per subgroup classification for analysis. All meta-analyses were performed using STATA version 17.0 (StataCorp) within the metaprop package.
      • Nyaga V.N.
      • Arbyn M.
      • Aerts M.
      Metaprop: a Stata command to perform meta-analysis of binomial data.
      For patients with a new diagnosis from reanalysis, we also extracted the primary reason responsible for diagnosis for each case. We excluded studies that did not provide a breakdown of reasons for new diagnoses. In studies that reanalyzed both diagnosed and undiagnosed cases from initial sequencing and analysis and provided case breakdown of reasons for new diagnosis in both originally diagnosed and undiagnosed cases, we excluded the studies in which we were unable to determine the breakdown of reasons for new diagnosis from the originally undiagnosed cases only.

      Certainty of assessment

      The certainty of evidence was considered in the context of the 5 Grading of Recommendations Assessment, Development and Evaluation (GRADE): study design, limitations, indirectness, consistency/heterogeneity, precision, and publication bias, with modifications for diagnostic tests.
      • Schünemann H.J.
      • Oxman A.D.
      • Brozek J.
      • et al.
      Grading quality of evidence and strength of recommendations for diagnostic tests and strategies.

      Sensitivity analysis

      We performed sensitivity analysis to determine the risk of bias and impact of including low-quality studies on the overall estimated diagnostic yield.

      Protocol registration and subsequent amendments

      The review protocol was originally registered in the PROSPERO international prospective register of systematic reviews (registration number CRD42021254519) on June 11, 2021. Compared with the original protocol, the following modifications were made in this study: (1) modification of the review title; (2) modification of the review team members; (3) extension of the review question to other factors that can influence diagnostic yield beyond sequencing modality; (4) extension of the timing subgroup analysis to <24 months and ≥24 months because we thought this is more relevant to real world clinical practice; (5) removal of clinical indication from subgroup analysis owing to lack of information provided in the studies; (6) addition of subgroup analysis based on the presence of research validation of novel variants/genes, study size, proband vs family based study, and presence of software automation owing to the clinical relevance of these elements; and (7) addition of sensitivity analysis owing to the significant heterogeneity noted in the studies’ methodology.

      Results

      Study selection

      A total of 9327 abstracts were identified from the initial literature search (Figure 1). After removal of duplicates, 4545 unique abstracts were reviewed. An additional 4485 records were excluded because they were nonhuman studies, review articles, conference abstracts, or studies that did not involve NGS reanalysis of unsolved patients with suspected Mendelian disease. Full text of 60 abstracts were reviewed with 28 articles being found eligible for data synthesis. We furthered reviewed 4 additional articles discovered through citation search of the reviewed full text articles as well as 1 article found via manual search for published literature associated with conference abstracts. We subsequently selected 1 more article leading to a total of 29 articles for data synthesis.
      Figure thumbnail gr1
      Figure 1Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram for article selection.
      Reasons for exclusion of articles selected for full text analysis (Supplemental Table 3) included those that performed reanalysis after laboratory resequencing, review/commentary articles, case reports and/or cohorts less than 20 patients, incomplete reporting of outcomes, reanalysis of gene panels, performance of low-depth GS, insufficient clinical details to establish genotype–phenotype associations, and extension of the initial analysis rather than conducting reanalysis on unsolved cases. One cohort of patients was separately analyzed in 2 published studies.
      • Nambot S.
      • Thevenon J.
      • Kuentz P.
      • et al.
      Clinical whole-exome sequencing for the diagnosis of rare disorders with congenital anomalies and/or intellectual disability: substantial interest of prospective annual reanalysis.
      ,
      • Bruel A.L.
      • Nambot S.
      • Quéré V.
      • et al.
      Increased diagnostic and new genes identification outcome using research reanalysis of singleton exome sequencing.
      Because these studies used independent reanalysis pipelines and had additional unique cohorts, we have treated them as 2 separate study populations.

      Characteristics of included studies and patients

      The 29 studies included in the analysis
      • Tan N.B.
      • Stapleton R.
      • Stark Z.
      • et al.
      Evaluating systematic reanalysis of clinical genomic data in rare disease from single center experience and literature review.
      ,
      • Nambot S.
      • Thevenon J.
      • Kuentz P.
      • et al.
      Clinical whole-exome sequencing for the diagnosis of rare disorders with congenital anomalies and/or intellectual disability: substantial interest of prospective annual reanalysis.
      ,
      • Bruel A.L.
      • Nambot S.
      • Quéré V.
      • et al.
      Increased diagnostic and new genes identification outcome using research reanalysis of singleton exome sequencing.
      • Al-Nabhani M.
      • Al-Rashdi S.
      • Al-Murshedi F.
      • et al.
      Reanalysis of exome sequencing data of intellectual disability samples: yields and benefits.
      • Alsubaie L.
      • Alturki S.
      • Alothaim A.
      • Alfares A.
      Clinical reassessment of post-laboratory variant call format (VCF) files.
      • Baker S.W.
      • Murrell J.R.
      • Nesbitt A.I.
      • et al.
      Automated clinical exome reanalysis reveals novel diagnoses.
      • Genetics Initiative Epilepsy
      The Epilepsy Genetics Initiative: systematic reanalysis of diagnostic exomes increases yield.
      • Bowling K.M.
      • Thompson M.L.
      • Amaral M.D.
      • et al.
      Genomic diagnosis for children with intellectual disability and/or developmental delay.
      • Brunet T.
      • Jech R.
      • Brugger M.
      • et al.
      De novo variants in neurodevelopmental disorders-experiences from a tertiary care center.
      • Costain G.
      • Jobling R.
      • Walker S.
      • et al.
      Periodic reanalysis of whole-genome sequencing data enhances the diagnostic advantage over standard clinical genetic testing.
      • Eldomery M.K.
      • Coban-Akdemir Z.
      • Harel T.
      • et al.
      Lessons learned from additional research analyses of unsolved clinical exome cases.
      • Ewans L.J.
      • Schofield D.
      • Shrestha R.
      • et al.
      Whole-exome sequencing reanalysis at 12 months boosts diagnosis and is cost-effective when applied early in Mendelian disorders.
      • Jalkh N.
      • Corbani S.
      • Haidar Z.
      • et al.
      The added value of WES reanalysis in the field of genetic diagnosis: lessons learned from 200 exomes in the Lebanese population.
      • James K.N.
      • Clark M.M.
      • Camp B.
      • et al.
      Partially automated whole-genome sequencing reanalysis of previously undiagnosed pediatric patients can efficiently yield new diagnoses.
      • Lassmann T.
      • Francis R.W.
      • Weeks A.
      • et al.
      A flexible computational pipeline for research analyses of unsolved clinical exome cases.
      • Li J.
      • Gao K.
      • Yan H.
      • et al.
      Reanalysis of whole exome sequencing data in patients with epilepsy and intellectual disability/mental retardation.
      • Liu P.
      • Meng L.
      • Normand E.A.
      • et al.
      Reanalysis of clinical exome sequencing data.
      • Liu Y.
      • Teng Y.
      • Li Z.
      • Cui J.
      • Liang D.
      • Wu L.
      Increase in diagnostic yield achieved for 174 whole-exome sequencing cases reanalyzed 1-2 years after initial analysis.
      • Machini K.
      • Ceyhan-Birsoy O.
      • Azzariti D.R.
      • et al.
      Analyzing and reanalyzing the genome: findings from the MedSeq project.
      • Matalonga L.
      • Hernández-Ferrer C.
      • Piscia D.
      • et al.
      Solving patients with rare diseases through programmatic reanalysis of genome-phenome data.
      • Mitani T.
      • Isikay S.
      • Gezdirici A.
      • et al.
      High prevalence of multilocus pathogenic variation in neurodevelopmental disorders in the Turkish population.
      • Ngo K.J.
      • Rexach J.E.
      • Lee H.
      • et al.
      A diagnostic ceiling for exome sequencing in cerebellar ataxia and related neurological disorders.
      • Salfati E.L.
      • Spencer E.G.
      • Topol S.E.
      • et al.
      Re-analysis of whole-exome sequencing data uncovers novel diagnostic variants and improves molecular diagnostic yields for sudden death and idiopathic diseases.
      • Basel-Salmon L.
      • Orenstein N.
      • Markus-Bustani K.
      • et al.
      Improved diagnostics by exome sequencing following raw data reevaluation by clinical geneticists involved in the medical care of the individuals tested.
      • Schmitz-Abe K.
      • Li Q.
      • Rosen S.M.
      • et al.
      Unique bioinformatic approach and comprehensive reanalysis improve diagnostic yield of clinical exomes.
      • Shashi V.
      • Schoch K.
      • Spillmann R.
      • et al.
      A comprehensive iterative approach is highly effective in diagnosing individuals who are exome negative.
      • Sundercombe S.L.
      • Berbic M.
      • Evans C.A.
      • et al.
      Clinically responsive genomic analysis pipelines: elements to improve detection rate and efficiency.
      • Wenger A.M.
      • Guturu H.
      • Bernstein J.A.
      • Bejerano G.
      Systematic reanalysis of clinical exome data yields additional diagnoses: implications for providers.
      • Wright C.F.
      • McRae J.F.
      • Clayton S.
      • et al.
      Making new genetic diagnoses with old data: iterative reanalysis and reporting from genome-wide data in 1,133 families with developmental disorders.
      encompassed a total of 9419 patients with suspected Mendelian disorders that did not reach a diagnosis (Supplemental Table 2B). The studies originated from the United States, Europe, China, Australia, and the Middle East. Most studies (19/29) had a predominance of pediatric patients (n = 4430) with 1 study in adults only (n = 60), and the rest (9/29) not providing an age breakdown of their reanalyzed patients (n = 4989). There were 3 studies that performed reanalysis of cGS data (n = 146), 3 studies that performed both cGS and cES (n = 4702), and 23 studies that performed reanalysis of cES data (n = 4571). There were 10 studies that did not provide any information on timing of reanalysis (n = 5085) and 3 studies stating that reanalysis was conducted later than 12 months from original analysis only (n = 390) without providing further data. Among the remaining 16 studies (n = 3944), 50% had a median time between original analysis and reanalysis of <24 months.
      Most studies examined either singleton-only (13/29 studies; n = 2791) or singletons, trios, and other mixed family compositions (12/29 studies; n = 5611) reanalysis with 3 studies having trios only (n = 978) and 1 study (n = 39) not reporting its composition. There were 7 studies (n = 5040) that did not report on their sequencing methodology with most of the remaining studies (20/22) using an Illumina-based platform (n = 4170). Of the 29 studies, 24 used a Genome Analysis Toolkit-based pipeline
      • McKenna A.
      • Hanna M.
      • Banks E.
      • et al.
      The Genome Analysis Toolkit: a MapReduce framework for analyzing next-generation DNA sequencing data.
      for variant calling with a mixture of different software packages for variant annotation. Variant curation was predominantly limited to single-nucleotide variants and indels in known disease-causing genes, with 9 studies (n = 3202) also examining CNVs. There were 8 studies (n = 5259) that used semiautomation/AI-based software in variant interpretation, with the rest conducting manual variant curation.

      Generation of a standardized reanalysis data set

      To generate a standardized data set, we reviewed the original data presented in the text, tables, and supplementary information to determine the diagnostic yield by applying strict ACMG/AMP criteria. We reclassified the data to exclude diagnoses from the use of other methods (such as microarrays
      • Tan N.B.
      • Stapleton R.
      • Stark Z.
      • et al.
      Evaluating systematic reanalysis of clinical genomic data in rare disease from single center experience and literature review.
      ,
      • Liu Y.
      • Teng Y.
      • Li Z.
      • Cui J.
      • Liang D.
      • Wu L.
      Increase in diagnostic yield achieved for 174 whole-exome sequencing cases reanalyzed 1-2 years after initial analysis.
      ), repeat NGS sequencing,
      • Tan N.B.
      • Stapleton R.
      • Stark Z.
      • et al.
      Evaluating systematic reanalysis of clinical genomic data in rare disease from single center experience and literature review.
      ,
      • Shashi V.
      • Schoch K.
      • Spillmann R.
      • et al.
      A comprehensive iterative approach is highly effective in diagnosing individuals who are exome negative.
      secondary diagnoses,
      • Liu Y.
      • Teng Y.
      • Li Z.
      • Cui J.
      • Liang D.
      • Wu L.
      Increase in diagnostic yield achieved for 174 whole-exome sequencing cases reanalyzed 1-2 years after initial analysis.
      ,
      • Schmitz-Abe K.
      • Li Q.
      • Rosen S.M.
      • et al.
      Unique bioinformatic approach and comprehensive reanalysis improve diagnostic yield of clinical exomes.
       unvalidated VUS and/or novel genes,
      • Tan N.B.
      • Stapleton R.
      • Stark Z.
      • et al.
      Evaluating systematic reanalysis of clinical genomic data in rare disease from single center experience and literature review.
      ,
      • Al-Nabhani M.
      • Al-Rashdi S.
      • Al-Murshedi F.
      • et al.
      Reanalysis of exome sequencing data of intellectual disability samples: yields and benefits.
      • Alsubaie L.
      • Alturki S.
      • Alothaim A.
      • Alfares A.
      Clinical reassessment of post-laboratory variant call format (VCF) files.
      • Baker S.W.
      • Murrell J.R.
      • Nesbitt A.I.
      • et al.
      Automated clinical exome reanalysis reveals novel diagnoses.
      ,
      • Costain G.
      • Jobling R.
      • Walker S.
      • et al.
      Periodic reanalysis of whole-genome sequencing data enhances the diagnostic advantage over standard clinical genetic testing.
      ,
      • Eldomery M.K.
      • Coban-Akdemir Z.
      • Harel T.
      • et al.
      Lessons learned from additional research analyses of unsolved clinical exome cases.
      ,
      • Lassmann T.
      • Francis R.W.
      • Weeks A.
      • et al.
      A flexible computational pipeline for research analyses of unsolved clinical exome cases.
      ,
      • Mitani T.
      • Isikay S.
      • Gezdirici A.
      • et al.
      High prevalence of multilocus pathogenic variation in neurodevelopmental disorders in the Turkish population.
      ,
      • Basel-Salmon L.
      • Orenstein N.
      • Markus-Bustani K.
      • et al.
      Improved diagnostics by exome sequencing following raw data reevaluation by clinical geneticists involved in the medical care of the individuals tested.
      • Schmitz-Abe K.
      • Li Q.
      • Rosen S.M.
      • et al.
      Unique bioinformatic approach and comprehensive reanalysis improve diagnostic yield of clinical exomes.
      • Shashi V.
      • Schoch K.
      • Spillmann R.
      • et al.
      A comprehensive iterative approach is highly effective in diagnosing individuals who are exome negative.
      incorrect zygosity,
      • Alsubaie L.
      • Alturki S.
      • Alothaim A.
      • Alfares A.
      Clinical reassessment of post-laboratory variant call format (VCF) files.
      miscounting of 2 causative variants as 2 diagnoses in 1 patient,
      • James K.N.
      • Clark M.M.
      • Camp B.
      • et al.
      Partially automated whole-genome sequencing reanalysis of previously undiagnosed pediatric patients can efficiently yield new diagnoses.
      misclassification of a clinically irrelevant variant as pathogenic on the basis of ACMG/AMP criteria,
      • Lassmann T.
      • Francis R.W.
      • Weeks A.
      • et al.
      A flexible computational pipeline for research analyses of unsolved clinical exome cases.
      and miscounting of 2 class 4/5 variants in the supplemental table.
      • Machini K.
      • Ceyhan-Birsoy O.
      • Azzariti D.R.
      • et al.
      Analyzing and reanalyzing the genome: findings from the MedSeq project.
      In total, this resulted in 14 out of 29 studies
      • Tan N.B.
      • Stapleton R.
      • Stark Z.
      • et al.
      Evaluating systematic reanalysis of clinical genomic data in rare disease from single center experience and literature review.
      ,
      • Al-Nabhani M.
      • Al-Rashdi S.
      • Al-Murshedi F.
      • et al.
      Reanalysis of exome sequencing data of intellectual disability samples: yields and benefits.
      • Alsubaie L.
      • Alturki S.
      • Alothaim A.
      • Alfares A.
      Clinical reassessment of post-laboratory variant call format (VCF) files.
      • Baker S.W.
      • Murrell J.R.
      • Nesbitt A.I.
      • et al.
      Automated clinical exome reanalysis reveals novel diagnoses.
      ,
      • Costain G.
      • Jobling R.
      • Walker S.
      • et al.
      Periodic reanalysis of whole-genome sequencing data enhances the diagnostic advantage over standard clinical genetic testing.
      ,
      • Eldomery M.K.
      • Coban-Akdemir Z.
      • Harel T.
      • et al.
      Lessons learned from additional research analyses of unsolved clinical exome cases.
      ,
      • James K.N.
      • Clark M.M.
      • Camp B.
      • et al.
      Partially automated whole-genome sequencing reanalysis of previously undiagnosed pediatric patients can efficiently yield new diagnoses.
      ,
      • Lassmann T.
      • Francis R.W.
      • Weeks A.
      • et al.
      A flexible computational pipeline for research analyses of unsolved clinical exome cases.
      ,
      • Liu Y.
      • Teng Y.
      • Li Z.
      • Cui J.
      • Liang D.
      • Wu L.
      Increase in diagnostic yield achieved for 174 whole-exome sequencing cases reanalyzed 1-2 years after initial analysis.
      ,
      • Machini K.
      • Ceyhan-Birsoy O.
      • Azzariti D.R.
      • et al.
      Analyzing and reanalyzing the genome: findings from the MedSeq project.
      ,
      • Mitani T.
      • Isikay S.
      • Gezdirici A.
      • et al.
      High prevalence of multilocus pathogenic variation in neurodevelopmental disorders in the Turkish population.
      ,
      • Basel-Salmon L.
      • Orenstein N.
      • Markus-Bustani K.
      • et al.
      Improved diagnostics by exome sequencing following raw data reevaluation by clinical geneticists involved in the medical care of the individuals tested.
      • Schmitz-Abe K.
      • Li Q.
      • Rosen S.M.
      • et al.
      Unique bioinformatic approach and comprehensive reanalysis improve diagnostic yield of clinical exomes.
      • Shashi V.
      • Schoch K.
      • Spillmann R.
      • et al.
      A comprehensive iterative approach is highly effective in diagnosing individuals who are exome negative.
      being assigned a different diagnostic yield to that reported by the authors (Supplemental Table 2B). These data reinforce the need for standardized reporting of reanalysis studies.

      Quality and bias assessment

      There was significant variation in the quality of studies as assessed by the modified Standards for Reporting of Diagnostic Accuracy checklist. (Supplemental Table 2A). Of the 29 studies, only 10 studies had described specific eligibility criteria for patient entry into original sequencing and analysis. Methodology on bioinformatic processing of the raw data was reported in 28 of the 29 studies with all studies giving a description of variant curation strategies. In addition, 22 out of 29 studies specifically reported methodological differences between original sequencing and analysis and reanalysis.
      Most studies (22 out of 29) did not provide individual clinical information of sequenced probands, eg, diagnosed variant, or timing of reanalysis but instead provided summary data of the overall population. There were only 4 studies that did not provide ACMG/AMP classification for diagnostic variants in known disease-causing genes but most of the studies (22 out of 29) did not provide specific criteria to justify the ACMG/AMP classification.
      • Richards S.
      • Aziz N.
      • Bale S.
      • et al.
      Standards and guidelines for the interpretation of sequence variants: a joint consensus recommendation of the American College of Medical Genetics and Genomics and the Association for Molecular Pathology.
      There were 11 studies that reported the finding of VUS and/or variants in novel genes but only 8 studies provided research evidence confirming their pathogenicity. Only 3 studies discussed the impact of reanalysis on clinical management and only 16 out of 29 articles discussed study limitations. There was no evidence of publication bias as shown by the funnel plot (Supplemental Figure 1) or Egger’s test (P = .89).

      Overall yield and subgroup analysis

      The summarized pool estimated yield from genomic reanalysis among patients without a diagnosis after initial NGS analysis in all studies was 0.10 (95% CI = 0.06-0.13)
      • Tan N.B.
      • Stapleton R.
      • Stark Z.
      • et al.
      Evaluating systematic reanalysis of clinical genomic data in rare disease from single center experience and literature review.
      ,
      • Nambot S.
      • Thevenon J.
      • Kuentz P.
      • et al.
      Clinical whole-exome sequencing for the diagnosis of rare disorders with congenital anomalies and/or intellectual disability: substantial interest of prospective annual reanalysis.
      ,
      • Bruel A.L.
      • Nambot S.
      • Quéré V.
      • et al.
      Increased diagnostic and new genes identification outcome using research reanalysis of singleton exome sequencing.
      • Al-Nabhani M.
      • Al-Rashdi S.
      • Al-Murshedi F.
      • et al.
      Reanalysis of exome sequencing data of intellectual disability samples: yields and benefits.
      • Alsubaie L.
      • Alturki S.
      • Alothaim A.
      • Alfares A.
      Clinical reassessment of post-laboratory variant call format (VCF) files.
      • Baker S.W.
      • Murrell J.R.
      • Nesbitt A.I.
      • et al.
      Automated clinical exome reanalysis reveals novel diagnoses.
      • Genetics Initiative Epilepsy
      The Epilepsy Genetics Initiative: systematic reanalysis of diagnostic exomes increases yield.
      • Bowling K.M.
      • Thompson M.L.
      • Amaral M.D.
      • et al.
      Genomic diagnosis for children with intellectual disability and/or developmental delay.
      • Brunet T.
      • Jech R.
      • Brugger M.
      • et al.
      De novo variants in neurodevelopmental disorders-experiences from a tertiary care center.
      • Costain G.
      • Jobling R.
      • Walker S.
      • et al.
      Periodic reanalysis of whole-genome sequencing data enhances the diagnostic advantage over standard clinical genetic testing.
      • Eldomery M.K.
      • Coban-Akdemir Z.
      • Harel T.
      • et al.
      Lessons learned from additional research analyses of unsolved clinical exome cases.
      • Ewans L.J.
      • Schofield D.
      • Shrestha R.
      • et al.
      Whole-exome sequencing reanalysis at 12 months boosts diagnosis and is cost-effective when applied early in Mendelian disorders.
      • Jalkh N.
      • Corbani S.
      • Haidar Z.
      • et al.
      The added value of WES reanalysis in the field of genetic diagnosis: lessons learned from 200 exomes in the Lebanese population.
      • James K.N.
      • Clark M.M.
      • Camp B.
      • et al.
      Partially automated whole-genome sequencing reanalysis of previously undiagnosed pediatric patients can efficiently yield new diagnoses.
      • Lassmann T.
      • Francis R.W.
      • Weeks A.
      • et al.
      A flexible computational pipeline for research analyses of unsolved clinical exome cases.
      • Li J.
      • Gao K.
      • Yan H.
      • et al.
      Reanalysis of whole exome sequencing data in patients with epilepsy and intellectual disability/mental retardation.
      • Liu P.
      • Meng L.
      • Normand E.A.
      • et al.
      Reanalysis of clinical exome sequencing data.
      • Liu Y.
      • Teng Y.
      • Li Z.
      • Cui J.
      • Liang D.
      • Wu L.
      Increase in diagnostic yield achieved for 174 whole-exome sequencing cases reanalyzed 1-2 years after initial analysis.
      • Machini K.
      • Ceyhan-Birsoy O.
      • Azzariti D.R.
      • et al.
      Analyzing and reanalyzing the genome: findings from the MedSeq project.
      • Matalonga L.
      • Hernández-Ferrer C.
      • Piscia D.
      • et al.
      Solving patients with rare diseases through programmatic reanalysis of genome-phenome data.
      • Mitani T.
      • Isikay S.
      • Gezdirici A.
      • et al.
      High prevalence of multilocus pathogenic variation in neurodevelopmental disorders in the Turkish population.
      • Ngo K.J.
      • Rexach J.E.
      • Lee H.
      • et al.
      A diagnostic ceiling for exome sequencing in cerebellar ataxia and related neurological disorders.
      • Salfati E.L.
      • Spencer E.G.
      • Topol S.E.
      • et al.
      Re-analysis of whole-exome sequencing data uncovers novel diagnostic variants and improves molecular diagnostic yields for sudden death and idiopathic diseases.
      • Basel-Salmon L.
      • Orenstein N.
      • Markus-Bustani K.
      • et al.
      Improved diagnostics by exome sequencing following raw data reevaluation by clinical geneticists involved in the medical care of the individuals tested.
      • Schmitz-Abe K.
      • Li Q.
      • Rosen S.M.
      • et al.
      Unique bioinformatic approach and comprehensive reanalysis improve diagnostic yield of clinical exomes.
      • Shashi V.
      • Schoch K.
      • Spillmann R.
      • et al.
      A comprehensive iterative approach is highly effective in diagnosing individuals who are exome negative.
      • Sundercombe S.L.
      • Berbic M.
      • Evans C.A.
      • et al.
      Clinically responsive genomic analysis pipelines: elements to improve detection rate and efficiency.
      • Wenger A.M.
      • Guturu H.
      • Bernstein J.A.
      • Bejerano G.
      Systematic reanalysis of clinical exome data yields additional diagnoses: implications for providers.
      • Wright C.F.
      • McRae J.F.
      • Clayton S.
      • et al.
      Making new genetic diagnoses with old data: iterative reanalysis and reporting from genome-wide data in 1,133 families with developmental disorders.
      (Figure 2). Considerable heterogeneity was observed between studies (I2 = 95.33%; P < .01). In total, 13 studies were excluded because they either did not provide any information on timing or the timing reported was not sufficiently detailed (eg, 1-7 years) to assign them to a category for the subgroup analysis. Within the remaining studies, there were 2 studies
      • Nambot S.
      • Thevenon J.
      • Kuentz P.
      • et al.
      Clinical whole-exome sequencing for the diagnosis of rare disorders with congenital anomalies and/or intellectual disability: substantial interest of prospective annual reanalysis.
      ,
      • Schmitz-Abe K.
      • Li Q.
      • Rosen S.M.
      • et al.
      Unique bioinformatic approach and comprehensive reanalysis improve diagnostic yield of clinical exomes.
      that provided reanalysis timing for individual patients that allowed the results to be stratified into different time points. The remaining studies were allocated to time points on the basis of median or mean timing of reanalysis. From this subgroup analysis, studies that reanalyzed at ≥24 months after the original analysis had a higher diagnostic yield with a rate of 0.13 (95% CI = 0.09-0.18; I2 = 84%; P = .000) than those that reanalyzed at <24 months with a rate of 0.09 (95% CI = 0.06-0.13; I2 = 66%; P = .000), although this was not statistically significant (Supplemental Figure 2).
      Figure thumbnail gr2
      Figure 2Forest plot of overall diagnostic yield. The GRADE certainty of assessment is scored as moderate-quality evidence. GRADE, Grading of Recommendations Assessment, Development and Evaluation.
      Studies that limited the reanalysis to known disease-causing genes had a lower diagnostic yield of 0.08 (95% CI = 0.4-0.12; I2 = 95.89%; P < .01) compared with a yield of 0.15 (95% CI = 0.09-0.22; I2 = 88.28%; P < .01) for those that also performed validation studies of novel variants and/or genes, although this was not statistically significant (Supplemental Figure 3A).
      There was 1 study
      • Matalonga L.
      • Hernández-Ferrer C.
      • Piscia D.
      • et al.
      Solving patients with rare diseases through programmatic reanalysis of genome-phenome data.
      that used both cES and cGS, but it did not provide yield broken down by sequencing technology and was therefore excluded. Among the remaining 28 studies (Supplemental Figure 3B), there were 2 studies that used both cES and cGS
      • Alsubaie L.
      • Alturki S.
      • Alothaim A.
      • Alfares A.
      Clinical reassessment of post-laboratory variant call format (VCF) files.
      ,
      • Bowling K.M.
      • Thompson M.L.
      • Amaral M.D.
      • et al.
      Genomic diagnosis for children with intellectual disability and/or developmental delay.
      with the rest either reanalyzing either cES or cGS data only. Studies reanalyzing cES had an increased pooled yield of 0.11 (95% CI = 0.08-0.14; I2 = 84.30%; P < .01) compared with a yield of 0.04 (95% CI = 0.01-0.09; I2 = 62.59%; P < .01) in cGS studies, and although the CIs overlap, this was statistically significant with a P value for heterogeneity between groups of .025.
      Size of the study (Supplemental Figure 3C) and the use of familial segregation (Supplemental Figure 3D) did not increase the diagnostic yield of genomic reanalysis. Interestingly, use of AI-based tools in variant curation was associated with a small decrease in diagnostic yield from 0.10 (95% CI = 0.07-0.14; I2 = 86%; P = .00) to 0.08 (95% CI = 0.03-0.15; I2 = 98%; P = .00), which was not statistically significant (Supplemental Figure 3E).

      Reasons for new diagnosis

      The reasons for achieving a diagnosis with reanalysis in originally undiagnosed patients were provided in 23 studies involving 429 patients (Supplemental Figure 4). The major reason was updates in the literature/disease database owing to new gene discovery (62%). In 15% of cases the reasons for new diagnosis were unknown or not specified. Other reasons for reaching a diagnosis included research validation of candidate variants through external collaboration and/or functional studies (11%), bioinformatic pipeline improvements, updated patient clinical phenotypes, additional segregation studies in relatives, and laboratory errors/misinterpretations. A total of 14 new diagnoses were made through bioinformatic improvements: 5 by examining for low-quality variants,
      • Al-Nabhani M.
      • Al-Rashdi S.
      • Al-Murshedi F.
      • et al.
      Reanalysis of exome sequencing data of intellectual disability samples: yields and benefits.
      ,
      • Bowling K.M.
      • Thompson M.L.
      • Amaral M.D.
      • et al.
      Genomic diagnosis for children with intellectual disability and/or developmental delay.
      ,
      • Basel-Salmon L.
      • Orenstein N.
      • Markus-Bustani K.
      • et al.
      Improved diagnostics by exome sequencing following raw data reevaluation by clinical geneticists involved in the medical care of the individuals tested.
      ,
      • Sundercombe S.L.
      • Berbic M.
      • Evans C.A.
      • et al.
      Clinically responsive genomic analysis pipelines: elements to improve detection rate and efficiency.
      4 by relaxation of the population allele frequency filtration,
      • Bowling K.M.
      • Thompson M.L.
      • Amaral M.D.
      • et al.
      Genomic diagnosis for children with intellectual disability and/or developmental delay.
      ,
      • Liu Y.
      • Teng Y.
      • Li Z.
      • Cui J.
      • Liang D.
      • Wu L.
      Increase in diagnostic yield achieved for 174 whole-exome sequencing cases reanalyzed 1-2 years after initial analysis.
      2 by software improvements for better variant calling sensitivity and joint calling,
      • Ewans L.J.
      • Schofield D.
      • Shrestha R.
      • et al.
      Whole-exome sequencing reanalysis at 12 months boosts diagnosis and is cost-effective when applied early in Mendelian disorders.
      ,
      • Schmitz-Abe K.
      • Li Q.
      • Rosen S.M.
      • et al.
      Unique bioinformatic approach and comprehensive reanalysis improve diagnostic yield of clinical exomes.
      and 3 cases by software improvements to include intronic variants and indels of 10 to 50-base pair length
      • Liu Y.
      • Teng Y.
      • Li Z.
      • Cui J.
      • Liang D.
      • Wu L.
      Increase in diagnostic yield achieved for 174 whole-exome sequencing cases reanalyzed 1-2 years after initial analysis.
      (Supplemental Figure 5).

      Influence on management

      Only 3 studies discussed whether a genetic diagnosis led to management changes.
      • Ewans L.J.
      • Schofield D.
      • Shrestha R.
      • et al.
      Whole-exome sequencing reanalysis at 12 months boosts diagnosis and is cost-effective when applied early in Mendelian disorders.
      ,
      • James K.N.
      • Clark M.M.
      • Camp B.
      • et al.
      Partially automated whole-genome sequencing reanalysis of previously undiagnosed pediatric patients can efficiently yield new diagnoses.
      ,
      • Liu P.
      • Meng L.
      • Normand E.A.
      • et al.
      Reanalysis of clinical exome sequencing data.
      In these studies, impact of genetic diagnosis on management was only discussed in a subgroup of patients.

      Sensitivity analysis

      There were 4 studies
      • Genetics Initiative Epilepsy
      The Epilepsy Genetics Initiative: systematic reanalysis of diagnostic exomes increases yield.
      ,
      • Eldomery M.K.
      • Coban-Akdemir Z.
      • Harel T.
      • et al.
      Lessons learned from additional research analyses of unsolved clinical exome cases.
      ,
      • Jalkh N.
      • Corbani S.
      • Haidar Z.
      • et al.
      The added value of WES reanalysis in the field of genetic diagnosis: lessons learned from 200 exomes in the Lebanese population.
      ,
      • Mitani T.
      • Isikay S.
      • Gezdirici A.
      • et al.
      High prevalence of multilocus pathogenic variation in neurodevelopmental disorders in the Turkish population.
      that did not provide ACMG/AMP classification for their candidate variants in known disease-causing genes and an another study
      • Matalonga L.
      • Hernández-Ferrer C.
      • Piscia D.
      • et al.
      Solving patients with rare diseases through programmatic reanalysis of genome-phenome data.
      that only curated variants in ClinVar. Sensitivity analysis excluding these 5 studies did not change the overall pooled estimates of reanalysis efficacy, nor did it significantly alter the heterogeneity between studies (Supplemental Figure 3F). Of the 29 studies, 19 did not report eligibility criteria for recruitment of patients (Supplemental Table 2A). Sensitivity analysis limited to the 10 studies that did report eligibility criteria
      • Nambot S.
      • Thevenon J.
      • Kuentz P.
      • et al.
      Clinical whole-exome sequencing for the diagnosis of rare disorders with congenital anomalies and/or intellectual disability: substantial interest of prospective annual reanalysis.
      ,
      • Bowling K.M.
      • Thompson M.L.
      • Amaral M.D.
      • et al.
      Genomic diagnosis for children with intellectual disability and/or developmental delay.
      • Brunet T.
      • Jech R.
      • Brugger M.
      • et al.
      De novo variants in neurodevelopmental disorders-experiences from a tertiary care center.
      • Costain G.
      • Jobling R.
      • Walker S.
      • et al.
      Periodic reanalysis of whole-genome sequencing data enhances the diagnostic advantage over standard clinical genetic testing.
      ,
      • Ewans L.J.
      • Schofield D.
      • Shrestha R.
      • et al.
      Whole-exome sequencing reanalysis at 12 months boosts diagnosis and is cost-effective when applied early in Mendelian disorders.
      ,
      • Machini K.
      • Ceyhan-Birsoy O.
      • Azzariti D.R.
      • et al.
      Analyzing and reanalyzing the genome: findings from the MedSeq project.
      ,
      • Mitani T.
      • Isikay S.
      • Gezdirici A.
      • et al.
      High prevalence of multilocus pathogenic variation in neurodevelopmental disorders in the Turkish population.
      • Ngo K.J.
      • Rexach J.E.
      • Lee H.
      • et al.
      A diagnostic ceiling for exome sequencing in cerebellar ataxia and related neurological disorders.
      • Salfati E.L.
      • Spencer E.G.
      • Topol S.E.
      • et al.
      Re-analysis of whole-exome sequencing data uncovers novel diagnostic variants and improves molecular diagnostic yields for sudden death and idiopathic diseases.
      ,
      • Wright C.F.
      • McRae J.F.
      • Clayton S.
      • et al.
      Making new genetic diagnoses with old data: iterative reanalysis and reporting from genome-wide data in 1,133 families with developmental disorders.
      ,
      • Yang Y.
      • Muzny D.M.
      • Xia F.
      • et al.
      Molecular findings among patients referred for clinical whole-exome sequencing.
      did not change the overall estimate of efficacy nor heterogeneity (Supplemental Figure 3G).

      Certainty of assessment

      Diagnostic accuracy studies are considered high-quality studies.
      • Schünemann H.J.
      • Oxman A.D.
      • Brozek J.
      • et al.
      Grading quality of evidence and strength of recommendations for diagnostic tests and strategies.
      However, we downgraded the certainty because of statistical heterogeneity between studies that could not be adequately explained by the subgroup analysis. Only 10 of the 29 studies
      • Nambot S.
      • Thevenon J.
      • Kuentz P.
      • et al.
      Clinical whole-exome sequencing for the diagnosis of rare disorders with congenital anomalies and/or intellectual disability: substantial interest of prospective annual reanalysis.
      ,
      • Bowling K.M.
      • Thompson M.L.
      • Amaral M.D.
      • et al.
      Genomic diagnosis for children with intellectual disability and/or developmental delay.
      • Brunet T.
      • Jech R.
      • Brugger M.
      • et al.
      De novo variants in neurodevelopmental disorders-experiences from a tertiary care center.
      • Costain G.
      • Jobling R.
      • Walker S.
      • et al.
      Periodic reanalysis of whole-genome sequencing data enhances the diagnostic advantage over standard clinical genetic testing.
      ,
      • Ewans L.J.
      • Schofield D.
      • Shrestha R.
      • et al.
      Whole-exome sequencing reanalysis at 12 months boosts diagnosis and is cost-effective when applied early in Mendelian disorders.
      ,
      • Machini K.
      • Ceyhan-Birsoy O.
      • Azzariti D.R.
      • et al.
      Analyzing and reanalyzing the genome: findings from the MedSeq project.
      ,
      • Mitani T.
      • Isikay S.
      • Gezdirici A.
      • et al.
      High prevalence of multilocus pathogenic variation in neurodevelopmental disorders in the Turkish population.
      • Ngo K.J.
      • Rexach J.E.
      • Lee H.
      • et al.
      A diagnostic ceiling for exome sequencing in cerebellar ataxia and related neurological disorders.
      • Salfati E.L.
      • Spencer E.G.
      • Topol S.E.
      • et al.
      Re-analysis of whole-exome sequencing data uncovers novel diagnostic variants and improves molecular diagnostic yields for sudden death and idiopathic diseases.
      ,
      • Wright C.F.
      • McRae J.F.
      • Clayton S.
      • et al.
      Making new genetic diagnoses with old data: iterative reanalysis and reporting from genome-wide data in 1,133 families with developmental disorders.
      ,
      • Yang Y.
      • Muzny D.M.
      • Xia F.
      • et al.
      Molecular findings among patients referred for clinical whole-exome sequencing.
      reported eligibility criteria for patient recruitment. However, we ruled out this potential source of bias by performing a sensitivity analysis limited to these 10 studies, which obtained the same 10% estimates of efficacy. Thus, the finding of 10% diagnostic yield in NGS data reanalysis is rated as moderate-certainty evidence according to the GRADE assessment (Supplemental Table 2C).

      Discussion

      We performed a systematic review and meta-analysis of the available literature to determine efficacy of NGS data reanalysis for the diagnosis of suspected Mendelian diseases. This work builds on the initial systematic review by Tan et al
      • Tan N.B.
      • Stapleton R.
      • Stark Z.
      • et al.
      Evaluating systematic reanalysis of clinical genomic data in rare disease from single center experience and literature review.
      but differs from it in several ways. We used stringent inclusion/exclusion criteria and limited the meta-analysis to studies with ≥20 patients with NGS who had bioinformatic reanalysis. We applied ACMG/AMP diagnostic criteria and calculated diagnostic yields using strict definitions. This resulted in 14 out of 29 studies
      • Tan N.B.
      • Stapleton R.
      • Stark Z.
      • et al.
      Evaluating systematic reanalysis of clinical genomic data in rare disease from single center experience and literature review.
      ,
      • Al-Nabhani M.
      • Al-Rashdi S.
      • Al-Murshedi F.
      • et al.
      Reanalysis of exome sequencing data of intellectual disability samples: yields and benefits.
      • Alsubaie L.
      • Alturki S.
      • Alothaim A.
      • Alfares A.
      Clinical reassessment of post-laboratory variant call format (VCF) files.
      • Baker S.W.
      • Murrell J.R.
      • Nesbitt A.I.
      • et al.
      Automated clinical exome reanalysis reveals novel diagnoses.
      ,
      • Costain G.
      • Jobling R.
      • Walker S.
      • et al.
      Periodic reanalysis of whole-genome sequencing data enhances the diagnostic advantage over standard clinical genetic testing.
      ,
      • Eldomery M.K.
      • Coban-Akdemir Z.
      • Harel T.
      • et al.
      Lessons learned from additional research analyses of unsolved clinical exome cases.
      ,
      • James K.N.
      • Clark M.M.
      • Camp B.
      • et al.
      Partially automated whole-genome sequencing reanalysis of previously undiagnosed pediatric patients can efficiently yield new diagnoses.
      ,
      • Lassmann T.
      • Francis R.W.
      • Weeks A.
      • et al.
      A flexible computational pipeline for research analyses of unsolved clinical exome cases.
      ,
      • Liu Y.
      • Teng Y.
      • Li Z.
      • Cui J.
      • Liang D.
      • Wu L.
      Increase in diagnostic yield achieved for 174 whole-exome sequencing cases reanalyzed 1-2 years after initial analysis.
      ,
      • Machini K.
      • Ceyhan-Birsoy O.
      • Azzariti D.R.
      • et al.
      Analyzing and reanalyzing the genome: findings from the MedSeq project.
      ,
      • Mitani T.
      • Isikay S.
      • Gezdirici A.
      • et al.
      High prevalence of multilocus pathogenic variation in neurodevelopmental disorders in the Turkish population.
      ,
      • Basel-Salmon L.
      • Orenstein N.
      • Markus-Bustani K.
      • et al.
      Improved diagnostics by exome sequencing following raw data reevaluation by clinical geneticists involved in the medical care of the individuals tested.
      • Schmitz-Abe K.
      • Li Q.
      • Rosen S.M.
      • et al.
      Unique bioinformatic approach and comprehensive reanalysis improve diagnostic yield of clinical exomes.
      • Shashi V.
      • Schoch K.
      • Spillmann R.
      • et al.
      A comprehensive iterative approach is highly effective in diagnosing individuals who are exome negative.
      having a different diagnostic yield to that reported by the authors. This is because of the fact that we consistently used the number of patients without a diagnosis from the original analysis as the denominator. We also excluded cases with VUS and unvalidated novel variants and/or genes and those who originally reached a diagnosis but subsequently reached a secondary additional diagnosis from reanalysis. This allowed direct comparisons to be made between studies, leading to an overall pooled diagnostic yield of 10% that was not affected by study size or segregation. We assessed this estimate as moderate-certainty using the GRADE criteria.
      Most of the new diagnoses were attributable to updates in the literature and annotated disease databases cataloging disease-causing variants and genes. This is expected because annually there are >120,000 submissions to ClinVar
      National Library of Medicine
      ClinVar Celebrates 1 million submissions 2019. NCBI Insights.
      and 250 genes added to the literature,
      • Seaby E.G.
      • Rehm H.L.
      • O’Donnell-Luria A.
      Strategies to uplift novel Mendelian gene discovery for improved clinical outcomes.
      and there is an estimated 6000 Mendelian conditions remaining to be discovered in a recent review.
      • Bamshad M.J.
      • Nickerson D.A.
      • Chong J.X.
      Mendelian gene discovery: fast and furious with no end in sight.
      This is supported by our finding that reanalysis conducted after ≥24 months is associated with a higher diagnostic yield than that conducted after <24 months, although this was not statistically significant owing to the small number of studies and heterogeneity between study groups. Accordingly, we would suggest that reanalysis be delayed to ≥24 months unless there was urgent clinical need to reanalyze earlier.
      Novel variant and/or gene discovery was the third highest cause of positive diagnosis in our study. In this instance, the study authors validated the pathogenicity of the VUS with external data sharing and functional studies. This is distinct from the publication of new data showing pathogenicity by other authors (updates in the literature). This is consistent with the position statement by the ACMG
      Acmg Board Of Directors
      Laboratory and clinical genomic data sharing is crucial to improving genetic health care: a position statement of the American College of Medical Genetics and Genomics.
      on laboratory and clinical genomic data sharing to improve diagnosis. It also highlights the importance of functional studies being available to validate novel variants and/or genes to improve diagnostic yield, particularly in cases where segregation studies are not possible.
      • Casanova J.L.
      • Conley M.E.
      • Seligman S.J.
      • Abel L.
      • Notarangelo L.D.
      Guidelines for genetic studies in single patients: lessons from primary immunodeficiencies.
      This reinforces the need to develop and make accessible high throughput functional assays to assist in genomic diagnosis.
      • Starita L.M.
      • Ahituv N.
      • Dunham M.J.
      • et al.
      Variant interpretation: functional assays to the rescue.
      We also showed the importance of software updates including CNV calling,
      • Nambot S.
      • Thevenon J.
      • Kuentz P.
      • et al.
      Clinical whole-exome sequencing for the diagnosis of rare disorders with congenital anomalies and/or intellectual disability: substantial interest of prospective annual reanalysis.
      ,
      • James K.N.
      • Clark M.M.
      • Camp B.
      • et al.
      Partially automated whole-genome sequencing reanalysis of previously undiagnosed pediatric patients can efficiently yield new diagnoses.
      ,
      • Liu Y.
      • Teng Y.
      • Li Z.
      • Cui J.
      • Liang D.
      • Wu L.
      Increase in diagnostic yield achieved for 174 whole-exome sequencing cases reanalyzed 1-2 years after initial analysis.
      ,
      • Ngo K.J.
      • Rexach J.E.
      • Lee H.
      • et al.
      A diagnostic ceiling for exome sequencing in cerebellar ataxia and related neurological disorders.
      ,
      • Shashi V.
      • Schoch K.
      • Spillmann R.
      • et al.
      A comprehensive iterative approach is highly effective in diagnosing individuals who are exome negative.
      better identification of low quality
      • Sundercombe S.L.
      • Berbic M.
      • Evans C.A.
      • et al.
      Clinically responsive genomic analysis pipelines: elements to improve detection rate and efficiency.
      and/or coverage variants,
      • Al-Nabhani M.
      • Al-Rashdi S.
      • Al-Murshedi F.
      • et al.
      Reanalysis of exome sequencing data of intellectual disability samples: yields and benefits.
      ,
      • Bowling K.M.
      • Thompson M.L.
      • Amaral M.D.
      • et al.
      Genomic diagnosis for children with intellectual disability and/or developmental delay.
      and intronic variant calling
      • Liu Y.
      • Teng Y.
      • Li Z.
      • Cui J.
      • Liang D.
      • Wu L.
      Increase in diagnostic yield achieved for 174 whole-exome sequencing cases reanalyzed 1-2 years after initial analysis.
      (Supplemental Figure 5). Furthermore, accurate and updated phenotyping during reanalysis also contributes to improved diagnostic yield because additional phenotype information allows improved genotype–phenotype correlation. This can be further improved by involving the referring clinician in assessing the relevance of candidate variants alongside the expertise of genetic scientists and pathologists in a multidisciplinary team setting.
      • Basel-Salmon L.
      • Orenstein N.
      • Markus-Bustani K.
      • et al.
      Improved diagnostics by exome sequencing following raw data reevaluation by clinical geneticists involved in the medical care of the individuals tested.
      There were also 3 studies in which extension of reanalysis from singleton to trios allowed identification of diagnostic variants. This was because of better identification of de novo variants
      • James K.N.
      • Clark M.M.
      • Camp B.
      • et al.
      Partially automated whole-genome sequencing reanalysis of previously undiagnosed pediatric patients can efficiently yield new diagnoses.
      ,
      • Li J.
      • Gao K.
      • Yan H.
      • et al.
      Reanalysis of whole exome sequencing data in patients with epilepsy and intellectual disability/mental retardation.
      and unusual inheritance patterns including digenic inheritance and inheritance of a somatic mosaic variant from an unaffected parent.
      • Eldomery M.K.
      • Coban-Akdemir Z.
      • Harel T.
      • et al.
      Lessons learned from additional research analyses of unsolved clinical exome cases.
      Unexpectedly, original laboratory errors also contributed to nearly 3% of positive diagnostic yield from reanalysis. These included errors in patient data handling,
      • James K.N.
      • Clark M.M.
      • Camp B.
      • et al.
      Partially automated whole-genome sequencing reanalysis of previously undiagnosed pediatric patients can efficiently yield new diagnoses.
      interpretation of the clinical context,
      • Li J.
      • Gao K.
      • Yan H.
      • et al.
      Reanalysis of whole exome sequencing data in patients with epilepsy and intellectual disability/mental retardation.
      ,
      • Salfati E.L.
      • Spencer E.G.
      • Topol S.E.
      • et al.
      Re-analysis of whole-exome sequencing data uncovers novel diagnostic variants and improves molecular diagnostic yields for sudden death and idiopathic diseases.
      and variant interpretation.
      • Nambot S.
      • Thevenon J.
      • Kuentz P.
      • et al.
      Clinical whole-exome sequencing for the diagnosis of rare disorders with congenital anomalies and/or intellectual disability: substantial interest of prospective annual reanalysis.
      Although this unexpectedly large figure is confounded by the fact that only half of the diagnosed patients from our collated studies had a reason reported for their diagnosis, it nevertheless shows the importance of having rigorous quality control systems in both the bioinformatic and variant interpretation processes.
      Interestingly, use of AI-based semiautomation in the reanalysis pipeline only led to a small decrease in the diagnostic yield compared with manual curation from 10% to 8%, which was not statistically significant. The AI-based methodologies included semiautomation in phenotype–genotype matching with databases and published literature, variant filtration and/or classification, and extraction of patient history from clinical notes using natural language processing.
      • Baker S.W.
      • Murrell J.R.
      • Nesbitt A.I.
      • et al.
      Automated clinical exome reanalysis reveals novel diagnoses.
      ,
      • Lassmann T.
      • Francis R.W.
      • Weeks A.
      • et al.
      A flexible computational pipeline for research analyses of unsolved clinical exome cases.
      ,
      • Liu P.
      • Meng L.
      • Normand E.A.
      • et al.
      Reanalysis of clinical exome sequencing data.
      ,
      • Matalonga L.
      • Hernández-Ferrer C.
      • Piscia D.
      • et al.
      Solving patients with rare diseases through programmatic reanalysis of genome-phenome data.
      ,
      • Salfati E.L.
      • Spencer E.G.
      • Topol S.E.
      • et al.
      Re-analysis of whole-exome sequencing data uncovers novel diagnostic variants and improves molecular diagnostic yields for sudden death and idiopathic diseases.
      Manual variant curation is labor-intensive and requires multidisciplinary expertise in several domains of knowledge.
      • Mardis E.R.
      The $1,000 genome, the $100,000 analysis?.
      Use of AI in clinical genetics as a labor-saving tool has the potential to bypass this bottleneck and facilitate reanalysis of more undiagnosed cases, thereby increasing access of patients to the clinical benefits of reanalysis. We anticipate that future improvements will lead to even better performance by AI-based tools and bridge the narrow gap between manual curation and automation.
      Counterintuitively, we also found reanalysis of cES to have higher yields than that of cGS data despite the clear advantage of the latter in terms of more uniform coverage, ability to detect noncoding variants and better detection of structural variants.
      • Adams D.R.
      • Eng C.M.
      Next-generation sequencing to diagnose suspected genetic disorders.
      One possibility is that in the majority of the studies that conducted research validation of novel variants/genes, they were conducted using ES.
      • Nambot S.
      • Thevenon J.
      • Kuentz P.
      • et al.
      Clinical whole-exome sequencing for the diagnosis of rare disorders with congenital anomalies and/or intellectual disability: substantial interest of prospective annual reanalysis.
      ,
      • Genetics Initiative Epilepsy
      The Epilepsy Genetics Initiative: systematic reanalysis of diagnostic exomes increases yield.
      ,
      • Eldomery M.K.
      • Coban-Akdemir Z.
      • Harel T.
      • et al.
      Lessons learned from additional research analyses of unsolved clinical exome cases.
      ,
      • Basel-Salmon L.
      • Orenstein N.
      • Markus-Bustani K.
      • et al.
      Improved diagnostics by exome sequencing following raw data reevaluation by clinical geneticists involved in the medical care of the individuals tested.
      ,
      • Shashi V.
      • Schoch K.
      • Spillmann R.
      • et al.
      A comprehensive iterative approach is highly effective in diagnosing individuals who are exome negative.
      In addition, within the studies that reported their diagnostic yield from the initial analysis of the original cohort, the 3 studies that performed pure cGS analysis
      • Costain G.
      • Jobling R.
      • Walker S.
      • et al.
      Periodic reanalysis of whole-genome sequencing data enhances the diagnostic advantage over standard clinical genetic testing.
      ,
      • James K.N.
      • Clark M.M.
      • Camp B.
      • et al.
      Partially automated whole-genome sequencing reanalysis of previously undiagnosed pediatric patients can efficiently yield new diagnoses.
      ,
      • Machini K.
      • Ceyhan-Birsoy O.
      • Azzariti D.R.
      • et al.
      Analyzing and reanalyzing the genome: findings from the MedSeq project.
      had a higher yield than the 15 studies
      • Tan N.B.
      • Stapleton R.
      • Stark Z.
      • et al.
      Evaluating systematic reanalysis of clinical genomic data in rare disease from single center experience and literature review.
      ,
      • Nambot S.
      • Thevenon J.
      • Kuentz P.
      • et al.
      Clinical whole-exome sequencing for the diagnosis of rare disorders with congenital anomalies and/or intellectual disability: substantial interest of prospective annual reanalysis.
      ,
      • Baker S.W.
      • Murrell J.R.
      • Nesbitt A.I.
      • et al.
      Automated clinical exome reanalysis reveals novel diagnoses.
      ,
      • Genetics Initiative Epilepsy
      The Epilepsy Genetics Initiative: systematic reanalysis of diagnostic exomes increases yield.
      ,
      • Brunet T.
      • Jech R.
      • Brugger M.
      • et al.
      De novo variants in neurodevelopmental disorders-experiences from a tertiary care center.
      ,
      • Ewans L.J.
      • Schofield D.
      • Shrestha R.
      • et al.
      Whole-exome sequencing reanalysis at 12 months boosts diagnosis and is cost-effective when applied early in Mendelian disorders.
      ,
      • Jalkh N.
      • Corbani S.
      • Haidar Z.
      • et al.
      The added value of WES reanalysis in the field of genetic diagnosis: lessons learned from 200 exomes in the Lebanese population.
      ,
      • Lassmann T.
      • Francis R.W.
      • Weeks A.
      • et al.
      A flexible computational pipeline for research analyses of unsolved clinical exome cases.
      ,
      • Liu P.
      • Meng L.
      • Normand E.A.
      • et al.
      Reanalysis of clinical exome sequencing data.
      ,
      • Mitani T.
      • Isikay S.
      • Gezdirici A.
      • et al.
      High prevalence of multilocus pathogenic variation in neurodevelopmental disorders in the Turkish population.
      • Ngo K.J.
      • Rexach J.E.
      • Lee H.
      • et al.
      A diagnostic ceiling for exome sequencing in cerebellar ataxia and related neurological disorders.
      • Salfati E.L.
      • Spencer E.G.
      • Topol S.E.
      • et al.
      Re-analysis of whole-exome sequencing data uncovers novel diagnostic variants and improves molecular diagnostic yields for sudden death and idiopathic diseases.
      • Basel-Salmon L.
      • Orenstein N.
      • Markus-Bustani K.
      • et al.
      Improved diagnostics by exome sequencing following raw data reevaluation by clinical geneticists involved in the medical care of the individuals tested.
      • Schmitz-Abe K.
      • Li Q.
      • Rosen S.M.
      • et al.
      Unique bioinformatic approach and comprehensive reanalysis improve diagnostic yield of clinical exomes.
      ,
      • Wright C.F.
      • McRae J.F.
      • Clayton S.
      • et al.
      Making new genetic diagnoses with old data: iterative reanalysis and reporting from genome-wide data in 1,133 families with developmental disorders.
      ,
      Reanalysis of clinical whole-exome sequence data yields multiple new diagnoses: A time-intensive but successful strategy highlights the benefits of data sharing and international collaborations.
      that performed pure cES analysis (0.36 vs 0.2). One study performed both cES and cGS.
      • Bowling K.M.
      • Thompson M.L.
      • Amaral M.D.
      • et al.
      Genomic diagnosis for children with intellectual disability and/or developmental delay.
      It is possible that a higher initial yield from cGS analysis resulted in a lower reanalysis yield. In addition, it is also possible that the reanalysis timing in cGS studies were shorter than the cES studies. We did not have sufficient information to make this conclusion but within 1 study
      • Bowling K.M.
      • Thompson M.L.
      • Amaral M.D.
      • et al.
      Genomic diagnosis for children with intellectual disability and/or developmental delay.
      that reported both cGS and cES reanalysis rates, the cGS studies had less time between initial analysis and reanalysis and this was associated with a lower reanalysis diagnostic yield. Finally, it is also possible that more patients in the cGS cohorts were not naive to sequencing and had undergone previous analysis such as cES. These patients may therefore represent the most difficult to diagnose cases. However, owing to the lack of stringent patient eligibility criteria, we did not have the information available to make this conclusion.
      We did not specifically examine the role of repeat sequencing or complementary/additional genetic testing in the reanalysis. This deserves further study because cGS after cES is associated with increased diagnostic yield.
      • Tan N.B.
      • Stapleton R.
      • Stark Z.
      • et al.
      Evaluating systematic reanalysis of clinical genomic data in rare disease from single center experience and literature review.
      Complementary testing such as chromosomal microarrays have also identified additional pathogenic variants that were missed by cES.
      • Tan N.B.
      • Stapleton R.
      • Stark Z.
      • et al.
      Evaluating systematic reanalysis of clinical genomic data in rare disease from single center experience and literature review.
      ,
      • Liu P.
      • Meng L.
      • Normand E.A.
      • et al.
      Reanalysis of clinical exome sequencing data.
      We also did not specifically examine the effect of reanalysis on cases in which a previous diagnosis had been established. However, it was noted in a few studies
      • Eldomery M.K.
      • Coban-Akdemir Z.
      • Harel T.
      • et al.
      Lessons learned from additional research analyses of unsolved clinical exome cases.
      ,
      • Liu P.
      • Meng L.
      • Normand E.A.
      • et al.
      Reanalysis of clinical exome sequencing data.
      that some cases had downgraded the pathogenicity of their diagnostic variants whereas others reached additional pathogenic variants resulting in a multilocus contribution to disease phenotype. We believe this deserves further study in the future. However, given the likely low rates of reclassification and the labor-intensive nature of variant curation, it is questionable whether systematic reanalysis of diagnosed variants can ever be achieved on a large scale.
      A major limitation of our study is the considerable heterogeneity between studies, leading us to downgrade the certainty of assessment to moderate-quality. We were not able to explain the interstudy heterogeneity despite subgroup classifications by factors that are known to affect diagnostic yield. Reasons for the heterogeneity include lack of granular individual patient clinical details, which hampered subgroup analysis; lack of strict eligibility criteria for patient recruitment; and lack of uniform standards and methods for bioinformatic processing of raw data and variant curation.
      • Amendola L.M.
      • Jarvik G.P.
      • Leo M.C.
      • et al.
      Performance of ACMG-AMP Variant-Interpretation Guidelines among nine laboratories in the Clinical Sequencing Exploratory Research Consortium.
      The availability of such granular patient details may also make conclusions regarding the impact of individual conditions on the diagnostic yield possible.
      Given these limitations, we make the following recommendations for minimum standards in future reanalysis studies to promote more comparable and generalizable results (Supplemental Figure 6): (1) detailed inclusion and exclusion criteria for patient recruitment, (2) reporting of clinical details for each patient undergoing initial analysis and subsequent reanalysis, (3) clear documentation of sequencing, bioinformatic, and variant curation methodologies, (4) clear documentation of methodological differences between original analysis and subsequent reanalysis, (5) adherence to ACMG/AMP variant classification consensus guidelines during variant curation, (6) reporting of diagnostic yield from both initial analysis and subsequent reanalysis, (7) reporting of diagnostic variants associated with each patient and their ACMG/AMP classification, and (8) provision of detailed rationale for attributing pathogenicity to each diagnostic variant. These details can include specific ACMG/AMP criteria underlying classification and research evidence and/or results from external data sharing in novel variants and/or genes.
      We also asked what is the best way to maximize the diagnostic yield from genomic data reanalysis in undiagnosed patients. On the basis of moderate-quality data from our meta-analysis, we make the following best practice recommendations in NGS data reanalysis (Figure 3): (1) data reanalysis should be routinely performed, preferably ≥24 months after the original analysis, or earlier depending on the clinical need and availability of resources. This time interval is based on low-quality data and will likely change as new higher quality studies are published. Genetic diagnosis is a dynamic new field and it is possible that the rate of gene discoveries will slow over time as the field matures and as we approach the asymptote. Under these circumstances, it is likely that the optimal timing of reanalysis will converge on a new equilibrium and the recommended time interval will change, (2) data reanalysis should involve reannotation with the most up-to-date disease databases, software, and alignment/variant calling tools, (3) data reanalysis should include repeated clinical phenotyping of patients with ambiguous results discussed in an multidisciplinary team with the patient’s referring clinician, (4) candidate gene variants should undergo family segregation studies when possible, (5) laboratories providing reanalysis should have access to research pipelines and external collaboration to validate novel variants and/or genes, and (6) reanalysis should consider adopting automation as a labor-saving tool to increase patient access to reanalysis, particularly because AI-based tools continue to improve. This recommendation is based on the fact that the use of AI-based approaches was noninferior (did not result in a statistically significant loss in diagnostic accuracy).
      Figure thumbnail gr3
      Figure 3Suggested best practice for next generation sequencing data reanalysis. VUS, variants of uncertain significance.
      In conclusion, there was considerable heterogeneity in the data reanalysis studies. Future reanalysis studies should follow a set of minimum standards to reduce the heterogeneity and improve the confidence of conclusions from future meta-analyses. NGS data reanalysis in patients with undiagnosed Mendelian disorders have an overall yield of 10% and should be performed routinely. Although the data suggests reanalysis performed at longer time intervals, such as 24 months, will provide higher diagnostic yields, the optimal timing of reanalysis will also depend on individual patient needs and laboratory resources.

      Conflict of Interest

      The authors declare no conflicts of interests.

      Acknowledgments

      We thank the patients, their families, and treating clinicians who made this work possible. P.D. is a recipient of the Inaugural David Cooper Memorial Fund Fellowship and the RCPA Postgraduate Research Fellowship Award. T.G.P. is supported by an NHMRC Senior Research Fellowship (ID 1155678) and Mrs Janice Gibson and the Ernest Heine Family Foundation. A.H. received Summer Research Scholarships from the Kinghorn Centre for Clinical Genomics, Garvan Institute of Medical Research. This work is supported by the St Vincent’s Clinic Foundation, Australia, Allergy and Immunology Foundation of Australia (AIFA), and Lowy Foundation Garvan-Weizmann grant.

      Author Information

      Conceptualization: P.D., T.G.P.; Data Curation: P.D., A.H., L.B.; Formal Analysis: P.D., A.H., L.B., M.L., T.G.P.; Funding Acquisition: L.B., T.G.P.; Investigation: T.G.P.; Methodology: P.D., L.B., M.L., T.G.P.; Project Administration: T.G.P.; Resources: T.G.P.; Supervision: M.L., T.G.P.; Validation: L.B., A.H., M.L., T.G.P.; Visualization: P.D., T.G.P.; Writing-original draft: P.D., L.B., M.L., T.G.P.; Writing-review and editing: P.D., A.H., J.M., L.E., L.B., M.L., T.G.P.

      Supplementary Material

      References

        • Katsanis S.H.
        • Katsanis N.
        Molecular genetic testing and the future of clinical genomics.
        Nat Rev Genet. 2013; 14: 415-426https://doi.org/10.1038/nrg3493
        • Wang J.
        • Gotway G.
        • Pascual J.M.
        • Park J.Y.
        Diagnostic yield of clinical next-generation sequencing panels for epilepsy.
        JAMA Neurol. 2014; 71: 650-651https://doi.org/10.1001/jamaneurol.2014.405
        • Lee H.
        • Deignan J.L.
        • Dorrani N.
        • et al.
        Clinical exome sequencing for genetic identification of rare Mendelian disorders.
        JAMA. 2014; 312: 1880-1887https://doi.org/10.1001/jama.2014.14604
        • Phan T.G.
        • Gray P.E.
        • Wong M.
        • et al.
        The Clinical Immunogenomics Research Consortium Australasia (CIRCA): a distributed network model for genomic healthcare delivery.
        J Clin Immunol. 2020; 40: 763-766https://doi.org/10.1007/s10875-020-00787-6
        • Sarmady M.
        • Abou Tayoun A.
        Need for automated interactive genomic interpretation and ongoing reanalysis.
        JAMA Pediatr. 2018; 172: 1113-1114https://doi.org/10.1001/jamapediatrics.2018.2675
        • Tan T.Y.
        • Dillon O.J.
        • Stark Z.
        • et al.
        Diagnostic impact and cost-effectiveness of whole-exome sequencing for ambulant children with suspected monogenic conditions.
        JAMA Pediatr. 2017; 171: 855-862https://doi.org/10.1001/jamapediatrics.2017.1755
        • Deignan J.L.
        • Chung W.K.
        • Kearney H.M.
        • et al.
        Points to consider in the reevaluation and reanalysis of genomic test results: a statement of the American College of Medical Genetics and Genomics (ACMG).
        Genet Med. 2019; 21: 1267-1270https://doi.org/10.1038/s41436-019-0478-1
        • Ji J.
        • Leung M.L.
        • Baker S.
        • Deignan J.L.
        • Santani A.
        Clinical exome reanalysis: current practice and beyond.
        Mol Diagn Ther. 2021; 25: 529-536https://doi.org/10.1007/s40291-021-00541-7
        • Richards S.
        • Aziz N.
        • Bale S.
        • et al.
        Standards and guidelines for the interpretation of sequence variants: a joint consensus recommendation of the American College of Medical Genetics and Genomics and the Association for Molecular Pathology.
        Genet Med. 2015; 17: 405-424https://doi.org/10.1038/gim.2015.30
        • Tan N.B.
        • Stapleton R.
        • Stark Z.
        • et al.
        Evaluating systematic reanalysis of clinical genomic data in rare disease from single center experience and literature review.
        Mol Genet Genomic Med. 2020; 8e1508https://doi.org/10.1002/mgg3.1508
        • Page M.J.
        • McKenzie J.E.
        • Bossuyt P.M.
        • et al.
        The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.
        BMJ. 2021; 372: n71https://doi.org/10.1136/bmj.n71
        • Nambot S.
        • Thevenon J.
        • Kuentz P.
        • et al.
        Clinical whole-exome sequencing for the diagnosis of rare disorders with congenital anomalies and/or intellectual disability: substantial interest of prospective annual reanalysis.
        Genet Med. 2018; 20: 645-654https://doi.org/10.1038/gim.2017.162
        • Whiting P.F.
        • Rutjes A.W.
        • Westwood M.E.
        • et al.
        QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies.
        Ann Intern Med. 2011; 155: 529-536https://doi.org/10.7326/0003-4819-155-8-201110180-00009
        • Bossuyt P.M.
        • Reitsma J.B.
        • Bruns D.E.
        • et al.
        STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies.
        BMJ. 2015; 351: h5527https://doi.org/10.1136/bmj.h5527
        • Egger M.
        • Davey Smith G.
        • Schneider M.
        • Minder C.
        Bias in meta-analysis detected by a simple, graphical test.
        BMJ. 1997; 315: 629-634https://doi.org/10.1136/bmj.315.7109.629
        • Nyaga V.N.
        • Arbyn M.
        • Aerts M.
        Metaprop: a Stata command to perform meta-analysis of binomial data.
        Arch Public Health. 2014; 72: 39https://doi.org/10.1186/2049-3258-72-39
        • Schünemann H.J.
        • Oxman A.D.
        • Brozek J.
        • et al.
        Grading quality of evidence and strength of recommendations for diagnostic tests and strategies.
        BMJ. 2008; 336 (Published correction appears in BMJ. 2008;336(7654)): 1106-1110
        https://doi.org/10.1136/bmj.39500.677199.AE
        • Bruel A.L.
        • Nambot S.
        • Quéré V.
        • et al.
        Increased diagnostic and new genes identification outcome using research reanalysis of singleton exome sequencing.
        Eur J Hum Genet. 2019; 27: 1519-1531https://doi.org/10.1038/s41431-019-0442-1
        • Al-Nabhani M.
        • Al-Rashdi S.
        • Al-Murshedi F.
        • et al.
        Reanalysis of exome sequencing data of intellectual disability samples: yields and benefits.
        Clin Genet. 2018; 94: 495-501https://doi.org/10.1111/cge.13438
        • Alsubaie L.
        • Alturki S.
        • Alothaim A.
        • Alfares A.
        Clinical reassessment of post-laboratory variant call format (VCF) files.
        JBCGenetics. 2018; 1: 31-36https://doi.org/10.24911/JBCGenetics/183-1529928114
        • Baker S.W.
        • Murrell J.R.
        • Nesbitt A.I.
        • et al.
        Automated clinical exome reanalysis reveals novel diagnoses.
        J Mol Diagn. 2019; 21: 38-48https://doi.org/10.1016/j.jmoldx.2018.07.008
        • Genetics Initiative Epilepsy
        The Epilepsy Genetics Initiative: systematic reanalysis of diagnostic exomes increases yield.
        Epilepsia. 2019; 60: 797-806https://doi.org/10.1111/epi.14698
        • Bowling K.M.
        • Thompson M.L.
        • Amaral M.D.
        • et al.
        Genomic diagnosis for children with intellectual disability and/or developmental delay.
        Genome Med. 2017; 9: 43https://doi.org/10.1186/s13073-017-0433-1
        • Brunet T.
        • Jech R.
        • Brugger M.
        • et al.
        De novo variants in neurodevelopmental disorders-experiences from a tertiary care center.
        Clin Genet. 2021; 100: 14-28https://doi.org/10.1111/cge.13946
        • Costain G.
        • Jobling R.
        • Walker S.
        • et al.
        Periodic reanalysis of whole-genome sequencing data enhances the diagnostic advantage over standard clinical genetic testing.
        Eur J Hum Genet. 2018; 26: 740-744https://doi.org/10.1038/s41431-018-0114-6
        • Eldomery M.K.
        • Coban-Akdemir Z.
        • Harel T.
        • et al.
        Lessons learned from additional research analyses of unsolved clinical exome cases.
        Genome Med. 2017; 9: 26https://doi.org/10.1186/s13073-017-0412-6
        • Ewans L.J.
        • Schofield D.
        • Shrestha R.
        • et al.
        Whole-exome sequencing reanalysis at 12 months boosts diagnosis and is cost-effective when applied early in Mendelian disorders.
        Genet Med. 2018; 20: 1564-1574https://doi.org/10.1038/gim.2018.39
        • Jalkh N.
        • Corbani S.
        • Haidar Z.
        • et al.
        The added value of WES reanalysis in the field of genetic diagnosis: lessons learned from 200 exomes in the Lebanese population.
        BMC Med Genomics. 2019; 12: 11https://doi.org/10.1186/s12920-019-0474-y
        • James K.N.
        • Clark M.M.
        • Camp B.
        • et al.
        Partially automated whole-genome sequencing reanalysis of previously undiagnosed pediatric patients can efficiently yield new diagnoses.
        NPJ Genom Med. 2020; 5: 33https://doi.org/10.1038/s41525-020-00140-1
        • Lassmann T.
        • Francis R.W.
        • Weeks A.
        • et al.
        A flexible computational pipeline for research analyses of unsolved clinical exome cases.
        NPJ Genom Med. 2020; 5: 54https://doi.org/10.1038/s41525-020-00161-w
        • Li J.
        • Gao K.
        • Yan H.
        • et al.
        Reanalysis of whole exome sequencing data in patients with epilepsy and intellectual disability/mental retardation.
        Gene. 2019; 700: 168-175https://doi.org/10.1016/j.gene.2019.03.037
        • Liu P.
        • Meng L.
        • Normand E.A.
        • et al.
        Reanalysis of clinical exome sequencing data.
        N Engl J Med. 2019; 380: 2478-2480https://doi.org/10.1056/NEJMc1812033
        • Liu Y.
        • Teng Y.
        • Li Z.
        • Cui J.
        • Liang D.
        • Wu L.
        Increase in diagnostic yield achieved for 174 whole-exome sequencing cases reanalyzed 1-2 years after initial analysis.
        Clin Chim Acta. 2021; 523: 163-168https://doi.org/10.1016/j.cca.2021.09.015
        • Machini K.
        • Ceyhan-Birsoy O.
        • Azzariti D.R.
        • et al.
        Analyzing and reanalyzing the genome: findings from the MedSeq project.
        Am J Hum Genet. 2019; 105: 177-188https://doi.org/10.1016/j.ajhg.2019.05.017
        • Matalonga L.
        • Hernández-Ferrer C.
        • Piscia D.
        • et al.
        Solving patients with rare diseases through programmatic reanalysis of genome-phenome data.
        Eur J Hum Genet. 2021; 29 (Published correction appears in Eur J Hum Genet. 2021;29(9):1466–1469): 1337-1347
        https://doi.org/10.1038/s41431-021-00852-7
        • Mitani T.
        • Isikay S.
        • Gezdirici A.
        • et al.
        High prevalence of multilocus pathogenic variation in neurodevelopmental disorders in the Turkish population.
        Am J Hum Genet. 2021; 108: 1981-2005https://doi.org/10.1016/j.ajhg.2021.08.009
        • Ngo K.J.
        • Rexach J.E.
        • Lee H.
        • et al.
        A diagnostic ceiling for exome sequencing in cerebellar ataxia and related neurological disorders.
        Hum Mutat. 2020; 41: 487-501https://doi.org/10.1002/humu.23946
        • Salfati E.L.
        • Spencer E.G.
        • Topol S.E.
        • et al.
        Re-analysis of whole-exome sequencing data uncovers novel diagnostic variants and improves molecular diagnostic yields for sudden death and idiopathic diseases.
        Genome Med. 2019; 11: 83https://doi.org/10.1186/s13073-019-0702-2
        • Basel-Salmon L.
        • Orenstein N.
        • Markus-Bustani K.
        • et al.
        Improved diagnostics by exome sequencing following raw data reevaluation by clinical geneticists involved in the medical care of the individuals tested.
        Genet Med. 2019; 21: 1443-1451https://doi.org/10.1038/s41436-018-0343-7
        • Schmitz-Abe K.
        • Li Q.
        • Rosen S.M.
        • et al.
        Unique bioinformatic approach and comprehensive reanalysis improve diagnostic yield of clinical exomes.
        Eur J Hum Genet. 2019; 27: 1398-1405https://doi.org/10.1038/s41431-019-0401-x
        • Shashi V.
        • Schoch K.
        • Spillmann R.
        • et al.
        A comprehensive iterative approach is highly effective in diagnosing individuals who are exome negative.
        Genet Med. 2019; 21: 161-172https://doi.org/10.1038/s41436-018-0044-2
        • Sundercombe S.L.
        • Berbic M.
        • Evans C.A.
        • et al.
        Clinically responsive genomic analysis pipelines: elements to improve detection rate and efficiency.
        J Mol Diagn. 2021; 23: 894-905https://doi.org/10.1016/j.jmoldx.2021.04.007
        • Wenger A.M.
        • Guturu H.
        • Bernstein J.A.
        • Bejerano G.
        Systematic reanalysis of clinical exome data yields additional diagnoses: implications for providers.
        Genet Med. 2017; 19: 209-214https://doi.org/10.1038/gim.2016.88
        • Wright C.F.
        • McRae J.F.
        • Clayton S.
        • et al.
        Making new genetic diagnoses with old data: iterative reanalysis and reporting from genome-wide data in 1,133 families with developmental disorders.
        Genet Med. 2018; 20: 1216-1223https://doi.org/10.1038/gim.2017.246
        • McKenna A.
        • Hanna M.
        • Banks E.
        • et al.
        The Genome Analysis Toolkit: a MapReduce framework for analyzing next-generation DNA sequencing data.
        Genome Res. 2010; 20: 1297-1303https://doi.org/10.1101/gr.107524.110
        • Yang Y.
        • Muzny D.M.
        • Xia F.
        • et al.
        Molecular findings among patients referred for clinical whole-exome sequencing.
        JAMA. 2014; 312: 1870-1879https://doi.org/10.1001/jama.2014.14601
        • National Library of Medicine
        ClinVar Celebrates 1 million submissions 2019. NCBI Insights.
        • Seaby E.G.
        • Rehm H.L.
        • O’Donnell-Luria A.
        Strategies to uplift novel Mendelian gene discovery for improved clinical outcomes.
        Front Genet. 2021; 12: 674295https://doi.org/10.3389/fgene.2021.674295
        • Bamshad M.J.
        • Nickerson D.A.
        • Chong J.X.
        Mendelian gene discovery: fast and furious with no end in sight.
        Am J Hum Genet. 2019; 105: 448-455https://doi.org/10.1016/j.ajhg.2019.07.011
        • Acmg Board Of Directors
        Laboratory and clinical genomic data sharing is crucial to improving genetic health care: a position statement of the American College of Medical Genetics and Genomics.
        Genet Med. 2017; 19: 721-722https://doi.org/10.1038/gim.2016.196
        • Casanova J.L.
        • Conley M.E.
        • Seligman S.J.
        • Abel L.
        • Notarangelo L.D.
        Guidelines for genetic studies in single patients: lessons from primary immunodeficiencies.
        J Exp Med. 2014; 211: 2137-2149https://doi.org/10.1084/jem.20140520
        • Starita L.M.
        • Ahituv N.
        • Dunham M.J.
        • et al.
        Variant interpretation: functional assays to the rescue.
        Am J Hum Genet. 2017; 101: 315-325https://doi.org/10.1016/j.ajhg.2017.07.014
        • Mardis E.R.
        The $1,000 genome, the $100,000 analysis?.
        Genome Med. 2010; 2: 84https://doi.org/10.1186/gm205
        • Adams D.R.
        • Eng C.M.
        Next-generation sequencing to diagnose suspected genetic disorders.
        N Engl J Med. 2018; 379: 1353-1362https://doi.org/10.1056/NEJMra1711801
      1. Reanalysis of clinical whole-exome sequence data yields multiple new diagnoses: A time-intensive but successful strategy highlights the benefits of data sharing and international collaborations.
        Am J Med Genet A. 2018; 176: 264-265https://doi.org/10.1002/ajmg.a.38608
        • Amendola L.M.
        • Jarvik G.P.
        • Leo M.C.
        • et al.
        Performance of ACMG-AMP Variant-Interpretation Guidelines among nine laboratories in the Clinical Sequencing Exploratory Research Consortium.
        Am J Hum Genet. 2016; 98 (Published correction appears in Am J Hum Genet. 2016;99(1):247): 1067-1076
        https://doi.org/10.1016/j.ajhg.2016.03.024