If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Correspondence and requests for materials should be addressed to Robin Z. Hayeems, Child Health Evaluative Sciences, The Hospital for Sick Children (SickKids), 686 Bay Street, Toronto, Ontario M5G 0A4, Canada.
Child Health Evaluative Sciences, The Hospital for Sick Children (SickKids), Toronto, Ontario, CanadaInstitute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, The University of Toronto, Toronto, Ontario, Canada
Child Health Evaluative Sciences, The Hospital for Sick Children (SickKids), Toronto, Ontario, CanadaInstitute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, The University of Toronto, Toronto, Ontario, Canada
Division of Clinical and Metabolic Genetics, The Hospital for Sick Children (SickKids), Toronto, Ontario, CanadaDepartment of Pediatrics, Temetry Faculty of Medicine, The University of Toronto, Toronto, Ontario, Canada
Department of Medical Genetics, Alberta Children’s Hospital, Calgary, Alberta, CanadaAlberta Children’s Hospital Research Institute, University of Calgary, Calgary, Alberta, Canada
Department of Genetics, Children’s Hospital of Eastern Ontario, Ottawa, Ontario, CanadaChildren’s Hospital of Eastern Ontario Research Institute, University of Ottawa, Ottawa, Ontario, Canada
Cumming School of Medicine, University of Calgary, Calgary, Alberta, CanadaO’Brien Institute for Public Health, University of Calgary, Calgary, Alberta, Canada
To facilitate robust economic analyses of clinical exome and genome sequencing, this study was taken up with the objective of establishing a framework for organizing diagnostic testing trajectories for patients with rare disease.
Methods
We collected diagnostic investigations–related data before exome sequencing from the medical records of 228 cases. Medical geneticist experts participated in a consensus building process to develop the SOLVE Framework for organizing the complex range of observed tests. Experts categorized tests as indicator or nonindicator tests on the basis of their specificity for diagnosing rare diseases. Face validity was assessed using case vignettes.
Results
Most cases had symptom onset at birth (42.5%) or during childhood (43.4%) and had intellectual disability (73.3%). On average, the time spent seeking a diagnosis before sequencing was 1989 days (SD = 2137) and included 16 tests (SD = 14). Agreement across experts on test categories ranged from 83% to 96%. The SOLVE Framework comprised observed tests, including 186 indicator and 39 nonindicator tests across cytogenetic/molecular, biochemical, imaging, electrical, and pathology test categories.
Conclusion
Real-world diagnostic testing data can be ascertained and organized to reflect the complexity of the journey of the patients with rare diseases. SOLVE Framework will improve the accuracy and certainty associated with value-based assessments of genomic sequencing.
The clinical application of genome-wide sequencing for monogenic diseases in Canada: Position Statement of the Canadian College of Medical Geneticists.
inadequacies in the evidence related to its clinical utility and cost-effectiveness, combined with limited coverage offered by public and private payors, hinder its implementation into routine care in multiple jurisdictions.
Being precise about precision medicine: what should value frameworks incorporate to address precision medicine? A report of the Personalized Precision Medicine Special Interest Group.
Value Health.2020; 23 (Published correction appears in Value Health. 2020;23(11):1522): 529-539
Broadly, barriers relate to defining health and nonhealth outcomes attributable to genomic sequencing itself, defining appropriate time horizons for key outcomes, measuring family cascade effects of sequencing and incorporating them into traditional health technology assessment frameworks, and constructing robust comparative study designs in the context of small, heterogeneous patient groups. Even when robust design parameters can be established, the rapid evolution of sequencing platforms often outpaces the time required to complete required evaluative studies. As a result, studies often rely on less robust evidence of value that is hypothetical or contains uncertainty.
Being precise about precision medicine: what should value frameworks incorporate to address precision medicine? A report of the Personalized Precision Medicine Special Interest Group.
Value Health.2020; 23 (Published correction appears in Value Health. 2020;23(11):1522): 529-539
The use of real-world evidence from patient medical records, registries, surveys, and observational studies presents a practical and robust alternative.
Use of real-world evidence in US payer coverage decision-making for next-generation sequencing–based tests: challenges, opportunities, and potential solutions.
Care4Rare-SOLVE (SOLVE) is a Genome Canada funded initiative that aims to (1) use new sequencing technologies to establish the molecular etiology of currently unsolved rare diseases and (2) generate evidence of the clinical utility and cost consequences of clinical exome sequencing (ES) to inform adoption and reimbursement decisions across provincial ministries of health in Canada. SOLVE includes a prospective cohort study in 2 Canadian provinces collecting data related to pre-ES diagnostic testing and post-ES medical management to generate evidence of clinical utility and cost consequences.
The objective of this substudy was to use an expert-driven consensus process and real-world data from medical genetics practice to establish a structure for organizing a complex set of diagnostic investigations for patients with rare diseases. This structure can be used to develop comparator groups to facilitate clinical utility and cost consequence analyses related to ES at different points in the diagnostic journey of patients with rare diseases.
Materials and Methods
Setting and sample
The target sample for the SOLVE prospective cohort study is 650 participants from 12 genetics clinics in Ontario and Alberta. Recruitment began in January 2019. To facilitate recruitment, consulting geneticists or subspecialists with expertise in genetics notify eligible families about the study during clinical consultations. A research coordinator follows up with interested families to provide further study details and complete the informed consent process. To be eligible for the cohort study, participants must have 2 or more of the following: (1) moderate to severe developmental or functional impairment, (2) multisystem involvement, (3) progressive clinical course that cannot be explained by another cause, (4) a differential diagnosis that includes 2 or more conditions that would require evaluation by separate gene panels, or (5) a suspected severe genetic syndrome for which multiple family members are affected or parents are consanguineous. Individuals with the following clinical indications were not eligible for enrollment: (1) isolated mild intellectual disability (ID) or learning disabilities, (2) nonsyndromic autism, (3) isolated neurobehavioral disabilities (eg, attention deficit disorder), or (4) isolated neuropsychiatric conditions (eg, schizophrenia, bipolar disease, Tourette syndrome). To achieve the objective of the substudy described in this paper, we used data collected from the first 228 SOLVE participants.
Data collection
We developed a data collection tool called the PhenoTips Care Pathway
that captures the following: (1) patient characteristics, (2) dates of referral to Genetics, (3) pre-ES diagnostic testing, (4) ES results, and (5) post-ES medical management recommendations. We focused on data pertaining to participants’ pre-ES diagnostic investigations and participants’ clinical characteristics (ie, clinical presentation when ES was ordered, age of symptom onset) to inform an organizing structure to support the development of latent comparator groups for future analyses of clinical utility, effectiveness, and cost-effectiveness. Patient characteristics collected in the Care Pathway include the clinical subgroup (ie, syndromic ID, isolated ID, multiple congenital anomalies without ID, multisystem disorder without ID, single organ disorder without ID) and age of symptom onset (ie, prenatal, birth, childhood, adolescence, adulthood). The pre-ES diagnostic investigations component of the Care Pathway was developed with input from 2 medical geneticists with specific expertise in rare diseases and included (1) DNA-based testing options (ie, cytogenetic, molecular) and (2) non-DNA–based testing options (ie, biochemistry, pathology, imaging, and electrical activity). For DNA-based testing options, structured response options included karyotyping, florescent in situ hybridization (FISH), microarray, single gene and gene panel tests. For FISH, single gene test, and gene panel test, specific gene or test names were entered into the Care Pathway as free text.
For non-DNA–based testing options (ie, biochemistry, pathology, imaging, and electrical activity), structured subcategories and response options were included for each test category. For biochemistry, structured subcategories included basic biochemistry tests (12 response options), small molecule disorder tests (13 response options), mitochondrial disorder tests (8 response options), lysosomal disorder tests (5 response options), and peroxisomal disorder tests (7 response options). For pathology tests, 5 structured response options were provided. For imaging, structured subcategories included magnetic resonance imaging (MRI; 5 response options), computerized tomography (5 response options), X-ray (4 response options), ultrasound (4 response options), and 5 other structured response options (ie, bone age, bone density, echocardiogram, magnetic resonance spectroscopy of the brain, and skeletal survey). For electrical activity tests, 6 structured response options were included. After each set of structured response options for each subcategory, an unstructured “other” response option was included where diagnostic tests identified in medical records that did not fit into a structured response option could be entered as free text. Test type and the date the test result was reported by the laboratory were recorded in the Care Pathway.
All unique diagnostic investigations reported in the Care Pathway database, including those that were entered into structured and unstructured response options, were extracted for this substudy. Data entries were audited quarterly by a central auditing team. Each entry was verified according to the criteria related to completeness, accuracy, and consistency with data entry instructions. Where discrepancies were identified, the source record was re-reviewed by the audit team and corrected. Data entry personnel at the source site were notified of the correction(s) to minimize future discrepancies. The list of observed pre-ES diagnostic tests served as our core data source for the expert-driven consensus building process.
Expert-driven consensus process
Three clinical experts from the SOLVE leadership team, located in the largest clinical recruitment sites in Ontario and Alberta, participated in the consensus building process. Experts were clinical geneticists who have been in practice for >15 years and have specific expertise in rare disease diagnosis. As outlined in Figure 1, this process involved 5 steps. Step 1 involved gathering Canadian guidelines specific to diagnostic testing algorithms for rare diseases.
Early identification of treatable inborn errors of metabolism in children with intellectual disability: the Treatable Intellectual Disability Endeavor protocol in British Columbia.
From these guidelines, we generated a list of standard diagnostic tests for individuals with unexplained developmental delay or treatable ID. This comprised the Guideline Framework and served as the reference point for our observed data, organized as an inventory-unique diagnostic tests included in our database.
Figure 1Expert-driven consensus process to organizepre-ES diagnostictesting data.
Because this test inventory reflected more extensive diagnostic testing than the tests included in the Guideline Framework, steps 2 to 5 invoked a consensus process to establish and validate a data-driven SOLVE Framework to better characterize the real-world complexities of the diagnostic journey for our cohort. In step 2, we presented the inventory of observed pre-ES diagnostic tests to our clinical experts. During videoconference 1, we asked experts to examine the list of tests to assist with defining test categories. To guide their thinking, we constructed de-identified case vignettes from our data set to show diagnostic journeys of participants in each of the 4 clinical subgroups and with various ages of symptom onset. Each vignette represented a participant’s pre-ES diagnostic test sequence in chronological order.
To categorize the tests, experts distinguished between indicator and nonindicator tests. Indicator tests were defined as those with high specificity for diagnosing rare diseases and as such were likely to contribute specific information toward achieving a clinically valid molecular diagnosis. These included tests that were specific to characterizing both the underlying genetic etiology of the disease and defining important components of the associated phenotype. Nonindicator tests were defined as those performed as a routine part of a diagnostic work-up for a patient referred for evaluation of a rare disorder. To further differentiate indicator and nonindicator tests, clinicians articulated that indicator tests are typically of higher cost, potentially invasive, less accessible, and ordered/interpreted by a subspecialist and that nonindicator tests are typically of lower cost, noninvasive, locally accessible, and ordered/interpreted by a generalist (Table 1).
Table 1Definitions and criteria for indicator and nonindicator tests
Indicator Tests
Nonindicator Tests
Definition
Diagnostic tests likely to contribute specific information toward achieving a clinically valid molecular diagnosis of a rare disease
Diagnostic tests performed as a routine part of a diagnostic work-up for a patient referred to clinical genetics
Criteria
High cost, invasive, less accessible, and ordered/interpreted by a subspecialist
Low cost, noninvasive, locally accessible, and ordered/interpreted by a generalist
Once test category definitions were established, we provided each expert with an excel worksheet and asked them to independently assign each test in the inventory to a test category (ie, indicator vs nonindicator). In making these assignments, we asked them to consider the most common rare disease indication for testing in our data set as a starting point (ie, ID). The excel worksheet included the test inventory in one column, their category assignment in a second column, and notes to reflect their thinking on category assignment in a third column to use at their discretion. The completed worksheets were returned to the study team. We assessed agreement among the 3 experts and resolved discrepancies through discussion during videoconferences 2 and 3. We asked experts to complete this task once for the tests reported in the structured section of the Care Pathway (step 3) and a second time for the tests reported in the unstructured (ie, free text) section (step 4). Once consensus (ie, complete agreement) was achieved on test categorizations for the SOLVE Framework, we convened videoconference 4 to assess the face validity of this Framework (step 5). To facilitate step 5, we constructed a final set of case vignettes using our data set in which the pre-ES testing journey was presented according to the Guideline and SOLVE Frameworks. The vignettes were constructed to represent the range of clinical subgroups and ages of symptom onset observed in the data (Figure 2, Supplemental Figure 1). Clinicians were asked to endorse or refute their test category assignments after reviewing the case vignettes, enabling active consideration of the role of clinical context in making test category assignments. Given the multistep and iterative nature of this consensus process, a standard Delphi method
Figure 2Vignette-based illustration of pre-EStesting according to the Guideline andSOLVE Frameworks. The arrows connect a test date (M-D-Y) to the name of the test itself contained within a cluster of tests. Each cluster of tests is represented by a box and each cluster is triggered by the presence of an indicator test. The size of the box reflects the time period over which the cluster of tests was recorded in the medical record. Age of onset and actual test dates are not presented to maintain participant anonymity. This pathway represents a female with syndromic intellectual disability (moderate-severe) and multi-system involvement. ALT, alanine aminotransferase test; AST, aspartate aminotransferase test; CBC, complete blood count; Chem, Chemistry test C; CK, creatine kinase; CMA, chromosome microarray; CT, computerized tomography; ECG, electrocardiogram; ES, exome sequencing; MRI, magnetic resonance imaging.
We used frequency counts and descriptive statistics to summarize the characteristics of the participants. These included participant sex, age at clinical presentation at the time of ES, age of symptom onset, duration of pre-ES testing period, and number of diagnostic tests pursued during this period. To characterize the SOLVE Framework, we used frequency counts to summarize the number of tests in the structured and unstructured data sets that required categorization, the number of discrepancies identified in each of these sections of the data set, and the final number of tests in each category. We audio-recorded the videoconference calls and reviewed the audio files to verify definitions of test categories and test assignments.
Results
Patient characteristics
Table 2 summarizes the characteristics of the 228 participants. The sample was almost evenly split between males and females (51.8% and 48.2%, respectively) and the mean age at the time ES was ordered was 9.6 years (SD = 10.9). The largest proportions of cases had symptom onset at birth (42.5%) or during childhood (43.4%) and most cases had ID (73.3%). On average, the pre-ES diagnostic testing journey occurred over 1989 days (SD = 2137, range = 0-18,393) or approximately 5.5 years and included 16 diagnostic tests (SD = 14, range = 0-97). On average, 4 tests were classified as indicator tests and 12 were classified as nonindicator tests per case (Table 2, Supplemental Figure 1A and B).
All of the diagnostic tests that were included in the Guideline Framework were present in our data set. In this Framework, indicator tests included chromosomal microarray, 49 unique single gene tests, and 51 unique gene panel tests. Nonindicator tests included 10 unique FISH tests, 7 unique metabolic tests, brain MRI, electroencephalogram, echocardiogram, and skin and muscle biopsies.
SOLVE Framework
Table 3 presents a summary of the test categorization process that was used to develop and validate the SOLVE Framework. Of the 176 tests derived from the structured section of the Care Pathway that experts were asked to classify as indicator or nonindicator tests in step 3, agreement on classification was achieved for 96.0% of tests before discussion. After the videoconference discussion of discrepancies, 147 tests were defined as indicator tests and 39 were defined as nonindicator tests. Of the 195 tests derived from the unstructured section of the Care Pathway that experts were asked to classify in step 4, agreement was achieved for 82.6% of tests before discussion. After videoconference discussion of discrepancies, 39 tests were defined as indicator tests and 146 tests were defined as nonindicator tests. Clinicians agreed that 3 biochemical tests (ie, urine purines and pyrimidines, glycosaminoglycans, and ceruloplasmin) and 10 FISH tests that were assigned to the nonindicator test category in the Guideline Framework should be assigned to the indicator test category for the SOLVE Framework. Ultimately, tests from the unstructured data set that were classified as nonindicator tests were removed from the final SOLVE Framework because they were neither identified as essential to the diagnostic journey at the time of database construction nor during the consensus process. As such, the SOLVE Framework included 225 tests, in which 186 were categorized as indicator tests and 39 as nonindicator tests. After reviewing vignettes that showed examples of indicator and nonindicator tests associated with Guideline and SOLVE Frameworks (Figure 2) in step 5, experts confirmed that test categorizations remained appropriate. No changes in test categorizations were made when a range of patient clinical presentations and age of symptom onset were presented in the vignettes. Supplemental Table 1 presents the final Guideline and SOLVE Frameworks, each comprising a set of specific indicator and nonindicator tests organized into the following test categories: molecular/cytogenetic, biochemistry, imaging, electrical activity, and invasive pathology.
Table 3Summary of consensus process and test categories for the SOLVE Framework
Our real-world evidence and expert-driven process for classifying diagnostic tests among individuals with rare disease reveals the complexity of this aspect of their care. For a sample of 228 cases, most of whom had ID with symptom onset at birth or during childhood, the pre-ES diagnostic testing journey occurred, on average, over a 5.5-year period and included 16 targeted diagnostic tests. There is significant variability in both duration of the testing period and number of tests, highlighting the heterogeneity of this patient population. Despite this complexity and heterogeneity, our data enabled us to articulate guideline and data-driven frameworks as a first step toward devising latent comparator groups to conduct economic analyses.
Our findings are both similar to and different from related literature. Collaborators at our institution conducted a retrospective analysis of diagnostic testing pursued by 63 children enrolled in a structured complex care program.
Similar to the findings of this study, the median duration of the testing period was 4 years. Although only genetic tests were included in that analysis (ie, metabolic, imaging, and physiological activity tests were excluded), a median of 6 tests was observed. In a retrospective review of data from a Dutch cohort of 50 patients presenting with complex pediatric neurologic disorders of suspected genetic etiology,
researchers found that, on average, patients made 16 physician visits and underwent 30 diagnostic tests (ie, 4 imaging, 2 neurophysiologic tests, 8 genetic tests, 16 other diagnostic tests). In this cohort, the mean duration of the diagnostic journey was 40 months. Although higher in mean number of tests and shorter in length of testing period than those observed in our cohort, the range of test types in the Dutch cohort was more limited than that identified in our cohort. Finally, in a Utah-based retrospective review of data from 64 children with inherited leukodystrophies, diagnostic tests, including brain MRIs and disease-specific tests (eg, blood chemistry, hemoglobin, leukocyte lysosomal enzymes, chromosome karyotype, very long-chain fatty acids, and Pelizaeus-Merzbacher gene testing), were tabulated over an 8-year period.
On average, patients in that cohort had 20 tests and 2.5 brain MRIs each. Although this cohort reflected the diagnostic testing pathway for a single clinical indication (ie, inherited leukodystrophies), it is similarly complex as those observed in our heterogeneous cohort of undiagnosed complex rare diseases.
In addition to characterizing the complexity of the real-world diagnostic journey for patients with a rare disease, we developed the SOLVE Framework to represent a structure for organizing diagnostic tests that retains the complexity and attends to the nuances of establishing a clinically valid molecular diagnosis for this population. First, we established a fundamental distinction between indicator tests (ie, specific to achieving a clinically valid molecular diagnosis of rare diseases) and nonindicator tests (ie, commonly ordered tests for patients referred for genetic evaluation). In doing so, we distinguish between diagnostic tests that carry higher predictive value for rare disease diagnosis from those that are more routine. Characterizing this distinction can assist with efforts to optimize testing algorithms for this patient population because testing algorithms incorporate clinicians’ thinking about the predictive value of available tests. Although counting and costing all tests pursued for a given cohort is essential to understanding the patient and system-related intensity and economic impact of the diagnostic journey, assuming that all tests contribute equally to the desired outcome (ie, a clinically valid diagnosis) may result in an oversimplification of the testing pathway. This oversimplification may compromise clinical recommendations and policy decisions related to testing algorithms for rare diseases, including the optimal positioning of tests such as ES. Determining the optimal positioning of ES is fundamental to health technology assessment aimed at assessing value of alternative testing strategies for funding and policy decision-making in a range of jurisdictions considering the adoption of ES and genome sequencing (GS);
the SOLVE Framework offers an essential analytical tool for such value assessment.
Building on the organizing framework we developed for indicator and nonindicator tests, our findings highlight differences between testing pathways that have informed previous cost-effectiveness analyses of ES and GS (ie, using Guideline Frameworks)
Being precise about precision medicine: what should value frameworks incorporate to address precision medicine? A report of the Personalized Precision Medicine Special Interest Group.
Value Health.2020; 23 (Published correction appears in Value Health. 2020;23(11):1522): 529-539
and testing pathways that reflect real-world clinical practice (ie, using the SOLVE Framework). The Guideline Framework is exemplified by a recent health technology assessment of ES and GS conducted by collaborators in Ontario, Canada. In that study, a discrete event simulation model was used to determine the cost-effectiveness of ES and GS for patients with unexplained developmental delay and congenital anomalies compared with standard care.
Early identification of treatable inborn errors of metabolism in children with intellectual disability: the Treatable Intellectual Disability Endeavor protocol in British Columbia.
± brain MRI) and, when negative, tier 2 investigations (ie, single gene testing, gene panel testing). The volume of use of nongenetic tests in standard testing was based on expert opinion and assumed to be the same in all comparators. The use and costs of genetic tests in standard testing were varied on the basis of real-world data from a retrospective analysis of diagnostic testing pathways in children in a structured complex care program.
The authors found that if ES was used as a second-tier test (after first tier chromosomal microarray failed to provide a diagnosis), it would be less costly and more effective than standard testing because of its ability to avert an extended and costly series of single gene and gene panel tests and to identify more molecular diagnoses.
In this hybrid approach, a hypothetical model was constructed on the basis of guidelines and expert-driven definitions of standard care and real-world data to inform costs of genetic testing and clinical utility. The SOLVE Framework included up to 186 indicator and 39 nonindicator tests in its standard (ie, pre-ES) pathway; the range of plausible tests and the relative predictive value of each suggests a complex standard care pathway. As such, using the SOLVE Framework as a basis for a standard testing pathway comprising genetic and nongenetic tests in clinical utility and cost consequence analyses for introducing ES may provide a more realistic reflection of routine clinical practice.
Alongside these contributions, we acknowledge several limitations. First, the data extraction required to generate the inventory of tests used to create the SOLVE Framework was resource intensive, requiring the review of long testing histories and extensive data entry training and auditing. This may have resulted in data entry or coding errors. Machine learning approaches to create synthetic data
that reflect the test classification system embedded in the SOLVE Framework warrants further study. Second, the distinction between indicator and nonindicator tests relied upon expert opinion and was specific to a largely pediatric population with rare diseases in Canada and ordering practices that reflect currently available tests. In addition, most cases reflected testing pathways that were focused on establishing an etiologic diagnosis for phenotypes inclusive of ID. Although the final SOLVE Framework was derived from observed data and achieved face validity in our study context, its face validity in other populations with rare diseases, in other jurisdictions, and over time (ie, as diagnostic testing technologies and norms change) will warrant assessment in the future. Given the importance of further validation, we acknowledge the presence of some uncertainty in the current classification scheme. Finally, although generating this Framework was motivated by the desire to establish comparator groups to facilitate planned analyses related to the clinical utility and cost consequences of genome-wide sequencing, additional parameters (ie, decision rules related to test clusters and test intervals) need to be applied to this Framework to generate actual comparator groups. Limitations notwithstanding, planned analyses of clinical utility and cost consequences of ES, using the SOLVE Framework, are underway. For example, a model has been designed to simulate patients’ test trajectories and evaluate the costs associated with achieving a diagnosis and changing management by performing ES at different timepoints. Each trajectory will be structured according to the indicator tests (ie, events) and nonindicator tests (ie, latent intervals) established in the SOLVE Framework.
In conclusion, the real-world evidence approach we used showed that diagnostic testing data for patients with rare disease can be ascertained and organized in a way that reflects the true complexity of this journey. Informed by a robust sample of 228 patients with rare diseases from multiple clinical settings, the SOLVE Framework will improve the accuracy and the certainty associated with value-based analyses of genome-wide sequencing.
Data Availability
The data that support the findings in this study are available within the published article and Supplemental Table 1 and Supplemental Figure 1. Identifiable participant information are not publicly available in accordance with research ethics policies. For computer code availability, please contact the authors directly.
Conflict of Interest
The authors declare no conflict of interest.
Acknowledgments
This work was performed under the Care4Rare Canada Consortium funded by Genome Canada and the Ontario Genomics Institute (OGI-147), the Canadian Institutes of Health Research, Ontario Research Fund, Genome Alberta, Genome British Columbia, Genome Quebec, and Children’s Hospital of Eastern Ontario Foundation. Deborah A. Marshall was supported by a Canada Research Chair (Health Systems and Services Research) and the Arthur J.E. Child Chair in Rheumatology. We acknowledge all study team members involved in participant recruitment and data entry. We thank Trevor Seeger for his statistical support.
This study was reviewed by Clinical Trials Ontario (1577) and the Research Ethics Board at the University of Calgary (180-0744). Informed consent was obtained from all participants as required by Research Ethics Board and Clinical Trials Ontario. In addition, all individual-level data was de-identified.
The clinical application of genome-wide sequencing for monogenic diseases in Canada: Position Statement of the Canadian College of Medical Geneticists.
Being precise about precision medicine: what should value frameworks incorporate to address precision medicine? A report of the Personalized Precision Medicine Special Interest Group.
Value Health.2020; 23 (Published correction appears in Value Health. 2020;23(11):1522): 529-539
Use of real-world evidence in US payer coverage decision-making for next-generation sequencing–based tests: challenges, opportunities, and potential solutions.
Early identification of treatable inborn errors of metabolism in children with intellectual disability: the Treatable Intellectual Disability Endeavor protocol in British Columbia.
05383-1&id=gr1.jpg){kind=link}
05383-1&id=gr2.jpg){kind=link}