If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Correspondence and requests for materials should be addressed to Gabriel Recchia, Centre for Mathematical Sciences, Wilberforce Rd, Cambridge CB3 0WA, United Kingdom.
Winton Centre for Risk and Evidence Communication, Department of Pure Mathematics and Mathematical Statistics, University of Cambridge, Cambridge, United Kingdom
Winton Centre for Risk and Evidence Communication, Department of Pure Mathematics and Mathematical Statistics, University of Cambridge, Cambridge, United Kingdom
Winton Centre for Risk and Evidence Communication, Department of Pure Mathematics and Mathematical Statistics, University of Cambridge, Cambridge, United Kingdom
Our objective was to apply a user-centered design process to identify phrases, graphics, and ways of communicating numerical risks that could be used to help patients understand their cancer risk and next steps on receiving BRCA1 genetic test results (positive, negative, and variants of uncertain significance).
Methods
The first phase of the study, a user-centered design process, consisted of 4 rounds of interviews (N = 42, including 13 health care professionals and 16 patients having undergone BRCA testing). The second was a randomized, between-participants experimental study of 456 United Kingdom residents that compared the resulting reports to reports used in a United Kingdom national genomic laboratory hub. Outcomes were subjective and objective comprehension, communication efficacy, actionability, and perceived risk.
Results
Subjective comprehension, communication efficacy, and actionability were all higher for the user-centered reports, with no difference in perceived risk. Comprehension of participants viewing user-centered reports was significantly better on 2 items, directionally (but not significantly) better on 6 items, and directionally (but not significantly) worse on 2 items.
Conclusion
Our results imply that user-centered design is a promising approach for developing materials about complex genetic risks. We suggest wordings that are likely to lead to improved comprehension when communicating BRCA-associated risks.
For genetic testing to deliver maximum benefit to patients, results must be acted on appropriately by patients and clinicians and disseminated accurately through families. Across many state and private health care systems, genetic testing is being increasingly embedded in mainstream testing pathways. As a result, for an increasing number of tests, including gene panels for breast and ovarian cancer,
Letter comments on: mainstreaming informed consent for genomic sequencing: A call for action: A single-centre comparison of mainstreamed genetic testing with a specialist genetics service—focus on consent discussions.
clinicians without genetic specialty training are expected to order tests directly instead of referring patients to a clinical genetics service. Unfortunately, a lack of specialized training and experience in communicating genetic risk means that many ordering clinicians do not feel they have the knowledge or confidence to communicate test results to patients effectively.
A substantial body of literature over the past 2 decades suggests that these concerns are often well founded. For example, in a survey primarily of US obstetricians/gynecologists, 40% of whom had personally provided BRCA1/2 genetic counseling, only 81% recognized that BRCA1/2 pathogenic variants can be inherited via the male line.
In another survey, only 60% of oncologists recognized that a healthy woman whose sister has an inherited BRCA1 pathogenic variant has a 50% chance of having inherited the same variant.
for a summary of research on primary care physicians’ genetics knowledge and examples of poor patient outcomes resulting from a non-genetically trained medical provider giving inaccurate interpretations of genetic test results, and Ha et al
shared anonymized patient genetic test reports from 2 United Kingdom (UK) genetics laboratories with UK breast cancer specialists and asked them what actions they would take. One of these reports referred to a variant of uncertain significance (VUS) as "not previously reported" and "significance unknown" but did not provide patient management guidance. For this report, only slightly more than half (54%) of specialists indicated communication/management strategies classified as appropriate (such as "refer patients to a genetics consultant"); 39% of responses were classified as "don’t know" and 7% as "inappropriate". In contrast, after reading a VUS report from a laboratory that provided management guidance within the report, 94% chose an appropriate strategy. A US-based study surveying physicians about their communication/management strategies in vignettes about hypothetical patients with a BRCA1 VUS found that only 15% selected the most appropriate answer in all 3 scenarios.
“Not pathogenic until proven otherwise”: perspectives of UK clinical genomics professionals toward secondary findings in context of a Genomic Medicine Multidisciplinary Team and the 100,000 Genomes Project.
Some patients likewise fail to fully understand the implications of their results, and BRCA test results present particular challenges. Although patients’ understanding of positive and true negative results is generally good, understanding of uninformative negatives is often poor.
A systematic review of the effects of communications about VUS and uninformative negatives in cancer genetic counseling found that in most studies investigating recall and understanding, "counselees’ interpretation and understanding of a VUS was difficult or incorrect... Only a few studies found that most counselees correctly interpreted a VUS or understood an inconclusive test result".
Across different cultural settings, the perception that a test result has no implications for family members and the perception that family members would not understand the results present barriers to result sharing within the family.
A recent review of patient-facing materials relevant to precision medicine, with a focus on genetic test result reports, concluded that “the breadth of knowledge in this field is limited in scope”,
and called for more research involving patients in the development of patient-facing materials.
The object of this study was to determine whether a user-centered design process involving clinicians, patients, and members of the public would result in reports of BRCA panel test results that are comprehended better, and are rated as more actionable and effective at communicating key points, than those currently used in the UK National Health Service. We hypothesized that an alternative document designed using recommended practices for patient-facing genetic reports
and refined through user-centered design would have advantages in terms of actionability, comprehension, and communication efficacy.
Exploratory findings from previous work on p.Phe508del reports found that participants were much more apt to describe the probability that a child would have cystic fibrosis as "likely" if they had seen a standard positive report vs a positive report developed with patient input, despite no differences in the numeric probabilities that the 2 report formats assigned to this outcome.
We speculated this was the case because of language in the original reports that could have been perceived as indicating that something was highly abnormal, which was a concern across negative and positive reports in this study. As such, we also predicted that the level of perceived risk would be higher for participants viewing the standard reports than for participants viewing reports refined through user-centered design, across test result types.
Materials and Methods
Reports were developed through user-centered design, involving 4 rounds of semistructured interviews. Formative testing occurred throughout the development process and focused on identifying issues with the proposed reports and making adjustments to address them at the end of each round until all serious issues had been addressed. As in Recchia et al,
"serious" issues were operationalized as ones that could severely limit the participant’s ability to use the report, including issues that could leave recipients with a serious misconception (Supplemental Table 1).
User-centered design process
Participants
A total of 42 participants took part in semistructured interviews conducted online. Interviews lasted 29 to 173 minutes (69 minutes average) and took place between September 2019 and April 2020. User groups were health care providers with and without experience ordering genetic tests, individuals having experienced BRCA testing, and the general public. Participants were recruited via purposive sampling (mailing lists of charities serving individuals with experience of BRCA-associated breast and ovarian cancer, flyers at conferences), convenience sampling (social media), and snowball sampling.
Included 7 individuals with no experience of genetic testing, 1 rare disease advocate, 1 individual who had experienced non-BRCA breast cancer, and 4 individuals having family members with BRCA variants.
13
(31%)
Individuals Participating in the Randomized Experiment
Gender
Female
226
(50%)
Male
220
(48%)
Nonbinary
6
(1%)
Prefer not to say
3
(1%)
Age group
18-34 years
124
(27%)
35-54 years
154
(34%)
55+ years
177
(39%)
Education
Below upper secondary
80
(18%)
Upper secondary
207
(45%)
Above upper secondary
168
(37%)
Ethnicity
Asian/Asian British
23
(5%)
Black/African/Caribbean/Black British
5
(1%)
Mixed/Multiple ethnic groups
6
(1%)
White
410
(90%)
Other ethnic group: Arab (1), Kazakh (1)
2
(0.4%)
No response or Prefer not to say
9
(2%)
Do you have any personal experience with breast cancer, ovarian cancer, or BRCA testing?
Yes
51
(11%)
No
403
(89%)
Prefer not to say
1
(0.2%)
Numeracy (Berlin Numeracy Score)
1 (low)
221
(49%)
2 (moderately low)
148
(33%)
3 (moderately high)
33
(7%)
4 (high)
53
(12%)
Numeracy (Combined score)
1 (lowest)
61
(13%)
2
89
(20%)
3
83
(18%)
4
91
(20%)
5
65
(14%)
6
23
(5%)
7 (highest)
43
(9%)
Subjective health literacy (Health Literacy Questionnaire subscale 9)
The authors of the Health Literacy Questionnaire do not use labels such as “high” or “low” health literacy, but for context, scores of 1, 2, 3, 4, and 5 correspond to responses of “cannot do or always difficult”, “usually difficult”, “sometimes difficult”, “usually easy”, and “always easy” to questions about how difficult or easy it is to perform tasks such as confidently and accurately filling in medical forms, understanding instructions from health care providers, etc.
1 (< 1.5)
7
(2%)
2 (1.5-2.49)
28
(6%)
3 (2.5-3.49)
152
(33%)
4 (3.5-4.49)
218
(48%)
5 (4.5-5.0)
49
(11%)
Missing
1
(0.2%)
a Included 7 individuals with no experience of genetic testing, 1 rare disease advocate, 1 individual who had experienced non-BRCA breast cancer, and 4 individuals having family members with BRCA variants.
b The authors of the Health Literacy Questionnaire do not use labels such as “high” or “low” health literacy, but for context, scores of 1, 2, 3, 4, and 5 correspond to responses of “cannot do or always difficult”, “usually difficult”, “sometimes difficult”, “usually easy”, and “always easy” to questions about how difficult or easy it is to perform tasks such as confidently and accurately filling in medical forms, understanding instructions from health care providers, etc.
We conducted 4 rounds of semistructured interviews with 42 participants total (rounds 1 and 2, 9 participants each; rounds 3 and 4, 12 participants each). Reports were designed for 3 diagnostic scenarios (positive, negative, VUS), 2 predictive scenarios (positive, negative), and an incidental findings scenario, informed by best-practice guidelines.
During the semistructured interview, participants were asked to provide feedback on 2 of the reports.
Interview protocol
The interviewer started by describing a fictional scenario associated with a particular report (eg, “Let’s imagine that this is the situation that Jane is in: a family member has had a genetic test suggesting that they might be genetically predisposed to breast or ovarian cancer…”), asked the participant to carefully read the report, and stepped through the interview protocol. This began by asking the participant to summarize the result in their own words, rate their understanding of the result, and provide feedback on the comprehensibility and the amount/adequacy of information provided in each section. Feedback was also solicited on the final, technical page modeled on the standard report format used by the collaborating laboratory. The process was repeated for the second report, followed by questions about the interviewee’s preferences regarding visualizations to communicate quantified cancer risks. Finally, participants provided demographic information and were debriefed. In some cases, interview protocol questions were modified at the end of a round. Reports and interview protocols are available at https://osf.io/znfae/?view_only=0408f9b7e7c24a5a87bd306a098f1b4b.
Analysis
Detailed notes on participant feedback were taken during round 1; interviews for rounds 2, 3, and 4 were fully transcribed. After each round, perceived issues—complaints, suggestions for changes, or comprehension issues—were extracted from notes and interview transcripts. All were classified into descriptive categories identified inductively and rated for severity. The study team then discussed each issue and what changes to the report format (if any) were reasonable to implement to address it. These tasks were conducted by consensus of the first two authors, who are researchers in risk communication with experience in cognitive science and user testing (G.R.) and qualitative methods (A.C.E.L.). Ambiguous cases and cases where the best way to address an issue was unclear were referred to collaborators at a UK national genomic laboratory hub (education and training lead, lead genetic counselor, and clinical scientist), who also signed off on the reports before the randomized experiment. Finally, reports were updated in line with these modifications.
Randomized experiment
Participants
Power analysis in G∗Power, seeking 95% power for effects sized f = 0.25 (based on previous research using similar measures
) with α = 0.05, indicated that a sample size of 456 would be required, assuming Benjamini-Hochberg alpha adjustment for the tests of the 5 preregistered hypotheses. We therefore recruited 456 participants to complete an online questionnaire using the panel provider Respondi in May 2020. One participant unable to view the stimuli was excluded. Eligible participants were UK residents aged 18 or older. Quota sampling ensured that age, gender, and education levels of the sample would be proportional to the United Kingdom population. See Supplemental Figure 1 for the participant flow diagram. The reward was £2.18, which participants could choose to receive or have donated to a cancer charity.
Materials and Measures
Endpoints were objective comprehension, subjective comprehension, communication efficacy, actionability, and perceived risk. See Table 2 for objective comprehension questions, Table 3 for preregistered measure items, and Supplemental Table 2 for exploratory measure items. Subjective comprehension was measured by averaging 2 Likert items asking “How clear is the information in the report?” and “How well did you understand the information in the report?”, as in previous work.
Objective comprehension was measured with the questions in Table 2. Communication efficacy and actionability were measured with scales modified slightly from those in Recchia et al.
Perceived risk was measured with the item indicated in Table 3. As secondary endpoints, we also measured perceived risk to descendants, explicability, and worry for descendants using single items modified slightly from items in Biesecker et al
How to measure risk comprehension in educated samples.
in: Garcia-Retamero R. Galesic M. Transparent Communication of Health Risks: Overcoming Cultural Differences. Springer Science+Business Media,
2013: 29-52
Characterisation of health literacy strengths and weaknesses among people at metabolic and cardiovascular risk: validity testing of the Health Literacy Questionnaire. SAGE Open Med,
2018
Q3 - Which of the following is a recommended course of action for Jane? (She should proceed with comprehensive genetic testing for hereditary cancer syndromes; She should proceed with targeted genetic testing for the BRCA1 variant of uncertain significance; She should not proceed with genetic testing for the BRCA1 variant of uncertain significance at this time
Q4 - Which of the following is a recommended course of action for Jane? (She should consider a prophylactic oophorectomy [ovary removal surgery] to greatly reduce risk for ovarian cancer; She should proceed with ovarian cancer surveillance in the form of CA-125 [a blood test that can be used to look for early signs of ovarian cancer] and transvaginal ultrasound [an imaging scan used to examine organs and structures within the female pelvis]; She should do neither prophylactic surgery nor surveillance at this time
Q5 - Which of the following is the most accurate way to explain Jane’s result? (This BRCA1 variant is responsible for Jane’s personal history of breast cancer; This BRCA1 variant is very likely responsible for Jane’s personal history of breast cancer; This BRCA1 variant is not responsible for Jane’s personal history of breast cancer; None of the above
Comprehension questions asked ONLY of participants in positive or negative conditions
Q6 – Which of the following best describes the situation? (Jane has an alteration in the BRCA1 gene, and each of her children have close to a 0% chance of having the same alteration; Jane has an alteration in the BRCA1 gene, and each of her children have a 50% chance of having the same alteration
; Jane has an alteration in the BRCA1 gene, and each of her children have more than a 50% chance of having the same alteration; Jane and her children all have the same alteration in the BRCA1 gene; Jane does not have a known alteration in the BRCA1 gene
; I don’t know) a = correct for positive reports, b = correct for negative reports
118/195 (61%)
102/179 (57%)
.49 (n.s.)
Positive reports only
80/113 (71%)
70/106 (66%)
.45 (n.s.)
Negative reports only
38/82 (46%)
32/73 (44%)
.75 (n.s.)
Q7 – True or False: Jane now knows that her risk of getting breast cancer in the future is substantially higher than she thought it was before she received this test result. (true
) a = correct for positive reports, b = correct for negative reports in specific circumstances, c = correct for negative reports
All reports if “a woman selected at random”, “not much difference” and “hard to say” are all treated as correct for negative reports
130/195 (67%)
125/179 (70%)
.51 (n.s.)
All reports if only “not much difference” and “hard to say” are treated as correct for negative reports
114/195 (58%)
107/179 (60%)
.80 (n.s.)
Positive reports only
61/113 (54%)
64/106 (60%)
.34 (n.s.)
Negative reports if “a woman selected at random”, “not much difference”, and “hard to say” are all treated as correct for negative reports
69/82 (84%)
61/73 (84%)
.92 (n.s.)
Negative reports if only “not much difference” and “hard to say” are treated as correct for negative reports
53/82 (65%)
43/73 (59%)
.46 (n.s.)
Q9 allowed selection of multiple answers; all correct answers must be selected to be counted correct Q9 – “Which of Jane’s close relatives have a fifty-fifty chance of having an alteration in the BRCA1 gene that is associated with an increased risk of cancer (or very close to a fifty-fifty chance)? You can check as many of the boxes as you want, or none of them.”
Correct for positive reports and predictive negative reports (as vignette for predictive negative case indicated that Jane's mother had tested positive).
Correct for positive reports and predictive negative reports (as vignette for predictive negative case indicated that Jane's mother had tested positive).
15/195 (8%)
66/179 (37%)
<.001
Positive reports only
6/113 (5%)
54/106 (51%)
<.001
Predictive negatives only
2/40 (5%)
7/34 (21%)
.041
Diagnostic negatives only
7/42 (17%)
5/39 (13%)
.63 (n.s.)
Comprehension question asked of participants in all conditions
Q10 – Can a man with a BRCA alteration pass it on to his children? (yes - BRCA alterations can be inherited from men or women [correct]; only in rare cases; no; I don’t know)
66/236 (28%)
102/219 (47%)
<.001
Positive reports only
29/113 (26%)
66/106 (62%)
<.001
Negative reports only
25/82 (30%)
20/73 (27%)
.67 (n.s.)
VUS reports only
12/41 (29%)
16/40 (40%)
.31 (n.s.)
Proportions are split out in cases where different answers were correct for different groups of reports and for Q6, because only the positive user-centered reports made reference to the fact that BRCA1 variants can be inherited from men as well as women.
n.s., not significant; VUS, variant of uncertain significance.
a Correct answer.
b Correct for diagnostic negative reports.
c Correct for predictive negative reports.
d Correct for positive reports.
e Correct for negative reports.
f Correct for negative reports in specific circumstances.
g Correct for positive reports and predictive negative reports (as vignette for predictive negative case indicated that Jane's mother had tested positive).
endpoint labeled: 1 = “not at all”, 7 = “completely”
4.27
1.54
5.21
1.29
51.53
0.104
<.001
How clear are you about the next steps that you could take…
4.22
1.82
5.35
1.50
53.56
0.108
<.001
Do you feel you would have the necessary information to decide what to do next…
4.14
1.76
5.16
1.50
44.74
0.092
<.001
How certain are you about what you would do next…
4.05
1.80
5.11
1.54
46.79
0.096
<.001
Do you feel you would have the necessary professional support to decide what to do next…
4.49
1.62
5.20
1.39
25.79
0.055
<.001
How ready would you feel to take any next steps…
4.43
1.78
5.24
1.44
28.60
0.061
<.001
Perceived risk “For a woman whose BRCA1 genes are similar to Jane's, how likely is it that she will develop breast or ovarian cancer sometime during her lifetime?” Fully labeled: 1 = “very unlikely”, 2 = “unlikely”, 3 = “somewhat unlikely”, 4 = “neither likely nor unlikely”, 5 = “somewhat likely”, 6 = “likely”, 7 = “very likely”
4.82
1.32
4.78
1.47
0.21
0.000
.644 (n.s.)
See Table 2 for comprehension items and Supplemental Table 2 for exploratory measures. Ellipses (“…”) appearing in subjective understanding/communication efficacy questions stand in for the phrase “in the report” (“of the report” for communication efficacy question 1). Ellipses appearing in actionability questions stand in for “if you had received this report in real life.” Endpoint labeled scales included numeric labels above each point on the scale (1, 2, 3, etc.) with the endpoints also having a verbal label.
Our preregistration noted that objective comprehension items would be analyzed separately if objective comprehension scales for positive, negative, and VUS reports did not meet certain criteria for reliability and validity. Count variables corresponding to the number of comprehension questions answered correctly by participants viewing positive, negative, and VUS reports were constructed; these had Cronbach’s alphas of αpositive = 0.67, αnegative = 0.49, and αVUS = 0.50, failing to meet the criterion of α ≥ 0.7 laid out in the preregistration. McDonald’s ωtotals
were likewise low (0.67, 0.61, 0.61). Combining these measures into an overall comprehension index is questionable because participants in positive, negative, and VUS conditions were necessarily presented with different comprehension questions (eg, questions about the meaning of a VUS report would not be appropriate to ask participants given some other report). Nevertheless, as exploratory analyses, we analyzed these measures individually as well as combined; see Analysis for details. Reliability was higher for the other multi-item measures employed, namely subjective comprehension (α = 0.92, ωtotal = 0.92), communication efficacy (α = 0.97, ωtotal = 0.97), actionability (α = 0.93, ωtotal = 0.93), and subjective health literacy (α = 0.91, ωtotal = 0.92).
Design and procedure
The study used a balanced between-participants 2 (format: user-centered or standard) × 6 (test result: diagnostic negative, diagnostic positive, predictive negative, predictive positive, diagnostic VUS, incidental findings) factorial design. Participants were blind to condition, which was assigned by the survey software via simple randomization. Participants read a vignette providing context about the fictional recipient of the report, followed by the report that they had been randomized to receive, which they were asked to read carefully. Participants then answered the questions corresponding to the subjective comprehension measure, were given the opportunity to provide free-text comments about the report, and finally answered the questions corresponding to the remaining measures: explicability, perceived risk to descendants, worry for descendants, actionability, perceived risk, objective comprehension, and communication efficacy. In each section, a link to the report with the text “Please answer these questions with respect to the report you saw earlier. To take another look at it, you may click here” appeared at the top of the screen. Finally, the report was removed and participants were shown the result statement at the top of their report and asked if they remembered seeing it and reading it, and participants who had been shown a VUS report were tested on how Jane’s variant had been classified. The survey closed with subjective health literacy questions, numeracy questions, demographic questions, and a debrief. Ethnicity was self-reported and was collected as part of the survey’s demographic questions to assess the degree to which the sample was or was not representative of the United Kingdom population in this respect.
Analysis
The study was preregistered at https://osf.io/9z7af (OSF registration 9z7af). We preregistered five 2 (format) × 6 (test result) factorial analyses of variance (ANOVAs) (α = 0.05, two-tailed) with subjective comprehension, objective comprehension, communication efficacy, actionability, and perceived risk as dependent variables. Each ANOVA included a format × test result interaction term and used Type 3 sums of squares. Analogous ANOVAs were conducted for the exploratory measures of perceived risk to descendants, explicability, and worry for descendants.
We analyzed objective comprehension items individually using 2-proportion z-tests. As exploratory analyses, we also tested for main effects of format on the number of objective comprehension questions answered correctly by participants viewing positive, negative, and VUS reports, as well as on the overall objective comprehension index (the proportion of questions answered correctly, irrespective of which report was viewed). Because these variables represent underdispersed count/proportion data, we conducted Hermite regression, although we also conducted exploratory ANOVAs for purposes of comparison (see Supplemental Table 4 footnotes for details).
Ceiling effects among the group viewing user-centered reports caused heteroscedasticity in the variables of communication efficacy, actionability, the objective comprehension index, and the number of questions answered correctly by individuals viewing positive reports. Nonparametric Mann-Whitney tests were therefore conducted on all variables as a robustness check. Finally, all analyses were repeated for the subgroups of individuals who scored below the median of the sample on health literacy and numeracy, although the study was not powered for this. All tests were 2-sided.
Results
Demographic characteristics of participants in the user-centered design process and randomized experiment are summarized in Table 1.
User-centered design process
Key categories of feedback identified inductively were actionability and risk communication, because problems with actionability or with how risks were communicated were deemed most likely to have implications for patient safety. The remaining issue categories were additional requested content, formatting, feedback on how the information was communicated, feedback on the type/amount of information provided, wording suggestions, and feedback about aspects of the genetic testing process that were out of scope of the present study.
Actionability issues identified in the initial drafts included a lack of clarity about who to speak to about test results, a perceived lack of specific recommendations for individuals with a positive predictive test result, and confusion about next steps for relatives. Risk communication issues included initial misinterpretation of ranges, such as “Out of 100 women… 65 to 79 (65% to 79%) will develop breast cancer in their lifetime” as age ranges and a lack of meaningful context around the numbers. The most frequent complaints were about the lack of information about possible treatment pathways, modifiable lifestyle factors, and whether concerning variants had been identified in any other genes on the panel (BRCA2, PALB2); a perceived lack of clarity around next steps regarding family testing; a request for more links to charities or support groups; a dislike of the phrase “The information on this page is for health professionals”; and confusion around the meaning of the phrase “clinically significant”. Changes were made to address issues at the end of each round. One substantial change was the inclusion and refinement of a pictograph illustrating risks of breast and ovarian cancer as compared with members of the general population, which we evaluated in a separate study.
Some specific differences in wording between the initial and final reports are shown in Table 4. By the final round of interviews, the most common categories of feedback were wording suggestions, additional requested content, and feedback on how the information was communicated (eg, points about sentences that could be stated more clearly or suggestions for paragraphs to be rearranged). Some clinicians commented that nongenetics health care professionals would have difficulties understanding some information on the final page of the report, which contained technical information, underscoring the importance of clear bullet points with action items separated from the technical detail.
Table 4Example differences in wording between initial and final reports
Initial Report Wording
Final Report Wording
“Ask your doctor for a referral to a genetic counselor if one has not already been made.”
“If this is the first time you have heard about these results and you do not already have an appointment with the Clinical Genetics service to discuss these results, please bring these results to the person who requested your test and ask for a referral. You can see who requested your test at the top of this page.”
“If your relatives would like to be tested, they should ask their GP for a referral to Genetics.”
(under What This Result Means for You) “Genetic testing is generally offered in a family step by step, starting with people who are a close relative (parents, siblings and children) of someone who has the alteration or a related type of cancer. Male and female family members may wish to consider testing, as both men and women can pass this alteration to children.” (under Next Steps) “Family testing is generally made available step by step, starting with parents, siblings and children (over the age of 18). Clinical Genetics can advise on having these conversations with your family. Family members who want to be tested should see their GP to discuss a referral to the Clinical Genetics Service, and bring a copy of this report with them.”
“YOUR RESULT: Clinically significant gene alteration detected in BRCA1.”
“YOUR RESULT: An alteration was found in the BRCA1 gene. Females with this alteration have an increased risk of breast and ovarian cancer. Males with this alteration have an increased risk of breast and possibly prostate cancer.”
“If you have children, each child has a 1 in 2 (50%) chance of also having this alteration to BRCA1. Each of your full siblings (if you have any) also has a 1 in 2 (50%) chance of also having this alteration to BRCA1.”
“There is a chance your children, brothers and sisters also have an alteration to the BRCA1 gene. If you have children, each child, male and female, has a 50% (1 in 2) chance of having this alteration to BRCA1. Each of your full brothers and sisters, if you have any, has a 50% (1 in 2) chance of having this alteration to BRCA1.”
“This test did not find any clinically significant gene alterations in BRCA1, BRCA2 or PALB2. This means that your cancer diagnosis is unlikely to be associated with BRCA1, BRCA2 or PALB2. It is still possible that your diagnosis of cancer has a genetic cause, as we know that alterations in other genes can increase cancer risk.”
“This test did not find any alterations associated with an increased risk of cancer in the BRCA1, BRCA2 or PALB2 genes. This means that your cancer diagnosis is unlikely to be associated with BRCA1, BRCA2 or PALB2. It is still possible that your diagnosis of cancer has a genetic cause as alterations in other genes or combinations of genes can increase cancer risk. However, many cancers develop spontaneously and are not inherited.”
Participants viewing the user-centered reports rated them higher on subjective comprehension, communication efficacy, and actionability than participants viewing the standard reports: ANOVAs found main effects of format on each of these variables (see Table 3). No differences were found between report formats with respect to perceived risk (Table 3). There were also main effects of test result for each of these variables (Supplemental Tables 4 and 5). Tukey’s post hoc tests suggested that VUS reports scored lower than 1 or more other test result types for subjective comprehension, communication efficacy, and actionability; perceived risk varied with test result such that greater risk was perceived for results conveying the presence of a BRCA1 alteration (Supplemental Table 6). VUS reports in the standard format were rated especially difficult to understand (Figure 1), resulting in a significant interaction for subjective comprehension (Supplemental Table 4). Interactions for communication efficacy, perceived risk, and actionability were not significant (Supplemental Table 4). Participants viewing the user-centered reports demonstrated better objective comprehension on items Q1, Q9, and Q10 (Table 2) at the P < .05 level; no differences in comprehension were observed for the remaining items. Alpha adjustment for multiple testing led the difference observed for Q1 (P = .046) to become insignificant, although the need to run separate tests for every comprehension question was not anticipated when the study power was calculated.
Figure 1Means and 95% CI for subjective comprehension, actionability, communication efficacy, and perceived risk for each combination of format and test result. See Supplemental Figure 2 for analogous graphs of the exploratory measures.
Regarding perceived risk to descendants, although there was a main effect of test result and a format × test result interaction (Supplemental Figure 2, Supplemental Tables 4 and 5), there was no main effect of format (Supplemental Table 2). The pattern of differences between means suggested by Tukey’s post hoc tests was identical to the pattern observed for perceived risk (Supplemental Table 6). For explicability, there was a main effect of format, with the user-centered reports deemed more explicable (Supplemental Table 2). There was also a main effect of test result, with VUS reports rated especially difficult to explain (Supplemental Tables 4-6, Supplemental Figure 2), and an interaction suggesting that standard negative reports (predictive and diagnostic) were less explicable than their user-centered counterparts but that standard diagnostic positive reports were more explicable (Supplemental Tables 4 and 5, Supplemental Figure 2). Finally, worry for descendants also exhibited a main effect of format; user-centered reports were associated with less worry (Supplemental Table 2). Worry also exhibited a main effect of test result (Supplemental Tables 4 and 5, Supplemental Figure 2). The pattern of differences between means was again identical to that observed for perceived risk, with the exception that the difference between incidental findings and VUS reports was no longer significant (Padj = .078). The format × test result interaction was not significant (Supplemental Table 4).
Hermite regressions (Supplemental Table 7), ANOVAs (Supplemental Table 2), and Mann-Whitney tests (Supplemental Table 8) all found that the mean objective comprehension index across all reports was higher among those viewing user-centered reports (0.57) vs standard reports (0.48), and that the same was true of the mean number of comprehension questions answered correctly by participants viewing a positive report (user-centered: 4.15 of 6; standard: 3.17 of 6). They found no differences for negative reports (user-centered: 2.97 of 6; standard: 2.96 of 6) or VUS reports (user-centered: 1.55 of 4; standard: 1.17 of 4). Subdividing the data by test result suggested that the difference may have been driven by improved comprehension for the user-centered predictive positive, incidental findings, and possibly diagnostic positive reports (Supplemental Tables 5, 6, and 8, Supplemental Figure 2), although format × test result interactions were not significant (Supplemental Table 4). Subgroup analyses suggested that among individuals below the median on numeracy, the user-centered reports were associated with greater subjective comprehension, explicability, actionability, communication efficacy, and more comprehension questions answered correctly on the positive reports; among those below the median on health literacy, user-centered reports were associated with greater actionability and communication efficacy and less worry for descendants (Supplemental Table 9).
Discussion
Despite the complexities of BRCA risk communication, our findings suggest that following a user-centered design process with clinicians, patients, and the general public—as well as best practices for risk communication—has benefits. This process yielded reports that were rated as subjectively more understandable, actionable, and effective at communicating key findings than reports currently in use at one national laboratory hub, which are typical of reports across the National Health Service in structure and content. Participants viewing the new reports were more likely to correctly answer questions concerning whether BRCA variants can be inherited down the male line and which relatives are at 50% risk of having inherited a pathogenic BRCA variant. This is unsurprising, given that the standard report provided no information on the former issue and only limited information on the latter (mentioning children but not siblings). However, the high proportion who answered these questions incorrectly, combined with existing literature highlighting similar misconceptions among many patients and even some clinicians,
found that merely mentioning the fact that men could carry BRCA variants was not enough, because this was apparently overlooked or misremembered by participants; the researchers ultimately decided it was necessary to emphasize it in more than one way.
We did not find significantly greater objective comprehension on other items. That said, 72% correctly answered the question “If one of Jane’s children has inherited a BRCA1 alteration from Jane, can the other child also have inherited it?” after reading a positive standard report, and 83% answered correctly after reading a positive user-centered report (P = .046). Although this was not significant after alpha adjustment, it suggests that the wording used to communicate risk to children in the user-centered reports (“If you have children, each child, male and female, has a 50% [1 in 2] chance of having this alteration to BRCA1”) could have more clearly communicated that 50% is the risk for each child, as opposed to the exact proportion of a couple’s children who will carry the variant, more clearly than the standard wording (“The patient’s children are at 50% risk of inheriting this BRCA1 pathogenic variant”).
The results for the objective comprehension questions highlighted opportunities for further work. Although all significant differences that were observed favored the user-centered reports, the number of questions which a substantial proportion answered incorrectly suggests room for improvement. In addition, when subdividing participants’ subjective ratings by test result, user-centered reports generally outperformed the corresponding standard reports on communication efficacy and actionability, but this was not necessarily the case for the diagnostic positive reports. Comments from the final round of interviews suggested that more efforts could be undertaken to make the diagnostic positive reports more clearly relevant to the diagnostic context (“At the moment that reads like they don't have cancer already and being at risk, you haven't really incorporated the ones who've got cancer because again, the ones who've got cancer might think, well, I've got my cancer it doesn't really matter. But actually, there's still an ongoing risk with contralateral breast cancer or ovarian cancer. So, there is a risk to them as well. And that doesn't really come across very well” [P31]). The comments of one of the participants in the randomized experiment likewise suggested a misconception about whether the report recipient had cancer. The diagnostic positive reports were also the only user-centered report rated as less explicable than the corresponding report in current use, suggesting that more could be done to clarify this particular report.
Risk perception did not differ based on whether participants viewed the standard or user-centered reports. Encouragingly, for the user-centered reports, perceived risk to descendants and worry for descendants were lower for predictive negatives (true negatives) than for diagnostic negatives (uninformative negatives), in line with the level of risk to Jane’s descendants. This was not true of the standard reports. Because there is no single correct level of qualitative risk perception or worry, we did not intend these measures to speak to whether it would be helpful or harmful for the modified report to produce a higher or lower number on these measures. That said, studies suggest that cancer risk among individuals at increased risk tends to be overestimated but that BRCA-positive women tend to have reasonably accurate risk perceptions.
This suggests that the reduced worry for descendants and perceived risk to descendants among those viewing user-centered (vs standard) true negative reports seems likely to be appropriate (Supplemental Table 8), but that the implications of the reduced worry for descendants among those viewing the user-centered (vs standard) diagnostic positive reports (Supplemental Table 8) are more ambiguous and possibly negative. The fact that laypeople rather than patients were the participants in the randomized experiment greatly limits the conclusions that can be drawn with respect to risk perception.
Other limitations of this study include a lack of power for subgroup analyses, a lack of ethnic diversity in the study participants, and the fact that the study design meant that it was inherently limited to individuals with online access, meaning that there are many ways that our sample is not representative of the populations served by many genetic laboratories in the United Kingdom and elsewhere. The hypothetical nature of the situation that participants in the randomized experiment were asked to consider adds further uncertainty to how these reports would be received by patients receiving them in a real-world context.
Despite these limitations, the increases in subjective comprehension, actionability, and communication efficacy give us hope that user-centered reports may facilitate appropriate sharing of results within families. Subjective comprehension may be of particular importance in this regard: patients who feel that a result is hard to understand may be more likely to conclude that relatives will not understand the implications and be less likely to share results with relatives.
Plain-language documents that cover what the test means for the patient, risks to other family members, and clear next steps have the potential to support clinicians in appropriate management and to support patients in understanding their situation and communicating it accurately to others. Our findings imply that this is possible for BRCA panel results despite their complexities and that user-centered design can yield reports that provide clear advantages.
This work was supported by Cancer Research UK (RG86786) and the David & Claudia Harding Foundation via the Winton Centre for Risk and Evidence Communication and sponsored by the University of Cambridge. L.C. was funded by Health Education England. We also wish to extend our deep appreciation to Beverley Speight, Gemma Chandratillake, and James Drummond for the time and expertise contributed to this project, as well as all participants.
This study was reviewed by the University of Cambridge Psychology Research Ethics Committee (PRE.2018.077). Informed consent was obtained from all participants as required by the Research Ethics Committee. After viewing the participant information sheet, participants had the opportunity to ask questions before consent was obtained. Individual-level data was de-identified. The research adhered to the principles set out in the Declaration of Helsinki. We have received and archived written consent for participation/publication from every individual whose data is included.
Conflict of Interest
The authors declare no conflict of interest. G.R., A.C.E.L., and A.L.J.F. are employed by the Winton Centre for Risk and Evidence Communication at the University of Cambridge, which is supported by the David & Claudia Harding Foundation. L.C. was supported by Health Education England and is employed by East and North Hertfordshire NHS Trust. Additional funding was provided by Cancer Research UK. These bodies have no financial interest in the research and had no role in the design of the study, collection/analysis of data, or decision to publish.
Supplemental Tables S1-S9 and Supplemental Figures S1 and S2
References
Dungey M.
Dalby M.C.
Moss E.L.
Barwell J.
Letter comments on: mainstreaming informed consent for genomic sequencing: A call for action: A single-centre comparison of mainstreamed genetic testing with a specialist genetics service—focus on consent discussions.
“Not pathogenic until proven otherwise”: perspectives of UK clinical genomics professionals toward secondary findings in context of a Genomic Medicine Multidisciplinary Team and the 100,000 Genomes Project.
How to measure risk comprehension in educated samples.
in: Garcia-Retamero R. Galesic M. Transparent Communication of Health Risks: Overcoming Cultural Differences. Springer Science+Business Media,
2013: 29-52
Characterisation of health literacy strengths and weaknesses among people at metabolic and cardiovascular risk: validity testing of the Health Literacy Questionnaire. SAGE Open Med,
2018 (2050312118801250)
00745-6&id=gr1.jpg){kind=link}