The study was approved by the Rutgers University Institutional Review Board (#Pro2023000964). All participants gave informed consent before participation. This study follows the CONSORT extension for reporting randomized pilot studies (Eldridge et al. 2016).

Development of the prototype DHGFormative research

We conducted focus groups to explore barriers and facilitators of GT uptake and elicit recommendations for the development of a chatbot to support cancer genetic education and testing. Research study staff identified patients who visited the Rutgers Cancer Institute and had been diagnosed with cancer, a PV or were family members of cancer patients. Cancer patients also referred their family members to participate, and family members were also recruited through the community and organizations that provide support for family members of cancer patients.

We conducted 8 focus groups with a total of 51 participants, comprising 3 focus groups of individuals with PV (BRCA1/2 and Lynch syndrome, n = 24), 3 groups of Black (n = 13) and Hispanic (n = 5) cancer patients, and 2 groups of relatives of Black (n = 3) and Hispanic (n = 6) cancer patients (see eTable 1 Supplement for the focus group participants sociodemographic and clinical characteristics). The primary themes discussed by the participants encompassed familial involvement in GT from both patient and relative perspectives, barriers and facilitators of GT decision-making, the mitigation of mistrust and stigma around GT, barriers (such as extra clinic appointments with genetic counselors, geographic distance) to GT uptake, and facilitators (such as perceived personal and family health benefits, personalized medical care based on results) of GT uptake.

Participants enthusiastically supported the development of a chatbot to support hereditary cancer genetic education and testing, and they provided specific recommendations for content. The key intervention features suggested included genetic literacy, measures to enhance trust and credibility, patient testimonials, understandable risk information, actionable steps for GT, and an easy user interface. The participants preferred a Rutgers-affiliated portal because of privacy concerns and cited it as trustworthy. The suggested chatbot functionalities include an accessible interface, an interculturally competent avatar, and options for text and voice communication, as well as referrals for GC, testing, and support groups. The Black and Hispanic groups proposed acknowledging the generational gap in families regarding technology, providing users with choices for receiving GT results (e.g., mail to home address, send to email address, or call with a genetic counselor), and employing social marketing strategies, such as testimonials and measures to instill trust (e.g., videos of patients with negative/positive GT result or their relatives). They also emphasized the need for multi-media elements (text, voice graphics, videos) and socially relevant content.

Theoretical framework

The Ottawa Decision Support Framework (ODSF) underpins our intervention and measurement approach (Hoefel et al. 2020a, b; Stacey et al. 2020). The ODSF provides a structure to facilitate and evaluate informed decision-making by ensuring that decisions are based on relevant knowledge and attitude alignment. Congruent with the ODSF, proactive genetic education via the DHG covers key components of typical GC, with the intent of facilitating awareness, knowledge, attitude clarification, alignment, and informed GT decisions, resulting in increased GT uptake and improved decision-making outcomes. In contrast, the lack of proactive genetic education and low uptake of GC in the EUC arm will likely yield lower GT uptake and poorer decision-making outcomes compared to the DHG arm.

Initial prototype development

The DHG, “Alex”, was designed as a structured, HIPAA-compliant guided user journey (Fig. 1) with required and optional modules. The required modules include log-in, account creation, onboarding, introduction to hereditary cancer, GT readiness, GT choice, GT result delivery, and GT family communication letter. The optional modules include educational and patient testimonial videos, frequently asked questions (FAQ), and cancer family history questionnaire. The cancer family history questionnaire was made optional because we prioritized NCCN cancer-based eligibility, which is reliably captured in the medical record, aligns with national guidelines, and has been shown to outperform family history in identifying patients with actionable hereditary cancer risk. In addition, completing a detailed pedigree early created additional participant burden, and many users lacked complete family information and preferred to decide about testing first. Additional DHG platform features include user preferences, voice chat, tabs (chat vs. questionnaire), menu option, and knowledge base. Participants spent 10.5 min (SD = 5.7) completing the pre-GT required modules and 12 min (SD = 9) on the post-GT required modules, while the optional modules varied in duration depending on user preference and level of engagement. For example, the optional educational video library includes seven videos with 4 patient testimonials, each approximately 2–4 min in length, while the optional cancer family history questionnaire could range from 10 min to 1 h, depending on the complexity and completeness of the participant’s family cancer history. Alex utilizes a Retrieval-Augmented Generation framework powered by the GPT-4 Large Language Model. The system is tailored and guard-railed for Rutgers’ needs by limiting reliance on the model’s general knowledge and instead leveraging structured, credible information sourced from national resources and vetted by the study team’s clinical experts. Users’ data are protected through multiple layers of security built into the DHG. The HIPAA-compliant DHG requires secure password-protected accounts and is hosted within Rutgers-affiliated medical infrastructure. Additionally, because of the restricted and vetted knowledge base, the DHG minimizes inappropriate data handling and prevents inaccurate responses.

Fig. 1

Digital Health Guide user journey before the user and usability testing

Alex’s functionalities are organized into two main tabs: Chat and Questionnaires. The Chat tab stores user interactions, allowing participants to review their conversation history, while the Questionnaires tab contains various assessments that become available as users progress. This section enables users to track their progress, complete questionnaires, and download personalized documents like risk summaries and family trees. There is a toggle option for participants to switch between enabling and disabling the voice chat option. A permanent Menu offers an overview of key checkpoints, aiding navigation. To enhance user experience, Alex features interactive tools like educational resources and personalized content. Participants can access videos, FAQs, and questionnaires at any point and download communication aids, such as “Questions for their Doctor” and a “Family Communication Letter,” to facilitate discussions about hereditary cancer and their GT results with healthcare providers and relatives, respectively.

Alex also facilitates GT and relays GT results. When a participant indicates that they would like GT, the DHG offers two options: proceed directly to GT or schedule a GC session with a certified genetic counselor (by phone, televideo, or in-person). The DHG notifies study staff when a patient requests GT, and a testing kit is mailed to them from a commercial lab, including instructions, consent forms, and prepaid return packaging. GC staff assist with obtaining insurance authorization or financial aid through medical assistance programs if needed. Since the study participants met the NCCN testing criteria and the lab was in-network with most health plans, all the DHG participants used their insurance coverage, and none had any out-of-pocket costs. None of the participants elected the patient‑pay (self‑pay) option of $250. The DHG alerts patients via email and/or text (depending on patient preference) when their GT results are available. GT results are disclosed by a certified genetic counselor for patients who have a PV or by mail/email for negative or VUS results. GT results are also available via the patient portal in the electronic health record (EHR) and the DHG platform. Participants received their results, a clinical summary letter, and a family sharing letter via the DHG platform as well as through postal mail and email, and were encouraged to follow up with their physician or schedule additional GC sessions if desired.

User and usability testing

The initial prototype was refined through user and usability testing. User testing sessions addressed content and design, such as clarity, formatting, and engagement, while usability testing focused on navigation, error management, and the time it took to complete a task.

Participant selection

Participants were approached through multiple recruitment pathways, including identification via the electronic health record with oncologist permission-to-contact procedures, clinician referral at Rutgers Cancer Institute sites, flyers posted in clinic settings, and outreach through online and community cancer support groups. Eligible individuals were then invited directly by email, phone, or in person to complete the screening and schedule the HIPAA-compliant Zoom or Microsoft Teams interview (eFigure 1, Supplement).

Eligibility criteria included (1) age ≥18 years, (2) ability to read and speak English, and (3) a personal history of either ovarian, breast, pancreatic, colorectal, endometrial, or prostate cancer. The recommended sample size for user and usability testing is 3–5 participants, with additional recruitment until thematic saturation is reached (Guest et al. 2006; Pernice and Nielson 2012). Saturation is reached when no new or significant information is gathered from additional data collection. Saturation was achieved for this study after 14 interviews (6 user and 8 usability).

Procedures

We employed a mixed-methods approach, combining flexible, open-ended exploration of participants’ experiences with structured, quantifiable usability testing data (eTable 2, Supplement). In-depth, semi-structured interviews utilizing think-aloud techniques captured participants’ preferences and expectations and identified platform issues. Interviews for both user and usability testing were conducted remotely using HIPAA-compliant Zoom or Microsoft Teams. Each interview was conducted individually, with one research coordinator interviewing one participant at a time. Interviews were typically scheduled for 60 to 90 min. Participants were provided with $75 gift card for participation.

Measures

The user and usability testing interview guides incorporated both open-ended and structured questions, covering participants’ health and medical history, device usage, and sociodemographic information. Quantitative measures included the Chatbot Usability Questionnaire (CUQ), which was used to evaluate Alex’s acceptability (Holmes et al. 2019). The CUQ is a validated, standardized instrument designed to assess users’ perceived usability of conversational agents. The CUQ consists of 16 items adapted from established usability frameworks, including the System Usability Scale, and evaluates multiple domains such as ease of use, clarity of responses, efficiency, engagement, and overall satisfaction with the chatbot interaction. Items are rated on a Likert scale, with higher scores indicating greater perceived usability. The CUQ has demonstrated good internal consistency (Cronbach’s alpha = 0.90) and construct validity across diverse chatbot applications and is commonly used to evaluate user experience and usability in health-related conversational systems (Holmes et al. 2019, 2023). CUQ has a benchmark score of 68 out of 100 (Boyd et al. 2022; Larbi et al. 2022). The usability testing was conducted as a facilitated, online screenshare session, during which the facilitator directly observed participants as they navigated the platform using a think-aloud approach. Task performance, navigation challenges, and any required assistance were documented in real time. All sessions were audio- and video-recorded to allow for later verification and additional detail during analysis. Participants then completed the CUQ with the facilitator present during the same guided session. Participants were encouraged to navigate the platform independently while thinking aloud. Coordinators offered support only when explicitly requested or when significant issues arose.

Data analysis

Interviews were recorded, transcribed, and analyzed using Atlas.ti 24 software. We employed the Framework Method for qualitative data analysis (Gale et al. 2013), combining inductive and deductive approaches to develop themes from participants’ experiences and existing literature. Two coders independently analyzed the transcripts to ensure reliability, followed by batch meetings to review them line by line and resolve discrepancies through consensus. When consensus was not achieved, a third coder adjudicated. Inter-rater reliability was high, with an overall percent agreement of 70.2% and Cohen’s κ = 0.96, indicating almost perfect agreement. Descriptive statistics were used to summarize quantitative sociodemographic and clinical characteristics, along with average CUQ values.

Pilot-testing of the DHG

The feasibility, acceptability, and preliminary efficacy of the DHG intervention vs. EUC were assessed among cancer survivors through a randomized pilot trial.

Participant selection

We used a proactive outreach and clinical informatics approach for recruitment. Participants were identified through the EHR. Eligibility criteria were (1) age ≥18 years, (2) the ability to read and speak English, and (3) a personal history of ovarian, breast, pancreatic, colorectal, endometrial, or prostate cancer that met national criteria for germline genetic testing (NCCN 2025a, 2025b) (eFigure 2, Supplement). Patients were excluded if they lacked internet access, had previous cancer GT, self-reported inability to use the internet independently, or had previously participated in user/usability testing. Since the data are descriptive in nature and will help inform a future definitive randomized trial, a sample size of 30 patients is sufficient for this study (Eldridge et al. 2016; Lewis et al. 2021). It would provide a margin of error of 0.167 when the true proportion is 0.3, a margin of error of 0.178 when the true proportion is 0.4, and a margin of error of 0.182 when the true proportion is 0.5.

Procedures

Eligible patients were sent a study information letter (via email, postal mail, and/or patient portal), a study flyer, and an opt-out postcard/toll-free telephone number/email address. Interested patients completed the internet eligibility survey, consent and HIPAA authorization form, and baseline survey. Following completion of the baseline survey, participants were randomized 1:1 to either the EUC or DHG arm. Randomization was implemented in REDCap using a computer-generated algorithm stratified by sex and cancer type, with block randomization (block size = 4) to maintain balance across strata. Infeasible sex–disease combinations (e.g., male breast cancer) were removed. Both arms received a clinical letter signed by the medical director of the hereditary cancer genetics program at Rutgers Cancer Institute. Participants assigned to the EUC arm were sent the letter via email, postal mail, and/or patient portal informing them of their and their relatives’ potential hereditary cancer risk. The letter emphasized the participant’s eligibility for GT, recommended scheduling a GC appointment, and included contact information for the GC clinic, including a link to the website. In contrast, DHG participants were granted access to Alex (DHG), and after completing the required genetic education modules, they could proceed directly to making a GT decision. Participants completed post-intervention surveys 1 and 6 months after receiving the EUC letter or engaging with the DHG platform. Participants received a $50 gift card for each completed survey (baseline, 1- and 6-month post-intervention surveys), and an additional $30 for completing both follow-up (1- and 6-month) surveys, for a potential total of $180 in gift cards.

Measures

The baseline survey assessed demographics and clinical characteristics. Both baseline and post-intervention surveys assessed GT knowledge, a 10-item knowledge index that measures a patient’s understanding of inherited cancer concepts covered during a pretest GC session (Cronbach’s alpha = 0.69) (Cragun et al. 2020). Additionally, the post-intervention assessment collected data on (1) GC and GT uptake measured by the proportion of participants in each study arm who received GC and/or GT, verified by EHR and lab results; (2) Decisional Conflict Scale (O’Connor, 1993), a 16-item measure assessing how informed, clear, supported, autonomous, and confident a person feels in making a GT decision, and the degree of uncertainty they experience (Cronbach’s alpha = 0.88); (3) Decisional Regret (Brehaut et al. 2003), a 5-item measure assessing the level of distress or remorse regarding their decision (Cronbach’s alpha equals 0.89); and (4) DHG acceptability for DHG arm participants using the CUQ (Holmes et al. 2019).

Data analysis

Descriptive statistics were calculated for sociodemographic and clinical characteristics. Feasibility was determined by calculating the cooperation rate (number enrolled divided by number contacted and screened eligible on the internet eligibility survey) and retention rate (number who completed the follow-up survey divided by number enrolled). Average CUQ scores (median and interquartile range (IQR)) were calculated to assess the acceptability of the chatbot for those in the DHG arm. Chi-square tests (and Fisher’s exact test, when appropriate) were used to evaluate GC and GT uptake between EUC and DHG patients 6 months post-intervention. The Wilcoxon matched-pairs signed-rank test was used to compare pre- and post-intervention GT knowledge for all patients and stratified by study arm. The Wilcoxon rank sum test was used to compare decisional conflict, decision regret, and decision satisfaction between the EUC and DHG arms. The non-parametric Wilcoxon tests were used due to the skewness of the data. All analyses were conducted using an intention-to-treat approach.