This multicenter, randomized, controlled, parallel-group pilot study was conducted in routine clinical care settings across 10 German local trial sites and one central trial site for telemedical participation throughout Germany. The study compared the efficacy and safety of a newly developed self-directed d-PMT in addition to TAU with TAU alone. The study was carried out in accordance with good clinical practice (DIN EN ISO 14155:2021-05) and the Declaration of Helsinki in its latest version. The study was approved by the Ethics Committee of the Medical Faculty of the University Heidelberg, Germany as leading ethics committee on July 7, 2022. The study was prospectively registered in the German Registry of Clinical Studies (DRKS00030086). The randomized controlled trial reported in this paper was part of a larger study setting that also included two observational studies. These studies examined the effects of the application in two other cohorts: children with a suspected diagnosis of ADHD/ODD and children with ADHD receiving pharmacotherapy but experiencing residual symptoms requiring treatment. The publication of the results of these studies is in preparation.

Changes to the clinical investigational plan during the study are summarized in Online Resource 1. Recruitment was carried out during routine clinical care of the child either by the local investigator (treating physician) or by a central telemedical investigator after referral by a treating physician who provided a signed report with information about the clinical diagnosis of the child, previous/current treatments and comorbid conditions. Families received written and oral information about the study from an investigator and were free to decide on study participation. All parents/legal guardians had to give written informed consent. The children had to give at least oral assent and were additionally offered the opportunity to add their names on the informed consent form. Upon giving informed consent, families were screened for eligibility, and, if eligible, were enrolled into the study. Demographic, socioeconomic and disorder-specific baseline parameters were documented, and participants received a link to a baseline (T0) survey via email. This survey contained all rating scales used to assess the outcomes listed below. Upon completion, the investigator randomized the participants using a build-in randomization tool of the electronic data capture system. The investigator had no influence on the result of the randomization. Randomization with a 1:1 allocation ratio was carried out separately at each trial site. Allocation sequences were stratified by sex of the child and were based on permuted blocks of varying (4 or 6) length. Participants were informed about their group allocation by the investigator via email. As part of this email, participants randomized to the d-PMT + TAU group received an activation code for the d-PMT. Following activation, access to the app was valid for a total of 180 days. After 8 weeks (T1/W8), 12 weeks (T2/W12) and 16 weeks (T3/W16), all participants were again invited to complete an online survey via email for outcome ratings. At the end of each of these surveys, participants were asked to report any adverse events (AEs) to the investigator. Within three weeks after W16, a final visit with the investigator took place to capture potential further AEs as well as device deficiencies and to document therapeutic measures taken throughout the study. During this visit, children in the d-PMT + TAU group were asked whether they had noticed any changes in their parents’ behavior during the study. The baseline visit and the final visit could be conducted on site or via a telemedicine-based online visit (end-to-end encrypted). Participants were compensated for their participation in the study, and those in the TAU group were given access to the d-PMT after completing the study.

Eligible children (patients) were aged between 4;0 and 11;11 years and had a definitive clinical diagnosis of ADHD (International Statistical Classification of Diseases and Related Health Problems, 10th revision [ICD-10]: F90.x, F98.80) and/or ODD (ICD-10: F91.3) documented by the child’s treating physician (i.e., the clinician was certain that the clinical criteria for the diagnosis were met). Furthermore, children had elevated parent-rated symptoms of ADHD and/or ODD (mean score > 1.0 = 78th -89th percentile in age specific representative samples) based on the German Fremdbeurteilungsbogen (FBB)-ADHS (Symptom Check List (SLC)-ADHD; parts A and B) and/or FBB-SSV (SCL-ODD; part A) at screening [17, 18]. Parents (participants) were required to have sufficient reading and German language skills, as well as a smartphone or tablet PC with internet connection. Key exclusion criteria included (1) current or planned ADHD medication for the child within the next four months, (2) intensive behavioral therapy for the child or regular parent training both with at least two contacts per month within the last 12 months, currently ongoing or planned within the next four months, (3) another severe mental disorder of the child (e.g., intellectual disability, autism, psychosis), (4) a known severe mental disorder of the participant (e.g., severe substance use disorder, severe emotionally unstable personality disorder, psychosis) and (5) planned (partial) inpatient therapy for the child due to a mental disorder.

The d-PMT (brand name hiToco®) is a CE-marked class I medical device software according to regulation (EU) 2017/745 on medical devices (MDR). This mobile application (for smartphone and tablet) is based on the established German therapy program for children with hyperkinetic and oppositional problem behavior (THOP; Therapieprogramm für Kinder mit hyperkinetischem und oppositionellem Problemverhalten [19]) and self-help workbooks for parents that were derived from THOP [20, 21]. The d-PMT is a comprehensive, integrated and personalized program consisting of five modules: (1) psychoeducation on characteristics, causes, and further course of ADHD/ODD, (2) psychoeducation on assessment and interventions, (3) coping with parental challenges (e.g., reducing own stress, improving own self-control, improving own organizational skills, handling partner problems), (4) strengthening family resources and the parent-child relationship, (5) solving child behavior problems at home (parents select one out of four typical problem situations and are guided to analyze the problem and to develop a specific intervention). Transfer tasks aim to help parents implement the interventions developed in modules 3 to 5 at home. Depending on the feedback of the parents regarding implementation success, the parents were positively reinforced or received further support to overcome implementation barriers. In general, there were no spatial or temporal requirements for the use of the app. However, parents were advised to use the app regularly (e.g., two to three times per week).

The total usage time of the app is flexible since the usage is personalized according to the individual needs of the parents. In case of technical issues with the app or problems with the proposed interventions that could not be resolved by the app, parents had the option to seek online support from the manufacturer. In the latter case, feedback was limited to the content of the app and did not include psychotherapeutic or other therapeutic advice. All investigators and treating physicians were given access to the d-PMT to familiarize themselves with the program, but they were not required for the intended use of this self-directed d-PMT during the study. A detailed description of the d-PMT is given in Online Resource 2.

Following guideline recommendations, TAU was understood as a patient-specific therapy that was developed together with the family by the treating physician of the child, taking into account the age of the child, the severity of the symptoms, the family’s wishes, and the current care situation. As stated above, pharmacotherapy or psychological interventions were exclusion criteria. However, if indicated, the treating physician could initiate both pharmacological and non-pharmacological treatments during the study. The investigator documented previous and current treatments at baseline as well as any therapeutic measures taken during the study at the final study visit. Investigators at the central trial site did not provide TAU; this was provided by the treating physician.

The primary outcome measure was the mean item score of the 28 items on the parent-rated SCL-ADHD/ODD scale, which includes the 20 items of the SCL-ADHD (parts A and B) and the eight items of SCL-ODD (part A) [18, 22,23,24]. The combination of both scales was used to measure the externalizing problem behavior of the child. All items were rated on a 4-point Likert scale (0 = not at all, 1 = a little, 2 = to large extent, 3 = extremely). A detailed description of this measure is provided in Online Resource 3.

Parent-rated symptoms of ADHD and oppositional behavior were measured using the 20 items of the SCL-ADHD and the eight items of the SCL-ODD, respectively. Parent-rated functional impairment of the child was assessed using five items of the SCL-ADHD (part F) [17]. Parenting behavior was measured via the Assessment Scale of Positive and Negative Parenting Behavior (FPNE, German: Fragebogen zum positiven und negativen Erziehungsverhalten) [25]. Family strain was measured using the Family Strain Index (FSI) [26]. Details of these measures are provided in Online Resource 3.

Participants were advised to report any AEs and device deficiencies throughout the study and were specifically asked by the investigator at the final visit. AEs and device deficiencies were coded according to the Medical Dictionary for Regulatory Activities (MedDRA) version 27.0 and the terminology of the International Medical Device Regulators Forum (IMDRF, Annex A), respectively. Any changes in concomitant ADHD and/or ODD-specific therapies were documented throughout the study. At the final visit, the investigator rated the child’s condition using the German version of the validated Clinical Global Impression-Severity/Improvement scale (CGI-S/I) [27, 28].

App user data from participants who voluntarily consented to tracking were collected to gain insights into user behavior (N = 29). If participants opted in for tracking and– for iPhone users– the app-specific settings in the Apple iOS did not block tracking, user data could also be collected using the web-analytics platform matomo (www.matomo.org; N = 21), which was hosted by Medigital GmbH. The database was filtered for logins with a duration of at least 10 s to ignore logins that do not reflect any meaningful interaction with the program (e.g., brief reminders). Users who did not interact with the app for more than 30 min were automatically logged off. Based on the available data, the number of logins per user, the single login duration (minutes), the total usage time (minutes) and the period of use (time between registration and last observed login in days) were calculated. The completion rate of the individual training plan (ranging from 0 to 1) was calculated by dividing the number of completed articles by the number of articles that were part of the training plan as defined by the initial questionnaire.

Previous randomized controlled trials evaluating d-PMTs have observed small to moderate effect sizes for ADHD/ODD symptoms [6, 9]. Based on these results, an effect size of Cohen’s d = 0.3 was assumed for this pilot study. Thus, the sample size was calculated as 74 participants based on a two-sample t-test with a one-sided level of significance of 50% and a power of 90%. This pilot study is designed to observe with high probability a true effect in the correct direction. If an effect is observed in the incorrect direction (erroneously), it would be difficult to justify a subsequent (larger) confirmatory trial. Therefore, the sample size rationale serves as a safeguard against such misleading results. Allowing for an assumed pre-post correlation of 0.6, the preliminary sample size amounted to 48 participants. In addition, a dropout rate of 20% was assumed to account for participants with missing data over the course of 16 weeks. Thus, a total of 60 participants were needed for this study (74*(1–0.62)/0.8), i.e., 30 participants per randomized group.

Pre-specified statistical analyses were carried out using the statistical software SAS® (version 9.4). Post-hoc analyses were performed using R (version 4.4.1). In general, standard descriptive statistics for continuous and categorial variables were used to describe the study sample at baseline and to summarize AEs and further outcomes (i.e., CGI, therapeutic measures taken, usage data). In a post-hoc analysis, baseline characteristics were screened for significant differences between both randomization groups. For continuous variables (e.g., age), the two-sample t-test was used. For categorial variables (e.g., gender, educational qualification), a chi-square-test for independence was used. Analyses of primary and secondary efficacy variables were based on the intention-to-treat (ITT) population, which included all randomized participants and patients. Analyses of baseline data, secondary variables of safety (AEs, device deficiencies) and other variables were based on the safety population, which included all participants/patients who provided oral/written informed consent and were included in the study.

Inferential statistics were used to model between-group as well as within-group changes in the SCL-ADHD/ODD, SCL-ADHD, SCL-ODD, SCL-ADHD part F, FPNE and FSI scores. For that purpose, a linear mixed model for repeated measures (LMMRM) was used, with the change from baseline at each timepoint (W8, W12, W16) as dependent variable. Fixed effects were baseline value, group, time, and interaction group*time. Least squares (LS) means and the differences thereof were calculated as estimators for the within-group and between-group comparison, respectively, and displayed including the 95% confidence intervals (CI). This model contained data from all participants with at least one follow-up value. For the primary efficacy variable, sensitivity analyses were performed to investigate the robustness of the results under the not missing-at-random assumption by controlled multiple imputation [29]. Effect sizes (Cohen’s d) were calculated post-hoc based on the respective raw value for the change from baseline. For between-group effect sizes, the pooled standard deviation (SD) was used. An effect size of 0.2 was considered small, while effect sizes of 0.5 and 0.8 were considered moderate and large, respectively [30].

To address the clinical relevance of changes in the SCL-ADHD/ODD, post-hoc analyses were performed using the reliable change index (RCI) [31]. This method represents an established approach to address clinical relevance in the ADHD field [32,33,34]. The RCI indicates whether a change in an outcome variable is large enough to reflect a real change considering the precision of the respective instrument (here, the internal consistency of the SCL-ADHD/ODD). Combining this information with normative comparisons allows for classification of the observed changes. In this analysis, we used an SCL-ADHD/ODD mean item score of 1.0 as a cut-off to distinguish between a non-clinical (< 1.0) and a clinical (≥ 1.0) symptom severity, which corresponds to the 85th percentile in the normative sample [17]. This classification ultimately results in four different groups: (I) reliably recovered (reliable mean item score reduction and mean item score at W12/W16 below 85th percentile), (II) reliably improved (reliable mean item score reduction but mean item score at W12/W16 above 85th percentile), (III) no reliable change (no reliable mean item score reduction), (IV) reliably deteriorated (reliable mean item score increase). The analysis was conducted at W12 and W16 in a population restricted to children with an SCL-ADHD/ODD mean item score ≥ 1.0, as well as in the ITT population. Children with missing values at W12 and/or W16 were classified as “no reliable change”. Children classified as “reliably recovered” or “reliably improved” were considered responders, and the Pearson chi-square test was used to assess differences in the responder proportions between the d-PMT + TAU and TAU groups.