An IRB exemption was obtained for this research from the University of Michigan. The Pediatric Surgical Oncology Research Collaborative.

PSORC, was established in 2017 [8]. PSROC was originally founded in North America including US programs and a program in Toronto Ontario Canada. Currently there are 55 centers including two in Canada and one in the Netherlands with a few more in Europe working through the membership application. At the time of the meeting and for this publication PSORC had 37 centers just in the USA and Canada. A list of centers can be found at www.psorc.org. Its primary mission is to develop multi-institutional research collaborations to improve surgical outcomes for children and young adults with cancer. PSORC centers have an institutional membership. The leads are pediatric general surgeons but there are several pediatric urologists, pathologist, orthopedic oncologist and pediatric oncologists who participate and propose studies. PSORC has focused on surgical issues not easily studied by traditional cancer consortiums, such as the Children’s Oncology Group. In 2020, PSORC established a stakeholder advisory group to develop and guide research in PCOR.

Study timeline and framework

A detailed timeline for the study conceptualization, development and execution is shown in Table 1. Figure 1 is a graphic representation of the iterative and interactive methodology used for this process. study.

Fig. 1

Graphical representation of the iterative process for the study

Framework

We adapted a framework developed by PCORI for identifying and prioritizing research areas, focusing on input from parents, childhood cancer survivors, and key stakeholders [9]. A multisource five-component framework was used to develop the roadmap: (1) Education, (2) Topic Generation, (3) Gap Analysis Systematic Review, (4) Value of Information (VOI) Analysis, and (5) Peer/Stakeholder Review. (Fig. 2). Each of these components informed the subsequent steps and the development of the research agenda.

Fig. 2

Framework for the research roadmap for PSORC

Education

Education of all stakeholders focused on PCOR/CER and PCORI funding. This education was critical to developing questions that PCORI supports, which are not traditional. For example, PCORI would not fund a trial comparing one chemotherapeutic agent(s) to another but would consider a project comparing limb-sparing surgery to non-limb-sparing surgery and functional outcomes or a study that impacts adherence to treatments. This was particularly important for the medical professionals. Presentations from established and funded PCORI investigators helped elaborate a research roadmap, showcasing how other pediatric investigators had successfully developed extramurally funded research programs.

Topic generation

Topic generation involved physician and stakeholder meetings, focus groups, and word clouds. We also developed a social media (Facebook) survey, which was sent to 48 solid tumor disease and advocacy support groups. These groups were identified by stakeholders’ knowledge of the groups and a Facebook search using the terms liver tumors, renal tumors, sarcomas, and neuroblastoma. Once the survey was approved by the group’s administrator, it was posted and left up for 30 days [10].

Survey and thematic analysis

The survey consisted of 30 open- and closed-ended questions. A detailed qualitative analysis of the survey is beyond the scope of this publication and will be the subject of a separate report. Thematic analysis was performed on the results of the survey. The survey team created, sent out, and analyzed the survey prior to the conference. Qualitative analysis was used in a systematic fashion to interpret the non-numerical data from the survey. Data was organized, sorted, coded and then grouped into themes for development. Grounded theory was then used to explain the findings. The team was led by author JP, who holds a PhD in decision analysis and has conducted research in thematic and qualitative analysis. The full results of the survey will be published in a future publication. Thematic analysis, primarily used in qualitative research, identifies recurring patterns or themes within a dataset by carefully examining and interpreting the data. For example, two specific open-ended questions were “Please describe your decision-making experience for the surgery” and “What recommendations do you have for improving the surgical experience before or after surgery?” This method was used for identifying topics for the research agenda and analyzing the post-conference survey. Descriptive and thematic results are presented.

Word cloud

A word cloud session is a useful method to visualize the most frequently mentioned terms or topics from a collection of text data, like focus group discussions, open-ended survey responses, or interviews. This identified key themes, which helped to formulate preliminary questions that were eventually worked into structured questions for the value of information analysis and the peer review. This diverse input was used to involve stakeholders in the planning, agenda development, and implementation of the conference.

Systematic review and gap analysis

The systematic review was done using PubMed, Ovid and the Cochrane library. To be included in the review a paper had to be about a pediatric solid tumor in the pediatric population, a focus on surgery, have patient reported outcomes and or compare different surgical therapies or outcomes form a surgical cancer procedure.

The PubMed search terms used were - ((“patient-centered“[Title/Abstract] OR “patient centered“[Title/Abstract] OR “patient reported“[Title/Abstract] OR “quality of life“[Title/Abstract] OR “patient preferences“[Title/Abstract] OR “patient outcomes“[Title/Abstract] OR “Patient Outcome Assessment“[MeSH] OR “Quality of Life“[MeSH] OR “Patient Satisfaction“[MeSH]) AND (“comparative effectiveness“[Title/Abstract] OR “comparative study“[Title/Abstract] OR “outcomes research“[Title/Abstract] OR “Comparative Effectiveness Research“[MeSH]) AND (“pediatric“[Title/Abstract] OR “child“[Title/Abstract] OR “children“[Title/Abstract] OR “adolescent“[Title/Abstract] OR “Pediatrics“[MeSH] OR “Child“[MeSH] OR “Adolescent“[MeSH]) AND (“surgical oncology“[Title/Abstract] OR “surgery“[Title/Abstract] OR “neoplasm“[Title/Abstract] OR “cancer“[Title/Abstract] OR “tumor“[Title/Abstract] OR “malignancy“[Title/Abstract] OR “Surgical Procedures, Operative“[MeSH] OR “Neoplasms“[MeSH] OR “Oncology“[MeSH]))

The Ovid search terms were.

1.

(“patient-centered” or “patient centered” or “patient reported” or “quality of life” or “patient preferences” or “patient outcomes”).mp.

2.

Patient Outcome Assessment/.

3.

exp Quality of Life/.

4.

exp Patient Satisfaction/.

5.

1 OR 2 OR 3 OR 4.

6.

(“comparative effectiveness” or “comparative study” or “outcomes research”).mp.

7.

exp Comparative Effectiveness Research.

8.

6 OR 7.

9.

(pediatric or child or children or adolescent).mp.

10.

exp Pediatrics/.

11.

exp Child/.

12.

exp Adolescent/.

13.

9 OR 10 OR 11 OR 12\.

The Cochrane search terms were.

(“patient-centered” OR “patient centered” OR “patient reported” OR “quality of life” OR “patient preference” OR “patient satisfaction”) AND (“comparative effectiveness” OR “comparative study” OR “outcomes research”) AND (pediatric OR child OR children OR adolescent) AND (“surgical oncology” OR “surgery” OR “neoplasm” OR “cancer” OR “tumor” OR “malignancy”).

Tumor teams

In preparation for the conference. Surgeons and stakeholder leads were identified for each of the four solid tumors (Renal, Liver, Sarcoma, Neuroblastoma). These included three surgeons who were on the COG solid tumor disease group and three stakeholders (typically and patient advocate, parent and a leader of a survivor group etc.)

Value of information analysis

Value of Information Analysis is a decision-making tool. VOI analysis quantifies how much uncertainty in decision-making would be reduced if more information (from further research) were available, typically in the context of healthcare interventions or treatments. It helps prioritize research by showing where additional evidence might make the biggest impact on patient outcomes or healthcare costs. VOI analysis doesn’t directly generate research questions, but it identifies where uncertainty is most important to stakeholders. It helps you decide which research study to invest in based on the expected outcomes, patient/population benefit, feasibility, etc. The systematic engagement of multiple stakeholders, especially patients, is a critical component of topic generation. This is followed by a multistep prioritization beginning with systematic reviews and gap analysis to identify what is known, and what desirable knowledge is not known. Although these steps are depicted as linear, in practice, prioritization and reprioritization occur at all stages of the process. The integration of patient viewpoints is included at all stages of research prioritization [11].

Selection of participantsStakeholders

We identified stakeholders through multiple methods. First, we leveraged our association with the Children’s Oncology Group (COG). We contacted the head of the patient advocacy group – COG- and reached out to the patient advocate in each of the four solid organ groups (e.g., Neuroblastoma). The two PIs had prior associations with groups related to Sarcoma (RD) and renal and liver tumors (PFE), as well as organizations like the International WAGR Group. Additionally, we spoke with the leaders of the MOMcology project (momcology.org) and other solid organ tumor advocacy groups. Another method of recruitment was through word of mouth from other families. We had representatives from all four areas in the United States. All expenses for the stakeholders were covered, and those who participated in the conference development and pre-conference work were financially compensated as per PCORI policy.

Physicians. Twenty-two of the physicians’ participants were pediatric surgeons with a prior interest in pediatric oncology, were active in the PSORC consortium, and were interested in PCOR research. Two physicians were emergency medicine doctors with funding from PCORI for pediatric research. The final physician was a pediatric oncologist with a leadership position in a North American pediatric oncology research consortium.

Peer stakeholder review

After completing Steps 1–4, a set of questions was created and then underwent peer review and prioritization to form the research agenda. This list of questions was then discussed and refined to ensure clarity and eliminate duplicates. A final list of 10 questions was established. Each of the 50 participants was given four votes to choose the questions they found most interesting.