2.1 General

The objective was to assess the cost-utility of DERM when used pre-referral relative to standard care (SC). The approach was model-based consisting of a decision tree to capture differences in the number of true positives (TP), false positives (FP), false negatives (FN) and true negatives (TN) between the options (expanded below) and then Markov models at the terminal nodes to quantify the patient outcomes associated with being TP, FP, FN or TN. There were three diagnostic categories: (1) high-risk cancer (including MM, SCC and other rare cancers); BCC; and benign (including other low risk lesions, including pre-malignant lesions).

The model outputs were numbers of diagnostic outcomes (TP, FP, FN, TN), numbers of key events (GP appointments, DERM assessments, hospital referrals), costs, life-years (LY) and quality-adjusted life-years (QALY). Incremental cost-effectiveness ratios (ICER) were calculated as costs per QALY. Consistent with the NICE reference case, the perspective was NHS and personal social services, the time horizon was life-time up to a maximum of 100 years and the discount rate 3.5% per annum applied to costs and QALYs. The costs were measured in GBP and the cost year was 2023/4. A plan for the wider evaluation, including the health economic model, was prepared ahead of the commencement of the health economic analysis and is available from the corresponding author on request.

2.2 Population

This was any person in the community who attends primary care with a concern that a skin lesion might be cancer. The modelled population had an age of 61 years at entry.

2.3 Comparator (SC Pathways)

Patients with suspicious skin lesions see their GPs for assessment. There are two referral pathways if the GP decides to refer: routine referral for low-risk BCCs and urgent referral for MMs, SCCs and high-risk BCCs. Patients who do not need a referral are provided with an opportunity for a follow-up appointment with their GP after a period of 3–6 months to verify the benign diagnosis.

2.4 Intervention (DERM Pathways)

The first step is a reception triage where the eligibility for taking images is checked. The eligibility criteria are listed in Supplementary file Appendix 1. Those who are ineligible are referred to a GP appointment. For those who are eligible, a DERM image is taken, usually by a senior health care assistant, and sent for analysis by the AI system. Where DERM cannot assess the lesion, even though eligible, it is reviewed by a Skin Analytics (SA)-employed dermatologist. The SA dermatologist review can result in an urgent referral to secondary care, a non-urgent referral to community dermatologists or discharge. Patients who have been assessed by DERM are assigned a disease label (MM, SCC, BCC, IEC, AK, benign) and will either be referred for an urgent referral, non-urgent referral or discharged. However, if discharged, a SA dermatologist does a second read (2R). The outcomes of this 2R SA dermatologist review are the same as a review for ineligibility. The model considers the DERM pathway with 2R (‘DERM_2R’). It additionally hypothetically models a pathway where 2R is not used and patients are discharged without review (‘DERM_autonomous’).

2.5 Model Rationale and Consistency with DERM Value Proposition

These are set out in Supplementary file Appendix 2. It explains the need for three base-cases representing the widely differing evidence on the diagnostic accuracy of GPs in current practice. The three base cases are: BASE-CASE 1: GP accuracy research literature based, sensitivity maximal; BASE-CASE 2: GP accuracy research literature based, specificity maximal; and BASE-CASE 3: implied accuracy based on NHS routine data, particularly detection rate and conversion rate.

2.6 Decision Tree Model Structures

Owing to the complexity of the decision tree, key features of each of the three alternative strategies (SC, DERM_2R, DERM_autonomous) are described. The decision tree for SC was as shown in Fig. 1.

Fig. 1

Decision tree; standard care arms (all conditions)

In SC, GP sensitivity drives the numbers of correct and incorrect identifications of true skin cancer. GP specificity drives the numbers of correct and incorrect identifications of benign disease. The model structure separates urgent from non-urgent referrals, although as already noted these proportions were not known for SC.

A representative section of the decision tree for DERM_2R showing the arms for high-risk cancer is shown in Fig. 2.

Fig. 2

Part of decision tree for DERM_2R arms (high-risk cancers)

In DERM_2R, DERM sensitivity drives the numbers of correct and incorrect identifications of true skin cancer (only MM and SCC shown). DERM specificity drives the numbers of correct and incorrect identifications of benign disease (arm not shown in Fig. 2 but available in Supplementary file Appendix 3). As can be seen in the upper part of Fig. 2, an important feature of the model for this management option is the requirement for 2R, where the DERM label is benign and the management action would be to discharge. There are also arms to capture patients who cannot be assessed by DERM.

A representative section of the decision tree for DERM_autonomous showing the arms for benign disease is shown in Fig. 3.

Fig. 3

Part of decision tree for DERM_autonomous arms (benign)

The model is identical to DERM_2R, with the exception that the 2R arm in the uppermost DERM branch for all arms is replaced by discharge alone. This also occurs for the high-risk cancer and BCC portions of the DERM_autonomous decision tree, but is not shown in Fig. 3.

2.7 Decision Tree Parameters

These are given in Table 1. Table 1 also presents the uncertainty in parameters explored in the sensitivity analyses.

Table 1. Key parameters used in decision tree

It is worth noting that the accuracy of DERM is taken directly from the SBRI evaluations, taking the site with the least optimistic estimate as the parameter value, thus these could be considered conservative values. Sensitivity was further capped at 95% in DERM, but also GP base-case 1, so as not to exceed consultant performance on grounds of face validity. The cost of DERM is also worthy of note. The device is not priced on individual use, but rather on a price per 10,000 practice patients. The model was designed assuming a price per use could be obtained, so approximation was required by SA introducing uncertainty into the cost-effectiveness estimates. Knowing the average number of times DERM would be used if available in practices would have avoided the need for estimation.

2.8 Markov Models

There were three Markov models (M1, M2, M3) at the terminal nodes of the decision trees capturing the long-term outcomes associated with each accuracy outcome (TP, FP, FN and TN). M1 models TP patients, M2 FN patients, and M3 TN and FP patients. The last two states are combined because from year 2 onwards, when the Markov models commence, TN and FP patients are both healthy and disease free with respect to skin cancer. The cycle length of the Markov models was 1 year, with half-cycle correction applied. Patients enter the models at age 61 years. This was the value used in the original post-referral model, but this value was also consistent with data in the SBRI evaluation [11]. Concerning mortality, sex- and age-related mortality rates of the general population are taken from the UK Office for National Statistics (ONS) for 2018–2020 (Table Supplementary file Appendix 4.1).

The Markov model structures are shown in Figs. 4, 5, 6. The additional parameters are provided in Supplementary file Appendices 4.2–4.7.

Fig. 4

Markov model M1 for TP patients

Fig. 5

Markov model M2 for FN patients

Fig. 6

Markov model M3 for TN and FP patients

Patients with TN or FP results, as well as patients with BCC and patients with high-risk cancers in situ and stage Ia, are assumed to have a normal lifespan. Cancer-related mortality rates are taken from Edwards (2016) [12] (Table Supplementary file Appendix 4.2). Patients with high-risk cancers in stage Ib or higher, diagnosed and treated or undiagnosed, are expected to be at higher risk of mortality because of their cancer. Patients with unidentified high-risk cancer in stage Ib or higher are at increased risk of dying for the whole period over which their cancer remains unidentified. Patients with identified and treated high-risk cancer in stage 1b or higher are at increased risk of dying for the first 10 years after treatment; within this period, the risk is higher in the first 5 years and lower, but still higher than the general population, in the second 5 years. After 10 years they have the same mortality risk as the general population. The excess risk of mortality in patients with cancer is calculated by subtracting sex- and age-specific mortality of the general population from the annual mortality risk associated with the patient’s stage of disease.

Concerning progression in patients with unidentified cancer, the annual risk of progression from undiagnosed melanoma is taken from Wilson (2013) [13] (Supplementary file Appendix 4.4). The assumption is that 10% of patients with undiagnosed high-risk cancer will be opportunistically detected (regardless of the stage), 10% will progress to the next stage (e.g. stage I to II, or stage II to III) and 5% or 10% of patients will progress within stage (e.g. Ia to Ib), whilst most of the patients (70% or 80%) will remain in the same stage. We could not find data for the progression of BCC, but it is generally noted to progress slowly [14]. It was thus assumed that patients with BCC who were missed in the initial assessment do progress to a more advanced form of the disease but that this does not change the patient’s mortality risk but does require more invasive and expensive surgical treatment; the annual probability of opportunistic detection of such patients is assumed to be 20%.

Concerning costs in the Markov model (Supplementary file Appendices 4.5 and 4.6), most of the costs are incurred in the first year after presentation and are captured in the decision tree. Exceptions are the costs related to: patients with false negative results, as they will return, most likely with more advanced disease, and will incur additional diagnostic and treatment costs, and patients with true positive results for high-risk cancers, as they are at higher risk of dying, which means that in their final year they will incur costs related to the treatment of terminal cancer. For convenience, the follow-up costs incurred by patients in stage 1b or higher within 5 years after treatment are discounted and captured in the decision tree.

In all scenarios, costs related to diagnosis include the cost of face-to-face assessment and biopsy. Additional diagnostic costs are incurred in patients with more advanced disease (1b and higher), for example, sentinel lymph node biopsy and computerised tomography scans. The treatment of patients with high-risk cancer in situ or stage 1 and 2 involves surgery to remove the tumour. Patients with stage 3 may also undergo lymph node dissection and patients with terminal disease may undergo chemotherapy.

Concerning utility values, people in the model experienced utility (or disutility) associated with one or more of the following:

Disutility because of the excision and biopsy of a lesion suspected of MM that caused distress as well as anxiety whilst waiting for the results.

Disutility because of the permanent scarring following surgical excision of a lesion on the head or neck.

Health state-related utility, which was associated with the stage of MM (in people with MM) or with the average utility of the general population (in people without a MM). Patients treated for high-risk cancer will have a reduction in their health-related quality of life owing to their cancer. The reduction depends on the stage at diagnosis (for high-risk cancers) or whether they receive surgical treatment or not (for BCC) and whether they have scarring on the head or neck.

Patients with true positive results were treated in accordance with national guidelines. These patients will have a reduction in their health-related quality of life (HRQoL). A proportion of those who had a melanoma on their head or neck experienced an additional permanent reduction in their HRQoL because of the scarring following excision and biopsy.

People with true negative results will have the same HRQoL as the general population. Patients with false positive results also have the same HRQoL as the general population, except for a temporary reduction owing to anxiety whilst waiting for their biopsy results and the biopsy itself. In some events these could not be included because waiting times were unknown. A proportion of patients with false positive results will experience permanent reduction in their HRQoL owing to a scar on the head or neck (captured in the Markov model). The same applies to patients with true positive and false negative BCC results.

People with false negative results will have the same HRQoL as the general population of the same age, until their cancer is identified, in which case they experienced a reduction in their HRQoL depending on cancer stage at diagnosis (as for the true positives).

The age-related utilities of the general population are based on the catalogue of EQ-5D utilities for the UK population produced by Sullivan 2011 [15]. Cancer-related utilities are based on those reported in Tromme 2014 [16] and adjusted for age, using the age coefficient of −0.00029 reported by Sullivan 2011 [15] (as in the model reported in Edwards 2016 [12]).

Analysis:

The analytic strategy included multiple sensitivity analyses beyond examining the effect of variation in GP accuracy achieved by having multiple base-cases. The sensitivity of all other inputs was examined univariately in one-way sensitivity analyses using the ranges indicated in Table 1. These ranges were generally ±10% of the central estimate. These ranges were extended to include actually observed variation in the SBRI evaluations, that is, percentage of patients ineligible for photography. Probabilistic sensitivity analyses were also undertaken but are not presented in full because they did not meaningfully add to the results. An illustrative example of the probabilistic sensitivity analysis is, however, provided in Supplementary file Appendix 5. A scenario analysis was also done deterministically. It reduced the prevalence of skin cancer to the level of disease seen in the SBRI implementation evaluations. Decisions on whether variation in the sensitivity analyses led to a change in cost-effectiveness conclusions were made on the basis of the greatest value of the net monetary benefit (NMB), with a willingness to pay (WTP) value of £20,000 being the optimal strategy.

The model was implemented in Amua v. 0.3.3, free software for cost-effectiveness analysis [17]. A copy of the model is available from the corresponding author on request.