The aim of this study was to report the accuracy and precision of quantitative metrics of the biliary tree, measured from non-invasive MRCP images using commercially available software, which has received increasing attention for the assessment and monitoring of patients with PSC. The detection and measurement of strictures and dilatations had excellent accuracy and negligible bias across all the scanners tested, as demonstrated by the results on phantoms with known values. The In vivo precision was generally better for test-retest repeatability than cross-scanner reproducibility. However, for 6/7 metrics both test-retest and cross-scanner reproducibility coefficients, which capture measurement error were lower than the median differences observed between healthy volunteers and published PSC values. This suggests that the metrics are sufficiently precise to distinguish these populations.

Reproducibility of MRCP + metrics was assessed across scanners with varying field strengths (1.5T and 3T), and while precision was generally sufficient to distinguish PSC and healthy populations, some differences were observed. Metrics such as the total number of strictures and dilatations demonstrated slightly better reproducibility at 3T. However, certain metrics, including the total number of ducts with stricture or dilatation, showed consistent reproducibility across field strengths, reflecting their robustness to scanner variations. It was observed during the analysis that the 3T MRCP acquisitions often achieved superior background suppression, making the selection and modelling of the ducts easier, which may partly explain the slightly improved reproducibility of some metrics at 3T. Figure 4 presents selected case examples demonstrating varying levels of repeatability between scans acquired at 1.5T and 3T, which also illustrate the superior background suppression at 3T. These results indicate that while field strength can influence precision for some metrics, others remain stable across different imaging environments, supporting their suitability for multi-center trials. Differences in sequence parameters, including echo times, may also contribute to the observed variability, underlining the importance of harmonizing imaging protocols when incorporating quantitative MRCP metrics into clinical workflows.

Examples of challenging pairs of acquisitions with reduced cross-scanner consistency. (a) and (b) show MRCP MIPs of two separate subjects (cases 1 and 2, respectively), acquired on Siemens 3T, alongside their MRCP + models. The corresponding results shown for case 1 (c) and case 2 (d) were acquired on Philips 1.5T and GE 1.5T, respectively. Both 1.5T images have worse background suppression, making analysis more challenging. For case 2, more faint vessel-like structures are visible in the 1.5T scan (d), leading to more ducts being modelled than for the corresponding 3T scan (b)

The tighter limits of agreement for the phantom accuracy compared with that of the in vivo precision data is likely because the former is unaffected by issues such as patient motion and bright gastrointestinal structures located near the biliary tree if the patient has not fasted prior to the scan. This suggests that patient compliance with breathing and fasting instructions may be a dominant source of error in the quantitative metrics compared to error caused by the underlying acquisitions or image processing algorithms.

For each in vivo measure of metric consistency reported here, further contextualisation is needed to assess whether these values are ‘good’. The interpretation of the RC is that, in 95% of cases, noise induced by repeated measurements will not exceed this value, thus a change greater than the RC would indicate a true change. One way to contextualise the RC values is to compare them to the differences typically seen in healthy and PSC populations, as shown in Table 5. For example, the median (IQR) number of strictures in the PSC and healthy groups are 11 (11) and 2 (2), respectively. If a treatment was expected to reduce the number of strictures to the value seen in a healthy population, the change would require a reduction in at least 9 strictures. Any change of this magnitude would be greater than the range of RC values (3.1–6.5), (Tables 3 and 4), and therefore detectable on an individual level. By contrast, the difference in equivalent values for the percentage of ducts with diameter 3–5 mm: 22 (15) vs. 20 (11) for the PSC and healthy groups, respectively, is less than the range of RC values (10–30), suggesting this measure may be insensitive to differences between PSC and healthy patients. This is consistent with the findings of Trivedi et al. [15], where no significant difference in this metric between PSC and healthy populations were reported. However, while not diagnostic, this metric has shown promise as a prognostic marker for PSC [18, 19], suggesting that it may capture aspects of PSC relevant to later stage disease.

The MRCP + software has a relatively low threshold for detection of strictures (absolute change in width of 1 mm), which makes it sensitive to subtle changes in duct width that may not be recorded by a radiologist. This explains why the healthy reference range for number of strictures (as defined by MRCP+) is 0–6, while radiologists, who typically focus on severe duct stenosis, may report fewer or no strictures in healthy subjects. While the exact definition of strictures used by the software may lead to more strictures being detected in healthy biliary trees than may be reported by a radiologist, strictures measured according to this definition have nevertheless been found to have good utility for diagnosing and monitoring PSC cases [13,14,15,16,17,18,19]. In fact, the ability to detect subtle biliary structures through quantitative analysis may characterise diffuse patterns and explain why this approach has previously outperformed other traditional assessments [18, 19].

Conventional MRCP image interpretation is subjective, with many studies reporting poor inter-observer variability. For example, Selvaraj et al. [13] reported that even simple metrics of maximum duct diameter had poor inter-observer agreement when measured manually by expert radiologists (ICC = 0.40 for left hepatic duct). Furthermore, Grigoriadis et al. [8] found that the semi-quantitative assessment of more complex features, including dilatations, using the ANALI score had a Cohen’s kappa score of 0.38, while strictures characterised by the DiStrict score depended heavily upon radiologist experience, with less experienced radiologists demonstrating poor agreement (ICC 0.48; 95% CI 0.05–0.72) [7].

Therefore, the performance demonstrated by the quantitative MRCP metrics highlights the improvements possible by converting to a fully quantitative and objective assessment of MRCP examinations. This is particularly important when considering that standardizing assessments of biliary health could ensure patients receive the most appropriate care in a timely fashion. An objective assessment is also imperative in a drug development setting, where quantitative, reliable metrics could detect subtle changes in biliary tree health. In multi-centre trials with longitudinal assessments of disease, an understanding of the magnitude of change due to measurement noise is required. Future work should focus on estimating the clinically meaningful change in quantitative MRCP metrics, to strengthen the utility as clinical biomarkers and as objective endpoints in PSC clinical trials.

Due to practical constraints, it was not possible to scan all subjects on all scanners, which were located in different cities. Furthermore, the GE and Philips groups had far fewer subjects with biliary or other liver diseases, and thus the range of metric values was lower in this group. Nevertheless, the worst-case RC values across all scanners still enabled differentiation of healthy and PSC cases. Furthermore, the metric accuracy assessed using the phantom– which provided a true gold-standard and covered a broad range of metric values– was similar across all scanners.

While this study demonstrates promising results for quantitative MRCP metrics, practical challenges may affect their implementation in AI-driven clinical workflows. Our study population had a mean age of 39 years and mean BMI of 25.5 kg/m², which may not fully represent the diversity of patients encountered in clinical practice. When integrating these metrics into automated AI systems, variations in image quality due to patient factors (respiratory motion, body habitus) could impact performance. In practice, published studies of patients including those with PSC, PBC or AIH have shown successful return of MRCP + reports in 90–95% of prospectively collected MRCP scans [13, 15].

Several steps are already in place as part of the MRCP + workflow which address potential sources of variability introduced more generally within AI-driven workflows:

(1) MRCP + checks the DICOM files when loading to ensure the acquisition parameters are within acceptable ranges. (2) Operators are trained to recognize and reject cases affected by severe artefacts. (3) MRCP + uses a standardized MR-protocol where possible, which has been optimised for image quality and robustness. As new AI-enhanced and/or accelerated MR-imaging techniques become commonplace, underlying MRCP data quality may also be improved, reducing potential impact of motion artefacts. Within fully automated AI-workflows, step (2) above would likely be automated, although this would require significant training data to justify removing the security of the human-in-the-loop checkpoint.