In this study, we evaluated the use of machine learning models for early prediction of ICU-AW. To ensure long-term sustainability of health care systems, tailored treatment is increasingly enforced through an accountability burden for treatment choice. As costs tend to concentrate in selected patient groups, it may be necessary to devise specific protocols for these outliers [28]. In ICU, resource usage is the highest in long-stay patients, and among them, patients who will develop intensive care unit acquired weakness (ICU-AW) represent a subgroup with particularly inflated costs [29].

In addition, interventions prescribed for this patient group, such as long-term recovery of premorbid physical function and quality of life, are not necessarily successful [9, 29,30,31,32,33], ensuring high value care will increasingly depend on ability to identify the “high cost and high need” patients, and delineate from which interventions they benefit the most [2, 4, 28, 34].

ICU-AW is a well-known predictor of both short and long-term outcome in ICU- patients [9, 29, 35, 36]. Upfront patient stratification by risk of ICU-AW as surrogate marker for individual patient trajectory could guide proactive treatment planning. As such, it holds added value over ICU-AW assessment itself. Indeed, the current body of evidence linking ICU-AW to outcome focused on its presence or absence on day 9, which indicates the beginning of prolonged critical illness [13, 30]. The clinical testing of peripheral muscle strength is inexpensive and simple, but reproducibility depends on assessment by trained health care providers: systematic daily assessment of all ICU-patients by the same trained physiotherapists is not a feasible clinical reality. Conversely, delayed formal ICU-AW assessment until clinical suspicion due to its associated complications (e.g. delayed weaning) or due to lack of cooperation (e.g., due to sedation, use of neuromuscular blocking agents, delirium) precludes the possibility to anticipate the ICU-trajectory upfront, and impedes targeted patient inclusion in randomized controlled trials aiming to reduce the burden of ICU-AW.

We created a model for predicting ICU-AW using a cohort of adult critically ill patients demographically representative of a general ICU population and made it available online. We based our prediction on clinical ICU-AW assessments performed by the same trained physiotherapists in an internal validation setup. Using a random forest, we constructed an accurate and calibrated model using baseline characteristics and descriptors readily available within 24 h after ICU admission at the bedside.

Based on its inherent feature importance, the random forest identified APACHE as the most relevant descriptor. This was expected, as it combines several factors that are related to the development of weakness, such as: high severity of illness implying high likelihood of sepsis and high likelihood of prolonged mechanical ventilation, use of steroids and neuromuscular blocking agents, among others.

However, age and creatinine, which are already included in the APACHE, are surprisingly descriptive. We can interpret this as an indication that pre-ICU frailty (i.e., higher creatinine likely captures patients with preexisting renal disease) is a very strong predictor of more frailty and more adverse ICU outcomes. This could also be related to the non-linear relationship between creatinine and ICU-AW, when taking into account interactions with covariates such as APACHE II, as can be seen in the partial dependency plot presented in Supplementary Material Fig. 13.

This is in line with the perceived evolution in the task of critical care physicians, namely to include a long-term perspective in their care and incorporate clearly established predictors of ICU and post ICU disability in early goals of care discussions for patients likely to not do well both in the ICU and beyond [37].

Our research has several strengths. First, our modelling approach has several methodological advantages when compared to currently available prediction models for ICU-AW. Namely, we rely on well-established machine learning models, whereas the other studies use step-wise logistic regression, which is inherently flawed [38]. Moreover, other models use data collected up until ICU-AW evaluation, such as duration of mechanical ventilatory support, to increase accuracy for detection of prolonged critically ill patients [21, 22]. Alternatively, they depend on data available only at day 2 after admission [23]. Thus, they employ data collected throughout the disease course, which excludes or reduces the possibility of predicting the outcome upfront and consequently of early initiation of preventive care. Our model uses variables which are frequently recorded in most of the ICUs and are also extractable from electronic health records.

Second, our study reported not only AUROC, sensitivity and specificity, but also model calibration and decision curves (clinical usefulness), as opposed to other studies [39, 40].

Third, as shown in the decision curve (Fig. 3), our model shows benefits above default strategies in the risk range below 0.5. Hence, using a relatively low threshold would be advantageous, as it increases the number of correctly identified ICU-AW in the expense of few incorrectly diagnosed with ICU-AW. For instance, if we take a false negative cost three times as high as a false positive cost, our model yields 88% sensitivity and 53% specificity, compared to 84% and 53% for the logistic regression model, respectively. This is particularly interesting for the inclusion of patients in new RCTs aiming to prevent ICU-AW or to initiate goals of care discussions in patients expected to have very poor long-term outcome based on this model among others. This allows the implementation of more efficient RCTs to assess the effect of interventions, since most of them still have limited success [14].

Lastly, our experiments included a larger number of descriptors, which enables us to assess their influence on ICU-AW. In principle, this could be a hindrance to some machine learning models, nonetheless our employed model, random forest, can correctly handle and benefit from such higher number of descriptors, as it is robust to overfitting [41]. Further, we showcased that using all descriptors is superior to using solely APACHE II. Our model is also available as an online application, which allows clinicians to estimate model performance in a sample of their patients, compensating for the lack of formal external validation. Since external validation will likely require a large university center, our online application allows physicians working in smaller hospitals to assess the applicability of the model in their patient populations.

Our study has important limitations. The choice of an early prediction model, using data limited to the first 24 h of ICU-stay, inherently limits the expected optimal performance for prediction of ICU-AW and long-term outcome [30]. To ensure that our model was informative of the long-term trajectory without including temporal indicators reflecting the in-ICU trajectory, we validated our model to predict weakness at day 9 of ICU-stay, as this time frame was shown to represent the cut-off towards prolonged critical illness known to be related to incremental adverse long-term prospects.

Another potential limitation of our study is the use of the MRC sum score to diagnose ICUAW, while a two-tier approach using handgrip dynamometry and a through range 4-grade score showed better inter-operator agreement for ICUAW diagnosis [42]. Nevertheless the MRC sum score remains the most used diagnostic procedure for ICUAW [14] and has demonstrated good inter-operator agreement [43, 44], while the two-tier approach still requires further validation.

Thirdly, ICU-AW was only assessed on the patients who were awake and cooperative at day 9, which can lead to possible bias, as competing factors, such as mortality prior to day 9, were not taken into account. Furthermore, the clinical trial from which the data originates dates to 2010, while different patient management strategies such as different types of sedation and variations in feeding strategies might be currently used.

Lastly, our experiments were limited to a single-center. External validation is thus required to assess the validity of the model in other centers. However, the online application tool allows clinicians to estimate the out-of-sample performance in their local patient population.