Chronic liver disease and cirrhosis remain major health concerns, consistently being among the top causes of death worldwide.1 Despite significant progress in medical treatments, the mortality rates associated with liver diseases have not shown significant improvement over the past three decades.2 To reduce mortality in patients with chronic liver disease and cirrhosis, liver transplantation is a potential solution.3,4 However, often the treating physicians have to weigh the risks and benefits of early liver transplantation in patients who present with evidence of physiological decompensation. This is essential because in some cases the time window between the intervention and the patient’s death may be limited. Failing to address these factors promptly may render intensive care unit (ICU) care and advanced interventions such as liver transplantation futile for some of these high-risk individuals.5,6 Therefore, it is necessary to establish a rational basis for discontinuing ICU care based on futility. Various scores, including Child–Turcotte–Pugh (CTP) and Mayo End-Stage Liver Disease (MELD), have been developed to assess prognosis and guide treatment prioritization including liver transplantation.7 Research suggests that the Chronic Liver Failure Consortium Acute-on-Chronic Liver Failure (CLIF-C ACLF) score might outperform both CTP and MELD in predicting short and medium-term mortalities in a range of patient scenarios.8,9 While these scores offer valuable information, they lack accuracy as they group patients with different levels of disease severity into similar risk categories. Also, there are no prognostic models that offer personalized estimates of the risk of liver-related death for individuals with alcohol-associated cirrhosis.10 In addition, the traditional prognostic scores may not take into account some of the risk factors associated with short-term mortality, such as acute kidney injury, plasma ammonia level, sarcopenia, and increased platelet aggregation.11,12

Advanced statistical tools and machine learning (ML) techniques are found to be superior to traditional statistical methods in predicting the risk of mortality. By capturing higher-dimensional and potentially nonlinear effects of variables, they can process more variables.13 Artificial intelligence (AI) is an expanding domain of information technology, widely used in various sectors such as e-commerce, media, and finance. Although health sciences have been slower to adopt AI, particularly ML, it is now gaining attention. ‘Traditional’ ML trains models through mathematical functions/rulesets that employ non-linear relationships, yielding more precise classification and prediction outputs compared to classical statistics.14 Deep learning, a relatively new subset of ML, employs deep neural networks to analyze data through multiple layers of interconnected artificial neurons, mimicking the structure and function of the human cerebral cortex.

A patient with cirrhosis represents a complex, multidimensional system for prognosis prediction. Emerging research suggests that AI and ML should be capable of providing superior mortality predictions for individual cirrhosis cases.15,16 Yet, previous studies have been limited in scope, which have used AI and ML focused on 90-day mortality prediction.5,15,17 There is a need for more comprehensive investigations using AI and ML to develop more accurate models capable of predicting short-term mortality risk in cirrhosis cases.17,18 Therefore, the objective of this study is to develop an AI-ML-based model to predict 28-day mortality in cirrhosis patients.

Methods

This retrospective study was conducted at Sultan Qaboos University Hospital (SQUH), a multispecialty teaching hospital in Oman.19,20 The subjects comprised all adult patients diagnosed with acute decompensation of liver cirrhosis and hospitalized in SQUH from January 2015 to December 2021. The study was approved by the Medical Research Ethics Committee of the College of Medicine and Health Sciences of Sultan Qaboos University (SQU EC/349/2021 MREC #2375).

The methodology described by Al Kaabi et al,2 in 2023 was adopted. In case of multiple hospitalizations, the first admission with acute decompensation was taken as the index admission. The required data was extracted from the patients’ electronic health records. This included demographic details, admission diagnosis, relevant comorbidities, cirrhosis etiology, duration of hospital stay, intensive care requirements, and relevant laboratory results.

The outcome of interest was 28-day all-cause mortality during the follow-up period. This was ascertained by reviewing the electronic health records of patients or through phone calls where necessary.

Before proceeding with model development, the dataset was refined. Missing values in binary features were imputed using with the mode, while continuous features had their missing values imputed using the median. To ensure uniformity in range, continuous features were scaled using the MinMaxScaler.

Four general ML algorithms, namely, Decision Tree, Random Forest, Logistic Regression, and Naïve Bayes, were employed to develop the models. These algorithms were chosen for their ability to handle both continuous and binary features and their capability to capture complex relationships within the data.21 In addition, a deep learning algorithm that incorporated multilayer artificial neural networks (ANNs) and recurrent neural network (RNN) including long short-term memory (LSTM) was implemented to test the ability of neural networks in capturing non-linear dependencies in the data.22 For each algorithm, the dataset was split into training and testing sets at 80:20 ratio.

Various metrics were calculated to compare the models' performance, including accuracy, precision, recall, and F1-Score. Accuracy measures the overall correctness of predictions while precision quantifies the proportion of correctly predicted positive cases. Recall measures the sensitivity of the model in identifying positive cases. F1-score combines precision and recall into a single metric. The metrics were calculated for each model and reported. To provide a visual representation of the performance, bar plots were generated for the four metrics with each bar representing the performance of a given model.22

Categorical variables were expressed in numerical values and percentages while continuous variables were presented as means for normally distributed data and medians with IQR for non-normally distributed data. To compare the continuous variables between the two groups, the student’s t-test was employed for normally distributed variables, and the Wilcoxon rank-sum test for non-normally distributed variables. The chi-square test assessed the relationship between categorical variables. Statistical significance was set at a two-sided p-value < 0.050. All statistical analyses were conducted using Stata (StataCorp. 2023. Stata Statistical Software: Release 18. College Station, TX: StataCorp LLC).

For the development of the models and their evaluation, we used Anaconda distribution, including Python programming language and various libraries. We initially developed our models using four general ML algorithms, namely Decision Tree, Random Forest, Naïve Bayes, and Logistic Regression. Multilayer ANNs, LSTM, and RNN were employed as deep learning algorithms. Each generated model was trained and tested using a dataset containing features related to acute decompensations of chronic liver disease.

Results

The subjects comprised of 173 patients admitted with acute decompensated liver disease during the study period. The demographic, clinical and laboratory findings, and factors associated with 28-day mortality are reported in Table 1.

The scores of various ML models under different assessment scales are listed in Table 2. The Decision Tree model achieved an accuracy of 71.4%, with a precision of 38.5% and a recall of 71.4%. The F1-score for this model was 0.500. The Logistic Regression model showed higher performance with an accuracy of 82.9%. It achieved a precision of 55.6% and a recall of 71.4%. The F1-score was 0.625. Both the Naïve Bayes and Random Forest models performed similarly, both achieving an accuracy of 82.9% and a precision of 54.5%. The Naïve Bayes model showed a higher recall of 85.7%, resulting in an F1-score of 0.667. The Random Forest model had a recall of 85.7% and an F1-score of 0.667. The multilayer ANNs model achieved an accuracy of 74.3%, with a precision of 37.5% and a recall of 42.9%. The F1-score was 0.400. In contrast, the RNN and LSTM models exhibited lower performance metrics. The RNN model achieved an accuracy of 80.0% with a perfect precision of 100%. However, it had a recall of 0.0% and an F1-score of 0.000. The LSTM model also had an accuracy of 80.0% but performed poorly in terms of precision, recall, and the F1-score, all of which were 0.000.

Among the traditional ML models (those without Deep Learning), the Logistic Regression, Naïve Bayes, and Random Forest showed high AUROC values indicating good discrimination capability in predicting mortality. In contrast, the deep learning models— multilayer ANNs, RNN, and LSTM — demonstrated a lower capacity for discrimination, as revealed by their lower area under the receiver operating characteristics curve values [Table 2]. Overall, Naïve Bayes and Random Forest showed the highest performance metrics among the traditional ML models.

The Feature Importance Analysis using the Random Forest model revealed the top 10 contributing features for mortality prediction in patients with chronic liver disease and cirrhosis following admission with acute decompensation. The most influential feature was ICU admission (importance: 0.112), followed by mechanical ventilation (importance: 0.095), and mean arterial pressure (importance: 0.073). Other significant features included platelet count (importance: 0.063), alkaline phosphatase (importance: 0.056), bilirubin (importance: 0.054), and white blood cell count (importance: 0.048). Figure 1 shows the top 10 contributing features in the order of relative importance.

Discussion

Our study used traditional ML as well as deep learning techniques to develop a model to predict 28-day mortality of hospitalized patients with acute decompensation of liver cirrhosis. We evaluated the performance of various models, including Decision Tree, Logistic Regression, Naïve Bayes, Random Forest, multilayer ANNs, RNN, and LSTM, using a dataset that included demographic, clinical, and laboratory features related to chronic liver disease.

The Naïve Bayes and Random Forest models had the highest performance metrics among the traditional ML models. They achieved high accuracy, precision, recall, and F1-score. Despite its simplicity, the Naïve Bayes model was able to capture relevant patterns in the data and make accurate predictions for mortality.

The AUROC values further confirmed the discriminatory ability of these models with values ranging from 0.786 to 0.839. The Naïve Bayes model, although simpler in its assumption of feature independence, delivered competitive results. It showed a balanced trade-off between precision and recall, similar to the Decision Tree model. It is worth noting that the AUROC values for predicting 28-day mortality using the Naïve Bayes and Random Forest models were comparable to the AUROC values for CLIF-C and superior to those for CTP and MELD-Na within the same cohort of patients, as previously reported.2

In a 2021 study on a cohort of 2170 cirrhosis patients,23 the prediction of 90-day mortality was examined through the utilization of three AI models: Logistic Regression, Kernel Support Vector Machine, and Random Forest Classifiers. The study yielded a modest AUROC score of 0.67 for the prediction of 90-day mortality. The mortality predictions by the AI models were only as accurate as those by MELD-Na, a popular non-AI model.

The deep learning models used in our study—including the multilayer ANNs, RNN, and LSTM—showed lower performance metrics. The multilayer ANNs model achieved moderate performance, while the RNN and LSTM models showed poor performance, with recall and F1-score of 0 for predicting mortality. Meanwhile in a large study involving 34 575 cirrhosis patients, deep learning models including deep neural networks, outperformed the traditional MELD score in predicting mortality at various time frames (90, 180, and 365 days).17

In the current study, the superior performance of traditional ML models like Logistic Regression in comparison to the newer deep learning models such as multilayer ANN, RNNs, and LSTM could be attributed to several factors. First, with small sample sizes (173 in our case), traditional machine learning models are reported to be often more efficient in processing and more accurate in extracting meaningful patterns from data.24 Deep learning models require a larger amount of data to capture the complexity of the problem adequately. Additionally, traditional ML models have a simpler architecture and fewer hyperparameters to tune, making them less prone to overfitting in small datasets.25

In our study, feature importance analysis revealed the top 10 contributing features for mortality prediction in patients with chronic liver disease and cirrhosis [Figure 1]. Among these features, ICU admission was found to be the most influential, followed by mechanical ventilation, mean arterial pressure, platelet count, alkaline phosphatase, bilirubin, and white blood cell count. The AI and ML models used in a previous study identified alkaline phosphatase, alanine aminotransferase, and hemoglobin as top contributing features, besides MELD-Na variables in predicting mortality in patients with liver cirrhosis.17

Our findings and those from previous studies suggest that clinical indicators of disease severity and organ dysfunction can serve as important indicators for mortality prediction in this patient population. They provide additional factors associated with short-term poor outcomes which were not highlighted in traditional prognostic scores (i.e MELD, CLIF-C),9,26 and may further empower clinicians to make well-informed decisions regarding care prioritization, including the advisability of liver transplantation or a re-evaluation of appropriateness of certain treatments.

It is important to note the limitations of our study. First, our analysis was based on a relatively small dataset, from a single center, on a mostly ethnically homogeneous (Omani) cohort. These factors may impact the generalizability of our results. Future studies with larger and more diverse datasets in different populations are warranted to validate the performance of these models and to improve them. Additionally, external validation of the developed models using independent cohorts would provide further evidence of the models’ real-life applicability. Furthermore, our study focused on predicting mortality within a 28-day period, and longer-term predictions were not explored. Future research could use these models on different prediction horizons to enhance their clinical utility.

Conclusion

This study demonstrates the potential of ML and deep learning techniques in predicting 28-day mortality in patients admitted with acute decompensation of chronic liver disease. The Logistic Regression, Naïve Bayes, and Random Forest models showed favorable performance metrics, indicating their utility in mortality prediction. The top predictive features for 28-day mortality included ICU admission, mechanical ventilation, and mean arterial pressure. Implementing these models in clinical practice may enhance risk stratification and aid in timely intervention for patients with chronic liver disease and cirrhosis for optimal outcomes. More studies using larger cohorts, external datasets, and longer time frames are necessary to establish the robustness and generalizability of these models.

Disclosure

The authors declared no conflict of interest. No funding was received for this study.

references

1. 1. Devarbhavi H, Asrani SK, Arab JP, Nartey YA, Pose E, Kamath PS. Global burden of liver disease: 2023 update. J Hepatol 2023 Aug;79(2):516-537.
2. 2. Al Kaabi H, Al Alawi AM, Al Falahi Z, Al-Naamani Z, Al Busafi SA. Clinical Characteristics, Etiology, and Prognostic Scores in Patients with Acute Decompensated Liver Cirrhosis. J Clin Med 2023 Sep;12(17):5756.
3. 3. Mahmud N. Selection for Liver Transplantation: Indications and Evaluation. Curr Hepatol Rep 2020;19(3):203-212.
4. 4. Bajaj JS, Choudhury AK, Xie Q, Kamath PS, Topazian M, Hayes PC, et al; CLEARED Investigators. Global disparities in mortality and liver transplantation in hospitalised patients with cirrhosis: a prospective cohort study for the CLEARED Consortium. Lancet Gastroenterol Hepatol 2023 Jul;8(7):611-622.
5. 5. Cucchetti A, Vivarelli M, Heaton ND, Phillips S, Piscaglia F, Bolondi L, et al. Artificial neural network is superior to MELD in predicting mortality of patients with end-stage liver disease. Gut 2007 Feb;56(2):253-258.
6. 6. Trebicka J, Sundaram V, Moreau R, Jalan R, Arroyo V. Liver Transplantation for Acute-on-Chronic Liver Failure: Science or Fiction? Liver Transpl 2020 Jul;26(7):906-915.
7. 7. Peng Y, Qi X, Guo X. Child-pugh versus MELD score for the assessment of prognosis in liver cirrhosis: a systematic review and meta-analysis of observational studies. Medicine (Baltimore) 2016 Feb;95(8):e2877.
8. 8. Barosa R, Roque Ramos L, Patita M, Nunes G, Fonseca J. CLIF-C ACLF score is a better mortality predictor than MELD, MELD-Na and CTP in patients with acute on chronic liver failure admitted to the ward. Rev Esp Enferm Dig 2017 Jun;109(6):399-405.
9. 9. Ramzan M, Iqbal A, Murtaza HG, Javed N, Rasheed G, Bano K. Comparison of CLIF-C ACLF score and MELD score in predicting ICU mortality in patients with acute-on-chronic liver failure. Cureus 2020 Feb;12(2):e7087.
10. 10. Marot A, Henrion J, Knebel JF, Trépo E, Moreno C, Deltenre P. A model for individualized prediction of liver-related death in outpatients with alcohol-associated cirrhosis. Hepatol Commun 2023 Aug;7(9):e0229.
11. 11. Zanetto A, Campello E, Bulato C, Gavasso S, Farinati F, Russo FP, et al. Increased platelet aggregation in patients with decompensated cirrhosis indicates higher risk of further decompensation and death. J Hepatol 2022 Sep;77(3):660-669.
12. 12. Thuluvath PJ, Alukal JJ, Zhang T. A model to predict inhospital mortality in patients with cirrhosis, ascites and hyponatremia. Eur J Gastroenterol Hepatol 2022 Jun;34(6):591-597.
13. 13. Angraal S, Mortazavi BJ, Gupta A, Khera R, Ahmad T, Desai NR, et al. Machine learning prediction of mortality and hospitalization in heart failure with preserved ejection fraction. JACC Heart Fail 2020 Jan;8(1):12-21.
14. 14. Bhat M, Rabindranath M, Chara BS, Simonetto DA. Artificial intelligence, machine learning, and deep learning in liver transplantation. J Hepatol 2023 Jun;78(6):1216-1233.
15. 15. Banerjee R, Das A, Ghoshal UC, Sinha M. Predicting mortality in patients with cirrhosis of liver with application of neural network technology. J Gastroenterol Hepatol 2003 Sep;18(9):1054-1060.
16. 16. Cross SS, Harrison RF, Kennedy RL. Introduction to neural networks. Lancet 1995 Oct;346(8982):1075-1079.
17. 17. Guo A, Mazumder NR, Ladner DP, Foraker RE. Predicting mortality among patients with liver cirrhosis in electronic health records with machine learning. PLoS One 2021 Aug;16(8):e0256428.
18. 18. Hou Y, Zhang Q, Gao F, Mao D, Li J, Gong Z, et al. Artificial neural network-based models used for predicting 28- and 90-day mortality of patients with hepatitis B-associated acute-on-chronic liver failure. BMC Gastroenterol 2020 Mar;20(1):75.
19. 19. Al-Yarabi A, Al Balushi H, Al Hatmi K, Al Yahyaie R, Al Alawi AM, Al Zeedy K, et al. Inappropriate hospital stay of patients admitted under care of general medicine units: a retrospective study. Sultan Qaboos Univ Med J 2023 May;23(2):174-181.
20. 20. Al Sibani M, Al-Maqbali JS, Yusuf Z, Al Alawi AM. Incidence and risk factors for 28 days hospital readmission: a retrospective study from Oman. Oman Med J 2022 Sep;37(5):e423.
21. 21. Sarker IH. Machine learning: algorithms, real-world applications and research directions. SN Comput Sci 2021;2(3):160.
22. 22. Choi RY, Coyner AS, Kalpathy-Cramer J, Chiang MF, Campbell JP. Introduction to machine learning, neural networks, and deep learning. Transl Vis Sci Technol 2020 Feb;9(2):14.
23. 23. Hu C, Anjur V, Saboo K, Reddy KR, O’Leary J, Tandon P, et al. Low predictability of readmissions and death using machine learning in cirrhosis. Am J Gastroenterol 2021 Feb;116(2):336-346.
24. 24. Maniar KM, Lassarén P, Rana A, Yao Y, Tewarie IA, Gerstl JV, et al. Traditional machine learning methods versus deep learning for meningioma classification, grading, outcome prediction, and segmentation: a systematic review and meta-analysis. World Neurosurg 2023 Nov;179:e119-e134.
25. 25. Gupta R, Srivastava D, Sahu M, Tiwari S, Ambasta RK, Kumar P. Artificial intelligence to deep learning: machine intelligence approach for drug discovery. Mol Divers 2021 Aug;25(3):1315-1360.
26. 26. Shi Y, Shu Z, Sun W, Yang Q, Yu Y, Yang G, et al. Risk stratification of decompensated cirrhosis patients by chronic liver failure consortium scores: classification and regression tree analysis. Hepatol Res 2017 Mar;47(4):328-337.