Predicting Mortality and ICUs Transfer in Hospitalized COVID-19 Patients Using Random Forest Model
,
Abstract
Background: The objective of the present study was to identify prognostic factors associated with mortality and transfer to intensive care units (ICUs) in hospitalized COVID-19 patients using random forest (RF). Also, its performance was compared with logistic regression (LR).
Methods: In this retrospective cohort study, information of 329 COVID-19 patients were analyzed. These patients were hospitalized in Besat hospital in Hamadan province, the west of Iran. The RF and LR models were used for predicting mortality and transfer to ICUs. These models' performance was assessed using area under the receiver operating characteristic curve (AUC) and accuracy.
Results: Of the 329 COVID-19 patients, 57 (15.5%) patients died and 106 (32.2%) patients were transferred to ICUs. Based on multiple LR model, there was a significant association between age (OR=1.02; 95% CI=1.00-1.05), cough (OR=0.24; 95% CI=0.10-0.56), and ICUs (OR=7.20; 95% CI=3.30-15.69) with death. Also, a significant association was found between kidney disease (OR=3.90; 95% CI=1.04-14.63), decreased sense of smell (OR=0.28; 95% CI=0.10-0.73), Kaletra (OR=2.53; 95% CI=1.39-4.59), and intubation (OR=8.32; 95% CI=3.80-18.24) with transfer to ICUs. RF showed that the order of variable importance has belonged to age, ICUs, and cough for predicting mortality; and age, intubation, and Kaletra for predicting transfer to ICUs.
Conclusion: This study showed that the performance of RF provided better results compared to LR for predicting mortality and ICUs transfer in hospitalized COVID-19 patients.
[1] Organization WH. Coronavirus disease 2019 (COVID-19): situation report, 73 2020 [Available from: Availableonline:https://www.who.int/emergencies/diseases/novel-coronavirus-2019/situation-reports.
[2] Yeşilkanat CM. Spatio-temporal estimation of the daily cases of COVID-19 in worldwide using random forest machine learning algorithm. Chaos Solitons Fractals. 2020; 140:110210.
[3] Subudhi S, Verma A, Patel AB, Hardin CC, Khandekar MJ, Lee H, et al. Comparing machine learning algorithms for predicting ICU admission and mortality in COVID-19. NPJ digital medicine. 2021;4(1):1-7.
[4] Covid C, Team R, COVID C, Team R, COVID C, Team R, et al. Severe outcomes among patients with coronavirus disease 2019 (COVID-19)—United States, February 12–March 16, 2020. MMWR Morb Mortal Wkly Rep. 2020; 69(12):343-6.
[5] Myers LC, Parodi SM, Escobar GJ, Liu VX. Characteristics of hospitalized adults with COVID-19 in an integrated health care system in California. Jama. 2020; 323(21):2195-8.
[6] Richardson S, Hirsch JS, Narasimhan M, Crawford JM, McGinn T, Davidson KW, et al. Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City area. Jama. 2020; 323(20):2052-9.
[7] Hui DS, Azhar EI, Madani TA, Ntoumi F, Kock R, Dar O, et al. The continuing 2019-nCoV epidemic threat of novel coronaviruses to global health—The latest 2019 novel coronavirus outbreak in Wuhan, China. Int J Infect Dis. 2020; 91:264-6.
[8] Huang C, Wang Y, Li X, Ren L, Zhao J, Hu Y, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet. 2020; 395(10223):497-506.
[9] Chen N, Zhou M, Dong X, Qu J, Gong F, Han Y, et al. Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. Lancet. 2020; 395(10223):507-13.
[10] Chen Y, Liu Z, Li X, Zhao J, Wu D, Xiao M, et al. Risk factors for mortality due to COVID-19 in intensive care units: a single-center study. Ann Transl Med. 2021; 9(4):276.
[11] Du Y, Tu L, Zhu P, Mu M, Wang R, Yang P, et al. Clinical features of 85 fatal cases of COVID-19 from Wuhan. A retrospective observational study. Am J Respir Crit Care Med. 2020 Jun 1;201(11):1372-9.
[12] Liang W-h, Guan W-j, Li C-c, Li Y-m, Liang H-r, Zhao Y, et al. Clinical characteristics and outcomes of hospitalised patients with COVID-19 treated in Hubei (epicentre) and outside Hubei (non-epicentre): a nationwide analysis of China. Eur Respir J. 2020; 55(6):2000562.
[13] García LF, Gutiérrez ABP, Bascones MG. Relationship between obesity, diabetes and ICU admission in COVID-19 patients. Med Clin (English Ed). 2020;155(7):314.
[14] Jackson BR, Gold JA, Natarajan P, Rossow J, Neblett Fanfair R, da Silva J, et al. Predictors at admission of mechanical ventilation and death in an observational cohort of adults hospitalized with COVID-19. Clin infect dis. 2021. 73(11):e4141-e4151.
[15] Venturini S, Orso D, Cugini F, Crapis M, Fossati S, Callegari A, et al. Classification and analysis of outcome predictors in non‐critically ill COVID‐19 patients. Intern Med J. 2021; 51(4):506-14.
[16] Jeung M, Baek S, Beom J, Cho KH, Her Y, Yoon K. Evaluation of random forest and regression tree methods for estimation of mass first flush ratio in urban catchments. Journal of Hydrology. 2019; 575:1099-110.
[17] Chen W, Xie X, Wang J, Pradhan B, Hong H, Bui DT, et al. A comparative study of logistic model tree, random forest, and classification and regression tree models for spatial prediction of landslide susceptibility. Catena. 2017; 151:147-60.
[18] Izquierdo-Verdiguier E, Zurita-Milla R. An evaluation of Guided Regularized Random Forest for classification and regression tasks in remote sensing. International Journal of Applied Earth Observation and Geoinformation. 2020; 88:102051.
[19] Hengl T, Nussbaum M, Wright MN, Heuvelink GB, Gräler B. Random forest as a generic framework for predictive modeling of spatial and spatio-temporal variables. PeerJ. 2018;6:e5518.
[20] Najafi-Vosough R, Faradmal J, Hosseini SK, Moghimbeigi A, Mahjub H. Predicting hospital readmission in heart failure patients in Iran: a comparison of various machine learning methods. Healthc Inform Res. 2021; 27(4):307-14.
[21] Breiman L. Random forests. Machine learning. 2001; 45(1):5-32.
[22] Hosseini-Esfahani F, Alafchi B, Cheraghi Z, Doosti-Irani A, Mirmiran P, Khalili D, et al. Using machine learning techniques to predict factors contributing to the incidence of metabolic syndrome in tehran: Cohort study. JMIR Public Health Surveill. 2021; 7(9):e27304.
[23] Pourhomayoun M, Shakibi M. Predicting mortality risk in patients with COVID-19 using machine learning to help medical decision-making. Smart Health. 2021; 20:100178.
[24] Cheng F-Y, Joshi H, Tandon P, Freeman R, Reich DL, Mazumdar M, et al. Using machine learning to predict ICU transfer in hospitalized COVID-19 patients. J Clin Med. 2020; 9(6):1668.
[25] Tezza F, Lorenzoni G, Azzolina D, Barbar S, Leone LAC, Gregori D. Predicting in-Hospital Mortality of Patients with COVID-19 Using Machine Learning Techniques. J Pers Med. 2021;11(5):343.
[26] Wollenstein-Betech S, Cassandras CG, Paschalidis IC. Personalized predictive models for symptomatic COVID-19 patients using basic preconditions: hospitalizations, mortality, and the need for an ICU or ventilator. Int J Med Inform. 2020; 142:104258.
[27] Das AK, Mishra S, Gopalan SS. Predicting CoVID-19 community mortality risk using machine learning and development of an online prognostic tool. PeerJ. 2020; 8:e10083.
[28] Wang J, Yu H, Hua Q, Jing S, Liu Z, Peng X, et al. A descriptive study of random forest algorithm for predicting COVID-19 patients outcome. PeerJ. 2020; 8:e9945.
[29] Kafieh R, Arian R, Saeedizadeh N, Amini Z, Serej ND, Minaee S, et al. Covid-19 in iran: Forecasting pandemic using deep learning. Computational and mathematical methods in medicine. 2021;2021.
[30] Mahdavi M, Choubdar H, Zabeh E, Rieder M, Safavi-Naeini S, Jobbagy Z, et al. A machine learning based exploration of COVID-19 mortality risk. Plos one. 2021;16(7):e0252384.
[31] Orooji A, Kazemi-Arpanahi H, Kaffashian M, Kalvandi G, Shanbehzadeh M. Comparing of Machine Learning Algorithms for Predicting ICU admission in COVID-19 hospitalized patients. Health Education and Health Promotion. 2021;9(3):229-36.
[32] Team RC. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. URLhttps://www.R-project.org/. 2020.
[33] Zhao Z, Chen A, Hou W, Graham JM, Li H, Richman PS, et al. Prediction model and risk scores of ICU admission and mortality in COVID-19. PloS one. 2020;15(7):e0236618.
[34] Ji D, Zhang D, Xu J, Chen Z, Yang T, Zhao P, et al. Prediction for progression risk in patients with COVID-19 pneumonia: the CALL score. Clin Infect Dis. 2020; 71(6):1393-9.
[35] Lu J, Hu S, Fan R, Liu Z, Yin X, Wang Q, et al. ACP risk grade: a simple mortality index for patients with confirmed or suspected severe acute respiratory syndrome coronavirus 2 disease (COVID-19) during the early stage of outbreak in Wuhan, China. MedRxiv. 2020.
[36] Flythe JE, Assimon MM, Tugman MJ, Chang EH, Gupta S, Shah J, et al. Characteristics and outcomes of individuals with pre-existing kidney disease and COVID-19 admitted to intensive care units in the United States. Am J Kidney Dis. 2021; 77(2):190-203. e1.
[37] Buonamico E, Quaranta VN, Boniello E, Dimitri M, Di Lecce V, Labate L, et al. Risk factors for transfer from respiratory intermediate care unit to intensive care unit in COVID-19. Respir Investig. 2021; 59(5):602-7.
[38] Moulaei K, Shanbehzadeh M, Mohammadi-Taghiabad Z, Kazemi-Arpanahi H. Comparing machine learning algorithms for predicting COVID-19 mortality. BMC Med Inform Decis Mak. 2022; 22(1):1-12.
[39] Yadaw AS, Li Y-c, Bose S, Iyengar R, Bunyavanich S, Pandey G. Clinical features of COVID-19 mortality: development and validation of a clinical prediction model. Lancet Digit Health. 2020; 2(10):e516-e25.
[40] Gao Y, Cai G-Y, Fang W, Li H-Y, Wang S-Y, Chen L, et al. Machine learning based early warning system enables accurate mortality risk prediction for COVID-19. Nat Commun. 2020; 11(1):1-10.
[41] Shanbehzadeh M, Haghiri H, Afrash MR, Amraei M. Comparison of Machine Learning Tools for the Prediction of ICU Admission in COVID-19 Hospitalized Patients. Shiraz E Medical Journal. 2022.
[42] Lorenzen SS, Nielsen M, Jimenez-Solem E, Petersen TS, Perner A, Thorsen-Meyer H-C, et al. Using machine learning for predicting intensive care unit resource use during the COVID-19 pandemic in Denmark. Scientific reports. 2021; 11(1):1-10.
[2] Yeşilkanat CM. Spatio-temporal estimation of the daily cases of COVID-19 in worldwide using random forest machine learning algorithm. Chaos Solitons Fractals. 2020; 140:110210.
[3] Subudhi S, Verma A, Patel AB, Hardin CC, Khandekar MJ, Lee H, et al. Comparing machine learning algorithms for predicting ICU admission and mortality in COVID-19. NPJ digital medicine. 2021;4(1):1-7.
[4] Covid C, Team R, COVID C, Team R, COVID C, Team R, et al. Severe outcomes among patients with coronavirus disease 2019 (COVID-19)—United States, February 12–March 16, 2020. MMWR Morb Mortal Wkly Rep. 2020; 69(12):343-6.
[5] Myers LC, Parodi SM, Escobar GJ, Liu VX. Characteristics of hospitalized adults with COVID-19 in an integrated health care system in California. Jama. 2020; 323(21):2195-8.
[6] Richardson S, Hirsch JS, Narasimhan M, Crawford JM, McGinn T, Davidson KW, et al. Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City area. Jama. 2020; 323(20):2052-9.
[7] Hui DS, Azhar EI, Madani TA, Ntoumi F, Kock R, Dar O, et al. The continuing 2019-nCoV epidemic threat of novel coronaviruses to global health—The latest 2019 novel coronavirus outbreak in Wuhan, China. Int J Infect Dis. 2020; 91:264-6.
[8] Huang C, Wang Y, Li X, Ren L, Zhao J, Hu Y, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet. 2020; 395(10223):497-506.
[9] Chen N, Zhou M, Dong X, Qu J, Gong F, Han Y, et al. Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. Lancet. 2020; 395(10223):507-13.
[10] Chen Y, Liu Z, Li X, Zhao J, Wu D, Xiao M, et al. Risk factors for mortality due to COVID-19 in intensive care units: a single-center study. Ann Transl Med. 2021; 9(4):276.
[11] Du Y, Tu L, Zhu P, Mu M, Wang R, Yang P, et al. Clinical features of 85 fatal cases of COVID-19 from Wuhan. A retrospective observational study. Am J Respir Crit Care Med. 2020 Jun 1;201(11):1372-9.
[12] Liang W-h, Guan W-j, Li C-c, Li Y-m, Liang H-r, Zhao Y, et al. Clinical characteristics and outcomes of hospitalised patients with COVID-19 treated in Hubei (epicentre) and outside Hubei (non-epicentre): a nationwide analysis of China. Eur Respir J. 2020; 55(6):2000562.
[13] García LF, Gutiérrez ABP, Bascones MG. Relationship between obesity, diabetes and ICU admission in COVID-19 patients. Med Clin (English Ed). 2020;155(7):314.
[14] Jackson BR, Gold JA, Natarajan P, Rossow J, Neblett Fanfair R, da Silva J, et al. Predictors at admission of mechanical ventilation and death in an observational cohort of adults hospitalized with COVID-19. Clin infect dis. 2021. 73(11):e4141-e4151.
[15] Venturini S, Orso D, Cugini F, Crapis M, Fossati S, Callegari A, et al. Classification and analysis of outcome predictors in non‐critically ill COVID‐19 patients. Intern Med J. 2021; 51(4):506-14.
[16] Jeung M, Baek S, Beom J, Cho KH, Her Y, Yoon K. Evaluation of random forest and regression tree methods for estimation of mass first flush ratio in urban catchments. Journal of Hydrology. 2019; 575:1099-110.
[17] Chen W, Xie X, Wang J, Pradhan B, Hong H, Bui DT, et al. A comparative study of logistic model tree, random forest, and classification and regression tree models for spatial prediction of landslide susceptibility. Catena. 2017; 151:147-60.
[18] Izquierdo-Verdiguier E, Zurita-Milla R. An evaluation of Guided Regularized Random Forest for classification and regression tasks in remote sensing. International Journal of Applied Earth Observation and Geoinformation. 2020; 88:102051.
[19] Hengl T, Nussbaum M, Wright MN, Heuvelink GB, Gräler B. Random forest as a generic framework for predictive modeling of spatial and spatio-temporal variables. PeerJ. 2018;6:e5518.
[20] Najafi-Vosough R, Faradmal J, Hosseini SK, Moghimbeigi A, Mahjub H. Predicting hospital readmission in heart failure patients in Iran: a comparison of various machine learning methods. Healthc Inform Res. 2021; 27(4):307-14.
[21] Breiman L. Random forests. Machine learning. 2001; 45(1):5-32.
[22] Hosseini-Esfahani F, Alafchi B, Cheraghi Z, Doosti-Irani A, Mirmiran P, Khalili D, et al. Using machine learning techniques to predict factors contributing to the incidence of metabolic syndrome in tehran: Cohort study. JMIR Public Health Surveill. 2021; 7(9):e27304.
[23] Pourhomayoun M, Shakibi M. Predicting mortality risk in patients with COVID-19 using machine learning to help medical decision-making. Smart Health. 2021; 20:100178.
[24] Cheng F-Y, Joshi H, Tandon P, Freeman R, Reich DL, Mazumdar M, et al. Using machine learning to predict ICU transfer in hospitalized COVID-19 patients. J Clin Med. 2020; 9(6):1668.
[25] Tezza F, Lorenzoni G, Azzolina D, Barbar S, Leone LAC, Gregori D. Predicting in-Hospital Mortality of Patients with COVID-19 Using Machine Learning Techniques. J Pers Med. 2021;11(5):343.
[26] Wollenstein-Betech S, Cassandras CG, Paschalidis IC. Personalized predictive models for symptomatic COVID-19 patients using basic preconditions: hospitalizations, mortality, and the need for an ICU or ventilator. Int J Med Inform. 2020; 142:104258.
[27] Das AK, Mishra S, Gopalan SS. Predicting CoVID-19 community mortality risk using machine learning and development of an online prognostic tool. PeerJ. 2020; 8:e10083.
[28] Wang J, Yu H, Hua Q, Jing S, Liu Z, Peng X, et al. A descriptive study of random forest algorithm for predicting COVID-19 patients outcome. PeerJ. 2020; 8:e9945.
[29] Kafieh R, Arian R, Saeedizadeh N, Amini Z, Serej ND, Minaee S, et al. Covid-19 in iran: Forecasting pandemic using deep learning. Computational and mathematical methods in medicine. 2021;2021.
[30] Mahdavi M, Choubdar H, Zabeh E, Rieder M, Safavi-Naeini S, Jobbagy Z, et al. A machine learning based exploration of COVID-19 mortality risk. Plos one. 2021;16(7):e0252384.
[31] Orooji A, Kazemi-Arpanahi H, Kaffashian M, Kalvandi G, Shanbehzadeh M. Comparing of Machine Learning Algorithms for Predicting ICU admission in COVID-19 hospitalized patients. Health Education and Health Promotion. 2021;9(3):229-36.
[32] Team RC. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. URLhttps://www.R-project.org/. 2020.
[33] Zhao Z, Chen A, Hou W, Graham JM, Li H, Richman PS, et al. Prediction model and risk scores of ICU admission and mortality in COVID-19. PloS one. 2020;15(7):e0236618.
[34] Ji D, Zhang D, Xu J, Chen Z, Yang T, Zhao P, et al. Prediction for progression risk in patients with COVID-19 pneumonia: the CALL score. Clin Infect Dis. 2020; 71(6):1393-9.
[35] Lu J, Hu S, Fan R, Liu Z, Yin X, Wang Q, et al. ACP risk grade: a simple mortality index for patients with confirmed or suspected severe acute respiratory syndrome coronavirus 2 disease (COVID-19) during the early stage of outbreak in Wuhan, China. MedRxiv. 2020.
[36] Flythe JE, Assimon MM, Tugman MJ, Chang EH, Gupta S, Shah J, et al. Characteristics and outcomes of individuals with pre-existing kidney disease and COVID-19 admitted to intensive care units in the United States. Am J Kidney Dis. 2021; 77(2):190-203. e1.
[37] Buonamico E, Quaranta VN, Boniello E, Dimitri M, Di Lecce V, Labate L, et al. Risk factors for transfer from respiratory intermediate care unit to intensive care unit in COVID-19. Respir Investig. 2021; 59(5):602-7.
[38] Moulaei K, Shanbehzadeh M, Mohammadi-Taghiabad Z, Kazemi-Arpanahi H. Comparing machine learning algorithms for predicting COVID-19 mortality. BMC Med Inform Decis Mak. 2022; 22(1):1-12.
[39] Yadaw AS, Li Y-c, Bose S, Iyengar R, Bunyavanich S, Pandey G. Clinical features of COVID-19 mortality: development and validation of a clinical prediction model. Lancet Digit Health. 2020; 2(10):e516-e25.
[40] Gao Y, Cai G-Y, Fang W, Li H-Y, Wang S-Y, Chen L, et al. Machine learning based early warning system enables accurate mortality risk prediction for COVID-19. Nat Commun. 2020; 11(1):1-10.
[41] Shanbehzadeh M, Haghiri H, Afrash MR, Amraei M. Comparison of Machine Learning Tools for the Prediction of ICU Admission in COVID-19 Hospitalized Patients. Shiraz E Medical Journal. 2022.
[42] Lorenzen SS, Nielsen M, Jimenez-Solem E, Petersen TS, Perner A, Thorsen-Meyer H-C, et al. Using machine learning for predicting intensive care unit resource use during the COVID-19 pandemic in Denmark. Scientific reports. 2021; 11(1):1-10.
| Files | ||
| Issue | Vol 9 No Supp. 2 (2023): Supplement 2 |
|
| Section | Research Article(s) | |
| DOI | https://doi.org/10.18502/aacc.v9i6.14444 | |
| Keywords | ||
| COVID-19 Mortality Intensive Care Units Random Forest Logistic Regression. | ||
| Rights and permissions | |
|
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
How to Cite
1.
Najafi-Vosough R, Bakhshaei MH, Farzian M. Predicting Mortality and ICUs Transfer in Hospitalized COVID-19 Patients Using Random Forest Model. Arch Anesth & Crit Care. 2023;9(Supp. 2):479-487.
