A comparison of machine learning classifiers for pediatric epilepsy using resting‑state functional MRI latency data

Nguyen,Ryan D.; Smyth,Matthew D.; Zhu,Liang; Pao,Ludovic P.; Swisher,Shannon K.; Kennady,Emmett H.; Mitra,Anish; Patel,Rajan P.; Lankford,Jeremy E.; Von Allmen,Gretchen; Watkins,Michael W.; Funke,Michael E.; Shah,Manish N.

doi:10.3892/br.2021.1453

A comparison of machine learning classifiers for pediatric epilepsy using resting‑state functional MRI latency data

Authors:
- Ryan D. Nguyen
- Matthew D. Smyth
- Liang Zhu
- Ludovic P. Pao
- Shannon K. Swisher
- Emmett H. Kennady
- Anish Mitra
- Rajan P. Patel
- Jeremy E. Lankford
- Gretchen Von Allmen
- Michael W. Watkins
- Michael E. Funke
- Manish N. Shah
View Affiliations

Affiliations: Division of Pediatric Neurosurgery, McGovern Medical School at UTHealth, Houston, TX 77030, USA, Department of Neurological Surgery, Washington University School of Medicine, St. Louis, MO 63110, USA, Biostatistics and Epidemiology Research Design Core, Institute for Clinical and Translational Sciences, McGovern Medical School at UTHealth, Houston, TX 77030, USA, Mallinckrodt Institute of Radiology, Washington University School of Medicine, St. Louis, MO 63110, USA, Department of Diagnostic and Interventional Imaging, McGovern Medical School at UTHealth, Houston, TX 77030, USA, Department of Pediatric Neurology, McGovern Medical School at UTHealth, Houston, TX 77030, USA
Published online on: July 23, 2021 https://doi.org/10.3892/br.2021.1453
Article Number: 77
Copyright: © Nguyen et al. This is an open access article distributed under the terms of Creative Commons Attribution License.

Metrics: Total Views: 0 (Spandidos Publications: | PMC Statistics: )

Metrics: Total PDF Downloads: 0 (Spandidos Publications: | PMC Statistics: )

Cited By (CrossRef): 0 citations Loading Articles...

Abstract

Epilepsy affects 1 in 150 children under the age of 10 and is the most common chronic pediatric neurological condition; poor seizure control can irreversibly disrupt normal brain development. The present study compared the ability of different machine learning algorithms trained with resting‑state functional MRI (rfMRI) latency data to detect epilepsy. Preoperative rfMRI and anatomical MRI scans were obtained for 63 patients with epilepsy and 259 healthy controls. The normal distribution of latency z‑scores from the epilepsy and healthy control cohorts were analyzed for overlap in 36 seed regions. In these seed regions, overlap between the study cohorts ranged from 0.44‑0.58. Machine learning features were extracted from latency z‑score maps using principal component analysis. Extreme Gradient Boosting (XGBoost), Support Vector Machines (SVM), and Random Forest algorithms were trained with these features. Area under the receiver operating characteristics curve (AUC), accuracy, sensitivity, specificity and F1‑scores were used to evaluate model performance. The XGBoost model outperformed all other models with a test AUC of 0.79, accuracy of 74%, specificity of 73%, and a sensitivity of 77%. The Random Forest model performed comparably to XGBoost across multiple metrics, but it had a test sensitivity of 31%. The SVM model did not perform >70% in any of the test metrics. The XGBoost model had the highest sensitivity and accuracy for the detection of epilepsy. Development of machine learning algorithms trained with rfMRI latency data could provide an adjunctive method for the diagnosis and evaluation of epilepsy with the goal of enabling timely and appropriate care for patients.

View Figures

View References

1	Aaberg KM, Gunnes N, Bakken IJ, Lund Søraas C, Berntsen A, Magnus P, Lossius MI, Stoltenberg C, Chin R and Surén P: Incidence and prevalence of childhood epilepsy: A nationwide cohort study. Pediatrics. 139(e20163908)2017.PubMed/NCBI View Article : Google Scholar
2	Kwan P and Brodie MJ: Early identification of refractory epilepsy. N Engl J Med. 342:314–319. 2000.PubMed/NCBI View Article : Google Scholar
3	Smith SJ: EEG in the diagnosis, classification, and management of patients with epilepsy. J Neurol Neurosurg Psychiatry. 76 (Suppl 2):ii2–ii7. 2005.PubMed/NCBI View Article : Google Scholar
4	Mihara T, Inoue Y, Matsuda K, Tottori T, Otsubo T, Watanabe Y, Hiyoshi T, Kubota Y, Yagi K and Seino M: Recommendation of early surgery from the viewpoint of daily quality of life. Epilepsia. 37 (Suppl 3):S33–S36. 1996.PubMed/NCBI View Article : Google Scholar
5	Westerveld M, Sass KJ, Chelune GJ, Hermann BP, Barr WB, Loring DW, Strauss E, Trenerry MR, Perrine K and Spencer DD: Temporal lobectomy in children: Cognitive outcome. J Neurosurg. 92:24–30. 2000.PubMed/NCBI View Article : Google Scholar
6	Baulac M, de Boer H, Elger C, Glynn M, Kälviäinen R, Little A, Mifsud J, Perucca E, Pitkänen A and Ryvlin P: Epilepsy priorities in Europe: A report of the ILAE-IBE epilepsy advocacy Europe task force. Epilepsia. 56:1687–1695. 2015.PubMed/NCBI View Article : Google Scholar
7	Berg AT, Baca CB, Loddenkemper T, Vickrey BG and Dlugos D: Priorities in pediatric epilepsy research: Improving children's futures today. Neurology. 81:1166–1175. 2013.PubMed/NCBI View Article : Google Scholar
8	Dlugos DJ: The early identification of candidates for epilepsy surgery. Arch Neurol. 58:1543–1546. 2001.PubMed/NCBI View Article : Google Scholar
9	Gilliam F, Kuzniecky R, Meador K, Martin R, Sawrie S, Viikinsalo M, Morawetz R and Faught E: Patient-oriented outcome assessment after temporal lobectomy for refractory epilepsy. Neurology. 53:687–694. 1999.PubMed/NCBI View Article : Google Scholar
10	Guerrini R: Epilepsy in children. Lancet. 367:499–524. 2006.PubMed/NCBI View Article : Google Scholar
11	Shah MN, Nguyen RD, Pao LP, Zhu L, CreveCoeur TS, Mitra A and Smyth MD: Role of resting state MRI temporal latency in refractory pediatric extratemporal epilepsy lateralization. J Magn Reson Imaging. 49:1347–1355. 2019.PubMed/NCBI View Article : Google Scholar
12	Guidelines for neuroimaging evaluation of patients with uncontrolled epilepsy considered for surgery. Commission on neuroimaging of the international league against epilepsy. Epilepsia. 39:1375–1376. 1998.PubMed/NCBI View Article : Google Scholar
13	Recommendations for neuroimaging of patients with epilepsy. Commission on neuroimaging of the international league against epilepsy. Epilepsia. 38:1255–1256. 1997.PubMed/NCBI View Article : Google Scholar
14	Biswal B, Yetkin FZ, Haughton VM and Hyde JS: Functional connectivity in the motor cortex of resting human brain using echo-planar MRI. Magn Reson Med. 34:537–541. 1995.PubMed/NCBI View Article : Google Scholar
15	Mitra A, Snyder AZ, Blazey T and Raichle ME: Lag threads organize the brain's intrinsic activity. Proc Natl Acad Sci USA. 112:E2235–E2244. 2015.PubMed/NCBI View Article : Google Scholar
16	Mitra A, Snyder AZ, Hacker CD and Raichle ME: Lag structure in resting-state fMRI. J Neurophysiol. 111:2374–2391. 2014.PubMed/NCBI View Article : Google Scholar
17	Jirsa VK, Proix T, Perdikis D, Woodman MM, Wang H, Gonzalez-Martinez J, Bernard C, Bénar C, Guye M, Chauvel P and Bartolomei F: The virtual epileptic patient: Individualized whole-brain models of epilepsy spread. Neuroimage. 145:377–388. 2017.PubMed/NCBI View Article : Google Scholar
18	Pizoli CE, Shah MN, Snyder AZ, Shimony JS, Limbrick DD, Raichle ME, Schlaggar BL and Smyth MD: Resting-state activity in development and maintenance of normal brain function. Proc Natl Acad Sci USA. 108:11638–11643. 2011.PubMed/NCBI View Article : Google Scholar
19	Proix T, Bartolomei F, Guye M and Jirsa VK: Individual brain structure and modelling predict seizure propagation. Brain. 140:641–654. 2017.PubMed/NCBI View Article : Google Scholar
20	Beckmann CF, DeLuca M, Devlin JT and Smith SM: Investigations into resting-state connectivity using independent component analysis. Philos Trans R Soc Lond B Biol Sci. 360:1001–1013. 2005.PubMed/NCBI View Article : Google Scholar
21	Bharath RD, Panda R, Raj J, Bhardwaj S, Sinha S, Chaitanya G, Raghavendra K, Mundlamuri RC, Arimappamagan A, Rao MB, et al: Machine learning identifies ‘rsfMRI epilepsy networks’ in temporal lobe epilepsy. Eur Radiol. 29:3496–3505. 2019.PubMed/NCBI View Article : Google Scholar
22	Boerwinkle VL, Mirea L, Gaillard WD, Sussman BL, Larocque D, Bonnell A, Ronecker JS, Troester MM, Kerrigan JF, Foldes ST, et al: Resting-state functional MRI connectivity impact on epilepsy surgery plan and surgical candidacy: Prospective clinical work. J Neurosurg Pediatr. 1–8. 2020.PubMed/NCBI View Article : Google Scholar : (Online ahead of print).
23	Boerwinkle VL, Mohanty D, Foldes ST, Guffey D, Minard CG, Vedantam A, Raskin JS, Lam S, Bond M, Mirea L, et al: Correlating resting-state functional magnetic resonance imaging connectivity by independent component analysis-based epileptogenic zones with intracranial electroencephalogram localized seizure onset zones and surgical outcomes in prospective pediatric intractable epilepsy study. Brain Connect. 7:424–442. 2017.PubMed/NCBI View Article : Google Scholar
24	Shah MN, Mitra A, Goyal MS, Snyder AZ, Zhang J, Shimony JS, Limbrick DD, Raichle ME and Smyth MD: Resting state signal latency predicts laterality in pediatric medically refractory temporal lobe epilepsy. Childs Nerv Syst. 34:901–910. 2018.PubMed/NCBI View Article : Google Scholar
25	Darcy AM, Louie AK and Roberts LW: Machine learning and the profession of medicine. JAMA. 315:551–552. 2016.PubMed/NCBI View Article : Google Scholar
26	Deo RC: Machine learning in medicine. Circulation. 132:1920–1930. 2015.PubMed/NCBI View Article : Google Scholar
27	Rajkomar A, Dean J and Kohane I: Machine learning in medicine. N Engl J Med. 380:1347–1358. 2019.PubMed/NCBI View Article : Google Scholar
28	Jordan MI and Mitchell TM: Machine learning: Trends, perspectives, and prospects. Science. 349:255–260. 2015.PubMed/NCBI View Article : Google Scholar
29	Cortes C and Vapnik V: Support-vector networks. Mach Learn. 20:273–297. 1995.
30	Chen T and Guestrin C: XGBoost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp785-794, 2016.
31	Liaw A and Wiener M: Classification and regression by randomForest. R News. 2:18–22. 2002.
32	Fern A and Givan R: Online ensemble learning: An empirical study. Mach Learn. 53:71–109. 2003.
33	Mennes M, Biswal BB, Castellanos FX and Milham MP: Making data sharing work: The FCP/INDI experience. Neuroimage. 82:683–691. 2013.PubMed/NCBI View Article : Google Scholar
34	Bellec P, Chu C, Chouinard-Decorte F, Benhajali Y, Margulies DS and Craddock RC: The neuro bureau ADHD-200 preprocessed repository. Neuroimage. 144:275–286. 2017.PubMed/NCBI View Article : Google Scholar
35	Brier MR, Thomas JB, Snyder AZ, Benzinger TL, Zhang D, Raichle ME, Holtzman DM, Morris JC and Ances BM: Loss of intranetwork and internetwork resting state functional connections with Alzheimer's disease progression. J Neurosci. 32:8890–8899. 2012.PubMed/NCBI View Article : Google Scholar
36	Kowerko D: https://www.mathworks.com/matlabcentral/fileexchange/49823-calc_overlap_twonormal(s1,s2,mu1,mu2,xstart,xend,xinterval). MATLAB Central File Exchange. Retrieved July 19, 2021.
37	Jenkinson M, Beckmann CF, Behrens TE, Woolrich MW and Smith SM: FSL. Neuroimage. 62:782–790. 2012.PubMed/NCBI View Article : Google Scholar
38	Verleysen M and François D: The Curse of Dimensionality in Data Mining and Time Series Prediction. Vol 3521. IWANN 2005: Computational Intelligence and Bioinspired Systems, pp758-770, 2005.
39	Meyer D, Dimitriadou E, Hornik K, Weingessel A and Leisch F: e1071: Misc functions of the department of statistics, probability theory group (Formerly: E1071), TU Wien, 2017.
40	R Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2019. Available from: http://www.r-project.org/index.html.
41	RStudio Team: RStudio: Integrated Development for R. RStudio, Inc., Boston MA (Computer Software v0.98.1074), 2015.
42	Sasaki Y: The truth of the F-measure. Teach Tutor Mater, 2007.
43	Baca CB, Vickrey BG, Vassar S, Hauptman JS, Dadour A, Oh T, Salamon N, Vinters HV, Sankar R and Mathern GW: Time to pediatric epilepsy surgery is related to disease severity and nonclinical factors. Neurology. 80:1231–1239. 2013.PubMed/NCBI View Article : Google Scholar
44	Loddenkemper T, Holland KD, Stanford LD, Kotagal P, Bingaman W and Wyllie E: Developmental outcome after epilepsy surgery in infancy. Pediatrics. 119:930–935. 2007.PubMed/NCBI View Article : Google Scholar
45	Berg AT, Vickrey BG, Testa FM, Levy SR, Shinnar S, DiMario F and Smith S: How long does it take for epilepsy to become intractable? A prospective investigation. Ann Neurol. 60:73–79. 2006.PubMed/NCBI View Article : Google Scholar
46	Trevathan E and Gilliam F: Lost years: Delayed referral for surgically treatable epilepsy. Neurology. 61:432–433. 2003.PubMed/NCBI View Article : Google Scholar
47	Elger CE, Helmstaedter C and Kurthen M: Chronic epilepsy and cognition. Lancet Neurol. 3:663–672. 2004.PubMed/NCBI View Article : Google Scholar
48	Sperling MR: Sudden unexplained death in epilepsy. Epilepsy Curr. 1:21–23. 2001.PubMed/NCBI View Article : Google Scholar
49	Nguyen RD, Kennady EH, Smyth MD, Zhu L, Pao LP, Swisher SK, Rosas A, Mitra A, Patel RP, Lankford J, et al: Convolutional neural networks for pediatric refractory epilepsy classification using resting-state functional magnetic resonance imaging. World Neurosurg. 149:e1112–e1122. 2021.PubMed/NCBI View Article : Google Scholar
50	Hastie T, Tibshirani R and Friedman J: The Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer, New York, NY, 2016.
51	Di Martino A, O'Connor D, Chen B, Alaerts K, Anderson JS, Assaf M, Balsters JH, Baxter L, Beggiato A, Bernaerts S, et al: Enhancing studies of the connectome in autism using the autism brain imaging data exchange II. Sci Data. 4(170010)2017.PubMed/NCBI View Article : Google Scholar
52	Di Martino A, Yan CG, Li Q, Denio E, Castellanos FX, Alaerts K, Anderson JS, Assaf M, Bookheimer SY, Dapretto M, et al: The autism brain imaging data exchange: Towards a large-scale evaluation of the intrinsic brain architecture in autism. Mol Psychiatry. 19:659–667. 2014.PubMed/NCBI View Article : Google Scholar
53	Schölkopf B and Smola AJ: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. Adaptive Computation and Machine Learning Series, 2018.