Evaluating Machine Learning Models on Classification of Novel Cyber Attacks in the Healthcare Domain

Author

• Promise Ehimen, Minnesota State University MoorheadFollow

Date of Award

Summer 7-31-2026

Document Type

Project (696 or 796 registration)

Degree Name

Master of Science in Cybersecurity

Department

Graduate Studies

Committee Chair

Dr. Adaeze Nwaigwe

Keywords

Cybersecurity, CICIoMT2024, Internet of Medical Things, Logistic Regression, Machine Learning, Neural Network

Abstract

The increasing adoption of the Internet of Medical Things (IoMT) has improved healthcare delivery through connected medical devices while simultaneously expanding the cybersecurity risks facing healthcare organizations. Although machine learning based intrusion detection systems have demonstrated high detection accuracy, their ability to respond reliably to previously unseen cyberattacks remains uncertain. This study investigated how a Neural Network model and a Logistic Regression model classified novel cyberattacks within the IoMT environment. The Neural Network and Logistic Regression models were both trained and tested using a subset of the CICIoMT2024 benchmark dataset. The Neural Network achieved 99.82% test accuracy and a 0.94 Macro F1, while Logistic Regression achieved 99.77% accuracy and 0.92 Macro F1 score. For the generalization testing the Neural Network and Logistic Regression models were tested on a subset of the CICIoMT2024 dataset which contained a new attack category previously unseen by the models. In terms of how well each model was able to classify the new attack type as some form of attack, both models performed about the same, 78% for the Neural Network model and 77% for the Logistic Regression model. However, the Neural Network model was 8% better at classifying Benign cases during generalization testing. The resulting misclassifications may have incident response implications, including delayed threat investigation, unnecessary alerts and increased workload for a cybersecurity analyst. However, the ability of the simple Logistic Regression model and the more complex Neural Network to generalize to new cases of an unseen attack make them good candidates for consideration in identifying novel and possibly zero-day cyber-attacks in the IoMT domain.

Recommended Citation

Ehimen, Promise, "Evaluating Machine Learning Models on Classification of Novel Cyber Attacks in the Healthcare Domain" (2026). Dissertations, Theses, and Projects. 1158.
https://red.mnstate.edu/thesis/1158