[Research Briefing] SHIELD: An Efficient ML Framework for Healthcare IoT Anomaly Detection

Introduction

In modern medical environments, the adoption of IoT devices has drastically increased patient monitoring and diagnostic efficiency. However, this expansion of connectivity has simultaneously increased the attack surface for external cyberattacks and the risk of malfunctions due to inherent device faults.

Given the nature of medical data, minute numerical errors or brief system outages can directly impact patient lives. Therefore, there is an urgent need for an integrated detection system capable of identifying both physical device faults and network-based attacks, moving beyond simple security solutions.

From an operational perspective for enterprises and hospitals, the feasibility of implementation depends on securing ‘lightweight yet accurate’ models that can operate rapidly at the wearable or edge device level, rather than deploying expensive, high-performance servers.

This research provides an engineering basis for determining which algorithms are most effective under the specific constraints of medical IoT by comparing various ML paradigms.

Core Findings

This study aims to resolve the issues of security vulnerabilities and operational errors caused by increased connectivity in IoT medical devices, which threaten patient safety and data privacy. To this end, a dataset consisting of 200,000 records collected from patient monitoring sensors and control units in an ICU environment was utilized.

The researchers employed a three-stage approach: data preprocessing, feature selection, and model evaluation. In the feature selection stage, an optimal feature set was constructed by integrating ANOVA F-value, Mutual Information, and Recursive Feature Elimination (RFE). A total of eight models were analyzed, including supervised learning (XGBoost, KNN), semi-supervised learning (GAN, VAE), and unsupervised learning (One-Class SVM, Isolation Forest, GNN, LSTM Autoencoder).

Regarding device fault detection, XGBoost demonstrated the superior performance, recording 99% accuracy and a low computational cost of 0.04 seconds. Isolation Forest also showed high accuracy and recall, whereas GANs entailed relatively high computational costs. LSTM Autoencoders and GNNs proved inefficient due to low accuracy and high latency.

In cyberattack detection, KNN exhibited nearly perfect performance in precision, recall, and F1-score, with the lowest computational cost of 0.05 seconds. VAE followed with 97% accuracy, while GAN showed the lowest performance with 83% accuracy and an ROC-AUC of 0.72.

Deep Analysis of Key Failure Factors

First is the risk of model mismatch relative to the characteristics of data generated in healthcare IoT environments. Results indicate that complex models such as LSTM Autoencoders and GNNs required high computational costs yet yielded low anomaly detection accuracy for medical data. This implies that data complexity does not necessarily correlate with the required structural complexity of the model; introducing excessively heavy models actually degrades real-time detection performance.

Second is the reliability issue of semi-supervised and unsupervised learning models. GANs showed poor performance in cyberattack detection, with 83% accuracy and an ROC-AUC of 0.72. Notably, GANs had the highest computational cost despite the poor results, suggesting that adversarial learning methods have limitations in learning specific attack patterns within healthcare IoT.

Third is the risk of losing the response ‘golden time’ due to computational latency. The LSTM Autoencoder recorded an extremely high computational cost of 292.127 seconds during cyberattack detection. In an ICU environment where responses are required in seconds, such latency is a critical flaw, as damage would have already occurred by the time detection succeeds.

Source: Photo by Hitesh Choudhary on Unsplash

Details on SHIELD Framework Data Preprocessing and Feature Engineering

The SHIELD framework is based on 200,000 records collected from patient monitoring sensors and control units in an ICU environment. The analyzed data is divided into ‘IoT Medical Infrastructure Attack Data’ (Table I) and ‘Medical Device Data’ (Table II). Attack data captures malicious network activity, including network traffic metadata, IP and port information, and TCP/MQTT protocol flags. Device data focuses on identifying physical hardware faults, including patient identification, sensor types, temperature, blood pressure (systolic/diastolic), heart rate, battery levels, and control parameters such as target health states.

During the data cleaning phase, Median Imputation was applied to handle missing values in numerical fields. This choice maintains data integrity by minimizing the impact of extreme outliers—common in medical sensor data—on the mean value. Categorical variables, such as Sensor Type and Source IP (ip.src), were converted into binary format via One-Hot Encoding to prevent the model from misinterpreting categorical data as ordinal data and to preserve intrinsic characteristics.

In the normalization process, different scaling techniques were strategically used based on the characteristics of the ML models. For traditional ML models like XGBoost, StandardScaler was used to standardize features and improve learning efficiency. Conversely, deep learning models such as VAE, GAN, and LSTM Autoencoders used MinMax Scaling to accelerate convergence and stabilize the learning process during gradient descent optimization.

To increase detection precision beyond raw data, feature engineering incorporating domain knowledge was performed. Specifically, Heart Rate Deviation (HRD) and Blood Pressure Deviation (BPD) features were created to capture variability in vital signs. These were calculated as the absolute difference between real-time measurements and the rolling mean within a predefined window, enabling the effective identification of abnormal fluctuations or sensor malfunctions even when values remain within a general numerical range.

Additionally, time-based features were extracted for analysis. By extracting the hour of the day and day of the week, the framework can identify temporal patterns, such as cyberattack spikes during shift changes or sensor faults occurring after prolonged operation. This temporal context helps the model learn correlations between the timing of anomalies and environmental factors.

From a network security perspective, a TCP Anomaly Score was introduced by analyzing TCP flag distributions and traffic behavior. Specifically, scores were calculated by analyzing statistical deviations of protocol flags specified in Table I, such as tcp.flags.ack, tcp.flags.push, tcp.flags.reset, and tcp.flags.syn. A higher score indicates a higher probability of malicious activity, serving as a key indicator for detecting sophisticated cyberattacks that are difficult to identify via network metadata alone.

This rigorous preprocessing and feature engineering process provides the model with deeper insights than raw data. HRD, BPD, and the TCP Anomaly Score, reflecting domain knowledge, served as the foundation for precisely distinguishing between physical faults and network threats unique to healthcare IoT, ultimately enriching the dataset for higher precision detection.

Integrated 3-Stage Feature Selection Process

To improve analysis accuracy and optimize computational efficiency, the SHIELD framework integrated three different feature selection techniques: ANOVA F-value, Mutual Information (MI), and Recursive Feature Elimination (RFE). This multifaceted approach eliminates the bias inherent in any single technique and is a strategic choice to precisely capture anomalies of different natures, namely cyberattacks and device faults.

First, the ANOVA F-value technique was used to select features with the highest statistical influence. For cyberattack detection, key features selected included tcp.connection.syn, frame.time delta, ip.proto, frame.len, tcp.dstport, mqtt.ver, tcp.ack, tcp.flags.syn, and mqtt.msgtype. For device fault detection, indicators directly linked to patient vital signs, such as Temperature (°C), Systolic BP (mmHg), and Heart Rate (bpm), were derived as primary features.

Next, the Mutual Information (MI) technique was applied to capture both linear and non-linear relationships, unlike ANOVA. This allowed for the identification of complex network pattern features in cyberattack detection, such as tcp.time delta, mqtt.topic len, tcp.flags.reset, tcp.flags.push, tcp.hdr len, ip.ttl, and mqtt.dupflag. For device fault detection, Diastolic BP (mmHg), Heart Rate (bpm), and Device Battery Level (%) were ranked as features with high predictive power.

Finally, the Recursive Feature Elimination (RFE) technique further refined the optimal feature subset contributing to model performance. For cyberattack detection, mqtt.qos, mqtt.retain, tcp.flags.ack, tcp.flags.fin, frame.time relative, tcp.connection.rst, and mqtt.clientid len were finally selected. For device fault detection, Temperature (°C), Diastolic BP (mmHg), and Heart Rate (bpm) remained the most relevant features, consistent with previous analyses.

The final dataset, integrating the results of these three techniques, was optimized for each detection objective. The final feature set for cyberattack detection consisted of 28 unique features, including network metadata, TCP flags, and MQTT protocol parameters. This provides a basis for comprehensively analyzing minute fluctuations in network traffic and protocol-level anomalies.

The final feature set for device fault detection was compressed into key indicators deemed important across all three techniques. Specifically, Temperature (°C), Systolic BP (mmHg), Diastolic BP (mmHg), Heart Rate (bpm), and Device Battery Level (%) were selected. By retaining only core vital sign indicators, model complexity was reduced and detection speed was maximized.

Consequently, this integrated feature selection process not only improved model accuracy by removing unnecessary data noise but also improved real-time performance by reducing computation time. This was a decisive factor in ensuring the model operates rapidly in resource-constrained healthcare IoT environments and laid the groundwork for the high-efficiency performance of XGBoost and KNN models.

Comparative Performance Analysis by Learning Paradigm

The SHIELD framework validated eight models across three distinct learning paradigms: supervised, semi-supervised, and unsupervised learning. Analysis revealed a clear performance gap across paradigms in terms of overall performance metrics (precision, recall, F1-score, ROC-AUC) and computational efficiency. Supervised learning models, utilizing datasets with clear ground-truth labels, demonstrated overwhelmingly superior detection performance and real-time processing capabilities compared to other paradigms.

Supervised Learning models, XGBoost and KNN, recorded top-tier performance across all evaluation metrics. In device fault detection, XGBoost achieved 99% accuracy with perfect precision and recall, while its computational cost was the lowest at 0.04 seconds. In cyberattack detection, KNN achieved nearly perfect performance in precision, recall, and F1-score, with a very low computational cost of 0.05 seconds. This proves that supervised learning, which learns predefined patterns, is most efficient for identifying standardized anomalies in healthcare IoT.

Semi-supervised Learning models, VAE and GAN, showed contrasting results. VAE maintained respectable performance with 97% accuracy in cyberattack detection and produced balanced results across precision, recall, and ROC-AUC in device fault detection. In contrast, GAN exhibited the lowest efficiency among semi-supervised models. It performed poorly in cyberattack detection (83% accuracy, 0.72 ROC-AUC) and had very high computational costs, confirming limitations for real-time system application.

Unsupervised Learning models showed significant performance variance depending on the algorithm. Isolation Forest demonstrated strong performance in device fault detection, showing nearly perfect accuracy and recall. One-Class SVM recorded an F1-score of 0.7894 and a low computational cost of 0.1122 seconds in device fault detection, positioning it as a lightweight alternative for simple systems where high precision is not required. However, it suffered from high computational costs during cyberattack detection.

Deep learning models within the unsupervised group, GNN and LSTM Autoencoder, generally showed inadequate performance. Although GNN recorded a very high ROC-AUC in cyberattack detection, its low precision and recall reduced actual detection efficiency. The LSTM Autoencoder showed even more severe efficiency issues, with a computational cost of 292.127 seconds for cyberattack detection—the slowest among all models. It also showed low ROC-AUC and accuracy in device fault detection, demonstrating that complex model architecture does not necessarily lead to high detection performance.

In conclusion, synthesizing the metrics from Figures 2 through 13, supervised learning models implemented maximum detection accuracy with minimum resources. Conversely, unsupervised deep learning models failed to match supervised models in precision and recall despite high computational costs. These results provide engineering evidence that efficient supervised models like XGBoost and KNN are the optimal choice for domains like healthcare IoT, where rapid response speed and high reliability are simultaneously required.

Optimal Application Scenarios by Model for Healthcare IoT

Based on the research results, optimal application scenarios for each model differ according to the nature of the detection target and system resource constraints. First, XGBoost is most suitable for scenarios detecting physical faults and operational anomalies in medical devices. With 99% accuracy and perfect precision/recall, and a very low computational cost of 0.04 seconds, it provides optimal performance for device monitoring in ICU environments where real-time responsiveness is essential.

For cyberattack detection scenarios, KNN has the highest utility. KNN proved nearly perfect performance in precision, recall, and F1-score while maintaining a minimal computational cost of 0.05 seconds. This suggests that KNN is a highly efficient alternative for network security where known attack patterns must be identified rapidly, ensuring high reliability even in resource-constrained environments.

In unsupervised learning scenarios where new types of anomalies must be captured without labels, Isolation Forest is an effective choice. It balanced performance and efficiency, recording high accuracy and recall in fault detection. Despite some limitations in ROC-AUC, its practical utility is high as it can effectively isolate outliers without complex predefined rules.

Meanwhile, VAE showed respectable performance with approximately 97% accuracy, making it viable as a supplementary detection system to augment supervised models. However, GAN’s efficiency is low for current healthcare IoT security scenarios; despite decent performance in fault detection, its computational cost was excessive, and it ranked lowest in cyberattack detection (83% accuracy, 0.72 ROC-AUC).

One-Class SVM showed moderate performance with an F1-score of 0.7894 but has the strength of a very low computational cost (0.1122 seconds). Thus, it can be considered a practical choice for simple monitoring environments that must minimize system load or for low-power edge devices requiring a lightweight detection layer.

The scalability of the SHIELD framework lies in the modular integration of these models. Real-time detection can be implemented on low-power edge devices, such as wearable health monitors or IoT-connected medical sensors, using lightweight models like XGBoost and KNN. Integration with cloud-based security solutions further enables the expansion into large-scale anomaly detection systems across multiple medical facilities.

For future performance enhancement, as suggested in the original text, the framework can evolve by integrating Federated Learning to collaboratively train models while protecting patient privacy. Additionally, introducing time-series models and reinforcement learning is expected to provide the ability to adapt more flexibly to evolving cyber threats and operational anomalies.

In conclusion, to simultaneously satisfy the two core values of ‘accuracy’ and ‘real-time responsiveness’ in mission-critical medical environments, the SHIELD framework proposes a hierarchical defense strategy centered on supervised XGBoost and KNN, supplemented by unsupervised models like Isolation Forest.

Model Selection and Validation Checklist for Medical IoT Anomaly Detection

To prevent system collapse in practical engineering pipelines, the following core defense mechanisms must be mandated during the operational phase:

• Inference Latency Measurement: Does the model’s prediction time satisfy the real-time response threshold (e.g., within 0.1 seconds)?
• Model Lightweighting: Does the model have a computational complexity suitable for operation on edge devices or low-power sensors?
• Supervised Data Acquisition: Are known attack patterns and fault data sufficiently labeled to enable the use of XGBoost/KNN?
• Feature Selection Optimization: Have unnecessary features been removed via ANOVA, MI, and RFE to reduce model load?
• Recall-First Validation: Is the threshold set to minimize false negatives to prevent medical accidents?
• Data Normalization Strategy: Are StandardScaler (traditional ML) and MinMax scaling (deep learning) applied distinctly based on model characteristics?
• Outlier Handling Method: Is median imputation used instead of the mean to reduce the impact of extreme values in medical data?
• Cross-Validation Performance: Is performance consistency verified across supervised, semi-supervised, and unsupervised models to prevent model bias?

Data Collection and ICU Configuration

The dataset used to verify the performance of the SHIELD framework consists of 200,000 records. This dataset is divided into two core areas: Faulty Device Data, which captures functional anomalies of medical devices, and Attack Data, which records malicious network activity targeting medical infrastructure. This binary configuration is a strategic choice to integrally detect anomalies of different natures: physical hardware faults and external cyber threats.

The data was collected from a system simulating a real-world Intensive Care Unit (ICU) setup. Specifically, a two-bed ICU environment was constructed, with nine patient monitoring sensors deployed per bed to monitor patient status in real-time. This multi-sensor configuration secures complex vital sign data and enables anomaly detection not only for single sensor faults but also through correlations between multiple sensors.

To manage and transmit the collected data, a Bedx-Control-Unit was installed at each bed. This control unit plays a pivotal role in aggregating data from each sensor and transmitting it externally. Therefore, the dataset includes both individual sensor measurements and network traffic information delivered via the control unit, providing a basis for simultaneous analysis of device-level errors and network-level attacks.

Faulty device data focuses on functional anomalies such as temperature fluctuations, power failures, and sensor misreads. This includes essential vital monitoring parameters such as body temperature, systolic/diastolic blood pressure, heart rate, and battery levels. Furthermore, the system was designed to collect Control Parameters, such as target blood pressure or target heart rate, alongside current values, allowing anomalies to be judged based on the discrepancy between expected and actual measurements.

In contrast, attack data focuses on malicious network activity targeting IoT-based medical infrastructure. This includes network traffic metadata such as frame time delta, relative time, and frame length, as well as source/destination IP addresses, TCP port numbers, MQTT client IDs, and message types. Specifically, it analyzes TCP flags (ACK, FIN, PUSH, RESET, SYN) and MQTT protocol flags (QoS, Retain, Topic) to identify attacks exploiting protocol vulnerabilities or abnormal communication patterns.

As a result, the SHIELD framework has secured data encompassing both the physical layer (sensors and control units) and the network layer (TCP/MQTT protocols) of the specialized ICU environment. This comprehensive data collection system serves as a powerful foundation for efficiently detecting and analyzing various forms of threats and errors within a single framework.

Feature Engineering Details

Beyond the basic information provided by raw data, the SHIELD framework significantly increased detection precision through feature engineering combined with domain knowledge. Recognizing that temporal variation patterns in vital signs are more meaningful than absolute values in medical settings, derived features called Heart Rate Deviation (HRD) and Blood Pressure Deviation (BPD) were created.

These deviation values were calculated as the absolute difference between real-time measurements and the Rolling Mean within a predefined window. This allows the system to identify abrupt volatility or subtle irregularities rather than simply finding values outside a normal range, enabling more sophisticated identification of patient health anomalies or medical sensor malfunctions.

Additionally, Time-based features were extracted and reflected in the model to understand the context of anomaly occurrence. Specifically, ‘Hour of the day’ and ‘Day of the week’ information were added to the dataset. This accounts for the specificity of medical environments, analyzing temporal correlations such as cyberattack patterns concentrating during medical staff shift changes or sensor degradation and faults occurring due to prolonged continuous operation.

In the network security domain, a new metric called the TCP Anomaly Score was introduced, moving beyond simple packet analysis. This score was calculated by statistically analyzing the distribution of TCP flags, such as tcp.flags.ack, tcp.flags.push, tcp.flags.reset, and tcp.flags.syn, and the overall behavior of network traffic.

A higher TCP Anomaly Score, based on statistical deviation, indicates a higher probability that the traffic is malicious, providing a basis for more effectively detecting sophisticatedly disguised cyberattacks. These network-based derived features play a key role in identifying attacks on MQTT protocol vulnerabilities.

Consequently, SHIELD expanded the dataset through feature engineering across three perspectives: vital sign variability, temporal context, and statistical characteristics of network traffic. This approach converted complex anomaly patterns, difficult to discover with raw data alone, into explicit features for the model, resulting in higher overall detection accuracy and reliability.

These derived features are then passed to the subsequent feature selection stage. By selecting only the core features that actually contribute to detection performance through techniques like ANOVA F-value, Mutual Information (MI), and Recursive Feature Elimination (RFE), the framework lowered computational costs and optimized real-time detection performance.

Model Hyperparameters

The SHIELD framework precisely configured the hyperparameters of each model to match the characteristics of device fault detection and cyberattack detection. Model specifications are detailed in Table III (Device Fault Detection) and Table IV (Cyberattack Detection), reflecting the differences in dataset characteristics and learning paradigms rather than simple performance comparisons.

For the supervised learning model XGBoost, a learning rate of 0.1, max depth of 6, and logloss as the evaluation metric were used for both tasks. The KNN model adopted k=5 and Euclidean distance measurement to perform distance-based classification between data points.

For the semi-supervised model GAN, the latent space was set to 10. For device fault detection, 100 epochs and a batch size of 64 were applied; however, for cyberattack detection, epochs were significantly increased to 1,000 considering the complexity of the learning task. The VAE model used a latent space of 2, 100 epochs, and a batch size of 32 for device fault detection, but expanded the latent space to 10 and increased epochs to 200 for cyberattack detection to enhance model expressiveness.

Among unsupervised models, One-Class SVM used the RBF kernel. For device fault detection, the nu value was set to 0.2; for cyberattack detection, nu 0.1 and gamma 0.1 were applied to define the decision boundary for normal data. For Isolation Forest, the contamination parameter was set to 0.2 for device fault detection and 0.1 for cyberattack detection to adjust the outlier judgment criteria.

Deep learning-based GNN used 16 hidden channels, 100 epochs, and a learning rate of 0.01 for device fault detection. For cyberattack detection, hidden channels were expanded to 32 to accommodate the complexity of the network structure, while maintaining 100 epochs and a 0.01 learning rate. Finally, the LSTM Autoencoder used 64 LSTM units for both tasks, with 100 epochs and a batch size of 32 for device fault detection, and 200 epochs and a batch size of 32 for cyberattack detection to learn the reconstruction error of time-series data.

These hyperparameter settings are strategic choices to maximize the algorithmic characteristics of each model. Increasing epochs and latent space size for cyberattack detection models aims to learn more complex patterns in network traffic data. Conversely, device fault detection models maintained relatively lower hyperparameters to ensure computational efficiency and manage overfitting risks.

As a result, the SHIELD framework implemented eight models based on these systematically defined hyperparameters. This process went beyond simple benchmarking to explore the optimal performance points reachable by each learning paradigm within the specific constraints of a healthcare IoT environment.

Scalability and Future Research Directions

One of the greatest strengths of the SHIELD framework is its exceptional Scalability. The framework is designed to adapt flexibly to a wide range of medical environments, from small clinics to large hospital networks. A key advantage is the ability to perform real-time anomaly detection on low-specification edge devices with low power consumption by adopting computationally efficient models like XGBoost and KNN. This means immediate security and fault responses are possible at the endpoint level, such as wearable health monitors or IoT-connected medical sensors.

Furthermore, SHIELD’s modular design allows for seamless integration with cloud-based security solutions. This design makes it possible to build and operate large-scale anomaly detection systems across multiple medical facilities regardless of location. By combining primary detection at the edge—where hardware constraints are severe—with integrated management at the cloud level, a foundation has been laid to simultaneously ensure the security and reliability of the entire medical infrastructure.

Regarding future research, the integration of Federated Learning was proposed to enhance model performance while strengthening patient privacy. Introducing federated learning would allow individual medical institutions to collaboratively train models without sharing sensitive patient data externally, expanding the universality of detection models while minimizing privacy breach risks.

The introduction of Time-series models to more sophisticatedly analyze temporal contexts was also mentioned as an important research task. Since medical data has strong time-series characteristics with continuous flow, specialized models could enable more precise anomaly capture. Additionally, by applying Reinforcement Learning, the goal is to build a system that actively adapts to evolving, sophisticated cyber threats and new forms of operational anomalies over time.

Finally, to maximize system effectiveness, the optimization of Real-time streaming analytics and Low-latency decision-making technologies will be pursued. These technical advancements will increase the speed of analysis and response immediately as data is generated, further increasing the value of the SHIELD framework in high-demand medical network environments requiring high-precision management.

In conclusion, SHIELD aims not to settle for current efficient detection performance but to play a core role in protecting next-generation smart healthcare infrastructure through distributed learning, adaptive modeling, and ultra-low-latency analysis systems. This direction will be a powerful means of preemptively responding to security threats and device fault risks that will become more complex as the medical IoT ecosystem expands.

Author’s Perspective and Practical Suggestions

The most interesting point in the research results is that the latest deep learning techniques were outperformed by classical ML models. In the industry, there is often a fallacy that ‘latest model = best performance,’ leading to excessive resource investment. However, this study reaffirms that depending on the characteristics of the domain data, simple tree-based or distance-based models can be far more powerful.

Particularly in the healthcare field, recall is as important as precision because missing an attack is far more dangerous than a false positive. The high recall shown by Isolation Forest suggests that a strategy of deploying unsupervised models in early stages when labels are scarce, and then transitioning to XGBoost as data accumulates, would be effective.

However, this research is based on a dataset from a specific ICU environment. Actual hospital environments are far noisier and involve a wider variety of equipment; therefore, when applying the proposed SHIELD framework, a continuous monitoring system for data drift must be implemented in parallel.

In conclusion, engineering leaders should set ‘inference time’ and ‘resource consumption’ as core KPIs beyond mere ‘accuracy’ figures, as these directly correlate with patient safety.

Conclusion and Enterprise Governance Guide

To ensure that internal AI projects do not remain as early-stage PoC demos but settle into actual business value, the following governance principles must be established from the planning stage:

1. Real-time Priority Principle: For IoT detection models connected to life-support systems, manage inference latency as the primary metric over accuracy.
2. Model Efficiency Validation Principle: Before introducing complex deep learning models, always perform a cost-performance analysis against simple ML baselines.
3. Multi-layered Detection Strategy Principle: Build a hybrid system that detects known threats via supervised learning and unknown new threats via unsupervised learning.
4. Data Integrity Assurance Principle: Standardize preprocessing pipelines (such as median imputation) that are robust to outliers, considering the specificity of medical data.
5. Edge-First Processing Principle: Design detection logic to be processed at the edge as much as possible to reduce network latency and data leakage risks.

References and Original Analysis Materials