Monthly Archives: September 2026

You are browsing the site archives by month.

Time series events recognition in embedded devices

September 10, 2026 11:48 ,

Pau Gairí, Tomas Palleja, Marcel Tresanchez, Real-time recognition of multivariate event-based time series on embedded devices using recurrent neural networks: A practical study, Engineering Applications of Artificial Intelligence, Volume 182, Part 1, 2026,10.1016/j.engappai.2026.115787.

Real-time time-series recognition on embedded devices remains challenging due to limited computational resources and memory capacity, but also because of the operational constraints imposed by randomly triggered, event-driven sensing applications with variable duration. Most deep learning approaches rely on offline processing, complete-sequence inference, and pre-established deployment frameworks. In contrast, hardware-aware design along with recurrent architectures can distribute inference across the time-series evolution, enabling low latency solutions that better suit time-critical applications. This work combines recurrent neural networks with gated units and microcontroller-oriented processing to enable multivariate, event-based classification directly on embedded hardware. On one hand, a peripheral-driven approach for acquisition, preprocessing, and sample-rate adaptation is introduced, including real-time dynamic linearization and normalization strategies for incomplete time series. On the other hand, a lightweight deep recurrent architecture is tailored to the characteristics of each input and extended with a multi-objective output. Additionally, a dedicated multi-label dataset together with an automated clustering and labelling methodology enables an event-based training strategy with shared objectives that ensures stable inference and efficient memory usage on the device. The proposed methodology was evaluated across different architectural configurations, identifying optimal trade-offs between computational cost and classification accuracy. The on-device implementation shows that 32-bit floating-point inference using the microcontroller Floating-Point Unit avoids additional compression while preserving up to 99% of the best model performance. The optimized kernel achieves a macro F1-score (average of per-class harmonic means of precision and recall) of 0.840, while real-time validation under real application conditions confirms robust operation with a macro F1-score of 0.898.

Posted in: Systems and Signals , Tagged: , , ,

Explaining the results of RL in continuous spaces

September 3, 2026 09:35 ,

R. Zuo, S. Khan, Z. Wang, G. E. Katz and Q. Qiu, Beyond VisionMask: Explaining Reinforcement Learning Via Contrastive-State Masking, IEEE Intelligent Systems, vol. 41, no. 4, pp. 56-63, July-Aug. 2026,10.1109/MIS.2026.3683960.

Existing explainable reinforcement learning (RL) approaches often fail to capture the contrastive nature of human reasoning—answering “why this action instead of that one?”. To address this challenge, we present Continuous VisionMask (cVM), a contrastive learning framework that explains RL agents in continuous state and action spaces. cVM extends our prior work, VisionMask, which was restricted to vision-based discrete settings, by introducing a quantized representation of continuous state and a set of contrastive learning objectives. This generalization enables cVM to provide faithful, robust, and sparse attributions highlighting which aspects of the sensed input are most important for the agent’s decision making, and to do so across a broader range of real-world RL scenarios. We evaluate cVM in multiple continuous-control environments and compare it against existing explainability baselines. Our results demonstrate that cVM produces more faithful and stable explanations, thereby enhancing transparency and interpretability in continuous RL systems.

Posted in: Reinforcement learning in AI , Tagged:

Unlearning repeatitive cycles in RL

September 3, 2026 09:04 ,

Dinesh Kumar Banjare, Mithilesh Atulkar, Mitul Kumar Ahirwal, Utilizing machine unlearning for faulty behavior patterns in human decision-making model: A sharded reinforcement learning approach, Engineering Applications of Artificial Intelligence, Volume 181, Part 6, 2026,10.1016/j.engappai.2026.115652.

In this paper, enhance the computational models of human decision-making by proposing a novel sharded reinforcement learning framework based on the principle of machine unlearning. The work proposes a data sharding and selective machine unlearning approach to unlearn faulty behavior patterns. It overcomes the drawbacks of conventional reinforcement learning methods, in which the players tend to play the same deck over and over again, showing a lack of adaptability. These problems are frequently seen in tasks such as the Iowa gambling task. To handle this issue, the concept of machine unlearning is used. The proposed method detects repeated patterns of faulty behavior, such as repeated deck selections. Such patterns are corrected locally at the level of shards, while retaining the last stable expectancy state from the previous learning stage. The mean squared deviation was used to evaluate the model’s performance. The results indicate that the unlearned model learns faster and has better performance in the early phases of learning. The framework supports behavioral adaptability by selectively excluding identified faulty shards while preserving previous valid learning. These findings show the potential of shard-based learning and selective unlearning in sequential decision-making.

Posted in: Reinforcement learning in AI , Tagged:

A survey on DRL applied to control engineering

September 3, 2026 08:51 ,

Klinsmann Agyei, Pouria Sarhadi, Daniel Polani, Deep reinforcement learning in applied control: Challenges, analysis, and insights, Engineering Applications of Artificial Intelligence, Volume 181, Part 4, 2026, 10.1016/j.engappai.2026.115524.

Deep reinforcement learning (DRL) algorithms have shown increasing promise for autonomous decision-making and continuous control, yet their practical value relative to established control methods remains difficult to assess under realistic control-engineering conditions. Existing evaluations are often narrow, application-specific, or based on inconsistent assumptions. As a result, it remains unclear when learned controllers are suitable alternatives to conventional designs. This paper presents a systematic assessment framework that compares four prominent DRL controllers with a classical control baseline across a diverse set of applied control problems, including non-minimum phase dynamics, flexible mechanical systems, nonlinear marine control, and aerial robotics. The framework standardises modelling assumptions, reward design, controller tuning, computational budgets, performance metrics, robustness tests, and deployment conditions. This allows fair and reproducible analysis across methods and applications. The evaluation covers tracking accuracy, settling time, overshoot, control effort, actuator saturation, disturbance rejection, model uncertainty, robustness margins, and sim-to-real transfer. The results show that performance depends strongly on the algorithmic structure, system dynamics, and operating conditions. Learned controllers provide advantages in some cases, particularly for nonlinearities, constraints, and uncertain operating regimes. Classical control remains competitive in accuracy, simplicity, and reliability for several benchmark conditions. Under combined saturation and unmodelled dynamics, the learned controllers maintained stable operation where the classical baseline became unstable. Real-time deployment on a quadrotor platform further demonstrated stable sim-to-real transfer. The study establishes a rigorous and reproducible evidence base for assessing the practical suitability of these data-driven control algorithms in applied control. It clarifies the trade-offs between learning-based and conventional control. This addresses a critical gap that cannot be easily inferred from isolated, system-specific studies.

Posted in: Applications of reinforcement learning to control engineering , Tagged:

Hierarchical adversarial RL

September 3, 2026 08:19 ,

Xiaofeng Wang, Shaopu Wang, Jipeng Zhang, Yue Ma, Hierarchical multi-policy adversarial reinforcement learning, Engineering Applications of Artificial Intelligence, Volume 181, Part 1, 2026, 10.1016/j.engappai.2026.115297.

In complex and dynamic autonomous driving environments, deep reinforcement learning policies often struggle with generalization due to distribution shifts between training and deployment. While conventional approaches like domain randomization and adversarial training offer partial solutions, they face notable limitations. Specifically, domain randomization relies on manually defined distributions that poorly capture real-world perturbations, while adversarial training suffers from high computational costs, unstable optimization, and behavioral homogenization. To address these issues, we propose a hierarchical multi-policy adversarial reinforcement learning framework that enhances policy generalization under complex perturbations. It incorporates three key components: a multi-policy generation architecture with a shared backbone and independent heads for diverse adversary modeling; a dual-perspective policy expansion regularization mechanism; and an adaptive curriculum-based adversarial training framework. The engineering value translates to a reliable, high-performance solution for autonomous driving, as validated on high-fidelity driving benchmarks. Experimentally, our approach demonstrates superior convergence, stability, and out-of-distribution generalization, achieving performance improvements ranging from 7.36% to 50.06% over established baseline methods while maintaining computational efficiency. These results, obtained in a high-fidelity simulator, demonstrate promise for real-world autonomous driving applications and lay the groundwork for future real-world deployment.

Posted in: Reinforcement learning in AI , Tagged: ,

Analyzing Li-Ion battery health

September 3, 2026 08:14 ,

Haixu Yang, Fengwei Liang, Kerui Li, Jichao Hong, Jinghan Zhang, Xiaohui Chen, State of health estimation and remaining useful life prediction of Li-ion battery oriented to real-world electric vehicles: A comprehensive evaluation methodology, Engineering Applications of Artificial Intelligence, Volume 181, Part 1, 2026, 10.1016/j.engappai.2026.115311.

State of health estimation is essential to ensure the battery’s reliability, safety and economy. It is an integral part of the battery management system. Estimation often faces the problems of a lone indicator and ineffective real-vehicle application. The use of a lone indicator leads to biased and inaccurate state estimation. The complexity of real-world driving scenarios renders the method impractical and inaccurate. Here, we propose a comprehensive evaluation method oriented to real-world electric vehicles. Capacity, ohmic internal resistance, cell voltage inconsistency, and battery pack temperature inconsistency are considered as four indicators of battery degradation. For each indicator, a state of health estimation value can be obtained. A fuzzy matrix is used to fuse the four indicators for a comprehensive evaluation. Also, the fuzzy matrix gives the weights of different health indicators as 0.1988, 0.2463, 0.3183, and 0.2366. The inconsistency of cell voltages carries the highest weighting among these indicators, exerting a more significant influence than the other three. In terms of artificial intelligence, this paper proposes a complete set of data processing, feature computation, and fusion estimation methods for battery health state. In engineering applications, the metrics for health state assessment are greatly enriched and the degree of accuracy is significantly improved. The method breaks through the limitations of the traditional evaluation based on a singular indicator, achieving a multi-dimensional comprehensive and objective evaluation of automotive batteries.

Posted in: Electronics , Tagged: ,