Time series events recognition in embedded devices

September 10, 2026 11:48 ,

Pau Gairí, Tomas Palleja, Marcel Tresanchez, Real-time recognition of multivariate event-based time series on embedded devices using recurrent neural networks: A practical study, Engineering Applications of Artificial Intelligence, Volume 182, Part 1, 2026,10.1016/j.engappai.2026.115787.

Real-time time-series recognition on embedded devices remains challenging due to limited computational resources and memory capacity, but also because of the operational constraints imposed by randomly triggered, event-driven sensing applications with variable duration. Most deep learning approaches rely on offline processing, complete-sequence inference, and pre-established deployment frameworks. In contrast, hardware-aware design along with recurrent architectures can distribute inference across the time-series evolution, enabling low latency solutions that better suit time-critical applications. This work combines recurrent neural networks with gated units and microcontroller-oriented processing to enable multivariate, event-based classification directly on embedded hardware. On one hand, a peripheral-driven approach for acquisition, preprocessing, and sample-rate adaptation is introduced, including real-time dynamic linearization and normalization strategies for incomplete time series. On the other hand, a lightweight deep recurrent architecture is tailored to the characteristics of each input and extended with a multi-objective output. Additionally, a dedicated multi-label dataset together with an automated clustering and labelling methodology enables an event-based training strategy with shared objectives that ensures stable inference and efficient memory usage on the device. The proposed methodology was evaluated across different architectural configurations, identifying optimal trade-offs between computational cost and classification accuracy. The on-device implementation shows that 32-bit floating-point inference using the microcontroller Floating-Point Unit avoids additional compression while preserving up to 99% of the best model performance. The optimized kernel achieves a macro F1-score (average of per-class harmonic means of precision and recall) of 0.840, while real-time validation under real application conditions confirms robust operation with a macro F1-score of 0.898.

Posted in: Systems and Signals , Tagged: , , ,

Explaining the results of RL in continuous spaces

September 3, 2026 09:35 ,

R. Zuo, S. Khan, Z. Wang, G. E. Katz and Q. Qiu, Beyond VisionMask: Explaining Reinforcement Learning Via Contrastive-State Masking, IEEE Intelligent Systems, vol. 41, no. 4, pp. 56-63, July-Aug. 2026,10.1109/MIS.2026.3683960.

Existing explainable reinforcement learning (RL) approaches often fail to capture the contrastive nature of human reasoning—answering “why this action instead of that one?”. To address this challenge, we present Continuous VisionMask (cVM), a contrastive learning framework that explains RL agents in continuous state and action spaces. cVM extends our prior work, VisionMask, which was restricted to vision-based discrete settings, by introducing a quantized representation of continuous state and a set of contrastive learning objectives. This generalization enables cVM to provide faithful, robust, and sparse attributions highlighting which aspects of the sensed input are most important for the agent’s decision making, and to do so across a broader range of real-world RL scenarios. We evaluate cVM in multiple continuous-control environments and compare it against existing explainability baselines. Our results demonstrate that cVM produces more faithful and stable explanations, thereby enhancing transparency and interpretability in continuous RL systems.

Posted in: Reinforcement learning in AI , Tagged:

Unlearning repeatitive cycles in RL

September 3, 2026 09:04 ,

Dinesh Kumar Banjare, Mithilesh Atulkar, Mitul Kumar Ahirwal, Utilizing machine unlearning for faulty behavior patterns in human decision-making model: A sharded reinforcement learning approach, Engineering Applications of Artificial Intelligence, Volume 181, Part 6, 2026,10.1016/j.engappai.2026.115652.

In this paper, enhance the computational models of human decision-making by proposing a novel sharded reinforcement learning framework based on the principle of machine unlearning. The work proposes a data sharding and selective machine unlearning approach to unlearn faulty behavior patterns. It overcomes the drawbacks of conventional reinforcement learning methods, in which the players tend to play the same deck over and over again, showing a lack of adaptability. These problems are frequently seen in tasks such as the Iowa gambling task. To handle this issue, the concept of machine unlearning is used. The proposed method detects repeated patterns of faulty behavior, such as repeated deck selections. Such patterns are corrected locally at the level of shards, while retaining the last stable expectancy state from the previous learning stage. The mean squared deviation was used to evaluate the model’s performance. The results indicate that the unlearned model learns faster and has better performance in the early phases of learning. The framework supports behavioral adaptability by selectively excluding identified faulty shards while preserving previous valid learning. These findings show the potential of shard-based learning and selective unlearning in sequential decision-making.

Posted in: Reinforcement learning in AI , Tagged:

A survey on DRL applied to control engineering

September 3, 2026 08:51 ,

Klinsmann Agyei, Pouria Sarhadi, Daniel Polani, Deep reinforcement learning in applied control: Challenges, analysis, and insights, Engineering Applications of Artificial Intelligence, Volume 181, Part 4, 2026, 10.1016/j.engappai.2026.115524.

Deep reinforcement learning (DRL) algorithms have shown increasing promise for autonomous decision-making and continuous control, yet their practical value relative to established control methods remains difficult to assess under realistic control-engineering conditions. Existing evaluations are often narrow, application-specific, or based on inconsistent assumptions. As a result, it remains unclear when learned controllers are suitable alternatives to conventional designs. This paper presents a systematic assessment framework that compares four prominent DRL controllers with a classical control baseline across a diverse set of applied control problems, including non-minimum phase dynamics, flexible mechanical systems, nonlinear marine control, and aerial robotics. The framework standardises modelling assumptions, reward design, controller tuning, computational budgets, performance metrics, robustness tests, and deployment conditions. This allows fair and reproducible analysis across methods and applications. The evaluation covers tracking accuracy, settling time, overshoot, control effort, actuator saturation, disturbance rejection, model uncertainty, robustness margins, and sim-to-real transfer. The results show that performance depends strongly on the algorithmic structure, system dynamics, and operating conditions. Learned controllers provide advantages in some cases, particularly for nonlinearities, constraints, and uncertain operating regimes. Classical control remains competitive in accuracy, simplicity, and reliability for several benchmark conditions. Under combined saturation and unmodelled dynamics, the learned controllers maintained stable operation where the classical baseline became unstable. Real-time deployment on a quadrotor platform further demonstrated stable sim-to-real transfer. The study establishes a rigorous and reproducible evidence base for assessing the practical suitability of these data-driven control algorithms in applied control. It clarifies the trade-offs between learning-based and conventional control. This addresses a critical gap that cannot be easily inferred from isolated, system-specific studies.

Posted in: Applications of reinforcement learning to control engineering , Tagged:

Hierarchical adversarial RL

September 3, 2026 08:19 ,

Xiaofeng Wang, Shaopu Wang, Jipeng Zhang, Yue Ma, Hierarchical multi-policy adversarial reinforcement learning, Engineering Applications of Artificial Intelligence, Volume 181, Part 1, 2026, 10.1016/j.engappai.2026.115297.

In complex and dynamic autonomous driving environments, deep reinforcement learning policies often struggle with generalization due to distribution shifts between training and deployment. While conventional approaches like domain randomization and adversarial training offer partial solutions, they face notable limitations. Specifically, domain randomization relies on manually defined distributions that poorly capture real-world perturbations, while adversarial training suffers from high computational costs, unstable optimization, and behavioral homogenization. To address these issues, we propose a hierarchical multi-policy adversarial reinforcement learning framework that enhances policy generalization under complex perturbations. It incorporates three key components: a multi-policy generation architecture with a shared backbone and independent heads for diverse adversary modeling; a dual-perspective policy expansion regularization mechanism; and an adaptive curriculum-based adversarial training framework. The engineering value translates to a reliable, high-performance solution for autonomous driving, as validated on high-fidelity driving benchmarks. Experimentally, our approach demonstrates superior convergence, stability, and out-of-distribution generalization, achieving performance improvements ranging from 7.36% to 50.06% over established baseline methods while maintaining computational efficiency. These results, obtained in a high-fidelity simulator, demonstrate promise for real-world autonomous driving applications and lay the groundwork for future real-world deployment.

Posted in: Reinforcement learning in AI , Tagged: ,

Analyzing Li-Ion battery health

September 3, 2026 08:14 ,

Haixu Yang, Fengwei Liang, Kerui Li, Jichao Hong, Jinghan Zhang, Xiaohui Chen, State of health estimation and remaining useful life prediction of Li-ion battery oriented to real-world electric vehicles: A comprehensive evaluation methodology, Engineering Applications of Artificial Intelligence, Volume 181, Part 1, 2026, 10.1016/j.engappai.2026.115311.

State of health estimation is essential to ensure the battery’s reliability, safety and economy. It is an integral part of the battery management system. Estimation often faces the problems of a lone indicator and ineffective real-vehicle application. The use of a lone indicator leads to biased and inaccurate state estimation. The complexity of real-world driving scenarios renders the method impractical and inaccurate. Here, we propose a comprehensive evaluation method oriented to real-world electric vehicles. Capacity, ohmic internal resistance, cell voltage inconsistency, and battery pack temperature inconsistency are considered as four indicators of battery degradation. For each indicator, a state of health estimation value can be obtained. A fuzzy matrix is used to fuse the four indicators for a comprehensive evaluation. Also, the fuzzy matrix gives the weights of different health indicators as 0.1988, 0.2463, 0.3183, and 0.2366. The inconsistency of cell voltages carries the highest weighting among these indicators, exerting a more significant influence than the other three. In terms of artificial intelligence, this paper proposes a complete set of data processing, feature computation, and fusion estimation methods for battery health state. In engineering applications, the metrics for health state assessment are greatly enriched and the degree of accuracy is significantly improved. The method breaks through the limitations of the traditional evaluation based on a singular indicator, achieving a multi-dimensional comprehensive and objective evaluation of automotive batteries.

Posted in: Electronics , Tagged: ,

Converting logical formulae into geometrical shapes in order to solve problems in symbolic logic

June 11, 2026 11:14 ,

Thomas C. Henderson, Amelia Lessen, Ishaan Rajan, Tessa Nishida, Kutay Eken, Xiuyi Fan, David Sacharny, Amar Mitiche, Thatcher Geary, GEO-SAT: A new approach for knowledge-based agent decision making, Robotics and Autonomous Systems, Volume 203, 2026, 10.1016/j.robot.2026.105522.

Logical agents base their action selection decisions on inferences made over a logical knowledge base. Given a propositional logic knowledge base expressed in Conjunctive Normal Form (CNF; we will refer to a logical well-formed formula in CNF as a sentence), the knowledge can be converted into a geometrical format, and subsequent analysis takes place as geometrical operations on the feasible region in that representation. Two geometric representations are presented: the n-dimensional hypercube in Euclidean geometry and the n-dimensional Poincaré disk in non-Euclidean geometry. Based on these representations, two novel methods are proposed to: (1) find SAT solutions for the knowledge base (i.e., a truth assignment to each logical variable which makes the sentence true), and (2) find a reasonable approximation to the atom probabilities given the current set of information. This allows agents to determine the semantics (truth) of the world as well as to estimate the probability of truth. The geometric method provides an efficient heuristic approach to solving SAT for CNF knowledge bases, and provides polynomial-time solutions of probabilistic SAT (PSAT) for independent variables, and good PSAT estimates for non-independent logical variables. The polynomial time result for PSAT is due to the fact that the feasible region for the logical sentence is found using convex linear programming, and any point in the feasible region provides a set of valid atom probabilities. From these, if the variables are independent, any complete conjunction is defined as the product of literal probabilities. There is still no reduction of SAT to a geometric problem with a polynomial time solution; this approach to solving SAT and PSAT is called GEO-SAT.

Posted in: Artificial Intelligence , Tagged: , ,

RL for critical system through a top-level control of reward shaping

June 4, 2026 07:39 ,

Yang, Z., Feng, X. & Yu, H. , Equipping With Cognition: A Metacognition-Inspired Reinforcement Learning Approach for Multiobjective Safety-Critical Systems, Cogn Comput 18, 60 (2026), 10.1007/s12559-026-10602-w.

With the increasing complexity of safety-critical systems, reinforcement learning (RL) agents are required to optimize multiple objectives while facing progressively higher safety requirements in dynamic and nonstationary environments. Existing multiobjective reinforcement learning (MORL) methods typically assume stationary dynamics and rely on fixed reward structures, limiting their ability to maintain safety-performance trade-offs under environmental shifts. To address this limitation, we propose MMOSRL, a hierarchical metacognition-inspired reinforcement learning framework that introduces a dedicated metacognitive control layer above the RL agent. The metacognitive layer is structured into three components—knowledge, monitoring, and reflection—which collectively enable safety feature decomposition, real-time safety evaluation, and adaptive reward regulation. In particular, safety degradation is detected through a normalized survival score, and a dynamic reward reshaping mechanism is triggered via Safe Bayesian Optimization to adjust reward hyperparameters within a constrained search space. This design allows the agent to continuously realign its learning objective with evolving safety requirements without retraining. Simulation experiments conducted in autonomous driving scenarios demonstrate that in the most challenging high-density traffic conditions, MMOSRL achieves a 3.9% absolute improvement in success rate over the state-of-the-art baseline (PSL-MORL), while reducing collision and lane departure rates by 2.8% and 1.1%, respectively. The results validate that incorporating structured metacognitive control effectively enhances robustness and safety compliance in multiobjective, nonstationary environments.

Posted in: Applications of reinforcement learning to robots , Tagged: ,

A review of shared control in robot navigation

May 22, 2026 13:06 ,

Haojie Zhang, Chuankai Liu, Yikang Liu, Qing Li, hared control for autonomous robotic navigation: A review, Robotics and Autonomous Systems, Volume 202, 2026, 10.1016/j.robot.2026.105467.

Robots have been widely deployed in a variety of fields. However, full autonomous or full manual navigation remains difficult to implement in real-world environments due to environmental uncertainty and operational complexity. Shared control has therefore attracted increasing attention as an effective approach to enhance the performance of autonomous robotic navigation. This paper presents a comprehensive review of shared control methods applied to autonomous robotic navigation. First, the principle of shared control is introduced. Then, four representative shared control system frameworks are given in a chronological manner. Subsequently, human–robot interfaces are described, covering both simple parameter-based interfaces and mixed-initiative operational interfaces. The implementation of shared control methods for navigation is discussed in detail. Finally, the main findings and future research directions are summarized. The review indicates that shared control has been developed rapidly and plays a critical role in improving navigation performance in complex environments. It is expected to see broader adoption of shared control methods in real cases.

Posted in: Robot motion planning , Tagged: ,

Selecting the best RL result from offline RL in order to minimize risks

April 10, 2026 16:58 ,

Giorgio Angelotti, Nicolas Drougard, Caroline P․ C. Chanel, An offline risk-aware policy selection method for Bayesian Markov decision processes, Artificial Intelligence, Volume 354, 2026, 10.1016/j.artint.2026.104519.

In Offline Model Learning for Planning and in Offline Reinforcement Learning, the limited data set hinders the estimate of the Value function of the relative Markov Decision Process (MDP). Consequently, the performance of the obtained policy in the real world is bounded and possibly risky, especially when the deployment of a wrong policy can lead to catastrophic consequences. For this reason, several pathways are being followed with the scope of reducing the model error (or the distributional shift between the learned model and the true one) and, more broadly, obtaining risk-aware solutions with respect to model uncertainty. But when it comes to the final application which baseline should a practitioner choose? In an offline context where computational time is not an issue and robustness is the priority we propose Exploitation vs Caution (EvC), a paradigm that (1) elegantly incorporates model uncertainty abiding by the Bayesian formalism, and (2) selects the policy that maximizes a risk-aware objective over the Bayesian posterior between a fixed set of candidate policies provided, for instance, by the current baselines. We validate EvC with state-of-the-art approaches in different discrete, yet simple, environments offering a fair variety of MDP classes. In the tested scenarios EvC manages to select robust policies and hence stands out as a useful tool for practitioners that aim to apply offline planning and reinforcement learning solvers in the real world.

Posted in: Reinforcement learning in AI , Tagged: