Category Archives: Cognitive Sciences

The physics of the mouth influences symbols

November 5, 2026 09:57 ,

Luca S. Rinaldi, Vittorio Gallese, How the mouth became symbolic, Trends in Cognitive Sciences,
Volume 30, Issue 11, 2026, Pages 973-986, 10.1016/j.tics.2026.02.011.

Before we name, we touch. We propose that the roots of language lie not in abstract, amodal symbols but in early bodily experience. Early haptic and oral interactions ground conceptual knowledge through active exploration. The mouth, acting as a cognitive organ, functions not only as a site of articulation but also as a locus of tactile perception and intersubjective exchange. We suggest that language may have evolved through the neural reuse of circuits originally dedicated to ingestion and action understanding. These circuits were progressively shaped by affordances, embodied simulation, and sound symbolism. From sensorimotor patterns to iconic vocal forms, we trace a pathway linking bodily experience to symbolic reference, through which language acquisition could arise.

Posted in: Psycho-physiological bases of engineering , Tagged: ,

Explaining the results of RL in continuous spaces

September 3, 2026 09:35 ,

R. Zuo, S. Khan, Z. Wang, G. E. Katz and Q. Qiu, Beyond VisionMask: Explaining Reinforcement Learning Via Contrastive-State Masking, IEEE Intelligent Systems, vol. 41, no. 4, pp. 56-63, July-Aug. 2026,10.1109/MIS.2026.3683960.

Existing explainable reinforcement learning (RL) approaches often fail to capture the contrastive nature of human reasoning—answering “why this action instead of that one?”. To address this challenge, we present Continuous VisionMask (cVM), a contrastive learning framework that explains RL agents in continuous state and action spaces. cVM extends our prior work, VisionMask, which was restricted to vision-based discrete settings, by introducing a quantized representation of continuous state and a set of contrastive learning objectives. This generalization enables cVM to provide faithful, robust, and sparse attributions highlighting which aspects of the sensed input are most important for the agent’s decision making, and to do so across a broader range of real-world RL scenarios. We evaluate cVM in multiple continuous-control environments and compare it against existing explainability baselines. Our results demonstrate that cVM produces more faithful and stable explanations, thereby enhancing transparency and interpretability in continuous RL systems.

Posted in: Reinforcement learning in AI , Tagged:

Unlearning repeatitive cycles in RL

September 3, 2026 09:04 ,

Dinesh Kumar Banjare, Mithilesh Atulkar, Mitul Kumar Ahirwal, Utilizing machine unlearning for faulty behavior patterns in human decision-making model: A sharded reinforcement learning approach, Engineering Applications of Artificial Intelligence, Volume 181, Part 6, 2026,10.1016/j.engappai.2026.115652.

In this paper, enhance the computational models of human decision-making by proposing a novel sharded reinforcement learning framework based on the principle of machine unlearning. The work proposes a data sharding and selective machine unlearning approach to unlearn faulty behavior patterns. It overcomes the drawbacks of conventional reinforcement learning methods, in which the players tend to play the same deck over and over again, showing a lack of adaptability. These problems are frequently seen in tasks such as the Iowa gambling task. To handle this issue, the concept of machine unlearning is used. The proposed method detects repeated patterns of faulty behavior, such as repeated deck selections. Such patterns are corrected locally at the level of shards, while retaining the last stable expectancy state from the previous learning stage. The mean squared deviation was used to evaluate the model’s performance. The results indicate that the unlearned model learns faster and has better performance in the early phases of learning. The framework supports behavioral adaptability by selectively excluding identified faulty shards while preserving previous valid learning. These findings show the potential of shard-based learning and selective unlearning in sequential decision-making.

Posted in: Reinforcement learning in AI , Tagged:

Hierarchical adversarial RL

September 3, 2026 08:19 ,

Xiaofeng Wang, Shaopu Wang, Jipeng Zhang, Yue Ma, Hierarchical multi-policy adversarial reinforcement learning, Engineering Applications of Artificial Intelligence, Volume 181, Part 1, 2026, 10.1016/j.engappai.2026.115297.

In complex and dynamic autonomous driving environments, deep reinforcement learning policies often struggle with generalization due to distribution shifts between training and deployment. While conventional approaches like domain randomization and adversarial training offer partial solutions, they face notable limitations. Specifically, domain randomization relies on manually defined distributions that poorly capture real-world perturbations, while adversarial training suffers from high computational costs, unstable optimization, and behavioral homogenization. To address these issues, we propose a hierarchical multi-policy adversarial reinforcement learning framework that enhances policy generalization under complex perturbations. It incorporates three key components: a multi-policy generation architecture with a shared backbone and independent heads for diverse adversary modeling; a dual-perspective policy expansion regularization mechanism; and an adaptive curriculum-based adversarial training framework. The engineering value translates to a reliable, high-performance solution for autonomous driving, as validated on high-fidelity driving benchmarks. Experimentally, our approach demonstrates superior convergence, stability, and out-of-distribution generalization, achieving performance improvements ranging from 7.36% to 50.06% over established baseline methods while maintaining computational efficiency. These results, obtained in a high-fidelity simulator, demonstrate promise for real-world autonomous driving applications and lay the groundwork for future real-world deployment.

Posted in: Reinforcement learning in AI , Tagged: ,

Converting logical formulae into geometrical shapes in order to solve problems in symbolic logic

June 11, 2026 11:14 ,

Thomas C. Henderson, Amelia Lessen, Ishaan Rajan, Tessa Nishida, Kutay Eken, Xiuyi Fan, David Sacharny, Amar Mitiche, Thatcher Geary, GEO-SAT: A new approach for knowledge-based agent decision making, Robotics and Autonomous Systems, Volume 203, 2026, 10.1016/j.robot.2026.105522.

Logical agents base their action selection decisions on inferences made over a logical knowledge base. Given a propositional logic knowledge base expressed in Conjunctive Normal Form (CNF; we will refer to a logical well-formed formula in CNF as a sentence), the knowledge can be converted into a geometrical format, and subsequent analysis takes place as geometrical operations on the feasible region in that representation. Two geometric representations are presented: the n-dimensional hypercube in Euclidean geometry and the n-dimensional Poincaré disk in non-Euclidean geometry. Based on these representations, two novel methods are proposed to: (1) find SAT solutions for the knowledge base (i.e., a truth assignment to each logical variable which makes the sentence true), and (2) find a reasonable approximation to the atom probabilities given the current set of information. This allows agents to determine the semantics (truth) of the world as well as to estimate the probability of truth. The geometric method provides an efficient heuristic approach to solving SAT for CNF knowledge bases, and provides polynomial-time solutions of probabilistic SAT (PSAT) for independent variables, and good PSAT estimates for non-independent logical variables. The polynomial time result for PSAT is due to the fact that the feasible region for the logical sentence is found using convex linear programming, and any point in the feasible region provides a set of valid atom probabilities. From these, if the variables are independent, any complete conjunction is defined as the product of literal probabilities. There is still no reduction of SAT to a geometric problem with a polynomial time solution; this approach to solving SAT and PSAT is called GEO-SAT.

Posted in: Artificial Intelligence , Tagged: , ,

Selecting the best RL result from offline RL in order to minimize risks

April 10, 2026 16:58 ,

Giorgio Angelotti, Nicolas Drougard, Caroline P․ C. Chanel, An offline risk-aware policy selection method for Bayesian Markov decision processes, Artificial Intelligence, Volume 354, 2026, 10.1016/j.artint.2026.104519.

In Offline Model Learning for Planning and in Offline Reinforcement Learning, the limited data set hinders the estimate of the Value function of the relative Markov Decision Process (MDP). Consequently, the performance of the obtained policy in the real world is bounded and possibly risky, especially when the deployment of a wrong policy can lead to catastrophic consequences. For this reason, several pathways are being followed with the scope of reducing the model error (or the distributional shift between the learned model and the true one) and, more broadly, obtaining risk-aware solutions with respect to model uncertainty. But when it comes to the final application which baseline should a practitioner choose? In an offline context where computational time is not an issue and robustness is the priority we propose Exploitation vs Caution (EvC), a paradigm that (1) elegantly incorporates model uncertainty abiding by the Bayesian formalism, and (2) selects the policy that maximizes a risk-aware objective over the Bayesian posterior between a fixed set of candidate policies provided, for instance, by the current baselines. We validate EvC with state-of-the-art approaches in different discrete, yet simple, environments offering a fair variety of MDP classes. In the tested scenarios EvC manages to select robust policies and hence stands out as a useful tool for practitioners that aim to apply offline planning and reinforcement learning solvers in the real world.

Posted in: Reinforcement learning in AI , Tagged:

Using evolutionary computation to find better rewards in the case of partial-observable RL

February 19, 2026 10:26 ,

Zhengwei Zhu, Zhixuan Chen, Chenyang Zhu, Wen Si, Fang Wang, Optimizing potential-based reward automata in partially observable reinforcement learning using genetic local search, Engineering Applications of Artificial Intelligence, Volume 169, 2026, 10.1016/j.engappai.2026.114054.

Partially observable reinforcement learning extends the reinforcement learning framework to environments in which agents have limited visibility of the state space, making it particularly relevant for applications in robotics and autonomous vehicle navigation. However, a primary challenge in partially observable reinforcement learning is defining effective reward functions that can guide the learning process despite partial observability. To address this challenge, this paper introduces a novel approach for constructing potential-based reward automata by employing genetic local search methods. Specifically, our method constructs these automata from compressed representations of exploration trajectories, which succinctly capture critical decision points and essential state transitions while eliminating redundant steps. By optimizing trajectory samples and shortening agent trajectories to their crucial transitions, our technique significantly reduces computational overhead. Formally, we define the learning objective as an optimization problem aimed at maximizing the log-likelihood of future observations while simultaneously minimizing the structural complexity of the learned reward automata. Furthermore, by incorporating value-based strategies to estimate potential values within the reward automata, our approach improves learning efficiency and facilitates the identification of optimal reward structures. We empirically evaluate our proposed method on seven partially observable grid-world benchmarks. Experimental results demonstrate that our method achieves superior performance relative to state-of-the-art reward automata-based techniques, exhibiting both accelerated learning speeds and higher accumulated rewards. Additionally, our genetic local search algorithm consistently outperforms comparative heuristic methods in terms of learning curves and reward accumulation.

Posted in: Reinforcement learning in AI , Tagged: , ,

Uncovering time variations in decision making of agents that do not always respond with the same policy

January 8, 2026 12:21 ,

Anne E. Urai, Structure uncovered: understanding temporal variability in perceptual decision-making, Trends in Cognitive Sciences, Volume 30, Issue 1, 2026, Pages 54-65 10.1016/j.tics.2025.06.003.

Studies of perceptual decision-making typically present the same stimulus repeatedly over the course of an experimental session but ignore the order of these observations, assuming unrealistic stability of decision strategies over trials. However, even ‘stable,’ ‘steady-state,’ or ‘expert’ decision-making behavior features significant trial-to-trial variability that is richly structured in time. Structured trial-to-trial variability of various forms can be uncovered using latent variable models such as hidden Markov models and autoregressive models, revealing how unobservable internal states change over time. Capturing such temporal structure can avoid confounds in cognitive models, provide insights into inter- and intraindividual variability, and bridge the gap between neural and cognitive mechanisms of variability in perceptual decision-making.

See also: the no so strong influence of time in some cognitive processes, such as speech processing (https://doi.org/10.1016/j.tics.2025.05.017)
Posted in: Psycho-physiological bases of engineering , Tagged: , ,

Evidences in the natural world of the benefits of communication errors within collaborative agents

January 8, 2026 12:16 ,

Bradley D. Ohlinger, Takao Sasaki, How miscommunication can improve collective performance in social insects, Trends in Cognitive Sciences, Volume 30, Issue 1, 2026, Pages 10-12, 10.1016/j.tics.2025.10.005.

Communication errors are typically viewed as detrimental, yet they can benefit collective foraging in social insects. Temnothorax ants provide a powerful model for studying how such errors arise during tandem running and how they might improve group performance under certain environmental conditions.

Posted in: Psycho-physiological bases of engineering , Tagged: ,

RL with both discrete and continuous actions

November 6, 2025 15:27 ,

Chengcheng Yan, Shujie Chen, Jiawei Xu, Xuejie Wang, Zheng Peng, Hybrid Reinforcement Learning in parameterized action space via fluctuates constraint, Engineering Applications of Artificial Intelligence, Volume 162, Part C, 2025 10.1016/j.engappai.2025.112499.

Parameterized actions in Reinforcement Learning (RL) are composed of discrete-continuous hybrid action parameters, which are widely employed in game scenarios. However, previous works have often concentrated on the network structure of RL algorithms to solve hybrid actions, neglecting the impact of fluctuations in action parameters for agent move trajectory. Due to the coupling between discrete and continuous actions, instability in discrete actions influences the selection of corresponding continuous parameters, resulting in the agent deviating from the optimal move path. In this paper, we propose a parameterized RL approach based on parameter fluctuation restriction (PFR) to address this problem, called CP-DQN. Our method effectively mitigated value fluctuation in action parameters by constraining the action parameter between adjacent time steps. Additionally, we have incorporated a supervision module to optimize the entire training process. To quantify the superiority of our approach in minimizing trajectory deviations for agents, we propose an indicator to measure the influence of parameter fluctuations on performance in hybrid action space. Our method is evaluated in three environments with hybrid action spaces, and the experiments demonstrate the superiority of our method compared to existing approaches.

Posted in: Reinforcement learning in AI