Category Archives: Cognitive Sciences

Detecting novelties in the model of the world within MCTS

May 29, 2025 11:48 ,

Bryan Loyall, Avi Pfeffer, James Niehaus, Michael Harradon, Paola Rizzo, Alex Gee, Joe Campolongo, Tyler Mayer, John Steigerwald, Coltrane: A domain-independent system for characterizing and planning in novel situations, Artificial Intelligence, Volume 345, 2025, 10.1016/j.artint.2025.104336.

AI systems operating in open-world environments must be able to adapt to impactful changes in the world, immediately when they occur, and be able to do this across the many types of changes that can occur. We are seeking to create methods to extend traditional AI systems so that they can (1) immediately recognize changes in how the world works that are impactful to task accomplishment; (2) rapidly characterize the nature of the change using the limited observations that are available when the change is first detected; (3) adapt to the change as well as feasible to accomplish the system’s tasks given the available observations; and (4) continue to improve the characterization and adaptation as additional observations are available. In this paper, we describe Coltrane, a domain-independent system for characterizing and planning in novel situations that uses only natural domain descriptions to generate its novelty-handling behavior, without any domain-specific anticipation of the novelty. Coltrane’s characterization method is based on probabilistic program synthesis of perturbations to programs expressed in a traditional programming language describing domain transition models. Its planning method is based on incorporating novel domain models in an MCTS search algorithm and on automatically adapting the heuristics used. Both a formal external evaluation and our own demonstrations show that Coltrane is capable of accurately characterizing interesting forms of novelty and of adapting its behavior to restore its performance to pre-novelty levels and even beyond.

Posted in: Artificial Intelligence , Tagged: , ,

A possible explanation for the formation of concepts in the human brain

May 8, 2025 11:09 ,

Luca D. Kolibius, Sheena A. Josselyn, Simon Hanslmayr, On the origin of memory neurons in the human hippocampus, Trends in Cognitive Sciences, Volume 29, Issue 5, 2025, Pages 421-433 10.1016/j.tics.2025.01.013.

The hippocampus is essential for episodic memory, yet its coding mechanism remains debated. In humans, two main theories have been proposed: one suggests that concept neurons represent specific elements of an episode, while another posits a conjunctive code, where index neurons code the entire episode. Here, we integrate new findings of index neurons in humans and other animals with the concept-specific memory framework, proposing that concept neurons evolve from index neurons through overlapping memories. This process is supported by engram literature, which posits that neurons are allocated to a memory trace based on excitability and that reactivation induces excitability. By integrating these insights, we connect two historically disparate fields of neuroscience: engram research and human single neuron episodic memory research.

Posted in: Psycho-physiological bases of engineering , Tagged: , , ,

On the problem of choice overload for human cognition

May 8, 2025 11:06 ,

Jessie C. Tanner, Claire T. Hemingway, Choice overload and its consequences for animal decision-making, Trends in Cognitive Sciences, Volume 29, Issue 5, 2025, Pages 403-406 10.1016/j.tics.2025.01.003.

Animals routinely make decisions with important consequences for their survival and reproduction, but they frequently make suboptimal decisions. Here, we explore choice overload as one reason why animals may make suboptimal decisions, arguing that choice overload may have important ecological and evolutionary consequences, and propose future directions.

Posted in: Psycho-physiological bases of engineering , Tagged: ,

Improving reward shaping in Deep RL for avoiding user’s biases and boosting learning efficiency

April 24, 2025 08:32 ,

Jiawei Lin, Xuekai Wei, Weizhi Xian, Jielu Yan, Leong Hou U, Yong Feng, Zhaowei Shang, Mingliang Zhou, Continuous reinforcement learning via advantage value difference reward shaping: A proximal policy optimization perspective, Engineering Applications of Artificial Intelligence, Volume 151, 2025 10.1016/j.engappai.2025.110676.

Deep reinforcement learning has shown great promise in industrial applications. However, these algorithms suffer from low learning efficiency because of sparse reward signals in continuous control tasks. Reward shaping addresses this issue by transforming sparse rewards into more informative signals, but some designs that rely on domain experts or heuristic rules can introduce cognitive biases, leading to suboptimal solutions. To overcome this challenge, this paper proposes the advantage value difference (AVD), a generalized potential-based end-to-end exploration reward function. The main contribution of this paper is to improve the agent’s exploration efficiency, accelerate the learning process, and prevent premature convergence to local optima. The method leverages the temporal difference error to estimate the potential of states and uses the advantage function to guide the learning process toward more effective strategies. In the context of engineering applications, this paper proves the superiority of AVD in continuous control tasks within the multi-joint dynamics with contact (MuJoCo) environment. Specifically, the proposed method achieves an average increase of 23.5% in episode rewards for the Hopper, Swimmer, and Humanoid tasks compared with the state-of-the-art approaches. The results demonstrate the significant improvement in learning efficiency achieved by AVD for industrial robotic systems.

Posted in: Reinforcement learning in AI , Tagged: , , ,

When to rely on memories versus sampling sensory information anew to guide behavior

April 3, 2025 06:41 ,

Levi Kumle, Anna C. Nobre, Dejan Draschkow, Sensorimnemonic decisions: choosing memories versus sensory information, Trends in Cognitive Sciences, Volume 29, Issue 4, 2025, Pages 311-313, 10.1016/j.tics.2024.12.010.

We highlight a fundamental psychological function that is central to many of our interactions in the environment – when to rely on memories versus sampling sensory information anew to guide behavior. By operationalizing sensorimnemonic decisions we aim to encourage and advance research into this pivotal process for understanding how memories serve adaptive cognition.

Posted in: Psycho-physiological bases of engineering , Tagged:

On the innate ability of vertebrates for number recognition and the one of distinguishing ratios of numbers

March 6, 2025 10:33 ,

Elena Lorenzi, Dmitry Kobylkov, Giorgio Vallortigara, Is there an innate sense of number in the brain?, Cerebral Cortex, Volume 35, Issue 2, February 2025, DOI: 10.1093/cercor/bhaf004.

The approximate number system or «sense of number» is a crucial, presymbolic mechanism enabling animals to estimate quantities, which is essential for survival in various contexts (eg estimating numerosities of social companions, prey, predators, and so on). Behavioral studies indicate that a sense of number is widespread across vertebrates and invertebrates. Specific brain regions such as the intraparietal sulcus and prefrontal cortex in primates, or equivalent areas in birds and fish, are involved in numerical estimation, and their activity is modulated by the ratio of quantities. Data gathered across species strongly suggest similar evolutionary pressures for number estimation pointing to a likely common origin, at least across vertebrates. On the other hand, few studies have investigated the origins of the sense of number. Recent findings, however, have shown that numerosity-selective neurons exist in newborn animals, such as domestic chicks and zebrafish, supporting the hypothesis of an innateness of the approximate number system. Control-rearing experiments on visually naïve animals further support the notion that the sense of number is innate and does not need any specific instructive experience in order to be triggered.

Posted in: Psycho-physiological bases of engineering , Tagged:

It seems that the human brain working memory uses pointers

March 6, 2025 10:21 ,

Edward Awh, Edward K. Vogel, Working memory needs pointers, Trends in Cognitive Sciences, Volume 29, Issue 3, 2025, Pages 230-241, DOI: 10.1016/j.tics.2024.12.006.

Cognitive neuroscience has converged on a definition of working memory (WM) as a capacity-limited system that maintains highly accessible representations via stimulus-specific neural patterns. We argue that this standard definition may be incomplete. We highlight the fundamental need to recognize specific instances or tokens and to bind those tokens to the surrounding context. We propose that contextual binding is supported by spatiotemporal ‘pointers’ and that pointers are the source of neural signals that track the number of stored items, independent of their content. These content-independent pointers may provide a productive perspective for understanding item-based capacity limits in WM and the role of WM as a gateway for long-term storage.

Posted in: Psycho-physiological bases of engineering , Tagged:

On the reasons of the pervasiveness of the myth of meritocracy

February 6, 2025 13:15 ,

Ian R. Hadden, Céline Darnon, Lewis Doyle, Matthew J. Easterbrook, Sébastien Goudeau, Andrei Cimpian, Why the belief in meritocracy is so pervasive, Trends in Cognitive Sciences, Volume 29, Issue 2, 2025, Pages 101-104, DOI: 10.1016/j.tics.2024.12.008.

People worldwide tend to believe that their societies are more meritocratic than they actually are. We propose the belief in meritocracy is widespread because it is rooted in simple, seemingly obvious causal–explanatory intuitions. Our proposal suggests solutions for debunking the myth of meritocracy and increasing support for equity-oriented policies.

Posted in: Cognitive sciences , Tagged:

On the two-ways of learning language in humans: both abstracting detailed knowledge and refining still-only-abstract one

January 9, 2025 16:38 ,

Susan Goldin-Meadow, Inbal Arnon, Whole-to-part development in language creation, Trends in Cognitive Sciences, Volume 29, Issue 1, 2025, Pages 12-14, DOI: 10.1016/j.tics.2024.09.015.

Children approach language by learning parts and constructing wholes. But they can also first learn wholes and then discover parts. We demonstrate this understudied yet impactful process in children creating language without input. Whole-to-part learning thus need not be driven by hard-to-segment input and is a bias that children bring to language.

Posted in: Psycho-physiological bases of engineering , Tagged:

On the limited throughput of the human cognition and its implications, e.g., in Engineering

December 19, 2024 11:49 ,

Jieyu Zheng1, and Markus Meister, The unbearable slowness of being: Why do we live at 10 bits/s?, Neuron (2024), DOI: 10.1016/j.neuron.2024.11.008.

This article is about the neural conundrum behind the slowness of human behavior. The information throughput of a human being is about 10 bits/s. In comparison, our sensory systems gather data at 10 bits/s. The stark contrast between these numbers remains unexplained and touches on fundamental aspects of brain function: what neural substrate sets this speed limit on the pace of our existence? Why does the brain need billions of neurons to process 10 bits/s? Why can we only think about one thing at a time? The brain seems to operate in two distinct modes: the ‘‘outer’’ brain handles fast high-dimensional sensory and motor signals, whereas the ‘‘inner’’ brain processes the reduced few bits needed to control behavior. Plausible explanations exist for the large neuron numbers in the outer brain, but not for the inner brain, and we propose new research directions to remedy this.

Posted in: Psycho-physiological bases of engineering , Tagged: , ,