Category Archives: Cognitive Sciences

Leveraging the unexplainability and opacity of NNs to generate random numbers

December 15, 2023 16:17 ,

Y. Almardeny, A. Benavoli, N. Boujnah and E. Naredo, A Reinforcement Learning System for Generating Instantaneous Quality Random Sequences, IEEE Transactions on Artificial Intelligence, vol. 4, no. 3, pp. 402-415, June 2023 DOI: 10.1109/TAI.2022.3161893.

Random numbers are essential to most computer applications. Still, producing high-quality random sequences is a big challenge. Inspired by the success of artificial neural networks and reinforcement learning, we propose a novel and effective end-to-end learning system to generate pseudorandom sequences that operates under the upside-down reinforcement learning framework. It is based on manipulating the generalized information entropy metric to derive commands that instantaneously guide the agent toward the optimal random behavior. Using a wide range of evaluation tests, the proposed approach is compared against three state-of-the-art accredited pseudorandom number generators (PRNGs). The experimental results agree with our theoretical study and show that the proposed framework is a promising candidate for a wide range of applications.

Posted in: Artificial Intelligence , Tagged: ,

Limiting human intervention in the design of RL solutions (now called “Automated RL”)

December 5, 2023 10:52 ,

Marco Mussi, Davide Lombarda, Alberto Maria Metelli, Francesco Trov�, Marcello Restelli, ARLO: A framework for Automated Reinforcement Learning, Expert Systems with Applications, Volume 224, 2023 DOI: 10.1016/j.eswa.2023.119883.

Automated Reinforcement Learning (AutoRL) is a relatively new area of research that is gaining increasing attention. The objective of AutoRL consists in easing the employment of Reinforcement Learning (RL) techniques for the broader public by alleviating some of its main challenges, including data collection, algorithm selection, and hyper-parameter tuning. In this work, we propose a general and flexible framework, namely ARLO: Automated Reinforcement Learning Optimizer, to construct automated pipelines for AutoRL. Based on this, we propose a pipeline for offline and one for online RL, discussing the components, interaction, and highlighting the difference between the two settings. Furthermore, we provide a Python implementation of such pipelines, released as an open-source library. Our implementation is tested on an illustrative LQG domain and on classic MuJoCo environments, showing the ability to reach competitive performances requiring limited human intervention. We also showcase the full pipeline on a realistic dam environment, automatically performing the feature selection and the model generation tasks.

Posted in: Reinforcement learning in AI , Tagged:

Multi-task RL through common perceptions

December 5, 2023 10:50 ,

Jinling Meng, Fei Zhu, Seek for commonalities: Shared features extraction for multi-task reinforcement learning via adversarial training, Expert Systems with Applications, Volume 224, 2023 DOI: 10.1016/j.eswa.2023.119975.

Multi-task reinforcement learning is promising to alleviate the low sample efficiency and high computation cost of reinforcement learning algorithms. However, current methods mostly focus on unique features that are not conducive to the transfer between tasks. Moreover, they usually lack a balance mechanism among tasks, which often leads to the unnecessary occupation of training resources by tasks that have already been trained. To address the problems, a simple yet effective method referred to as Adaptive Experience buffer with Shared Features Multi-Task Reinforcement Learning (AESF-MTRL) is proposed. In AESF-MTRL, input observation of the environment is divided into shared features and unique features, which are extracted using different feature extractors. Unique features are extracted by simple gradient descent, while shared features are extracted using adversarial training, with an additional discriminator trained to ensure that the extracted features are indeed shared features. AESF-MTRL also maintains a reward stack to adjust the sampling ratio of trajectories from different tasks dynamically during the update period to balance the learning process of different tasks. Experiments on multiple robotics control environments demonstrate the effectiveness of the proposed method.

Posted in: Reinforcement learning in AI , Tagged: ,

Further support for a multi-tool approach for consciusness

December 5, 2023 10:12 ,

Biyu J. He, Towards a pluralistic neurobiological understanding of consciousness, Trends in Cognitive Sciences, Volume 27, Issue 5, 2023 DOI: 10.1016/j.tics.2023.02.001.

Theories of consciousness are often based on the assumption that a single, unified neurobiological account will explain different types of conscious awareness. However, recent findings show that, even within a single modality such as conscious visual perception, the anatomical location, timing, and information flow of neural activity related to conscious awareness vary depending on both external and internal factors. This suggests that the search for generic neural correlates of consciousness may not be fruitful. I argue that consciousness science requires a more pluralistic approach and propose a new framework: joint determinant theory (JDT). This theory may be capable of accommodating different brain circuit mechanisms for conscious contents as varied as percepts, wills, memories, emotions, and thoughts, as well as their integrated experience.

Posted in: Psycho-physiological bases of engineering , Tagged:

Emergence of number meaning from sensorimotor experiences

November 24, 2023 17:48 ,

Elena Sixtus, Florian Krause, Oliver Lindemann, Martin H. Fischer, A sensorimotor perspective on numerical cognition, Trends in Cognitive Sciences, Volume 27, Issue 4, 2023, Pages 367-378 DOI: 10.1016/j.tics.2023.01.002.

Numbers are present in every part of modern society and the human capacity to use numbers is unparalleled in other species. Understanding the mental and neural representations supporting this capacity is of central interest to cognitive psychology, neuroscience, and education. Embodied numerical cognition theory suggests that beyond the seemingly abstract symbols used to refer to numbers, their underlying meaning is deeply grounded in sensorimotor experiences, and that our specific understanding of numerical information is shaped by actions related to our fingers, egocentric space, and experiences with magnitudes in everyday life. We propose a sensorimotor perspective on numerical cognition in which number comprehension and numerical proficiency emerge from grounding three distinct numerical core concepts: magnitude, ordinality, and cardinality.

Posted in: Psycho-physiological bases of engineering , Tagged:

Review of emotions in AI

November 17, 2023 09:51 ,

G. Assun��o, B. Patr�o, M. Castelo-Branco and P. Menezes, An Overview of Emotion in Artificial Intelligence, IEEE Transactions on Artificial Intelligence, vol. 3, no. 6, pp. 867-886, Dec. 2022 DOI: 10.1109/TAI.2022.3159614.

The field of artificial intelligence (AI) has gained immense traction over the past decade, producing increasingly successful applications as research strives to understand and exploit neural processing specifics. Nonetheless emotion, despite its demonstrated significance to reinforcement, social integration, and general development, remains a largely stigmatized and consequently disregarded topic by most engineers and computer scientists. In this article, we endorse emotion\u2019s value for the advancement of artificial cognitive processing, as well as explore real-world use cases of emotion-augmented AI. A schematization is provided on the psychological-neurophysiologic basics of emotion in order to bridge the interdisciplinary gap preventing emulation and integration in AI methodology, as well as exploitation by current systems. In addition, we overview three major subdomains of AI greatly benefiting from emotion, and produce a systematic survey of meaningful yet recent contributions to each area. To conclude, we address crucial challenges and promising research paths for the future of emotion in AI with the hope that more researchers will develop an interest for the topic and find it easier to develop their own contributions.

Posted in: Psycho-physiological bases of engineering , Tagged:

Normal blindness to visible objects seems to be the result of limited-capacity prediction mechanisms in the brain

November 3, 2023 09:05 ,

Jeremy M. Wolfe, Anna Kosovicheva, Benjamin Wolfe, Normal blindness: when we Look But Fail To See, Trends in Cognitive Sciences, Volume 26, Issue 9, 2022, Pages 809-819 DOI: 10.1016/j.tics.2022.06.006.

Humans routinely miss important information that is \u2018right in front of our eyes\u2019, from overlooking typos in a paper to failing to see a cyclist in an intersection. Recent studies on these \u2018Looked But Failed To See\u2019 (LBFTS) errors point to a common mechanism underlying these failures, whether the missed item was an unexpected gorilla, the clearly defined target of a visual search, or that simple typo. We argue that normal blindness is the by-product of the limited-capacity prediction engine that is our visual system. The processes that evolved to allow us to move through the world with ease are virtually guaranteed to cause us to miss some significant stimuli, especially in important tasks like driving and medical image perception.

Posted in: Psycho-physiological bases of engineering , Tagged:

On the existence of multiple fundamental “languages” in the brain that use discrete symbols and a few basic structures

November 3, 2023 09:03 ,

Stanislas Dehaene, Fosca Al Roumi, Yair Lakretz, Samuel Planton, Mathias Sabl�-Meyer, Symbols and mental programs: a hypothesis about human singularity, Trends in Cognitive Sciences, Volume 26, Issue 9, 2022, Pages 751-766 DOI: 10.1016/j.tics.2022.06.010.

Natural language is often seen as the single factor that explains the cognitive singularity of the human species. Instead, we propose that humans possess multiple internal languages of thought, akin to computer languages, which encode and compress structures in various domains (mathematics, music, shape\u2026). These languages rely on cortical circuits distinct from classical language areas. Each is characterized by: (i) the discretization of a domain using a small set of symbols, and (ii) their recursive composition into mental programs that encode nested repetitions with variations. In various tasks of elementary shape or sequence perception, minimum description length in the proposed languages captures human behavior and brain activity, whereas non-human primate data are captured by simpler nonsymbolic models. Our research argues in favor of discrete symbolic models of human thought.

Posted in: Psycho-physiological bases of engineering , Tagged: ,

Unexpected consequences of training smarthome systems with reinforcement learning: effects on human behaviours

October 27, 2023 06:02 ,

S. Suman, A. Etemad and F. Rivest, TPotential Impacts of Smart Homes on Human Behavior: A Reinforcement Learning Approach, IEEE Transactions on Artificial Intelligence, vol. 3, no. 4, pp. 567-580, Aug. 2022 DOI: 10.1109/TAI.2021.3127483.

Smart homes are becoming increasingly popular as a result of advances in machine learning and cloud computing. Devices, such as smart thermostats and speakers, are now capable of learning from user feedback and adaptively adjust their settings to human preferences. Nonetheless, these devices might in turn impact human behavior. To investigate the potential impacts of smart homes on human behavior, we simulate a series of hierarchical-reinforcement learning-based human models capable of performing various activities\u2014namely, setting temperature and humidity for thermal comfort inside a Q-Learning-based smart home model. We then investigate the possibility of the human models\u2019 behaviors being altered as a result of the smart home and the human model adapting to one another. For our human model, the activities are based on hierarchical-reinforcement learning. This allows the human to learn how long it must continue a given activity and decide when to leave it. We then integrate our human model in the environment along with the smart home model and perform rigorous experiments considering various scenarios involving a model of a single human and models of two different humans with the smart home. Our experiments show that with the smart home, the human model can exhibit unexpected behaviors such as frequent changing of activities and an increase in the time required to modify the thermal preferences. With two human models, we interestingly observe that certain combinations of models result in normal behaviors, while other combinations exhibit the same unexpected behaviors as those observed from the single human experiment.

Posted in: Psycho-physiological bases of engineering , Tagged: , ,

Improving the quality of memory replay in RL through an evolutionary algorithm biologically inspired

October 20, 2023 09:42 ,

M. Ramicic and A. Bonarini, Augmented Memory Replay in Reinforcement Learning With Continuous Control, IEEE Transactions on Cognitive and Developmental Systems, vol. 14, no. 2, pp. 485-496, June 2022 DOI: 10.1109/TCDS.2021.3050723.

Online reinforcement learning agents are currently able to process an increasing amount of data by converting it into a higher order value functions. This expansion of the information collected from the environment increases the agent\u2019s state space enabling it to scale up to more complex problems but also increases the risk of forgetting by learning on redundant or conflicting data. To improve the approximation of a large amount of data, a random mini-batch of the past experiences that are stored in the replay memory buffer is often replayed at each learning step. The proposed work takes inspiration from a biological mechanism which acts as a protective layer of higher cognitive functions found in mammalian brain: active memory consolidation mitigates the effect of forgetting previous memories by dynamically processing the new ones. Similar dynamics are implemented by the proposed augmented memory replay or AMR algorithm. The architecture of AMR , based on a simple artificial neural network is able to provide an augmentation policy which modifies each of the agents experiences by augmenting their relevance prior to storing them in the replay memory. The function approximator of AMR is evolved using genetic algorithm in order to obtain the specific augmentation policy function that yields the best performance of a learning agent in a specific environment given by its received cumulative reward. Experimental results show that an evolved AMR augmentation function capable of increasing the significance of the specific memories is able to further increase the stability and convergence speed of the learning algorithms dealing with the complexity of continuous action domains.

Posted in: Psycho-physiological bases of engineering , Tagged: