Monthly Archives: June 2025

You are browsing the site archives by month.

Hierarchical optimization based on learning from mistakes

L. Zhang, B. Garg, P. Sridhara, R. Hosseini and P. Xie, Learning From Mistakes: A Multilevel Optimization Framework, IEEE Transactions on Artificial Intelligence, vol. 6, no. 6, pp. 1651-1663, June 2025, 10.1109/TAI.2025.3534151.

Bi-level optimization methods in machine learning are popularly effective in subdomains of neural architecture search, data reweighting, etc. However, most of these methods do not factor in variations in learning difficulty, which limits their performance in real-world applications. To address the above problems, we propose a framework that imitates the learning process of humans. In human learning, learners usually focus more on the topics where mistakes have been made in the past to deepen their understanding and master the knowledge. Inspired by this effective human learning technique, we propose a multilevel optimization framework, learning from mistakes (LFM), for machine learning. We formulate LFM as a three-stage optimization problem: 1) the learner learns, 2) the learner relearns based on the mistakes made before, and 3) the learner validates his learning. We develop an efficient algorithm to solve the optimization problem. We further apply our method to differentiable neural architecture search and data reweighting. Extensive experiments on CIFAR-10, CIFAR-100, ImageNet, and other related datasets powerfully demonstrate the effectiveness of our approach. The code of LFM is available at: https://github.com/importZL/LFM.

The brain is organized to minimize energy consumption while maximizing computation

Sharna D. Jamadar, Anna Behler, Hamish Deery, Michael Breakspear, The metabolic costs of cognition, Trends in Cognitive Sciences, Volume 29, Issue 6, 2025, Pages 541-555, 10.1016/j.tics.2024.11.010.

Cognition and behavior are emergent properties of brain systems that seek to maximize complex and adaptive behaviors while minimizing energy utilization. Different species reconcile this trade-off in different ways, but in humans the outcome is biased towards complex behaviors and hence relatively high energy use. However, even in energy-intensive brains, numerous parsimonious processes operate to optimize energy use. We review how this balance manifests in both homeostatic processes and task-associated cognition. We also consider the perturbations and disruptions of metabolism in neurocognitive diseases.

A possible explanation of the origin of the concept of number and some arithmetical operations based on language concepts

Stanislas Dehaene, Mathias Sablé-Meyer, Lorenzo Ciccione, Origins of numbers: a shared language-of-thought for arithmetic and geometry?, Trends in Cognitive Sciences, Volume 29, Issue 6, 2025, Pages 526-540, 10.1016/j.tics.2025.03.001.

Concepts of exact number are often thought to originate from counting and the successor function, or from a refinement of the approximate number system (ANS). We argue here for a third origin: a shared language-of-thought (LoT) for geometry and arithmetic that involves primitives of repetition, concatenation, and recursive embedding. Applied to sets, those primitives engender concepts of exact integers through recursive applications of additions and multiplications. Links between geometry and arithmetic also explain the emergence of higher-level notions (squares, primes, etc.). Under our hypothesis, understanding a number means having one or several mental expressions for it, and their minimal description length (MDL) determines how easily they can be mentally manipulated. Several historical, developmental, linguistic, and brain imaging phenomena provide preliminary support for our proposal.