Author Archives: Juan-antonio Fernández-madrigal

Non-parameterical detection of regimes in time series data (SODA), and its use in fuzzy forecasting

November 10, 2023 09:17 ,

Shivani Pant, Sanjay Kumar, IFS and SODA based computational method for fuzzy time series forecasting, Expert Systems with Applications, Volume 209, 2022 DOI: 10.1016/j.eswa.2022.118213.

Time series forecasting has fascinated a great deal of interest from various research communities due to its wide applications in medicine, economics, finance, engineering and many other crucial fields. Various studies in past have shown that intuitionistic fuzzy sets (IFSs) not only handle non-stochastic non-determinism in time series forecasting but also enhance accuracy in forecasted outputs. Clustering is another one of the methods that improves accuracy of time series forecasting. The contribution of this research work is a novel computational fuzzy time series (FTS) forecasting method which relies on IFSs and self-organized direction aware (SODA) approach of clustering. The usage of SODA aids in making the proposed FTS forecasting method as autonomous as feasible, as it does not require human intervention or prior knowledge of the data. Forecasted outputs in proposed FTS forecasting method are computed using a weighted formula and weights are optimized using grey wolf optimization (GWO) method. Proposed FTS is applied to forecast enrolments of the University of Alabama and market price of State Bank of India (SBI) share at Bombay stock exchange (BSE), India and performance is compared in terms of root mean square error (RMSE), average forecasting error (AFE) and mean absolute deviation (MAD). Goodness of the proposed FTS forecasting method in forecasting enrolments of the University of Alabama and market price of SBI share is also tested using coefficient of correlation and determination, criteria of Akaike and Bayesian information.

See also https://babel.isa.uma.es/kipr/?p=1550

Posted in: Probability and statistics , Tagged: ,

On the extended use of RL for navigation in UAVs

November 3, 2023 10:07 ,

Fadi AlMahamid, Katarina Grolinger, Autonomous Unmanned Aerial Vehicle navigation using Reinforcement Learning: A systematic review, Engineering Applications of Artificial Intelligence, Volume 115, 2022 DOI: 10.1016/j.engappai.2022.105321.

There is an increasing demand for using Unmanned Aerial Vehicle (UAV), known as drones, in different applications such as packages delivery, traffic monitoring, search and rescue operations, and military combat engagements. In all of these applications, the UAV is used to navigate the environment autonomously \u2014 without human interaction, perform specific tasks and avoid obstacles. Autonomous UAV navigation is commonly accomplished using Reinforcement Learning (RL), where agents act as experts in a domain to navigate the environment while avoiding obstacles. Understanding the navigation environment and algorithmic limitations plays an essential role in choosing the appropriate RL algorithm to solve the navigation problem effectively. Consequently, this study first identifies the main UAV navigation tasks and discusses navigation frameworks and simulation software. Next, RL algorithms are classified and discussed based on the environment, algorithm characteristics, abilities, and applications in different UAV navigation problems, which will help the practitioners and researchers select the appropriate RL algorithms for their UAV navigation use cases. Moreover, identified gaps and opportunities will drive UAV navigation research.

Posted in: Applications of reinforcement learning to robots , Tagged: , , ,

Current state of the practical use of real-time systems, got through industry questionnaires

November 3, 2023 09:54 ,

Akesson, B., Nasri, M., Nelissen, G. et al. A comprehensive survey of industry practice in real-time systems, Real-Time Syst 58, 358\u2013398 (2022) DOI: 10.1007/s11241-021-09376-1.

This paper presents results and observations from a survey of 120 industry practitioners in the field of real-time embedded systems. The survey provides insights into the characteristics of the systems being developed today and identifies important trends for the future. It extends the results from the survey data to the broader population that it is representative of, and discusses significant differences between application domains. The survey aims to inform both academics and practitioners, helping to avoid divergence between industry practice and academic research. The value of this research is highlighted by a study showing that the aggregate findings of the survey are not common knowledge in the real-time systems community.

Posted in: Real-Time Systems , Tagged: ,

A remote Matlab laboratory for LTI system identification

November 3, 2023 09:32 ,

Z. Lei, H. Zhou, W. Hu and G. -P. Liu, Teaching and Comprehensive Learning With Remote Laboratories and MATLAB for an Undergraduate System Identification Course, EEE Transactions on Education, vol. 65, no. 3, pp. 402-408, Aug. 2022 DOI: 10.1109/TE.2021.3123302.

Contribution: This article introduces the teaching and learning with remote laboratories and MATLAB for an undergraduate system identification (SI) course, which can be employed for students at the advanced level with a control background. Background: SI has been widely used in all engineering fields; thus, the SI course that includes complex theories, concepts, and formulas is crucial for engineering education. Constraints, such as time, space, cost, and maintenance work, pose limitations for conventional laboratories, and current remote laboratories may not offer experiences to enhance control-oriented practical skills. Intended Outcomes: The proposed teaching and learning using remote laboratories is intended to facilitate the understanding of theories and concepts, and enhance the ability of design and implementation of control algorithms, the conducting of experiments, data collecting, data analysis, and the conducting of SI with MATLAB. Application Design: In the classroom teaching, theoretical lectures regarding SI are delivered to students by the teacher, along with the classroom demonstration with the networked control system laboratory for online experimentation. Then, the laboratory work is required to be completed by the students using the remote laboratory. A tailored laboratory report is supposed to be handed in by each student after the experimentation. Findings: The effectiveness of the proposed method was evaluated through the analysis of student performance and student responses to surveys. The student performance analysis indicates that the application of the remote laboratories is effective, and the feedback from students shows that they can benefit from the application of remote laboratories, and they would like the remote laboratories to be expanded to other courses.

Posted in: Control Engineering , Tagged: , , ,

On the importance of the static structures + execution flow in learning programming languages

November 3, 2023 09:27 ,

B. Bettin, M. Jarvie-Eggart, K. S. Steelman and C. Wallace, Preparing First-Year Engineering Students to Think About Code: A Guided Inquiry Approach, IEEE Transactions on Education, vol. 65, no. 3, pp. 309-319, Aug. 2022 DOI: 10.1109/TE.2021.3140051.

In the wake of the so-called fourth industrial revolution, computer programming has become a foundational competency across engineering disciplines. Yet engineering students often resist the notion that computer programming is a skill relevant to their future profession. Here are presented two activities aimed at supporting the early development of engineering students\u2019 attitudes and abilities regarding programming in a first-year engineering course. Both activities offer students insights into the way programs are constructed, which have been identified as a source of confusion that may negatively affect acceptance. In the first activity, a structured, language-independent way to approach programming problems through guided questions was introduced, which has previously been used successfully in introductory computer science courses. The team hypothesized that guiding students through a structured reflection on how they construct programs for their class assignments might help reveal an understandable structure to them. Results showed that students in the intervention group scored nearly a full letter grade higher on the unit\u2019s final programming assessment than those in the control condition. The second activity aimed to help students recognize how their experience with MATLAB might help them interpret code in other programming languages. In the intervention group, students were asked to review and provide comments for code written in a variety of programming languages. A qualitative analysis of their reflections examined what skills students reported they used and, specifically, how prior MATLAB experience may have aided their ability to read and comment on the unfamiliar code. Overall, the ability to understand and recognize syntactic constructs was an essential skill in making sense of code written in unfamiliar programming languages. Syntactic constructs, lexical elements, and patterns were all recognized as essential landmarks used by students interpreting code they did not write, especially in new languages. Developing an understanding of the static structure and dynamic flow required of programs was also an essential skill which helped the students. Together, the results from the first activity and the insights gained from the second activity suggest that guided questions to build skills in reading code may help mitigate confusion about program construction, thereby better preparing engineering students for computing-intensive careers.

Posted in: Computer science , Tagged: ,

Doing a more intelligent exploration in RL based on measuring uncertainty through prediction

November 3, 2023 09:12 ,

Xiaoshu Zhou, Fei Zhu, Peiyao Zhao, Within the scope of prediction: Shaping intrinsic rewards via evaluating uncertainty, Expert Systems with Applications, Volume 206, 2022 DOI: 10.1016/j.eswa.2022.117775.

The agent of reinforcement learning based approaches needs to explore to learn more about the environment to seek optimal policy. However, simply increasing the frequency of stochastic exploration sometimes fails to work or even causes the agent to fall into traps. To solve the problem, it is essential to improve the quality of exploration. An approach, referred to as the scope of prediction based on uncertainty exploration (SPE), is proposed, taking advantage of the uncertainty mechanism and considering the stochasticity of prospecting. As by uncertainty mechanism, the unexpected states make more curiosity, the model derives higher uncertainty by projecting future scenarios to compare with the actual future to explore the world. The SPE method utilizes a prediction network to predict subsequent observations and calculates the mean squared difference value of the real observations and the following observations to measure uncertainty, encouraging the agent to explore unknown regions more effectively. Moreover, to reduce the noise interference caused by uncertainty, a reward-penalty model is developed to discriminate the noise by current observations and action prediction for future rewards to improve the interference ability against noise so that the agent can escape from the noisy region. Experiment results showed that deep reinforcement learning approaches equipped with SPE demonstrated significant improvements in simulated environments.

Posted in: Reinforcement learning theory , Tagged: , ,

Normal blindness to visible objects seems to be the result of limited-capacity prediction mechanisms in the brain

November 3, 2023 09:05 ,

Jeremy M. Wolfe, Anna Kosovicheva, Benjamin Wolfe, Normal blindness: when we Look But Fail To See, Trends in Cognitive Sciences, Volume 26, Issue 9, 2022, Pages 809-819 DOI: 10.1016/j.tics.2022.06.006.

Humans routinely miss important information that is \u2018right in front of our eyes\u2019, from overlooking typos in a paper to failing to see a cyclist in an intersection. Recent studies on these \u2018Looked But Failed To See\u2019 (LBFTS) errors point to a common mechanism underlying these failures, whether the missed item was an unexpected gorilla, the clearly defined target of a visual search, or that simple typo. We argue that normal blindness is the by-product of the limited-capacity prediction engine that is our visual system. The processes that evolved to allow us to move through the world with ease are virtually guaranteed to cause us to miss some significant stimuli, especially in important tasks like driving and medical image perception.

Posted in: Psycho-physiological bases of engineering , Tagged:

On the existence of multiple fundamental “languages” in the brain that use discrete symbols and a few basic structures

November 3, 2023 09:03 ,

Stanislas Dehaene, Fosca Al Roumi, Yair Lakretz, Samuel Planton, Mathias Sabl�-Meyer, Symbols and mental programs: a hypothesis about human singularity, Trends in Cognitive Sciences, Volume 26, Issue 9, 2022, Pages 751-766 DOI: 10.1016/j.tics.2022.06.010.

Natural language is often seen as the single factor that explains the cognitive singularity of the human species. Instead, we propose that humans possess multiple internal languages of thought, akin to computer languages, which encode and compress structures in various domains (mathematics, music, shape\u2026). These languages rely on cortical circuits distinct from classical language areas. Each is characterized by: (i) the discretization of a domain using a small set of symbols, and (ii) their recursive composition into mental programs that encode nested repetitions with variations. In various tasks of elementary shape or sequence perception, minimum description length in the proposed languages captures human behavior and brain activity, whereas non-human primate data are captured by simpler nonsymbolic models. Our research argues in favor of discrete symbolic models of human thought.

Posted in: Psycho-physiological bases of engineering , Tagged: ,

Survey on RL applied to cyber-security

October 27, 2023 06:19 ,

Amrin Maria Khan Adawadkar, Nilima Kulkarni, Cyber-security and reinforcement learning \u2014 A brief survey, Engineering Applications of Artificial Intelligence, Volume 114, 2022, DOI: 10.1016/j.engappai.2022.105116.

This paper presents a comprehensive literature review on Reinforcement Learning (RL) techniques used in Intrusion Detection Systems (IDS), Intrusion Prevention Systems (IPS), Internet of Things (IoT) and Identity and Access Management (IAM). This study reviews scientific documents such as journals and articles, from 2010 to 2021, extracted from the Science Direct, ACM, IEEEXplore, and Springer database. Most of the research articles published in 2020 and 2021, for cybersecurity and RL are for IDS classifiers and resource optimization in IoTs. Some datasets used for training RL-based IDS algorithms are NSL-KDD, CICIDS, and AWID. There are few datasets and publications for IAM. The few that exist focus on the physical layer authentication. The current state of the art lacks standard evaluation criteria, however, we have identified parameters like detection rate, precision, and accuracy which can be used to compare the algorithms employing RL. This paper is suitable for new researchers, students, and beginners in the field of RL who want to learn about the field and identify problem areas.

Posted in: Computer vision , Tagged: ,

Hierarchical RL with diverse methods integrated in the framework

October 27, 2023 06:12 ,

Ye Zhou, Hann Woei Ho, Online robot guidance and navigation in non-stationary environment with hybrid Hierarchical Reinforcement Learning, Engineering Applications of Artificial Intelligence, Volume 114, 2022 DOI: 10.1016/j.engappai.2022.105152.

Hierarchical Reinforcement Learning (HRL) provides an option to solve complex guidance and navigation problems with high-dimensional spaces, multiple objectives, and a large number of states and actions. The current HRL methods often use the same or similar reinforcement learning methods within one application so that multiple objectives can be easily combined. Since there is not a single learning method that can benefit all targets, hybrid Hierarchical Reinforcement Learning (hHRL) was proposed to use various methods to optimize the learning with different types of information and objectives in one application. The previous hHRL method, however, requires manual task-specific designs, which involves engineers\u2019 preferences and may impede its transfer learning ability. This paper, therefore, proposes a systematic online guidance and navigation method under the framework of hHRL, which generalizes training samples with a function approximator, decomposes the state space automatically, and thus does not require task-specific designs. The simulation results indicate that the proposed method is superior to the previous hHRL method, which requires manual decomposition, in terms of the convergence rate and the learnt policy. It is also shown that this method is generally applicable to non-stationary environments changing over episodes and over time without the loss of efficiency even with noisy state information.

Posted in: Applications of reinforcement learning to robots , Tagged: ,