Fast Algorithm for Outlier Detection in Time Series: Finding a Solution with a Minimum Amount of Rejected Measurement Data | Chapter 1 | Physical Science: New Insights and Developments Vol. 3

 

This study addresses the challenge of detecting coarse measurements (outliers) in time series data, a common issue in fields such as space geodynamics, geodesy, and other measurement-driven sciences. The proposed outlier detection algorithm solves two key problems: first, it finds a solution containing the maximum possible amount of measurement data remaining after detecting and removing outliers. Second, it requires the minimum possible number of arithmetic operations, estimated by the value O(NlogN), which cannot be improved in order N, to find a solution. To construct the algorithm, it was shown that the required solution should be sought in the ordered sequence of input data in the form of a set of consecutive numbers. The search for the set of maximum length is performed step by step, with the range of possible values of the set length being halved at each step.  The transition to one of the two halves is performed by checking the fulfilment of certain criteria, not for all possible values of lengths from the range under consideration, but only for its mid value. Testing the algorithm on real data obtained from laser rangefinder measurements showed the advantages of it over others, both in terms of time consumption and the amount of data remaining after detection and removal of outliers. The algorithm is robust and always finds a solution, if one exists. Its time efficiency is evident when cleaning a large amount of measurement data of outliers. It can be used for automated cleaning from outliers of observation data in information and measuring systems, in systems with artificial intelligence, as well as when solving various scientific, applied managerial and other problems using modern computer systems in order to obtain promptly the most accurate final result.

 

 

Author(s) Details

Igor V.Bezmenov
Russian Metrological Institute of Technical Physics and Radio Engineering, Mendeleevo, Moscow Region, Russia.

 

Please see the link:- https://doi.org/10.9734/bpi/psniad/v3/6424

 

A Standardized Threshold Approach for Outlier Detection Using Python Algorithms| Chapter 2 | Mathematics and Computer Science: Research Updates Vol. 6

This study aims to standardise threshold techniques and dataset preprocessing steps to improve outlier detection using Python-based algorithms. The objective of the study is to get results that are more precise in the algorithm for most types of datasets. The methodology involved using samples datasets and testing the results when using the normal thresholds of the python outliers detection algorithm, and comparing that results with the results have been done by using the generalize threshold which is mean-median. The results obtained from the supervised results showed that when standardising the threshold using the formula (Mean-Med) produced more precise and more generalised outcomes. The study also applied algorithms that use quartiles (Q1, Median, Q3) and found that adjusting the first quartile to 15% instead of the standard 25% helped to better isolate outliers in the lower range. Similarly, modifying the third quartile threshold to 80% instead of 75% provided more effective detection of upper-range outliers. More precise and more generalize results were obtained when using the formula in the python algorithms use threshold or normal thresholds, which are 0.1 to 2.5 as datasets threshold, compare to use the formula (Mean-Med). The study used some sample datasets for analysis and indicated the potential for applying the method to many other unsupervised datasets in future research.

 

Author(s) Details

 Zahir Saif Alhashami
Ministry of Education, Oman.

 

Please see the book here:-  https://doi.org/10.9734/bpi/mcsru/v6/5746

Trend Detection in Time Series Using the Minimizing Sets Method: Applications in Space Geodynamics and Beyond | Chapter 8 | Physical Science: New Insights and Developments Vol. 1

A significant problem in achieving the required level of accuracy of modern computing complexes and systems is the detection of coarse measurements (outliers) in the time series of data at the preprocessing stage. The proper detection and removal of outliers from measurement data is a necessary step in any scientific application to obtain the most accurate final result. This chapter addresses the challenge of detecting trends in time series data contaminated with outliers, a common issue in fields such as space geodynamics, geodesy, and other measurement-driven sciences. The proposed solution builds on the author’s previously developed Minimising Sets (MS) method, which iteratively constructs a trend by maximising the usable portion of the data and minimising the influence of outliers. The method is extended beyond power polynomials to include trigonometric functions with fixed frequencies and harmonic functions with unknown parameters, increasing its applicability across diverse physical systems. Simulations demonstrate that the approach yields accurate trend approximations and robust outlier detection, without relying on arbitrary thresholds. The technique shows promise for high-precision scientific applications requiring reliable preprocessing of noisy measurement data.

 

Author(s) Details

Igor V. Bezmenov
Russian Metrological Institute of Technical Physics and Radio Engineering, Mendeleevo, Moscow Region, Russia.

 

Please see the book here:- https://doi.org/10.9734/bpi/psniad/v1/5621

Deep Ensemble Outlier Detection in Lymphography Dataset | Chapter 5 | Contemporary Perspective on Science, Technology and Research Vol. 1

 Main data instances famous as outliers are those whose characteristics differ from those of the most of the instances in a dataset. With a difference of uses, including deception detection in credit card undertakings and intrusion discovery in computer route, outlier detection is a critical field of study in statistics and dossier mining. In the healing field, diseases can also be recognized from a variety of lab reports using aberration detection techniques. Investigators have developed any of techniques to recognize outliers in healthcare systems. This research aims to identify the outliers utilizing Deep Ensemble Approach in lymphography dataset. In order to recognize outliers in the lymphography dataset, this research suggests an ensemble approach established deep learning and isolation thickets. The suggested approach outperforms the individual models, in accordance with experimental results utilizing the lymphography dataset from the UCI machine learning repository that is to say accessible to the society.

Author(s) Details:

J. E. Judith,
Noorul Islam Centre for Higher Education, Kumaracoil, India.

Roy Thomas,
Noorul Islam Centre for Higher Education, Kumaracoil, India.

C. Dhayananth Jegan,
Stella Mary’s College of Engineering, Nagercoil, India.

Please see the link here: https://stm.bookpi.org/CPSTR-V1/article/view/12690