Identifying Outliers through Unsupervised Machine Learning Techniques
In the realm of data analysis, outliers – data points that deviate significantly from the norm – can often distort analysis and affect modeling. Two popular methods for identifying these anomalous observations are the Local Outlier Factor (LOF) and Gaussian Mixture Models (GMM). Both algorithms are unsupervised, meaning they can identify outliers without the need for labeled data.
Local Outlier Factor (LOF)
The Local Outlier Factor (LOF) is a density-based method that compares the local density around a data point to the densities of its neighbors. It uses the concept of reachability distance, which considers the distances to the (k) nearest neighbors. By computing the ratio of the local reachability density of a point to that of its neighbors, LOF assigns an outlier score: a higher score indicates the point is in a sparser region relative to its neighborhood and is likely an outlier.
In Scikit-Learn, the class implements the LOF algorithm. You specify the number of neighbors (k) (parameter ). It computes the LOF score for each point based on local densities and labels points with high LOF scores as outliers.
Gaussian Mixture Models (GMM)
Gaussian Mixture Models (GMM) are probabilistic models that assume the data is generated from a mixture of several Gaussian distributions. GMM fits these Gaussians to the dataset by estimating parameters using methods like Expectation-Maximization. Then, the probability of each data point under the model is computed. Points with very low probability (i.e., that do not fit well into any Gaussian component) are flagged as outliers.
In Scikit-Learn, the class fits the GMM to the data. After fitting, you can compute the log-likelihood of each point under the model. Points with low likelihood are treated as anomalies. This is a soft clustering method and requires you to choose the number of Gaussian components.
Comparing LOF and GMM
While both methods detect outliers, they operate differently. LOF excels at detecting anomalies that are isolated or lie in low-density areas compared to their neighbors, while GMM detects points inconsistent with the overall probabilistic distribution structure.
Here's a summary table of key aspects:
| Aspect | Local Outlier Factor (LOF) | Gaussian Mixture Model (GMM) | |-----------------------------|---------------------------------------------|--------------------------------------------------| | Approach | Density-based (local density comparison) | Probabilistic (mixture of Gaussian distributions) | | Core concept | Local reachability density ratio | Low likelihood under Gaussian mixture components | | Key parameter | Number of neighbors ((k)) | Number of Gaussian components | | Outputs | LOF score (higher means more outlier-like) | Probability density or log-likelihood | | Suitable for | Detecting isolated outliers or sparse local neighborhoods | Detecting points inconsistent with global data distribution | | Implementation in Scikit-Learn | class | class |
These methods complement each other and may be selected based on the nature of data and expected outlier patterns.
Further Resources
For those interested in learning more about these topics, the books "Hands-on Machine Learning with Scikit-Learn, Keras & TensorFlow" by Aurélien Géron and "Data Cleaning and Exploration with Machine Learning" by Michael Walker are excellent resources. The full code for this example can be found on Gustavo Santos' GitHub. Additionally, there are other algorithms and methods available for finding outliers, such as Isolation Forest, Z-Score, and IQR.
Technology, such as data-and-cloud-computing platforms, enables the implementation of advanced algorithms like Local Outlier Factor (LOF) and Gaussian Mixture Models (GMM) for identifying outliers in data analysis. These techniques, including LOF's density-based approach using local reachability density ratio and GMM's probabilistic method with Gaussian mixture components, help in detecting and analyzing anomalous observations that might otherwise distort analysis and modeling.
