Exploring Your Dataset for Machine Learning

With all analysis, machine learning or not, it’s best to start by getting to grips with your dataset. In this notebook we will demonstrate how to use Python to visualise and characterise the dataset. This is essential preparation before we start building models, whether we use classical machine learning or deep neural networks.

Key objectives

• Load, explore and visualise the dataset
• Apply basic quality criteria, like checks for missing values
• Check for class imbalance
• Calculate correlations between variables

1. Introducing the Breast Cancer Dataset

The clinical samples used to form this dataset were collected from January 1989 to November 1991. Relevant features were extracted from the digital images using Multisurface Method-Tree (K.P. Bennett 1992). This is a classification method that uses linear programming to construct a decision tree – a form of machine learning!

The features are observable characteristics of the cells in the images – for example, radius, concavity, and texture. An example of one of the digitised images from an FNA sample is given below.

1b. Knowing your dataset

2. Setting Up Our Environment

Now that we understand the origins of our dataset, let’s explore it.

For this we will be using a combination of widely-used data science packages in Python. Here is a brief description of each:

• Pandas: A powerful data manipulation library that provides DataFrame structures ideal for working with labelled, tabular data
• Numpy: The fundamental package for numerical computing in Python, providing support for arrays and mathematical functions
• Matplotlib: A comprehensive plotting library capable of creating static, animated, and interactive visualisations
• Seaborn: A statistical data visualisation library that builds on matplotlib and provides a high-level interface for drawing attractive statistical graphs
• Scikit-learn: A machine learning library that provides simple and efficient tools for data analysis and modelling

Using a combination of these tools, we should be able to load, manipulate, and visualise our data effectively.

3. Loading and Initial Inspection

It looks like the dataframe is all numeric, with float64 features and an integer column for the diagnosis (the labels). We can check this by counting the values of the DataFrame’s dtypes.

4. Data Quality Checks

Data quality is crucial. We’re lucky enough to be using a well-prepared, clean dataset, but in real-world problems that’s not often the case! In this section we’ll run through some common data quality checks.

Null or missing values will often break machine learning models, so we need to appropriately handle them before we being training. There are a few common strategies for handling null values. You can remove the offending row altogether; use a replacement value (such as zero, or the average of the other values in the column); or interpolate the value based upon the other features in the row.

It’s also good to check for duplicates. This is more commons than you might think. Duplicates will introduce bias in training, slightly favouring the duplicated data point.

5. Target Variable Analysis

The goal of this kind of machine learning is to predict one aspect of a data point, based on the others. The thing you’re trying to predict is called the label, the class, or the target variable. In this case, the label is the diagnosis, with two possible values: 0 (benign) and 1 (malignant).

Class imbalance can significantly affect model performance, so understanding the class distribution is crucial for machine learning.

Note on Class Imbalance: An unbalanced distribution of classes in the target variable can affect your predictions with machine learning. If one class dominates, the algorithm might achieve high accuracy by simply predicting the majority class. In this dataset, we have a reasonable balance (approximately 1.7:1 ratio), though it’s worth noting that the malignant class is slightly more prevalent.

6. Statistical Summary of Features

A useful way to get an overview of the features is to look at the summary statistics – the mean, standard deviation, and quartile values – for each column. We can do that easily with pandas.

7. Feature Distributions

Understanding how features are distributed is essential for choosing appropriate preprocessing techniques and algorithms. Machine learning algorithms generally look for ways to separate points of different classes by finding high-dimensional patterns. These patterns are very difficult for our puny human brains to visualise, but what we can do is break down the problem and look at a couple of dimensions at a time.

The pairplot from seaborn is a great starting tool, to see roughly at a glance if there are any pairs of features that show clear differences between the classes. If you can see multiple pairs with visual separation between classes, there is a good chance a machine learning model will be perform well. Many pairs might have slight separation with a lot of ‘blur’ between the classes; however, in a higher dimensional space the boundry will hopefully be more defined.

The figure comes out qutie large because we have a lot of features, so we need to reduce the size and resolution a little to make it display nicely.

8. Feature Scaling and Comparison

Features often have different scales. Look at the summary statistics above; smoothness_mean ranges from 0.05 to 0.16, whereas area goes from 143.5 to 2501. Some machine learning algorithms find it difficult to compare features that range over such different magnitudes. Features whose units are larger can swamp the predictive space and have a disproportionately greater affect upon the predictions made by the algorithm. To combat this, we rescale the data so that all the features vary over the same range. We do this with the RobustScaler() class imported from scikit-learn.

9. Feature Relationships and Correlations

Highly correlated features can affect the prediction quality of some machine learning algorithms, particualy linear or logistic regression and linear SVMs. They affect our ability to determine feature importance, as correlated features essentialy capture the same information twice. Tree-based models and neural networks are more robust to correlated features, but it’s still beneficial to think about correlations before we proceed with modelling.

Understanding relationships between features helps identify multicollinearity and potential feature engineering opportunities.

From this analysis we can see that we have several highly correlated pairs of features. We could remove one feature from each pair for the most highly correlated features, such as perimeter_mean and perimeter_worst. However, a much cleaner method is to perform feature reduction through Principle Conponment Analysis (PCA). We’ll explore this below.

10. Dimensionality Reduction with PCA

Principal Component Analysis (PCA) is a statistical method to understand the main sources of variation in our data, reducing our feature space to a more managable number. PCA finds orthogonal directions in feature space along which the data has the most variation.

Given we get 95% of the variance explained with only 10 components, we could consisider only using these new features in our machine learning modelling. This would reduce the computational load and simplify the state space, at the expense of making the results a biut more difficult to explain.

In the next notebook, we’ll start to perform some modelling using classical machine learning methods.