Data transformation is one of those steps that quietly shapes the quality of downstream analysis. When features operate on different scales, even well-designed models can behave in unexpected ways. Z-scoring, also known as standardization, is a simple and widely used approach that brings numerical features onto a common scale while preserving their underlying structure. This article shows how to standardize data in Python and why the scaler should be fit on the training data only.
What Z-Scoring does
Z-score standardization transforms each data point into a standardized value that reflects its distance from the mean of the feature, measured in standard deviations. A positive z-score indicates a value above the mean, while a negative z-score indicates a value below it. After transformation, the feature has a mean of zero and a standard deviation of one on the sample the scaler was fitted on. Applied to new data with those stored parameters, the values are recentred on the training mean rather than on their own, so they will generally not have mean zero and unit spread.
This transformation does not change the shape of the distribution. Correlations and the ordering of observations remain intact, and differences between values are preserved up to a constant factor of one divided by the standard deviation. Ratios between values are not, because subtracting the mean moves the origin. What changes is the center and the scale, which makes features comparable in spread.
Figure 1 follows a resting heart rate through this transformation. Drag the marker along the scale, or jump to the reference points, and the panel below recomputes the z-score one step at a time. The reading keeps its place among the others. Only the axis beneath it changes from beats per minute to standard deviations.
From raw values to z-scores
A z-score re-expresses a value as its distance from the mean, counted in standard deviations. Below is a resting heart rate. Standardizing leaves the reading itself alone and changes only the scale you describe it on.
A resting heart rate of 84 bpm sits 1.00 standard deviation above the mean of 72 bpm. On the standardized scale its value is +1.00.
Here the mean is μ = 72 bpm and the standard deviation is σ = 12 bpm (population standard deviation, dividing by n, as scikit-learn’s StandardScaler does).
The Z-Score formula
The z-score for a single observation is defined as:
z = (x − μ) / σ
where x is the original value, μ is the mean of the feature, and σ is the standard deviation. The arithmetic is trivial. What it does to a modeling workflow is not.
Throughout this article, σ is the population standard deviation, which divides by n and matches the convention used by scikit-learn’s StandardScaler. A sample standard deviation divides by n minus 1 and returns a larger value, though the difference is negligible unless the dataset is small.
A practical best practice is to compute μ and σ using only the training data, then apply the same parameters to validation and test sets. This avoids leaking information across data splits and keeps model evaluation honest.
Figure 2 sets this correct workflow beside a common mistake. Use the toggle to fit the scaler on the training data alone, then on the training and test data together. The second option lets the test set influence the mean and standard deviation. That changes the z-scores it receives, and leaks information into the evaluation.
Fit the scaler on the training data only
The mean and standard deviation should be learned from the training set, then reused to standardize the test set. Switch the workflow below to see what changes when test data is allowed to influence those statistics.
Fitted on: the training set only
μ = 50.0 years
σ = 8.0 years
56 years → +0.75
62 years → +1.50
68 years → +2.25
Handling different feature scales
Many real datasets mix variables with very different units. Age might range from tens to hundreds, while laboratory values or financial ratios may be fractions or large integers. Without standardization, features with larger numeric ranges tend to dominate optimization and distance calculations. Z-scoring places all features on a comparable scale, so that no single variable has a wider spread than the rest simply because of its units. That puts the spreads on an equal footing, though a standardized feature can still dominate a distance or an optimization.
Supporting distance-based methods
Algorithms that rely on distances or similarities are particularly sensitive to scale. Methods such as k-nearest neighbors, k-means clustering, and hierarchical clustering implicitly treat one unit of any feature as comparable to one unit of any other. Z-score standardization gives every feature the same spread, so no feature dominates a distance simply because it was measured on a larger scale. Equal spread is not the same as equal influence, and whether an equal-spread feature space is the intended one remains a modeling assumption.
Enabling meaningful comparisons
Standardized values support comparisons across features, and sometimes across datasets. A z-score of 2 carries the same interpretation regardless of the original unit and it represents a value two standard deviations above the mean. That interpretation holds in standard deviations only. Equal z-scores mean equal distance from each feature’s own mean. Whether they also mean equal rarity, or the same percentile, depends on the two distributions sharing a shape. Comparing across datasets asks for more still, namely that both means and standard deviations were estimated on comparable populations. That common scale is useful for downstream model development.
Practical considerations
Z-scoring assumes that the mean and standard deviation are meaningful summaries of the data. For heavily skewed distributions or features with extreme outliers, robust alternatives such as median-based scaling may be more appropriate. In practice, inspecting distributions before standardization often reveals whether z-scoring is a good fit. Another important detail is interpretability. While models may perform better on standardized data, coefficients are expressed per standard deviation of their feature and often need to be reported in the original units. Keeping track of scaling parameters makes it easy to reverse the transformation when needed.
A constant feature is a special case. Its standard deviation is zero, so the z-score formula would divide by zero. scikit-learn’s StandardScaler guards against this by treating the scale as 1, which returns a column of zeros for that feature.
Figure 3 shows what standardizing does to distributions of different shapes. Switch between a roughly symmetric feature, a right-skewed feature, and a feature with a single outlier. In every case the mean moves to 0 and the standard deviation to 1, yet the shape of the distribution and the order of the values stay the same. A skewed feature stays skewed, and an outlier stays extreme, which is exactly why the shape of a feature tells you how well its mean and standard deviation summarize it.
What standardization changes and what it keeps
Standardizing moves the mean to 0 and makes one unit equal to one standard deviation. The shape of the distribution and the order of the values do not change. Switch between distribution shapes to see what survives the transformation.
Example: Standardizing continuous variables
This notebook focuses on a small set of continuous variables from a cardiovascular dataset and prepares them for further analyses. This notebook is intentionally lightweight. The goal is to keep the data and plots easy to inspect, and to make it straightforward to extend with correlation heatmaps or pairwise scatterplots later.
Notebook Overview:
- Load a local CSV (
heart.csv) into a pandas DataFrame (this load is not among the code blocks below, which begin from a DataFrame named df that already holds the file) - Select a few continuous features of interest
- Visualize feature distributions before and after standardization
Data sources
Heart Failure Prediction Dataset gathers 11 routinely collected patient features and asks whether heart disease is present. It is aimed at early detection and risk stratification.
It merges five well-known heart disease cohorts into one harmonized table. Removing duplicates leaves 918 unique patient observations.
The variables span demographics, clinical readings, electrocardiogram (ECG), exercise and laboratory results. Research on cardiovascular risk modeling and clinical decision support leans on it heavily.
Further information about the dataset can be found here: https://www.kaggle.com/datasets/fedesoriano/heart-failure-prediction
Import dependencies
The imports below cover three needs:
- Data handling:
pandasfor loading and slicing the dataset - Preprocessing utility:
StandardScalerfor a quick, consistent standardization of selected columns - Visualization:
matplotlibfor figure setup andseabornfor histogram and kernel density estimate (KDE) plots
If this notebook is being run in a fresh environment, make sure the standard scientific Python stack is available (pandas, matplotlib, seaborn, scikit-learn).
import sys
import os
import seaborn as sns
import matplotlib.pyplot as plt
from sklearn.preprocessing import StandardScaler
import pandas as pdDefine variables of interest and apply a simple filter
A focused variable list keeps exploratory plots readable and avoids mixing data types.
In this notebook:
continuous_variablesdefines the numeric columns to visualize.- A quick plausibility filter is applied to
RestingBPto reduce the influence of extreme values that can dominate plots and obscure the central mass of the distribution.
This is a pragmatic cleaning step for visualization. Any downstream modeling workflow should document filters like this and revisit thresholds if domain requirements change.
continuous_variables = ["Age", "RestingBP", "MaxHR"]
df = df[(df["RestingBP"] >= 50) & (df["RestingBP"] <= 200)]Plot feature distributions before standardizing
Before changing any columns, it helps to look at the raw distributions.
This block:
- Creates one histogram per variable (plus a KDE overlay)
- Uses a consistent bin count so that differences are visually comparable
- Produces a quick scan for issues like heavy tails, multi-modality, or suspicious spikes
The intent here is just to build intuition about what the data looks like prior to further analysis.
# Set up the figure and axes
num_vars = len(continuous_variables)
fig, axes = plt.subplots(1, num_vars, figsize=(6 * num_vars, 4))
# Ensure axes is iterable even if there's only one plot
if num_vars == 1:
axes = [axes]
# Plot histograms with KDE for each continuous variable
for ax, variable in zip(axes, continuous_variables):
sns.histplot(data=df, x=variable, kde=True, bins=30, ax=ax)
ax.set_title(f'Distribution of {variable}')
ax.set_xlabel(variable)
ax.set_ylabel('Frequency')
plt.tight_layout()
plt.show()Raw distributions before standardizing
Apply standardization to the selected continuous variables
Here the notebook creates a copy of the DataFrame and applies StandardScaler to the columns in continuous_variables.
Implementation notes:
df_standardized = df.copy()keeps the original values available for reference.- Only the selected continuous columns are transformed, leaving the rest of the dataset untouched.
- The fitted scaler is kept in the
scalerobject, which is what makes the transform reversible viascaler.inverse_transform. Do not carry its stored mean and standard deviation into a modeling workflow. They were fitted here on the full DataFrame. Reusing them would let test rows shape the scaling parameters, which is the leakage Figure 2 warns about. If this notebook is later extended with train/test splits, fit a fresh scaler on the training split alone.
This section is deliberately brief and procedural, focusing on what the code is doing rather than the general theory behind rescaling.
# Create a scaler object
scaler = StandardScaler()
# Fit and transform the continuous variables
df_standardized = df.copy()
df_standardized[continuous_variables] = scaler.fit_transform(df[continuous_variables])Verify the updated feature distributions
This block repeats the same histogram workflow, but uses the updated DataFrame (df_standardized).
# Now plot the histograms as before but with the standardized data
# Set up the figure and axes
num_vars = len(continuous_variables)
fig, axes = plt.subplots(1, num_vars, figsize=(6 * num_vars, 4))
# Ensure axes is iterable even if there's only one plot
if num_vars == 1:
axes = [axes]
# Plot histograms with KDE for each continuous variable in the standardized data
for ax, variable in zip(axes, continuous_variables):
sns.histplot(data=df_standardized, x=variable, kde=True, bins=30, ax=ax)
ax.set_title(f'Distribution of Standardized {variable}')
ax.set_xlabel(variable)
ax.set_ylabel('Frequency')
plt.tight_layout()
# Save the figure as a PNG file
fig.savefig('z_histograms.png')
# Display the plot
plt.show()
The same shapes after standardizing
We can confirm that the standardization did not change the fundamental shape of the variable distributions. However, the three standardized columns are now centered to a mean of 0 and a standard deviation of 1, while the rest of the DataFrame is unchanged.
Key Takeaways
Subtracting the mean and dividing by the standard deviation moves the center to 0 and the spread to 1. Nothing is reshaped: skew, multi-modality and outliers all survive, and the ordering of the observations is unchanged.
The mean and standard deviation are estimated parameters. Fit them on the training split, then apply them to validation and test. Figure 2 shows the cost: adding the test set to the fit moves μ from 50.0 to 54.5 years and σ from 8.0 to 9.1.
k-nearest neighbors, k-means and hierarchical clustering compare distances across features, so a variable measured in hundreds can drown out one measured in fractions. Standardizing equalizes the spread of each feature, and equal spread still leaves each feature a different amount of influence on the fit.
An extreme value still sits far from the mean afterwards, and it inflated the σ that every other value was divided by. For heavily skewed features, or features carrying extreme outliers, median-based scaling may be the better summary.
A z of 2 means two standard deviations above the mean whatever the original unit. Two features share a percentile at the same z only when their distributions share a shape. Comparing z across datasets asks for more still, namely that both were fitted on comparable populations.
A coefficient fitted on standardized data is expressed per standard deviation of its feature rather than per original unit. Holding the fitted μ and σ makes the transform reversible and keeps results reportable in years, mmHg or bpm.
Data & License
Heart Failure Prediction Dataset by fedesoriano (Kaggle, 2021), combining five UCI heart-disease databases over 11 shared clinical features (918 records). Original clinical data © its creators — Andras Janosi, M.D. (Budapest); William Steinbrunn, M.D. (Zurich); Matthias Pfisterer, M.D. (Basel); and Robert Detrano, M.D., Ph.D. (Long Beach / Cleveland); donor David W. Aha (UCI). Source: kaggle.com/datasets/fedesoriano/heart-failure-prediction.License: Open Database License (ODbL) v1.0 for the database; the contents remain © the original authors named above. Used with attribution. ODbL 1.0
© 2025 Philip Sarajlic. All rights reserved for the article’s original text and figures; the third-party data remains under the license shown above.
Code examples in this article are licensed under the Common Public Attribution License Version 1.0 (CPAL-1.0), an OSI-approved copyleft license based on the Mozilla Public License 1.1. Initial Developer: Philip Sarajlic.
Attribution required by CPAL Exhibit B: © 2025 Philip Sarajlic · “Based on code by Philip Sarajlic” · philipsarajlic.com · no graphic image. This attribution must be displayed in Larger Works.
Modifications must be released in Source Code form under CPAL-1.0. Making the code usable by anyone other than you over a network is External Deployment under the license and is treated as distribution, so the Source Code must be made available to those users.
Full text: opensource.org/license/cpal-1-0 (SPDX identifier CPAL-1.0)
