Variance Explained: Making Sense of Data Variability

In my previous post, I explained the concepts of mean and variance. However, I noticed that variance was unclear to many people. So, in this dedicated post, I’ll explain variance thoroughly.

Variance measures how spread out the data is. It tells us if the numbers in a dataset are close together or far apart from the average (mean).

👉 If variance is small → The numbers are similar and close to the mean.
👉 If variance is large → The numbers are very different and spread out.

Let's say we have a dataset:

Numbers: [2, 4, 6, 8, 10]

Step 1: Find the Mean (Average)

Mean = (2 + 4 + 6 + 8 + 10) / 5 = 6

Step 2: Find the Difference from the Mean (Deviation)

We subtract the mean from each number:

(2 - 6) = -4
(4 - 6) = -2
(6 - 6) = 0
(8 - 6) = 2
(10 - 6) = 4

Step 3: Square Each Difference

We square each value so that negatives don’t cancel out:

(-4)² = 16
(-2)² = 4
(0)² = 0
(2)² = 4
(4)² = 16

Step 4: Find the Average of These Squared Differences

Variance = (16 + 4 + 0 + 4 + 16) / 5 = 8

✅ Variance = 8
Since the variance is not zero, the numbers are spread out.

Low vs. High Variance

Let's compare two datasets:

1. Low Variance (Numbers are Close Together)

[4, 5, 5, 6, 6]

• Mean = 5.2
• Variance ≈ 0.56 (small spread)

2. High Variance (Numbers are Spread Out)

[1, 3, 7, 9, 11]

• Mean = 6.2
• Variance ≈ 16.56 (big spread)

👉 Low variance → All numbers are close to the mean.
👉 High variance → The numbers are spread far from the mean.

NumPy Example

Let’s compute variance in Python.

import numpy as np

data = np.array([2, 4, 6, 8, 10])
variance = np.var(data)

print("Variance:", variance)  # Output: 8.0

PyTorch Example

We can do the same in PyTorch.

import torch

tensor = torch.tensor([2, 4, 6, 8, 10], dtype=torch.float32)
variance = torch.var(tensor)

print("Variance:", variance.item())  # Output: 8.0

Why is Variance Important in Deep Learning?

✅ Feature Scaling: Normalize data so all features have similar variance.
✅ Weight Initialization: If weights have high variance, training becomes unstable.
✅ Loss Analysis: Track variance in errors to improve model performance.

Final Takeaway

• Small variance → Data is packed close together (consistent).
• Large variance → Data is spread out (inconsistent).