How to Learn the Canny Edge Detector: Gaussian Smoothing, Gradients, Non-Maximum Suppression, Hysteresis and Production Vision Pipelines

How does a computer turn a grey image into a clean map of meaningful boundaries? The Canny edge detector does not rely on one threshold or one derivative. It builds an edge map through a sequence of stages: reduce noise, estimate gradients, thin candidate edges, separate strong from weak responses, and keep weak responses only when they connect to stronger evidence.

This article teaches Canny from first principles to production use. It complements the existing Hough Line Transform article: Canny produces edge evidence; Hough can then aggregate that evidence into line hypotheses.

Quick Read

Canny is a multi-stage edge detector introduced by John Canny in 1986.
Gaussian smoothing reduces noise before differentiation.
Image gradients estimate both edge strength and edge direction.
Non-maximum suppression thins broad gradient responses into narrow edge candidates.
Double thresholding separates strong, weak and rejected responses.
Hysteresis keeps weak edges when they are connected to strong edges, improving continuity without accepting every weak response.
Professional performance depends on blur scale, thresholds, gradient operator, image bit depth, contrast, noise, domain-specific validation and downstream needs.

1. Beginner Level: What Is an Edge?

An edge is a location where image intensity changes rapidly across space. A dark object against a bright wall creates a strong transition. A shadow, texture boundary or noise spike can also create a transition, which is why edge detection is not simply “find every big difference between neighbouring pixels.”

Canny is useful because it treats edge detection as a chain of decisions. Each stage removes a different kind of ambiguity. That makes it an excellent algorithm for learning how a pipeline can be more reliable than one clever formula.

2. Stage One: Gaussian Smoothing

Differentiation amplifies high-frequency changes, including noise. Canny therefore smooths the image before estimating gradients. A Gaussian filter replaces each pixel with a weighted average of nearby pixels, with closer pixels usually contributing more strongly.

The blur scale matters. A small sigma preserves fine detail but leaves more noise. A larger sigma suppresses noise and tiny structures but can shift, weaken or merge nearby edges. There is no universally correct sigma independently of image resolution and the smallest structure the task needs to preserve.

3. Stage Two: Gradient Magnitude and Direction

After smoothing, the algorithm estimates horizontal and vertical derivatives, commonly using Sobel-style filters in practical libraries. Call them Gx and Gy. The gradient magnitude measures how strong the local change is, while the gradient direction tells us which way intensity rises most steeply.

magnitude = sqrt(Gx*Gx + Gy*Gy)
angle = atan2(Gy, Gx)

OpenCV can also use an L1-style approximation |Gx| + |Gy| when its L2gradient option is false. That is a small example of a wider engineering pattern: mathematically cleaner and computationally cheaper formulations can coexist, and the right choice depends on accuracy and performance requirements.

4. Stage Three: Non-Maximum Suppression

A raw gradient image often produces thick ridges. Canny thins them by asking whether each candidate pixel is a local maximum across the gradient direction. If neighbouring pixels on either side have a stronger response, the current pixel is suppressed.

This detail is easy to miss: the comparison is not merely left versus right in image coordinates. The relevant direction depends on the local gradient orientation. Implementations may quantise that angle into a few directions or interpolate neighbouring magnitudes for more precise suppression.

5. Stage Four: Double Thresholds

After thinning, candidate pixels are classified using two thresholds:

above the high threshold: strong edge evidence;
between low and high: weak edge evidence;
below the low threshold: reject.

One threshold cannot express the difference between “definitely edge,” “possibly edge if connected,” and “too weak to trust.” The two-threshold design creates that middle state.

6. Stage Five: Hysteresis Edge Tracking

Weak edge pixels are not accepted automatically. They survive only when connected, directly or through other weak pixels, to strong edge evidence. This process is called hysteresis.

The result is often more coherent than a single global threshold. A faint section of a real edge can survive because it belongs to a stronger structure, while isolated weak noise is discarded.

7. A Small Conceptual Example

Imagine a white rectangle on a dark background, but one side is partly shadowed. The bright sides produce strong gradients. The shadowed side may produce only weak gradients. With one high threshold, that side disappears. With one low threshold, background texture may flood the edge map. Canny keeps the shadowed side when its weak responses connect to strong rectangle edges while rejecting many isolated weak responses elsewhere.

8. Pseudocode

smoothed = gaussian_filter(image, sigma)
Gx, Gy = spatial_derivatives(smoothed)
magnitude = gradient_strength(Gx, Gy)
direction = gradient_direction(Gx, Gy)

thin = non_maximum_suppression(magnitude, direction)

strong = thin >= high_threshold
weak = (thin >= low_threshold) and not strong

edges = hysteresis_connect(strong, weak)
return edges

The teaching value comes from tracing what each array means after each stage. If a learner cannot explain why a pixel survived or disappeared, the code is running but the algorithm is not yet understood.

9. OpenCV Example

import cv2

image = cv2.imread("scene.png", cv2.IMREAD_GRAYSCALE)

blurred = cv2.GaussianBlur(image, (5, 5), 1.2)
edges = cv2.Canny(
    blurred,
    threshold1=60,
    threshold2=140,
    apertureSize=3,
    L2gradient=True,
)

cv2.imwrite("edges.png", edges)

The threshold values above are examples, not universal defaults. A robust workflow chooses parameters using representative images and a measurable downstream objective.

10. scikit-image Example

from skimage import feature, io, color

image = io.imread("scene.png")
gray = color.rgb2gray(image)

edges = feature.canny(
    gray,
    sigma=1.5,
    low_threshold=0.08,
    high_threshold=0.18,
)

Different libraries use different intensity conventions and parameter semantics. Copying threshold numbers between pipelines without checking image range and preprocessing is a common mistake.

11. Parameter Effects Strong Learners Should Predict

Increase sigma: fewer fine details and less noise; small edges may disappear.
Increase high threshold: fewer seed edges; weak chains may lose their anchors.
Decrease low threshold: longer weak chains can survive, but noise connectivity can increase.
Narrow the gap between thresholds: hysteresis behaves more like a single-threshold detector.
Change gradient norm: edge strengths shift, so previous thresholds may no longer mean the same thing.
Resize the image: edge width, noise scale and useful sigma can all change.

Before running code, predict the direction of change. That habit is more educational than blindly sweeping parameters until the picture looks good.

12. Complexity and Memory

For a fixed-size Gaussian and derivative kernel, each major stage touches the image a constant number of times, so practical complexity is linear in the number of pixels. Larger separable Gaussian kernels still scale predictably with image size and kernel radius. Memory is typically linear because implementations store intermediate gradient, magnitude, classification or edge arrays.

On large images or video, the real bottlenecks may be memory bandwidth, cache traffic, image transfer between CPU and accelerator, and repeated format conversion rather than arithmetic alone.

13. Failure Modes in Real Images

Texture creates many legitimate local gradients that are irrelevant to the task.
Motion blur spreads transitions and weakens localization.
Uneven illumination changes edge strength across the same object.
Compression artefacts introduce false high-frequency structure.
Low-contrast boundaries may never cross the high threshold.
Excessive smoothing removes the very feature the system is supposed to detect.
Edges from shadows can dominate edges from physical object boundaries.
Thresholds tuned on one camera or exposure fail after a hardware or lighting change.

14. Professional Validation: Evaluate the Job, Not the Screenshot

An edge map can look impressive and still be wrong for the downstream task. If Canny feeds line detection, document rectification, measurement, segmentation or robotics, evaluate the final geometric or decision error. For labelled edge datasets, precision, recall and localization tolerance can be useful. For industrial use, false-edge and missed-edge costs may differ sharply.

Parameter selection should therefore be tested across representative noise, lighting, contrast, blur and camera conditions. A professional pipeline records preprocessing, threshold ranges and library versions so behaviour can be reproduced.

15. Learning Progression: Beginner to Professional

Beginner: inspect a one-dimensional intensity step and compute simple finite differences.
Intermediate: display the image after each Canny stage and explain every visible change.
Advanced: implement non-maximum suppression and hysteresis yourself, then compare with a library.
Professional: build parameter sweeps across image conditions, measure downstream accuracy and runtime, and determine when a learned edge detector or domain-specific method is preferable.

16. Practice Problems

Create a synthetic rectangle with Gaussian noise. Compare sigma values 0.8, 1.5 and 3.0.
Visualise gradient magnitude before non-maximum suppression and explain why the edges look thick.
Implement four-direction non-maximum suppression and test a 45-degree line.
Construct a weak edge connected to a strong edge and verify hysteresis preserves it.
Create isolated weak pixels with the same magnitude and verify they are rejected.
Compare L1 and L2 gradient magnitude in OpenCV and retune thresholds fairly.
Feed the resulting edges into the Hough transform and measure how Canny settings change line detection.

17. Sources and Further Reading

Final idea: Canny is not one edge formula. It is a sequence of filters that progressively asks stronger questions of the evidence: Is the signal strong enough? Is it locally maximal? Is it strong by itself, or connected to something stronger? That layered decision process is one of the reasons the algorithm remains so instructive decades after its publication.

Small Group Tutorials