Wait, What?
An optimization algorithm can improve a population of imperfect solutions without ever computing a gradient.
Evolutionary algorithms search by maintaining multiple candidate solutions, evaluating them, selecting some to influence the next generation and applying variation through mutation, recombination or related operators. Their strength is not that they imitate biology perfectly; it is that population-based stochastic search can work when objectives are discontinuous, noisy, mixed, multimodal or difficult to differentiate.
This article owns the evolutionary-algorithm learning job. The existing Randomized Algorithms article owns randomness as a general algorithmic tool, the Numerical Optimisation article owns gradient and Newton-style methods, and the Bayesian-optimization article owns surrogate-guided expensive black-box search. Evolutionary algorithms have a separate job: population-based search through selection and variation.
Quick Answer
Learn evolutionary algorithms through the route candidate representation → population → fitness → selection → mutation → crossover → replacement → diversity → elitism → constraint handling → genetic algorithms → evolution strategies → differential evolution → covariance adaptation → multi-objective Pareto search → NSGA-II → benchmarking → stochastic reliability. A beginner should be able to evolve bit strings toward a simple target and explain each operator. A professional should be able to design representations and operators that respect problem structure, diagnose premature convergence, compare against strong baselines, handle constraints and multiple objectives, and report stochastic performance rather than a single lucky run.
1. Begin With a Population You Can See
Give learners ten 8-bit strings and define fitness as the number of 1s. Select higher-fitness strings, copy or recombine them, flip a few bits randomly and create the next generation. Repeat by hand for three generations.
The exercise makes the algorithm concrete: no candidate “understands” the goal, yet selection repeatedly changes which variations are more likely to survive.
2. Representation Is the First Algorithmic Decision
A candidate solution might be a bit string, real-valued vector, permutation, tree, graph or program. The representation determines what mutations and crossovers are legal and what kinds of neighbouring solutions are easy to reach.
For a travelling-salesperson tour, an ordinary one-point crossover can create duplicate cities and omit others. A permutation-aware operator is needed. This teaches a professional lesson early: generic operators can violate problem semantics.
3. Fitness Is an Objective Interface, Not an Explanation
The fitness function tells the algorithm how candidates are compared. It may be the real objective, a transformed score, a penalized objective or a multi-objective relation.
If fitness rewards a proxy, the population will optimize that proxy. Evolutionary search can exploit loopholes just as aggressively as any other optimizer.
4. Selection Converts Fitness Differences Into Reproductive Pressure
Fitness-proportionate selection gives higher-scoring individuals more chance to contribute offspring. Tournament selection samples a small group and selects the best among them. Rank-based selection uses ordering rather than raw score differences.
Selection pressure matters: too weak and search drifts; too strong and diversity can collapse before the population reaches a good region.
5. Mutation Creates Local Variation
Bit-flip mutation toggles bits. Gaussian mutation perturbs real-valued coordinates. Swap mutation exchanges positions in a permutation. Mutation rate and step size determine how far the search moves.
Ask learners to compare a mutation rate of 0, 1/length and 0.5 on the same toy problem. The extremes reveal why variation must be calibrated.
6. Crossover Recombines Existing Structure
Crossover combines information from multiple parents. One-point and uniform crossover are easy to teach on bit strings; arithmetic crossover fits real vectors; ordered or partially matched crossover can preserve permutation validity.
The key question is not whether crossover is biologically realistic. It is whether the operator tends to preserve useful problem structure while exploring new combinations.
7. Replacement Defines Who Survives Into the Next Generation
Generational replacement discards most or all parents and uses offspring as the next population. Steady-state methods replace only a few individuals at a time. Elitism copies one or more top candidates forward unchanged.
Elitism preserves the best-so-far solution but can also increase selection pressure. Every operator changes the search dynamics.
8. Diversity Is a Resource
If all candidates become nearly identical, crossover stops creating meaningful novelty and the search can stagnate around a local optimum. Diversity can be monitored through genotype distance, phenotype difference, fitness spread or niche occupancy.
Methods such as fitness sharing, crowding, novelty search, random immigrants and restart strategies explicitly protect exploration.
9. A Basic Genetic Algorithm Is a Repeated Population Loop
population = initialize()
while not stop:
fitness = evaluate(population)
parents = select(population, fitness)
offspring = crossover_and_mutate(parents)
population = replace(population, offspring)
return best_seenThe loop is deliberately simple. Most professional performance comes from the quality of representation, variation, selection, constraint handling and evaluation—not from adding more lines to this skeleton.
10. Premature Convergence Is the Signature Failure Mode
Premature convergence occurs when the population loses useful diversity and clusters around a solution that is not globally good. Symptoms include rapidly shrinking genotype variation, little improvement after early generations and repeated copies of the same candidate.
Possible responses include reducing selection pressure, increasing mutation, using larger populations, maintaining niches or restarting from diverse seeds.
11. Constraint Handling Must Preserve the Real Feasible Set
Some candidates violate hard constraints. Options include repair operators, penalty functions, feasibility-preserving encodings and comparison rules that prefer feasible candidates before objective value.
A huge penalty may distort the landscape; a weak penalty may reward invalid solutions. Whenever possible, encode feasibility directly into representation and operators.
12. Evolution Strategies Put Mutation at the Centre
Evolution strategies were developed for continuous optimization and often emphasize Gaussian mutation, step-size adaptation and selection among parent and offspring populations.
Self-adaptation is a major idea: the algorithm can evolve not only candidate parameters but also parameters controlling its own search behaviour.
13. Differential Evolution Builds Mutations From Population Differences
Differential evolution creates a trial vector using scaled differences between existing population members, then recombines that mutant with a target vector and selects between target and trial.
The current SciPy differential_evolution documentation describes it as a stochastic global optimizer that does not use gradients and may search broad candidate regions at the cost of more function evaluations than many conventional gradient-based methods.
14. Differential Evolution’s Scale and Crossover Parameters Control Search Geometry
The mutation scale controls how strongly difference vectors move a candidate. Recombination probability controls how much of the mutant enters the trial vector. Small values can make search conservative; large values can disrupt useful structure.
Plot candidate trajectories on a 2D function so learners can see parameter effects rather than treating them as magic defaults.
15. CMA-ES Learns the Shape of Promising Search Directions
Covariance Matrix Adaptation Evolution Strategy samples candidate points from a multivariate normal distribution and adapts its covariance matrix so future samples align with directions that have produced progress.
The covariance becomes a learned geometry of the local search landscape. This is why CMA-ES can perform strongly on continuous black-box problems with correlated variables.
16. Population Size Changes Both Exploration and Cost
Larger populations provide more diversity and can reveal multiple promising regions in one generation, but each generation requires more objective evaluations. Small populations update quickly but may collapse early.
When evaluations can run in parallel, a larger population may have attractive wall-clock behaviour even if the evaluation count is higher.
17. Multi-Objective Problems Do Not Have One Best Scalar Answer
Suppose a design should be cheap, fast and energy efficient. Unless the objectives are combined with chosen weights, there may be many solutions where improving one objective worsens another.
A candidate is Pareto-dominated if another candidate is at least as good on every objective and strictly better on at least one. Non-dominated candidates form an approximation to the Pareto front.
18. NSGA-II Balances Convergence and Diversity on the Pareto Front
NSGA-II ranks candidates by non-dominated sorting and uses crowding distance to prefer solutions in less crowded regions. Elitist survival keeps strong non-dominated candidates while maintaining spread across trade-off space.
The classic NSGA-II paper introduced the fast non-dominated sorting and elitist multi-objective design that remains foundational in evolutionary multi-objective optimization.
19. Pareto Fronts Are Decision Surfaces, Not Automatic Decisions
An optimizer can show efficient trade-offs, but a human or downstream decision rule must still choose among them. Hiding multiple objectives behind one arbitrary weighted sum may conceal important alternatives.
20. Evolutionary Algorithms Are Naturally Stochastic
Different random seeds can produce different results. Therefore report distributions over repeated runs: median, quartiles, confidence intervals, success rates and worst cases—not only the single best run.
Use the same evaluation budget and comparable stopping rules when benchmarking algorithms.
21. Benchmark Functions Teach Failure Modes, Not Real-World Victory
Sphere, Rosenbrock, Rastrigin and other synthetic functions expose conditioning, multimodality and separability. They are useful controlled tests, but performance on benchmark suites does not prove superiority on unrelated real tasks.
A professional benchmark includes both diagnostic synthetic functions and the actual problem distribution.
22. Compare Against Random Search and Problem-Specific Heuristics
If a genetic algorithm cannot beat random search under the same evaluation budget, complexity has not earned its keep. If a domain-specific greedy or local-search heuristic performs better, the evolutionary method needs justification.
Baselines are not an insult to sophisticated algorithms; they are how sophistication proves value.
23. Hybrid Algorithms Can Combine Global and Local Search
A memetic algorithm may use evolutionary search to locate promising regions, then apply local optimization to refine candidates. Hybrid methods can exploit global exploration and efficient local improvement.
The risk is evaluation-budget imbalance: an expensive local optimizer applied to every candidate can consume the budget before population search contributes meaningfully.
24. Common Learning Failure States
- Choosing a representation before understanding feasibility constraints.
- Using generic crossover on permutations and generating invalid solutions.
- Setting very high selection pressure and collapsing diversity.
- Treating mutation as random noise rather than a designed neighbourhood operator.
- Assuming elitism can only help.
- Reporting one lucky seed.
- Comparing methods with unequal evaluation budgets.
- Calling any population heuristic a genetic algorithm.
- Collapsing multiple objectives into weights without showing trade-offs.
- Claiming a benchmark win proves universal superiority.
25. A Beginner-to-Professional Learning Ladder
- Level 1: evolve bit strings toward a target by hand.
- Level 2: compare tournament, rank and fitness-proportionate selection.
- Level 3: implement mutation and crossover for bit strings.
- Level 4: design valid operators for permutations or constrained vectors.
- Level 5: measure diversity and diagnose premature convergence.
- Level 6: implement or use differential evolution on continuous functions.
- Level 7: study adaptive step size and CMA-ES search geometry.
- Level 8: solve a constrained real-world optimization problem under a fixed evaluation budget.
- Level 9: approximate a Pareto front with NSGA-II and explain crowding distance.
- Level 10: benchmark repeated stochastic runs against random, local and domain-specific baselines.
26. Teach Operators With Predictable Micro-Experiments
Show two parents and a fixed random seed. Ask learners to predict the tournament winner, crossover point and mutation result. Run the code, inspect the child, then change one operator. This isolates causal understanding before the full stochastic loop becomes visually noisy.
27. PRIMM and Faded Examples Reduce Blank-Page Load
The current Raspberry Pi Foundation PRIMM course uses Predict, Run, Investigate, Modify and Make. For evolutionary algorithms, learners can predict one generation, run it, investigate why fitness changed, modify selection pressure or mutation, then build a new representation.
The National Centre for Computing Education programming-pedagogy course explicitly includes worked examples, pair programming and Parsons problems alongside PRIMM. These scaffolds are useful here because the conceptual challenge is the interaction among operators, not syntax memorization.
28. Build Reproducibility Into the First Implementation
Record random seeds, population size, evaluation budget, operator parameters, stopping rule and best-so-far trajectory. Reproducibility should be part of the algorithm, not a professional habit added later.
29. Professional Direction
Advanced study includes evolution strategies, CMA-ES, differential evolution variants, self-adaptation, estimation-of-distribution algorithms, genetic programming, neuroevolution, quality-diversity methods, novelty search, coevolution, island models, surrogate-assisted evolutionary optimization, NSGA-II/III, MOEA/D and constrained multi-objective search.
Algorithm-learning rule: never ask only whether evolution found a good candidate. Ask what representation made variation meaningful, whether selection destroyed diversity, how many objective evaluations were spent, whether constraints were respected, how results vary across seeds, and which simpler baseline the method actually beat.
eduKate Sengkang 