2019 Evolutionary Algorithms Review

Title

A review of evolutionary algorithms proposes a new taxonomy focusing on control, explainability, causality, bias, and corrective measures.

Preface

An Evolutionary Algorithm replaces manual chemist experimentation to explore chemical problem spaces.

Introduction

AI science sits at the boundary of philosophy and science, combining theoretical ideas with practical engineering.

User Control Attributes

Rule-based ML is transitioning to outcome-oriented systems, with UCA (limiters, explainability, causality, fairness, and correction) and trust considerations.

Control Attributes in ML

Modern ML evaluates models by control attributes including limits, explainability, causality, fairness, and the ability to correct.

End of Moore's Law

Advances in silicon are hitting physical and economic limits, signaling the end of Moore’s Law.

SpiNNaker

SpiNNaker uses a million interconnected ARM cores to run Spiking Neural Networks, roughly equating to about 1% of a human brain.

Hybrid Evolutionary Algorithms

Hybridizing evolutionary algorithms with neural networks is a growing approach to automate complex problems.

Introduction

EAs balance exploitative local search with explorative stochastic search.

Figure 2.1

The figure illustrates how hardware capability and algorithmic efficiency co-evolve, expanding the unknown search space.

End of Dennard Scaling

End of Dennard scaling shifts power-density concerns and enables new hardware concepts and deep learning resurgence.

No Free Lunch Theorem

No single algorithm dominates across all problems; domain knowledge is needed to achieve efficiency.

Evolutionary Algorithms Overview

Evolutionary algorithms are population-based metaheuristics driven by selection, variation, and reproduction.

Fitness and Objectives in EAs

Fitness functions quantify success and guide selection; multi-objective optimization seeks Pareto-optimal solutions.

Introduction

EAs tackle problems where traditional methods struggle due to resources, dimensionality, or complexity.

Figure 2.1

A captioned figure linking hardware progress to algorithmic capability and unknown problem spaces.

End of Dennard Scaling

Shifts in power dynamics and new hardware concepts accompany the deep-learning resurgence.

No Free Lunch Theorem

No single algorithm outperforms all problems; domain-specific knowledge enhances efficiency.

Overview of Evolutionary Algorithms

EAs are population-based metaheuristics that evolve solutions through selection, reproduction, and variation.

Fitness and Objectives in EAs

Fitness measures guide selection; many-objective optimization seeks trade-offs along Pareto fronts.

Introduction

EAs apply when traditional exploitative or stochastic methods fail due to complexity or resources.

Figure 2.1

Shows the relationship between hardware improvement and algorithmic progress over time.

End of Dennard Scaling

Describes how diminishing power density leads to reliance on new computing models and AI advances.

No Free Lunch Theorem

No universal solver exists; domain knowledge increases efficiency for specific problems.

Overview of Evolutionary Algorithms

EAs are population-based metaheuristics that use evolution-inspired operators to explore problem spaces.

Fitness and Objectives in EAs

Fitness functions evaluate performance and multi-objective optimization seeks balanced trade-offs.

Introduction

EAs address problems where classic methods struggle due to complexity or resource limits.

Figure 2.1

Depicts how hardware scaling and algorithmic progress enable exploration of unknown problem regions.

End of Dennard Scaling

Points to new hardware paradigms and AI-driven software growth beyond Dennard limits.

No Free Lunch Theorem

Domain knowledge is essential for efficient problem solving when using EAs.

Evolutionary Algorithms Overview

EAs are population-based methods using selection, variation and reproduction to search spaces.

Fitness and Objectives in EAs

Fitness guides progression; multi-objective EAs aim for Pareto-optimal compromises.

Introduction

Introductory discussion on when EAs are advantageous relative to traditional methods.

Figure 2.1

Illustrates hardware vs software growth and the unknowns at the frontier.

End of Dennard Scaling

Hardware scaling constraints drive exploration of novel computing paradigms.

No Free Lunch Theorem

Algorithm performance is problem-dependent; no universal best method.

Overview of Evolutionary Algorithms

EAs are Darwinian metaheuristics consisting of population, variation, selection.

Fitness and Objectives in EAs

The fitness function determines success and guides evolution; multi-objective issues exist.

Introduction

EAs tackle hard problems where standard methods fail or are inefficient.

Figure 2.1

Depicts the interplay of hardware and algorithmic progress over time.

End of Dennard Scaling

Denotes the shift to new computing approaches alongside AI trends.

No Free Lunch Theorem

No single algorithm excels for all problems; domain knowledge improves efficacy.

Evolutionary Algorithms Overview

EA families like GAs, GP, GE, CGP, PushGP explore problem spaces via evolution.

Fitness and Objectives in EAs

Fitness quantifies progress; multi-objective optimization balances conflicting goals.

Introduction

EAs provide a framework for solving complex, high-dimensional problems.

Figure 2.1

Illustrates how hardware capabilities enable broader exploration of problem spaces.

End of Dennard Scaling

Outlines transitions to alternative computing models to sustain AI progress.

No Free Lunch Theorem

No universal algorithm exists; domain knowledge improves search efficiency.

Overview of Evolutionary Algorithms

EAs are population-based metaheuristics evolving solutions via selection and variation.

Fitness and Objectives in EAs

Fitness evaluation drives selection; multi-objective optimization seeks Pareto-optimal fronts.

Introduction

Introduces when EAs outperform traditional approaches.

Figure 2.1

Links hardware progress with AI algorithmic capabilities and unknown regions.

End of Dennard Scaling

Signals a move to new hardware paradigms and deep learning resurgence.

No Free Lunch Theorem

No single algorithm is universally best; domain knowledge improves efficiency.

Evolutionary Algorithms Overview

Overview of evolutionary algorithms and their metaheuristic nature.

Fitness and Objectives in EAs

Fitness functions guide evolution; multi-objective optimization navigates trade-offs.

Introduction

Discussion of when EAs are advantageous over traditional methods.

Figure 2.1

Illustrates the relationship between hardware improvements and algorithmic progress.

End of Dennard Scaling

End of scaling drives exploration of new computing models.

No Free Lunch Theorem

No universal algorithm exists; domain knowledge improves efficiency.

Overview of Evolutionary Algorithms

EA families evolve solutions via population-based processes.

Fitness and Objectives in EAs

Fitness measures success; multi-objective optimization seeks Pareto frontiers.

Introduction

EAs solve complex, high-dimensional problems where other methods fail.

Figure 2.1

Shows hardware and software progress and the unknown frontiers.

End of Dennard Scaling

Describes a shift to advanced hardware and AI-enabled systems.

No Free Lunch Theorem

Domain-specific knowledge is essential for efficient search.

Evolutionary Algorithms Overview

Overview of population-based metaheuristics.

Fitness and Objectives in EAs

Fitness guides evolution; multi-objective optimization balances several goals.

Introduction

Explains why EAs are used for challenging problems.

Figure 2.1

Depicts hardware progress enabling new problem-solving capabilities.

End of Dennard Scaling

Describes the move to novel hardware models to support AI.

No Free Lunch Theorem

No universal algorithm exists; domain knowledge drives efficiency.

Overview of Evolutionary Algorithms

EA families such as GA, GP, GE, CGP, and PushGP evolve programs and designs.

Fitness and Objectives in EAs

Fitness calculations drive selection; multi-objective may yield Pareto-optimal sets.

Introduction

Introduces the role of EAs in solving complex problems.

Figure 2.1

Relates hardware capability to algorithmic performance and unknown regions.

End of Dennard Scaling

Discusses shifts to new computing paradigms and AI advances.

No Free Lunch Theorem

No single algorithm is best for all problems; domain knowledge helps.

Evolutionary Algorithms Overview

Overview of evolutionary algorithms and their metaheuristic nature.

Fitness and Objectives in EAs

Fitness measures success; multiple objectives require Pareto optimization.

Introduction

Explains when EAs are advantageous relative to traditional methods.

Figure 2.1

Illustrates hardware progression and algorithmic capacity growth.

End of Dennard Scaling

Hardware scaling limits spur new computing approaches for AI.

No Free Lunch Theorem

There is no universal best algorithm; domain knowledge improves search.

Overview of Evolutionary Algorithms

Introduction to EAs as population-based metaheuristics.

Fitness and Objectives in EAs

Fitness guides evolution; multi-objective optimization seeks Pareto-optimal solutions.

Introduction

Discusses when EAs are favored over traditional optimization methods.

Figure 2.1

Illustrates the interplay of hardware progress and algorithmic capabilities.

End of Dennard Scaling

Describes the transition to new hardware paradigms to sustained AI progress.

No Free Lunch Theorem

No universal best algorithm; domain knowledge enhances efficiency.

Traditional Techniques Overview

An overview of established EAs used in industry and research.

Evolutionary Strategy (ES)

ES evolves continuous parameters using mutation and selection, with CMA-ES and CMSA-ES variants.

Genetic Algorithms (GA)

GA optimizes fixed-length strings representing variables or parameters using crossover and mutation.

Genetic Programming (GP)

GP evolves executable programs or equations using tree-like representations.

Genetic Improvement (GI)

GI optimizes existing working code to improve performance or correctness.

Grammatical Evolution (GE)

GE evolves programs by evolving grammars (BNF) to generate code.

Linear Genetic Programming (LGP)

LGP uses linear programs for sequential problems and low-level optimizations.

Cartesian Genetic Programming (CGP)

CGP uses Cartesian graphs with small populations to solve structured problems.

Differential Evolution (DE)

DE optimizes by weighted differences and self-organizes populations.

Gene Expression Programming (GEP)

GEP uses fixed-length strings encoding expression trees to generate valid programs.

Specialized Techniques

Covers exotic and hybrid EAs beyond traditional methods.

Auto-constructive Evolution

Entities evolve themselves without a central controller to form offspring.

Neuroevolution

Uses genetic algorithms to optimize neural networks and architectures.