Hierarchical Bayesian Optimization Algorithm

This book provides a framework for the design of competent optimization techniques by combining advanced evolutionary algorithms with state-of-the-art machine learning techniques. The primary focus of the book is on two algorithms that replace traditional variation operators of evolutionary algorithms, by learning and sampling Bayesian networks: the Bayesian optimization algorithm (BOA) and the hierarchical BOA (hBOA). They provide a scalable solution to a broad class of problems. The book provides an overview of evolutionary algorithms that use probabilistic models to guide their search, motivates and describes BOA and hBOA in a way accessible to a wide audience, and presents numerous results confirming that they are revolutionary approaches to black-box optimization.This book provides a framework for the design of competent optimization techniques by combining advanced evolutionary algorithms with state-of-the-art machine learning techniques. The book focuses on two algorithms that replace traditional variation operators of evolutionary algorithms by learning and sampling Bayesian networks: the Bayesian optimization algorithm (BOA) and the hierarchical BOA (hBOA). BOA and hBOA are theoretically and empirically shown to provide robust and scalable solution for broad classes of nearly decomposable and hierarchical problems. A theoretical model is developed that estimates the scalability and adequate parameter settings for BOA and hBOA. The performance of BOA and hBOA is analyzed on a number of artificial problems of bounded difficulty designed to test BOA and hBOA on the boundary of their design envelope. The algorithms are also extensively tested on two interesting classes of real-world problems: MAXSAT and Ising spin glasses with periodic boundary conditions in two and three dimensions. Experimental results validate the theoretical model and confirm that BOA and hBOA provide robust and scalable solution for nearly decomposable and hierarchical problems with only little problem-specific information.