An Introduction to Applied Optimization

Applied optimization belongings to applied mathematics is not a hazard mathematical subject but it is applied widely and effectively in many realistic applications, especially, in computer science as well as artificial intelligence (AI) that is the main engine of the current hot topic “large language model” where optimizing parameters of deep neural network by association of applied optimization and backpropagation algorithm is the key solution. Moreover, applied optimization can be applied recurrently into learning hyper-parameters of deep neural network. Although the fundamental of applied optimization is theoretical convex optimization and theoretical global optimization with full of strict mathematical definitions and proofs, applied optimization with support of heuristic assumptions proves itself by its effectiveness of searching global optimizer. Particularly, global optimization can be classified into three groups such as analytic methods (purely mathematical methods), probabilistic methods, and heuristic methods so that the book focuses on probabilistic methods and heuristic methods as main targets of applied optimization.