Companion Website

Computational Physics

From Equations to Simulation

Chapter 13

Optimization and Data Fitting

These practical programs accompany Chapter 13 of Computational Physics: From Equations to Simulation. They cover local and heuristic optimization, least-squares fitting, model comparison, and bootstrap uncertainty. Each example is a standalone source file with editable settings and reusable numerical routines.

Choose a language below. The examples use the same physical or mathematical tasks across languages. Fixed seeds support reproducible work within each implementation; random sequences and noisy data can differ across languages and compilers. The source files include a commented alternative for initializing from system entropy or the clock. A heuristic search reports the best result found, without certifying global optimality.

Chapter 13 exercises & projects Chapter 13 resources Chapter 13 updates & errata Chapter 13 overview

On a narrow screen, scroll horizontally to view all language columns.

Practical programs in four languages
Program	Fortran	C++	MATLAB	Python
Golden-section searchLocalizes the minimum of a unimodal scalar function on a finite interval without derivatives.	Source	Source	Source	Source
Gradient descentUnconstrained local minimization with an analytic gradient and Armijo backtracking.	Source	Source	Source	Source
Simulated annealingBounded one-dimensional search with symmetric proposals and a cooling schedule; retains the best point found.	Source	Source	Source	Source
Particle swarm optimizationBounded multidimensional search with personal and shared best positions, velocity limits, and reflected boundary overshoots.	Source	Source	Source	Source
Ant colony optimizationSearches for a short symmetric traveling-salesman tour, including the return to the starting city.	Source	Source	Source	Source
Differential evolutionBounded current-to-best/1/bin search with frozen generations, greedy selection, and uniform boundary repair.	Source	Source	Source	Source
Genetic algorithmReal-valued chromosomes, tournament selection, uniform crossover, Gaussian mutation, and one elite individual.	Source	Source	Source	Source
Straight-line least squaresFits an intercept and slope with optional known measurement uncertainties; reports their covariance and standard errors.	Source	Source	Source	Source
Nonlinear exponential fitFits A exp(−t/tau) directly to the observations using Nelder–Mead in A and log(tau); reports point estimates.	Source	Source	Source	Source
Polynomial model comparisonFits centered and scaled polynomial models and compares training error with independent validation error.	Source	Source	Source	Source
Pairs-bootstrap uncertaintyResamples complete observation pairs to estimate standard errors and covariance of straight-line fit parameters.	Source	Source	Source	Source

Running the programs

Fortran: console programs in .f95 files. Compile each file separately with a compatible Fortran compiler. There are no plotting options.
C++: C++11 or newer, with the standard library only. Compile each .cpp file separately. Output is printed to the console.
MATLAB: run the script in MATLAB with support for local functions in scripts. The examples need no additional toolbox. Set make_plot = false to suppress the optional plots.
Python: Python 3 and NumPy. The nonlinear fit also uses SciPy. Optional plots require Matplotlib; run with --no-plot for console output only.

For example, after opening the relevant language folder:

gfortran -O2 ch13_golden_section.f95 -o ch13_golden_section
c++ -std=c++11 -O2 ch13_golden_section.cpp -o ch13_golden_section
python3 ch13_golden_section.py --no-plot

Fortran and C++ commands build an executable; run that executable after compilation, for example ./ch13_golden_section on macOS or Linux. Each source file has its own main program, so compile one example at a time.

Adapting an example

Start by running the supplied example, then replace its objective, gradient, bounds, or measurements. Use tolerances appropriate to the variable scales, inspect the stopping reason, and compare heuristic searches over several seeds and evaluation budgets. Plot evaluations are separate from search evaluation counts.

The linear fit assumes independent errors in the measured dependent variable and negligible errors in the independent variable. Supplied standard deviations are treated as known absolute uncertainties; without them, the common variance is estimated from the residuals. The nonlinear exponential examples report point estimates, not parameter uncertainties. Bootstrap standard errors are not confidence intervals. The Chapter 13 resources explain these distinctions and provide benchmark checks; the additional exercises and projects offer further applications.

← Back to Chapter 13