Part 7 · Scientific Computing & Data Science · Module 50
SciPy for Science & Engineering
NumPy gives you arrays; SciPy gives you decades of battle-tested numerical algorithms built on them: optimizers, equation solvers, integrators, filters, spatial indexes, and statistics. Knowing which tool to reach for, and how to check its answer, turns a week of hand-rolled numerics into ten lines.
- Navigate the SciPy subpackages and pick the right one for a problem
- Minimize functions, find roots with
brentq, fit models withcurve_fit, and solve linear programs withlinprog - Integrate functions with
quadand simulate differential equations withsolve_ivp - Interpolate data with splines, filter noisy signals, find peaks, and read a frequency spectrum with
scipy.fft - Answer nearest-neighbor and geometry questions with
KDTree, distance matrices, andConvexHull - Recognize floating-point error, ill-conditioning, and tolerance settings, and verify numerical results
The SciPy map
SciPy is a collection of subpackages, each wrapping well-studied algorithms (many in Fortran and C libraries such as
LAPACK, QUADPACK, and HiGHS). You import the subpackage you need: from scipy import optimize. You
already used scipy.linalg and scipy.sparse in M49.
A habit for the whole module: every numerical answer is an approximation, so check it. Plug the root back into the function, compare a fitted curve with the data, integrate a case you know by hand, and read the solver's status flags.
scipy.optimize: minimize, find roots, fit, and plan
Minimization
"Find the inputs that make this function as small as possible" covers a huge range of problems: the cheapest
design, the best-fitting model (minimum error), the lowest-energy molecule. optimize.minimize(f, x0)
starts at a guess x0 and walks downhill. Think of standing on a foggy hillside: you can feel the slope
under your feet (the gradient) and step downhill until the ground is flat. Methods like BFGS also learn the
curvature as they go, so they take smarter steps. Try it with the visualizer: the step size (learning rate) decides
whether you crawl, converge, or overshoot.
Notice the second BFGS run: without an exact gradient it estimates slopes numerically, lands almost on (1, 1),
and still reports success: False with "precision loss". The answer is fine here, but the lesson is
general: read res.success and res.message, and judge the result yourself.
Root finding with brentq
A root of f is an x with f(x) = 0: the break-even price,
the time a ball hits the ground, the interest rate that makes a loan's payments add up. If f is
continuous, f(a) < 0, and f(b) > 0, the graph must cross zero somewhere between
(the intermediate value theorem). Bisection halves the interval repeatedly, keeping the half where
the sign changes, just like binary search in M30. brentq combines
bisection's guarantee with faster interpolation steps. You must give it a bracket [a, b] with
opposite signs.
Fitting models with curve_fit
M49's least squares fits models that are linear in their parameters. Many real models aren't:
exponential decay A·e^(−kt), logistic growth, a damped spring. optimize.curve_fit(model, x, y,
p0) adjusts the parameters to minimize the sum of squared errors, starting from your guess p0
(a sensible guess matters for non-linear fits). It returns the best parameters and their covariance
matrix; the square roots of its diagonal are the parameters' standard errors, a measure of how precisely
the data pins each one down.
Linear programming with linprog
A linear program optimizes a linear objective subject to linear inequality constraints. A small workshop makes tables (profit $70, 3 hours of labour, 4 m of wood) and chairs (profit $50, 2 hours, 1.5 m). This week it has 120 hours and 140 m of wood, and can sell at most 30 chairs. How many of each maximizes profit?
linprog minimizes c @ x subject to A_ub @ x ≤ b_ub (and optional
equalities and bounds). To maximize profit, minimize its negative.
Remember: minimize for smooth objectives (watch for local minima),
brentq for roots inside a sign-changing bracket, curve_fit for non-linear models (check
residuals and standard errors), linprog for linear objectives with linear constraints.
scipy.integrate: areas and differential equations
quad: the area under a curve
An integral ∫ₐᵇ f(x) dx is the area under the graph of f between
a and b: total distance from a speed curve, total energy from a power curve, a probability
from a density. You can approximate it by slicing the area into thin rectangles and adding them up.
integrate.quad does this adaptively (more slices where the curve bends) and returns the value
and an estimate of its error.
solve_ivp: simulate how things change
Many laws of nature describe rates of change. A differential equation says how fast a
quantity changes given its current state. A falling object with air drag: its velocity v changes at rate
dv/dt = g − (c/m)·v² (gravity pulls, drag pushes back harder the faster it goes). Population growth with
a limited food supply: dP/dt = r·P·(1 − P/K). Given the starting state (the initial value), a
solver takes many small time steps, each time using the rate to predict the next state. solve_ivp
chooses the step sizes automatically to stay within your tolerances (rtol, atol).
The state can be a vector. For the falling object, the state is [height, velocity], and the rate
function returns [dh/dt, dv/dt]. An event function lets the solver stop exactly when
something happens, like height reaching zero.
Remember: quad returns (value, error_estimate). solve_ivp(fun,
(t0, t1), y0) with fun(t, y) returning rates; use t_eval for output times and
events to stop on a condition.
scipy.interpolate: filling in between measurements
You measured temperature every 3 hours but need it at 10:30. Interpolation builds a function that
passes exactly through your data points and estimates values between them. The simplest is straight lines between
neighbours (np.interp). Splines join smooth cubic pieces so the curve and its slope have
no kinks, which looks and behaves more like physical data.
np.interp(x_new, x, y): fast piecewise-linear, 1-D.make_interp_spline(x, y, k=3): an interpolating B-spline of degreek, callable, with derivatives available.CubicSplineis similar, with boundary-condition options.interp1dstill exists but SciPy now documents it as legacy; prefer the two above in new code.
Remember: interpolants pass through every point. Linear for speed, splines for smoothness; don't extrapolate.
scipy.signal and scipy.fft: cleaning and reading signals
Frequencies and the FFT
Any signal sampled over time (sound, vibration, heartbeat, electricity use) can be described as a sum of sine
waves of different frequencies (cycles per second, Hz). The Fourier transform
measures how much of each frequency is present; the FFT (fast Fourier transform) computes it in
O(n log n). The result, the spectrum, shows peaks at the frequencies that make up the signal. Use
scipy.fft.rfft for real-valued signals and rfftfreq(n, d=1/fs) for the matching
frequencies, where fs is the sampling rate.
Filtering
A low-pass filter keeps slow changes and removes fast jitter (noise); a high-pass filter does the
opposite. signal.butter(order, cutoff, fs=fs, output="sos") designs a smooth Butterworth filter, and
signal.sosfiltfilt applies it forwards and backwards so the output isn't shifted in time.
signal.find_peaks locates local maxima, with options like height, distance
(minimum spacing), and prominence (how much a peak stands out) to ignore noise bumps.
Remember: the FFT turns time into frequency; filter with butter +
sosfiltfilt; tune find_peaks with distance and prominence.
scipy.spatial: nearest neighbours, distances, hulls
"Which store is closest to each customer?" Checking every store for every customer costs O(n·m). A
KD-tree splits space into boxes, alternating between x and y (and further dimensions), like a binary
search tree for points (M24). A query only visits the few boxes near the target, about
O(log n) per query in low dimensions. Build it once with KDTree(points), then call
tree.query(targets, k) for the k nearest, or tree.query_ball_point(x, r) for
everything within radius r.
For all pairwise distances between two smallish sets, use scipy.spatial.distance.cdist(A, B,
metric=...) (many metrics: "euclidean", "cityblock" (L1), "cosine",
…). The convex hull is the smallest convex polygon containing all the points: imagine a rubber band
snapped around nails. ConvexHull(points) gives its vertices, area, and perimeter.
Remember: build a KDTree once, query many times; cdist for full distance
matrices; ConvexHull for outer boundaries (2-D volume = area).
scipy.stats: a preview
scipy.stats has about a hundred probability distributions, each with the same methods:
pdf/pmf (density or probability), cdf (probability of being ≤ x),
ppf (the inverse: which x has this cumulative probability), rvs (random samples), and
fit. It also has descriptive statistics and hypothesis tests. M54
covers them properly; here is a taste.
Numerical pitfalls: floating point, conditioning, tolerance
Every result in this module is computed with 64-bit floating-point numbers, which have about 16 significant
decimal digits (the IEEE-754 format; see the bits visualizer below). Three ideas explain almost every
numerical surprise:
- Rounding error: most decimals, like 0.1, can't be stored exactly. Each operation rounds, and errors accumulate. Worst of all is catastrophic cancellation: subtracting two nearly equal numbers wipes out the leading digits and leaves mostly noise.
- Conditioning: some problems amplify small input errors no matter how good the algorithm (M49's condition number). An ill-conditioned problem needs reformulating, not a better solver.
- Tolerance: iterative algorithms (roots, optimizers, ODE solvers) stop when a change is below a
tolerance. Tolerances too loose give wrong answers; too tight waste time or fail to converge. Compare floats with
np.isclose(a, b, rtol=..., atol=...), never==, and remember thatatolmatters near zero, where relative tolerance is meaningless.
Remember: expect rounding, avoid subtracting near-equal numbers, respect conditioning, and treat tolerances as part of the answer.
Practice
Use SciPy. Tests compare with tolerances.
A product's profit at price p is profit(p) = (p - unit_cost) * demand(p) - fixed_cost
with demand(p) = 1000 * exp(-p / 40). Write break_even_prices(unit_cost, fixed_cost)
returning a sorted tuple of the two prices where profit is exactly 0. First locate the price of maximum profit
with optimize.minimize_scalar on -profit over bounds (unit_cost, 500); then
use optimize.brentq on each side ([unit_cost, p_best] and [p_best, 500]).
Raise ValueError if even the best price makes a loss.
Hint
res = optimize.minimize_scalar(lambda p: -profit(p),
bounds=(unit_cost, 500), method="bounded"). If profit(res.x) <= 0, raise. Otherwise the signs
flip on both sides of res.x.
Show solution
import numpy as np
from scipy import optimize
def break_even_prices(unit_cost, fixed_cost):
def profit(p):
return (p - unit_cost) * 1000 * np.exp(-p / 40) - fixed_cost
res = optimize.minimize_scalar(lambda p: -profit(p), bounds=(unit_cost, 500), method="bounded")
best = res.x
if profit(best) <= 0:
raise ValueError("no price is profitable")
lo = optimize.brentq(profit, unit_cost, best)
hi = optimize.brentq(profit, best, 500)
return (lo, hi)
Write logistic(t, K, r, t0) returning K / (1 + exp(-r * (t - t0))), and
fit_logistic(t, y) returning (K, r, t0) fitted with optimize.curve_fit.
Use a data-driven initial guess: K ≈ max(y), r ≈ 1, t0 ≈ the time where
y is closest to half its maximum.
Hint
t0_guess = t[np.argmin(np.abs(y - y.max() / 2))], then
popt, _ = optimize.curve_fit(logistic, t, y, p0=[y.max(), 1.0, t0_guess]).
Show solution
import numpy as np
from scipy import optimize
def logistic(t, K, r, t0):
return K / (1 + np.exp(-r * (t - t0)))
def fit_logistic(t, y):
t, y = np.asarray(t, dtype=float), np.asarray(y, dtype=float)
t0_guess = t[np.argmin(np.abs(y - y.max() / 2))]
popt, _ = optimize.curve_fit(logistic, t, y, p0=[y.max(), 1.0, t0_guess])
K, r, t0 = popt
return float(K), float(r), float(t0)
Write cheapest_diet(cost, nutrients, minimums). cost[j] is the price per serving of food
j; nutrients[i, j] is the amount of nutrient i per serving of food
j; minimums[i] is the daily minimum of nutrient i. Return
(servings, total_cost) minimizing cost with all servings ≥ 0. Raise ValueError if the
problem is infeasible. (Remember linprog wants ≤ constraints: multiply
≥ rows by −1.)
Hint
linprog(c=cost, A_ub=-nutrients, b_ub=-minimums, bounds=[(0,
None)] * len(cost)). Check res.success.
Show solution
import numpy as np
from scipy.optimize import linprog
def cheapest_diet(cost, nutrients, minimums):
cost = np.asarray(cost, dtype=float)
res = linprog(c=cost, A_ub=-np.asarray(nutrients, dtype=float), b_ub=-np.asarray(minimums, dtype=float),
bounds=[(0, None)] * cost.size, method="highs")
if not res.success:
raise ValueError(f"no feasible diet: {res.message}")
return res.x, float(res.fun)
Write fall_time(height, mass, drag, g=9.81) that simulates a dropped object with
dv/dt = g - (drag / mass) * v * |v| and dh/dt = -v, starting at rest, and returns the
time when h reaches 0, using a terminal event. With drag=0 the answer must match
sqrt(2 h / g).
Hint
Define def ground(t, s): return s[0], set
ground.terminal = True and ground.direction = -1, integrate over a long span such as
(0, 10_000) with rtol=1e-9, and return sol.t_events[0][0].
Show solution
import numpy as np
from scipy.integrate import solve_ivp
def fall_time(height, mass, drag, g=9.81):
def rates(t, s):
h, v = s
return [-v, g - (drag / mass) * v * abs(v)]
def ground(t, s):
return s[0]
ground.terminal = True
ground.direction = -1
sol = solve_ivp(rates, (0, 10_000), [height, 0.0], events=ground, rtol=1e-9, atol=1e-9)
return float(sol.t_events[0][0])
Write assign_customers(stores, customers, max_distance). Return an integer array with the index of
the nearest store for each customer, or -1 if the nearest store is farther than
max_distance. Use scipy.spatial.KDTree. Then write store_loads(assignment,
n_stores) returning how many customers each store serves (ignore -1).
Hint
dist, idx = KDTree(stores).query(customers, k=1), then
np.where(dist <= max_distance, idx, -1). For loads, np.bincount(a[a >= 0],
minlength=n_stores).
Show solution
import numpy as np
from scipy.spatial import KDTree
def assign_customers(stores, customers, max_distance):
dist, idx = KDTree(stores).query(customers, k=1)
return np.where(dist <= max_distance, idx, -1).astype(int)
def store_loads(assignment, n_stores):
assignment = np.asarray(assignment)
return np.bincount(assignment[assignment >= 0], minlength=n_stores)
Mini-project: model a cooling cup of coffee
Newton's law of cooling says an object's temperature T approaches the room temperature
T_room at a rate proportional to the difference: dT/dt = −k (T − T_room). Its exact
solution is T(t) = T_room + (T₀ − T_room)·e^(−kt). Here is the whole workflow in miniature: fit
k from noisy thermometer readings with curve_fit, then simulate with solve_ivp
and answer a practical question ("when is it drinkable at 60 °C?") with an event.
Quiz
Recap
- SciPy is a toolbox of subpackages on top of NumPy: optimize, integrate, interpolate, linalg, sparse, signal, fft, spatial, stats, and more.
minimizewalks downhill (local);brentqneeds a sign-change bracket;curve_fitfits non-linear models with uncertainties;linprogsolves linear programs at corners.quadintegrates with an error estimate;solve_ivpsimulates ODEs, with events to stop on conditions.- Splines interpolate smoothly between points; don't extrapolate or interpolate noise.
- FFT reveals frequencies; Butterworth +
sosfiltfiltremoves noise;find_peaksfinds events. KDTreefor nearest neighbours,cdistfor distance matrices,ConvexHullfor boundaries.- Floating point rounds, problems can be ill-conditioned, and tolerances are part of every answer: verify.
Going further
- SciPy user guide (one chapter per subpackage)
- Optimization tutorial and
solve_ivpreference - Interpolation tutorial (including the legacy status of
interp1d) - Python tutorial: floating-point arithmetic, issues and limitations
- Next: M51 · pandas: Series & DataFrames; for statistics, M54