Residuals and Model Misspecification

Introduction

A fitted equation compresses chemical observations into a small set of parameters. Its residuals—the parts the equation does not explain—can show where that compression fails. A straight-line calibration may have a high correlation yet consistently overpredict middle standards. A kinetic model may miss early induction or late product inhibition. Residuals are not leftover rubbish; their pattern is evidence about omitted chemistry, measurement conditions or error assumptions.

Core explanation

Compute each residual as observed response − predicted response . Plot residuals against predictor values, fitted response and measurement order. If the model is adequate for its purpose, residuals should have no important systematic structure at the scale relevant to the decision. A U-shape versus concentration suggests curvature in the mean response. A funnel suggests changing variance. A steady trend versus time suggests drift or sample instability. Alternating signs may indicate a periodic process or run-order artifact.

Residuals can diagnose several different failures. If a reaction-rate model fits low concentrations but misses high ones in one direction, substrate inhibition, adsorption saturation or mass transport may be plausible. If a spectroscopic calibration fails only after a lamp change, instrument state is a stronger first explanation than new chemistry. A battery fade curve may have an early rapid decline followed by slower loss; one straight line averages two regimes. Diagnose the physical cause before adding polynomial terms solely to improve fit.

Replicate observations at the same predictor level help distinguish pure measurement variation from lack of fit . If repeated standards at 2 mg/L scatter only 0.01 absorbance but the fitted line misses their mean by 0.10, the systematic gap is unlikely to be ordinary repeat noise. If scatter is 0.15, the same gap is less decisive. NIST's residual-check guidance stresses that structured residuals can reveal unmodeled effects and motivate a better model.

Residual plots are not infallible. With very few points, a pattern may be hard to see. A flexible model can force tiny residuals while overfitting noise. An influential observation can distort the fit and make other residuals look patterned. Scale residuals by uncertainty when variance differs, and remember that repeated observations on the same sample are correlated. A residual near zero at a point does not prove that the underlying model is chemically correct; several mechanisms can produce the same mean curve.

Model revision should be constrained by independent knowledge. A physically motivated saturation equation is preferable to a high-degree polynomial when detector saturation is known. If measurement drift is the cause, fix or model the instrument process; do not call it a reaction law. If a factor was omitted, collect new experiments that vary it independently. Compare alternative models using residuals, parameter plausibility and predictions on independent data. Document revisions made after looking at the original data, because exploratory model search can overstate confidence.

Step-by-step reasoning

Fit the planned model and save predicted values. Plot raw data with the fit and residuals against predictor, fitted value and run order. Compare pattern size with replicate measurement noise and intended tolerance. List chemical and instrumental explanations for each pattern. Test the most plausible explanation with a controlled new measurement or revised design. Refit only when the change is justified, then validate on independent observations and update uncertainty claims.

Visual explanation

Imagine four residual panels sharing a horizontal zero line. Random scatter represents an adequate local model; a U-shaped panel indicates curvature; a widening cone indicates unequal variance; a sloping time-order panel indicates drift. Put a raw-data graph above each to show that the same apparently strong line can hide these different problems. Label the vertical residual units so readers can judge practical importance.

Real-world analogy

If a map predicts every morning journey to be five minutes too short and every evening journey ten minutes too long, its average error may be near zero but the pattern reveals missing traffic conditions. A chemical model with positive and negative residuals can likewise have a small overall average while failing systematically in important regions.

Real-world example

A laboratory calibrates a UV absorbance assay from 0 to 20 mg/L. Its line has an impressive R-squared, but residuals are positive at low and high concentration and negative in the middle. The lab checks for concentration-dependent chemical association and stray-light effects, then narrows the linear working range to 0–8 mg/L. Unknowns above 8 mg/L are diluted. This is more defensible than forcing the original line through all standards because the broad fit looked visually close.

Why?

Why can an overall average residual of zero be uninformative? Many fitting methods force residuals to balance around zero, so positive and negative systematic errors cancel in the average. Plotting them against concentration or time reveals whether they alternate randomly or follow a meaningful pattern. The decision may depend on one narrow region where bias does not cancel.

Common misconception

“Small residuals prove the mechanism.” A flexible equation can fit data without being physically correct. “A high R-squared means no lack of fit” ignores structured residuals. “Every outlying residual must be deleted” confuses unexpected chemistry with error. Investigate preparation and model assumptions before exclusion, and preserve the original record.

Worked example

A linear model predicts reaction rates of 1.0, 2.0, 3.0 and 4.0 mmol/min at increasing substrate concentration. Observed rates are 1.2, 2.1, 2.9 and 3.3, giving residuals +0.2, +0.1, −0.1 and −0.7. The increasingly negative high-end residual suggests the line overpredicts at high concentration. If replicate rate uncertainty is about 0.1, the final −0.7 is substantial. Substrate saturation or inhibition is plausible, but check mixing and detector range first. A nonlinear kinetic model and new high-concentration measurements can distinguish chemical curvature from instrument failure.

Quick check

1. What does a residual trend with measurement order suggest when concentration order was randomized? Answer: Instrument drift, sample aging or another time-dependent process may remain. Check controls and timestamps rather than attributing the pattern immediately to concentration chemistry.

Exam focus

Calculate residual sign correctly and match a pattern to a possible issue. Distinguish mean-model curvature from variance change and time dependence. Compare residual size with experimental noise and application tolerance. Suggest a controlled follow-up rather than simply adding polynomial terms. State that residual adequacy supports use of a model in its domain but does not prove a mechanism.

Advanced insight

Residual checks are model-dependent: a log transformation changes which deviations appear large, and weighting changes the appropriate residual scale. Cross-validation tests predictive behavior on held-out data, while residuals test structure in fitted data. Both are useful, but neither substitutes for a chemically meaningful measurement design. A systematic residual may reveal a new mechanism if instrument artifacts are ruled out.

Summary

Residuals expose what a fitted chemical model leaves unexplained. Curvature, funnels and run-order trends suggest different modeling or measurement problems. Compare patterns with replicate noise, investigate physical causes and validate revised models independently. A summary fit statistic cannot replace this diagnostic reasoning.

Practice questions

1. A residual plot widens with concentration but remains centered on zero. What is the likely statistical issue? Answer: Error variance increases with concentration, so constant-variance assumptions may fail. Estimate variance with replicates and consider a justified weighted or variance model.

2. A kinetic line misses the high-concentration points in one direction. Name one chemical and one instrumental possibility. Answer: Chemical saturation or substrate inhibition could bend the true rate law; detector saturation or poor mixing could also cause the pattern. Controlled tests should distinguish them.

3. Why should a researcher keep records of the first planned model after revising it based on residuals? Answer: The revision was informed by the observed data. Preserving the plan and exploration history prevents the final model from being presented as if it had been predicted independently.