What Hartree–Fock Misses

Introduction

Hartree–Fock includes the electron–electron Coulomb interaction and the exchange consequences of antisymmetry, yet it does not let electrons respond fully to one another's instantaneous positions. That omitted behavior is called electron correlation beyond the mean field. The phrase covers more than one physical problem. Fine adjustments around a good single determinant are often called dynamical correlation; situations where several configurations are comparably important are often called static or near-degeneracy correlation. Separating them helps choose the next computational method and prevents the mistaken claim that a more expensive correction will always rescue a poor reference.

Core explanation

For a defined fixed-nuclei nonrelativistic electronic Hamiltonian and an appropriate complete-basis Hartree–Fock reference, correlation energy is commonly written E corr = E exact − E HF. Because a variational Hartree–Fock ground-state energy is an upper bound, E corr is zero or negative under that matched comparison. IUPAC's correlation-energy entry describes the difference between Hartree–Fock and exact nonrelativistic energy. The definition requires care when actual finite-basis numbers are used: basis-set error and correlation error are separate, and comparing an exact energy for one geometry with Hartree–Fock for another does not measure E corr.

The electron–electron repulsion in Hartree–Fock is represented through average Coulomb fields and exchange terms from the antisymmetric determinant. Exchange is therefore already present in Hartree–Fock; it should not be counted as wholly missing correlation. For electrons of opposite spin in different orbitals, their instantaneous separation can still matter strongly even though an average density is known. If one electron happens to be close to a region, another can rearrange to avoid it more than the fixed mean field permits. Dynamical correlation refers broadly to such additional fluctuations around a useful single-reference description. It often affects binding energies and barrier heights because correlation corrections differ between reactant, transition state and product.

London dispersion is an important manifestation of long-range correlated fluctuations. Two neutral, nonpolar fragments can attract when instantaneous charge fluctuations in one are correlated with a response in the other. Ordinary Hartree–Fock does not capture the correct long-range dispersion attraction. Primary analysis of intermolecular interactions identifies attractive dispersion as a correlation effect, while a review of dispersion-corrected mean-field methods explains why bare Hartree–Fock lacks that long-range contribution. This matters for molecular crystals, biomolecular contacts, stacked aromatic molecules and adsorption. It does not mean all intermolecular attraction is dispersion; electrostatics, induction and hydrogen bonding can also contribute.

Static or near-degeneracy correlation appears when no single determinant dominates. Consider a covalent bond stretched toward dissociation. Near equilibrium, both electrons may be modeled in a bonding orbital with opposite spins. At large separation, a correct singlet description must avoid falsely placing both electrons on one fragment, and multiple occupancy patterns become comparable. A restricted single-determinant picture can give qualitatively wrong dissociation. Transition-metal complexes with nearly degenerate d configurations, diradicals and some excited states present related challenges. A perturbative correction built around one poor determinant may become unstable or misleading because the assumed separation between the reference and alternatives is too small.

How large an energy error matters depends on the chemical question. A difference of 1 kJ mol⁻¹ is small compared with many total electronic energies, but at room temperature it can noticeably change a conformer population or product ratio. A method could recover most total correlation energy while still predicting a reaction energy poorly if correlation corrections fail to cancel between compared states. Conversely, a low-cost method might predict a specific trend well because its errors cancel. A benchmark should target the observable of interest, such as an interaction energy, spin splitting or barrier, rather than only a large absolute energy.

Methods beyond Hartree–Fock address correlation in different ways. Perturbation theory, configuration interaction and coupled cluster can improve a good single-reference state; active-space or multireference methods explicitly include several important configurations. Density-functional approximations use a different framework that models exchange and correlation through functionals of electron density. No category is uniformly best: cost, basis size, electronic character and validation determine suitability. The next pages introduce those options in detail.

Step-by-step reasoning

1. Define the Hamiltonian, geometry, basis convention and Hartree–Fock reference for a comparison. 2. Ask whether one determinant dominates or several occupations are nearly degenerate. 3. Identify likely dynamical effects, including local electron avoidance and long-range dispersion. 4. Choose a correlation treatment appropriate to the electronic character and system size. 5. Compare differences relevant to the chemistry, not just a lowered absolute total energy. 6. Benchmark against independent evidence or a higher-level calculation and report basis and method uncertainty separately.

Visual explanation

Draw two panels. In the first, two electrons move in a shared region: Hartree–Fock shows average clouds, while a correlated sketch shows one electron's likely position changing the conditional location of the other. In the second, draw a bond-energy curve from short to long separation; a single restricted determinant rises toward an incorrect separated-atom limit, while a multideterminant curve approaches the correct fragments. Add a third small diagram of two neutral fragments with synchronized fluctuating dipoles to illustrate dispersion.

Real-world analogy

A city plan based only on each person's average location misses how people adjust their movement when others are nearby. A mean-field electron picture similarly misses conditional motion beyond average density. The analogy is limited: electrons are quantum particles, exchange is built into antisymmetry and dispersion is a correlated electronic fluctuation rather than ordinary crowd avoidance.

Real-world example

A chemist compares two stacked aromatic conformers. Bare Hartree–Fock predicts weak or unfavorable binding because it misses much of the long-range dispersion attraction. A suitable correlated or dispersion-corrected method changes the interaction-energy balance. The chemist still checks basis-set superposition error, solvent environment and conformational sampling before comparing with an experimental association constant. The finding does not imply Hartree–Fock is useless; its orbitals and exchange treatment can remain a useful reference for a method that adds missing correlation.

Why?

Why can a Hartree–Fock calculation include electron–electron repulsion yet omit dispersion? Average Coulomb fields and exchange describe important parts of the interaction, but dispersion requires correlated fluctuations of electrons in separated fragments. A fixed independent-particle mean field cannot produce the full coupled fluctuation response. Including the repulsion operator in the Hamiltonian is not the same as solving its many-electron consequences exactly.

Common misconception

“Correlation energy is always the same correction per electron.” It depends on electronic structure and can change markedly across states or geometries. Another mistake is saying exchange is absent from Hartree–Fock; exchange is central to its determinant. A third is treating a large negative correlation correction as proof that a particular reaction energy is accurate. What matters for a reaction is the difference in correlation contributions between states. A fourth is assuming perturbation theory is reliable when a bond-breaking state has strong near-degeneracy.

Worked example

Suppose two hypothetical states A and B have Hartree–Fock energies −100.000 and −99.990 hartree, so HF predicts B above A by 0.010 hartree, about 26.3 kJ mol⁻¹. A correlated method gives corrections of −0.200 hartree for A and −0.215 hartree for B, yielding −100.200 and −100.205 hartree. The correlated prediction now places B below A by 0.005 hartree, about 13.1 kJ mol⁻¹. Both absolute corrections are large and negative, but the 0.015-hartree difference between corrections reverses the state ordering. These invented values illustrate why cancellation cannot be assumed. The result still needs basis convergence and validation of whether a single-reference correction is suitable.

Quick check

1. Is exchange wholly missing from a Hartree–Fock calculation? Answer: No. Exchange follows from the antisymmetric Slater determinant and is included in Hartree–Fock. 2. Why can two large correlation corrections have a small effect on a reaction energy? Answer: If the corrections to reactant and product are nearly equal, they largely cancel in the energy difference.

Exam focus

Define E corr with a matched exact and Hartree–Fock comparison, and state its usual negative sign for a variational ground state. Distinguish exchange from missing dynamical and static correlation. Use dispersion and stretched-bond dissociation as separate examples of failure modes. Explain why reaction energies depend on differences in errors. Mention basis incompleteness as a distinct source of error and choose multireference treatment when a single determinant is qualitatively inadequate.

Advanced insight

Correlation is partly method-dependent language in practical calculations. A particular density-functional approximation mixes modeled exchange and correlation terms that do not map one-to-one onto Hartree–Fock plus a correction. Coupled-cluster and perturbation methods rely on the quality of their reference state, while active-space methods shift attention to selecting orbitals that carry near-degeneracy. A good research workflow tests whether the electronic state changes character along the reaction coordinate. A low barrier computed at one geometry may be meaningless if the method misses a state crossing or symmetry change elsewhere.

Summary

Hartree–Fock treats average Coulomb repulsion and fermionic exchange, but misses electron correlation beyond one optimized determinant. Dynamical correlation includes local and long-range fluctuations, including dispersion; static correlation signals that several configurations matter. Correlation energy is defined relative to a matched exact nonrelativistic and Hartree–Fock comparison. Chemical predictions depend on how correlation errors differ among the states being compared, not only on how much a total energy is lowered.

Practice questions

1. Under the matched variational ground-state definition, can E exact − E HF be positive? Answer: No. E HF is an upper bound to E exact for the same Hamiltonian, so the difference is zero or negative. 2. Which missing effect is especially relevant for binding between two neutral nonpolar fragments at long range? Answer: London dispersion, a correlated fluctuation effect absent from bare Hartree–Fock at long range. 3. Why is a stretched covalent bond often a poor single-determinant problem? Answer: Several electronic occupations become comparably important, so one determinant cannot describe the separated fragments correctly. 4. If reactant and product each receive a −0.1-hartree correlation correction, what is the change in their energy difference from those corrections alone? Answer: Zero, because equal corrections cancel when product energy is subtracted from reactant energy.