Consistency testing when unit-testing is infeasible

I use a number of unit and integrative testing in my C++ package for fluids simulation channelflow.

  1. Unit tests on component functions. E.g. the 3d solver uses a 1d Helmholtz equation boundary-value solver, so I test the 1d solver on a variety of test problems.
  2. In those unit tests, test convergence towards known analytic solutions and appropriately small residuals in artificially generated test problems. E.g. for an artificial 1d Helmholtz problem u'' - \lambda u = f plus boundary conditions, generate some random BCs and smooth functions f(x), solve for u(x), and then verify that the norm of the residual is small, \|u'' - \lambda u - f\|/\|f\| < \epsilon_{tol}.
  3. For integrative testing, concoct some test case that can be specified from a small initial data set and that reaches a solution that can be easily characterized. E.g. my code computes 3d, nonlinear equilibrium and periodic-orbit solutions of Navier-Stokes. So from the solutions I’ve computed with a trusted version of the code, I choose a few that are smooth and small in terms of discretization, and from them devise relatively simple initial guesses that converge towards those solutions when fed to the solver. (By simple I mean small enough to be specified in a small text file.) The integrative testing then runs these initial guesses through the solver and checks the norm of the residual and a few global (norm-like) properties of the solutions to verify that the solver reached the solution I intended.
3 Likes