Enable single-precision floating point unit tests for CI #1698
Conversation
Codecov Report
@@ Coverage Diff @@
## master #1698 +/- ##
==========================================
+ Coverage 73.24% 73.32% +0.08%
==========================================
Files 13 13
Lines 4518 4525 +7
==========================================
+ Hits 3309 3318 +9
+ Misses 1209 1207 -2
|
| @@ -16,7 +16,7 @@ | |||
| _FD_STEP = 1e-4 | |||
|
|
|||
| # The tolerance for the adjoint and finite difference gradient comparison | |||
| _TOL = 2e-2 | |||
| _TOL = 0.1 if mp.is_single_precision() else 0.02 | |||
There was a problem hiding this comment.
It makes sense that the error in finite-difference approximations would depend sensitively on the precision. If you are computing [f(x+δ)-f(x)]/δ, the optimal choice of δ depends sensitively on the precision, and if δ is too small compared to the precision you can see a large "catastrophic cancellation" error.
| @@ -53,7 +52,8 @@ def test_3rd_harm_1d(self): | |||
|
|
|||
| harmonics = [self.k, self.amp, mp.get_fluxes(self.trans1)[0], mp.get_fluxes(self.trans3)[0]] | |||
|
|
|||
| np.testing.assert_allclose(expected_harmonics, harmonics) | |||
| tol = 3e-5 if mp.is_single_precision() else 1e-7 | |||
There was a problem hiding this comment.
This also makes sense — there is little hope of getting 7 significant digits in single precision.
|
We will also want to change the default tolerance of the CW solver — it currently defaults to |
|
|
||
| class TestMultiLevelAtom(unittest.TestCase): | ||
|
|
||
| @unittest.skipIf(mp.is_single_precision(), "double-precision floating point specific test") |
There was a problem hiding this comment.
Would be good to open an issue about this and ping Alex Cerjan. I'm not sure why a Maxwell–Bloch simulation would be super sensitive to floating-point errors.
There was a problem hiding this comment.
Maybe we are entering some sort of chaotic regime, in which the precise details (phase etcetera) are extremely sensitive to initial conditions and small calculational errors.
| @@ -977,7 +977,9 @@ int main(int argc, char **argv) { | |||
| #endif | |||
|
|
|||
| if (!nonlinear_ex(vol1d(1.0, 30.0), one)) meep::abort("error in 1D nonlinear vacuum\n"); | |||
| if (!nonlinear_ex(vol3d(1.0, 1.2, 0.8, 10.0), one)) meep::abort("error in 3D nonlinear vacuum\n"); | |||
| if (sizeof(realnum) == sizeof(double)) { | |||
* lower tolerance of various tests when compiled with single precision * add more tests * fix more tests * fix for test_get_array_metadata.py * only compile with single precision for Python 3.9 with MPI * cleanups and tweaks * fix harmonics.cpp
* lower tolerance of various tests when compiled with single precision * add more tests * fix more tests * fix for test_get_array_metadata.py * only compile with single precision for Python 3.9 with MPI * cleanups and tweaks * fix harmonics.cpp
Enables single-precision floating point unit tests as part of the CI only for Python 3.9 with MPI. I have tested that all tests (C++ and Python) are passing for the entire CI matrix: [serial, MPI] x [Python 3.6, Python 3.9]. The changes in this PR involve mostly reducing the relative tolerance when checking against hard-coded values for a handful of the unit tests. In some of these modified tests involving a comparison of two NumPy arrays, the comparator function
numpy.testing.assert_allclosewas replaced withassertVectorsClosedue to #1575. (We should probably replace all uses ofnumpy.testing.assert_allclosebut that can be done in a separate PR.)Three of the unit tests are skipped for single precision:
test_multilevel_atom.py: the change in values was larger than just a few decimal digits.test_eigfreq.py: the CW solver fails to converge.test_epsilon_warningoftest_simulation.py: the dispersive material Molybdenum (Mo) from the materials library is causing the fields to blow up.