Regression Tests
pySpline includes a set of built-in regression tests that are used to ensure that the code reproduces a consistent set of reproducible results. It is HIGHLY recommended that the regression tests are run after installation on a new system to ensure consistency of the code build.
To run the regression tests go to the following directory:
python/reg_tests
and run the automatic testing script:
python run_reg_tests.py
If successful, you should see the following line:
pyspline: Success!
if you don’t, then something went wrong in the test. By default an
xxdiff
window will appear that should quickly highlight where the
discrepency is.
After running the regression tests, there will be three important files to look at:
pyspline_reg.ref
pyspline_reg
pyspline_ref.orig
pyspline_reg.ref
is the reference file that contains the‘truth’ values that we should reproduce
pyspline_ref
is the generated output that is identical to the reference output except for @value lines where a discrepency has been detected. A graphical diff ofpyspline_reg
andpyspline_reg.ref
should quickly show where the discrepancies are.pyspline_ref.org
is the actual output of the regression run. Generally speaking there will be small discrepancies in the 15 or 16th digit which depends on the compiler. This will trigger a text-based diff program which would make it difficult to determine which values are incorrect in a file with several thousand lines.
Note
Only lines that contain the @value magic string are actually compared by the regression tester.
Regression Test Script Help
To see a list of options for the regression test script run:
python run_reg_test.py --help
which will show:
usage: run_reg_tests.py [-h] [-mode {train,compare}] [-diff_cmd DIFF_CMD]
[-nodiff]
optional arguments:
-h, --help show this help message and exit
-mode {train,compare}
Train generates reference file. Compare runs test
-diff_cmd DIFF_CMD Command to run for displaying diff. Default: xxdiff
-nodiff Suppress displaying the comparison if not successful