The Tests Required

Before beginning any migration project it is important to consider how to test it.

• How will individual routines be tested - Unit Testing?

• How will the entire system be tested - System Testing?

• How complete are the tests - Coverage Analysis?

Unit Tests - Record-replay - Capturing and Replaying I/O Transactions and Selected Interfaces

INSERT RECORD REPLAY	Statements are inserted into the code to capture all terminal and file input and the data returned from selected sub-programs to file. The data in the file can then be replayed into the program, and the user can select which I/O unit numbers and which sub-program interfaces are to be replayed.
DEFAULT LENGTH OF RECORD REPLAY STRINGS	Sets the length of strings captured by the record/replay mechanism when the length cannot be inferred from context.
NUMBER RECORD REPLAY CALLS	All record/replay call sites are identified by a unique number. This command optionally specifies the starting number for each file. This prevents files from changing simply because the record/replay numbers have changed, and therefore shortens the build time.

These commands were used to capture the I/O for a real-time controller of a chemical plant over a six hour run. Several hundred key strokes were captured, with interface data from clocks, thermometers, flow meters and colorimeters. The data could be replayed into the code to provide a test frame which ran from a single command and in only a few minutes.

System Tests - Run-time Trace - Comparing Runs on Different Hosts, Eliminating Numerical Drift

INSERT RUN-TIME TRACE

Statements are added to the code to trace program execution and to capture every scalar numeric value (LOGICAL, INTEGER, REAL or COMPLEX) to a trace file as it is computed. This may be used to investigate error conditions, and to validate the performance of the code under different compilers. The trace file may be used to eliminate numerical drift, and therefore to expose coding errors and compiler bugs (see "The Removal of Numerical Drift from Scientific Models", Collins J, Farrimond B, Flower D, Anderson M and Gill D. 2013, arXiv:1304.3260 [cs.SE]).

RESOLVE AMBIGUOUS FLOW

Fortran statements which contain two or more function evaluations may be ambiguous. The order of evaluation of the functions is not defined by the Fortran standard, and different compilers make different choices of order. This may lead to inconsistent results, and causes spurious error reports from the run-time trace facility. This command modifies the code so that the order of evaluation is uniquely defined.

Coverage Analysis - Verifying the Completeness of the Tests

INSERT COVERAGE	Test coverage shows the proportion of a code which is actually exercised by a program test suite.
EXCEPTION	Optionally, this command is written in the code to mark exception handlers. It is assumbed that code sections reached from, for example, ERR= handlers in I/O statements are exception handlers. Coverage is reported separately for the main code path and for exception handlers because it is often very difficult to include the exception handlers in test suite runs.
MAXIMUM COVERAGE COUNT	Sets a maximum value for the 4-byte coverage counters to prevent the risk of overflow.