fpt and WinFPT Reference Manual - Command-line Commands
| SimCon Home | Ref Manual Home |
Testing a Migration
The Tests Required
Before beginning any migration project it is important to consider how to test it.
Unit Tests - Record-replay - Capturing and Replaying I/O Transactions and Selected Interfaces
The command INSERT RECORD REPLAY instruments the code to capture data at inerfaces. Statements are inserted to capture all terminal and file input and the data returned from selected sub-programs. The captured data can then be replayed into the program, and the user can select which I/O unit numbers and which sub-program interfaces are to be replayed. Two additional commands are used to tune this process.
DEFAULT LENGTH OF RECORD REPLAY STRINGS sets the length of strings captured by the record/replay mechanism when the length cannot be inferred from context.
NUMBER RECORD REPLAY CALLS FROM shortens the build process. All record/replay call sites are identified by a unique number. This command optionally specifies the starting number for each file. This prevents files from changing simply because the record/replay numbers have changed, and therefore prevents unnecessary compilations.
These commands were used to capture the I/O for a real-time controller of a chemical plant over a six hour run. Several hundred key strokes were captured, with interface data from clocks, thermometers, flow meters and colorimeters. The data could be replayed into the code to provide a test frame which ran from a single command and in only a few minutes.
System Tests - Run-time Trace - Comparing Runs on Different Hosts, Eliminating Numerical Drift
The command INSERT RUN-TIME TRACE is used to validate performance. Statements are added to the code to trace program execution and to capture every scalar numeric value (LOGICAL, INTEGER, REAL or COMPLEX) to a trace file as it is computed. This may be used to investigate error conditions, and to check the performance of the code under different compilers. The exact numerical results of a program will usually change when the compiler changes. Even when the compilers use the same number format small differences in the order of computation or expressions will cumulate to change the results. This is numerical drift, and does not indicate that there is a problem in the migration. The trace file may be used to eliminate numerical drift, and therefore to expose coding errors and compiler bugs (see "The Removal of Numerical Drift from Scientific Models", Collins J, Farrimond B, Flower D, Anderson M and Gill D. 2013, arXiv:1304.3260 [cs.SE]).
A detail in handling run-time trace is that the order of evaluation in Fortran statements which contain two or more function calls may be ambiguous. The order of evaluation of the functions is not defined by the Fortran standard, and different compilers make different choices of order. The command RESOLVE AMBIGUOUS FLOW is used to modify the code so that the order of evaluation is uniquely defined.
Run-time trace was used to eliminate numerical drift in WRF (which is about half a million lines long) and to compare runs under ifort and gfortran. The analysis exposed coding errors and a compiler bug in ifort.
Coverage Analysis - Verifying the Completeness of the Tests
The command INSERT COVERAGE instruments a code to check test coverage. This shows the proportion of a code which is actually exercised by a program test suite. Two additional commands are used to control the check.
The command EXCEPTION may optionally be written in the code to mark exception handlers. It is assumed that code sections reached from, for example, ERR= handlers in I/O statements are exception handlers. Coverage is reported separately for the main code path and for exception handlers because it is often very difficult to include the exception handlers in test suite runs.
MAXIMUM COVERAGE COUNT sets a maximum value for the 4-byte coverage counters to prevent the risk of overflow.
Copyright ©1995 to 2025 Software Validation Ltd. All rights reserved.