Fortran Analysis, Engineering & Migration

WRF - Errors in the Precision of Real Numbers

Precision of Real Numbers in WRF - The Issue

An error in the precision of a real number may occur where:

• the precision is not specified, and therefore defaults to low precision;
• the number is used in a context where a higher precision number would be more appropriate.

In WRF Version 3.6.1, the code contains 157,120 real numbers. Of these, 26,814 (18%) are of the default precision but are used where a higher precision would probably be appropriate. In WRF Version 3.4.1, there are 140,296 real numbers. 25,355 (18%) are of the default precision but are associated with objects of higher precision.

Real Number Precision in Fortran

A real number in Fortran code may be written, for example, as:

• 3.14159265358979_kr8 where kr8 is an integer number or integer parameter which specifies the kind. The kind explicitly determines the precision of the number.
• 3.14159265358979D0 where the exponent character, in this case "D", specifies the precision.
• 3.14159265358979 with no specification of precision. In this case the Fortran standard specifies that the number will have the default real kind, which in most implementations of WRF will be a 4-byte IEEE single precision number.

In consequence, a statement like that shown below (WRFV3.4.1, WRFV3/share/module_llxy.F line 1950)

REAL (KIND=HIGH) , PARAMETER :: pi = 3.141592653589793

will set the supposedly high precision parameter pi to the low precision value 3.1415927410125732 instead of 3.1415926535897931.

The Engineering Change

A facility was added to fpt to change the precision of every literal real number in a program according to the following rules:

1. If a KIND tag is present, no change is made.
2. If an exponent character other than "E" or "e" is present, no change is made.
3. If the number is associated through an arithmetic or relational operator, or by assignment, with a double precision real quantity, the number is promoted to double precision. The change is made by rewriting the exponent character or by adding the exponent "D0".
4. If the number occurs in an intrinsic function where the precision of the result is the same as the precision of the argument (Such as SIN or MAX) the number is promoted according to the operator or assignment associations of the intrinsic function invocation as in case 3 above.
5. All other single precision real numbers are unchanged.

The exponent character was used to promote the literal numbers for convenience. The addition of a kind tag would conform better to the Fortran standard, but different kind parameters are used in different components of WRF (And other programs) so this complexity was avoided in the first implementations.

An example of the modified code is shown below (WRFV3.4.1, WRFV3/phys/module_ra_cam_support, line 77)

!From radar.F90 module REAL(r8),PARAMETER :: min_tp_h2o = 160.0D0 ! min T_p for pre-calculated abs/emis REAL(r8),PARAMETER :: max_tp_h2o = 349.999999D0! max T_p for pre-calculated abs/emis REAL(r8),PARAMETER :: dtp_h2o = 21.111111111111D0! difference in adjacent elements of tp_h2o REAL(r8),PARAMETER :: min_te_h2o = -120.0D0 ! min T_e-T_p for pre-calculated abs/emis REAL(r8),PARAMETER :: max_te_h2o = 79.999999D0! max T_e-T_p for pre-calculated abs/emis REAL(r8),PARAMETER :: dte_h2o = 10.0D0 ! difference in adjacent elements of te_h2o REAL(r8),PARAMETER :: min_rh_h2o = 0.0D0 ! min RH for pre-calculated abs/emis REAL(r8),PARAMETER :: max_rh_h2o = 1.19999999D0! max RH for pre-calculated abs/emis REAL(r8),PARAMETER :: drh_h2o = 0.2D0 ! difference in adjacent elements of RH

The original code, reformatted but otherwise un-modified, is:

!From radar.F90 module REAL(r8),PARAMETER :: min_tp_h2o = 160.0 ! min T_p for pre-calculated abs/emis REAL(r8),PARAMETER :: max_tp_h2o = 349.999999 ! max T_p for pre-calculated abs/emis REAL(r8),PARAMETER :: dtp_h2o = 21.111111111111! difference in adjacent elements of tp_h2o REAL(r8),PARAMETER :: min_te_h2o = -120.0 ! min T_e-T_p for pre-calculated abs/emis REAL(r8),PARAMETER :: max_te_h2o = 79.999999 ! max T_e-T_p for pre-calculated abs/emis REAL(r8),PARAMETER :: dte_h2o = 10.0 ! difference in adjacent elements of te_h2o REAL(r8),PARAMETER :: min_rh_h2o = 0.0 ! min RH for pre-calculated abs/emis REAL(r8),PARAMETER :: max_rh_h2o = 1.19999999 ! max RH for pre-calculated abs/emis REAL(r8),PARAMETER :: drh_h2o = 0.2 ! difference in adjacent elements of RH

The changes affect PARAMETER statements, initialisation in DATA statements and declarations, and arithmetic statements in the code.

Changing the Precision of Real Numbers in WRF

If you have not already done so, set up the WRF test environment as described here.

The script tools/WRFV3_fpt_experient.csh has been set up to run an experiment as follows:

1. The fpt_output directories to which fpt will write re-engineered code are cleaned.
2. An fpt run is made for a specific engineering change. The re-engineered files are written to the fpt_output directory structure.
3. The output files are copied to a directory labelled with the WRF version and the name of the experiment so that they may be compared with un-modified files.
4. The output files are copied to the build directories. A small number of modified Makefiles are also copied to these directories.
5. The executable wrf.exe is built. The build procedure is almost identical to that distributed by UCAR.
6. The wrf executable is re-named to label it with the experiment name and is moved to the <WRF version>_run_test/WRFV3/main/ directory.
7. A run of the ideal case em_b_wave is made with the new executable.
8. The wrfout output file is compared with the file generated by a standard UCAR build.

This process is described in more detail in the wrf_experiments page.

Run the experiment correct_precision:

$ tools/WRFV3_fpt_experiment.csh WRFV3.6.1 correct_precision

      or

$ tools/WRFV3_fpt_experiment.csh WRFV3.4.1 correct_precision

The WRF Fortran files built for this experiment are copied to the directories, e.g. WRFV3.4.1_correct_precison/WRFV3/... . Compare these files with the original UCAR sources, and with the corresponding vanilla files if you have already made the vanilla (Unchanged) run. We strongly recommend Beyond Compare from Scooter Software to make this comparison. Specify the rule that Fortran comments are delimited by "!" and that comments are unimportant differences.

In our runs, with gfortran and ifort, the wrfout output files are identical to those generated by a standard UCAR build.

This is a very surprising result. We have just changed the precision of over 25,000 real numbers in the WRF code with no effect on the behaviour.

Why Did Nothing Change?

There was, of course, the usual witch-hunt to prove that we had really built, and were really running, the right program!

A change in the precision of a real number does not necessarily cause a change in value. With IEEE numbers, any number, X such that X = N * 2 ** M where N has up to 6 digits and -128 < N < +128 has the same value in single and double precision. Adding an exponent character to numbers like 0.5 should change nothing. The engineering tool therefore marks with a diagnostic only those numbers where the change in precision also causes a change in value. The diagnostic (Which may be suppressed for production runs) show, for example (WRFV3/phys/module_cu_sas.F line 247):

pgcon_use = 0.55D0 ! Zhang & Wu (2003,JAS), used in GFS (25km res spectral) !---------------------------^-------------------------------------------------- !!! FPT - 3743 Real literal precision increased with a small change in value !------------------------------------------------------------------------------

The total number of literals encountered and modified is reported in the list (fpl) file by fpt. The report shows, e.g. for WRFV3.4.1:

Promotion of Literal REAL Numbers ================================= Count Percent REAL literal numbers: 140296 With KIND tags: 16885 12.04% Promoted to higher precision: 25288 18.02% Promoted with change in value: 22526 16.06% Not promoted: 98090 69.92%

Most of the numbers with changed precision have small changes in value. The changes are marked by the fpt diagnostic number 3743, and the number of changes in individual files may therefore be examined. It was found that:

• In WRFV3.4.1, there were no changes in the code in the directories dyn_em, external, main, inc or share.
• Two files showed changes in the frame directory, but the changed values were very unlikely to be encountered.
• All of the significant changes were in the phys directory.

Any single test run only uses one of over a dozen physics options, so most of the code in phys is unused in any run. The occurrences of changed values in the phys directory in WRFV3.4.1 are:

Almost all of the changed values are in DATA statements in module_ra_goddard.F which is unused in the standard run of em_b_wave.

Counting Visits to Numbers With Changed Values

The fpt command insert monitor of changed real numbers enforces the precision rules described above, and inserts function invocations and subroutine calls to count visits to the numbers with changed values. The modification, for example, is (WRFV3.4.1, File WRFV3/phys/module_bl_gfs.F, line 250):

cpm = cp*(1.D0+fun_fpt_changed_precision(3,0.8D0)*qv3d(i,kts,j))

The original code is:

cpm = cp*(1.+0.8*qv3d(i,kts,j))

Note that the number 1. in the original code is promoted, but is not marked by the function fun_fpt_changed_precision because its value has not changed. The function has two arguments. The first is a unique integer identifier which allows the call to be located in the code and the second is the value of the number which has changed. In the case of changes to PARAMETER and DATA statements, the sub-programs where the objects affected are in scope are marked by a subroutine call which records that the change could potentially be visited.

The number of visits to each changed value is counted, and code is inserted to report the counts at the end of the program. The report is written to the file fpt_monitor_literal_precision.txt.

Run the experiment monitor_precision to count visits to changed real numbers in a run of em_b_wave, for example:.

$ tools/WRFV3_fpt_experiment.csh WRFV3.6.1 monitor_precision

The file fpt_monitor_literal_precision.txt is written to the directory run_tests/WRFV3/run. In our runs, it contains only the text:

No literal numbers with changed precision were visited
 
End of file

The Next Step

The next step in this study is to set up a test-suite which exercises a range of WRF physics options. It should then be possible to assess the extent to which changes in precision affect performance of the code.
 
 

Copyright ©1995 to 2024 Software Validation Ltd. All rights reserved.