The TIEGCM model is formally supported on two platform systems: 64-bit Linux Desktop, and a Linux cluster supercomputer (e.g., NCAR’s yellowstone system. However, the model has been built and executed on several other platforms. The source code is f90 standard compliant, and is mostly fixed-format fortran.
The tiegcm can be built with three compilers on 64-bit Linux Desktop systems, but only the intel compiler is used on the NCAR supercomputer yellowstone. The default is intel, since it out-performs the other compilers on both systems. As of January, 2015, these are the versions of each compiler we are using:
Each compiler has a makefile in the scripts directory that specifies compiler-specific flags, library paths, and other parameters necessary for the build priocess. These files are provided in the model scripts/ directory, and are included in the main Makefile at build time:
One of these files, or the user’s own, is specified by the job script variable “make” in the job script. The specified file is included in the main Makefile. User’s outside NCAR are encouraged to copy and rename one of these files, and customize it for your own operating system and compiler.
External library dependencies are netCDF, MPI, and ESMF. The MPI implementation is often bundled in with the compiler. Local paths to these libraries are specified in the compiler-specific Make files described above.
The electro-dynamo code see source file pdynamo.F) in the TIEGCM is calculated on the geomagnetic grid. Since the dynamo receives inputs from the neutral atmosphere, which is on the geographic grid, there is a need for regridding capability between the two grid systems. The same horizontal geomagnetic coordinates are used regardless of the 5-deg or 2.5-deg resolution of the geographic grid.
The Earth System Modeling Framework (see also Modeling Infractructure for the Geoscience Community is used in the TIEGCM to perform the grid remapping in an parallel MPI environment, see src/esmf.F. To build the TIEGCM, the ESMF library must be included in the link step. If the ESMF library is not already on your system, you will need to download and build it, using the same compiler you are using to build the TIEGCM.
HAO is using the following ESMF libraries built on a 64-bit Linux desktop for each compiler/MPI implementation:
ESMF libraries at HAO for use on Linux desktop systems (these paths are provided in the scripts/Make.xxxx files described above). The esmf makefiles esmf.mk are included in the model’s main makefile scripts/Makefile
The Network Common Data Form (NetCDF) is a cross-platform, self-describing metadata file format, developed by UNIDATA at UCAR. Please see NetCDF for more information. It is necessary to link the netCDF library when the model is built, since all data files imported to the model, and all model output history files are in NetCDF format. Because NetCDF is platform-independent, all history and data files can be used on either linux desktops or the NCAR yellowstone system. At HAO on linux desktops, we are using NetCDF version 4.1.1. On the NCAR yellowstone system, we are using version 4.3.2.
The model can be built with debug flags set in the compiler. To do this, simply set debug = TRUE in the job script, and resubmit (see also job scripts section).
The debug flags are set in the compiler-specific Make files described above. They can be adjusted there, of course, but usually they include floating-point exception traps, and core dumps with traceback. If debug was false in a previous run, the entire code will be rebuilt with the debug flags set, however, it doesn’t hurt to go to the execdir and type “gmake clean” before resubmitting. Keep in mind that because optimization is turned off when debug flags are set, performance will be destroyed, and the model will run agonizingly slow.
Although we do not support the model with MPI turned off, it can also be useful for debugging to run the model with only a single MPI task. To do this, set nproc=1 in the linux job script, or set #BSUB -n 1 in the yellowstone job script.