Procedure to add Processor Binding and 64-K Paging to TIEGCM/TIMEGCM jobs run on the NCAR IBM/AIX bluefire system

This procedure can improve model performance by as much as 20%, varying with the number of processors and nodes used.
See also Processor Binding and 64-KB Paging in the CISL Bluefire Quickstart Guide

Step 1:

In your ibm job file (e.g., $TGCMROOT/tiegcm1.9/scripts/tiegcm-ibm.job), replace this line:

  mpirun.lsf $execdir/$model < $namelist >&! $output || \

with these two lines:

  setenv TARGET_CPU_LIST "-1"
  mpirun.lsf /usr/local/bin/launch $execdir/$model < $namelist >&! $output || \

Be sure to include the backslash at the end of the second line.

Step 2:

In the Makefile template (e.g., $TGCMROOT/tiegcm1.9/scripts/Makefile), replace this line:

  LDFLAGS    += -bloadmap:loadmap -q64

with these two lines:

  LDFLAGS    += -bloadmap:loadmap -q64 $(OPTIM)
  LDFLAGS    += -bdatapsize:64K -bstackpsize:64K -btextpsize:64K

Notes:

The following files at both HAO and CISL have been modified for processor binding (models tiegcm1.9 and timegcm1-2dev7):

  $TGCMROOT/tiegcm1.9/scripts/tiegcm-ibm.job
  $TGCMROOT/tiegcm1.9/scripts/Makefile.job

  $TGCMROOT/timegcm1-2dev7/scripts/timegcm-ibm.job
  $TGCMROOT/timegcm1-2dev7/scripts/Makefile.job

If your tiegcm1.9 or timegcm1-2dev7 job script allows the default $utildir (i.e., "set utildir" is commented), then the job script, when submitted, will use the corresponding Makefile above, so you will not have to change your Makefile.

If you change a Makefile in your *aix exec directory (i.e., one that was already made by tgcm_config), you should rename it Makefile.new, and add "-f Makefile.new" to the gmake command in your job script.