Molpro2006.1 on Cobalt

Running jobs on Cobalt is slightly more complicated. Cobalt only has 2 GB of memory per processor. This amounts to about 200 MW of
memory per process, which simply isn’t enough for large Molpro jobs. You can run the job with a larger memory set
and fewer processors. For example, you could use 4 processors to actually compute, but use the memory from 8 processors. This will get you up to 400 MW
memory per process, which should be enough for most large jobs (you will still need multi-passing in the integral transformation, but the jobs will run). The PBS scripts
are somewhat annoying, so I have devised a python script to simplify things. The script can be found in my zuul home area under the very original name “submit.py”

Put the script in your home area on Cobalt. At the command line, you would then type something like the following in the folder containing your input file.

python ~/submit.py 18 4 8 input.dat output.dat

This would create and submit a PBS script to run your job for 18 hours with 4 compute processors and an 8 processor memory set with input file “input.dat” and output file “output.dat.” If no input and output file are given, the script defaults to input.dat and output.dat. If no command line options are given, the script defaults to 18 hours, 8 compute processors, 16 processor memory set. For those who want to do the submitting themselves, below is an example PBS script… but you will insult both me and Python by doing so:

Warnings!!!

1. Using more than 8 compute processors is not advised! It doesn’t seem to scale well past 8, but you are welcome to try.
Unlike at NERSC, the jobs do run faster on 16 processors than on 8, but the scaling is poor… going from 8 to 16 compute processors seems
to give you about 33% faster whereas going from 4 to 8 just about doubles the speed.

2. Walltime maximum is 18 hours! For 460 basis functions, 50 electrons, C1 symmetry, 8 compute processors, 16 processor memory set
the CCSD(T) job finished in 11.7 hours, which is well under the limit.

#!/bin/csh

#PBS -l ncpus=16,walltime=18:00:00

#PBS -q standard

#PBS -l mem=32GB

#PBS -N molprojob

######################################################

#

#

#

set IAm="jjwilke"

cd $SCR

pwd

setenv SCRATCH `/bin/pwd`

cp /u/ac/jjwilke/SN2/COMPLEX/TZ/input.dat .

cp /u/ac/jjwilke/SN2/COMPLEX/TZ/* .

# set up environmental variables

#

setenv MOLPRO /usr/apps/chemistry/molpro/PAR/molpro2006.1

setenv MOLPRO_PWD /u/ac/jjwilke/SN2/COMPLEX/TZ

#

# run molpro

#

alias molprox $MOLPRO/bin/molpro

#

# run the program

unlimit

limit

limit -h

ln -sf /u/ac/jjwilke/SN2/COMPLEX/TZ/output.dat output.dat

molprox -k id=ncsaedu,date=:2007/10/10,version=:9999,mpp=32767,modules=mpp\&qOrLMosDzGS3cVcG -N 8 -o output.dat -n 8 -d $SCR -I /u/ac/jjwilke/SN2/COMPLEX/TZ -W /u/ac/jjwilke/SN2/COMPLEX/TZ -L $MOLPRO/lib/ < input.dat

#

ls -la

#

# move the output and restart files to local disk

#

#/usr/sbin/unlink /u/ac/jjwilke/SN2/COMPLEX/TZ/output.dat

/bin/cp output.dat /u/ac/jjwilke/SN2/COMPLEX/TZ/output.dat