Login
CCC PC Cluster (32- and 64-bit machines)There currently exists three (3) distinct queues on the Ghostbuster cluster. They are:
Not all the programs that are available to us at the CCC are compiled for all the above systems. The following table denotes which programs are known to either work or not work based on architecture:
Following is the email that was sent to the group regarding submitting to the opt queue.
Group- If you would like to begin running on the Opterons, please read this email in its entirety: The Opteron nodes are ready for all to use. A few will remain offline for further benchmarking and compiling of programs. Here are some general guidelines for using the Opterons: - A single job can not use more than 8 processors (make note that each node contains 2 processors) - Each node has a total of 6 GB of memory shared between the 2 processors. - When making requests for nodes you must REQUEST BOTH processors for a node (shown below) - Some generalize scripts have been created to ease the running of parallel jobs - Molpro and NWChem are currently the only codes available to run in parallel. MPQC and QChem will follow soon. When you specify which queue to submit to use the "opt" description (ie "qsub -q opt"). You must specifiy how many nodes to use and how many processors on each node. PBS has been setup so that you MUST request both processors. This is done by this line in the PBS script: #PBS -l nodes=1:ppn=2 or #PBS -l nodes=2:ppn=2 or #PBS -l nodes=3:ppn=2 or #PBS -l nodes=4:ppn=2 The first line requests 1 node and 2 processors on this node. The others request 4, 6, and 8 processors, respectively. You must specify the paths for the 64-bit version of Molpro and NWChem which are: /usr/local/share/molpro/x86_64/bin /usr/local/share/nwchem/bin64 Now, rather than just set these paths in your .tcshrc, or whatever shell resource file you're using, you probably would like to first see if you're on an Intel (32-bit) machine, or an Opteron (64-bit). To do this, add an IF statement to your resource files, to the effect of: -------------------------------------- if ($MACHTYPE == "x86_64") then set path = ( paths to 64-bit binaries and other system directories ) else set path = ( paths to 32-bit binaries and other system directories ) endif --------------------------------------- This way, when you log onto a node (either you logging on explicitly, or PBS opening a shell to run one of your jobs) your paths will be set to the proper binaires. If you cruise around /usr/local/share, you'll observe new LAPACK/BLAS libraries, Athlon Core Math Libraries (ACML), MPI libraries, etc. -- and the 32-bit vs. 64-bit subdirectories directories should be obvious (given their name) if you choose to include these in your path declarations. The scripts that you should use to run Molpro and NWChem are (these are located in the above paths): run_molpro inputfile run_nwchem inputfile You can try and run whatever you'd like -- but for the time being only NWChem and Molpro are guaranteed to work! Also, we will not be compiling Gaussian94 for the opterons. Rather, we will focus on other DFT-ready codes that may be run in parallel. Here is an example PBS submission file that requests 2 processors: ======START PBS FILE====== #PBS -l nodes=1:ppn=2 #PBS -q opt cd $PBS_O_WORKDIR rm -fr /tmp*/$USER/* run_molpro input.dat rm -fr /tmp*/$USER/* =======END PBS FILE======= If you have any questions about Molpro or NWChem not working properly, please contact me and/or Michael about it. If you have problems with other codes, by all means let us know, but we'll probably just "take it under consideration" for the time being. -Justin and Michael |
