NAMD

The main server for Molecular Dynamics Calculations is PENZIAS which supports both GPU and non GPU versions of NAMD. However the MPI only (no GPU support) parallel versions of NAMD are also installed on SALK and ANDY.

In order to use the code please always load the NAMD module first. The following line will load application environment for NAMD on PENZIAS. GPU is not requested in below SLURM script.

module load namd

A batch submit script for NAMD that runs the CPU-only version on ANDY and PENZIAS using 'mpirun' on 16 processors, 4 to a compute node, follows. Please note that on PENZIAS the que is called production.

#!/bin/bash
#SBATCH --partition production_qdr
#SBATCH --job-name NAMD_MPI
#SBATCH --nodes=4
#SBATCH --ntasks=4
#SBATCH --mpiprocs=4


# Find out name of master execution host (compute node)
echo -n ">>>> SLURM Master compute node is: "
hostname

# You must explicitly change to the working directory in SLURM
cd $SLURM_SUBMIT_DIR

# Use 'mpirun' and point to MPI parallel executable to run 

mpirun -np 16 namd2  ./hivrt.conf > hivrt_mpi.out

echo ">>>> End   Non-Threaded NAMD MPI Parall Run ...

In order to use GPU enabled NAMD versions users must use PENZIAS. First load the proper module file:

module load namd/2.10

The similar as above job, but one that uses 4 CPUs for the bonded interactions and an additional 4 GPUs for the non-bonded interactions, the following script could be used. Please note that this set up is valid only on PENZIAS since it is the only server which has GPU.

#!/bin/bash
#SBATCH --partition production
#SBATCH --job-name NAMD_GPU
#SBATCH --nodes=2
#SBATCH --ntasks=2
#SBATCH --gres=gpu:1
#SBATCH --accel=kepler
#SBATCH --mpiprocs=2

# Find out name of master execution host (compute node)
echo -n ">>>> SLURM Master compute node is: "
hostname

# You must explicitly change to the working directory in SLURM
cd $SLURM_SUBMIT_DIR

# Use 'mpirun' and point to the Non-Threaded MPI parallel executable to run
echo ">>>> Begin Non-Threaded NAMD MPI-GPU Parall Run ..."
mpirun -np 4 namd2  +idlepoll +devices 0,1,0,1   ./hivrt.conf > hivrt_gpu.out
echo ">>>> End    Non-Threaded NAMD MPI-GPU Parall Run ..."

Job submission on SALK via SLURM is somewhat different. First there is no module file. The script below shows how to run 16 processor (core) job on SALK:

#!/bin/bash
#SBATCH --partition production
#SBATCH --job-name NAMD_MPI
#SBATCH --nodes=16
#SBATCH --ntasks=1
#SBATCH --mem=2048
#SBATCH --j oe
#SBATCH --o NAMD_MPI.out


# Find out name of master execution host (compute node)
echo -n ">>>> SLURM Master compute node is: "
hostname

# You must explicitly change to the working directory in SLURM
cd $SLURM_SUBMIT_DIR


# Use 'aprun' and point to the Non-Threaded MPI parallel executable to run
echo ">>>> Begin NAMD Non-Threaded MPI Run ..."
aprun -n 16 -N 16 -cc cpu /share/apps/namd/default/CRAY-XT-g++/namd2  ./hivrt.conf > hivrt.out
echo ">>>> End   NAMD Non-Threaded MPI Run ..."

The most important difference to note is that on SALK is that the 'mpirun' command is replaced with the Cray's 'aprun' command. The 'aprun' command is used to start all jobs on SALK's compute nodes and mediates the interaction between the SLURM script's resource requests and the ALPS resource manager on the Cray. SALK users should familiarize themselves with 'aprun' and its options by reading 'man aprun' on SALK. Users cannot request more resources on their 'aprun' command-lines than are defined by the SLURM script's resource request lines. There is useful discussion elsewhere on the Wiki about the interaction between SLURM and ALPS as mediated by the 'aprun' command and the error message generated when there is a mismatch.

For any of these jobs to run, all the required auxiliary files must be present in the directory from which the job is run.