James: Text replacement - "[pP][bB][sS]" to "SLURM"

2022-10-27T19:49:20Z

Text replacement - "[pP][bB][sS]" to "SLURM"

← Older revision		Revision as of 19:49, 27 October 2022
Line 98:		Line 98:
	directory which has much more disk space than the SLURM spool directory on /var.		directory which has much more disk space than the SLURM spool directory on /var.

	Details on the meaning of the ~~PBS~~ script are covered below in the SLURM section. The most important		Details on the meaning of the SLURM script are covered below in the SLURM section. The most important
	lines are the '#SBATCH --nodes ntasks=1 mem=2880'. The		lines are the '#SBATCH --nodes ntasks=1 mem=2880'. The
	first instructs SLURM to select 2 resource 'chunks' each with 1 processor (core) and 2,880 MBs of		first instructs SLURM to select 2 resource 'chunks' each with 1 processor (core) and 2,880 MBs of
Line 117:		Line 117:

	# Find out name of master execution host (compute node)		# Find out name of master execution host (compute node)
	echo -n ">>>> ~~PBS~~ Master compute node is: "		echo -n ">>>> SLURM Master compute node is: "
	hostname		hostname

James: Created page with "Currently, BEST is available on ANDY at the CUNY HPC Center. To run BEST, first a NEXUS-formatted, DNA sequence comparison input file (e.g. a '.nex' file) must be created using MrBayes. See the section on MrBayes below for this. Here is an example NEXUS input file:

 #NEXUS  begin data;    dimensions ntax=17 nchar=432;    format datatype=dna missing=?;    matrix    human       ctgactcctgaggagaagtctgccgttactgccctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcaggctg..."

2022-10-20T19:54:51Z

Created page with "Currently, BEST is available on ANDY at the CUNY HPC Center. To run BEST, first a NEXUS-formatted, DNA sequence comparison input file (e.g. a '.nex' file) must be created using MrBayes. See the section on MrBayes below for this. Here is an example NEXUS input file: <pre> #NEXUS begin data; dimensions ntax=17 nchar=432; format datatype=dna missing=?; matrix human ctgactcctgaggagaagtctgccgttactgccctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcaggctg..."

New page

Currently, BEST is available on ANDY at the CUNY HPC Center.

To run BEST, first a NEXUS-formatted, DNA sequence comparison input file (e.g. a '.nex' file)
must be created using MrBayes. See the section on MrBayes below for this. Here is an
example NEXUS input file:

<pre>
#NEXUS

begin data;
dimensions ntax=17 nchar=432;
format datatype=dna missing=?;
matrix
human ctgactcctgaggagaagtctgccgttactgccctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcaggctgctggtggtctacccttggacccagaggttctttgagtcctttggggatctgtccactcctgatgctgttatgggcaaccctaaggtgaaggctcatggcaagaaagtgctcggtgcctttagtgatggcctggctcacctggacaacctcaagggcacctttgccacactgagtgagctgcactgtgacaagctgcacgtggatcctgagaacttcaggctcctgggcaacgtgctggtctgtgtgctggcccatcactttggcaaagaattcaccccaccagtgcaggctgcctatcagaaagtggtggctggtgtggctaatgccctggcccacaagtatcac
tarsier ctgactgctgaagagaaggccgccgtcactgccctgtggggcaaggtagacgtggaagatgttggtggtgaggccctgggcaggctgctggtcgtctacccatggacccagaggttctttgactcctttggggacctgtccactcctgccgctgttatgagcaatgctaaggtcaaggcccatggcaaaaaggtgctgaacgcctttagtgacggcatggctcatctggacaacctcaagggcacctttgctaagctgagtgagctgcactgtgacaaattgcacgtggatcctgagaatttcaggctcttgggcaatgtgctggtgtgtgtgctggcccaccactttggcaaagaattcaccccgcaggttcaggctgcctatcagaaggtggtggctggtgtggctactgccttggctcacaagtaccac
bushbaby ctgactcctgatgagaagaatgccgtttgtgccctgtggggcaaggtgaatgtggaagaagttggtggtgaggccctgggcaggctgctggttgtctacccatggacccagaggttctttgactcctttggggacctgtcctctccttctgctgttatgggcaaccctaaagtgaaggcccacggcaagaaggtgctgagtgcctttagcgagggcctgaatcacctggacaacctcaagggcacctttgctaagctgagtgagctgcattgtgacaagctgcacgtggaccctgagaacttcaggctcctgggcaacgtgctggtggttgtcctggctcaccactttggcaaggatttcaccccacaggtgcaggctgcctatcagaaggtggtggctggtgtggctactgccctggctcacaaataccac
hare ctgtccggtgaggagaagtctgcggtcactgccctgtggggcaaggtgaatgtggaagaagttggtggtgagaccctgggcaggctgctggttgtctacccatggacccagaggttcttcgagtcctttggggacctgtccactgcttctgctgttatgggcaaccctaaggtgaaggctcatggcaagaaggtgctggctgccttcagtgagggtctgagtcacctggacaacctcaaaggcaccttcgctaagctgagtgaactgcattgtgacaagctgcacgtggatcctgagaacttcaggctcctgggcaacgtgctggttattgtgctgtctcatcactttggcaaagaattcactcctcaggtgcaggctgcctatcagaaggtggtggctggtgtggccaatgccctggctcacaaataccac
rabbit ctgtccagtgaggagaagtctgcggtcactgccctgtggggcaaggtgaatgtggaagaagttggtggtgaggccctgggcaggctgctggttgtctacccatggacccagaggttcttcgagtcctttggggacctgtcctctgcaaatgctgttatgaacaatcctaaggtgaaggctcatggcaagaaggtgctggctgccttcagtgagggtctgagtcacctggacaacctcaaaggcacctttgctaagctgagtgaactgcactgtgacaagctgcacgtggatcctgagaacttcaggctcctgggcaacgtgctggttattgtgctgtctcatcattttggcaaagaattcactcctcaggtgcaggctgcctatcagaaggtggtggctggtgtggccaatgccctggctcacaaataccac
cow ctgactgctgaggagaaggctgccgtcaccgccttttggggcaaggtgaaagtggatgaagttggtggtgaggccctgggcaggctgctggttgtctacccctggactcagaggttctttgagtcctttggggacttgtccactgctgatgctgttatgaacaaccctaaggtgaaggcccatggcaagaaggtgctagattcctttagtaatggcatgaagcatctcgatgacctcaagggcacctttgctgcgctgagtgagctgcactgtgataagctgcatgtggatcctgagaacttcaagctcctgggcaacgtgctagtggttgtgctggctcgcaattttggcaaggaattcaccccggtgctgcaggctgactttcagaaggtggtggctggtgtggccaatgccctggcccacagatatcat
sheep ctgactgctgaggagaaggctgccgtcaccggcttctggggcaaggtgaaagtggatgaagttggtgctgaggccctgggcaggctgctggttgtctacccctggactcagaggttctttgagcactttggggacttgtccaatgctgatgctgttatgaacaaccctaaggtgaaggcccatggcaagaaggtgctagactcctttagtaacggcatgaagcatctcgatgacctcaagggcacctttgctcagctgagtgagctgcactgtgataagctgcacgtggatcctgagaacttcaggctcctgggcaacgtgctggtggttgtgctggctcgccaccatggcaatgaattcaccccggtgctgcaggctgactttcagaaggtggtggctggtgttgccaatgccctggcccacaaatatcac
pig ctgtctgctgaggagaaggaggccgtcctcggcctgtggggcaaagtgaatgtggacgaagttggtggtgaggccctgggcaggctgctggttgtctacccctggactcagaggttcttcgagtcctttggggacctgtccaatgccgatgccgtcatgggcaatcccaaggtgaaggcccacggcaagaaggtgctccagtccttcagtgacggcctgaaacatctcgacaacctcaagggcacctttgctaagctgagcgagctgcactgtgaccagctgcacgtggatcctgagaacttcaggctcctgggcaacgtgatagtggttgttctggctcgccgccttggccatgacttcaacccgaatgtgcaggctgcttttcagaaggtggtggctggtgttgctaatgccctggcccacaagtaccac
elephseal ttgacggcggaggagaagtctgccgtcacctccctgtggggcaaagtgaaggtggatgaagttggtggtgaagccctgggcaggctgctggttgtctacccctggactcagaggttctttgactcctttggggacctgtcctctcctaatgctattatgagcaaccccaaggtcaaggcccatggcaagaaggtgctgaattcctttagtgatggcctgaagaatctggacaacctcaagggcacctttgctaagctcagtgagctgcactgtgaccagctgcatgtggatcccgagaacttcaagctcctgggcaatgtgctggtgtgtgtgctggcccgccactttggcaaggaattcaccccacagatgcagggtgcctttcagaaggtggtagctggtgtggccaatgccctcgcccacaaatatcac
rat ctaactgatgctgagaaggctgctgttaatgccctgtggggaaaggtgaaccctgatgatgttggtggcgaggccctgggcaggctgctggttgtctacccttggacccagaggtactttgatagctttggggacctgtcctctgcctctgctatcatgggtaaccctaaggtgaaggcccatggcaagaaggtgataaacgccttcaatgatggcctgaaacacttggacaacctcaagggcacctttgctcatctgagtgaactccactgtgacaagctgcatgtggatcctgagaacttcaggctcctgggcaatatgattgtgattgtgttgggccaccacctgggcaaggaattcaccccctgtgcacaggctgccttccagaaggtggtggctggagtggccagtgccctggctcacaagtaccac
mouse ctgactgatgctgagaagtctgctgtctcttgcctgtgggcaaaggtgaaccccgatgaagttggtggtgaggccctgggcaggctgctggttgtctacccttggacccagcggtactttgatagctttggagacctatcctctgcctctgctatcatgggtaatcccaaggtgaaggcccatggcaaaaaggtgataactgcctttaacgagggcctgaaaaacctggacaacctcaagggcacctttgccagcctcagtgagctccactgtgacaagctgcatgtggatcctgagaacttcaggctcctaggcaatgcgatcgtgattgtgctgggccaccacctgggcaaggatttcacccctgctgcacaggctgccttccagaaggtggtggctggagtggccactgccctggctcacaagtaccac
hamster ctgactgatgctgagaaggcccttgtcactggcctgtggggaaaggtgaacgccgatgcagttggcgctgaggccctgggcaggttgctggttgtctacccttggacccagaggttctttgaacactttggagacctgtctctgccagttgctgtcatgaataacccccaggtgaaggcccatggcaagaaggtgatccactccttcgctgatggcctgaaacacctggacaacctgaagggcgccttttccagcctgagtgagctccactgtgacaagctgcacgtggatcctgagaacttcaagctcctgggcaatatgatcatcattgtgctgatccacgacctgggcaaggacttcactcccagtgcacagtctgcctttcataaggtggtggctggtgtggccaatgccctggctcacaagtaccac
marsupial ttgacttctgaggagaagaactgcatcactaccatctggtctaaggtgcaggttgaccagactggtggtgaggcccttggcaggatgctcgttgtctacccctggaccaccaggttttttgggagctttggtgatctgtcctctcctggcgctgtcatgtcaaattctaaggttcaagcccatggtgctaaggtgttgacctccttcggtgaagcagtcaagcatttggacaacctgaagggtacttatgccaagttgagtgagctccactgtgacaagctgcatgtggaccctgagaacttcaagatgctggggaatatcattgtgatctgcctggctgagcactttggcaaggattttactcctgaatgtcaggttgcttggcagaagctcgtggctggagttgcccatgccctggcccacaagtaccac
duck tggacagccgaggagaagcagctcatcaccggcctctggggcaaggtcaatgtggccgactgtggagctgaggccctggccaggctgctgatcgtctacccctggacccagaggttcttcgcctccttcgggaacctgtccagccccactgccatccttggcaaccccatggtccgtgcccatggcaagaaagtgctcacctccttcggagatgctgtgaagaacctggacaacatcaagaacaccttcgcccagctgtccgagctgcactgcgacaagctgcacgtggaccctgagaacttcaggctcctgggtgacatcctcatcatcgtcctggccgcccacttcaccaaggatttcactcctgactgccaggccgcctggcagaagctggtccgcgtggtggcccacgctctggcccgcaagtaccac
chicken tggactgctgaggagaagcagctcatcaccggcctctggggcaaggtcaatgtggccgaatgtggggccgaagccctggccaggctgctgatcgtctacccctggacccagaggttctttgcgtcctttgggaacctctccagccccactgccatccttggcaaccccatggtccgcgcccacggcaagaaagtgctcacctcctttggggatgctgtgaagaacctggacaacatcaagaacaccttctcccaactgtccgaactgcattgtgacaagctgcatgtggaccccgagaacttcaggctcctgggtgacatcctcatcattgtcctggccgcccacttcagcaaggacttcactcctgaatgccaggctgcctggcagaagctggtccgcgtggtggcccatgccctggctcgcaagtaccac
xenlaev tggacagctgaagagaaggccgccatcacttctgtatggcagaaggtcaatgtagaacatgatggccatgatgccctgggcaggctgctgattgtgtacccctggacccagagatacttcagtaactttggaaacctctccaattcagctgctgttgctggaaatgccaaggttcaagcccatggcaagaaggttctttcagctgttggcaatgccattagccatattgacagtgtgaagtcctctctccaacaactcagtaagatccatgccactgaactgtttgtggaccctgagaactttaagcgttttggtggagttctggtcattgtcttgggtgccaaactgggaactgccttcactcctaaagttcaggctgcttgggagaaattcattgcagttttggttgatggtcttagccagggctataac
xentrop tggacagctgaagaaaaagcaaccattgcttctgtgtgggggaaagtcgacattgaacaggatggccatgatgcattatccaggctgctggttgtttatccctggactcagaggtacttcagcagttttggaaacctctccaatgtctccgctgtctctggaaatgtcaaggttaaagcccatggaaataaagtcctgtcagctgttggcagtgcaatccagcatctggatgatgtgaagagccaccttaaaggtcttagcaagagccatgctgaggatcttcatgtggatcccgaaaacttcaagcgccttgcggatgttctggtgatcgttctggctgccaaacttggatctgccttcactccccaagtccaagctgtctgggagaagctcaatgcaactctggtggctgctcttagccatggctacttc
;
end;

begin mrbayes;
charset non_coding = 1-90 358-432;
charset coding = 91-357;
partition region = 2:non_coding,coding;
set partition = region;
lset applyto=(2) nucmodel=codon;
prset ratepr=variable;
mcmc ngen=5000 nchains=1 samplefreq=10;
end;
</pre>

Next, a SLURM batch script must be created to run your job. The first script below shows a MPI parallel script
for the above '.nex' input file. Note that the number of processors that can be used by the job is limited to
the number of chains in the input file. Here, we have just 2 chains and therefore can only request 2 processors.
If you make the mistake of asking for more processors than input file chains, you will get the following error
message:

<pre>
The number of chains must be at least as great
as the number of processors (in this case 4)
</pre>

Also, to include all required environmental variables and the path to the BEST executable run the modules load
command (the modules utility is discussed in detail above):

<pre>
module load best
</pre>

Here is the MPI parallel SLURM batch script for BEST that request 2 processors, one for each chain in the input file:

<pre>
#!/bin/bash
#SBATCH --partition production
#SBATCH --job-name BEST_parallel
#SBATCH --nodes=2
#SBATCH --ntasks=1
#SBATCH --mem=2880

# Find out name of master execution host (compute node)
echo -n ">>>> SLURM Master compute node is: "
hostname

# You must explicitly change to the working directory in SLURM
cd $SLURM_SUBMIT_DIR

# Use 'mpirun' and point to the MPI parallel executable to run
echo ">>>> Begin BEST Parallel Run ..."
mpirun -np mbbest ./bglobin.nex > best_mpi.out 2>&1
echo ">>>> End BEST Parallel Run ..."
</pre>

This script can be dropped into a file (say 'best_mpi.job) on ANDY and run with:

<pre>
qsub best_mpi.job
</pre>

It should take less five minutes to run and will produce SLURM output and error files beginning
with the job name 'BEST_parallel'. The primary BEAST application results will be written into
the user-specified file at the end of the BEST command line after the greater-than sign. Here
it is named 'best_mpi.out'. The expression '2>&1' combines Unix standard output from the
program with Unix standard error. Users should always explicitly specify the name of the
application's output file in this way to ensure that it is written directly into the user's working
directory which has much more disk space than the SLURM spool directory on /var.

Details on the meaning of the PBS script are covered below in the SLURM section. The most important
lines are the '#SBATCH --nodes ntasks=1 mem=2880'. The
first instructs SLURM to select 2 resource 'chunks' each with 1 processor (core) and 2,880 MBs of
memory in it for the job. The second instructs SLURM to place this job wherever the least used
resources are found (freely). The master compute node that it finally selects to run your job
will be printed in the SLURM output file by the 'hostname' command.

The CUNY HPC Center also provides a serial version of BEST. A SLURM batch script for running the
serial version of BEST follows:

<pre>
#!/bin/bash
#SBATCH --partition production
#SBATCH --job-name BEST_serial
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --mem=2880

# Find out name of master execution host (compute node)
echo -n ">>>> PBS Master compute node is: "
hostname

# You must explicitly change to the working directory in SLURM
cd $SLURM_SUBMIT_DIR

# Just point to the serial executable to run
echo ">>>> Begin BEST Serial Run ..."
mbbest_serial ./bglobin.nex > best_ser.out 2>&1
echo ">>>> End BEST Serial Run ..."
</pre>

BEST - Revision history

James: Text replacement - "[pP][bB][sS]" to "SLURM"