Modules and Available Third Party Software - Revision history

Alex: /* Useful Modules Commands */

2024-12-21T17:13:47Z

Useful Modules Commands

← Older revision		Revision as of 17:13, 21 December 2024
Line 70:		Line 70:

	=== Useful Modules Commands ===		=== Useful Modules Commands ===
	[[File:Modules summary.png\|left\|~~thumb~~]]		[[File:Modules summary.png\|left\|frameless\|799x799px]]

Alex: /* Useful Modules Commands */

2024-12-21T17:12:22Z

Useful Modules Commands

← Older revision		Revision as of 17:12, 21 December 2024
Line 70:		Line 70:

	=== Useful Modules Commands ===		=== Useful Modules Commands ===
			[[File:Modules summary.png\|left\|thumb]]

Alex at 17:08, 21 December 2024

2024-12-21T17:08:12Z

@@ Line 67: / Line 67: @@
 :•	Determine what modules should be unloaded.
 :•	Determine what modules should be loaded.
 :•	Add the needed modules, i.e., '''<font face="courier">module load</font>'''
-The first step, see what modules are loaded, is shown below.
+=== Useful Modules Commands ===

Alex: /* TCL Modules and available third party software */

2024-12-21T17:06:00Z

TCL Modules and available third party software

← Older revision		Revision as of 17:06, 21 December 2024
Line 1:		Line 1:
	=~~TCL Modules and available third party software~~=		=Jobs Environment Management =
	==Modules==		==Modules==
	“Modules” makes it easier for users to run a standard or customized application and/or system environment. HPCC uses '''Lmod''' - an advanced module system that easily handles the MODULEPATH hierarchical problem common in UNIX based "modules" implementation. Application packages can be loaded and unloaded cleanly through the module system using modulefiles. This includes easily adding or removing directories to the PATH environment variable. Modulefiles for Library packages provide environment variables that specify where the library and header files can be found.		“Modules” makes it easier for users to run a standard or customized application and/or system environment. HPCC uses '''Lmod''' - an advanced module system that easily handles the MODULEPATH hierarchical problem common in UNIX based "modules" implementation. Application packages can be loaded and unloaded cleanly through the module system using modulefiles. This includes easily adding or removing directories to the PATH environment variable. Modulefiles for Library packages provide environment variables that specify where the library and header files can be found.

Alex at 02:12, 30 June 2023

2023-06-30T02:12:32Z

← Older revision		Revision as of 02:12, 30 June 2023
Line 1:		Line 1:
	=Modules and available third party software=		=TCL Modules and available third party software=
	==Modules==		==Modules==
	“Modules” makes it easier for users to run a standard or customized application and/or system environment. HPCC uses '''Lmod''' - an advanced module system that easily handles the MODULEPATH hierarchical problem common in UNIX based "modules" implementation. Application packages can be loaded and unloaded cleanly through the module system using modulefiles. This includes easily adding or removing directories to the PATH environment variable. Modulefiles for Library packages provide environment variables that specify where the library and header files can be found.		“Modules” makes it easier for users to run a standard or customized application and/or system environment. HPCC uses '''Lmod''' - an advanced module system that easily handles the MODULEPATH hierarchical problem common in UNIX based "modules" implementation. Application packages can be loaded and unloaded cleanly through the module system using modulefiles. This includes easily adding or removing directories to the PATH environment variable. Modulefiles for Library packages provide environment variables that specify where the library and header files can be found.

James: Text replacement - "[pP][bB][sS]" to "SLURM"

2022-10-27T19:48:03Z

Text replacement - "[pP][bB][sS]" to "SLURM"

← Older revision		Revision as of 19:48, 27 October 2022
Line 97:		Line 97:
	22) xtpe-mc8		22) xtpe-mc8
	23) cray-mpich2/5.5.3		23) cray-mpich2/5.5.3
	24) ~~pbs~~/11.3.0.121723		24) SLURM/11.3.0.121723

	From the list, we see that the Cray Programming Environment ('''<font face="courier">PrgEnv-cray/4.0.46</font>''') and the Cray Compiler environment are loaded ('''<font face="courier">cce/8.0.7</font>''') by default. To unload these Cray modules and load in the PGI equivalents we need to know the names of the PGI modules. The '''<font face="courier">module avail</font>''' command shows this.		From the list, we see that the Cray Programming Environment ('''<font face="courier">PrgEnv-cray/4.0.46</font>''') and the Cray Compiler environment are loaded ('''<font face="courier">cce/8.0.7</font>''') by default. To unload these Cray modules and load in the PGI equivalents we need to know the names of the PGI modules. The '''<font face="courier">module avail</font>''' command shows this.

James: Created page with "=Modules and available third party software= ==Modules== “Modules” makes it easier for users to run a standard or customized application and/or system environment. HPCC uses '''Lmod''' - an advanced module system that easily handles the MODULEPATH hierarchical problem common in UNIX based "modules" implementation. Application packages can be loaded and unloaded cleanly through the module system using modulefiles. This includes easily adding or removing directories to..."

2022-10-17T17:01:24Z

Created page with "=Modules and available third party software= ==Modules== “Modules” makes it easier for users to run a standard or customized application and/or system environment. HPCC uses '''Lmod''' - an advanced module system that easily handles the MODULEPATH hierarchical problem common in UNIX based "modules" implementation. Application packages can be loaded and unloaded cleanly through the module system using modulefiles. This includes easily adding or removing directories to..."

New page

=Modules and available third party software=
==Modules==
“Modules” makes it easier for users to run a standard or customized application and/or system environment. HPCC uses '''Lmod''' - an advanced module system that easily handles the MODULEPATH hierarchical problem common in UNIX based "modules" implementation. Application packages can be loaded and unloaded cleanly through the module system using modulefiles. This includes easily adding or removing directories to the PATH environment variable. Modulefiles for Library packages provide environment variables that specify where the library and header files can be found.

All the popular shells are supported: '''bash, ksh, csh, tcsh, zsh.''' LMOD is also available for perl and python. Modules is available on KARLE, PENZIAS, APPEL and SALK.

===Modules - getting started ===
The basic module commands are listed below. Note that almost all applications have default version and several other versions. The default version is marked with (D). For example:

python/2.7.13_anaconda (D)

The default version can be loaded via its short name: . Non default version(s) require use of their full name. For example:

module load python

will load the default 2.7.13_anaconda version of python interpreter.To load non-default 3.7.6 the user must type

module load python/3.7.6_anaconda

The module load command can be used to load several application environments at once:

module load package1 package2 ...

For documentation on “Modules”:

man module

For help enter:

module help

To see a list of currently loaded “Modules” run:

module list

To see a complete list of all modules available on the system run:

module avail

To show content of a module enter:

module show <module_name>

To change from one application to another ( example. default versions of gnu and intel compiler):

module swap gcc intel

To go back to an initial set of modules:

module reset

===Using LMOD commands===
To get a list of all modules available
module spider

To get information about a specific module
module spider python

===Modules for the advanced user===
A “Modules” example for advanced users who need to change their environment.

The HPC Center supports a number of different compilers, libraries, and utilities. In addition, at any given time different versions of the software may be installed. “Modules” is employed to define a default environment, which generally satisfies the needs of most users and eliminates the need for the user to create the environment. From time to time, a user may have a specific requirement that differs from the default environment.

In this example, the user wishes to use a version of the NETCDF library on the HPC Center’s Cray Xe6 (SALK) that is compiled with the Portland Group, Inc. (PGI) compiler instead of the installed default version, which was compiled with the Cray compiler. The approach to do this is:

:• Run '''module list''' to see what modules are loaded by default.
:• Determine what modules should be unloaded.
:• Determine what modules should be loaded.
:• Add the needed modules, i.e., '''module load'''

The first step, see what modules are loaded, is shown below.

'''user@SALK:~> module list'''
Currently Loaded Modulefiles:

1) modules/3.2.6.6
2) nodestat/2.2-1.0400.31264.2.5.gem
3) sdb/1.0-1.0400.32124.7.19.gem
4) MySQL/5.0.64-1.0000.5053.22.1
5) lustre-cray_gem_s/1.8.6_2.6.32.45_0.3.2_1.0400.6453.5.1-1.0400.32127.1.90
6) udreg/2.3.1-1.0400.4264.3.1.gem
7) ugni/2.3-1.0400.4374.4.88.gem
8) gni-headers/2.1-1.0400.4351.3.1.gem
9) dmapp/3.2.1-1.0400.4255.2.159.gem
10) xpmem/0.1-2.0400.31280.3.1.gem
11) hss-llm/6.0.0
12) Base-opts/1.0.2-1.0400.31284.2.2.gem
13) xtpe-network-gemini
14) cce/8.0.7
15) acml/5.1.0
16) xt-libsci/11.1.00
17) pmi/3.0.0-1.0000.8661.28.2807.gem
18) rca/1.0.0-2.0400.31553.3.58.gem
19) xt-asyncpe/5.13
20) atp/1.5.1
21) PrgEnv-cray/4.0.46
22) xtpe-mc8
23) cray-mpich2/5.5.3
24) pbs/11.3.0.121723

From the list, we see that the Cray Programming Environment ('''PrgEnv-cray/4.0.46''') and the Cray Compiler environment are loaded ('''cce/8.0.7''') by default. To unload these Cray modules and load in the PGI equivalents we need to know the names of the PGI modules. The '''module avail''' command shows this.

'''user@SALK:~> module avail'''
•
•
•

We see that there are several versions of the PGI compilers and two versions of the PGI Programming Environment installed. For this example, we are interested in loading PGI's 12.10 release (not the default, which is '''pgi/12.6''') and the most current release of the PGI programming environment ('''PrgEnv-pgi/4.0.46'''), which is the default.

The following module commands will unload the Cray defaults, load the PGI modules mentioned, and load version 4.2.0 of NETCDF compiled with the PGI compilers.

user@SALK:~> module unload PrgEnv-cray
user@SALK:~> module load PrgEnv-pgi
user@SALK:~> module unload pgi
user@SALK:~> module load pgi/12.10
user@SALK:~>
user@SALK:~> module load netcdf/4.2.0
user@SALK:~>
user@SALK;~> cc -V

/opt/cray/xt-asyncpe/5.13/bin/cc: INFO: Compiling with CRAYPE_COMPILE_TARGET=native.

pgcc 12.10-0 64-bit target on x86-64 Linux
Copyright 1989-2000, The Portland Group, Inc. All Rights Reserved.
Copyright 2000-2012, STMicroelectronics, Inc. All Rights Reserved.

A few additional comments:

:• The first three commands do not include version numbers and will therefore load or unload the current default versions.
:• In the third line, we unload the default version of the PGI compiler (version 12.6), which is loaded with the rest of the PGI Programming Environment in the second line. We then load the non-default and more recent release from PGI, version 12.10 in the fourth line.
:• Later, we load NETCDF version 4.2.0 which, because we have already loaded the PGI Programming Environment, will load the version of NETCDF 4.2.0 compiled with the PGI compilers.
:• Finally, we check to see which compiler the Cray "cc" compiler wrapper actually invokes after this sequence of module commands by again entering '''module list'''.