Data Storage and Management System - Revision history

Alex: /* /scratch */

2026-03-13T00:46:13Z

/scratch

← Older revision		Revision as of 00:46, 13 March 2026
Line 1:		Line 1:
	This file system is resides on Hybrid Parallel File System (HPFS).		This file system resides on '''Hybrid Parallel File System (HPFS).'''

	__TOC__		__TOC__
	=="Home" directories are on <font face="courier">/global/u</font>==		=="Home" directories are on <font face="courier">/global/u</font>==
	'''<font face="courier">/global/u</font>''' is ~~a standard~~ parallel high performance Linux file system based on HPE parallel file system (~~PFFS~~) It holds the home directories of individual users. When users request and are granted an allocation of HPC resources, they are assigned a '''<font face="courier"><font color="red"><userid></font color></font>''' and a 100 GB allocation of disk space for home directories on '''<font face="courier">/global/u/<font color="red"><userid></font color></font>'''. These '''home''' directories are on global file system ~~not~~ on local ~~disks at~~ nodes. That means that 1. data can be accesses only from PFFS and 2. no local storage on nodes is available. ~~The exception~~ are ~~few~~ nodes ~~in CONDO tier~~ that ~~are designed to~~ have local ~~"buffer" space~~. All home directories are backed up on weekly basis.		'''<font face="courier">/global/u</font>''' is partition in parallel high performance Linux file system based on HPE parallel file system (HPFS). It holds the home directories of '''all''' individual users. When users request and are granted an allocation of HPC resources, they are assigned a '''<font face="courier"><font color="red"><userid></font color></font>''' and a 100 GB allocation of disk space for home directories on '''<font face="courier">/global/u/<font color="red"><userid></font color></font>'''. These '''home''' directories are on global file system mounted only on login nodes. There is no local storage on nodes . That means that 1. data can be accesses only from PFFS and login node(s) 2. no local storage on nodes is available. Codes that write intermediate results on disk are typically slow and should be run only on condo nodes that have local disk storage. All home directories are backed up on weekly basis.

	==<font face="courier">/scratch</font>==		==<font face="courier">/scratch</font>==
	/Scratch is fast file system from which all jobs start. There is no quota on scratch, but '''scratch''' files are temporary and are '''<font color="red">not backed up</font color>'''. '''<font face="courier">/scratch</font>''' is used by jobs ~~queued for or in execution~~. Output from jobs may ONLY temporarily be ~~located in~~ '''<font face="courier">/scratch (up to 30 days).</font>'''		/Scratch is fast file system from which all jobs start. There is no quota on scratch, but '''scratch''' files are temporary and are '''<font color="red">not backed up</font color>'''. Single user cannot however take whole space with his/her data. That means users can run data sets that exceed their home space but they cannot use /scratch for storage. It is important to understand that '''<font face="courier">/scratch</font>''' <u>must be used only for submitting jobs.</u> Output from jobs may ONLY temporarily be stored on '''<font face="courier">/scratch (up to 10 days). Consequently in</font>''' order to submit a job for execution, a user must '''stage''' or '''mount''' the files required by the job to '''<font face="courier">/scratch</font>''' from '''<font face="courier">/global/u</font>''' using UNIX commands and/or from '''<font face="courier">SR1</font>''' using '''iRODS''' commands. The later is mounting data directly from project space. Note that SR is slower file system so large files is better to be staged from /home directories. Files in '''<font face="courier">/scratch</font>''' are '''automatically relocated to local archive''' when their inactive residence on scratch exceeds 90 days. Local archive has limited capacity and serves as data buffer, so '''strict policies of cleaning up temporary archive are in place'''. Upon relocation the user will get a warning ( via e-mail) and must either move files to his/her home directory or to SR1. Note that files left in temp archive will be purged after 30 days. Users must not '''store valuable data or compiled codes on scratch''' since these are static type of files.

	In order to submit a job for execution, a user must '''stage''' or '''mount''' the files required by the job to '''<font face="courier">/scratch</font>''' from '''<font face="courier">/global/u</font>''' using UNIX commands and/or from '''<font face="courier">SR1</font>''' using '''iRODS''' commands. The later is moaning data directly from project space.

	Files in '''<font face="courier">/scratch</font>''' on a system are '''automatically relocated to local archive''' when file(s)' residence on scratch exceeds 90 days. Active file are not relocated. Thus user must not store valuable data or compiled codes on scratch since the latter are static type of files. Upon move the user get warning from HPCC and must either move files to his/her home directory. Files in temporary archive left for 45 days are removed.


Line 18:		Line 15:
	'''“Project”''' directories are managed through '''iRODS''' and accessible through iRODS commands, not standard UNIX commands. In iRODS terminology, a “collection” is the equivalent of “directory”.		'''“Project”''' directories are managed through '''iRODS''' and accessible through iRODS commands, not standard UNIX commands. In iRODS terminology, a “collection” is the equivalent of “directory”.

	A '''“Project”''' is an activity that usually involves multiple users and/or many individual data files. A '''“Project”''' is normally led by a “Principal Investigator” (PI), who is a faculty member or a research scientist. The PI is the individual responsible to the University or a granting agency for the “Project”. The PI has overall responsibility for “Project” data and “Project” data management. To establish a Project, the PI completes and submits the online “Project Application Form”. Project data are stored on project space on main file system. Valuable parts of the projects must be curated by PI's and stored in HPCC archive.		A '''“Project”''' is an activity that usually involves multiple users and/or many individual data files. A '''“Project”''' is normally led by a “Principal Investigator” (PI), who is a faculty member or a research scientist. The PI is the individual responsible to the University or a granting agency for the “Project”. The PI has overall responsibility for “Project” data and “Project” data management. To establish a Project, the PI completes and submits the online “Project Application Form”. Project data are stored on project space on main file system. Valuable parts of the projects must be curated by PI's and stored in HPCC long term archive.

	<br />		<br />

Alex: /* "Home" directories are on /global/u */

2026-03-13T00:25:45Z

"Home" directories are on /global/u

← Older revision		Revision as of 00:25, 13 March 2026
Line 3:		Line 3:
	__TOC__		__TOC__
	=="Home" directories are on <font face="courier">/global/u</font>==		=="Home" directories are on <font face="courier">/global/u</font>==
	'''<font face="courier">/global/u</font>''' is a standard ~~Unix~~ file system ~~that~~ holds the home directories of individual users. When users request and are granted an allocation of HPC resources, they are assigned a '''<font face="courier"><font color="red"><userid></font color></font>''' and a 50 GB allocation of disk space for home directories on '''<font face="courier">/global/u/<font color="red"><userid></font color></font>'''. These '''home''' directories are on ~~the '''DSMS''',~~ not on ~~the HPC systems, but~~ can be ~~accessed~~ from ~~any Center system~~. All home directories are backed up on weekly basis.		'''<font face="courier">/global/u</font>''' is a standard parallel high performance Linux file system based on HPE parallel file system (PFFS) It holds the home directories of individual users. When users request and are granted an allocation of HPC resources, they are assigned a '''<font face="courier"><font color="red"><userid></font color></font>''' and a 100 GB allocation of disk space for home directories on '''<font face="courier">/global/u/<font color="red"><userid></font color></font>'''. These '''home''' directories are on global file system not on local disks at nodes. That means that 1. data can be accesses only from PFFS and 2. no local storage on nodes is available. The exception are few nodes in CONDO tier that are designed to have local "buffer" space. All home directories are backed up on weekly basis.

	==<font face="courier">/scratch</font>==		==<font face="courier">/scratch</font>==
	~~Disk storage~~ on ~~the HPC systems is used only for '''~~scratch~~''' files.~~ '''scratch''' files are temporary and are '''<font color="red">not backed up</font color>'''. '''<font face="courier">/scratch</font>''' is used by jobs queued for or in execution. Output from jobs may temporarily be located in '''<font face="courier">/scratch</font>'''.		/Scratch is fast file system from which all jobs start. There is no quota on scratch, but '''scratch''' files are temporary and are '''<font color="red">not backed up</font color>'''. '''<font face="courier">/scratch</font>''' is used by jobs queued for or in execution. Output from jobs may ONLY temporarily be located in '''<font face="courier">/scratch (up to 30 days).</font>'''

	In order to submit a job for execution, a user must '''stage''' or '''mount''' the files required by the job to '''<font face="courier">/scratch</font>''' from '''<font face="courier">/global/u</font>''' using UNIX commands and/or from '''<font face="courier">SR1</font>''' using '''iRODS''' commands.		In order to submit a job for execution, a user must '''stage''' or '''mount''' the files required by the job to '''<font face="courier">/scratch</font>''' from '''<font face="courier">/global/u</font>''' using UNIX commands and/or from '''<font face="courier">SR1</font>''' using '''iRODS''' commands. The later is moaning data directly from project space.

	Files in '''<font face="courier">/scratch</font>''' on a system are '''automatically ~~purged~~''' when (1) ~~usage reaches 70% of available space, or (2) file~~ residence on scratch exceeds ~~two weeks, whichever occurs first~~.		Files in '''<font face="courier">/scratch</font>''' on a system are '''automatically relocated to local archive''' when file(s)' residence on scratch exceeds 90 days. Active file are not relocated. Thus user must not store valuable data or compiled codes on scratch since the latter are static type of files. Upon move the user get warning from HPCC and must either move files to his/her home directory. Files in temporary archive left for 45 days are removed.


Line 16:		Line 16:
	==“Project” directories==		==“Project” directories==

	'''“Project”''' directories are managed through '''iRODS''' and accessible through iRODS commands, not standard UNIX commands. In iRODS terminology, a “collection” is the equivalent of “directory”.		'''“Project”''' directories are managed through '''iRODS''' and accessible through iRODS commands, not standard UNIX commands. In iRODS terminology, a “collection” is the equivalent of “directory”.

	A '''“Project”''' is an activity that usually involves multiple users and/or many individual data files. A '''“Project”''' is normally led by a “Principal Investigator” (PI), who is a faculty member or a research scientist. The PI is the individual responsible to the University or a granting agency for the “Project”. The PI has overall responsibility for “Project” data and “Project” data management. To establish a Project, the PI completes and submits the online “Project Application Form”.		A '''“Project”''' is an activity that usually involves multiple users and/or many individual data files. A '''“Project”''' is normally led by a “Principal Investigator” (PI), who is a faculty member or a research scientist. The PI is the individual responsible to the University or a granting agency for the “Project”. The PI has overall responsibility for “Project” data and “Project” data management. To establish a Project, the PI completes and submits the online “Project Application Form”. Project data are stored on project space on main file system. Valuable parts of the projects must be curated by PI's and stored in HPCC archive.

	~~Additional information on the '''DSMS''' is available in Section 4 of the User Manual~~ <br />		<br />
	~~http://www.csi.cuny.edu/cunyhpc/pdf/User_Manual.pdf~~


Line 37:		Line 37:
	</font>		</font>

	If working with '''SR1''':<br />		If working with '''SR1 (storage repository)''':<br />
	~~<font face="courier">~~
	cd /scratch/<font color="red"><userid></font color>		cd /scratch/<font color="red"><userid></font color>
	mkdir <font color="red">mySLURM_Job</font color> && cd <font color="red">mySLURM_Job</font color>		mkdir <font color="red">mySLURM_Job</font color> && cd <font color="red">mySLURM_Job</font color>

Alex at 17:04, 21 December 2024

2024-12-21T17:04:02Z

← Older revision		Revision as of 17:04, 21 December 2024
Line 66:		Line 66:
	<font face="courier">		<font face="courier">
	sbatch ./<font color="red">mySLURM_script</font color>		sbatch ./<font color="red">mySLURM_script</font color>
	~~</font>~~


Line 75:		Line 74:
	mv ./myoutputs /global/u/<font color="red"><userid></font color>/<font color="red">myProject</font color>/.		mv ./myoutputs /global/u/<font color="red"><userid></font color>/<font color="red">myProject</font color>/.
	cd ../		cd ../
	rm -rf <font color="red">mySLURM_Job</font color>		rm -rf <font color="red">mySLURM_Job</font color> </font>
	~~</font>~~		== iRODS (The iRODS Section is in REVIEW and may not be CURRENT) ==

	~~If working with '''SR1''':~~<br />
	<font ~~face="courier"~~>
	~~iput ./myoutputs <font color="red">myProject</font color>/.~~
	~~cd ../~~
	~~rm -rf <font color="red">mySLURM_Job</font color>~~
	~~</font>~~


	~~5. If output files are stored in '''SR1''' tag them with metadata.<br />~~
	~~<font face="courier">~~
	~~imeta addw -d myoutput zvalue 15 meters~~
	~~imeta addw -d myoutput colorLabel RED~~
	~~</font>~~
	~~<br />~~

	== iRODS (~~THE FOLLOWING SECTION~~ is in REVIEW) ==

	'''iRODS''' is the integrated Rule-Oriented Data-management System, a		'''iRODS''' is the integrated Rule-Oriented Data-management System, a
Line 413:		Line 395:
	<br />		<br />

	==~~Good data handling practices~~==		==Data Handling Practices ==
	===HPFS, i.e., /global/u ===		===HPFS, i.e., /global/u ===

Alex: /* Data retention and account expiration policy */

2024-12-21T02:12:03Z

Data retention and account expiration policy

← Older revision		Revision as of 02:12, 21 December 2024
Line 345:		Line 345:

	Possible levels of access to a data object are null/read/write/own.		Possible levels of access to a data object are null/read/write/own.
			==Backups (IN REVIEW)==

			'''Backups.''' '''<font face="courier">/global/u</font>''' user directories. Project files are backed up automatically to a remote tape silo system over a fiber optic network. Backups are performed daily.

	~~<br />~~		If the user deletes a file from '''<font face="courier">/global/u</font>''', it will remain on the tape silo system for 30 days, after which it will be deleted and cannot be recovered. If a user, within the 30 day window finds it necessary to recover a file, the user must expeditiously submit a request to [mailto:hpchelp@csi.cuny.edu hpchelp@csi.cuny.edu].

	~~==Backups==~~

	'''Backups.''' '''<font face="courier">/global/u</font>''' user directories and '''<font face="courier">SR1</font>''' Project files are backed up automatically to a remote tape silo system over a fiber optic network. Backups are performed daily.

	If the user deletes a file from '''<font face="courier">/global/u~~</font>''' or '''<font face="courier">SR1~~</font>''', it will remain on the tape silo system for 30 days, after which it will be deleted and cannot be recovered. If a user, within the 30 day window finds it necessary to recover a file, the user must expeditiously submit a request to [mailto:hpchelp@csi.cuny.edu hpchelp@csi.cuny.edu].

	Less frequently accessed files are automatically transferred to the HPC Center robotic tape system, freeing up space in the disk storage pool and making it available for more actively used files. The selection criteria for the migration are age and size of a file. If a file is not accessed for 90 days, it may be moved to a tape in the tape library – in fact to two tapes, for backup. This is fully transparent to the user. When a file is needed, the system will copy the file back to the appropriate disk directory. No user action is required.		Less frequently accessed files are automatically transferred to the HPC Center robotic tape system, freeing up space in the disk storage pool and making it available for more actively used files. The selection criteria for the migration are age and size of a file. If a file is not accessed for 90 days, it may be moved to a tape in the tape library – in fact to two tapes, for backup. This is fully transparent to the user. When a file is needed, the system will copy the file back to the appropriate disk directory. No user action is required.
	~~<br />~~

			==Data retention and account expiration policy (IN REVIEW)==


	==Data retention and account expiration policy==

	Project directories on SR1 are retained as long as the project is active. The HPC Center will coordinate with the Principal Investigator of the project before deleting a project directory. If the PI is no longer with CUNY, the HPC Center will coordinate with the PI’s departmental chair or Research Dean, whichever is appropriate.		Project directories on SR1 are retained as long as the project is active. The HPC Center will coordinate with the Principal Investigator of the project before deleting a project directory. If the PI is no longer with CUNY, the HPC Center will coordinate with the PI’s departmental chair or Research Dean, whichever is appropriate.

	For user accounts, current user directories under /global/u are retained as long as the account is active. If a user account is inactive for one year, the HPC Center will attempt to contact the user and request that the data be removed from the system. If there is no response from the user within three months of the initial notice, or if the user cannot be reached, the user directory will be purged.		For user accounts, current user directories under /global/u are retained as long as the account is active. If a user account is inactive for one year, the HPC Center will attempt to contact the user and request that the data be removed from the system. If there is no response from the user within three months of the initial notice, or if the user cannot be reached, the user directory will be purged.
	~~<br />~~




	==DSMS Technical Summary (IN REVIEW)==		==DSMS Technical Summary (IN REVIEW)==

Alex at 02:10, 21 December 2024

2024-12-21T02:10:22Z

← Older revision		Revision as of 02:10, 21 December 2024
Line 1:		Line 1:
	~~This section describes the use of Data Storage Management System (DSMS) which serves only Penzias, Karle and Appel.~~ This file system is ~~separate from Global~~ Hybrid File System (~~GHFS~~) ~~which serves Arrow and all Condominium servers~~.		This file system is resides on Hybrid Parallel File System (HPFS).

	__TOC__		__TOC__

	~~Key features of the DSMS system include:~~

	~~:• '''User''' home directories in a standard Unix file system called /global/u.~~
	~~:• Enhanced parallel scratch space on the HPC systems.~~
	:• '''Project''' directories in an Integrated Rule-Oriented Data-management System (iRODS) managed resource. Project directories exist in a “virtual file space” called '''cunyZone''' which contains a resource called '''Storage Resource 1 (SR1)'''. For the purpose of this document, we will use the terminology SR1 to describe '''Project file space.'''
	~~:• Automated backups.~~


	~~The '''DSMS''' is the HPC Center’s primary file system and is accessible from all existing HPC systems, except for '''HERBERT''' . It will similarly be accessible from all future HPC systems.~~

	~~The '''DSMS''' provides a 3-level data storage infrastructure: - '''HOME''' filesystem, '''SCRATCH''' filesystems, '''SR1''' (long-tern storage resource)~~

	~~'''DSMS''' features are explained below.~~


	=="Home" directories are on <font face="courier">/global/u</font>==		=="Home" directories are on <font face="courier">/global/u</font>==
	'''<font face="courier">/global/u</font>''' is a standard Unix file system that holds the home directories of individual users. When users request and are granted an allocation of HPC resources, they are assigned a '''<font face="courier"><font color="red"><userid></font color></font>''' and a 50 GB allocation of disk space for home directories on '''<font face="courier">/global/u/<font color="red"><userid></font color></font>'''. These '''home''' directories are on the '''DSMS''', not on the HPC systems, but can be accessed from any Center system. All home directories are backed up on weekly basis.		'''<font face="courier">/global/u</font>''' is a standard Unix file system that holds the home directories of individual users. When users request and are granted an allocation of HPC resources, they are assigned a '''<font face="courier"><font color="red"><userid></font color></font>''' and a 50 GB allocation of disk space for home directories on '''<font face="courier">/global/u/<font color="red"><userid></font color></font>'''. These '''home''' directories are on the '''DSMS''', not on the HPC systems, but can be accessed from any Center system. All home directories are backed up on weekly basis.
Line 109:		Line 93:
	<br />		<br />

	== iRODS ==		== iRODS (THE FOLLOWING SECTION is in REVIEW) ==

	'''iRODS''' is the integrated Rule-Oriented Data-management System, a		'''iRODS''' is the integrated Rule-Oriented Data-management System, a
Line 387:		Line 371:


	==DSMS Technical Summary==		==DSMS Technical Summary (IN REVIEW)==
	{\| class="wikitable"		{\| class="wikitable"
	\|+		\|+
Line 442:		Line 426:

	==Good data handling practices==		==Good data handling practices==
	===~~DSMS~~, i.e., /global/u ~~and SR1~~===		===HPFS, i.e., /global/u ===

	• The ~~'''DSMS'''~~ is not an archive for non-HPC users. It is an archive for users who are processing data at the HPC Center. “Parking” files on the '''~~DSMS~~''' as a back-up to local data stores is prohibited.		• The HPFS is not an archive for non-HPC users. It is an archive for users who are processing data at the HPC Center. “Parking” files on the '''HPFS''' as a back-up to local data stores is prohibited.

	• Do not store more than 1,000 files in a single directory. Store collections of small files into an archive (for example, tar). Note that for every file, a stub of about 4MB is kept on disk even if the rest of the file is migrated to tape, meaning that even migrated files take up some disk space. It also means that files smaller than the stub size are never migrated to tape because that would not make sense. Storing a large number of small files in a single directory degrades the file system performance.		• Do not store more than 1,000 files in a single directory. Store collections of small files into an archive (for example, tar). Note that for every file, a stub of about 4MB is kept on disk even if the rest of the file is migrated to tape, meaning that even migrated files take up some disk space. It also means that files smaller than the stub size are never migrated to tape because that would not make sense. Storing a large number of small files in a single directory degrades the file system performance.

Alex: /* Typical Workflow */

2023-06-30T02:18:22Z

Typical Workflow

← Older revision		Revision as of 02:18, 30 June 2023
Line 42:		Line 42:

	==Typical Workflow==		==Typical Workflow==
	Typical workflows in are described below:		Typical workflows for Penzias Appel and Karle in are described below:

	1. Copying files from a user’s home directory or from '''SR1''' to '''SCRATCH'''.<br />		1. Copying files from a user’s home directory or from '''SR1''' to '''SCRATCH'''.<br />

Alex: /* Penzias, Karle and Appel Global Data Storage (GDS) */

2023-06-30T02:17:52Z

Penzias, Karle and Appel Global Data Storage (GDS)

← Older revision		Revision as of 02:17, 30 June 2023
Line 1:		Line 1:
	=Penzias, Karle and Appel Global ~~Data Storage~~ (~~GDS~~)=		This section describes the use of Data Storage Management System (DSMS) which serves only Penzias, Karle and Appel. This file system is separate from Global Hybrid File System (GHFS) which serves Arrow and all Condominium servers.

	__TOC__		__TOC__

	Key features of the ~~GDS~~ system include:		Key features of the DSMS system include:

	:• '''User''' home directories in a standard Unix file system called /global/u.		:• '''User''' home directories in a standard Unix file system called /global/u.

Alex: /* Global Data Storage (GDS) */

2023-06-29T03:47:37Z

Global Data Storage (GDS)

← Older revision		Revision as of 03:47, 29 June 2023
Line 1:		Line 1:
	=Global Data Storage (GDS)=		=Penzias, Karle and Appel Global Data Storage (GDS)=

	__TOC__		__TOC__

Alex: /* Data Storage and Management System (DSMS) */

2023-04-17T16:13:38Z

Data Storage and Management System (DSMS)

← Older revision		Revision as of 16:13, 17 April 2023
Line 1:		Line 1:
	=Data Storage ~~and Management System~~ (~~DSMS~~)=		=Global Data Storage (GDS)=

	__TOC__		__TOC__

	Key features of the ~~'''DSMS'''~~ system include:		Key features of the GDS system include:

	:• '''User''' home directories in a standard Unix file system called /global/u.		:• '''User''' home directories in a standard Unix file system called /global/u.

James: Text replacement - "[pP][bB][sS]" to "SLURM"

2022-11-07T17:33:06Z

Text replacement - "[pP][bB][sS]" to "SLURM"

← Older revision		Revision as of 17:33, 7 November 2022
Line 48:		Line 48:
	<font face="courier">		<font face="courier">
	cd /scratch/<font color="red"><userid></font color>		cd /scratch/<font color="red"><userid></font color>
	mkdir <font color="red">mySLURM_Job</font color> && cd <font color="red">~~myPBS_Job~~</font color>		mkdir <font color="red">mySLURM_Job</font color> && cd <font color="red">mySLURM_Job</font color>
	cp /global/u/<font color="red"><userid></font>/<font color="red">myProject</font>/a.out ./		cp /global/u/<font color="red"><userid></font>/<font color="red">myProject</font>/a.out ./
	cp /global/u/<font color="red"><userid></font>/<font color="red">myProject</font>/<font color="red">mydatafile </font>./		cp /global/u/<font color="red"><userid></font>/<font color="red">myProject</font>/<font color="red">mydatafile </font>./
Line 56:		Line 56:
	<font face="courier">		<font face="courier">
	cd /scratch/<font color="red"><userid></font color>		cd /scratch/<font color="red"><userid></font color>
	mkdir <font color="red">~~myPBS_Job~~</font color> && cd <font color="red">~~myPBS_Job~~</font color>		mkdir <font color="red">mySLURM_Job</font color> && cd <font color="red">mySLURM_Job</font color>
	iget <font color="red">myProject</font color>/a.out		iget <font color="red">myProject</font color>/a.out
	iget <font color="red">myProject</font color>/<font color="red">mydatafile</font color>		iget <font color="red">myProject</font color>/<font color="red">mydatafile</font color>
Line 91:		Line 91:
	mv ./myoutputs /global/u/<font color="red"><userid></font color>/<font color="red">myProject</font color>/.		mv ./myoutputs /global/u/<font color="red"><userid></font color>/<font color="red">myProject</font color>/.
	cd ../		cd ../
	rm -rf <font color="red">~~myPBS_Job~~</font color>		rm -rf <font color="red">mySLURM_Job</font color>
	</font>		</font>

Line 98:		Line 98:
	iput ./myoutputs <font color="red">myProject</font color>/.		iput ./myoutputs <font color="red">myProject</font color>/.
	cd ../		cd ../
	rm -rf <font color="red">~~myPBS_Job~~</font color>		rm -rf <font color="red">mySLURM_Job</font color>
	</font>		</font>