HPCC Wiki - User contributions [en]

File:Bluemoon-Rack2.jpg

2025-06-02T20:10:56Z

Bimal:

MediaWiki:Sidebar

2025-01-03T16:06:01Z

Bimal:

* navigation
** Main Page | ◊ Overview of the CUNY HPCC
** Administrative Information | ◊ Administrative Information

** How to Login | ◊ How to Login
** Basic Unix & Linux Commands | ◊ Basic Unix & Linux Commands
** File Transfers | ◊ File Transfers
** Running Jobs | ◊ Running Jobs
** Data Storage and Management System | ◊ Data Store and Management System
** Modules and Available Third Party Software | ◊ Modules and Available Third Party Software
** Applications | ◊ Applications
** Program Compilation | ◊ Program Compilation
** Training & Workshops | ◊ Training & Workshops
** FAQs | ◊ FAQs
** Contact | ◊ Contact
** Forbidden Practices | ◊ Forbidden Practices
 
* SEARCH
* TOOLBOX
* LANGUAGES

How to Login

2025-01-03T15:39:07Z

Bimal: /* Logging in from Unix/Linux */

<blockquote>''Notice: Users may not access CUNY computer resources without authorization or use it for purposes beyond the scope of authorization. This includes attempting to circumvent CUNY computer resource system protection facilities by hacking, cracking or similar activities, accessing or using another person's computer account, and allowing another person to access or use the user's account. CUNY computer resources may not be used to gain unauthorized access to another computer system within or outside of CUNY. Users are responsible for all actions performed from their computer account that they permitted or failed to prevent by taking ordinary security precautions.''</blockquote>

__TOC__

For security reasons, CUNY only allows users to communicate using SSH. Secure Shell (abbreviated SSH) is a secure means of connecting to a remote server over an encrypted channel. SSH is a protocol designed to allow logging into a remote machine and executing commands on a remote machine using improved secure encrypted communication between two non-trusted hosts over an insecure network, while other protocols like Telnet cannot.

The HPC systems located at the CUNY HPCC accept IP addresses only from the CSI campus. Users not located on the CSI campus must first log into an gateway server. The gateway server for the HPCC is <code>chizen.csi.cuny.edu</code>.

'''To log into any of the the HPC systems, the user must then ssh from <code>chizen.csi.cuny.edu</code> to the desired HPC system. Currently, HPCC uses single login node MHN. Thus users must ssh to MHN after they login to chizen (gateway).'''
 
== Logging in from windows machine ==

If you are using Windows machine locally you need to have SSH client installed on it. While many SSH clients may exist, CUNY strongly recommends the use of CygWin emulator. [http://www.cygwin.com/ Cygwin]. The popular windows clients like [http://winscp.net/eng/index.php WinSCP] or [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY] can also be used as SSH client. Note that PuTTY scp and WinSCP may not work as Unix scp. Once the suitable SSH client is installed,
users can run it and connect to HPCC. Using the above links you may find documentations on these applications.
 
== Logging in from Unix/Linux ==
On Unix/Linux machines, the user should use ssh to log in to the HPCC systems. chizen is the gateway to all HPCC servers. The command: <syntaxhighlight lang="abap">
$ ssh <userid>@chizen.csi.cuny.edu
</syntaxhighlight>will log you onto this authentication server. Once logged the users must '''do second login''' to actual login node on production server. '''No jobs can be submitted from gateway.''' HPCC uses single login node so users must login to MHN (Master Head Node) as it shown on following example. Please read carefully message of the day (in red below) since it holds critical information. <syntaxhighlight lang="abap">
ssh arrow
Password:
Last login: Tue Apr 18 10:22:51 2023 from 163.238.128.59

+---------------------------------------------------------------------------------------+
| |
| Please NOTE |
| |
| 1. Running production jobs on login node(s) is strictly forbidden. |
| |
| 2. Access to server may be limited or denied if abuse of above policy is recorded. |
| |
| 3. All jobs must be submitted via SLURM batch system. All jobs running on login |
| node will be killed and output data will be lost. |
| |
| |
| Please report all non-emergency probems by opening a ticket at: |
| https://hpchelp.csi.cuny.edu |
| |
| Users who cannot open a ticket, but need further help should send E-mail to |
| hpchelp@csi.cuny.edu |
| |
+---------------------------------------------------------------------------------------+
</syntaxhighlight>When connecting to any of HPCC hosts for the first time you are asked to validate the authenticity of the key presented by that host. Once you answer yes, that key will be stored. Future logging attempts to that same server will check they key against what is stored in the file:
~/.ssh/known_hosts

In a very rare cases when our team performs reinstallation of the OS host identification will change. If we perform this sort of maintenance all users get a notice from HPC team.

When this happens you get message similar to this:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
5c:0b:18:56:b6:cd:12:10:32:cd:1d:a2:9a:cd:e5:1c.
Please contact your system administrator.
Add correct host key in /home/userid/.ssh/known_hosts to get rid of this message.
Offending key in /home/user/.ssh/known_hosts:3
RSA host key for chizen.csi.cuny.edu has changed and you have requested strict checking.
Host key verification failed.

This reads as "remote host information that is kept in your /.ssh/known_hosts at line #3 does not match that remote host, therefore ssh connection can not be established".

To get rid of this message you need to modify you ~/.ssh/known_hosts. Open it in your favorite text editor and delete line #3. After that try to ssh again. You will be asked if you want to save host identification (which is -- if you want to add host identification to your ~/.ssh/known_hosts). Answer "yes" and proceed normally.

However if you get the above ssh warning without our maintenance notice it will be a good idea to contact HPC stuff.

 

== X11 Forwarding or Tunneling==

X11 forwarding is required when logging in to a remote location, but an application GUI must be display locally. This could be done with [[#Mathematica|Mathematica]] for instance, if using the command-line interface was not acceptable.

=== UNIX clients ===
X11 forwarding or tunneling back through the 'ssh' connection can be provided by including the flag '-X' to 'ssh' command. For users off the CSI campus, the following forwards X11 traffic back from the HPCC gateway system, CHIZEN, to your desktop:

$ ssh -XY <userid>@chizen.csi.cuny.edu

If you need to then login to ANDY to run Mathematica you will have forward X11 traffic again through the second connection with:

$ ssh -XY <userid>@andy.csi.cuny.edu

Note that double-forwarding will be significantly slower and may make working with a GUI from outside of CSI campus inconvenient.

=== WINDOWS clients ===
In order to allow X11 forwarding for Windows-based client CUNY recommends to install [http://sourceforge.net/projects/xming/ Xming] -- X Server for Windows. Once Xming is downloaded and installed users should

:* start the server
:* [[#Logging from windows machine|connect]] to remote machine using [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY] with X11 forwarding enabled:
[[Image:configputty.JPG]]
:* once the connection is established start your X application. For example, type in console <pre>xterm</pre> This will give you xterm session.

Administrative Information

2024-12-19T13:54:53Z

Bimal: /* Reset Password */

__TOC__
==How to get an account==
===User accounts overview===
HPCC supports '''5''' types of accounts: '''a)''' CUNY full time faculty and research staff '''b)''' CUNY adjunct faculty and master students '''c)''' doctoral graduate students '''d)''' undergraduate students (must have sponsor) '''e)''' collaborators from other universities working with CUNY faculty, and '''f)''' public and private sector partners directly collaborating with CUNY faculty/department/school/center. Users from all '''a) to f)''' groups may receive authorization to use the CUNY HPC Center systems. Applications for accounts are accepted at any time. The accounts in groups '''a)''' and '''c)''' are valid for one year and must be renewed on or before Sept 30th each year. Accounts from '''b)''' and '''d)''' are valid for one semester and must be renewed at the beginning of fall and spring semester. The accounts in groups '''e)''' and '''f)''' are good for the duration of collaborative project. In addition non-CUNY researchers can obtain a research account at CUNY-HPCC and use the resources by paying cost recovery fee (proportional to use). Please contact the director of HPCC for details.

A user account is issued to an '''''individual user'''.'' Accounts are '''not to be shared'''. Users are responsible for choose secure passwords and to protect their passwords. Passwords '''are not to be shared.''' Users from '''a), b), c)''' and '''d)''' groups must have and use a '''valid CUNY e-mail address''' when register to HPC. Public mail accounts such as gmail, hotmail, yahoo ,outlook etc. can be used only as a second (backup) e-mail. Users in group '''e)''' must provide an e-mail which can be verified and state the CUNY collaborator e-mail as second mailbox. The users from group '''f)''' and outside users doing research on CUNY HPCC must provide valid professional e-mail address of their CUNY counterpart(s) including valid CUNY e-mail(s). Note that HPCC '''will not send''' warning and any messages to non CUNY e-mails unless is needed for emergency and registered users from groups '''e)''' and '''f).'''

All users of HPCC resources must register with HPCC for a type of account (described above). Accounts will be renewed with the same time of account unless users notifies the HPCC via e-mail to '''hpchelp@csi.cuny.edu''' for changes it his/her/them academic status. To do so all users must provide '''required information (see the list below) directly to HPC''' via '''hpchelp@csi.cuny.edu''' e-mail. It is mandatory to specify information (or NA) on all points from below list. Please do not forget to provide information about past and pending '''publications''' and funded projects and information about your local available resources (local servers and workstations/desktops only). Think carefully about the resources needed and try to estimate as accurate as possible. Note that '''by applying for and obtaining an account, the user agrees to the Center’s [https://cunyhpc.csi.cuny.edu/acceptableuse Acceptable Use Policy] and the [https://cunyhpc.csi.cuny.edu/passwordpolicy User Account and Password Policy].'''

''1. Full name as stated at the CUNY ID card:''

''2. For group '''e)''' only - Full name as stated at the University ID card (Jeanette Smith):''

''3. For group '''f)''' only- Full name as stated on State ID or Federal ID (e.g. Smith John Peter):''

''4. CUNY EID and valid CUNY e-mail (e.g. John A. Smith 22341356 jsmith@csi.cuny.edu):''

''5. Affiliation within CUNY - campus name and Department ( e.g Hunter College, Biology):''

''6. Affiliation outside CUNY if any(e.g. Rutgers University) and valid professional e-mail: ( e.g. John Doe, Rutgers University, jd@rutgers.edu)''

''7. Department at above (6) institution:''

''8. Second CUNY affiliation - campus name and Department (e.g. Graduate Center, Biology):''

''9. E-mail at (6):''

''10. Academic status ( faculty, adjunct faculty, graduate student, undergraduate student, research staff, collaborator to CUNY researcher, partners , external researcher):''

''11. Brief project description and project duration. In case of teaching/participating in class please state class number and semester (e.g. CS 456, fall 2023):''

''12. Comma separated list of Principal investigator(s) or research advisor(s) name, status, campus and department:''

''13. Resources needed:''

''- cpu cores''

''- GPU cores''

''- cpu hours''

''- GPU hours''

''- storage GB''

14.. '''If out of College of Staten Island:''' description of available local resources. Please state NONE if you do not have access to local computational resources. Otherwise provide:

''- type of computational (cluster, advanced workstation):''

''- number of nodes:''

''- number of cores per node:''

''-- memory per node:''

''- total number of GPU for server:''

''- number of GPU per node:''

''- type of GPU (list of all types e.e. 2 x K20m, 4 x K80):''

''15. Consent that HPCC will be cited properly (see our wiki for details) in all your published work including conferences and talks.''

Upon creation every research user account is provided with 50 GB (and max of 10000 files on /global/u) home directory mounted as /global/u/<userid>. If required, a user may request an increase in the size of their home directory. The HPC Center will endeavor to satisfy reasonable requests. If you expect to have more than 10 000 files please zip several small files into larger single zip file. Please keep only wrangled information in your space in order to optimize use of the existing storage.

Student class accounts ( group d)) are provided with 10 GB home directory. Please note that class accounts and data will be deleted 30 days after the semester ends (unless otherwise agreed upon). Students are responsible for backing up their own data prior to the end of the semester.

When a user account is established, only the user has read/write to his files. The user can change his UNIX permissions to allow others in his group to read/write to his file.

Please be sure to notify the HPC Center if user accounts need to be removed or added to a specific re group. Please read below a policies for accounts. Note that accounts are not timeless and non accessed and non active accounts are removed ( see below).

=== User accounts policies ===
CUNY HPCC applies strict security standards in user accounts management. HPCC uses “account periods”. Account period is '''one year''' for accounts in '''a,c,e, and f''' type of accounts and '''one semester''' for '''b''' and '''d''' type of accounts. All accounts are periodically reviewed and inactive accounts are removed. All student accounts will expire automatically and will be removed after each semester unless student’s advisor request extension of the student’s account. All user accounts for groups '''a,c,e and f''' must be renewed once a year by Sept 30th. All user accounts in groups '''b''' and '''d''' must be renewed within 2 weeks after each semester . Accounts not accessed for one account period and/or not renewed are automatically disabled/locked and will be deleted 60 days from locking. Delete of particular account means unrecoverable remove of all data associated with that account.

===Reset Password ===

Users must use automatic password reset system. Click on [https://hpcauth1.csi.cuny.edu/reset/ Reset Password]. Upon resetting the users will get their individual security token on the e-mail address registered with HPCC.

===Close of account===
If a user would like to close their account, please contact the CUNY at HPCHelp@csi.cuny.edu.
Supervisors that would like to modify the access of their researchers and/or students working for them should contact the HPC to remove, add or modify access.
User accounts that are not accessed or renewed for more than a year and one day will be purged along with any data associated with the account. User accounts that are not renewed on time will be locked and users must contact HPCC to get access recovered.

=== Message of the day (MOTD) ===
Users are encouraged to read the "Message of the day” (MOTD), which is displayed to the user upon logging onto a system. The MOTD provides information on scheduled maintenance time when systems will be unavailable and/or important changes in the environment that are of import to the user community. The MOTD is the HPC Center’s only efficient mechanism for communicating to the broader user community as bulk email messages are often blocked by CUNY SPAM filters.

=== Required citations ===
The CUNY HPC Center appreciates the support it has received from the National Science Foundation (NSF). It is the policy of NSF that researchers who are funded by NSF or who make use of facilities funded by NSF acknowledge the contribution of NSF by including the following citation in their papers and presentations:

'''''This research was supported, in part, under National Science Foundation Grants: CNS-0958379, CNS-0855217, ACI-1126113 and OAC-2215760 (2022) and the City University of New York High Performance Computing Center at the College of Staten Island.'''''

The HPC Center, therefore, requests its users to follow this procedure as it helps the Center to demonstrate that NSF’s investments aided the research and educational missions of the University.

== Reporting requirements ==
The Center reports on its support of the research and educational community to both funding agencies and CUNY on an annual basis. Citations are an important factor that is included in these reports. Therefore, is mandatory users to send copies of research papers developed, in part, using the HPC Center resources to [mailto:hpchelp@csi.cuny.edu hpchelp@csi.cuny.edu]. Accounts for users who violate that requirement may not be renewed. Reporting results obtained with HPC resources also helps the Center to keep abreast of user research requirement directions and needs.

== Funding of computational resources and storage ==
Systems at HPC Center are purchased with grants from National Science Foundation (NSF), grants from NYC, a grant from DASNY and a grant from CUNY's office of the CIO. In addition all systems in condo tier are purchased with direct funds from research groups. The largest financial support comes from '''NSF MRI grants (more than 80% of all funding).''' CUNY own investment constitute '''8.6%''' of all funds. Here is the list of all grants for CUNY-HPCC.

:'''PFSS and GPU Nodes:''' NSF Grant OAC-2215760 (operational)
:'''DSMS''' NSF Grant ACI-1126113 (server is partially retired)
:'''BLUE MOON''', Grant NYC 042-ST030-015 (operational)
:'''CRYO,''' Grant DASNY 208684-000 OP (operational)
:'''ANDY''', NSF Grant CNS-0855217 and the New York City Council through the efforts of Borough President James Oddo ( server is fully retired)
:'''APPEL''', New York State Regional Economic Development Grant through the efforts of State Senator Diane Savino (operational)
:'''PENZIAS''', The Office of the CUNY Chief Information Officer ( Server is partially retired)
:'''SALK''', NSF Grant CNS-0958379 and a New York State Regional Economic Development Grant (Server is fully retired)