Chen Yulin's BlogChen Yulin's Blog
HomeArchivesCategoriesTagsAbout
  目录
SJTU-HPC基本操作
Posted 2025-04-07Updated 2025-05-28Note4 minutes read (About 616 words)   visits

SJTU-HPC基本操作

官方文档

LMOD软件管理

https://lmod.readthedocs.io/en/latest/010_user.html

交大hpc使用lmod来管理用户软件,每一次重新登陆都会重置module,需要将软件重新load。
常用指令:

1
2
3
4
5
ml # list these loaded modules
ml miniconda3 # load the miniconda3 module
ml -miniconda3 # unload the miniconda3 module
module spider # inspect the possible modules that can be loaded
module spider cuda # show available cuda versions

oh-my-zsh更好的shell

hpc默认已安装zsh, 通过omzsh的安装脚本进行安装,并安装插件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
export CHSH=no  # 避免它自动切换默认 shell(HPC 通常不允许)
sh -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"
git clone https://github.com/zsh-users/zsh-syntax-highlighting.git \
${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-syntax-highlighting
git clone https://github.com/zsh-users/zsh-autosuggestions \
${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-autosuggestions
git clone --depth 1 https://github.com/junegunn/fzf.git ~/.fzf
~/.fzf/install
git clone https://github.com/psprint/zsh-navigation-tools ~/.zsh-navigation-tools

## vim add configuration in `.zshrc`
plugins=(
git
zsh-navigation-tools
rails
ruby
zsh-syntax-highlighting
zsh-autosuggestions
)
source ${(q-)ZSH_CUSTOM:-$ZSH/custom}/plugins/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh

## :wq

nvim 更好的编辑器

因为在服务器上没有sudo权限所以需要手动编译

首先使用conda创建编译环境

1
conda create -n nvim-build cmake libtool gettext curl unzip ninja gcc gxx -y -c conda-forge

clone & build

1
2
3
4
git clone https://github.com/neovim/neovim.git
cd neovim
git checkout stable
make CMAKE_BUILD_TYPE=Release

Tricks

交互式计算资源

某些情况下需要进行并行编译,编译需要耗费大量资源,并不一定可以在登录节点使用:

在这种情况下使用提交作业的方式较为低效,可以申请交互式计算资源
1
srun -p cpu -n 4 --pty /bin/bash
example:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
(base) chenyulin@pilogin2 ~> srun -p cpu -n 4 --pty /bin/bash
srun: job 43090290 queued and waiting for resources
srun: job 43090290 has been allocated resources
(base) chenyulin@cas242 ~> ls
condalist.txt dinov2 log test.slurm
(base) chenyulin@cas242 ~> cd dinov2
(base) chenyulin@cas242 ~/dinov2> conda activate dinov2
(dinov2) chenyulin@cas242 ~/dinov2> export CC=$CONDA_PREFIX/bin/gcc main
export CXX=$CONDA_PREFIX/bin/g++
(dinov2) chenyulin@cas242 ~/dinov2> which g++ main
~/.conda/envs/dinov2/bin/g++
(dinov2) chenyulin@cas242 ~/dinov2> vim conda_cyl.yaml main
(dinov2) chenyulin@cas242 ~/dinov2> conda env update -f conda_cyl.yaml main
Retrieving notices: ...working... done
Channels:
- xformers
- pytorch
- nvidia
- conda-forge
- defaults
- nvidia/label/cuda-11.7.0
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done
Installing pip dependencies: |

文件传输

本地向云端传输

使用hpc studio传输
https://studio.hpc.sjtu.edu.cn/pun/sys/dashboard/files/fs//lustre/home/acct-umjbyy/chenyulin/Data

云端之间互相传输

例如拷贝conda环境

1
cp -r /lustre/home/acct-umjbyy/chenyulin/.conda /dssg/home/acct-umjbyy/chenyulin/

SJTU-HPC基本操作

http://chen-yulin.github.io/2025/04/07/[OBS]linux-SJTU-HPC-SJTU-HPC基本操作/

Author

Chen Yulin

Posted on

2025-04-07

Updated on

2025-05-28

Licensed under

#PythonLinuxHPC
OmegaConf Python Package
Tuxguitar on Archlinux

Comments

Chen Yulin

Chen Yulin

SJTU student

Manchester by the Sea

Posts

263

Categories

8

Tags

189

Follow

Catalogue

  • LMOD软件管理
  • oh-my-zsh更好的shell
  • nvim 更好的编辑器
  • Tricks
    • 交互式计算资源
  • 文件传输
    • 本地向云端传输
    • 云端之间互相传输

Archives

  • May 202510
  • April 202517
  • March 202545
  • February 202512
  • January 202513
  • December 202412
  • November 20244
  • October 202418
  • September 202416
  • August 202413
  • July 20243
  • June 20245
  • May 202413
  • April 202417
  • March 20241
  • January 20241
  • December 20231
  • May 202346
  • August 20221
  • May 20226
  • April 20229

Recents

《禅与摩托车维修艺术》读书会p10

2025-05-24

《禅与摩托车维修艺术》读书会p10

读读噜

Pixtral 12B API Inference

2025-05-23

Pixtral 12B API Inference

Note

2025-05-19

2025 Disneyland

Note

Matlab on Archlinux

2025-05-13

Matlab on Archlinux

Note

2025-05-13

Part-level Dataset Available for Augmentation

Note

Tags

3D-Scene4
6-D3
AI10
AIGC1
API1
AR2
Academic1
Algorithm1
Aliyun1
App2
Atlas1
BS41
Beautify1
Behaviorism1
Business1
C1
CADC1
CD1
CLIP5
CNN1
CV28
Capstone10
Communication2
Contrastive-Learning3
Control2
Csharp9
Css1
Cuda3
DD1
DINO4
DT1
Dataframe1
Debate5
Debugger1
Diffusion1
Discrete-Mathematics1
Disney1
Docker1
Docs2
Dynamic-programming1
ESP322
Education1
Embeded-System9
Embodied-AI8
Emoation1
Emotion12
Ethic1
FL1
FPN2
Family1
Federated-Learning1
Foundation1
Functional programming1
GPT3
Game5
Gated-NN2
Git7
Github1
Godot3
HPC1
HRI2
Haskell1
Health2
Hexo10
Hierarchical1
Html5
Humanism1
Hyprland2
IK1
Image-Grounding1
Image-Text5
Image-generation1
ImitationLearning3
Jolt1
Json1
LLM12
LSP2
Latex2
Life4
LinearAlgebra1
Linux21
Live2d1
Love3
Lua1
MBTI1
ML5
MR/AR3
Mason1
Math3
Meme1
Message-Passing1
Mod3
Motivation1
Movie1
Multi-modal6
Multi-view1
Music5
NLP4
NN4
Network2
Nodejs5
Numpy1
Nvim9
Object-Detection4
Open-Vocabulary9
OpenCV1
Oral1
PHD1
PSY5
Pandas2
Panoptic1
Path1
Philosophy3
PhysX1
Physical-Scene4
Physics-engine1
Pio2
Planning1
Plugin8
PoseEstimation3
Postgraduate1
Prefab1
Probability1
Python27
Pytorch1
QML1
Quantum1
RNN4
ROS3
Reading19
Real2Sim1
Reconstruct9
Regex2
Reinforcement-learning1
Repository5
Representation-Learning1
Research-paper86
Robot1
Robotics16
SJTU-Lecture1
SQL2
SSH2
Scene-graph29
Scene-synthesis1
Science-fiction1
Scrap1
Script2
Segmentation7
Semantic12
Shader3
Shell4
Signals and Systems1
Sim2Real1
Sklearn1
Snippets1
Society4
Star-rail1
Subgraph1
Submodule1
Supervised-learning2
Survey3
TC1
TOEFL1
Task-Planning6
Tasks4
Tech Communication1
Torch4
Transformer11
Translation-Embedding2
Travel3
Unity20
Unsupervised-learning1
VLM6
VLP2
Version-management1
ViT4
VideoEditing2
Vim1
Visual-Relation20
WSL1
Waybar1
Wayland1
Web1
Website1
Well-being1
Window-manager2
YKLL3
Zen2
🐱1
Chen Yulin's BlogChen Yulin's Blog

© 2025 Chen Yulin  Powered by Hexo & Icarus

×