Linux Commands
The commands data engineers, MLOps folks, and anyone running models on a server use daily. Every heading below is a job you need to do; the table tells you the tool.
Table of Contents
Installation / Setup
uname -a # kernel & OS info
cat /etc/os-release # distro (Ubuntu, Debian, ...)
bash --version # your shell
Run one command with man <cmd> for docs; type <cmd> tells you what a command actually is. Commands below are grouped by job — nearly all ship with any Linux install.
Core Syntax
Navigation
| Command |
Does |
pwd |
print working directory |
ls / ls -la / ls -lh |
list (all, human sizes) |
cd dir / cd .. / cd ~ / cd - |
change directory |
find . -name "*.csv" |
locate files by name |
ls -la /data
cd ~/projects && pwd
find /home -name "*.py" -type f 2>/dev/null | head
Files & directories
| Task |
Command |
| Copy |
cp file.txt backup/ · cp -r dir/ newdir/ |
| Move/rename |
mv old.txt new.txt |
| Delete file |
rm file.txt |
| Delete directory |
rm -r dir/ (recursive) · rm -rf (force — dangerous!) |
| Create dir |
mkdir -p a/b/c |
| Create empty file |
touch notes.md |
| View file |
cat file · less file (scroll, q to quit) · head -20 · tail -20 |
| Follow a log |
tail -f app.log |
| Count lines |
wc -l file.csv |
| Symlink |
ln -s /target link_name |
| Compare |
diff -u old.py new.py · vimdiff a b |
Search
grep "error" app.log # find lines
grep -i -n "error" *.log # case-insensitive + line numbers
grep -r "TODO" src/ # recursive
grep -c "error" app.log # count matches
grep -v "DEBUG" app.log # exclude lines
grep -E "err(or|ror)" app.log # extended regex
grep -l "error" *.log # files that match
# ripgrep if installed: rg "pattern" / rg -l / rg -C 3
Text processing
| Tool |
Job |
sed 's/old/new/g' file |
search & replace (stream) |
awk '{print $1, $3}' file |
column extraction |
cut -d, -f1,2 file |
split fields |
sort -k2 -n · uniq -c |
sort · count unique |
tr 'a-z' 'A-Z' |
character translation |
jq '.key' data.json |
JSON querying |
head/tail/wc |
slices & counts |
sed -i 's/\r$//' script.sh # strip Windows CRLF in place
cut -d, -f1 train.csv | sort | uniq -c | sort -rn | head # top classes
cat events.json | jq -r '.user_id' | sort -u | wc -l
Processes
ps aux # all processes
ps aux | grep python # find a process
top / htop # live view
kill 1234 # graceful stop by PID
kill -9 1234 # force kill (last resort)
pkill -f train.py # kill by pattern
pgrep -f train.py # find PID by pattern
jobs / bg / fg # background/foreground in a shell
nohup python train.py & # survive logout
Networking
curl -I https://example.com # headers only
curl -O https://.../file.zip # download
curl -s https://api.example.com/x | jq
ping -c 4 host
ip addr / ip route # modern ip commands
ss -tlnp # listening ports & owners (was netstat)
nc -zv host 22 # is a port open?
dig example.com / nslookup example.com
wget -q https://.../model.pt
Permissions & ownership
ls -l file # -rw-r--r-- : owner/group/other
chmod 755 script.sh # rwxr-xr-x — executable
chmod +x script.sh # add execute
chmod 600 key.pem # private key: owner-only!
chown user:group file
umask # default permission mask
Permission triplet rwx = read/write/execute; numeric: r=4, w=2, x=1. 644 = owner rw, others r.
Users
whoami / id
sudo -i # root shell
sudo apt install pkg # run as root per command
useradd -m -s /bin/bash bob
passwd bob # set password
usermod -aG sudo bob # add to sudo group
groups bob
su - bob # switch user
Packages (Debian/Ubuntu; see notes)
sudo apt update # refresh index
sudo apt upgrade -y # upgrade all
sudo apt install -y htop # install
apt search pandas
apt show python3-pandas # info
sudo apt remove pkg
# Red Hat/Fedora: sudo dnf install ...
# macOS Homebrew: brew install ... (see 30-Other-Tools if added)
Environment variables
echo $PATH
export MY_VAR="value" # session only
export PATH="$PATH:/opt/bin" # extend PATH
unset MY_VAR
env | sort # all environment variables
# persistent: add export lines to ~/.bashrc, then: source ~/.bashrc
Archives
tar -czf out.tar.gz folder/ # create gzipped archive
tar -xzf out.tar.gz # extract
tar -tf out.tar.gz # list contents
zip -r out.zip folder/ ; unzip out.zip
gzip file.csv ; gunzip file.csv.gz
Disk & system info
df -h # disk usage by filesystem
du -sh folder/ # folder size
du -sh * | sort -h # biggest items here
free -h # RAM
nproc # CPU cores
uptime # load average
lsblk # block devices
dmesg | tail # kernel messages
Practical Examples
1. Inspect what's eating the GPU/disk
nvidia-smi # GPU memory per process
ps aux --sort=-%mem | head -10 # top memory consumers
du -sh ~/data ~/.cache 2>/dev/null # space hogs
2. Find and kill a runaway training job
ps aux | grep train.py | grep -v grep
kill -9 <PID>
# or in one step:
pkill -f "python train.py --lr 0.1"
3. Download, extract, and inspect a dataset
curl -L -O https://example.com/data.tar.gz
tar -xzf data.tar.gz
head -5 data/train.csv
wc -l data/train.csv
4. Copy files to/from a server
scp -i ~/.ssh/key.pem model.pt user@host:/home/user/models/
rsync -avz --progress data/ user@host:/data/ # resume-able sync
Common Patterns
# Run a job that survives logout
nohup python train.py > train.log 2>&1 &
# stdout+stderr -> train.log, background, immune to hangup
# then: tail -f train.log
# Daily automation building block
find . -name "*.tmp" -delete # clean temp files
grep -rn "print(" src/ | wc -l # count debug prints
# Safe-ish deletion habit: list first, delete second
find . -name "*.log" -mtime +30 # preview
find . -name "*.log" -mtime +30 -delete
# Pipe discipline: one tool per stage, read left to right
cat access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head -5
Best Practices
ls before rm, git status before git add — look before you delete.- Use
less for big files — never cat a multi-GB log.
- Quote paths with spaces (
cd "/my dir") and use tab completion.
sudo only when needed — a typo as root can destroy a system.- Private keys must be
chmod 600 or SSH refuses them.
- Check disk (
df -h) before writing big artifacts — full disks corrupt training runs.
- Prefer
find/rg over grep -r on huge trees; prefer rsync over scp for syncs.
- Keep defensive commands (network scans, user queries) read-only unless you own the target — see Cybersecurity category for the rules.
Common Mistakes
| Mistake |
Consequence |
Fix |
rm -rf with a typo (rm -rf / data) |
Deletes the wrong tree |
Triple-check; use rm -i; full paths in quotes |
cating a huge file |
Terminal freeze / OOM |
less or head |
| Killing the wrong PID |
Stops another service |
ps aux \| grep first; pgrep -af |
Forgetting sudo / overusing it |
Permissions errors or damage |
Right tool per task |
| Unquoted spaces |
"No such file or directory" |
Quote or escape |
grep on binary/model files |
Binary noise |
Add -I or search text dirs only |
| Editing system files without backup |
Broken config |
Copy first: cp file file.bak |
Troubleshooting
| Symptom |
Likely cause |
Fix |
command not found |
Not on PATH / not installed |
which cmd; install package |
Permission denied |
Wrong file perms/owner |
ls -l; chmod/chown or sudo |
Disk full (No space left) |
Data/logs grew |
du -sh * → remove/archive |
| Port already in use |
Another process |
ss -tlnp → kill or use another port |
-bash: python: command not found |
Python not on PATH (venv?) |
which python3; activate venv |
| SSH hangs / refused |
Firewall or sshd down |
ping, nc -zv host 22, check systemctl status ssh |
| Slow commands on big dirs |
Full scans |
Scope with find -maxdepth, rg, indexes |
Official Documentation
Last Updated
2026-09-06 — Initial version.