SSH Hardening: Lock Down Remote Access

If your server is on the internet, it’s being scanned for open SSH right now. Default SSH config + password auth + a weak password = compromised in hours. Five small changes block 99% of these attacks. Do them on every server before anything else.

The five-minute checklist

1. Use SSH keys (no passwords)
2. Disable root login
3. Restrict who can log in
4. Move SSH off port 22 (optional, low value but cuts log noise)
5. Install fail2ban

Step 1: Set up SSH keys (if you haven’t)

On YOUR machine:

ssh-keygen -t ed25519
ssh-copy-id user@server.com

Test login WITHOUT a password before continuing. Don’t disable password auth until you’ve confirmed key auth works.

Step 2: Edit /etc/ssh/sshd_config

On the server:

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak
sudo nano /etc/ssh/sshd_config

Set or add these lines:

# Disable root login over SSH
PermitRootLogin no

# Require keys; refuse passwords
PasswordAuthentication no
PubkeyAuthentication yes
ChallengeResponseAuthentication no
KbdInteractiveAuthentication no

# Restrict who can SSH (only these users)
AllowUsers alice bob

# Or restrict by group
# AllowGroups sshusers

# Disconnect idle clients
ClientAliveInterval 300
ClientAliveCountMax 2

# No X11 forwarding unless needed
X11Forwarding no

# No empty passwords ever
PermitEmptyPasswords no

# Use modern protocols
Protocol 2

# Optional: change port (low security value, less log spam)
# Port 2222

Step 3: Validate and reload

sudo sshd -t                     # syntax check
sudo systemctl reload ssh        # apply (Debian/Ubuntu)
sudo systemctl reload sshd       # apply (Fedora/RHEL)

From a SECOND terminal, test that you can still log in. Don’t disconnect your first session until you confirm the second works.

Step 4: Install fail2ban

fail2ban watches your auth log and bans IPs that fail too many login attempts.

sudo apt install fail2ban           # Debian/Ubuntu
sudo dnf install fail2ban           # Fedora/RHEL

sudo systemctl enable --now fail2ban

Default config bans an IP for 10 minutes after 5 failed attempts. Customize /etc/fail2ban/jail.local:

[DEFAULT]
bantime  = 1h
findtime = 10m
maxretry = 5

[sshd]
enabled = true
port = ssh
logpath = %(sshd_log)s

Restart and verify:

sudo systemctl restart fail2ban
sudo fail2ban-client status sshd

Step 5: Firewall

Restrict SSH to specific IPs if possible:

# ufw
sudo ufw allow from 198.51.100.5 to any port 22
sudo ufw enable

# Or via cloud security groups (much better when available)

Bonus: 2FA with Google Authenticator

sudo apt install libpam-google-authenticator
google-authenticator     # as the user, sets up TOTP

# Then in /etc/pam.d/sshd add:
# auth required pam_google_authenticator.so

# And in sshd_config:
# ChallengeResponseAuthentication yes
# UsePAM yes
# AuthenticationMethods publickey,keyboard-interactive

sudo systemctl reload ssh

Now SSH requires both your key AND a 6-digit code.

Audit current state

# Check effective sshd config
sudo sshd -T | grep -i "permitroot|password|allowus"

# See active SSH sessions
who
last | head

# Recent login attempts
sudo journalctl -u ssh --since "1 day ago" | grep -i fail
sudo lastb | head            # bad login attempts

# Currently banned IPs (fail2ban)
sudo fail2ban-client status sshd

SSH key best practices

• Use ed25519 (smaller, faster) over RSA where possible.
• Set a passphrase on your private key.
• Never commit private keys to git.
• Rotate keys periodically.
• Use a different key per machine for blast-radius isolation.
• Use ssh-agent + AddKeysToAgent yes in config.

Recovery if you lock yourself out

• Use the cloud provider’s web console / serial console.
• Boot the VM in rescue mode.
• Mount the filesystem and edit /etc/ssh/sshd_config to fix.

Common mistakes

• Disabling password auth before testing key auth — instant lockout.
• Forgetting sudo systemctl reload ssh after editing config.
• Allowing AllowUsers without including yourself.
• Changing the port without also opening it in the firewall.

What to learn next

For deeper hardening — beyond who logs in, controlling what processes can do — SELinux and AppArmor are the next stop.