Securing a Small Linux Server: SSH Keys, Firewall, Updates

Hello again, tech enthusiasts! 👋

You rented a small cloud server for a report script or a website. Great! But within minutes, robots on the internet start trying to log in. They guess passwords day and night.

The good news: a few simple steps stop almost all of these attacks. Today we lock the front door (SSH keys), build a fence (firewall), and keep the walls fresh (automatic updates). The examples use Ubuntu, but other Linux systems are very similar.

What you need

  • A Linux server (Ubuntu 22.04 or 24.04) and a user with sudo rights
  • A terminal on your own computer (Windows, Mac and Linux all have ssh built in)

In the examples, the server is server.example.com and the user is admin. Change them to yours.

Step 1: Create an SSH key and copy it to the server

SSH is how you log in to the server from far away. A password is like a door code that anyone can guess. An SSH key is like a real metal key: there are two parts, a private key that stays on your computer and a public key that you give to the server.

Run this on your own computer, not on the server:

# Create a modern key. Choose a passphrase when it asks (recommended).
ssh-keygen -t ed25519 -C "my-laptop"

# Copy the public key to the server (it asks for your password one last time)
ssh-copy-id admin@server.example.com

# Test: this should log you in without the server password
ssh admin@server.example.com

On Windows without ssh-copy-id, copy the line from .ssh\id_ed25519.pub into ~/.ssh/authorized_keys on the server.

Step 2: Turn off password login

Now that the key works, we tell the server to stop accepting passwords. On the server, create a small settings file:

sudo nano /etc/ssh/sshd_config.d/00-hardening.conf

Put these lines in it:

# Only keys, no passwords
PasswordAuthentication no
KbdInteractiveAuthentication no
# root may log in with a key only, never with a password
PermitRootLogin prohibit-password

Why the name 00-? The SSH server uses the first value it reads. Files in this folder are read in name order, and some cloud images add a file like 50-cloud-init.conf that turns passwords back on. A name starting with 00- is read first, so your choice wins.

Check the settings, then restart the service:

sudo sshd -t                     # no output means no errors
sudo systemctl restart ssh
sudo sshd -T | grep -Ei 'passwordauthentication|permitrootlogin'

Step 3: Test in a second window before you close anything

This is the most important step. Keep your current SSH window open. Open a new terminal and log in again:

ssh admin@server.example.com

If it works, great. If not, use the first window to fix the problem. If you close both, you could lock yourself out!

Step 4: Turn on the firewall

A firewall is like a guard at the gate. It only lets visitors in through the doors you name. Ubuntu includes a simple one called ufw.

sudo ufw default deny incoming    # block everything coming in...
sudo ufw default allow outgoing   # ...but let the server reach out
sudo ufw allow OpenSSH            # IMPORTANT: allow SSH before enabling
sudo ufw allow 80/tcp             # only if you run a website
sudo ufw allow 443/tcp            # only if you run a website (HTTPS)
sudo ufw enable
sudo ufw status verbose

Only open the ports you really use. A database port should usually stay closed.

Step 5: Automatic security updates

Most attacks use old, known bugs. Updates fix them. Let the server install security updates by itself:

sudo apt update
sudo apt install unattended-upgrades
sudo dpkg-reconfigure --priority=low unattended-upgrades   # choose "Yes"

# Check that it works (this only tests, it changes nothing)
sudo unattended-upgrade --dry-run --debug

Some updates need a restart. If the file /var/run/reboot-required exists, plan one at a quiet time.

Bonus: fail2ban

Even with keys only, the robots keep knocking. fail2ban watches the log and blocks an address for a while after too many failed tries.

sudo apt install fail2ban
sudo nano /etc/fail2ban/jail.local
[sshd]
enabled = true
maxretry = 5
findtime = 10m
bantime = 1h
sudo systemctl restart fail2ban
sudo fail2ban-client status sshd

Common problems

  • “Permission denied (publickey)” after Step 2. The key was not copied correctly. Use your still-open first window and check ~/.ssh/authorized_keys. The folder ~/.ssh should have permission 700 and the file 600.
  • Passwords still work. Another file sets PasswordAuthentication yes. Run sudo grep -ri password /etc/ssh/sshd_config.d/ and check the file names.
  • The website stopped working after ufw. You forgot to allow 80 or 443. Add them with sudo ufw allow.
  • fail2ban does not start. Some systems have no /var/log/auth.log. Add backend = systemd under [sshd] and restart.

Wrap-up

With keys, a firewall, automatic updates and fail2ban, your server is now much safer than many servers on the internet.

Next idea: set up a daily backup of your server to another place. A backup protects you from everything else, including your own mistakes. 🙂

fsaeed

, , , ,