VPS operations

Monitor one VPS without Prometheus, Grafana or a stack

· 9 min read

On this page (11)
  1. Who needs this
  2. What to alert on, and what not to
  3. The outside observer
  4. Check 1: is the app answering?
  5. Check 2: disk
  6. Check 3: memory
  7. Check 4: certificate expiry
  8. Check 5: systemd tells you when something gave up
  9. Scheduling it all
  10. Mistakes we see
  11. FAQ

A single VPS with one app does not need a metrics database, a dashboard server and an alert manager. Those tools solve fleet problems, and installing them on the same 2 GB machine they are meant to watch costs more memory than the app uses. They also share its fate: when the box goes down, the monitor goes down with it and says nothing.

What you need is a short list of signals, a few shell scripts, and one outside observer. This post builds that with tools already on the machine: cron, curl, df, openssl and systemd.

Checks on the box, watchers outside it

Cron and systemd run checks on the VPS for the app, disk, memory, certificate and failed units, then send alerts to your phone; an outside uptime monitor and a heartbeat service watch the server from the internet and alert you when it goes silent.cron checksApp, disk, memory,certsystemd OnFailureFailed unitsAlert scriptChat, push or e-mailOutside observerUptime check andheartbeatYour phonesilence or failure

Who needs this

  • A Go or Node app with paying users and one server. If it is down for six hours on a Saturday you hear about it from a customer. You want to hear about it from your phone within five minutes, and a fleet monitoring system is not going to be maintained by you.
  • A server with scheduled jobs. Backups, report generators, certificate renewals. The dangerous failure is the job that stopped running without an error. Nothing fails, so nothing alerts, and three weeks later you need the backup that was never made.
  • A box that fills up slowly. Logs, Docker images, old releases. The failure arrives on a random night as a "database or disk is full" error, and by the time you see it, writes are failing.
  • Someone who will not watch dashboards. You are not going to open Grafana on Tuesday morning. A system that is silent unless something is wrong suits you better.

What to alert on, and what not to

Start with the answer to a different question: what would you want to be woken up for? On a one-person server the list is short.

SignalWhy it mattersAlert when
App health URLThe site is actually answering and its database worksCheck fails, or the check stops running
Disk usageA full disk breaks writes, logs, backups and deploysOver 80% on / (and data volumes)
Backup finishedA backup that silently stopped is a missing backupThe success ping does not arrive on schedule
Certificate expiryRenewal can fail quietly for weeksFewer than 14 days left
Memory pressureOOM kills look like random crashesAvailable memory low, or an OOM kill appears in the kernel log
Failed systemd unitsServices and timers that gave upAny unit enters the failed state

Things that usually do not deserve a page: CPU at 90% for two minutes, a single slow request, load average on its own, and any metric without a clear action attached. Alerts you learn to ignore are worse than none. If you cannot say what you would do when it fires, it belongs on a dashboard or not at all.

The outside observer

Every check below runs on the server, so every check shares the server's fate. You need one thing that does not: an external service. There are two kinds, and you want both.

An uptime check from outside. A free-tier uptime monitor requests https://example.com/health every minute or five from the internet and alerts you by e-mail, push or chat. It sees what your customers see: DNS, TLS, the proxy, the app. It catches the case where the whole machine is dead.

A dead-man's-switch (heartbeat) service. Your server pings a URL at the end of each job. The service alerts you if a ping does not arrive within an expected window. This is the only approach that catches "the cron job silently stopped" and "the whole server is off". Many services offer this; the pattern is a URL for success and the same URL with /fail appended for explicit failure.

A bare-bones pattern with cron:

# crontab -e   (as the user that runs the job)
15 3 * * * /opt/scripts/backup.sh && curl -fsS -m 10 --retry 3 -o /dev/null https://hc.example/ping/BACKUP-UUID

The && means the ping is sent only if the script exits 0. If the script fails, no ping goes out, and the service raises the alarm when the window passes. Flags: -f makes curl fail on HTTP errors, -sS is quiet but still shows errors, -m 10 bounds the time, --retry 3 rides over a blip.

For anything where you want a fast signal instead of waiting for a window, report both outcomes:

/opt/scripts/backup.sh \
  && curl -fsS -m 10 --retry 3 -o /dev/null "$HC/ping/BACKUP-UUID" \
  || curl -fsS -m 10 --retry 3 -o /dev/null "$HC/ping/BACKUP-UUID/fail"

How a heartbeat catches a silent failure

Four steps: cron runs the job, the job exits zero, curl pings the heartbeat URL, and if no ping arrives within the window the service alerts you.1Cron runs thejobbackup.sh at03:152Exit code 0&& sends theping only onsuccess3Ping arrivesWindow resets,no noise4No ping inwindowService alertsyou

Check 1: is the app answering?

The local half of health monitoring is a script that asks the app for its health URL and reports the result. This is the shape of the one in our handbook:

#!/usr/bin/env bash
set -euo pipefail
HEALTH_URL=${HEALTH_URL:-http://127.0.0.1:3000/health}
HC_URL=${HC_URL:-}
if curl -fsS -m 5 -o /dev/null "$HEALTH_URL"; then
  [ -n "$HC_URL" ] && curl -fsS -m 10 --retry 2 -o /dev/null "$HC_URL" || true
  exit 0
fi
logger -t healthcheck "app NOT healthy: $HEALTH_URL"
[ -n "$HC_URL" ] && curl -fsS -m 10 --retry 2 -o /dev/null "$HC_URL/fail" || true
exit 1

Run it every minute or five from cron. Because the success ping stops when the machine dies, the heartbeat service also covers total failure, and logger leaves a line in the journal for later (journalctl -t healthcheck).

The health endpoint itself should check something real. A handler that returns 200 ok as long as the process runs will pass while the database file is unreadable. Have it run a cheap query and return 503 when it fails. We cover what a meaningful health check looks like, including what to check after a deploy, in deploy health checks that mean something.

Check 2: disk

#!/usr/bin/env bash
set -euo pipefail
THRESHOLD=${THRESHOLD:-80}
CHECK_PATHS=${CHECK_PATHS:-/}
rc=0
for p in $CHECK_PATHS; do
  use=$(df --output=pcent "$p" | tail -1 | tr -dc '0-9')
  msg="disk $p at ${use}% (threshold ${THRESHOLD}%) on $(hostname)"
  echo "$msg"
  if [ "$use" -ge "$THRESHOLD" ]; then
    logger -t check-disk "ALERT $msg"
    [ -n "${ALERT_CMD:-}" ] && echo "$msg" | bash -c "$ALERT_CMD" || true
    rc=1
  fi
done
exit "$rc"

df --output=pcent gives a clean percentage column, which avoids parsing the human format. Set ALERT_CMD to whatever sends you a message: mail -s disk [email protected] if you have a working mail setup, or a curl to a chat webhook. Run it every 30 minutes. If the disk fills faster than that, you also need an alert on rate of growth, but for most small servers the slow fill is the usual story.

Also check inodes once in a while with df -i; an inode-exhausted filesystem reports free space and still refuses new files. Journald, logrotate and the disk-full hunt covers finding what is eating the space.

Check 3: memory

You can watch MemAvailable, the kernel's own estimate of memory that can be given to programs without swapping:

avail_mb=$(awk '/MemAvailable/ {print int($2/1024)}' /proc/meminfo)
[ "$avail_mb" -lt 100 ] && logger -t check-mem "ALERT only ${avail_mb} MB available"

A threshold on a snapshot is noisy. More useful is to look for what actually hurts, the OOM killer. After the fact, journalctl -k | grep -i "killed process" shows kills. You can alert on that by checking the last hour:

if journalctl -k --since "-1h" --no-pager | grep -qi "killed process"; then
  logger -t check-mem "ALERT OOM kill in the last hour"
  exit 1
fi

For a service you care about, put MemoryMax= in its unit so a runaway app is the thing that dies, and let Restart=on-failure bring it back. That makes memory problems show up as a failed service, which the next check catches.

Check 4: certificate expiry

Caddy and certbot both renew on their own. The check exists because renewal can fail quietly, for example after DNS changes or a blocked port, and the first symptom is an expired certificate. openssl has the test built in. -checkend takes seconds and exits non-zero if the certificate expires within that time:

#!/usr/bin/env bash
host=example.com
days=14
if ! echo | openssl s_client -servername "$host" -connect "$host:443" 2>/dev/null \
     | openssl x509 -noout -checkend $((days*86400)) >/dev/null; then
  logger -t check-cert "ALERT certificate for $host expires within $days days (or is unreachable)"
  exit 1
fi

Run it daily. To see the actual date by hand: ... | openssl x509 -noout -enddate. An outside uptime monitor often offers certificate expiry checks as well, which is better because it validates what the world sees.

What deserves a page

Two cards: signals worth alerting on, such as a failed health check, disk over 80 percent, a missed backup ping, a certificate under 14 days and a failed unit; and noisy signals to skip, such as brief CPU spikes, single slow requests and load average alone.Alert onHealth check fails or stops runningDisk over 80%, backup ping missingCertificate under 14 days leftA systemd unit enters failed stateLeave to a dashboardCPU at 90% for two minutesA single slow requestLoad average on its ownAny metric with no clear action

Check 5: systemd tells you when something gave up

Cron job wrappers are one way. systemd has a better one for services and timers: OnFailure=. When a unit enters the failed state, systemd starts another unit of your choice. Write a template unit once and attach it to anything:

# /etc/systemd/system/[email protected]
[Unit]
Description=Alert: %i failed

[Service]
Type=oneshot
ExecStart=/usr/local/bin/alert "unit %i failed on %H"
# /etc/systemd/system/app.service.d/onfailure.conf
[Unit]
OnFailure=notify-failure@%n.service

/usr/local/bin/alert is your script: it can send a curl to a chat webhook, a push notification service, or an e-mail. After systemctl daemon-reload, test it by making a service fail on purpose in a scratch unit.

One nuance: with Restart=on-failure, a service going through restarts is not yet in the failed state. systemd only treats it as failed when it gives up, which happens when the start limit is hit (by default five starts in ten seconds). So OnFailure= is your "the service is crash-looping and systemd stopped trying" alarm. A single crash that restarts fine will not trigger it, and your health check covers that case.

If you want a quick view during a bad day, systemctl --failed lists failed units and journalctl -p err -b lists error-level messages since boot.

Scheduling it all

All of it fits in one /etc/cron.d file:

SHELL=/bin/bash
PATH=/usr/local/bin:/usr/bin:/bin
*/5  * * * * root HEALTH_URL=http://127.0.0.1:3000/health HC_URL=https://hc.example/ping/APP-UUID /opt/scripts/healthcheck.sh
*/30 * * * * root THRESHOLD=80 CHECK_PATHS="/ /var" /opt/scripts/check-disk.sh
17 6 * * *   root /opt/scripts/check-cert.sh
15 3 * * *   root /opt/scripts/backup.sh

Files in /etc/cron.d have a user column, which the per-user crontab -e does not. Cron's environment is minimal, so set PATH and SHELL and use full paths in the commands. Test each script by running it by hand with sudo, and then once from cron with a very short schedule to prove the environment works.

Rule of thumb: if you cannot say what you would do when an alert fires, delete the alert.

Mistakes we see

Alerting only from the server. If the machine is off, nothing is sent. Always add an outside check or a heartbeat service.

Checking that the process is running instead of that it works. systemctl is-active app says nothing about whether it can reach its database. Ask the health URL.

Never testing the alert. Run the check with a threshold of zero, or break something on purpose, and make sure the message reaches a person. Alerts that go to a mailbox nobody reads, or to a mail setup that was never completed, are common.

Alert fatigue. Thirty alerts a week teach you to ignore them. Keep the list above, tune thresholds so that a firing alert means action, and delete the rest.

Pinging success before the work is done. Put the ping after the command with &&, not before it, or the "success" ping means "cron started".

Ignoring the cron output. Cron e-mails output to the local user, and on most VPS images local mail goes nowhere. Redirect to a log file or logger, and rely on the heartbeat for silence.

If you take one idea away: silence must be a signal. A system that only speaks when something breaks cannot tell the difference between "all fine" and "dead". Heartbeats can. The same idea underlies monitoring backups so silence is not success.

FAQ

Is cron plus curl really enough for production?

For one server and one app, yes, as long as there is an outside observer. The scripts are small, you can read them in a minute, and they have almost nothing to break. What you give up is history and graphs. If you later need those, add them; the alerts you built still work.

Which free uptime service should I use?

Any reputable one that supports HTTPS checks, heartbeat (dead-man) pings and alerts to a channel you actually read. Compare check interval, how many monitors the free tier allows, and whether heartbeat monitoring is included, because that is the feature that catches silent failures. We do not recommend a particular vendor.

How often should the checks run?

Health every one to five minutes, disk every 30 minutes to an hour, certificates once a day, backups at their own schedule with a heartbeat window a bit longer than the job takes. More often than that costs you noise and gains nothing.

Should I install Prometheus and Grafana anyway to learn them?

If you want to learn them, on a separate machine or a laptop. Running them on the production VPS takes memory and disk from the app and gives you a monitor that dies with the thing it watches.

The VPS Handbook includes tested versions of the health, disk and heartbeat scripts, a cron template, a failure playbook table and a weekly checklist. They are described in the handbook.

monitoringcronsystemd