On this page
Monitor whether visitors can use the site before watching machine numbers
CPU at 20% and memory at 60% look healthy, but broken DNS or an expired certificate can still take a website completely offline. Monitor from the outside in: a public HTTPS request, reverse proxy, application health, dependencies, system resources, and backups. Every layer needs an owner and a first response; otherwise the dashboards are just pictures.
For one server, a small set of high-signal checks is more useful than dozens of charts. Begin with the minimum set that will tell the right person about a real failure at night.
First layer: request the real domain from another network
Every minute or five minutes, request a URL such as https://www.example.com/health from outside the server. Check DNS, connection, TLS, status code, and response time. A local curl on the server cannot reveal a broken public route, certificate, or firewall rule.
Keep the health response lightweight and avoid exposing software versions, database addresses, or secrets. Liveness can mean that a process is running; readiness means it can accept traffic. Monitor certificate expiry and domain renewal separately.
Second layer: processes, containers, and dependencies
For systemd, inspect active state, restart count, and recent failures. For Compose, inspect container state and actual health checks. For a database, check connectivity, capacity, and sudden growth in slow queries. A PID existing is not a substitute for requesting a working endpoint.
systemctl is-active myapp caddy
systemctl show myapp -p NRestarts
docker compose ps
journalctl -u myapp --since '15 minutes ago' --no-pagerThese commands help during investigation; ongoing monitoring can collect the same signals automatically. Keep at least one alert route outside the server being monitored, or a server outage may take its own alarms down with it.
Third layer: CPU, memory, disk, and load
A brief CPU spike may be a build or compression job. Sustained high usage together with slower responses deserves attention. For memory, consider available, swap activity, and out-of-memory events—not only used, since Linux uses free RAM for cache. Interpret load alongside CPU core count and I/O wait.
Disk space is often more urgent. If the root filesystem approaches 85%, identify whether logs, container layers, backups, or uploads are growing before cleaning anything. Do not automate rm -rf against an unknown directory. Watch inode use too: many tiny files can stop writes even when byte capacity remains.
Backup age needs its own alert
Track the time, size, and integrity result of the last successful backup. A file half the expected size may be a problem even when the job exits with code 0. Failed off-site uploads, unchecked local backup growth, and overdue restore rehearsals belong on the checklist.
Set thresholds from your recovery-point objective. If you back up daily, a gap of 26–30 hours without a successful copy may justify an alert; waiting a week to discover a broken timer is too late.
Alert on log patterns, not every line
Watch for a burst of 5xx responses, unusual login failures, exhausted database connections, out-of-memory events, a read-only disk, or repeated service restarts. An occasional 404 should not wake anyone. Include time, host, service, recent changes, and a troubleshooting entry point in alerts, but filter passwords, cookies, and request bodies.
Pair a threshold with a duration—for example, disk usage above 85% for ten minutes. Use warning and critical levels, and send a recovery notification so responders know when an incident has ended.
Every alert needs a first action
“High CPU” is not enough: link to steps for checking the busy process, recent releases, and traffic. For “site unavailable,” retry externally, then work through DNS, TLS, Caddy, and the upstream. Review false alarms, missed incidents, and ignored alerts monthly. Remove noise that nobody can act on.
You are ready when a public outage, service exit, nearly full disk, stale backup, and unusual 5xx burst can be detected before a visitor reports them—and the recipient knows the first step. Next, apply those signals in the 502, 504, and HTTPS troubleshooting guide.
Sources
- journalctlsystemd
Next steps
Continue with the next useful task; you do not need to read everything at once.