Server surveillance first looks at what:CPU, memory, disk, log and alarm thresholds

Monitor the user's access and then monitor the machine numbers.

CPU 20%, memory 60%, looks normal, and a domain name parsing error or certificate expiry will leave the site completely out of use. HTTPS Requests, reverse agents, application health, dependency, system resources, backups. For single servers, a small number of high signal indicators are more practical than dozens of dashboards.

First tier: request a real domain name from outside the server

Every minute or five minutes from another network. https://www.example.com/health，检查 DNS, TCP, TLS, state code and response time. Only the server has its own system. curl Public access routes, certificates and firewalls will be missed. Health pages should be light, without leaking versions, database addresses, or keys. liveness and readiness The first indicates that the process is alive and the second indicates that the request is ready to be received.

Second layer: processes, containers and dependence

systemd Service inspections active Status, number of restarts and recent failures;Compose Check the container status and health. The database monitors the success of the connection, the capacity is close to the limit, and the slow query is suddenly increasing. PID There is a requirement to request the actual functionality.

systemctl is-active myapp caddy
systemctl show myapp -p NRestarts
docker compose ps
journalctl -u myapp --since '15 minutes ago' --no-pager

Manual commands are suitable for checking, and continuous monitoring can be collected by the existing surveillance system.

Third floor:CPU, memory, disk and load

CPU Short-term 100% may only be constructed or compressed, sustained high occupancy and with slow response, it is worth warning. Memory must be looked at simultaneously. available, Swap Activities and OOM Records, don't just look. used Because... Linux The load values are combined. CPU Number and I/O Wait for an explanation. Disk often compares CPU More urgent. Root Division is 85%, then source of growth, logs, container layers, backups and uploads separately; do not write an automatic rm -rf Clear the unknown directory. Monitor it. inode And a lot of small files make it impossible for disks to continue writing.

Back-up age is an operational indicator of the need to warn.

Monitoring the recent successful backup time, volume, and validation results. Today’s document is only half yesterday, and missions may be unusual even if the exit code is zero. By RPO Decision threshold: A website that is backed up every day and that has no successful copy for more than 26 or 30 hours should be reminded; do not wait a week to discover that the scheduled task is long overdue.

Log alert mode, not all text

Focus on Short Time 5 xx Surges, login failures, database connections run out,OOM The alert is time, host, service, recent changes and check-in entrances, but filter passwords,Cookie And request body. Thresholds contain both values and duration, such as the use of disks with more than 85% duration of 10 minutes. warning and critical Two tiers, and the design of a recovery notice, to avoid the problem being closed but not known.

Every alarm must have a first step.

“CPU High is not enough; alerts are attached to the viewing process, recent publication and flow; “Weak is not available” is retested from outside and then viewed DNS/TLS/Caddy/upstream. Monthly review of misreporting, underreporting and unprocessed alarms to remove inoperable noises. The completion criteria are: public network malfunctions, service exits, disk closing caps, backup expirys and 5 xx The anomaly is detected before the user complains, and the recipient knows what to do first. 502, 504 and HTTPS Queued error.