Just record the phenomenon. Don't start again.
Reboot sometimes restores the service, but it clears the most valuable instantaneous state and makes you wonder where the problem is. 502 Generally indicating that the agent is unable to obtain an effective response from upstream, 504 being more upstream than within the deadline,TLS The error occurred in entering HTTP before application. They may be interconnected, but the order of screening should be external.
First floor:DNS Whether or not to the right entrance
dig +short A www.example.com
dig +short AAAA www.example.comChecks if the result is the current server. Error or obsolete AAAA The record will lead some users to the non-existent. IPv6. From different web tests, distinguish between locals. DNS Cache and authority recording problems. If the domain name has just been changed, see TTL, do not change frequently.
Second floor:80/443 and TLS Availability
Use curl -Iv https://www.example.com Check the cloud safety unit.UFW and Caddy Whether or not to listen 80/443. View on certificate issuance failed Caddy Log specific challenge error, confirm DNS The port has arrived on the current server and is not occupied by another process. The browser displays that the certificate is not trusted, do not allow the user to continue to access it. CDN_Other Organiser HTTPS Frequent fault check Provides a more ground-based approach.
Third floor:Caddy Configure whether to send the request to the right upstream
sudo caddy validate --config /etc/caddy/Caddyfile
systemctl status caddy --no-pager
journalctl -u caddy --since '20 minutes ago' --no-pagerCheck Domain Name Site Blocks, Path Matches and reverse_proxy address. In the container scene,127.0.0.1 The meaning depends on Caddy Whether the host or the container is written wrongly upstream is the common reason for 502. When you have recently modified the configuration, do not overlay the tentative changes compared to the backup.
Fourth floor: bypassing agent direct request upstream
curl -v --max-time 10 http://127.0.0.1:3000/health
ss -lntp | grep ':3000'
systemctl status myapp --no-pagerLocal requests have also failed, and the problem is applied or deeper; local successes and external 502 have focused on checking proxy addresses, protocols,Host, routing, and network. Do not change the firewall before the service start-up log is overheard at the port, because the internal loop requests do not go through the public portal.
Level 5: whether application is crash, jam or time-consuming.
Check logs near downtime, restarts,CPU, memory, disk and OOM. It is usually a configuration, permission or port conflict that exits immediately after start-up; process exists but the request for timeout may be an event cycle block, connection pool depleted, disk disk I/O Or downstream. API. If the request should have been completed in one second, a one-minute extension would have only kept the user waiting longer and filled the connection. Find where the slowness is before deciding whether to optimise, asymmetrical tasks, or to rationalize the time out.
Sixth floor: database and diskette
Checks whether the database can be connected, connected, locked, slow query and disk surplus.SQLite The directory has no write permission or disk full which may cause the application to read the page properly and write to fail;PostgreSQL When the connection pool is exhausted, you can queue a new request. Protect the data first and do not force the deletion of a locked file or database file in an unknown state.
Verify and record simultaneously after restoration
From external HTTPS Record root causes, trigger conditions, evidence, repairs, and repeat activations. If recovery is only restarted, the event remains incomplete for unknown reasons. The next article will be a day-to-day system of deployment, surveillance and error:Patches, capacity, failure records and recovery manual.
Sources
- Caddy reverse_proxy directiveCaddy Documentation
- journalctlsystemd