Production Operations on a Single Server · 9/10

502, 504 and HTTPS How to check the fault: from DNS Error stratification to application

Press DNS Networks,TLS, Caddy and the order of local upstream, application and database HTTPS Error, narrowing the range with each layer of evidence to avoid any luck in restarting the server.

Just record the phenomenon. Don't start again.

Reboot sometimes restores the service, but it clears the most valuable instantaneous state and makes you wonder where the problem is. 502 Generally indicating that the agent is unable to obtain an effective response from upstream, 504 being more upstream than within the deadline,TLS The error occurred in entering HTTP before application. They may be interconnected, but the order of screening should be external.

First floor:DNS Whether or not to the right entrance

bash
dig +short A www.example.com
dig +short AAAA www.example.com

Checks if the result is the current server. Error or obsolete AAAA The record will lead some users to the non-existent. IPv6. From different web tests, distinguish between locals. DNS Cache and authority recording problems. If the domain name has just been changed, see TTL, do not change frequently.

Second floor:80/443 and TLS Availability

Use curl -Iv https://www.example.com Check the cloud safety unit.UFW and Caddy Whether or not to listen 80/443. View on certificate issuance failed Caddy Log specific challenge error, confirm DNS The port has arrived on the current server and is not occupied by another process. The browser displays that the certificate is not trusted, do not allow the user to continue to access it. CDN_Other Organiser HTTPS Frequent fault check Provides a more ground-based approach.

Third floor:Caddy Configure whether to send the request to the right upstream

bash
sudo caddy validate --config /etc/caddy/Caddyfile
systemctl status caddy --no-pager
journalctl -u caddy --since '20 minutes ago' --no-pager

Check Domain Name Site Blocks, Path Matches and reverse_proxy address. In the container scene,127.0.0.1 The meaning depends on Caddy Whether the host or the container is written wrongly upstream is the common reason for 502. When you have recently modified the configuration, do not overlay the tentative changes compared to the backup.

Fourth floor: bypassing agent direct request upstream

bash
curl -v --max-time 10 http://127.0.0.1:3000/health
ss -lntp | grep ':3000'
systemctl status myapp --no-pager

Local requests have also failed, and the problem is applied or deeper; local successes and external 502 have focused on checking proxy addresses, protocols,Host, routing, and network. Do not change the firewall before the service start-up log is overheard at the port, because the internal loop requests do not go through the public portal.

Level 5: whether application is crash, jam or time-consuming.

Check logs near downtime, restarts,CPU, memory, disk and OOM. It is usually a configuration, permission or port conflict that exits immediately after start-up; process exists but the request for timeout may be an event cycle block, connection pool depleted, disk disk I/O Or downstream. API. If the request should have been completed in one second, a one-minute extension would have only kept the user waiting longer and filled the connection. Find where the slowness is before deciding whether to optimise, asymmetrical tasks, or to rationalize the time out.

Sixth floor: database and diskette

Checks whether the database can be connected, connected, locked, slow query and disk surplus.SQLite The directory has no write permission or disk full which may cause the application to read the page properly and write to fail;PostgreSQL When the connection pool is exhausted, you can queue a new request. Protect the data first and do not force the deletion of a locked file or database file in an unknown state.

Verify and record simultaneously after restoration

From external HTTPS Record root causes, trigger conditions, evidence, repairs, and repeat activations. If recovery is only restarted, the event remains incomplete for unknown reasons. The next article will be a day-to-day system of deployment, surveillance and error:Patches, capacity, failure records and recovery manual.

Sources

  1. Caddy reverse_proxy directiveCaddy Documentation
  2. journalctlsystemd
RESOURCES I USE · REFERRAL

Two services to compare when you are ready to launch

This is not an automated ranking, and neither service is necessary for everyone. These are services I use, with the use case and limitations kept visible.

Cloud server · Used for early projects

RainYun

A practical candidate for a website or small service. Choose by user region, configuration and measured workload rather than the lowest headline price.

Referral disclosure: these links contain my referral information. I may receive a platform benefit if you sign up or order, at no additional charge from AIOOS. Check the order page for current pricing, availability, regions and terms. Read the full affiliate disclosure