How to reduce interruptions in single server releases: health check, switch and quick rollback

It's not a word to roll back.Git Back up!

If the rollback step is to check for submission, re-installation dependency, and guess whether the database is compatible, it is not a rollback option. An implementable rollback must be prepared for the previous version of the code or mirror, configuration, data protection point and flow switch before it is released. Single servers cannot remove all interruptions, but unnecessary downtimes can be avoided: candidate versions are activated at another local port, after health and critical path checks, and then release Caddy Give it a new request.

Define success and cessation conditions before release

Write down this change, the path affected, the database changes, the projected time and the person responsible. Success is not a service display. active, but a health endpoint, home page, key API, login or form to work as expected.xx The proportion has risen significantly and the migration has taken longer than the window. Freezing does not change, saves the current version number, configuration Harshi, and database backup. Do not upgrade the operating system, the large version of the database and the application depend on the same release, which makes it impossible to distinguish the cause of failure.

Start the candidate version at another port

Current version running 127.0.0.1:3000, the candidate version can run 127.0.0.1:3002 Two. release The directory is read-only and the compatibility of the old and new versions with the data structure must be confirmed before sharing the data.

curl -fsS http://127.0.0.1:3002/health
curl -fsS -H 'Host: www.example.com' http://127.0.0.1:3002/

The health end-point inspection process is ready for traffic, key path check pages and dependence. Do not write the health check as a constant for permanent return to 200; and do not include too heavy external calls, resulting in a normal shaking and death.

Toggle proxy authentication configuration and observe immediately after the switch

Put it on. Caddy Go from 3000 to 3002, run. caddy validate After adoption reload. Save the pre-modified configuration, reload and check externals at the same time HTTPS, Caddy Logs, new versions of logs and old versions are connected. The old process first retains a short view window and does not just switch and delete it. If the application supports multiple upstreams, the candidate can also be temporarily added and the flow controlled, but two versions of the same database must be compatible. The value of the single machine blue green is a quick switch and does not amount to an independent infrastructure.

Database migration decision rolls back to maximum

Deleting columns, re-assets or irreversible changes would prevent the old version from continuing to function. It would be safer to expand -- move -- shrink: by adding backward compatible fields, deploying versions that can read and write both old and new structures, by completing data migration and observation, and by eventually removing the old structures in subsequent releases. Do not hide the long migration into the application startup command, otherwise systemd Both reboot and container reconstruction may be repeated.

Backroll in pre-written order

Trigger the stop condition and then go back to the old upstream and reload, confirm the return of the user flow, and stop the candidate version. If the database is modified backwards, there is no need to restore the whole database immediately; if an error is written, the event-level recovery programme does not automatically cover the entire production database for the purpose of completing a table. Keeps the candidate log and time line when rolling back. Restoring services is a priority and the root causes are then analysed, but do not delete the lost field evidence.

A minimum release record.

You will know the true time-consuming and the most vulnerable stages of failure before you can continue to automate. The completion criteria are that the candidate version is independently authenticated, the switch does not need to restart the entire agent, the old version can be restored in a few minutes, and the data migration compatibility strategy is clear. The next entry will allow degradation to be detected in time. Server monitoring and alarm.