Immutable deploys on a single box, with no downtime
Part of my Krabber series, a Twitter clone in Go. The full source is on GitHub.
Intro
Krabber runs on a single Beanstalk instance, which sounds like it should mean “deploys have a downtime window while the box restarts.” It doesn’t. Deploys are zero-downtime and they roll themselves back if the new version is sick. In this post I’ll walk through how one box manages that, because the trick is to never actually deploy onto the box that’s serving.
I. Immutable, even on one instance
Krabber’s deployment policy is set to Immutable. Instead of stopping the running app and installing the new version in place, Beanstalk boots a brand-new instance with the new version, waits for it to pass health checks, and only then switches traffic to it. The old instance keeps serving the whole time, and it’s retired once the new one is healthy. If the new instance never gets healthy, Beanstalk throws it away and leaves the old one running, untouched.
This works even though I only run one instance. Beanstalk spins up a temporary instance alongside the current one just long enough to verify the new version, then swaps. The whole thing takes about five to eight minutes, and nobody using the site notices a thing. (I got an up-close look at this during an outage, watching the events log boot a fresh instance and wait on its health checks before cutting over.)
The reason I love this on a single box is that “immutable” gives you the safety of blue-green without running two instances full-time. You pay for the second instance only for the few minutes a deploy is in flight.
II. The pipeline around it
A merge to main kicks off the deploy. After the usual checks and the arm64 build, the steps that matter are:
# bundle the app, upload it, register it as a version
aws elasticbeanstalk create-application-version \
--version-label <git-sha> --process
# tell the environment to run that version, then wait
aws elasticbeanstalk update-environment \
--environment-name krabber-prod --version-label <git-sha>
aws elasticbeanstalk wait environment-updated
The version label is the git SHA, so there’s a one-to-one line from “what’s running” back to “which commit.” After the environment reports healthy, the pipeline runs a smoke test through CloudFront, not against the box directly: it checks that /healthz returns 200, that / returns 200, and that /krab/login actually renders its form. Testing through CloudFront means the test exercises the real path, DNS, TLS, the WAF, and the app, the same path a user hits.
III. Rollback that doesn’t need me
Here’s the part I’m proudest of for a solo project. If that smoke test fails, the pipeline redeploys the previous version label automatically and then fails the workflow. So a bad release that somehow passed its health checks but is broken in a way only a real request reveals gets reverted without me doing anything. I find out from a red build and an email, not from users.
And if I ever need to roll back by hand, it’s one command, because Beanstalk keeps the last ten versions:
aws elasticbeanstalk update-environment \
--environment-name krabber-prod --version-label <previous-sha>
Only one deploy runs at a time, enforced by a concurrency group, so two quick merges can’t race each other onto the environment.
Conclusion
Running on one instance doesn’t mean giving up zero-downtime deploys or automatic rollback. Immutable deployments boot a fresh box, prove it healthy before switching, and keep the old one as a safety net, and a smoke test through CloudFront turns “the deploy finished” into “the deploy actually works,” reverting itself when it doesn’t. For a site I run alone, having the pipeline catch and undo a bad release on its own is worth a lot. Thanks for reading, and may your deploys always have somewhere to fall back to.