The Blue-Green Rollback That Only Undoes the Traffic
What the Router Actually Reverses
The core promise of blue-green deployment is the speed of the cut-over. Martin Fowler’s original description, published in 2010, frames the technique as a way to minimize downtime by having two identical production environments. One is live, the other is idle or staging. When a new release is ready in the green environment, the operator switches the router so all incoming requests flow to green. If something goes wrong, the router is switched back to blue. This reversal is mechanical and fast. It changes which set of servers receives HTTP requests or API calls. It does not, however, change the state of the data store, the message queue, or the external systems that the application has already spoken to.
The illusion of safety comes from conflating the traffic switch with the release itself. In a purely stateless web service, where every request is self-contained and the application logic is the only variable, switching the router does indeed revert the user-facing behavior. The blue environment is still running the old code, and it is still connected to the same database. If the database schema has not changed, the old code can read and write to it without issue. The rollback is clean. But this is the narrowest possible case. It assumes that the new version of the software did not alter any persistent state during the period it was live. The moment the new version writes to a database, sends a message, or calls an external API, the symmetry between the two environments breaks. The router can be moved back, but the system is no longer in the state it was in before the green environment went live.
The State That Stays Behind
The most common point of failure is the database schema. Fowler explicitly notes that databases can be a challenge with this technique. The standard mitigation is to separate schema changes from application upgrades. You first apply a database refactoring that supports both the old and new versions of the code. Then you deploy the new application. This works for the forward path. But if you roll back to the blue environment, the database is still in the new schema. The old code in the blue environment might still work, but only if the schema change was strictly additive. If the new version dropped a column, renamed a field, or changed the type of a value, the old code will fail. The rollback restores the application logic, but it leaves the data in a format the old logic may not understand.
Beyond the database, there are other forms of state that do not roll back. If the new version of the service sends a message to a queue in a new format, and the downstream consumer in the blue environment expects the old format, the rollback does not fix the queue. The messages are already there. If the new version warms a cache with data that is only valid under the new logic, the blue environment may serve stale or incorrect data from that cache. If the new version triggers an external side effect, such as sending an email or creating a resource in a third-party API, that action has already happened. You cannot unsend an email. You cannot uncreate a resource. The traffic switch is a local operation within your infrastructure boundary. It has no authority over the external world.
Comparing Against Incremental Rollout
Canarying, or progressive rollout, addresses this problem by changing the exposure model. The Google SRE Workbook defines canarying as a partial and time-limited deployment of a change. Instead of switching all traffic to the new version at once, you send a small percentage of traffic to the canary. The rest of the traffic continues to flow to the control group, which is running the old version. This means that the state changes introduced by the new version are limited to the subset of users who hit the canary. If the new version writes to the database, it only writes for those users. If it sends a message, it only sends for those users.
The advantage of this approach is that the blast radius of the state change is contained. If you discover a problem, you can stop the rollout before the new version has processed a significant portion of the traffic. The state changes that have occurred are limited to the canary subset. This makes the rollback more manageable because you are not dealing with a system-wide state divergence. You are dealing with a small, bounded set of changes. However, canarying does not eliminate the problem of irreversible state changes. It only reduces the volume. If the new version sends an email to one user, that email is still sent. If it creates a resource in an external API, that resource is still created. The difference is that you are not doing it for every user. You are doing it for a small fraction. This makes the cleanup process more feasible, but it does not make it unnecessary.
Which Release Shapes Stay Dangerous
The choice between blue-green and canarying depends on the nature of the changes in the release. If the release is purely a code change with no schema changes, no message format changes, and no external side effects, blue-green is a perfectly valid and efficient strategy. The rollback is clean because there is no state divergence. If the release involves a schema change, blue-green requires careful planning to ensure that the schema is backward compatible. If it is not, the rollback will fail. If the release involves external side effects, both blue-green and canarying are dangerous, but canarying is less dangerous because the side effects are limited in scope.
The key insight is that the risk is not in the traffic switch. The risk is in the state change. The traffic switch is a reversible operation. The state change is often not. Teams that choose blue-green because it feels safer are often mistaken. They are confusing the ease of the rollback with the safety of the release. The release is only as safe as the most irreversible state change it introduces. If your release changes the schema in a way that the old code cannot handle, blue-green will not save you. If your release sends a message in a format that the old code cannot parse, blue-green will not save you. The only way to make a release truly risk-free is to ensure that all state changes are backward compatible and reversible. If you cannot do that, you need to use a strategy that limits the exposure of those changes, such as canarying.