New post idea: POCDR #64
Description
POCDR ("Proof of Concept Delivery/Rollback") attempts to define a generalized CI/CD methodology for engineering the maximization of confidence in a production deployment.
Essentially, it is a logical extension of the canary deployment model, but for systems in which a phased rollout is not realistic or feasible. It does this by offering a more explicit strategy for applying overlapping concepts from the 12-factor app, cloud native, infrastructure as code, and Google SRE manifestos. While it does more or less rephrase inter-related points between those approaches, it does so in a way that specifically focuses on solving traditionally hard problems in changing production systems which:
- are mission critical
- have sensitive uptime requirements
- are on the write path
- struggle to justify the risks associated with:
- adding new features
- basic system maintenance (e.g, dependency updates)
- decomposing responsibility (i..e., "microservice fragmentation")
Especially if they:
- have a write path which cannot be parallelized
- have a large "blast radius" of downstream systems
- rely on system-wide infrastructure with similar constraints
- is itself a said system-wide infrastructure component
- must detect erroneous system output and apply alterations afterwards
Kubernetes, kafka, cassandra, and git are referenced, but only for an example implementation.
Bedrock system components are expected to have a reasonable granularity to their access control features.
Include the "city water purifier" analogy.
Include a high level summary of the principles of a POCDR-compliant system, similar to the approach used in Google's SRE manifesto.
Includes a breakdown of the high level summary as a chart outlining the "Levels of Maturity", similar to how this article does it.