You have a CI pipelineWhat is pipeline?A sequence of automated steps (install, lint, test, build, deploy) that code passes through before reaching production. that lints, type checks, and tests your code. Now the question becomes: what happens after all checks pass? The answer is automated deployment, a CD pipeline that takes your validated code and moves it through staging, approval, and production without manual intervention at each step. This is where AI-generated pipelines fall the hardest, because deployment is not just "run a command." It is a controlled sequence of trust decisions.
The complete pipelineWhat is pipeline?A sequence of automated steps (install, lint, test, build, deploy) that code passes through before reaching production.
A production-grade CD pipeline has five stages. Each one is a gate that must pass before the next begins.
1. Build → Create Docker image, run final compilation
2. Test → Full test suite (unit + integration + type check)
3. Staging → Deploy to staging environment, run smoke tests
4. Approval → Human reviews staging, clicks "approve"
5. Production → Deploy to production, monitor healthIn a GitHub Actions workflow, this looks like a chain of jobs connected by needs and protected by environment rules:
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build Docker image
run: |
docker build -t myapp:${{ github.sha }} .
docker push ghcr.io/myorg/myapp:${{ github.sha }}
test:
runs-on: ubuntu-latest
needs: build
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: "pip"
- run: pip install -r requirements.txt -r requirements-dev.txt
- run: pytest --cov=src --cov-fail-under=80
- run: mypy src/ --strict
deploy-staging:
runs-on: ubuntu-latest
needs: test
environment:
name: staging
url: https://staging.myapp.com
steps:
- name: Deploy to staging
run: |
railway up --environment staging \
--image ghcr.io/myorg/myapp:${{ github.sha }}
deploy-production:
runs-on: ubuntu-latest
needs: deploy-staging
environment:
name: production
url: https://myapp.com
steps:
- name: Deploy to production
run: |
railway up --environment production \
--image ghcr.io/myorg/myapp:${{ github.sha }}Each needs keyword creates a dependencyWhat is dependency?A piece of code written by someone else that your project needs to work. Think of it as a building block you import instead of writing yourself.. If test fails, deploy-staging never runs. If staging never deploys, production never deploys. The chain is unbreakable.
Environment protection rules
GitHub environments let you attach rules to deployment targets. The most important rule: required reviewers.
To set this up:
- Go to your repositoryWhat is repository?A project folder tracked by Git that stores your files along with the complete history of every change, inside a hidden .git directory. Settings, then Environments
- Create a
productionenvironment - Add required reviewers (one or more team members)
- Optionally add a wait timer (e.g., 5 minutes after staging deploy)
When the workflow reaches the deploy-production job, it pauses and sends a notification to the required reviewers. They can inspect the staging deployment, run manual checks, and then approve or reject. The pipelineWhat is pipeline?A sequence of automated steps (install, lint, test, build, deploy) that code passes through before reaching production. does not continue until a human says "go."
environment:
name: production # Must match the environment name in GitHub Settings
url: https://myapp.com # Shows as a link in the deployment UIDockerWhat is docker?A tool that packages your application and all its dependencies into a portable container that runs identically on any machine. image tagging
Every deployment should reference an immutable artifactWhat is artifact?The output files produced by a build step (like a dist/ folder) that are ready to be deployed to a server., a Docker image that is built once and deployed everywhere. The key question is how you tag that image.
# Dangerous - "latest" is a moving target
docker build -t myapp:latest .
# Safe - commit SHA is immutable and traceable
docker build -t myapp:${{ github.sha }} .Why latest is dangerous
The latest tag is not special to Docker, it is just a convention. When you push myapp:latest, it overwrites whatever image had that tag before. If staging is running myapp:latest and you push a new build, staging silently picks up the new version on next restart. You lose the ability to know exactly what version is running where.
| Tag strategy | Traceability | Rollback | Risk |
|---|---|---|---|
latest | None, which commit is "latest"? | Impossible, previous image is overwritten | High |
| Commit SHA | Exact, every image maps to a commit | Trivial, deploy the previous SHA | Low |
| Git tag (v1.2.3) | Good, maps to a release | Easy, deploy the previous tag | Low |
| Timestamp | Moderate, ordered but not linked to code | Possible but requires a lookup | Medium |
The commitWhat is commit?A permanent snapshot of your staged changes saved in Git's history, identified by a unique hash and accompanied by a message describing what changed. SHA strategy is the simplest and safest. Every image is traceable to an exact commit. RollbackWhat is rollback?Undoing a database migration or deployment to restore the previous state when something goes wrong. means deploying ghcr.io/myorg/myapp:<previous-sha>.
docker build -t myapp:latest . in every workflow. This is because AI optimizes for the simplest command, not for operational safety. Always override with ${{ github.sha }} or a version tag.RollbackWhat is rollback?Undoing a database migration or deployment to restore the previous state when something goes wrong. strategies
Something will go wrong in production. The question is not if, but how fast you can recover. A rollback strategy answers: "How do we get back to the last known-good state?"
Strategy 1: Deploy the previous version
The simplest rollback: deploy the DockerWhat is docker?A tool that packages your application and all its dependencies into a portable container that runs identically on any machine. image from the previous successful deployment. If you tag images with commitWhat is commit?A permanent snapshot of your staged changes saved in Git's history, identified by a unique hash and accompanied by a message describing what changed. SHAs, this is one command:
# Current (broken) deployment: abc123
# Previous (working) deployment: def456
railway up --image ghcr.io/myorg/myapp:def456This works because the previous image still exists in your containerWhat is container?A lightweight, portable package that bundles your application code with all its dependencies so it runs identically on any machine. registryWhat is registry?A server that stores and distributes packages or container images - npm registry for JavaScript packages, Docker Hub for container images.. Nothing was overwritten. You are not reverting code, you are deploying a known-good artifactWhat is artifact?The output files produced by a build step (like a dist/ folder) that are ready to be deployed to a server..
Strategy 2: Feature flags
Instead of rolling back the entire deployment, disable the broken feature. Feature flags let you ship code that is turned off by default and enable it gradually.
# In your application code
from myapp.flags import is_enabled
@app.get("/users/{user_id}")
async def get_user(user_id: int):
user = await fetch_user(user_id)
if is_enabled("new_user_profile"):
return format_new_profile(user)
return format_legacy_profile(user)If the new profile format causes errors, you flip the flag off. No redeployment needed. The code is still there, but the broken path is not executed.
| Rollback method | Speed | Complexity | When to use |
|---|---|---|---|
| Deploy previous image | Minutes | Low | Total breakage, need to restore everything |
| Feature flag | Seconds | Medium | Specific feature is broken, rest is fine |
| Git revert + redeploy | 10-30 minutes | Low | Need a permanent code fix in the commit history |
Strategy 3: Git revert and redeploy
Create a revert commit that undoes the bad change, push it, and let the pipelineWhat is pipeline?A sequence of automated steps (install, lint, test, build, deploy) that code passes through before reaching production. deploy:
git revert abc123
git push origin main
# Pipeline runs: build → test → staging → approval → productionThis is the cleanest approach for permanent fixes, but it is the slowest, you wait for the full pipeline to run again. Use the "deploy previous image" strategy for immediate recovery, then follow up with a git revert for the permanent fix.
Smoke tests after deployment
A deployment that succeeds is not necessarily a deployment that works. Smoke tests are lightweight checks that verify the deployed application is responding correctly.
- name: Smoke test
run: |
sleep 10 # Wait for the deployment to stabilize
curl --fail https://staging.myapp.com/health || exit 1
curl --fail https://staging.myapp.com/api/v1/status || exit 1A health endpointWhat is endpoint?A specific URL path on a server that handles a particular type of request, like GET /api/users. returns a simple 200 OK if the application is running. A status endpoint might check database connectivity, cache availability, and external service reachability:
@app.get("/health")
async def health():
return {"status": "ok"}
@app.get("/api/v1/status")
async def status():
db_ok = await check_database()
cache_ok = await check_redis()
return {
"database": "ok" if db_ok else "error",
"cache": "ok" if cache_ok else "error",
}What AI generates vs what production needs
| What AI generates | What production needs |
|---|---|
| Single deploy-on-push job | Staged pipeline: build, test, staging, approval, production |
| No environments | GitHub environment protection rules with required reviewers |
docker build -t myapp:latest | Image tagged with commit SHA for traceability |
| No rollback plan | Previous image deploy, feature flags, git revert |
| No smoke tests | Health check and status endpoint verification after deploy |
| No staging step | Staging deployment with manual approval gate |
| Secrets in workflow file | Secrets in GitHub environment settings |
The complete picture
Here is how the full pipelineWhat is pipeline?A sequence of automated steps (install, lint, test, build, deploy) that code passes through before reaching production. looks, from push to production:
Developer pushes to main
│
├── Build: Docker image tagged with commit SHA
│
├── Test: pytest + mypy + ruff (parallel)
│ └── Fail → Pipeline stops, PR is blocked
│
├── Deploy to staging
│ └── Smoke tests verify staging is healthy
│
├── Manual approval (required reviewer)
│ └── Reject → Pipeline stops
│
└── Deploy to production
└── Smoke tests verify production is healthy
└── Fail → Rollback to previous SHAQuick reference
| Concept | Implementation |
|---|---|
| Staged pipeline | Chain jobs with needs: keyword |
| Manual approval | environment: with required reviewers in GitHub Settings |
| Image tagging | docker build -t myapp:${{ github.sha }} |
| Rollback | Deploy previous image: myapp:<previous-sha> |
| Feature flags | Toggle broken features off without redeploying |
| Smoke tests | curl --fail against health and status endpoints |
| Environment secrets | Store per-environment secrets in GitHub Settings |