Shipping Python APIs/
Lesson

You have a CI pipelineWhat is pipeline?A sequence of automated steps (install, lint, test, build, deploy) that code passes through before reaching production. that lints, type checks, and tests your code. Now the question becomes: what happens after all checks pass? The answer is automated deployment, a CD pipeline that takes your validated code and moves it through staging, approval, and production without manual intervention at each step. This is where AI-generated pipelines fall the hardest, because deployment is not just "run a command." It is a controlled sequence of trust decisions.

The complete pipelineWhat is pipeline?A sequence of automated steps (install, lint, test, build, deploy) that code passes through before reaching production.

A production-grade CD pipeline has five stages. Each one is a gate that must pass before the next begins.

1. Build        → Create Docker image, run final compilation
2. Test         → Full test suite (unit + integration + type check)
3. Staging      → Deploy to staging environment, run smoke tests
4. Approval     → Human reviews staging, clicks "approve"
5. Production   → Deploy to production, monitor health

In a GitHub Actions workflow, this looks like a chain of jobs connected by needs and protected by environment rules:

yaml
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build Docker image
        run: |
          docker build -t myapp:${{ github.sha }} .
          docker push ghcr.io/myorg/myapp:${{ github.sha }}

  test:
    runs-on: ubuntu-latest
    needs: build
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: "pip"
      - run: pip install -r requirements.txt -r requirements-dev.txt
      - run: pytest --cov=src --cov-fail-under=80
      - run: mypy src/ --strict

  deploy-staging:
    runs-on: ubuntu-latest
    needs: test
    environment:
      name: staging
      url: https://staging.myapp.com
    steps:
      - name: Deploy to staging
        run: |
          railway up --environment staging \
            --image ghcr.io/myorg/myapp:${{ github.sha }}

  deploy-production:
    runs-on: ubuntu-latest
    needs: deploy-staging
    environment:
      name: production
      url: https://myapp.com
    steps:
      - name: Deploy to production
        run: |
          railway up --environment production \
            --image ghcr.io/myorg/myapp:${{ github.sha }}

Each needs keyword creates a dependencyWhat is dependency?A piece of code written by someone else that your project needs to work. Think of it as a building block you import instead of writing yourself.. If test fails, deploy-staging never runs. If staging never deploys, production never deploys. The chain is unbreakable.

02

Environment protection rules

GitHub environments let you attach rules to deployment targets. The most important rule: required reviewers.

To set this up:

  1. Go to your repositoryWhat is repository?A project folder tracked by Git that stores your files along with the complete history of every change, inside a hidden .git directory. Settings, then Environments
  2. Create a production environment
  3. Add required reviewers (one or more team members)
  4. Optionally add a wait timer (e.g., 5 minutes after staging deploy)

When the workflow reaches the deploy-production job, it pauses and sends a notification to the required reviewers. They can inspect the staging deployment, run manual checks, and then approve or reject. The pipelineWhat is pipeline?A sequence of automated steps (install, lint, test, build, deploy) that code passes through before reaching production. does not continue until a human says "go."

yaml
environment:
  name: production          # Must match the environment name in GitHub Settings
  url: https://myapp.com    # Shows as a link in the deployment UI
AI pitfall
AI-generated deploy workflows never use environments or protection rules. They deploy straight from CI to production in a single job. This means a single merged PR with a subtle bug goes live instantly with no human checkpoint.
03

DockerWhat is docker?A tool that packages your application and all its dependencies into a portable container that runs identically on any machine. image tagging

Every deployment should reference an immutable artifactWhat is artifact?The output files produced by a build step (like a dist/ folder) that are ready to be deployed to a server., a Docker image that is built once and deployed everywhere. The key question is how you tag that image.

yaml
# Dangerous - "latest" is a moving target
docker build -t myapp:latest .

# Safe - commit SHA is immutable and traceable
docker build -t myapp:${{ github.sha }} .

Why latest is dangerous

The latest tag is not special to Docker, it is just a convention. When you push myapp:latest, it overwrites whatever image had that tag before. If staging is running myapp:latest and you push a new build, staging silently picks up the new version on next restart. You lose the ability to know exactly what version is running where.

Tag strategyTraceabilityRollbackRisk
latestNone, which commit is "latest"?Impossible, previous image is overwrittenHigh
Commit SHAExact, every image maps to a commitTrivial, deploy the previous SHALow
Git tag (v1.2.3)Good, maps to a releaseEasy, deploy the previous tagLow
TimestampModerate, ordered but not linked to codePossible but requires a lookupMedium

The commitWhat is commit?A permanent snapshot of your staged changes saved in Git's history, identified by a unique hash and accompanied by a message describing what changed. SHA strategy is the simplest and safest. Every image is traceable to an exact commit. RollbackWhat is rollback?Undoing a database migration or deployment to restore the previous state when something goes wrong. means deploying ghcr.io/myorg/myapp:<previous-sha>.

AI pitfall
AI defaults to docker build -t myapp:latest . in every workflow. This is because AI optimizes for the simplest command, not for operational safety. Always override with ${{ github.sha }} or a version tag.
04

RollbackWhat is rollback?Undoing a database migration or deployment to restore the previous state when something goes wrong. strategies

Something will go wrong in production. The question is not if, but how fast you can recover. A rollback strategy answers: "How do we get back to the last known-good state?"

Strategy 1: Deploy the previous version

The simplest rollback: deploy the DockerWhat is docker?A tool that packages your application and all its dependencies into a portable container that runs identically on any machine. image from the previous successful deployment. If you tag images with commitWhat is commit?A permanent snapshot of your staged changes saved in Git's history, identified by a unique hash and accompanied by a message describing what changed. SHAs, this is one command:

# Current (broken) deployment: abc123
# Previous (working) deployment: def456
railway up --image ghcr.io/myorg/myapp:def456

This works because the previous image still exists in your containerWhat is container?A lightweight, portable package that bundles your application code with all its dependencies so it runs identically on any machine. registryWhat is registry?A server that stores and distributes packages or container images - npm registry for JavaScript packages, Docker Hub for container images.. Nothing was overwritten. You are not reverting code, you are deploying a known-good artifactWhat is artifact?The output files produced by a build step (like a dist/ folder) that are ready to be deployed to a server..

Strategy 2: Feature flags

Instead of rolling back the entire deployment, disable the broken feature. Feature flags let you ship code that is turned off by default and enable it gradually.

# In your application code
from myapp.flags import is_enabled

@app.get("/users/{user_id}")
async def get_user(user_id: int):
    user = await fetch_user(user_id)
    if is_enabled("new_user_profile"):
        return format_new_profile(user)
    return format_legacy_profile(user)

If the new profile format causes errors, you flip the flag off. No redeployment needed. The code is still there, but the broken path is not executed.

Rollback methodSpeedComplexityWhen to use
Deploy previous imageMinutesLowTotal breakage, need to restore everything
Feature flagSecondsMediumSpecific feature is broken, rest is fine
Git revert + redeploy10-30 minutesLowNeed a permanent code fix in the commit history

Strategy 3: Git revert and redeploy

Create a revert commit that undoes the bad change, push it, and let the pipelineWhat is pipeline?A sequence of automated steps (install, lint, test, build, deploy) that code passes through before reaching production. deploy:

git revert abc123
git push origin main
# Pipeline runs: build → test → staging → approval → production

This is the cleanest approach for permanent fixes, but it is the slowest, you wait for the full pipeline to run again. Use the "deploy previous image" strategy for immediate recovery, then follow up with a git revert for the permanent fix.

05

Smoke tests after deployment

A deployment that succeeds is not necessarily a deployment that works. Smoke tests are lightweight checks that verify the deployed application is responding correctly.

yaml
- name: Smoke test
  run: |
    sleep 10  # Wait for the deployment to stabilize
    curl --fail https://staging.myapp.com/health || exit 1
    curl --fail https://staging.myapp.com/api/v1/status || exit 1

A health endpointWhat is endpoint?A specific URL path on a server that handles a particular type of request, like GET /api/users. returns a simple 200 OK if the application is running. A status endpoint might check database connectivity, cache availability, and external service reachability:

@app.get("/health")
async def health():
    return {"status": "ok"}

@app.get("/api/v1/status")
async def status():
    db_ok = await check_database()
    cache_ok = await check_redis()
    return {
        "database": "ok" if db_ok else "error",
        "cache": "ok" if cache_ok else "error",
    }
06

What AI generates vs what production needs

What AI generatesWhat production needs
Single deploy-on-push jobStaged pipeline: build, test, staging, approval, production
No environmentsGitHub environment protection rules with required reviewers
docker build -t myapp:latestImage tagged with commit SHA for traceability
No rollback planPrevious image deploy, feature flags, git revert
No smoke testsHealth check and status endpoint verification after deploy
No staging stepStaging deployment with manual approval gate
Secrets in workflow fileSecrets in GitHub environment settings
07

The complete picture

Here is how the full pipelineWhat is pipeline?A sequence of automated steps (install, lint, test, build, deploy) that code passes through before reaching production. looks, from push to production:

Developer pushes to main
  │
  ├── Build: Docker image tagged with commit SHA
  │
  ├── Test: pytest + mypy + ruff (parallel)
  │     └── Fail → Pipeline stops, PR is blocked
  │
  ├── Deploy to staging
  │     └── Smoke tests verify staging is healthy
  │
  ├── Manual approval (required reviewer)
  │     └── Reject → Pipeline stops
  │
  └── Deploy to production
        └── Smoke tests verify production is healthy
              └── Fail → Rollback to previous SHA
08

Quick reference

ConceptImplementation
Staged pipelineChain jobs with needs: keyword
Manual approvalenvironment: with required reviewers in GitHub Settings
Image taggingdocker build -t myapp:${{ github.sha }}
RollbackDeploy previous image: myapp:<previous-sha>
Feature flagsToggle broken features off without redeploying
Smoke testscurl --fail against health and status endpoints
Environment secretsStore per-environment secrets in GitHub Settings