What does CI/CD on Databricks actually look like in 2026?

GitHub Actions (or your preferred CI) runs against the repo on every PR. It validates the Asset Bundle, runs unit tests on shared library code, and runs notebook-based integration tests against a dev workspace. On merge to main, the CI deploys the bundle to dev, then staging, then prod, each with an approval gate. Authentication is OIDC, short-lived federated tokens, no long-lived PATs.

Why OIDC instead of a Databricks PAT in CI?

PATs are long-lived secrets that have to be rotated, stored in CI secret managers, and revoked when employees leave. OIDC federates the CI environment's identity to Databricks directly, every CI run gets a fresh, scoped token. The risk surface shrinks to zero stored secrets, and audit trails name the workflow run that did the deployment.

Do I need a separate Databricks workspace per environment?

Most mature setups run separate workspaces for dev/staging/prod, with separate UC catalogs and separate service principals. The cost is real but minor compared to the safety it buys. For smaller teams, separate UC catalogs in a single workspace is acceptable; just be religious about catalog-level grants.

How do I test notebooks in CI?

Two layers. Library code factored out of notebooks into Python modules, those get unit-tested with pytest like any other code. Notebook integration tests run via the Databricks CLI: deploy the bundle to a CI target, trigger the job, assert on the resulting Delta table. The pattern is fast enough that most teams run it on every PR.

How do you handle secrets in CI/CD pipelines?

CI itself runs without long-lived Databricks secrets thanks to OIDC. Application secrets (database passwords, external API keys) live in Databricks secret scopes backed by Azure Key Vault / AWS Secrets Manager / GCP Secret Manager. The pipeline references them by name; the actual values never enter the CI environment.

Should the same pipeline deploy ML models?

Yes, but with a separate promotion model. Code changes flow through the standard CI pipeline. Model promotion uses MLflow alias changes (move `@prod` to point at the new model version) which can be gated by a CI workflow that runs evaluation, compares against the current `@prod`, and only flips the alias if the new model wins. Covered in MLOps on Databricks .

Topic guide

· Covered in Volume 3 · The Production Lakehouse

CI/CD on Databricks: GitHub Actions, OIDC, and Asset-Bundle Deployment (2026)

A modern Databricks CI/CD pipeline has four properties: declarative deployment via Asset Bundles, OIDC-federated authentication (no PATs), multi-environment promotion with approval gates, and notebook integration tests that run on every PR. This is the practitioner-level walkthrough of how to assemble that pipeline.

10 min read Updated 2026-05-18 By Ritesh Modi

Every team adopts CI/CD on Databricks the same way: they try to wire it up the way they did elsewhere, hit two specific issues, and rebuild from the ground up. The first issue is authentication, personal access tokens don't belong in a CI environment but they're the path of least resistance. The second is testing, notebooks aren't naturally testable. Both have clean solutions in 2026, and once you have them, the rest of the pipeline is straightforward.

OIDC federation: stop putting PATs in GitHub secrets

GitHub Actions can federate its workflow identity to Databricks via OAuth 2.0 token exchange. The setup, done once per repo and environment: create an OAuth federation policy in your Databricks account that trusts the specific GitHub workflow (by repository + workflow + branch), grant the corresponding service principal the privileges it needs, and configure the workflow with id-token: write. Every CI run receives a short-lived OAuth token, scoped to the SP, issued only for that run. No secret in GitHub. No rotation. No quarterly audit nightmare.

The same pattern works on GitLab, CircleCI, and most modern CI platforms. The Databricks side is identical; only the federation policy's audience claim changes.

Promotion pipeline architecture: human Git actions trigger GitHub Actions workflows that produce workspace changes — Figure 35.1 · Volume 3, Chapter 35 Promotion pipeline architecture. Each human action on the left triggers an automated workflow in the middle, producing a workspace change on the right.

The pipeline shape

A typical workflow:

on:
  pull_request:
  push:
    branches: [main]

jobs:
  validate:
    permissions:
      id-token: write
    steps:
      - uses: actions/checkout@v4
      - uses: databricks/setup-cli@v0
      - run: databricks bundle validate -t dev
      - run: databricks bundle deploy -t dev
      - run: databricks bundle run integration_tests -t dev

  deploy_staging:
    if: github.ref == 'refs/heads/main'
    needs: validate
    environment: staging
    steps:
      - uses: actions/checkout@v4
      - uses: databricks/setup-cli@v0
      - run: databricks bundle deploy -t staging

  deploy_prod:
    if: github.ref == 'refs/heads/main'
    needs: deploy_staging
    environment: production   # gated by required reviewers
    steps:
      - uses: actions/checkout@v4
      - uses: databricks/setup-cli@v0
      - run: databricks bundle deploy -t prod

Same bundle, three targets, two gates (CI tests, manual prod approval). The promotion path is legible at a glance, which is what review and audit actually need.

Testing: unit, integration, contract

Unit tests: factor logic out of notebooks into Python modules under src/. Test them with pytest in the standard way. The CI runs pytest on every PR, no Databricks workspace needed.

Integration tests: databricks bundle deploy -t ci deploys to a CI-specific target, databricks bundle run integration_test_job -t ci triggers the test job, and a final step asserts on the resulting tables or model registry entries. Each PR gets a clean, isolated deployment.

Contract tests: SDP expectations double as data quality contracts. If a PR breaks the contract, the SDP pipeline fails its update; the CI sees the failure and blocks the merge. Quality enforcement happens where the data lives.

Environments: workspaces, catalogs, and SPs

The pattern that consistently holds up at scale:

Separate workspaces for dev / staging / prod.
Separate UC catalogs (dev_retail, staging_retail, prod_retail) plus a global shared_catalog for cross-environment reference data.
One SP per environment per service, narrow grants, named after the workload they own.
OIDC federation policies that grant only the specific SP each workflow needs.

The PR experience matters

A pipeline that takes 30 minutes to run on every PR is one nobody waits for. Cache the bundle deploy step (it's idempotent), parallelize the integration test job's tasks, and aim for a sub-10-minute total. The faster the loop, the more code people will trust the platform with.

What's in Volume 3 on this

Chapter 35 is the CI/CD chapter: OIDC federation setup end-to-end (GitHub side and Databricks side), the workflow template above expanded with caching and matrix strategies, the testing architecture, environment promotion patterns, and an opinionated rollback playbook. The foundation is Asset Bundles (Ch 34); everything else is wrapping that with the right gates.