--- title: "Automate Databricks Unity Catalog Access via Terraform" description: "Codify Databricks Unity Catalog grants in Terraform with an entitlement matrix, imported existing grants, and a plan gate that fails on access drift." canonical: "https://adamtheautomator.com/databricks-unity-catalog-terraform/" --- # Automate Databricks Unity Catalog Access via Terraform > Codify Databricks Unity Catalog grants in Terraform with an entitlement matrix, imported existing grants, and a plan gate that fails on access drift. Source: https://adamtheautomator.com/databricks-unity-catalog-terraform/ --- ATA Learning Tap to hide [ ATA Learning ](/) * [Home](/) * [Tutorials](/tutorials/) * [Instructors](/author/) * [Advertising](/advertising/) * [Recommended Resources](/resources/) * [About Adam](/about-adam/) Search for: * [](https://twitter.com/adbertram) * [](https://github.com/Adam-the-Automator) * [](https://www.linkedin.com/company/adam-the-automator-llc) * [](/feed/) ![Automate Databricks Unity Catalog Access via Terraform](https://adamtheautomator.com/wp-content/uploads/publisher/3ec5d9c85b2b81afa288c4d3051031c0/69c71e320fe0086b7a86a8db15d32b1b8a3e0933856b53ca066ad83d2df8012a.webp) # Automate Databricks Unity Catalog Access via Terraform [![](https://secure.gravatar.com/avatar/d0b9d42e21e5622713f8b693aa5c0f9244d5f7dd200ed29b8398f52dee5de337?s=192&d=mm&r=g)Adam Bertram](https://adamtheautomator.com/author/adam-bertram/)9 November 202610 min. read Categories: [Cloud](/category/cloud/) Tags:[Terraform](/tag/terraform/)[Azure Databricks](/tag/azure-databricks/)[Data Engineering](/tag/data-engineering/)[Governance](/tag/governance/) Table of Contents * [The Privilege Model You Are Encoding](#the-privilege-model-you-are-encoding) * [Inheritance Runs Down, With One Exception](#inheritance-runs-down-with-one-exception) * [Ownership Is a Separate Axis From MANAGE](#ownership-is-a-separate-axis-from-manage) * [Prerequisites for a Databricks Unity Catalog Terraform Pipeline](#prerequisites-for-a-databricks-unity-catalog-terraform-pipeline) * [Step 1: Wire Up the Account and Workspace Providers](#step-1-wire-up-the-account-and-workspace-providers) * [Step 2: Create the Catalog and Schema](#step-2-create-the-catalog-and-schema) * [Step 3: Drive Unity Catalog Grants With an Entitlement Matrix](#step-3-drive-unity-catalog-grants-with-an-entitlement-matrix) * [Declare the Pipeline’s Own MANAGE Grant](#declare-the-pipelines-own-manage-grant) * [Step 4: Adopt the Grants You Already Have](#step-4-adopt-the-grants-you-already-have) * [Step 5: Verify What Each Principal Can Actually Reach](#step-5-verify-what-each-principal-can-actually-reach) * [Run It From a Pipeline and Catch Drift](#run-it-from-a-pipeline-and-catch-drift) * [What Could Go Wrong](#what-could-go-wrong) * [Make Access Review a Diff](#make-access-review-a-diff) Before you point a `databricks_grants` resource at a catalog that already carries grants, pull the current list. The resource is authoritative for its securable: the first apply deletes every grant the configuration does not name, including the `MANAGE` your pipeline is using to make the change. Terraform treats the omission as [resource drift](https://developer.hashicorp.com/terraform/tutorials/state/resource-drift) and removes it mid-apply. The build below splits account work from workspace work across two providers, then creates the catalog and schemas from code. From there it drives `databricks_grants` from an entitlement matrix, adopts the grants already in place, and fails a pipeline when the plan is not empty. You finish with access evidence an auditor can diff. ## The Privilege Model You Are Encoding Terraform is only the notation. The shape of the object still has to be right, because a grant placed one level too high hands out more access than the pull request review implies. Unity Catalog [arranges securables](https://docs.databricks.com/aws/en/data-governance/unity-catalog/) from the metastore, through catalogs and schemas, down to tables, views and volumes. The [`databricks_grants` reference](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/grants) says: “Securable objects are hierarchical and privileges are inherited downward. The highest level object that privileges are inherited from is the catalog.” It adds “Privileges that are granted on a metastore are not inherited.” ### Inheritance Runs Down, With One Exception A catalog grant applies to every current and future object inside it, and a schema grant does the same one level down. The [privileges reference](https://docs.databricks.com/aws/en/data-governance/unity-catalog/access-control/privileges-reference) names the single exception: “When granted on the metastore, `READ METADATA` inherits down to all objects in the metastore.” Traversal causes most support tickets. Unity Catalog has no `USAGE` privilege; it uses `USE CATALOG` and `USE SCHEMA`, and the same reference says “Even if a table owner grants `SELECT` on a table to another user, that user cannot access the table unless they also have `USE CATALOG` on the parent catalog.” Any principal reading a table needs both. The diagram below maps the four levels where a privilege can land. ![Privilege inheritance path](https://adamtheautomator.com/wp-content/uploads/publisher/2d2cc9d1db41a5aa261de618319c0e3cb26cb54bfa0f359e398a5d302bf02674.jpg) How privileges inherit ### Ownership Is a Separate Axis From `MANAGE` The principal that creates an object owns it and receives `ALL_PRIVILEGES` on it plus the right to grant. `MANAGE` is a separate, grantable administration privilege, and the two [are documented apart](https://docs.databricks.com/aws/en/data-governance/unity-catalog/manage-privileges) because they behave differently in a review. `ALL_PRIVILEGES` excludes `MANAGE`, `READ METADATA`, `EXTERNAL USE SCHEMA` and `EXTERNAL USE LOCATION`, so a catalog-wide grant hands out `MODIFY` and `CREATE_TABLE` below it without administration. A matrix for a three-group domain looks like this: | Principal | Catalog | Schema | Table | | --- | --- | --- | --- | | `sales-analysts` group | `USE_CATALOG` | `USE_SCHEMA`, `SELECT` | none | | `sales-engineers` group | `USE_CATALOG`, `CREATE_SCHEMA` | `USE_SCHEMA`, `SELECT`, `MODIFY`, `CREATE_TABLE` | none | | Pipeline service principal | `USE_CATALOG`, `CREATE_SCHEMA`, `MANAGE` | `USE_SCHEMA`, `MANAGE` | none | | `orders-app` service principal | `USE_CATALOG` | `USE_SCHEMA`, `SELECT` | `SELECT` on three tables | The module below encodes the first three rows. The `orders-app` row shows the shape a table-level grant takes: it lives in a `databricks_grant` resource for one principal on one table, which this module describes and does not encode. Groups carry human access, so membership churn lands in the identity provider. Service principals carry machine access and appear by application ID inside a `grant` block. ## Prerequisites for a Databricks Unity Catalog Terraform Pipeline If you want to follow along against a real workspace, you will need: * Terraform 1.5 or later, because the [`import` block](https://developer.hashicorp.com/terraform/language/import) that adopts existing grants arrived in that release. * The [Databricks CLI](https://docs.databricks.com/aws/en/dev-tools/cli/reference/grants-commands) for its `grants` command group, which is how you read what the apply produced. * A workspace attached to a Unity Catalog metastore, plus `CREATE CATALOG` on that metastore. Grant it to a group rather than making the pipeline a metastore admin. * A service principal with an OAuth client ID and secret, so the pipeline identity stays separate from any person’s account. * A storage credential and external location if the catalog reads from cloud storage you manage. Skip both for managed tables. The configuration in this article was validated with Terraform v1.9.8 and `databricks/databricks` provider v1.141.0, the release `terraform init` resolves from the `~> 1.0` constraint. That constraint is loose enough to accept a later provider release, so read the version your own `.terraform.lock.hcl` records after init. ## Step 1: Wire Up the Account and Workspace Providers Two kinds of Unity Catalog work go to two different APIs, and mixing them produces errors that read like authentication failures. The [provider documentation](https://registry.terraform.io/providers/databricks/databricks/latest/docs#special-configuration-for-unity-catalog) is explicit that “Except for metastore, metastore assignment and storage credential objects, Unity Catalog APIs are accessible via workspace-level APIs.” | Work | Provider | Host | | --- | --- | --- | | Metastore, metastore assignment | Account | `https://accounts.azuredatabricks.net` on Azure, `https://accounts.cloud.databricks.com` on AWS | | Catalogs, schemas, grants | Workspace | Your workspace URL | Declare both in one root module: ```hcl terraform { required_providers { databricks = { source = "databricks/databricks" version = "~> 1.0" } } } provider "databricks" { alias = "account" host = "https://accounts.azuredatabricks.net" account_id = var.databricks_account_id } provider "databricks" { alias = "workspace" host = var.workspace_url } ``` The walkthrough targets Azure, so the account block points at the Azure accounts host. On AWS, point the same field at `https://accounts.cloud.databricks.com` and set `TF_VAR_workspace_url` to the `.cloud.databricks.com` workspace URL. On Azure, leave both as written. Set `account_id` on the account provider only. The provider docs warn that setting it on a workspace-level provider produces an “invalid Databricks Account configuration” error, which sends people hunting through credentials when the problem is the block they copied. Export the credentials and the declared inputs in one place: ```bash export DATABRICKS_CLIENT_ID="" export DATABRICKS_CLIENT_SECRET="" export TF_VAR_databricks_account_id="" export TF_VAR_workspace_url="https://.azuredatabricks.net" export TF_VAR_pipeline_principal_id="" ``` Declare the same four inputs in the root module, because a `TF_VAR_` value only populates an existing variable. `pipeline_principal_id` takes the application ID you exported as `DATABRICKS_CLIENT_ID`, and `environment` defaults to `prod`, the value Step 4 assumes: ```hcl variable "databricks_account_id" { type = string } variable "workspace_url" { type = string } variable "environment" { type = string default = "prod" } variable "pipeline_principal_id" { type = string } ``` Set `TF_VAR_environment=dev` only for a non-production workspace. The `prod` default is what lets `terraform plan -input=false` run unattended. Every resource below sets `provider = databricks.workspace`; a metastore takes `databricks.account`. ## Step 2: Create the Catalog and Schema A [catalog](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/catalog) maps to a business domain, and the environment belongs in its name so a misdirected apply fails on a missing object instead of writing production grants from a staging pipeline. ```hcl resource "databricks_catalog" "sales" { provider = databricks.workspace name = "sales_${var.environment}" comment = "Sales domain, managed by platform engineering" } locals { schemas = ["raw", "curated"] } resource "databricks_schema" "domain" { provider = databricks.workspace for_each = toset(local.schemas) catalog_name = databricks_catalog.sales.name name = each.key comment = "Managed in Terraform, changes go through pull request" } ``` The `for_each` keeps the file short as the domain grows, and the `catalog_name` reference orders the [schema](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/schema) behind the catalog without `depends_on`. Apply this much and open **Catalog Explorer**: the catalog and schemas appear with your comments, and the **Permissions** tab shows the owner and nothing else. Nobody can read the data yet. ![Catalog Explorer permissions](https://adamtheautomator.com/wp-content/uploads/publisher/3d96cb4393a6c0669e75f4d36b61510e5e401f42d035d207d802547ac354da2c.jpg) Creating the catalog ## Step 3: Drive Unity Catalog Grants With an Entitlement Matrix Hand-written grant blocks do not survive a real catalog. Six groups across forty schemas is 240 decisions scattered through HCL, and the one that drifts is invisible in review. Keep the decisions in a map keyed by the securable and let `for_each` build the resources. ```hcl locals { schema_privileges = { raw = { "sales-engineers" = ["USE_SCHEMA", "SELECT", "MODIFY", "CREATE_TABLE"] } curated = { "sales-analysts" = ["USE_SCHEMA", "SELECT"] "sales-engineers" = ["USE_SCHEMA", "SELECT", "MODIFY", "CREATE_TABLE"] } } } resource "databricks_grants" "schema" { provider = databricks.workspace for_each = local.schema_privileges schema = "${databricks_catalog.sales.name}.${databricks_schema.domain[each.key].name}" dynamic "grant" { for_each = each.value content { principal = grant.key privileges = grant.value } } } ``` The key is the schema, and every principal for that schema sits inside one entry. Two `databricks_grants` resources naming the same schema remove each other’s grants on every apply, so the pair trades access back and forth forever. When one data team needs a reader on one table, use [`databricks_grant`](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/grant): it is authoritative for a single principal on a single securable, so it leaves everyone else alone. ### Declare the Pipeline’s Own `MANAGE` Grant The identity running the apply needs its privileges written down. The provider docs put it bluntly: “When applying grants using an identity with `MANAGE` permission, their `MANAGE` permission must also be defined, otherwise Terraform will remove their permissions, leading to errors.” A config that grants `MANAGE` to everyone except the service principal applying it strips that principal on the first run. ```hcl resource "databricks_grants" "catalog" { provider = databricks.workspace catalog = databricks_catalog.sales.name grant { principal = var.pipeline_principal_id privileges = ["USE_CATALOG", "CREATE_SCHEMA", "MANAGE"] } grant { principal = "sales-analysts" privileges = ["USE_CATALOG"] } grant { principal = "sales-engineers" privileges = ["USE_CATALOG", "CREATE_SCHEMA"] } } ``` Privilege names use underscores in Terraform (`CREATE_TABLE`, `USE_CATALOG`); the SQL form with spaces fails validation. * * * _**Warning: `databricks_grants` owns its securable. If an existing SQL `GRANT` or an admin in the UI adds a principal your map does not list, the next apply removes it. On an estate that already carries grants, adopt first, then apply.**_ * * * ## Step 4: Adopt the Grants You Already Have Steps 2 and 3 create `sales_prod` from nothing, so a reader who ran them has nothing to import: Terraform owns those addresses. This step covers the other estate: catalog and schemas built by hand or in SQL. The Step 3 warning says adopt first, then apply, so import before that apply and skip it on the first pass. The catalog here is `sales_prod`, from the `prod` default on `var.environment`. Import attaches an existing object to an address in your configuration, so read the grants, write the map, then import. ```bash databricks grants get catalog sales_prod --profile prod ``` That prints the principals and privileges held directly on the catalog. Translate them into `grant` blocks, including the service principal’s `MANAGE`, then import: ```bash terraform import databricks_grants.catalog catalog/sales_prod ``` Terraform 1.5 and later accepts an import block, which keeps adoption inside the reviewed plan: ```hcl import { to = databricks_grants.schema["curated"] id = "schema/sales_prod.curated" } ``` For dozens of schemas, an [experimental resource exporter](https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/experimental-exporter) writes configuration for what it finds: ```bash ./terraform-provider-databricks_v exporter -skip-interactive ``` Copy it out of `.terraform/providers/registry.terraform.io/databricks/databricks//_/`, taking `` from `terraform version` or the lock file; for the versions above that file is `terraform-provider-databricks_v1.141.0`. The exporter authenticates with `DATABRICKS_HOST` and `DATABRICKS_TOKEN`, a different pair from Step 1’s OAuth client ID and secret. Treat that output as a junior engineer’s draft: it inherits every grant that exists today, including ones nobody can explain. For an adopted estate, the proof is a plan with no changes. Run `terraform plan` after the import and investigate any diff, because a diff means your map misses a principal about to lose access. ## Step 5: Verify What Each Principal Can Actually Reach Two [CLI commands](https://docs.databricks.com/aws/en/dev-tools/cli/reference/grants-commands) answer different questions. `databricks grants get` returns what is granted directly on the securable; `get-effective` adds what the principal inherits from the catalogs and schemas above it. ```bash databricks grants get-effective schema sales_prod.curated \ --principal sales-analysts \ --profile prod ``` Use `get` to confirm the configuration matches the securable, and `get-effective` for the question an auditor asks: can this group read this table? A principal can hold `SELECT` on a table and still fail to query it. The `get-effective` listing below shows the container privileges a user inherits, so a missing one surfaces before a user reports it. ![Effective grant output](https://adamtheautomator.com/wp-content/uploads/publisher/8cf0f2c7ec9c5e87e5b048b3c658846cf66b342cd672057c3b2434adc6022fcf.jpg) Verifying effective grants * * * _**Pro Tip: capture the `get-effective` output in the pipeline run that applied the change. A plan diff plus an effective-grants listing is what replaces the screenshot in an audit folder.**_ * * * ## Run It From a Pipeline and Catch Drift A plan that exits zero tells you the configuration parsed. `terraform plan -detailed-exitcode` tells you whether reality matches it, returning 2 when there is a difference. The [plan documentation](https://developer.hashicorp.com/terraform/cli/commands/plan) defines that code and the saved plan file `terraform apply` consumes. ```bash terraform init -input=false init_status=$? if [ "$init_status" -ne 0 ]; then echo "terraform init failed: the gate cannot compare anything until this passes" exit 1 fi terraform plan -input=false -detailed-exitcode -out=tfplan status=$? if [ "$status" -eq 0 ]; then echo "No changes: reality matches the configuration" elif [ "$status" -eq 2 ]; then echo "Drift or pending change: review the plan artifact" exit 1 else echo "terraform plan failed with status $status: the gate proves nothing" exit 1 fi ``` A plan that errors or cannot authenticate exits 1, and a gate that lets that status through reports success on a workspace it never read. A pull request job should fail on exit code 2 and publish `tfplan` as a build artifact. [A scheduled job against production](https://adamtheautomator.com/automate-terraform-azure-devops/) should do the same, which turns a silent manual edit into a red build the next morning. Recovery from a bad apply is a Git revert and a normal apply, and the correction stays in the history where the next audit can find it. Keep the service principal’s secret in your CI store, or use [workload identity federation](https://docs.databricks.com/aws/en/dev-tools/auth/oauth-federation-provider) where your provider supports it. ## What Could Go Wrong Most of the painful days on this pattern come from mixing these three resources up. | Resource | Scope of authority | Use it when | | --- | --- | --- | | `databricks_grants` | Every grant on one securable | Platform teams own the catalog or schema | | `databricks_grant` | Every grant for one principal on one securable | A data team adds one reader to one table | | `databricks_sql_permissions` | Hive metastore table ACLs, applied through a cluster | Never, on a Unity Catalog workspace | The [legacy resource](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/sql_permissions) deserves more than a table cell. `databricks_sql_permissions` runs `SHOW GRANT`, `REVOKE ALL PRIVILEGES` and `GRANT` against a cluster, creating one named `terraform-table-acl` when `cluster_id` is omitted. The provider recommends `databricks_grants` because it “doesn’t require a technical cluster to perform operations,” and it documents the collision error: `cannot create sql permissions: cannot read current grants: For unity catalog, please specify the catalog name explicitly`. That error means the workspace has a Unity Catalog `default_catalog_name`. Two failures read as one problem and are another: * Two `databricks_grants` resources on one securable remove each other’s grants on alternating applies, which resembles a race until you read both blocks. * A config that changes an object’s `owner` and other fields in one apply can fail midway, because the provider updates ownership first and attempts the rest as the new owner. ## Make Access Review a Diff The catalog, schemas and grants now live in one repository, and the pipeline refuses to let reality drift from it. A reviewer sees which group gains which privilege before the apply, and an auditor gets a commit, an approval, a plan artifact and an effective-grants listing instead of a screenshot that was accurate for one afternoon. The [Unity Catalog Terraform guide](https://docs.databricks.com/aws/en/dev-tools/terraform/automate-uc) covers the setup once; the repository keeps it true. Start with the catalog your team argues about most. Read its grants, write the map, import the resource, and let the plan run clean. Once one behaves, the second takes an afternoon. Share this article [Share on X](https://twitter.com/intent/tweet?url=https%3A%2F%2Fadamtheautomator.com%2Fdatabricks-unity-catalog-terraform%2F&text=Automate%20Databricks%20Unity%20Catalog%20Access%20via%20Terraform)[Share on Facebook](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fadamtheautomator.com%2Fdatabricks-unity-catalog-terraform%2F)[Share on LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fadamtheautomator.com%2Fdatabricks-unity-catalog-terraform%2F) ## Related Posts ![](https://adamtheautomator.com/wp-content/uploads/2026/02/featured_image-21.webp) ### [Azure Databricks Serverless Cost Optimization Guide](/azure-databricks-serverless-cost-optimization/) Learn to reduce Azure Databricks costs with serverless compute, Liquid Clustering, auto-termination tuning, and budget controls. ![](https://adamtheautomator.com/wp-content/uploads/2026/01/featured_image-1.webp) ### [Effortless Spark: A Beginner\\’s Guide to Serverless Workspaces in Azure Databricks](/effortless-spark-beginners-guide-serverless/) Learn how Azure Databricks serverless workspaces eliminate infrastructure complexity. This beginner's guide covers setup, costs, limitations, and when to use serverless vs traditional deployments. ![](https://adamtheautomator.com/wp-content/uploads/2026/08/featured_image-7.webp) ### [Build Azure Landing Zones That Prevent Cloud Sprawl](/build-azure-landing-zones/) Build governed Azure landing zones end to end with the IaC accelerator: management group hierarchies, the eight design areas, Terraform and Bicep deployments, ## Categories * [IT Ops](/category/it-ops/) * [Cloud](/category/cloud/) * [DevOps](/category/devops/) * [Home Ops](/category/home-ops/) * [Information Security](/category/infosec/) * [Software Development](/category/software-development/) ## Site * [Home](/) * [Tutorials](/tutorials/) * [Instructors](/author/) * [Advertising](/advertising/) * [Recommended Resources](/resources/) * [About Adam](/about-adam/) Copyright 2026© ATA Learning | [Privacy Policy](/privacy/)