Databricks Unity Catalog

Unity Catalog

Guidance for Unity Catalog governance — access control, the privilege model,

external locations, securable DDL, and fine-grained access — plus system tables and

volume file operations.

> Before running databricks CLI commands, confirm the CLI and the subcommand exist.

> Run databricks --version — this skill assumes the unified CLI (≥ v1.0.0). Several

> subcommands shown here (experimental aitools, system-schemas, external-lineage,

> grants) vary by version or workspace availability; if one is missing or rejects a flag,

> fall back to the SQL form or the Python SDK rather than guessing. Each reference notes its

> own version floor where relevant.

When to Use This Skill

Use this skill when:

Governance & access control (start here):

• Granting or revoking access — GRANT/REVOKE, the UC privilege model, ownership (ALTER … OWNER TO), SHOW GRANTS, "who can read/write this table?"
• Row- and column-level security — row filters, column masks, dynamic views with current_user() / is_account_group_member()
• External locations & storage credentials — CREATE STORAGE CREDENTIAL, CREATE EXTERNAL LOCATION, backing external tables/volumes
• Securable DDL & metadata — creating/altering catalogs, schemas, managed vs external tables, views; comments, tags, table properties, ownership

Observability & files:

• Working with volumes (upload, download, list files in /Volumes/)
• Querying lineage (table dependencies, column-level lineage)
• Analyzing audit logs (who accessed what, permission changes)
• Monitoring billing and usage (DBU consumption, cost analysis)
• Tracking compute resources (cluster usage, warehouse metrics)
• Reviewing job execution (run history, success rates, failures)
• Analyzing query performance (slow queries, warehouse utilization)
• Profiling data quality (data profiling, drift detection, metric tables)

Reference Files

| Topic | File | Description |

|-------|------|-------------|

| Access Control | references/1-access-control.md | Privilege model, securable hierarchy, GRANT/REVOKE, ownership, inheritance, SHOW GRANTS |

| External Locations | references/2-external-locations.md | Storage credentials (AWS/Azure/GCP), external locations, validation |

| Securables DDL | references/3-securables-ddl.md | CREATE/ALTER/DROP catalogs/schemas/tables/views, comments, tags, ownership |

| Fine-Grained Access | references/4-fine-grained-access.md | Row filters, column masks, dynamic views |

| System Tables | references/5-system-tables.md | Lineage, audit, billing, compute, jobs, query history |

| Volumes | references/6-volumes.md | Volume file operations, permissions, best practices |

| Data Profiling | references/7-data-profiling.md | Data profiling, drift detection, profile metrics |

Quick Start

Create Unity Catalog Objects (CLI)

Use --json for create commands. Positional argument order differs per command and

has changed across CLI versions, so --json is the order-independent, version-stable form

shown throughout this skill.

# Create a catalog
databricks catalogs create --json '{"name": "my_catalog"}'

# Create a schema
databricks schemas create --json '{"name": "my_schema", "catalog_name": "my_catalog"}'

# Create a managed volume
databricks volumes create --json '{
  "catalog_name": "my_catalog",
  "schema_name": "my_schema",
  "name": "my_volume",
  "volume_type": "MANAGED"
}'

# List catalogs, schemas, volumes (read commands take simple positional args)
databricks catalogs list
databricks schemas list my_catalog
databricks volumes list my_catalog.my_schema

Positional create args still work if you prefer them, but the order is not uniform

across commands — this is the per-command order (and the reason --json is recommended):

| Command | Positional create order |

|---------|---------------------------|

| databricks catalogs create | NAME |

| databricks schemas create | NAME CATALOG_NAME |

| databricks volumes create | CATALOG_NAME SCHEMA_NAME NAME VOLUME_TYPE |

> CLI surface varies by version. If a databricks subcommand or positional signature is

> missing in your install, prefer --json, the SQL form, or the Python SDK rather than

> guessing flags.

Volume File Operations (CLI)

databricks fs requires the dbfs: scheme prefix even for UC Volume paths — without it the CLI treats the path as local filesystem and errors with no such directory.

# List files in a volume
databricks fs ls dbfs:/Volumes/catalog/schema/volume/path/

# Upload a directory's contents to a volume (-r copies contents, not the directory itself)
databricks fs cp -r --overwrite /tmp/data dbfs:/Volumes/catalog/schema/volume/dest

# Download a file from a volume
databricks fs cp dbfs:/Volumes/catalog/schema/volume/file.csv /tmp/file.csv

# Create a directory in a volume
databricks fs mkdirs dbfs:/Volumes/catalog/schema/volume/new_folder

Grant & Revoke Access

GRANT/REVOKE is the core governance operation. See references/1-access-control.md for the full privilege model.

-- Grant read access on a schema to a group
GRANT USE CATALOG ON CATALOG analytics TO `data_readers`;
GRANT USE SCHEMA ON SCHEMA analytics.gold TO `data_readers`;
GRANT SELECT ON SCHEMA analytics.gold TO `data_readers`;

-- Who can access this table?
SHOW GRANTS ON TABLE analytics.gold.customers;

-- Revoke
REVOKE SELECT ON SCHEMA analytics.gold FROM `data_readers`;

Enable System Tables Access

-- Grant access to system tables
GRANT USE CATALOG ON CATALOG system TO `data_engineers`;
GRANT USE SCHEMA ON SCHEMA system.access TO `data_engineers`;
GRANT SELECT ON SCHEMA system.access TO `data_engineers`;

Common Queries

-- Table lineage: What tables feed into this table?
SELECT source_table_full_name, source_column_name
FROM system.access.table_lineage
WHERE target_table_full_name = 'catalog.schema.table'
  AND event_date >= current_date() - 7;

-- Audit: Recent permission changes
SELECT event_time, user_identity.email, action_name, request_params
FROM system.access.audit
WHERE action_name LIKE '%GRANT%' OR action_name LIKE '%REVOKE%'
ORDER BY event_time DESC
LIMIT 100;

-- Billing: DBU usage by workspace
SELECT workspace_id, sku_name, SUM(usage_quantity) AS total_dbus
FROM system.billing.usage
WHERE usage_date >= current_date() - 30
GROUP BY workspace_id, sku_name;

Running SQL from the CLI

> databricks experimental aitools tools query is an experimental command. The

> experimental namespace is not guaranteed to be stable across CLI versions and may be

> absent in your install. Prefer running system-table SQL from a SQL warehouse (SQL

> editor, scheduled query) or the Python SDK (w.statement_execution.execute_statement),

> or a notebook. Use the experimental CLI only for quick ad-hoc checks.

> Getting the IDs these examples use. WAREHOUSE_ID — run databricks warehouses list

> (or copy it from a SQL warehouse's *Connection details* in the UI). METASTORE_ID (used in

> references/5-system-tables.md) — w.metastores.current().metastore_id

> via the SDK, or the Catalog UI → metastore details.

Experimental CLI form (convenience only):

databricks experimental aitools tools query --warehouse WAREHOUSE_ID "
  SELECT source_table_full_name, target_table_full_name
  FROM system.access.table_lineage
  WHERE event_date >= current_date() - 7
"

Stable SDK fallback (works on any CLI version):

from databricks.sdk import WorkspaceClient

w = WorkspaceClient()
resp = w.statement_execution.execute_statement(
    warehouse_id="WAREHOUSE_ID",
    statement="""
        SELECT source_table_full_name, target_table_full_name
        FROM system.access.table_lineage
        WHERE event_date >= current_date() - 7
        LIMIT 100
    """,
)
for row in resp.result.data_array or []:
    print(row)

> CLI surface varies by version. If a databricks subcommand (e.g. an experimental

> tool, system-schemas, or external-lineage) is missing, fall back to the SQL warehouse

> or the Python SDK shown above rather than guessing flags.

Best Practices

• Grant minimal access - Apply least privilege; grant at the narrowest securable that works
• Filter by date - System tables can be large; always use date filters
• Use appropriate retention - Check your workspace's retention settings
• Schedule reports - Create scheduled queries for regular monitoring
• Prefer SQL/SDK over experimental CLI - For anything beyond quick checks

Related Skills

This skill owns Unity Catalog governance: access control, the privilege model,

external locations / storage credentials, securable DDL, fine-grained access, system

tables, and volumes. For adjacent concerns, use the sibling skill instead:

• databricks-core (declared parent) — auth, profile selection, generic CLI, and catalog/table *exploration*
• databricks-metric-views — metric view definitions / DDL (WITH METRICS LANGUAGE YAML)
• databricks-iceberg — Managed Iceberg, External Iceberg Reads (fka Uniform), and Iceberg REST Catalog (IRC) credential *vending* for external engines — distinct from UC storage credentials (see references/2-external-locations.md)
• databricks-ml-training — UC model registration and @prod/@challenger aliases
• databricks-vector-search — Vector Search indexes
• databricks-pipelines, databricks-jobs, databricks-lakeflow-connect — *producing* tables via pipelines/jobs/managed ingestion
• databricks-lakebase — Lakebase / synced tables (OLTP)
• databricks-ai-functions — AI functions such as ai_mask / ai_classify (AI *transforms*, not access control — see references/4-fine-grained-access.md)
• databricks-aibi-dashboards — AI/BI dashboards on UC data
• databricks-synthetic-data-gen — generating data stored in UC volumes

Roadmap (not yet covered — deferred to a later version)

These governance areas are intentionally out of scope for v0.3.0 and planned for later:

• Delta Sharing / Marketplace / Clean Rooms
• Lakehouse Federation (connections + foreign catalogs)
• ABAC / governed tags as policy

Resources

• Unity Catalog Privileges & Securable Objects
• Unity Catalog System Tables
• Audit Log Reference
• Manage External Locations and Storage Credentials