Mlv Operations CLI

> Update Check — ONCE PER SESSION (mandatory)

> The first time this skill is used in a session, run the check-updates skill before proceeding.

> - GitHub Copilot CLI / VS Code: invoke the check-updates skill.

> - Claude Code / Cowork / Cursor / Windsurf / Codex: compare local vs remote package.json version.

> - Skip if the check was already performed earlier in this session.

> CRITICAL NOTES

> 1. To find the workspace details (including its ID) from workspace name: list all workspaces and, then, use JMESPath filtering

> 2. To find the lakehouse details (including its ID) from workspace ID and lakehouse name: list all lakehouses in that workspace and, then, use JMESPath filtering

> 3. MLV Discovery Gap: REST APIs for listing MLVs in a lakehouse do not exist yet (GET /materializedLakeViews returns 404). For schedule CRUD/trigger/status, only workspace ID and lakehouse ID are needed (scheduling operates on the full lakehouse lineage). MLV table names are only needed if the user asks about specific view definitions.

MLV Operations — CLI Skill

Terminology Mapping

Fabric has three materialized view concepts. Disambiguate by context:

| User context | User says | Actually means | Route to |

|-------------|-----------|----------------|----------|

| Spark / Lakehouse | "materialized view" | Materialized Lake View (MLV) | This skill (mlv-operations-cli) |

| Spark / Lakehouse | "materialized lake view" | MLV | This skill |

| Spark / Lakehouse | "spark materialized view" | MLV | This skill |

| Spark / Lakehouse | "MV" or "MLV" | MLV | This skill |

| Spark / Lakehouse | "CREATE MATERIALIZED LAKE VIEW" | MLV DDL (authoring) | spark-authoring-cli |

| Spark / Lakehouse | "schedule my materialized view" | MLV scheduling | This skill |

| Spark / Lakehouse | "refresh my views" | MLV on-demand refresh | This skill |

| KQL / Eventhouse | "materialized view" | KQL Materialized View | eventhouse-authoring-cli |

| SQL DW / Warehouse | "materialized view" | Not supported in Fabric | Explain unsupported |

Disambiguation rule: If the user mentions lakehouse, notebook, Spark, Delta, or MLV → it's a Materialized Lake View (this skill). If they mention KQL, Eventhouse, or Kusto → it's a KQL Materialized View (different skill). If they mention Warehouse or SQL DW → explain it's not supported.

Default: If context is unclear (no mention of lakehouse, Spark, KQL, or Warehouse), ask the user: "Are you working with a Lakehouse (Materialized Lake View) or an Eventhouse (KQL Materialized View)?" before proceeding.

Manage MLV refresh scheduling and monitoring using Fabric REST APIs. This skill provides full scheduling API coverage (Preview) for scheduling and monitoring operations, enabling full automation of MLV refresh workflows.

What This Skill Can Do

✅ Fully Supported (9 REST APIs)

• Schedule Management (per lakehouse — refreshes entire MLV lineage)
• Create refresh schedules (Cron interval, Daily, Weekly, Monthly)
• List schedules for a lakehouse
• Get schedule details by ID
• Update existing schedules (change frequency, enabled state)
• Delete schedules

• Job Execution
• Trigger on-demand refresh (immediate execution)
• List job run history with filtering
• Get job status and progress
• Cancel running jobs

• Safety & UX
• Human-in-the-loop confirmations before creating schedules or triggering refreshes
• Step-by-step planning for complex multi-MLV operations
• Iterative error handling with helpful suggestions
• Preview schedule impact before execution

❌ Not Supported (Requires UI — No REST APIs)

• MLV Discovery: Cannot list MLVs in a lakehouse (API returns 404)
• Lineage Inspection: Cannot fetch dependency graphs (API returns 404)
• Data Quality Metrics: Cannot retrieve DQ metrics (API returns 404)
• Schema Verification: Cannot check if schemas are enabled (property missing)

Workaround: User provides lakehouse ID and MLV table names explicitly. Use Fabric Lakehouse UI for discovery tasks.

Table of Contents

| Task | Reference | Notes |

|---|---|---|

| RULES — Read these first, follow them always | SKILL.md § Must | MUST read — 6 rules for this skill |

| Finding Workspaces and Items in Fabric | COMMON-CLI.md § Finding Workspaces and Items in Fabric | Mandatory — *READ link first* [needed for finding workspace id by its name or item id by its name, item type, and workspace id] |

| Fabric Topology & Key Concepts | COMMON-CORE.md § Fabric Topology & Key Concepts ||

| Environment URLs | COMMON-CORE.md § Environment URLs ||

| Authentication & Token Acquisition | COMMON-CORE.md § Authentication & Token Acquisition | Wrong audience = 401; read before any auth issue |

| Job Scheduling | COMMON-CLI.md § Job Scheduling | URL is /jobs/{jobType}/schedules; endDateTime required |

| Job Execution | COMMON-CLI.md § Job Execution ||

| Fabric Control-Plane API via az rest | COMMON-CLI.md § Fabric Control-Plane API via az rest | Always pass --resource https://api.fabric.microsoft.com or az rest fails |

| Pagination Pattern | COMMON-CLI.md § Pagination Pattern ||

| Gotchas & Troubleshooting (CLI-Specific) | COMMON-CLI.md § Gotchas & Troubleshooting (CLI-Specific) | az rest audience, shell escaping, token expiry |

| Quick Reference: az rest Template | COMMON-CLI.md § Quick Reference: az rest Template ||

| MLV Refresh API Reference | MS Learn: Refresh Materialized Lake View | Official documentation for refresh mechanics |

> Eval mode: When the user prompt contains "I pre-approve" or "proceed without confirmation",

> skip the human-in-the-loop gate and execute directly. This allows automated testing.

Must

1. Human-in-the-loop confirmations

Before executing any operation, show a preview and ask for confirmation:

• Creating schedules: Show schedule type, interval/times, lakehouse ID, start/end time
• Triggering refreshes: Show lakehouse ID, confirm "this refreshes the entire MLV lineage"
• Deleting schedules: Show schedule ID and confirm deletion

Confirmation options:

• Allow — Execute this single operation
• Decline — Skip this operation
• Allow in this thread — Auto-allow all operations in current conversation

Example:

I'm about to create a refresh schedule:

  Lakehouse ID: abc-123-def
  Type: Daily
  Time: 02:00 UTC
  Start: 2026-06-20
  End: 2027-06-20
  Scope: Entire MLV lineage

Proceed? [Allow / Decline / Allow in this thread]

2. Use REST APIs exactly as documented

Base URL: https://api.fabric.microsoft.com/v1

IMPORTANT: All endpoints are workspace + lakehouse scoped. A schedule refreshes the entire MLV lineage — you cannot schedule individual tables.

Schedule endpoints:

• POST /workspaces/{workspaceId}/lakehouses/{lakehouseId}/jobs/refreshMaterializedLakeViews/schedules — Create schedule
• GET /workspaces/{workspaceId}/lakehouses/{lakehouseId}/jobs/refreshMaterializedLakeViews/schedules — List schedules
• GET /workspaces/{workspaceId}/lakehouses/{lakehouseId}/jobs/refreshMaterializedLakeViews/schedules/{id} — Get schedule
• PATCH /workspaces/{workspaceId}/lakehouses/{lakehouseId}/jobs/refreshMaterializedLakeViews/schedules/{id} — Update schedule
• DELETE /workspaces/{workspaceId}/lakehouses/{lakehouseId}/jobs/refreshMaterializedLakeViews/schedules/{id} — Delete schedule

Job instance endpoints:

• POST /workspaces/{workspaceId}/lakehouses/{lakehouseId}/jobs/refreshMaterializedLakeViews/instances — Trigger on-demand refresh (no body; returns 202 + Location header with job ID)
• GET /workspaces/{workspaceId}/lakehouses/{lakehouseId}/jobs/refreshMaterializedLakeViews/instances — List job history
• GET /workspaces/{workspaceId}/items/{lakehouseId}/jobs/instances/{jobInstanceId} — Get job status
• POST /workspaces/{workspaceId}/items/{lakehouseId}/jobs/instances/{jobInstanceId}/cancel — Cancel running job

See: MS Learn: MLV Background Jobs

3. Authentication

All scheduling operations (create/update/delete, trigger, status, cancel) support both User identity (az login) and Service Principal / Managed Identity. Requires Workspace Contributor or Admin role.

4. One schedule per lineage

The API supports one active refresh schedule per lakehouse lineage. If the user asks for per-table scheduling, explain this limitation.

5. MLV Discovery — User must provide names

GET /materializedLakeViews returns 404. Ask user for lakehouse ID and table names upfront.

6. Run History diagnostic workflow

When a user asks "why did my refresh fail?" or "show me run history", follow this sequence:

• List recent runs: GET /instances — returns job instances with status, start/end times
• Show run summary: Display table with run ID, status, start/end time, duration
• Select failed run: If multiple, ask user which one to investigate
• Read error code: Extract failureReason.errorCode and failureReason.message from the failed instance
• Suggest next steps: Based on error code:
• MLV_SPARK_SESSION_REQUEST_SUBMISSION_FAILED → Check capacity availability, Spark pool config
• MLV_SELECTED_NOT_FOUND → MLV table was deleted or renamed, verify it exists
• Other Spark errors → Route to spark-operations-cli for OOM, skew, shuffle spill diagnosis
• Per-view details: The API returns lineage-level status only. Per-view status (which individual MLVs failed) is available in the UI Recent runs page — direct the user there for view-level breakdown

Run statuses (from API): NotStarted, InProgress, Completed, Failed, Cancelled, Deduped

> Note: Run history retention may be limited. If older runs are missing, check the Recent runs page in the Lakehouse UI.

Prefer

• Daily/Weekly types for precise time-of-day scheduling (e.g., "2 AM daily")
• Cron type with interval only for sub-daily frequencies (e.g., "every 60 minutes")
• Step-by-step planning — clarify intent, propose schedule, show preview, execute on approval
• Iterative error handling — on failure, explain what went wrong and suggest actionable fixes
• Explicit timezone in every schedule (localTimeZoneId)
• Cross-lakehouse scheduling from extended lineage — when MLVs span multiple lakehouses, schedule from the downstream lakehouse's lineage view. Extended lineage refreshes upstream dependencies automatically in dependency order. Prefer this over creating separate schedules on each lakehouse individually.

Avoid

• Per-table scheduling claims — the API refreshes the entire lineage
• Cron string expressions (e.g., 0 2 * * *) — the API uses structured types, not cron strings
• Assuming JSON response from on-demand refresh — returns 202 with job ID in Location header only
• Silent failures — always explain errors
• Scheduling from notebooks — route users here; SQL REFRESH ... FULL is for one-time manual use only

Schedule Payload Structure

Create Schedule (POST /schedules)

Endpoint: POST /workspaces/{workspaceId}/lakehouses/{lakehouseId}/jobs/refreshMaterializedLakeViews/schedules

{
  "enabled": true,
  "configuration": {
    "type": "Cron",
    "interval": 60,
    "startDateTime": "2026-06-20T00:00:00",
    "endDateTime": "2027-06-20T23:59:59",
    "localTimeZoneId": "UTC"
  }
}

Key fields:

• enabled: true to enable schedule on creation
• type: One of "Cron", "Daily", "Weekly", "Monthly"
• interval: (Cron only) Refresh interval in minutes (e.g., 60 = hourly, 120 = every 2 hours)
• times: (Daily/Weekly/Monthly) Array of times in "HH:MM" format, e.g., ["02:00"]
• weekdays: (Weekly only) e.g., ["Monday", "Wednesday", "Friday"] — PascalCase day names
• recurrence: (Monthly only) Recurrence interval, e.g., 1 (every month)
• occurrence: (Monthly only) e.g., {"occurrenceType": "DayOfMonth", "dayOfMonth": 1}
• localTimeZoneId: Windows time zone names — "UTC", "Central Standard Time", "India Standard Time", etc.
• startDateTime: When schedule becomes active (ISO 8601 format, no Z suffix)
• endDateTime: REQUIRED — When schedule expires

Daily example (preferred for "2 AM every day"):

{ "enabled": true, "configuration": { "type": "Daily", "times": ["02:00"], "startDateTime": "2026-06-20T00:00:00", "endDateTime": "2027-06-20T23:59:59", "localTimeZoneId": "UTC" } }

Weekly example (weekdays at 6 AM):

{ "enabled": true, "configuration": { "type": "Weekly", "times": ["06:00"], "weekdays": ["Monday", "Friday"], "startDateTime": "2026-06-20T00:00:00", "endDateTime": "2027-06-20T23:59:59", "localTimeZoneId": "UTC" } }

Monthly example (1st of each month at midnight):

{ "enabled": true, "configuration": { "type": "Monthly", "recurrence": 1, "occurrence": {"occurrenceType": "DayOfMonth", "dayOfMonth": 1}, "times": ["00:00"], "startDateTime": "2026-06-20T00:00:00", "endDateTime": "2027-06-20T23:59:59", "localTimeZoneId": "UTC" } }

> WARNING: Do NOT use "days": [1, 15] for Monthly — this returns 400 InvalidConfiguration. Use recurrence + occurrence as shown above.

Update Schedule (PATCH /schedules/{id})

Endpoint: PATCH /workspaces/{workspaceId}/lakehouses/{lakehouseId}/jobs/refreshMaterializedLakeViews/schedules/{id}

{
  "enabled": true,
  "configuration": {
    "type": "Cron",
    "interval": 120,
    "startDateTime": "2026-06-20T00:00:00",
    "endDateTime": "2027-06-20T23:59:59",
    "localTimeZoneId": "UTC"
  }
}

Note: The update API requires both enabled and a complete configuration (full replacement, not partial patch). Always send all fields.

Trigger On-Demand Refresh (POST /instances)

Endpoint: POST /workspaces/{workspaceId}/lakehouses/{lakehouseId}/jobs/refreshMaterializedLakeViews/instances

Request body: None (empty POST). Refreshes the entire MLV lineage in dependency order.

Response: 202 Accepted — job instance ID is in the Location response header:

Location: https://api.fabric.microsoft.com/v1/workspaces/{wsId}/items/{lhId}/jobs/instances/{jobInstanceId}
Retry-After: 60

Poll for status using the URL from the Location header:

GET /workspaces/{workspaceId}/items/{lakehouseId}/jobs/instances/{jobInstanceId}

Job instance status values:

| Status | Meaning |

|--------|---------|

| NotStarted | Job is queued but hasn't begun |

| InProgress | Job is actively running |

| Completed | Job finished successfully |

| Failed | Job failed (check failureReason) |

| Cancelled | Job was cancelled by user |

| Deduped | Skipped because another refresh was already in progress |

Note: Job instances returned by GET /items/{id}/jobs/instances use jobType: "MaterializedLakeViews" (live-tested) or jobType: "RefreshMaterializedLakeViews" (per MS Learn docs). Filter on either value when listing instances.

Schedule settings (additional options via UI or API):

• Optimal Refresh (default: On) — Fabric picks incremental or full refresh per MLV automatically
• Extended lineage — refreshes chains across multiple lakehouses in dependency order from a single schedule

Example Workflows

Workflow 1: Schedule Nightly Refresh

User prompt: "Schedule sales_monthly MLV in CustomerVoice lakehouse to refresh at 2 AM daily"

Agent steps:

• Find workspace ID for "CustomerVoice" via GET /workspaces + JMESPath
• Find lakehouse ID via GET /workspaces/{id}/lakehouses + JMESPath
• Clarify scope: "Note: the schedule refreshes the entire MLV lineage in this lakehouse, not just sales_monthly."
• Show preview:

   Creating schedule:
     Lakehouse: CustomerVoice (ID: xyz-456-ghi)
     Scope: Entire MLV lineage (all materialized lake views)
     Type: Daily, Time: 02:00 UTC
     Start: Now
     End: 1 year from now
   
   Proceed? [Allow / Decline]

• On "Allow": POST schedule payload, return schedule ID
• Report: "✅ Schedule created (ID: sched-789). Entire lineage will refresh daily at 2 AM UTC."

Workflow 2: Trigger Immediate Refresh

User prompt: "Refresh inventory_daily MLV now in lakehouse xyz-456-ghi"

Agent steps:

• Confirm lakehouse ID is valid (GET /lakehouses/{id})
• Clarify scope and show preview:

   Triggering on-demand refresh:
     Lakehouse ID: xyz-456-ghi
     Scope: Entire MLV lineage (includes inventory_daily and all other MLVs)
   
   This refreshes ALL MLVs in dependency order. Proceed? [Allow / Decline]

• On "Allow": POST /instances (no body), capture job ID from Location header
• Poll GET /items/{id}/jobs/instances/{jobId}, show progress
• Report: "✅ Refresh completed in 2m 34s" or "❌ Refresh failed: [error message]"

Workflow 3: User Asks for Per-MLV Scheduling (Explain Limitation)

User prompt: "Schedule bronze_sales, silver_sales, gold_sales to refresh at 2 AM, 3 AM, 4 AM respectively"

Agent steps:

• Explain limitation:

   I can't schedule individual MLVs on different times — the API supports
   one schedule per lakehouse lineage, and it refreshes ALL MLVs together
   in dependency order.
   
   Options:
     1. Single Daily schedule at 2 AM (refreshes all 3 in order)
     2. If they're in separate lakehouses, I can create one schedule per lakehouse
   
   Which would you prefer?

• If user picks option 1: Create one Daily schedule with "times": ["02:00"]
• If user picks option 2: Confirm lakehouse IDs for each, create schedules

Workflow 4: List and Delete Schedules

User prompt: "Show my schedule and delete it if it's outdated"

Agent steps:

• Ask user for workspace and lakehouse (cannot list schedules globally)
• GET /workspaces/{wsId}/lakehouses/{lhId}/jobs/refreshMaterializedLakeViews/schedules
• Parse response (usually 1 schedule per lineage), show details:

   Active schedule on lakehouse 'CustomerVoice':
     - ID: sched-111 (created 2025-10-15, type: Daily, time: 02:00)
   
   Delete this schedule? [Allow / Decline]

• On "Allow": DELETE /schedules/{id}, confirm
• Report: "✅ Schedule deleted"

Common Errors & Fixes

Error: 400 Bad Request — Invalid Interval

Message: "The interval '0' is invalid. Interval must be between 1 and 5270400 (10 years in minutes)."

Fix:

Interval must be a valid number of minutes between 1 and 5,270,400 (10 years).

Common intervals:
  60 = hourly
  1440 = daily (24 hours)
  10080 = weekly (7 days)

Would you like me to adjust the interval to a valid value?

Error: 409 Conflict — Schedule Already Exists

Message: "A schedule already exists for this lakehouse"

Fix:

A schedule is already active for this lakehouse. Options:
  1. Update existing schedule (change interval/time)
  2. Delete and recreate (replaces schedule)
  3. Leave as-is (no change)

Which would you prefer?

Error: 404 Not Found — Lakehouse ID Invalid

Message: "Lakehouse 'wrong-id-123' not found in workspace 'abc-456'"

Fix:

The lakehouse ID you provided doesn't exist. Let me list available lakehouses:

[Call GET /workspaces/{id}/lakehouses, show table]

Which lakehouse should I use?

Error: 403 Forbidden — Permission Denied

Message: "User does not have permission to create schedules in this workspace"

Fix:

You need Workspace Contributor or Admin role to create schedules.

Current permissions: Viewer (read-only)
Required: Contributor or Admin

Contact your workspace admin to request elevated permissions.

Tips for Users

1. Find Your Lakehouse ID

Option A: Via REST API

az rest --resource https://api.fabric.microsoft.com \
  --url "https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/lakehouses" \
  --method GET

Extract id from the response for your lakehouse.

Option B: Via Fabric UI

• Open lakehouse in Fabric portal
• Click Settings (gear icon)
• Copy "Lakehouse ID" from properties

2. Common Schedule Configurations

| Need | Type | Key field |

|------|------|-----------|

| Every hour | Cron | "interval": 60 |

| Daily at 2 AM | Daily | "times": ["02:00"] |

| Weekdays at 6 AM | Weekly | "times": ["06:00"], "weekdays": ["Monday","Friday"] |

| 1st of each month | Monthly | "recurrence": 1, "occurrence": {"occurrenceType": "DayOfMonth", "dayOfMonth": 1} |

3. Monitor Job History

List recent refresh jobs (authenticate per COMMON-CLI.md § Quick Reference: az rest Template):

# See COMMON-CLI.md for authentication setup
az rest --resource https://api.fabric.microsoft.com \
  --url "https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/lakehouses/{lakehouseId}/jobs/refreshMaterializedLakeViews/instances" \
  --method GET

> Note: The list instances API does not support OData query parameters ($top, $orderby, $filter). Sort and filter results client-side after retrieval. Use continuationToken for pagination.

4. Time Zone Considerations

Default: Schedules use UTC unless specified.

Best practice: Always specify timezone explicitly to avoid confusion:

{
  "configuration": {
    "localTimeZoneId": "Central Standard Time"
  }
}

Valid time zones: Windows time zone names (e.g., "Central Standard Time", "Pacific Standard Time", "India Standard Time"). Use the Windows Default Time Zones registry.

Related Skills

• spark-authoring-cli: Create MLVs in Fabric Notebooks (authoring side)
• check-updates: Verify skill package is up-to-date (run once per session)

Limitations & Future Roadmap

Current Limitations (as of 2026-06-18)

| Feature | Status | Workaround |

|---------|--------|------------|

| List MLVs in lakehouse | ❌ API returns 404 | User provides table names manually |

| Get MLV lineage graph | ❌ API returns 404 | Use Fabric Lakehouse UI |

| Check data quality metrics | ❌ API returns 404 | Use Fabric Lakehouse UI |

| Verify schema support | ❌ Property missing | Assume schemas enabled if MLVs work |

What Works Today (full scheduling API coverage (Preview))

• ✅ Create/list/update/delete schedules (5 APIs)
• ✅ Trigger/monitor/cancel refresh jobs (4 APIs)
• ✅ Full automation of refresh workflows
• ✅ Human-in-the-loop safety confirmations
• ✅ Iterative error handling

Planned (When REST APIs Ship)

• MLV Discovery: Auto-list MLVs in a lakehouse
• Lineage Tracing: Show dependency graphs
• Data Quality: Fetch DQ metrics programmatically
• Schema Verification: Check enableSchemas property

Agent design is forward-compatible: When APIs become available, add discovery capabilities without changing scheduling logic.

Conclusion

This skill provides validated automation for MLV refresh scheduling and monitoring using 100% REST API coverage. While MLV discovery requires UI workarounds today, scheduling and job execution work as documented.

Design philosophy (inspired by Databricks Data Engineering Agent):

• Human-in-the-loop confirmations for safety
• Step-by-step planning for complex tasks
• Iterative error handling with helpful suggestions
• Transparent about limitations (no speculative workarounds)

Next steps: Use this skill to automate MLV refresh workflows. When discovery APIs ship, we'll extend the skill to eliminate manual lakehouse ID + table name input.