DevOps & Observability
Debugging Microsoft Fabric Pipelines and Notebooks in Production
Debug Microsoft Fabric pipelines and notebooks systematically using run history, execution logs, correlation IDs, parameters, data validation, capacity metrics and safe rerun patterns.
On this page
Most Fabric incidents start the same way: a red pipeline in the Monitoring hub, a message from someone whose report shows yesterday’s numbers, or a load that normally takes ten minutes still running after forty. The temptation is to open the notebook, find something that looks wrong, change it and rerun. Sometimes that works. Often it fixes the wrong thing, reruns a window that was not safe to rerun, or destroys the evidence of what actually happened.
This guide is a systematic way to debug Fabric pipelines and notebooks when they fail, become slow, or produce the wrong result while reporting success. It builds on two companion articles: Monitoring Microsoft Fabric Pipelines and Notebooks, which covers the signals that tell you something is wrong, and Designing an Execution Log, which covers the record that tells you what each run was doing. Debugging is what you do with both once an alert fires.
Fabric behavior referenced here was checked against Microsoft Learn in October 2026.
The debugging sequence
Every incident in this article follows the same ten steps. Skipping one is the most common reason an incident comes back.
- Identify the failed or abnormal run. The specific pipeline run, notebook run and business window, not “last night’s load”.
- Establish the scope. One workload or several? One window or every run since a date? One environment or all?
- Locate the failing layer. Orchestration, source, transformation, write, data, permissions, capacity.
- Inspect the evidence. Run history, activity input and output, the execution log, validation results, the capacity window.
- Form a hypothesis. One specific, testable explanation.
- Test the hypothesis with the smallest possible experiment, outside production if you can.
- Fix safely. One change at a time, through your normal deployment path where possible.
- Rerun safely. Only when you know the rerun cannot duplicate or skip data.
- Verify downstream state. Validation passes, watermark correct, Gold and semantic models fresh.
- Prevent recurrence. A validation check, an alert, a parameter guard or a design change, plus a short note on the cause.
The fastest way through this list is good evidence. The rest of the article is about where that evidence lives for each kind of failure.
First question: what kind of failure is this?
| Failure type | Typical symptom | First place to look | Evidence source |
|---|---|---|---|
| Orchestration | Activity skipped, wrong branch taken, child pipeline never started | Pipeline run detail view: activity statuses and dependency conditions | Monitoring hub, activity inputs |
| Notebook code | Notebook activity failed with a Python or Spark exception | Notebook run output and error | Spark application details, execution log error_code / error_stage |
| Data | Run failed on a validation check, or succeeded with odd counts | Validation results, row counts | Execution log, validation tables |
| Schema | Missing column, type mismatch, write rejected | Error message, source vs target schema | Spark error, DESCRIBE / printSchema output |
| Authentication / permission | Access denied, forbidden, login failure | Failed activity error and the identity it ran as | Activity error, connection settings |
| Source connectivity | Timeout, connection refused, gateway unreachable | Copy or Lookup activity error | Activity output, gateway status |
| Capacity / performance | Slow run, Spark job could not start, throttling errors | Duration against baseline, then the capacity window | Execution log, Capacity Metrics app |
| Downstream write / commit | Failure at the end of a long run; concurrent update errors | Last stage of the notebook, write error | Spark error, Delta table history |
| “Successful” but wrong | Green pipeline, wrong numbers, stale report | Validation results and row counts for the window | Execution log, validation, source totals |
Classifying first saves time. A permission error and a capacity rejection can both surface as “the activity failed”, but you investigate them in completely different places.
Start from monitoring, not from the code
Debugging starts from evidence. Before opening a notebook, collect:
- The run: pipeline run ID, status, duration, which activity failed. The Monitoring hub’s run detail view shows the pipeline diagram, every activity run, and each activity’s input, output and error.
- The run tree: the Monitoring hub can show upstream and downstream runs, so you can see which parent triggered this run and which children it started.
- The execution log: business date, window, row counts, retries, watermark state, error stage.
- Validation results for the window.
- The capacity window for the same period, if anything looks slow or Spark jobs failed to start.
The monitoring guide covers where each of these comes from. The point here is order: you want all of it in front of you before forming a hypothesis.
Use execution correlation
When a notebook fails inside a child pipeline, the top-level pipeline only reports that the invoked pipeline failed. The execution log turns that into one query, because every execution carries a correlation_id (the business operation), its own execution_id, and a parent_execution_id pointing to its caller:
SELECT item_type, pipeline_name, notebook_name, status, start_time, duration_ms,
window_start, window_end, rows_read, rows_inserted, retry_count,
rerun_of_execution_id, error_stage, error_message, pipeline_run_id
FROM dbo.execution_log
WHERE correlation_id = 'sales-daily:2026-10-24'
ORDER BY start_time;A typical result for a failed night reads like a story:
pl_daily_salesstarted at 05:00,failed.pl_load_orders(parent =pl_daily_sales)failed.- The Copy job activity
succeeded, 1,240 rows read. nb_silver_orders(parent =pl_load_orders)failedaterror_stage = merge_silver,retry_count = 1.- No validation or publish records, because they never started.
You now know the layer (the Silver MERGE), the window, that one automatic retry already happened, and that nothing downstream ran. The pipeline_run_id takes you straight to the right run in the Monitoring hub for the activity-level detail.
Pipeline failures
In the failed run’s detail view, work through the activity that failed:
- Error message and code. Read the whole message. Copy and Lookup errors usually name the source or sink side.
- Input. The activity’s input shows the parameters and expressions it actually received. Wrong dates, empty paths and the wrong environment show up here.
- Output. For a Notebook activity, the output links to the notebook run; for a Copy activity, it shows rows read and written and a duration breakdown.
- Child pipelines. For an Invoke pipeline activity, follow it into the child run; the real failure is usually two levels down. Invoke Pipeline covers how parent and child runs relate.
- Retries. Activities with retries enabled can fail several times before the run fails. Several attempts with the same error suggest a persistent cause; different errors suggest something unstable.
- Timeouts. An activity that hits its timeout (12 hours by default) fails as a timeout, not as an error in your logic. Treat it as a performance incident.
- Dependency conditions. An activity that never ran was usually skipped by a dependency condition: on success, on failure, on completion, on skip. Check the path you expected to execute.
Then decide which layer failed:
| Layer | Clues |
|---|---|
| Orchestration | Wrong branch, skipped activity, bad expression, missing parameter at pipeline level |
| Source | Copy or Lookup error naming the source, timeouts, gateway or credential errors |
| Transformation | Notebook or Dataflow activity failed; the error is inside the code |
| Destination | Sink errors, write conflicts, schema rejected by the target |
| External dependency | Web activity or API errors, upstream system unavailable |
Notebook failures
Notebook failures are where most debugging time goes. Common classes:
- Python or Spark exceptions in your logic.
- Missing column or schema mismatch. Spark reports an unresolved column, often with suggested alternatives (
UNRESOLVED_COLUMN.WITH_SUGGESTION). - Null or type issues: casts that fail, nulls in a column that must not have them, decimal overflow.
- Path or table not found: the wrong default Lakehouse, a wrong path parameter, a table that has not been created yet in this environment.
- Library or dependency issues: a package missing from the attached environment, or a version that changed.
- Session or runtime issues: a session that could not start, a runtime upgrade that changed behavior.
- Write failures: schema rejected on write, or a concurrent write to the same Delta table conflicting at commit.
A practical pattern, run in a development copy of the notebook against the same window:
- Isolate the failing operation. The error stage in the execution log, or the failing cell in the run output, tells you where to start.
- Validate the parameters the run actually received (next section).
- Inspect the input schema at the point of failure:
df.printSchema(). - Inspect row counts at each step before the failure. A join that suddenly multiplies rows or a filter that drops everything is visible immediately.
- Sample the bad records. Filter for the condition that breaks: nulls in a key, values that fail a cast, keys that appear twice.
- Rerun the smallest unit of work: the failing step on the failing window, not the whole pipeline.
from pyspark.sql import functions as F
# Rows that will break a decimal cast or a required key
bad = (df.where(F.col("sales_order_id").isNull() |
F.col("grand_total").cast("decimal(18,2)").isNull() & F.col("grand_total").isNotNull()))
print(bad.count())
bad.show(20, truncate=False)Run the investigation on a development workspace or a copy of the data where you can, so experiments do not write to production tables.
Parameter debugging
A surprising share of “the notebook is broken” incidents are really “the notebook was given the wrong inputs”. Typical causes:
- A missing parameter, so the notebook silently used its interactive default.
- The wrong type: a date passed as a string in an unexpected format.
- An empty value from an expression that evaluated to null.
- The wrong business date, often computed from the current time instead of the trigger time, so a rerun processes today instead of the failed day.
- A wrong path or table name.
- The wrong environment or workspace identifier, so a production run reads development data or the reverse.
Validate parameters in the first code cell, and fail loudly before doing any work:
from datetime import date
def require(name, value):
if value in (None, "", "None"):
raise ValueError(f"Parameter '{name}' is missing or empty")
return value
business_date = date.fromisoformat(require("business_date", business_date))
environment = require("environment", environment)
if environment not in {"dev", "test", "prod"}:
raise ValueError(f"Unexpected environment '{environment}'")
if business_date > date.today():
raise ValueError(f"business_date {business_date} is in the future")Compare what the notebook should have received with the Notebook activity’s input in the run detail view. Notebook parameters covers the full pattern, including logging an allow-list of parameter values.
Schema debugging
Schema problems come from both ends: the source changes, or the target does not accept what you write. Typical causes:
- an unexpected new column in the source or in a file
- a missing column that the code references
- a type change, for example an integer that became a string
- a nullability change, where a column that was always populated now has nulls
- gradual source drift across files or partitions
- a destination mismatch, where a write is rejected by Delta schema enforcement
Compare three schemas: what the code expects, what the source delivered, and what the target table currently has.
expected = {"sales_order_id": "string", "customer_id": "string",
"order_timestamp": "timestamp", "grand_total": "decimal(18,2)"}
source = dict(source_df.dtypes)
target = dict(spark.table("silver_sales_orders").dtypes)
print("missing in source:", set(expected) - set(source))
print("unexpected in source:", set(source) - set(expected))
print("type changes:", {c: (expected[c], source[c]) for c in expected.keys() & source.keys() if expected[c] != source[c]})
print("not in target:", set(source) - set(target))A write rejected by schema enforcement is Delta protecting the table; the fix is a deliberate decision about the schema, not a blanket mergeSchema to make the error go away. Delta Tables for SQL Server Engineers explains enforcement versus evolution.
Incremental-load failures
Incremental workloads fail in ways that are invisible to run status, and the execution log is how you reconstruct them. Follow the Incremental Loads design, and compare the log with the watermark table:
| Symptom | What to check |
|---|---|
| Wrong watermark | window_start and window_end in recent records against the watermark table; was it edited by hand? |
| Overlapping windows | Expected if a lookback is configured; a problem only if writes are not idempotent |
| Missing window | Gaps between one run’s window_end and the next run’s window_start |
| Duplicate processing | Same window processed twice with append-only writes; duplicate keys in Silver |
| Deletes not propagated | Watermark-only change detection cannot see physical deletes; compare key sets |
| Watermark advanced too early | A record with watermark_advanced = true and a status other than succeeded |
| Partial write before failure | failed with watermark_advanced = false after Silver committed; safe if the rerun is idempotent |
The last row is the common, recoverable case. Silver may already contain the window, but the watermark did not move, so the next run replays it, and the guarded MERGE turns the already-applied rows into no-ops.
MERGE debugging
Delta MERGE failures and wrong results have a small set of causes:
- Multiple source rows match one target row. Delta fails the MERGE rather than picking one. The source window contains duplicate keys, usually several versions of the same record, or a replay. Deduplicate to one row per business key before the MERGE.
- Unexpected duplicates in the target. Earlier appends inserted the same key twice, so even a correct MERGE can now match several target rows.
- Missing or null join keys. Rows with null keys never match, so they are inserted every time.
- Wrong predicate. A match condition on the wrong column, or with an extra filter, inserts rows that should have updated.
- Row-count mismatch. Compare
rows_readwithrows_inserted + rows_updatedin the log, and read the operation metrics fromDESCRIBE HISTORY. - Skewed keys make a MERGE slow rather than wrong; that is a tuning question.
-- Duplicate keys in the source window (cause of multiple-match failures)
SELECT sales_order_id, COUNT(*) AS versions
FROM bronze_sales_orders
WHERE modified_at > '2026-10-24 09:50' AND modified_at <= '2026-10-24 11:00'
GROUP BY sales_order_id HAVING COUNT(*) > 1;For performance rather than correctness, see Delta Lake MERGE performance patterns.
“The pipeline succeeded but the data is wrong”
This is the hardest category, because nothing is red. Technical success means every activity returned without an error. Business success means the right data arrived, complete and on time. They are different things. Common versions:
- Row count too low: the source path changed, a filter is too strict, a window parameter was a day off, or a join dropped rows.
- Duplicated rows: an append on rerun, or a MERGE with a bad key.
- Stale data: the run processed an empty window because the watermark had already moved, or the semantic model did not refresh after the load.
- A missing partition or file: the file arrived late or with a different name, and the load processed what was there.
- Incorrect aggregation: a change moved rows between groups and only the new group was recomputed.
- Validation did not run: it was on a branch that was skipped, or it logged a warning instead of failing.
Start with the execution log’s row counts for the window against the recent baseline, then the validation results, then the source totals. Most of these are found in minutes when counts are logged and in days when they are not. The prevention is always the same: validation that can fail the run.
Slow pipeline vs failed pipeline
A failure stops or errors. A performance incident completes successfully, but too slowly to meet its window. They need different investigations, so decide which one you have first.
For a slow run, split the time:
- Source slowness: the Copy or Lookup activity took longer; check the source system and the activity’s duration breakdown.
- Notebook startup: time from activity start until the first cell ran. Notebook startup time and session tags cover the causes.
- Processing: reading, transforming and shuffling took longer. Make Fabric notebooks faster covers the usual suspects.
- Write and commit: many small files, a wide MERGE, or compaction triggered after the write. Delta table maintenance covers file layout.
- Queueing and concurrency: Spark jobs waiting for cores.
- Capacity pressure: everything on the capacity slowed down at once.
The full workflow is the next step in this path, Fabric Performance Troubleshooting; here the goal is only to know which part to investigate.
Debugging a slow notebook
A short diagnostic sequence, in order:
- Is startup slow? Compare the activity start with the first cell’s log timestamp. If so, look at session configuration, environment and libraries, pool and capacity, not your code.
- Is data read slow? Look at the scan stages in the Spark application details. Are you reading far more files or columns than the window needs?
- Is the transformation slow? Long stages on joins, aggregations or window functions point to data volume or plan changes.
- Is shuffle or skew involved? A few tasks running far longer than the rest of their stage point to skewed keys.
- Is write or commit slow? A slow final stage, many output files, or a MERGE touching most of the table.
- Is capacity pressure present? Check the same window in the Capacity Metrics app before changing anything.
Compare against a comparable run: the same window size, the same code version, and a similar session state. A cold session and a reused one are not the same experiment.
Capacity-related debugging
When a workload is slow, or a Spark job failed to start, check the capacity before the code:
- Note the time window of the slow or failed run.
- Open the same window in the Capacity Metrics app. Was the capacity throttling? How close was it to its limits?
- Look for overlapping heavy workloads: other pipelines, refreshes, backfills, ad hoc queries.
- Correlate: did the slowdown start when the overlapping work did?
- Decide: code, or capacity?
Spark adds its own signal. When all Spark cores are in use, jobs started from pipelines are queued, while interactive notebook runs fail with a TooManyRequestsForCapacity error. When the capacity is throttled, new Spark jobs are rejected rather than queued. A notebook that “could not start” is therefore usually a capacity or concurrency problem, not a code problem. Microsoft Fabric Capacity Explained covers throttling and how to read the app.
Retry debugging
Retries are useful and misleading at the same time. They hide transient failures, which is their job, and they also hide instability, which is not.
retry_countis evidence. A run that succeeded withretry_count = 0and one that succeeded after four retries are not equally healthy.- Read the retried errors. The same error every attempt means a persistent cause that retries will not fix. Different errors suggest something flaky upstream.
- Watch the trend. One retry a month is noise. Retries every night, or growing in number, are an incident waiting to happen.
- Retries cost capacity. Each attempt restarts the activity’s work, so a retry storm during a source outage can turn into a capacity problem.
SELECT COALESCE(notebook_name, pipeline_name) AS workload,
COUNT(*) AS runs, SUM(CASE WHEN retry_count > 0 THEN 1 ELSE 0 END) AS runs_with_retries,
MAX(retry_count) AS max_retries
FROM dbo.execution_log
WHERE start_time >= DATEADD(day, -14, SYSUTCDATETIME())
GROUP BY COALESCE(notebook_name, pipeline_name)
HAVING SUM(CASE WHEN retry_count > 0 THEN 1 ELSE 0 END) > 0
ORDER BY runs_with_retries DESC;Rerun debugging
When you look at a workload’s history, distinguish four things:
- The original failed run: its own record, kept as evidence.
- Automatic retries inside that run:
retry_counton the same record. - A manual rerun of the whole workload: a new record with
rerun_of_execution_idpointing to the failed one,trigger_type = 'rerun', and the samecorrelation_id. - A partial rerun: Fabric can rerun a pipeline from the failed activity or from a selected activity. That is useful, but it skips the activities before it.
Rerunning only a failed child is dangerous when upstream state has changed since the original run. The source moved on, a Copy activity would now extract a different set of rows, or a later scheduled run has already advanced the watermark. Before a partial rerun, confirm that the inputs the failed step depends on are still the ones it would have seen, or rerun from the step that produces them.
Idempotency: make reruns safe before you need them
Every debugging session ends with a rerun, so whether reruns are safe is decided long before the incident:
- Append-only writes duplicate data on rerun. Use MERGE or overwrite a scoped slice for state tables.
- Duplicate writes from a failed attempt and a successful rerun are harmless only if the next step deduplicates.
- A partially completed MERGE cannot exist, because a Delta commit is atomic. But a multi-table load can leave one table updated and another not. Rerun the whole window, not just the table that failed.
- Duplicate file ingestion happens when processed-file tracking is missing or written before the load succeeds.
- Unsafe watermark advancement (moving it before downstream writes succeed) makes the rerun skip data.
Incremental Loads covers the patterns that make all of these safe. To prove a pipeline is rerun-safe before an incident, run the same batch twice as a test: see Testing Microsoft Fabric Pipelines and Notebooks.
File-ingestion failures
File loads fail one file at a time. When one does, check:
- Malformed file: open the raw file; look for broken quoting, ragged rows, truncated JSON.
- Missing file: did it arrive? Under which name and path? Did the producer send it late?
- Delimiter or encoding change: a semicolon instead of a comma, UTF-16 instead of UTF-8, a byte-order mark in the header.
- Duplicate file: same name or checksum as a file already processed, or a corrected resend under the same name.
- Partial file: still being written when the load listed the folder.
- Wrong filename pattern: the load’s pattern no longer matches what the producer sends.
- Schema drift: new, missing or retyped columns in this file only.
Process bad files into a quarantine area with the reason recorded, rather than failing the whole batch, when the business can accept partial loads. Fail the batch when it cannot.
Permission and authentication failures
The error usually says “access denied”, “forbidden” or “login failed”. The question is which identity was denied access to what:
- Workspace or item permission: the identity running the pipeline or notebook lacks a role on the workspace, or permission on a specific item (Lakehouse, Warehouse, another workspace’s item through a shortcut).
- Connection permission: pipelines use shared connections; the user or identity running the pipeline needs permission to use the connection, and the connection’s credential needs access to the source.
- Source credentials: an expired password or secret, a rotated key, or a revoked account on the source system.
- Gateway: for on-premises or private sources, the on-premises or virtual network data gateway must be online and allowed to reach the source.
- Service principal or managed identity: when a workload runs under one, its own permissions apply, not the developer’s. Many “works for me, fails in the schedule” incidents are exactly this.
Check what changed recently: an expired secret, a new workspace, a deployment that pointed a connection at a different source. Permission failures rarely appear spontaneously.
Environment and runtime problems
A notebook that worked yesterday and fails today without a code change often has a changed environment:
- Runtime change: a workspace or environment moved to a new Spark runtime, with different Spark, Delta and Python versions. Behavior changes are documented, but easy to miss. Fabric Runtime 2.0 vs 1.3 vs 1.2 covers what changed between versions.
- Library version mismatch: an unpinned package resolved to a newer version.
- Environment dependency: the attached environment was edited or republished, or a different environment is attached in this workspace after a deployment.
- Spark behavior change: a configuration default changed with the runtime, for example around V-Order or optimized writes.
Compare the failing run’s runtime and environment with the last successful run’s before debugging the code. Pin library versions in environments used by production workloads, and promote environment changes through the same deployment path as code.
Illustrative incident 1: a failed incremental notebook
This is an illustrative walk-through, not a measured incident.
The daily sales pipeline fails at 05:12. The execution log query for sales-daily:2026-10-24 shows the Copy job activity succeeded (1,240 rows), and nb_silver_orders failed at error_stage = merge_silver with a cast error, after one automatic retry. No validation or publish records exist. The failed execution’s record shows watermark_advanced = false, and in the watermark table last_successful_watermark is still at the previous run’s upper bound.
The notebook had already committed a Bronze append before failing, but Silver was not changed (the MERGE failed as one atomic operation) and the watermark did not move. A rerun of the same window is therefore safe.
Sampling the window finds one order where grand_total contains a currency symbol. The source application had a bug. The team adds a validation rule that quarantines unparseable amounts instead of failing the whole run, deploys it, and reruns the pipeline. The rerun gets a new execution record with rerun_of_execution_id pointing to the failed one; Silver and Gold update, validation passes, the watermark advances, and one quarantined row is reported to the source team.
Illustrative incident 2: slow but green
Illustrative, not measured.
A pipeline that usually finishes in about 12 minutes takes 37. It is green. The execution log shows the usual row counts and no retries. The notebook’s startup time is normal, but its processing time has roughly tripled. There was no deployment and no data growth. The Capacity Metrics app shows the capacity under heavy load in the same window, from a large backfill that someone scheduled in another workspace on the same capacity.
The fix is scheduling, not code: the backfill moves to a quieter window, and the next run returns to normal. Without the capacity check, the likely outcome was an afternoon spent optimizing a notebook that had not changed.
Debugging decision tree
- Problem detectedAlert, user report or failed validation
- Failed
- Pipeline activity
- Notebook exception
- Data validation
- Permission or source
- Slow
- Startup
- Processing
- Write
- Capacity
- Green but wrong
- Row counts
- Duplicates
- Freshness
- Skipped validation
- Evidence to fix
- Hypothesis
- Fix one thing
- Safe rerun
- Validate downstream
A few more diagnostic queries
Recent failures, newest first:
SELECT start_time, COALESCE(notebook_name, pipeline_name) AS workload,
error_code, error_stage, error_message, correlation_id, pipeline_run_id
FROM dbo.execution_log
WHERE status IN ('failed', 'timed_out') AND start_time >= DATEADD(day, -1, SYSUTCDATETIME())
ORDER BY start_time DESC;Pipeline failures across a workspace, if workspace monitoring is enabled:
ItemJobEventLogs
| where ItemKind == "Data Pipeline" and JobStatus == "Failed"
| where JobStartTime > ago(1d)
| project JobStartTime, ItemName, WorkspaceName, DurationMs, JobInstanceId
| order by JobStartTime desc
The monitoring article and execution log article include the duration-baseline and watermark queries. For quick command references during an incident, keep the pipeline debugging cheat sheet, the Fabric monitoring cheat sheet, the data platform monitoring checklist and, for slow notebooks, the notebook performance checklist at hand.
Common debugging mistakes
- Rerunning before understanding the failure. The rerun may fail the same way, or worse, succeed by duplicating data.
- Reading notebook logs without the parent pipeline. The notebook does not know which window or business date it was given unless you check the activity input or the execution log.
- Ignoring the retry count. Success after several retries is a symptom.
- Ignoring parameters. Wrong inputs look exactly like broken code.
- Assuming a successful pipeline means correct data. Status is not correctness.
- Debugging code before checking capacity. Contention looks like slow code from inside the notebook.
- Changing several things at once. If it works afterwards, you do not know why.
- Deleting failed-run evidence. Overwritten log records and truncated tables make the next incident harder.
- Manually fixing target data without recording it. An unlogged manual update is invisible to the next person and to reconciliation.
- Not verifying downstream state. A green rerun with a stale semantic model has not closed the incident.
Production debugging checklist
- Run ID of the failed or abnormal run.
- Correlation ID and the full run tree.
- Failed activity and its error, input and output.
- Parameters the run actually received.
- Source window (
window_start,window_end) and business date. - Row counts at each step, compared with the baseline.
- Schema of source and target, compared with expectations.
- Retry count and the errors of each attempt.
- Watermark state, and whether it moved.
- Capacity window for the same period.
- Validation results for the window.
- Safe rerun confirmed: idempotent writes, correct window, no skipped upstream steps.
- Downstream verification: validation passes, Gold and semantic models fresh.
- Root-cause note written, with the prevention step that will catch it next time.
When the problem is speed rather than failure, continue with the next step in this path, Fabric Performance Troubleshooting. Pipelines vs Notebooks vs Dataflow Gen2 vs Copy Job is also a useful check on whether each step is owned by the right tool, which decides where its evidence lives.
Was this helpful?
What could be better?
Thanks — that helps me improve Javi Builds.
Continue learning
- Cheat sheetDebug Fabric PipelinesTroubleshoot Microsoft Fabric pipeline failures by isolating the failed activity, separating transient from deterministic errors, validating parameters and identities, and using retries, logging, and controlled reruns safely.Open cheat sheet
- ArticleIncremental Loads in Microsoft Fabric: Watermarks, CDC and MERGE Without Reprocessing EverythingStop full reloads in Microsoft Fabric without losing correctness: bounded watermark windows, Change Tracking vs CDC, Delta MERGE, deletes, late data and safe reruns.Read article
What should I build next?
I'm building Javi Builds around practical problems data engineers actually face. Tell me what you'd like to see next.
Stay in the loop
Get the Javi Builds updates you actually care about.
Check your inbox
I sent you a confirmation link. Your subscription will be active after you confirm it.