Browse cheat sheets · Microsoft Fabric

Technology: Microsoft FabricCategory: TroubleshootingDifficulty: IntermediateUpdated

Debug Fabric Pipelines

Troubleshoot Microsoft Fabric pipeline failures by isolating the failed activity, separating transient from deterministic errors, validating parameters and identities, and using retries, logging, and controlled reruns safely.

On this page
  1. Problem
  2. Quick answer
  3. Code / commands
  4. 1. Start with the pipeline run, then drill into the activity
  5. 2. Find the first real failure, not every downstream failure
  6. 3. Record activity inputs before changing anything
  7. 4. Parameterize notebooks deliberately
  8. 5. Validate date parameters before using them
  9. 6. Validate source existence before expensive work
  10. 7. Add explicit notebook execution context
  11. 8. Add a pipeline run ID or batch ID to writes
  12. 9. Make notebook errors explicit
  13. 10. Distinguish transient errors from deterministic errors
  14. 11. Configure activity retries selectively
  15. 12. Use conditional retries where supported
  16. 13. Do not retry authentication failures automatically
  17. 14. Confirm which identity runs the notebook
  18. 15. Compare interactive notebook run vs pipeline run
  19. 16. Validate the default Lakehouse
  20. 17. Debug Notebook activity startup separately from notebook code
  21. 18. Check Spark stage/task failures for notebook errors
  22. 19. Watch for Delta concurrency conflicts
  23. 20. Check whether the failed run partially wrote data
  24. 21. Design activities to be idempotent
  25. 22. Check for duplicate batches before rerunning
  26. 23. Use a control table for operational state
  27. 24. Avoid concurrent blind appends to the same execution log without design
  28. 25. Fail a pipeline when business validation fails
  29. 26. Do not hard-code environment-specific paths
  30. 27. Debug Copy activity failures by layer
  31. 28. Validate source and sink row counts when available
  32. 29. Debug stored procedure activity failures separately
  33. 30. Set activity timeout deliberately
  34. 31. Differentiate timeout from performance regression
  35. 32. Use dependency conditions intentionally
  36. 33. Preserve the original error when adding failure handling
  37. 34. Pass only necessary error context to alerts
  38. 35. Use Secure Input / Secure Output for sensitive activities
  39. 36. Use workspace monitoring for recurring failure analysis
  40. 37. Query failed pipeline activity runs
  41. 38. Summarize failures by activity
  42. 39. Track duration regressions
  43. 40. Rerun the smallest safe scope
  44. What the result means
  45. What to look for
  46. Failure happens immediately
  47. Failure happens after long execution
  48. Only pipeline execution fails
  49. Failure disappears after retry
  50. Rerun succeeds but data duplicates
  51. Several downstream activities fail or skip
  52. Production notes
  53. Most activity retries should target transient faults
  54. Notebook activity execution context matters
  55. Pipeline runs are not one distributed transaction
  56. Logging should be durable
  57. Workspace Monitoring can centralize pipeline history
  58. Alerts should be actionable
  59. Retries consume time and capacity
  60. Common mistakes
  61. 1. Rerunning the whole pipeline immediately
  62. 2. Retrying every failure
  63. 3. Hiding notebook exceptions
  64. 4. Debugging skipped activities
  65. 5. Assuming interactive identity equals pipeline identity
  66. 6. Increasing timeout instead of investigating regression
  67. 7. Blind append on rerun
  68. 8. Adding retries before write safety
  69. 9. Logging secrets for debugging
  70. 10. Treating one successful rerun as proof the issue is solved
  71. Next step
  72. Related deep dives

Problem

A Microsoft Fabric pipeline fails, stalls, retries repeatedly, or produces incorrect output.

The failure can come from different layers:

  • pipeline orchestration
  • Notebook activity
  • Copy activity
  • stored procedure activity
  • Dataflow activity
  • credentials or workspace identity
  • parameters
  • Spark startup or runtime
  • source connectivity
  • destination write conflicts
  • schema drift
  • downstream dependency logic

The goal is to identify which layer failed and whether the failure is transient or deterministic before rerunning anything.

A production-safe debugging flow should answer:

  • Which activity failed first?
  • What did that activity receive as input?
  • What identity did it run under?
  • Was the error transient?
  • Did the failed activity partially write data?
  • Is rerunning the activity safe?
  • Should the retry happen automatically or require intervention?

Quick answer

Use this sequence:

1. Open the failed pipeline run
2. Identify the first failed activity
3. Read the activity error, not only the pipeline status
4. Capture run ID, activity name, start/end time, and input parameters
5. Classify the error:
   - transient
   - authentication/permission
   - parameter/configuration
   - data/schema
   - notebook/Spark
   - destination/write conflict
6. Check whether the activity partially committed data
7. Fix the root cause
8. Rerun only when the activity is idempotent or safely recoverable
9. Add targeted retry only for transient failures
10. Persist operational metadata so the next incident is easier

Do not start with:

"Run the whole pipeline again."

until you know whether previous activities already wrote data.

Code / commands

1. Start with the pipeline run, then drill into the activity

In Fabric, use the pipeline run history / Output view to identify:

Pipeline run
  ↓
Failed activity
  ↓
Activity input
  ↓
Activity output/error

Capture at minimum:

pipeline name
pipeline run ID
activity name
activity type
start time
end time
duration
status
error code
error message

The pipeline-level status tells you that something failed.

The activity output usually tells you what failed.


2. Find the first real failure, not every downstream failure

A pipeline can show several failed or skipped activities because one upstream dependency failed.

Example:

Copy_Raw_Data         Succeeded
Transform_Notebook    Failed
Publish_Warehouse     Skipped
Send_Success_Email    Skipped

The debugging target is:

Transform_Notebook

not the skipped downstream activities.

Always identify the earliest failure in the dependency chain.


3. Record activity inputs before changing anything

For a failed activity, capture the effective values passed into it.

Examples:

source_path
target_table
start_date
end_date
environment
workspace
lakehouse
batch_id
run_id

Parameter bugs often look like data or connectivity bugs.

A notebook may be correct but receive:

2026/13/01

instead of:

2026-10-01

or point to the wrong Lakehouse.


4. Parameterize notebooks deliberately

A Fabric Notebook activity can pass parameters to a notebook.

Inside the notebook, define parameters explicitly in the designated parameter cell.

Example:

source_table = "bronze_sales"
target_table = "silver_sales"
run_date = "2026-10-04"

Then validate them early:

required = {
    "source_table": source_table,
    "target_table": target_table,
    "run_date": run_date,
}

for name, value in required.items():
    if value in (None, ""):
        raise ValueError(
            f"Missing required parameter: {name}"
        )

Fail early with a clear message instead of allowing a vague downstream failure.


5. Validate date parameters before using them

from datetime import datetime

try:
    parsed_run_date = datetime.strptime(
        run_date,
        "%Y-%m-%d",
    ).date()
except ValueError as exc:
    raise ValueError(
        f"run_date must be YYYY-MM-DD; got {run_date!r}"
    ) from exc

Do not allow malformed dates to fail later inside Spark filters or file paths.


6. Validate source existence before expensive work

Example with Spark tables:

if not spark.catalog.tableExists(source_table):
    raise RuntimeError(
        f"Source table does not exist: {source_table}"
    )

For file-based workloads, validate the source path or expected partition before starting a large transformation.

Failing fast gives a clearer operational signal.


7. Add explicit notebook execution context

At the beginning of a production notebook:

print(
    {
        "source_table": source_table,
        "target_table": target_table,
        "run_date": run_date,
    }
)

Do not print:

  • passwords
  • access tokens
  • connection strings containing secrets
  • confidential payloads

Operational logs should help debugging without leaking credentials.


8. Add a pipeline run ID or batch ID to writes

If the pipeline passes a run identifier:

from pyspark.sql import functions as F

output = (
    output
    .withColumn(
        "pipeline_run_id",
        F.lit(pipeline_run_id),
    )
    .withColumn(
        "ingested_at_utc",
        F.current_timestamp(),
    )
)

This helps answer:

Which pipeline execution wrote this row?

It also helps detect replayed batches and partial reruns.


9. Make notebook errors explicit

Bad:

try:
    do_work()
except Exception:
    print("something failed")

This can hide failure from the orchestrator.

Better:

try:
    do_work()
except Exception as exc:
    print(
        f"Notebook failed: {type(exc).__name__}: {exc}"
    )
    raise

Let the notebook activity fail when the transformation actually failed.

A pipeline that reports success with incomplete data is worse than a visible failure.


10. Distinguish transient errors from deterministic errors

Transient examples:

temporary service unavailability
HTTP 429 throttling
network timeout
temporary endpoint failure
temporary resource contention

Deterministic examples:

wrong column name
invalid SQL
missing table
permission denied
bad parameter
schema mismatch
duplicate business keys
invalid credentials

Retry transient problems.

Fix deterministic problems.

Do not use retries to hide broken logic.


11. Configure activity retries selectively

Most Fabric Data Factory pipeline activities support retries.

Use activity-level retry settings for failures that may recover automatically.

Conceptually:

Activity
→ General
→ Enable retries
→ Retry count
→ Retry interval

For brief transient problems, a fixed delay may be appropriate.

For throttling or unstable external services, increasing delay can reduce repeated pressure.

Do not set large retry counts by default.


12. Use conditional retries where supported

Fabric supports retry conditions for selected activity types.

Typical criteria include:

error code
error message
failure type

Example policy:

Retry:
HTTP 429
temporary system error
connection timeout

Do not retry:
authentication failure
bad schema
invalid SQL
missing source object

Conditional retry capability is feature/activity dependent, so verify support for the activity being used.


13. Do not retry authentication failures automatically

Example:

401
403
invalid credential
permission denied

Repeated retries usually do not fix:

  • expired secret
  • missing role
  • incorrect workspace permission
  • wrong identity

Stop retrying and fix access.


14. Confirm which identity runs the notebook

Notebook execution context matters.

Depending on how execution is triggered, the identity can differ.

For pipeline-triggered notebooks, validate the effective pipeline authentication/security context before assuming the notebook runs as the notebook owner.

When a notebook works interactively but fails in a pipeline, check:

identity
workspace access
Lakehouse access
external data permissions
API permissions

before changing notebook code.


15. Compare interactive notebook run vs pipeline run

If:

Notebook succeeds interactively
Pipeline Notebook activity fails

compare:

identity
parameters
default Lakehouse
Environment
Spark configuration
session mode
workspace
source credentials

The code may be identical while the execution context is not.


16. Validate the default Lakehouse

A notebook can fail if it expects tables from one Lakehouse but the activity/session uses another.

At notebook startup, log or validate the expected table namespace.

Prefer explicit table/path references for critical cross-Lakehouse workloads when ambiguity would be dangerous.

Do not assume the interactive notebook attachment exactly matches pipeline execution context.


17. Debug Notebook activity startup separately from notebook code

A Notebook activity can fail before useful Spark work starts.

Separate:

session acquisition
Spark startup
environment initialization
library setup
notebook code execution

If the failure happens before the first expected log line, investigate session/environment/platform setup rather than the transformation.


18. Check Spark stage/task failures for notebook errors

If the Notebook activity failed during Spark work, inspect:

failed stage
failed task
executor error
shuffle error
Delta conflict
out-of-memory condition
data parsing exception

The pipeline error is often only the outer wrapper.

The Spark job details contain the distributed execution cause.


19. Watch for Delta concurrency conflicts

Example scenario:

Notebook A → MERGE target table
Notebook B → MERGE same target table

at the same time.

Symptoms may include concurrent transaction/write conflict errors.

Do not solve this by increasing pipeline retries indefinitely.

Consider:

append to staging
→ serialize/reconcile merges

or isolate writers by partition/business domain where safe.


20. Check whether the failed run partially wrote data

Before rerunning, ask:

Did earlier activities succeed?
Did this activity commit anything?
Did the target contain the batch already?

For Delta writes, a failed Delta transaction does not partially commit that transaction, but earlier successful activities may already have committed data.

The pipeline as a whole is not one global transaction.


21. Design activities to be idempotent

A safe rerun should produce the same logical result.

Examples:

MERGE by business key
replace known partition
append with batch replay protection
write to staging then promote

Unsafe rerun pattern:

blind append

when the same source batch can be processed twice.


22. Check for duplicate batches before rerunning

If you persist batch metadata:

existing = (
    spark.table("pipeline_execution_log")
    .filter(
        F.col("batch_id") == batch_id
    )
)

if existing.count() > 0:
    raise RuntimeError(
        f"Batch already processed: {batch_id}"
    )

In a real production design, avoid expensive count() checks when a more efficient keyed lookup or control-table design exists.

The point is to make replay state explicit.


23. Use a control table for operational state

Example logical schema:

pipeline_execution_log
----------------------
pipeline_name
pipeline_run_id
batch_id
activity_name
status
started_at
completed_at
rows_read
rows_written
error_type
error_message

Use this to answer:

Was this batch already processed?
Where did the last run stop?
How many rows were written?
Which activity failed?

Avoid storing secrets or huge stack traces directly in operational tables.


24. Avoid concurrent blind appends to the same execution log without design

An execution log can itself become a concurrency hotspot.

If multiple notebooks write simultaneously, design the logging layer for concurrency.

Possible approaches:

append-only per activity
unique run/activity key
controlled merge/upsert
central logging service/table

Do not let observability become the failure point.


25. Fail a pipeline when business validation fails

Example:

bad_rows = (
    output
    .filter(
        F.col("merchant_id").isNull()
    )
)

bad_count = bad_rows.count()

if bad_count > 0:
    raise RuntimeError(
        f"Validation failed: {bad_count} rows have null merchant_id"
    )

A technically successful Spark job can still produce invalid business data.

Operational success should include quality checks where required.


26. Do not hard-code environment-specific paths

Bad:

source = "abfss://prod@..."

inside every notebook.

Prefer parameters or environment configuration:

source_path = source_path_param

This reduces:

  • accidental production access
  • deployment drift
  • code changes between environments

27. Debug Copy activity failures by layer

For Copy activity, classify the error:

Source connection
Source query
Authentication
Network/connectivity
Schema mapping
Serialization
Sink connection
Sink constraint/write
Throughput/throttling

Do not treat every Copy failure as a connectivity issue.

Check whether the source read actually succeeded before investigating the sink.


28. Validate source and sink row counts when available

Record:

rows read
rows written
rows skipped/rejected

If:

rows read = 10,000,000
rows written = 0

the source is probably not the problem.

If:

rows read = 0

focus upstream first.


29. Debug stored procedure activity failures separately

A Stored Procedure activity may fail because of:

connection/authentication
missing procedure
invalid parameter
SQL exception
transaction/deadlock
timeout
business validation

Capture the SQL error number/message where available.

Do not automatically retry logic errors.

For deadlocks or transient connection failures, targeted retries may be reasonable.


30. Set activity timeout deliberately

Fabric activities support timeout configuration.

A timeout should represent:

the longest duration the activity is allowed to consume resources before being treated as abnormal

Do not set an extremely high timeout simply to avoid failures.

A timeout can reveal:

  • stuck external calls
  • runaway Spark jobs
  • blocked SQL
  • unexpected volume increases

Use historical duration to choose a defensible operational limit.


31. Differentiate timeout from performance regression

If a notebook usually takes:

20 minutes

and now hits a:

2-hour timeout

do not simply raise the timeout.

Investigate:

input volume
Spark plan
shuffle
skew
capacity
source latency
concurrency

A timeout increase can hide a real regression.


32. Use dependency conditions intentionally

Typical orchestration logic:

On success
On failure
On completion

Design failure paths explicitly.

Example:

Transform
  ├─ success → Publish
  └─ failure → Log Failure

Do not let a diagnostic/notification activity accidentally convert the logical pipeline outcome into success.


33. Preserve the original error when adding failure handling

Failure-handling logic should record or alert on the error, but the original failed business activity should remain visible.

Avoid patterns where:

Activity fails
→ logging succeeds
→ pipeline appears successful

unless that is intentionally the contract.


34. Pass only necessary error context to alerts

Useful alert fields:

pipeline name
run ID
activity
environment
failure type
error code
short error message
monitoring link/context

Avoid sending:

  • credentials
  • tokens
  • full sensitive payloads
  • large data samples

35. Use Secure Input / Secure Output for sensitive activities

Fabric activity settings can hide sensitive input/output from run logs.

Use secure logging controls when activity payloads may include confidential values.

This reduces observability detail, so pair it with safe operational identifiers.


36. Use workspace monitoring for recurring failure analysis

When Fabric Workspace Monitoring is enabled, pipeline and activity logs can be analyzed centrally.

Pipeline-level events can be queried from the monitoring eventhouse.

Example KQL pattern:

ItemJobEventLogs
| where ItemKind == "Pipeline"
| summarize Runs = count() by JobStatus

Use activity-level logs for deeper investigation when available.

Central monitoring is more useful than manually opening individual runs when failures recur.


37. Query failed pipeline activity runs

A practical activity-level pattern is:

FabricDataPipelinesActivityRunsLogs
| where ActivityStatus == "Failed"
| project
    PipelineName,
    PipelineRunId,
    ActivityName,
    ActivityType,
    ActivityStartTime,
    ActivityEndTime,
    FailureType
| order by ActivityStartTime desc

Column availability can vary with the monitoring schema/version, so inspect the actual table schema in your workspace.

Use this to identify:

  • repeat offenders
  • failure clusters
  • failing activity types
  • time-based patterns

38. Summarize failures by activity

FabricDataPipelinesActivityRunsLogs
| where ActivityStatus == "Failed"
| summarize Failures = count()
    by PipelineName, ActivityName, ActivityType
| order by Failures desc

This turns operational history into prioritization.

A pipeline that failed once because of a temporary outage is different from an activity that fails every day.


39. Track duration regressions

Example concept:

FabricDataPipelinesActivityRunsLogs
| where ActivityStatus == "Succeeded"
| summarize
    AvgDurationMs = avg(DurationMs),
    MaxDurationMs = max(DurationMs)
    by PipelineName, ActivityName

Use actual schema field names available in your monitoring table.

Do not invent universal performance thresholds.

Compare each activity against its own historical baseline.


40. Rerun the smallest safe scope

After fixing the root cause, prefer:

rerun failed activity / failed branch

when the platform/workflow supports it safely and upstream outputs are valid.

Otherwise rerun the whole pipeline only when:

  • earlier activities are idempotent
  • duplicates are prevented
  • partial state is understood

The safest rerun scope depends on the pipeline design.

What the result means

Signal Likely direction
Notebook works interactively, fails in pipeline Identity / parameters / environment / Lakehouse context
HTTP 429 / temporary service failure Transient retry candidate
401 / 403 Authentication or permission
Invalid column / schema mismatch Deterministic data or code issue
Spark stage failure Notebook distributed-execution problem
Delta conflict Concurrent write / merge problem
Timeout Performance, blocking, dependency, or resource issue
Upstream succeeded, downstream skipped Dependency behavior, not multiple independent failures
Rerun creates duplicates Non-idempotent write design
Repeated same activity failure Structural issue, not random incident

What to look for

Failure happens immediately

Likely causes:

  • invalid parameter
  • missing object
  • permission
  • configuration
  • wrong path
  • syntax

Check setup before performance.

Failure happens after long execution

Likely causes:

  • Spark stage failure
  • timeout
  • write conflict
  • memory pressure
  • source/sink instability
  • large-volume edge case

Inspect runtime telemetry.

Only pipeline execution fails

Compare pipeline execution context to interactive execution.

Especially:

  • identity
  • parameters
  • default Lakehouse
  • workspace
  • Environment
  • external permissions

Failure disappears after retry

That supports a transient-failure hypothesis.

Still inspect frequency.

If retries are constantly required, the dependency may be unhealthy or overloaded.

Rerun succeeds but data duplicates

The retry/rerun policy is unsafe.

Fix idempotency before increasing retry counts.

Several downstream activities fail or skip

Trace the dependency graph back to the first genuine failure.

Do not debug every downstream symptom independently.

Production notes

Most activity retries should target transient faults

Fabric Data Factory supports activity retries and retry intervals.

Current retry capabilities also include increasing-delay behavior and conditional retry support for selected activity types.

Use those features for recoverable infrastructure/service failures, not deterministic code errors.

Notebook activity execution context matters

Pipeline-triggered notebook execution can use a different security context from an interactive notebook run.

A notebook that works manually can therefore fail from the pipeline because access conditions differ.

Pipeline runs are not one distributed transaction

One activity can succeed and commit before a later activity fails.

Rerun safety must be designed per activity.

Logging should be durable

Notebook print output is useful for immediate troubleshooting.

For operations, persist:

  • run IDs
  • batch IDs
  • row counts
  • status
  • timestamps
  • safe error metadata

so incidents can be compared over time.

Workspace Monitoring can centralize pipeline history

Fabric Workspace Monitoring can expose pipeline-level and activity-level execution logs through a monitoring eventhouse/KQL database.

This is much better for recurring operational analysis than opening individual run pages manually.

Alerts should be actionable

Fabric supports pipeline/activity alerting patterns.

Alert on failures that need intervention, not every transient event that self-recovers within the expected retry policy.

Retries consume time and capacity

A retry is not free.

Excessive retries can:

  • increase pipeline latency
  • increase capacity consumption
  • increase load on an already unhealthy dependency
  • duplicate work if writes are not idempotent

Use retry counts deliberately.

Common mistakes

1. Rerunning the whole pipeline immediately

Earlier activities may already have committed data.

Check partial state first.

2. Retrying every failure

Authentication, bad SQL, invalid schema, and incorrect parameters will not improve because you waited 30 seconds.

3. Hiding notebook exceptions

Printing an error without re-raising can make the notebook appear successful.

Fail visibly when the workload failed.

4. Debugging skipped activities

Skipped downstream activities are usually consequences.

Find the earliest actual failure.

5. Assuming interactive identity equals pipeline identity

Execution context can differ.

Always compare identity and permissions.

6. Increasing timeout instead of investigating regression

A much slower activity may indicate volume growth, skew, blocking, or resource pressure.

7. Blind append on rerun

This is a common way to create duplicate data.

Design idempotency.

8. Adding retries before write safety

Automatic retries are dangerous when the activity can partially or repeatedly write non-idempotent data.

9. Logging secrets for debugging

Never expose credentials or tokens in pipeline output.

Use secure input/output controls where appropriate.

10. Treating one successful rerun as proof the issue is solved

A successful rerun suggests transience but does not explain repeated failures.

Trend the failure over time.

Next step

Use the failure type to choose the next action.

  • 429 / temporary service error → add targeted retry with appropriate delay/backoff.
  • 401 / 403 → fix identity and permission; do not keep retrying.
  • Notebook works interactively only → compare pipeline identity, parameters, Lakehouse, and Environment.
  • Spark stage fails → inspect Spark job/stage/task diagnostics.
  • Delta write conflicts → reduce overlapping writers or stage then reconcile.
  • Schema/data error → validate inputs before expensive work and fail early.
  • Timeout → compare runtime with historical baseline before increasing the limit.
  • Rerun duplicates data → redesign the activity for idempotency.
  • Failure recurs → move from one-off debugging to Workspace Monitoring and trend analysis.
  • Pipeline succeeds but data is wrong → add business-quality validation, not only technical success checks.

← All cheat sheets