Microsoft Fabric

Troubleshooting Microsoft Fabric Performance in Production

A repeatable workflow for slow Fabric workloads: find where elapsed time goes, then decide whether the cause is capacity, startup, reads, shuffle and skew, Delta layout, MERGE, writes, SQL, concurrency or data growth.

Par JaviPublié 19 min de lectureAvancé

Dans cet article

“Fabric is slow” is not a diagnosis. It is a symptom that can come from a dozen places: a capacity shared with a backfill, a Spark session that took six minutes to start, a join on a key that suddenly dominates the data, a table that has grown thousands of small files, a MERGE that now rewrites most of its target, or a load that still reprocesses everything it ever ingested.

Each of those has a different fix, and most of the fixes make things worse when applied to the wrong cause. Resizing the capacity does not fix skew. Running OPTIMIZE does not fix a busy capacity. Rewriting a notebook does not fix a slow source.

So the rule for this whole guide is: find where the elapsed time is going before tuning anything.

This is the last step of the Fabric Start Here path, and it works as a hub. The diagnostics here tell you which specialized guide to open; those guides hold the detailed fixes. It assumes you have the evidence the previous steps set up: monitoring, an execution log, and the debugging workflow for failures.

Fabric behavior referenced here was checked against Microsoft Learn in October 2026.

Where elapsed time goes

A workload’s total duration is the sum of several parts:

  • Queue and wait: waiting for orchestration, for Spark cores, for an upstream dependency.
  • Startup: acquiring a Spark session, loading the environment and libraries.
  • Read: listing and reading files or querying the source.
  • Processing: transformations, joins, aggregations.
  • Shuffle: redistributing data between tasks for joins, aggregations, windows.
  • Write and commit: writing files and committing the Delta transaction.
  • Downstream dependencies: validation, publishing, refreshes that the run waits for.

The first job is not to make any of these faster. It is to find which one changed. A pipeline that went from 12 minutes to 40 almost never got slower everywhere at once; one part grew, and that part points to the cause.

Establish a baseline first

“Slow” only means something relative to a normal run. Compare the slow run with runs that are actually comparable:

  • the same workload, not a similar one
  • similar data volume and the same kind of business window (a Monday catch-up is not a Tuesday)
  • the same runtime and environment
  • similar capacity conditions: the same time of day, a similar overlap with other workloads
  • recent history, typically the last few weeks, so the baseline follows gradual growth

Use the median and a high percentile, or the historical range, rather than an average, which one bad night distorts. There is no universal threshold. A five-minute micro-batch that takes eight minutes is a bigger regression than a two-hour nightly load that takes two and a quarter.

Requête SQL: Establish a baseline firstT-SQL (SQL database in Fabric)
-- Duration history for one workload (execution log, SQL database implementation)
SELECT CAST(start_time AS date) AS run_date, duration_ms / 60000.0 AS minutes,
       rows_read, rows_written, window_start, window_end
FROM dbo.execution_log
WHERE notebook_name = 'nb_silver_orders' AND status = 'succeeded'
  AND start_time >= DATEADD(day, -28, SYSUTCDATETIME())
ORDER BY start_time;
T-SQL (SQL database in Fabric)7 lignesAffichage en lecture seule. Aucune connexion à une base de données.

Reading duration and volume side by side answers the first question immediately: did the work grow, or did the same work get slower?

Symptom classification

Symptom Likely layer First evidence to inspect
Everything is slow at the same time Capacity or concurrency Capacity Metrics for the window; what else ran
One notebook is slow That notebook’s read, processing or write Duration history; Spark stages for the slow run
One pipeline is slow One of its activities Activity durations in the run; child execution timings
Startup is slow Session, environment, libraries, Spark capacity Time from activity start to first cell; session details
Reads are slow Scan size, files, partitions, source Spark scan stages; table file count; source latency
Transformations are slow Joins, shuffle, skew, plan change Stage durations; task-time distribution
MERGE is slow Target scan, file rewrites, source duplicates, skew DESCRIBE HISTORY MERGE metrics; target size and file count
Writes are slow Output file count, overwrite scope, commit, concurrency Final stage duration; files added per write
The workload queues Spark concurrency or capacity Spark job queue; capacity window
Throttling occurs Capacity overload Capacity Metrics throttling charts and system events
SQL queries slow down Query shape, data scanned, table layout, sync, capacity Query history; table file layout; capacity window
Performance worsens as data grows Full reloads, fragmentation, growing joins Duration vs rows trend; file count trend
A runtime change caused a regression Spark, Delta or library behavior Runtime and environment of normal vs slow runs
The slowdown is intermittent Contention at certain times; source load Slow runs by time of day; capacity window for each
The pipeline succeeds but takes much longer Any of the above; often capacity or data growth Baseline comparison, then phase split

The troubleshooting workflow

  1. Confirm the slowdown against the baseline, not against memory.
  2. Find the affected execution: run ID, correlation ID, window.
  3. Compare against the baseline for volume, window, runtime and time of day.
  4. Break elapsed time into phases: queue, startup, read, processing, shuffle, write, downstream.
  5. Check capacity pressure in the same window.
  6. Check concurrency: what else ran on the capacity, in the same Spark pool, on the same tables.
  7. Check data volume: rows, files and window size against the baseline.
  8. Check the source and read pattern.
  9. Check transformation and shuffle, including skew.
  10. Check Delta and file layout.
  11. Check write and commit.
  12. Check runtime and environment against the last normal run.
  13. Form one hypothesis that explains the phase that grew.
  14. Change one thing.
  15. Rerun under comparable conditions.
  16. Compare results with the baseline, not with the slow run alone.
  17. Validate correctness, because a faster run that writes the wrong data is not a fix.

Steps 5 and 6 come early on purpose. They are the cheapest checks and they rule out the most common cause that has nothing to do with your code.

Use the evidence you already have

You do not start from scratch. The earlier guides set up exactly the evidence this workflow needs:

  • Monitoring: run history, durations, retry counts, the capacity time window, alerts and baselines.
  • The execution log: run IDs, correlation_id, parent/child relationships, row counts, source windows, parameters and per-execution duration.
  • Debugging: the failure-oriented companion to this guide, for when the workload does not just slow down but breaks.

If those are not in place yet, set them up as part of the fix. The next incident will be much shorter.

Capacity diagnostics

Capacity is the first check because it is shared. Signs that the cause is capacity rather than the workload:

  • Several workloads slow down at the same time, not just one.
  • Spark jobs queue or interactive runs fail to start.
  • Throttling shows up in the Capacity Metrics app.
  • A competing workload is running: a backfill, a reprocessing job, a refresh storm.
  • The slowdown is intermittent and lines up with particular times of day.

The diagnostic path:

  1. Note the slow run’s time window.
  2. Open the same time range in the Capacity Metrics app.
  3. Identify the top consumers during that window.
  4. Decide whether the slowdown is workload-local (only this workload, capacity healthy) or capacity-wide (many workloads affected, capacity under pressure).

Capacity-wide slowdowns are fixed by scheduling, isolation or capacity changes, not by tuning one notebook. To check whether the capacity is genuinely undersized before resizing, run the Metrics app numbers through the Fabric Capacity Planner. Microsoft Fabric Capacity Explained covers smoothing, throttling and how to read the app; this guide does not repeat them.

Queueing vs execution time

A long pipeline duration does not necessarily mean slow code. Separate:

  • Orchestration wait: time between activities, waiting on dependencies or upstream runs.
  • Waiting to start: Spark jobs started from pipelines can queue when the capacity’s Spark cores are in use. Interactive notebook runs are not queued; they fail with a TooManyRequestsForCapacity error. When the capacity is throttled, new Spark jobs are rejected rather than queued.
  • Session startup: acquiring the session and loading the environment.
  • Active computation: the work you wrote.
  • Write and commit: the final stages.

The cheapest way to make this split visible is to record timestamps inside the notebook, at the first cell, before the final write and after the commit, in the execution log or its optional event table. The difference between the pipeline activity’s start and the first cell is waiting plus startup; the rest is execution.

Requête SQL: Queueing vs execution timeT-SQL (SQL database in Fabric)
-- Phase timings per run, from the optional execution_event table
-- (each phase logged as event_type = 'metric', metric_name = 'duration_s', stage = phase name)
SELECT l.execution_id, l.start_time,
       MAX(CASE WHEN ev.stage = 'startup' THEN ev.metric_value END) AS startup_s,
       MAX(CASE WHEN ev.stage = 'read'    THEN ev.metric_value END) AS read_s,
       MAX(CASE WHEN ev.stage = 'process' THEN ev.metric_value END) AS process_s,
       MAX(CASE WHEN ev.stage = 'write'   THEN ev.metric_value END) AS write_s
FROM dbo.execution_log AS l
JOIN dbo.execution_event AS ev
  ON ev.execution_id = l.execution_id AND ev.event_type = 'metric' AND ev.metric_name = 'duration_s'
WHERE l.notebook_name = 'nb_silver_orders' AND l.start_time >= DATEADD(day, -14, SYSUTCDATETIME())
GROUP BY l.execution_id, l.start_time
ORDER BY l.start_time;
T-SQL (SQL database in Fabric)13 lignesAffichage en lecture seule. Aucune connexion à une base de données.

Notebook startup

If the slowdown happens before your first cell runs, the cause is not in your code. Look at:

  • Session creation: whether the run got a pre-warmed starter pool session or fell back to an on-demand one.
  • Environment loading: an attached environment with many libraries, or one republished recently.
  • Dependencies installed inline at the start of the notebook.
  • Runtime configuration and pool size.
  • Concurrency: many notebooks starting at once, competing for sessions and cores.

Notebook startup time covers the causes and fixes in detail.

Session tags as a diagnostic aid

Session tags let compatible notebook activities in a pipeline share a high-concurrency session. That reduces startup cost, and it also changes how you read the evidence: several executions may report the same Spark session, the startup cost is paid once, and notebooks in one shared session compete for its resources. When a tagged notebook gets slower, check what else ran in the same session. Tags also make ownership clearer, because related notebooks are grouped visibly.

Notebook processing

When startup is normal and the processing phase grew, work through these as hypotheses to test, not as a checklist to apply:

  • Full scans where the window should limit the read.
  • Unnecessary columns read and carried through the job.
  • Filters applied late, after a join or aggregation instead of at the read.
  • Python loops over rows or files, or collecting data to the driver.
  • Repeated actions recomputing the same lineage several times.
  • Repeated reads of the same table in one notebook.
  • Shuffle-heavy operations that grew with the data.
  • Skew (next section).
  • Unnecessary cache or persist that spills or wastes memory.
  • Inefficient writes hiding in the processing phase because the final action triggers everything.

The Spark application details for the slow run show which stages took the time. Make Fabric notebooks faster covers the fixes, and the notebook performance checklist and notebook performance cheat sheet are quick references.

Data skew

Skew is when a few keys hold a disproportionate share of the data, so a few tasks do most of the work while the rest finish early and wait. Symptoms:

  • One task, or a handful, runs much longer than the others in its stage.
  • One key dominates a join or aggregation.
  • Partition sizes are uneven.
  • A join suddenly degrades even though total rows only grew modestly.

A quick check on the key distribution:

PySpark
from pyspark.sql import functions as F

dist = (spark.table("silver_sales_orders")
        .groupBy("customer_id").count()
        .orderBy(F.desc("count")))
dist.show(10)

stats = dist.agg(F.max("count").alias("max"), F.expr("percentile_approx(count, 0.5)").alias("median")).first()
print(f"largest key / median key = {stats['max'] / max(stats['median'], 1):.0f}x")
PySpark9 lignesCellule d’exemple en lecture seule. Rien sur cette page n’exécute de code.

A ratio that has grown sharply since the baseline is a strong clue. The fix depends on the cause (a placeholder value such as an “unknown” customer, a genuinely huge customer, a bad join key), so find out why one key dominates before changing the job.

Shuffle

A shuffle moves data between tasks so that rows with the same key end up together. Operations that commonly cause one:

  • joins (except when one side is small enough to broadcast)
  • groupBy aggregations
  • distinct and deduplication
  • repartition
  • window functions with partitionBy

Shuffles get expensive in two ways: the amount of data moved grows with volume, and skew concentrates it on a few tasks. If a stage’s shuffle read or write grew much faster than the input, look at what the job is shuffling and whether it needs to: filter earlier, select fewer columns, or avoid a repartition that is not needed. The PySpark performance cheat sheet collects the usual patterns.

Read performance

Slow reads usually mean the job is reading more than it needs, or reading it inefficiently:

  • Scanning too much data: a missing or ineffective window filter.
  • Too many files: listing and opening overhead.
  • Too many columns read from wide tables.
  • Weak predicate pruning: filters on columns whose values are spread across every file, so file statistics cannot skip anything.
  • Poor partition pruning: filters that do not match the partition column.
  • Repeated full-table reads of a large dimension.
  • Source latency for external sources.
  • Remote or gateway bottlenecks for on-premises or private sources.

The scan stages in the Spark details show how many files and bytes were read; compare them with the baseline run.

Small files

Many small files are one of the most common causes of gradual slowdown:

many small files
  → more metadata and file-open work per read
  → slower reads, MERGEs and SQL queries
  → workload duration creeps up even though the data grew only a little

Check the evidence before deciding to run maintenance:

PySpark
d = spark.sql("DESCRIBE DETAIL silver_sales_orders").first()
print(d["numFiles"], "files,", round(d["sizeInBytes"] / 1024**3, 2), "GB,",
      round(d["sizeInBytes"] / max(d["numFiles"], 1) / 1024**2, 1), "MB/file average")
PySpark3 lignesCellule d’exemple en lecture seule. Rien sur cette page n’exécute de code.

A file count that has grown much faster than the table’s size, with an average file size far below the table’s target, is the clue. Delta Table Maintenance in Microsoft Fabric covers the diagnosis per partition and what to do about it, including auto compaction.

Delta layout

Beyond file count, check:

  • average file size compared with the table’s target
  • table growth over the baseline period
  • partition count, and files per partition
  • clustering (liquid clustering or Z-Order) where the table uses it, and whether it has been applied by OPTIMIZE
  • recent maintenance: when compaction last ran, and whether auto compaction is enabled
  • the write pattern: many small appends, frequent MERGEs, overwrites

Maintenance should respond to evidence, not habit. A table with healthy files does not get faster from another OPTIMIZE.

MERGE performance

A MERGE that slows down over time usually has one of these causes:

  • The target scan is too large: the match condition cannot prune, so every run reads most of the target.
  • A poor join predicate: missing the columns that would allow pruning, or using a function on the key.
  • Source duplication: more source rows than necessary, or duplicates that were tolerated until they caused failures.
  • Skew in the match key.
  • Excessive file rewrites: changes spread across many files, so each run rewrites far more data than it changes.
  • Partition pruning not helping: filters that do not align with the partition column.
  • A growing target: the same change volume against a much larger table.

DESCRIBE HISTORY records MERGE metrics per run, such as files added and removed and rows updated, so compare a slow run’s metrics with a normal one. Delta Lake MERGE performance patterns covers the tuning.

Write performance

Separate the final write from the transformation before it. Because Spark is lazy, a slow “write” stage often includes processing triggered by the write. Look at:

  • too many output files from too many shuffle partitions
  • repeated tiny writes, such as a write per loop iteration
  • an expensive overwrite of a whole table when only a slice changed
  • commit overhead on tables with very long histories or many files
  • concurrent writers to the same table, causing conflicts and retries
  • a destination bottleneck for external sinks
  • over-partitioning, so each write touches many tiny partitions

Partitioning

Partitioning helps only when partitions are large and queries filter on the partition column. It hurts when:

  • there are too many partitions
  • the partition key has high cardinality (one partition per customer, per order, per timestamp)
  • partition directories are tiny
  • partition sizes are uneven
  • queries do not filter on the partition column, so pruning never happens

Microsoft’s guidance for Lakehouse tables read through the SQL analytics endpoint is to avoid high-cardinality partition columns and aim for partitions of around 1 GB or more. A dedicated Fabric partitioning guide is planned; until then, the Delta Lake partitioning checklist covers when partitioning earns its place.

SQL analytics endpoint

When queries against a Lakehouse’s SQL analytics endpoint slow down, check:

  • Query shape: SELECT * on wide tables, late filters, non-selective joins.
  • Data scanned: whether filters can skip files, given how the data is laid out.
  • Table and file layout: many small files and high-cardinality partitions slow scans, and they also slow the endpoint’s metadata sync.
  • Metadata synchronization: the endpoint discovers Lakehouse changes through a background sync that normally lags by under a minute, but can take longer with many lakehouses in a workspace, very large change volumes or poorly maintained tables. “Slow” can actually mean “not visible yet”. You can trigger a manual metadata refresh.
  • Capacity and concurrency in the same window.

Do not reach for SQL Server habits: there are no user-created indexes to add. Performance comes from file layout, statistics that the engine maintains automatically, and the query itself. Delta Tables for SQL Server Engineers explains why. For tables that need maintenance, Microsoft documents a sys.sp_get_table_health_metrics procedure you can use from a pipeline before running OPTIMIZE.

Warehouse performance

Fabric Warehouse manages its own layout, compaction and statistics, so troubleshooting focuses on the workload:

  • query complexity and the joins it performs
  • data volume scanned
  • concurrency: many queries at once
  • capacity and overlap with other workloads

Fabric Warehouse does not support user-created indexes. Its query insights views (such as recent and long-running queries) show what ran and how long it took, and the first run of a query can be slower because of a cold cache. Judge performance on repeat runs, not the first one. Lakehouse vs Warehouse covers when each engine is the better home for a workload.

Pipeline performance

Break a pipeline’s duration down into its parts:

pipeline duration
├── orchestration (gaps between activities, waits)
├── copy activities or Copy jobs
├── notebooks
├── validation
└── downstream publish or refresh

The parent pipeline’s duration hides where the time went. Use the execution log’s parent/child records, or the activity durations in the run detail view, to find the activity that grew. Often it is one child pipeline or one notebook, and the rest of the pipeline is unchanged.

Requête SQL: Pipeline performanceT-SQL (SQL database in Fabric)
-- Which execution in one operation took the time
SELECT item_type, COALESCE(notebook_name, pipeline_name) AS workload, duration_ms / 60000.0 AS minutes
FROM dbo.execution_log
WHERE correlation_id = 'sales-daily:2026-10-24'
ORDER BY duration_ms DESC;
T-SQL (SQL database in Fabric)5 lignesAffichage en lecture seule. Aucune connexion à une base de données.

Ingestion performance

Slow ingestion has its own set of causes:

  • Too many small files to list and copy.
  • One very large file that cannot be split, so one worker does all the work.
  • Source throughput: the source database or API is the bottleneck.
  • Gateway or network limits for on-premises and private sources.
  • Destination throughput.
  • Excessive parallelism that overloads the source.
  • Insufficient parallelism that leaves throughput unused.
  • A full reload where an incremental load would do.

Connector behavior differs. Parallelism and partitioning options depend on the source connector and on whether you use a Copy job or a Copy activity, so check what your connector supports before tuning. Pipelines vs Notebooks vs Dataflow Gen2 vs Copy Job covers which tool fits which kind of ingestion.

Incremental vs full reload

One of the most common long-term regressions looks like this:

the system was fast at go-live
  → the dataset grew
  → the process still reloads everything
  → duration grows in line with total size, every month

If duration tracks total table size rather than new data, the design is the problem, not the code or the capacity. Incremental Loads in Microsoft Fabric covers moving to bounded windows safely.

Runtime and environment regressions

When a workload slows down right after a platform change, compare the slow run’s runtime and environment with the last normal one:

  • runtime version: Spark, Delta and Python versions change between Fabric runtimes
  • library versions in the attached environment
  • environment changes: a republished environment, a different environment after deployment
  • dependency changes installed inline
  • Spark behavior differences: configuration defaults that changed with the runtime

Fabric Runtime 2.0 vs 1.3 vs 1.2 covers what changed between versions. Log the runtime and environment alongside each execution where practical; it turns “did anything change?” into a query.

“Nothing changed in the code, but it got slower”

This is the most common performance incident, and “nothing changed” is almost never true. Something changed, just not in the code. Investigate:

  • More rows: the source grew, or the window got bigger.
  • More files: fragmentation from frequent writes.
  • Larger joins: a dimension grew, or a join that used to broadcast no longer fits.
  • More skew: the distribution of a key changed.
  • More concurrency: new workloads on the same capacity or the same tables.
  • Longer incremental windows: a run that catches up after downtime processes days instead of hours.
  • Table fragmentation: maintenance stopped running, or never started.
  • A changed source distribution: a new customer, a new region or a new product line that dominates a key.

Plot duration against rows read and against file count over the baseline period. If duration grows with rows at the same rate, the work grew. If duration grows faster than rows, something else changed: layout, skew, capacity or concurrency.

Requête SQL: “Nothing changed in the code, but it got slower”T-SQL (SQL database in Fabric)
-- Duration vs volume trend: is the same work getting slower?
SELECT CAST(start_time AS date) AS run_date,
       rows_read,
       duration_ms / 1000.0 AS seconds,
       (duration_ms / 1000.0) / NULLIF(rows_read, 0) * 1000000 AS seconds_per_million_rows
FROM dbo.execution_log
WHERE notebook_name = 'nb_silver_orders' AND status = 'succeeded'
  AND start_time >= DATEADD(day, -60, SYSUTCDATETIME())
ORDER BY start_time;
T-SQL (SQL database in Fabric)9 lignesAffichage en lecture seule. Aucune connexion à une base de données.

A rising “seconds per million rows” means the job is doing the same work less efficiently, and the cause is in the layout, the plan or the environment, not the volume.

Regression comparison

Put a normal run and the slow run side by side. SQL Server performance engineers will recognize the discipline:

Compare Normal run Slow run
Data volume (rows read, written)
File count of the main tables
Parameters
Source window size
Code version
Runtime
Environment and library versions
Capacity conditions
Concurrent workloads
Source latency
Maintenance state

Most rows will be identical. The one or two that differ are where to look first.

Experiment discipline

Every performance change is an experiment:

baseline → hypothesis → one change → rerun → measure → validate correctness → record the conclusion

Do not change several things at once. Resizing the capacity, repartitioning, rewriting code, changing the runtime, running OPTIMIZE and moving the schedule in the same afternoon might make the next run faster, but you will not know which change did it, which ones were neutral, and which ones will hurt later. If multiple things change, the result teaches you very little.

Two more rules:

  • Do not trust one fast rerun. A rerun on a quiet capacity with a warm session is not comparable with the scheduled run. Compare over several runs under normal conditions.
  • Validate correctness after every change. A repartition or a new filter that makes the job faster by dropping rows is a regression, not a fix.

Illustrative incident: capacity contention

Illustrative, not a measured incident.

A nightly Silver load has run within a stable range for weeks. One week it takes roughly three times as long, every night. Rows read are unchanged, the window is unchanged, startup time is normal, but the processing phase has grown. There was no deployment. In the Capacity Metrics app, the same window shows a newly scheduled backfill in another workspace on the same capacity, running at the same hours.

The workload did not change; its neighbours did. Moving the backfill to a quieter window returns the load to its normal range. A capacity-wide fix for a capacity-wide cause. For a real, sanitized case where a predictable morning ingestion burst pushed a shared capacity past 200%, see How I Diagnosed a Microsoft Fabric Capacity Saturation Problem.

Illustrative incident: small-file degradation

Illustrative.

An incremental pipeline appends and merges small batches every 15 minutes. It stays correct, but its duration creeps up over a few months while the rows it processes grow only moderately. The “seconds per million rows” trend rises steadily. DESCRIBE DETAIL shows the active file count of the main Silver table has grown far faster than its size, and auto compaction was never enabled.

File layout is the dominant clue. Compacting the backlog once and enabling auto compaction brings the file count down, and the duration moves back towards its historical range.

Illustrative incident: skew

Illustrative.

A notebook that joins orders to customers becomes much slower although total order volume grew only slightly. The Spark stage for the join shows most tasks finishing quickly and a few running far longer. The key-distribution check shows one customer_id now holds a large share of the orders: a new marketplace integration assigns all its orders to one placeholder customer.

The regression is explained by a distribution change, not by volume. The team fixes the placeholder at the source, and adds the distribution check to the workload’s validation so the next skew shift is caught early.

Decision tree

  1. Slow workloadConfirmed against its baseline
  2. Capacity-wide or workload-specific?
    • Capacity-wide
      • Capacity pressure
      • Concurrency
      • Competing workloads
    • Startup
      • Runtime
      • Environment
      • Session
    • Read
      • Scan size
      • Files
      • Source
      • Partitions
    • Transform
      • Shuffle
      • Skew
      • Joins
    • Write
      • Output files
      • Commit
      • Concurrent writers
  3. Experiment
    • Hypothesis
    • One change
    • Rerun
    • Compare baseline
    • Validate correctness
Narrowing down a slow Fabric workload. First decide whether the cause is capacity-wide or specific to the workload. If it is the workload, decide whether the time went before processing (startup) or during it (read, transform, write). Then run one experiment at a time.

What not to do

  • Resize the capacity before measuring. It may hide the cause for a while and cost more indefinitely.
  • Run OPTIMIZE everywhere. Compacting healthy tables is pure cost.
  • Repartition blindly. A repartition adds a shuffle; it only helps when it fixes a specific imbalance.
  • Cache everything. Caching helps reused data and hurts everything else.
  • Compare unlike runs. A Monday catch-up against a Tuesday increment, a cold session against a warm one.
  • Apply SQL Server tuning assumptions blindly. There are no indexes to add to a Delta table, and Warehouse manages its own layout.
  • Change several variables at once.
  • Trust one fast rerun.
  • Ignore capacity.
  • Ignore data growth.
  • Optimize without validating the results.

Production checklist

  • Baseline defined: same workload, comparable runs, median and range.
  • Row volume compared with the baseline.
  • Source window size compared.
  • File count and average file size of the main tables checked.
  • Startup time separated from execution.
  • Processing time and the slowest stages identified.
  • Write time separated from processing.
  • Capacity window inspected for pressure and throttling.
  • Concurrency checked: other workloads, shared sessions, concurrent writers.
  • Runtime and environment compared with the last normal run.
  • Skew checked on join and aggregation keys.
  • Shuffle volume compared with the input.
  • Table layout reviewed: partitions, clustering, file sizes.
  • Maintenance state known: last compaction, auto compaction setting.
  • One hypothesis written down.
  • One change made.
  • Rerun under comparable conditions, measured over more than one run.
  • Correctness validated after the change.

That completes the Fabric Start Here path. For designing platforms that avoid many of these problems from the start, see Designing a Scalable Microsoft Fabric Architecture. For a wider operational review, the data platform monitoring checklist and the session tags cheat sheet are useful companions.

Preparing for a senior Fabric interview? The Microsoft Fabric interview questions guide covers throttling, slow notebooks and debugging as interview scenarios.

Étiquettes

  • Microsoft Fabric
  • Performance
  • Dépannage
  • Débogage
  • Capacité