Microsoft Fabric

Microsoft Fabric Notebook Parameters: Practical Patterns for Reusable Pipelines

Make one Fabric notebook serve many runs: the parameters cell, pipeline base parameters, validation, date windows, environments, secrets, safe logging and testing.

By JaviPublished 11 min readIntermediate

A notebook that only works for one client, one date and one environment is a script that has to be edited before every run. Once it runs in a pipeline, every edit becomes a risk: someone changes a date in a cell, commits it by accident, and production processes the wrong day.

Parameters fix that. The notebook’s logic stays fixed and reviewed; the values change per run. (If you are still deciding whether a step belongs in a notebook at all, see Pipelines vs Notebooks vs Dataflow Gen2 vs Copy Job.) This guide covers how Fabric notebook parameters actually work, then the patterns that make them safe in production: validation, date windows, environment values, secrets, logging and testing.

How Fabric notebook parameters work

A Fabric notebook has an optional parameters cell. You mark it with Toggle parameter cell in the cell’s More commands menu. The variables assigned in that cell are the notebook’s parameters, and their values are the defaults.

When a pipeline runs the notebook with a Notebook activity, it passes values under Base parameters in the activity’s Settings. At execution time, Fabric injects a new cell immediately after the parameters cell that assigns the passed values. Anything the pipeline does not pass keeps its default.

Three consequences follow:

  1. Names must match. A base parameter called RunDate does not override run_date. It creates a new variable and the default silently stays in effect.
  2. Types are simple. Notebook parameters support int, float, bool and string. Lists and dictionaries are not supported directly; pass them as a JSON string and parse them in the notebook.
  3. Code before the parameters cell sees only defaults. Put the parameters cell near the top, after imports, and do all work that depends on parameters after it.

When you select a notebook in a Notebook activity, Fabric can auto-populate base parameters from the parameters cell, with names, types and default values. It only discovers variables in the parameters cell, which is another reason to keep every parameter there.

A minimal parameters cell

Keep the parameters cell boring: plain assignments, safe defaults, no logic.

PythonParameters9 lines
Python
# Parameters cell (toggle "parameter cell" on this cell)
client_id = ""
run_date = ""
window_days = 1
pipeline_run_id = ""
environment = "dev"
source_path = "Files/landing/orders"
target_table = "silver_orders"
dry_run = True
Python9 linesRead-only example cell. Nothing on this page runs code.

The defaults are chosen so that an accidental interactive run is harmless: an empty client_id fails validation, and dry_run = True prevents writes. A real run must pass explicit values.

Passing values from a pipeline

In the Notebook activity, set each base parameter with a matching name and type. Values can be literals, pipeline parameters or expressions from the pipeline expression language, for example:

Base parameter Type Value
client_id String @pipeline().parameters.client_id
run_date String @formatDateTime(pipeline().TriggerTime, 'yyyy-MM-dd')
window_days Int @pipeline().parameters.window_days
environment String a pipeline parameter or a Variable Library value
dry_run Bool false

Prefer computing business dates in the pipeline from the trigger time, or passing them explicitly for reruns, rather than calling “now” inside the notebook. A notebook that reads the current time cannot be rerun for yesterday.

Pass a run identifier too. @pipeline().RunId gives each pipeline run a unique ID; passing it to the notebook as pipeline_run_id lets every log line and output row be traced back to the run that produced it.

Validate parameters first

Everything that comes from outside the notebook is input, and input needs validation. Put a validation cell straight after the parameters cell and fail fast with a clear message.

Python
from datetime import date, timedelta
import re

ALLOWED_ENVIRONMENTS = {"dev", "test", "prod"}
CLIENT_ID_PATTERN = re.compile(r"^[a-z0-9][a-z0-9-]{1,62}$")
TABLE_PATTERN = re.compile(r"^[a-z][a-z0-9_]{0,127}$")


def validate_parameters():
    errors = []

    if not CLIENT_ID_PATTERN.fullmatch(client_id or ""):
        errors.append("client_id must be lowercase letters, digits or hyphens")

    try:
        start = date.fromisoformat(run_date)
    except ValueError:
        errors.append("run_date must be an ISO date such as 2026-10-01")
        start = None

    if not isinstance(window_days, int) or not 1 <= window_days <= 31:
        errors.append("window_days must be an integer between 1 and 31")

    if environment not in ALLOWED_ENVIRONMENTS:
        errors.append(f"environment must be one of {sorted(ALLOWED_ENVIRONMENTS)}")

    if not TABLE_PATTERN.fullmatch(target_table or ""):
        errors.append("target_table must be a simple lowercase table name")

    if errors:
        raise ValueError("Invalid notebook parameters: " + "; ".join(errors))

    return start, start + timedelta(days=window_days)


window_start, window_end = validate_parameters()
Python36 linesRead-only example cell. Nothing on this page runs code.

Validation does three jobs. It catches pipeline misconfiguration early, before any Spark work runs. It produces an error message that an on-call engineer can act on. And it protects code that builds paths or table names from values that were never meant to reach it. Never build SQL or paths by concatenating unvalidated strings.

Date and window parameters

Date handling causes more reprocessing bugs than any other parameter. A few rules keep it predictable:

  • Pass dates as ISO strings (2026-10-01) and parse them once, in the validation cell.
  • Use half-open windows: include the start, exclude the end. window_start <= event_date < window_end never double-counts a boundary day.
  • Make the window explicit. run_date plus window_days lets the same notebook process one day, catch up a week, or reprocess a month without code changes.
  • Make writes idempotent for the window. Rerunning the same window must produce the same result, which usually means replacing or merging the rows for that window instead of appending them again. Incremental Loads in Microsoft Fabric shows how to derive these windows from a watermark and keep reruns safe.
PySpark
from pyspark.sql import functions as F

orders = (
    spark.read.format("delta")
    .load(source_path)
    .where(F.col("client_id") == client_id)
    .where(
        (F.col("order_date") >= F.lit(window_start))
        & (F.col("order_date") < F.lit(window_end))
    )
)
PySpark11 linesRead-only example cell. Nothing on this page runs code.

Filtering on the window as early as possible also lets Delta skip files and partitions it does not need, which matters on large tables.

Environment and runtime configuration

Not every value should be a pipeline parameter. Separate three kinds of configuration:

  1. Run parameters that change per execution: client_id, run_date, window_days, pipeline_run_id, dry_run. These belong in the parameters cell and the Notebook activity.
  2. Environment configuration that changes per workspace but not per run: source locations, retention settings, feature switches. Fabric Variable Libraries are designed for this. A notebook can read them with notebookutils.variableLibrary.getLibrary(“<library name>”), and pipelines can reference library variables in dynamic content. The active value set follows the environment.
  3. Runtime context that Fabric already knows: notebookutils.runtime.context returns read-only properties such as the current workspace ID and name, the notebook name, the default Lakehouse, and whether the run comes from a pipeline (isForPipeline).

Mixing these up creates fragile notebooks. An environment run parameter that the caller can set to prod from a development workspace is a risk; deriving environment-specific settings from a Variable Library bound to the workspace is safer. In deployment, item-to-item wiring such as default Lakehouse IDs is a separate concern again, handled by the deployment tool’s parameter file, as described in Microsoft Fabric CI/CD with fabric-cicd.

Tenant and client parameters

Multi-client notebooks are where parameters pay off most, and where mistakes are most expensive.

  • Treat client_id as a filter that must always apply. Apply it when reading, and include it in the write condition, so a bug cannot write one client’s data into another client’s partition.
  • Validate it against a known list when one exists, not just a pattern.
  • Keep client-specific behavior in data, not in code branches. A configuration table keyed by client_id is easier to review than if client_id == ... blocks.
  • Run clients separately when isolation matters more than throughput. One failed client should not roll back the others.

For fan-out, a pipeline ForEach over a list of clients that calls the same notebook is easier to monitor than one notebook looping internally. Each client gets its own activity run, status and duration. If many short notebook runs pay a high session start-up cost, session tags can let compatible runs reuse a Spark session.

Secrets are not parameters

Never pass passwords, keys or connection strings as notebook parameters. Parameter values appear in pipeline definitions, activity inputs and run history, and they end up in Git when the pipeline is committed.

Instead, pass the name of a secret and resolve it inside the notebook:

Python
api_key = notebookutils.credentials.getSecret(
    "https://your-key-vault.vault.azure.net/",
    "orders-api-key",
)
Python4 linesRead-only example cell. Nothing on this page runs code.

The executing identity needs permission on the Key Vault. Notebook output tries to redact a retrieved secret if you print it, but Microsoft describes that as a best-effort safeguard, not a security boundary. Do not print, log or write secret values, and avoid building them into strings that you then log.

Log parameters safely

Logging the parameters of every run is one of the cheapest debugging aids you can add. When a number looks wrong, the first question is always “what was this run asked to do?”.

Python
import json

ctx = notebookutils.runtime.context

run_record = {
    "pipeline_run_id": pipeline_run_id,
    "notebook": ctx["currentNotebookName"],
    "workspace_id": ctx["currentWorkspaceId"],
    "is_pipeline_run": ctx["isForPipeline"],
    "client_id": client_id,
    "run_date": run_date,
    "window_days": window_days,
    "target_table": target_table,
    "dry_run": dry_run,
}
print(json.dumps(run_record, default=str))
Python16 linesRead-only example cell. Nothing on this page runs code.

Log an explicit allow-list of parameters, not globals(), so a secret or personal value never leaks into logs by accident. For durable history, write the same record to an execution log table, as described in Designing an Execution Log for Data Pipelines, and use it alongside pipeline and notebook monitoring.

Returning a value to the pipeline

A notebook can end with notebookutils.notebook.exit(value). The value is always a string. The pipeline can read it from the Notebook activity’s output and use it in later activities, for example to skip a downstream step when no rows were processed. Return a small JSON string with counts or a status, not data.

Two cautions: do not call exit() inside a try/except block, because the exit does not take effect when wrapped in exception handling; and remember that exit() ends the run successfully. To fail the activity, raise an exception.

Reusing one notebook across pipelines

A well-parameterized notebook becomes a reusable building block:

  • One transformation, many inputs. The same notebook processes each client, each source system or each region, driven by parameters.
  • Child notebooks. notebookutils.notebook.run(“child_notebook”, 600, {“client_id”: client_id, “run_date”: run_date}) runs another notebook with arguments and returns its exit value. notebookutils.notebook.runMultiple() can run several notebooks with dependencies.
  • Session configuration. A %%configure cell can be parameterized so a pipeline passes values such as driver cores for a heavy run. A notebook uses the defaults in interactive runs, and scheduled notebook runs do not support parameterized session configuration.

Keep reuse honest: if two callers need materially different logic, two notebooks are clearer than one notebook full of mode flags.

Testing parameter combinations

Parameters multiply the number of ways a notebook can run. Test the combinations that matter:

  • Defaults. An interactive run with defaults should fail validation or do nothing harmful.
  • Typical run. One client, one day.
  • Boundaries. First day of a month, the last day, a leap day, a window that crosses a month or year.
  • Reprocessing. The same window twice; the result must not change.
  • Empty input. A client with no data in the window should succeed and report zero rows, not fail obscurely.
  • Invalid input. Unknown client, malformed date, window_days = 0; each should fail at validation with a clear message.

Keep the validation and window logic in plain Python functions so they can be unit tested without Spark. Run an integration test with a small dataset in a development workspace before merging changes to the notebook. For the Spark side of the same idea, see PySpark DataFrame Unit Testing with pytest. Testing Microsoft Fabric Pipelines and Notebooks turns these combinations into a parameter test matrix and runs them as pipeline tests.

Common mistakes

  • Parameter name mismatch between the activity and the parameters cell, so defaults silently win.
  • Logic inside the parameters cell, which runs before the injected values and then gets overwritten.
  • Reading the current date inside the notebook, which makes reruns impossible.
  • Passing secrets as parameters and finding them in run history and Git.
  • No validation, so a malformed value fails deep inside a Spark job with an unhelpful error.
  • Appending on rerun, which duplicates the window’s data.
  • Unsafe defaults such as environment = "prod" or dry_run = False.
  • Passing a list as a string without parsing it, then iterating over characters.

Recommendation

Design every pipeline notebook as a function: a parameters cell with safe defaults, a validation cell that fails fast, explicit half-open date windows, idempotent writes, and a structured log record of the run. Keep secrets in Key Vault, keep environment settings in Variable Libraries, and keep per-run values in the parameters cell.

That structure makes one notebook safe to call from many pipelines, easy to rerun, and easy to diagnose. It also keeps environment-specific edits out of code, which is what makes Git integration and automated deployment work cleanly.

Tags

  • Microsoft Fabric
  • Fabric Notebook
  • Data Pipelines
  • PySpark