Microsoft Fabric
Create Your First Fabric Notebook
Create a Microsoft Fabric notebook, attach it to a Lakehouse, build a small dataset with PySpark, write it as a Delta table and query it with Spark SQL, with every step explained.
En este artículo
A Lakehouse with no tables is just structure. In this tutorial you put the first data into it: you create a notebook, attach it to lh_learning, build a tiny sales dataset with PySpark, write it as a Delta table called bronze_sales, and query it with Spark SQL.
The dataset is created in code, so you do not need to download or upload anything. The point is not the five rows. It is to see each step clearly: what a Spark session is, what a DataFrame is, what “write as Delta” actually produces, and how the same table becomes visible to SQL.
This tutorial continues from Create Your First Fabric Lakehouse. You need the jb-fabric-dev workspace and the lh_learning Lakehouse from there.
What a Fabric notebook is
A Fabric notebook is an interactive document made of cells. Code cells run against a Spark session, Markdown cells explain what the code does, and results appear under each cell as tables, charts or text. Notebooks are items in a workspace: they are saved automatically, can be shared, versioned in Git, scheduled, and run from pipelines.
A few terms you will meet immediately:
- Spark session. The connection between your notebook and a Spark cluster. It starts when you run the first cell and stays alive while you work, so later cells reuse it. In code, the session is the
sparkobject, created for you. - PySpark. The Python API for Spark, and the default language of a Fabric notebook. You write Python; Spark distributes the work.
- Spark SQL. SQL that runs on the same Spark engine against the same tables. You can switch a single cell to SQL with the
%%sqlmagic. - Attached Lakehouse. A notebook can be attached to one or more Lakehouses. One of them is the default Lakehouse: table names without a prefix resolve to it, and relative paths such as
Files/landingpoint to it.
Why notebooks are useful
- Fast feedback. Run a cell, look at the result, adjust. Exploring data this way is much faster than editing and redeploying a job.
- Code and explanation together. Markdown cells document the reasoning next to the code that implements it.
- Real engineering power. PySpark handles parsing, joins, deduplication and
MERGElogic that visual tools struggle with, and scales from a few rows to billions. - A path to production. The same notebook can later be parameterized, run from a pipeline and promoted through environments. Microsoft Fabric Notebook Parameters covers that step.
Create the notebook and attach the Lakehouse
-
Open the
jb-fabric-devworkspace and openlh_learning. -
From the Lakehouse, choose the option to open a new notebook.
Creating the notebook from the Lakehouse attaches
lh_learningautomatically as its default Lakehouse. You see it in the notebook’s explorer pane, with its Tables and Files. -
Rename the notebook to
nb_first_sales. -
Confirm that the notebook language is PySpark (Python), the default.
If you create the notebook from the workspace instead, select or attach lh_learning from the notebook’s explorer pane. If several Lakehouses are attached, the pinned one is the default. Restart the session after changing the default Lakehouse.
Step 1: create a DataFrame
Paste this into the first cell and run it with Shift+Enter:
from datetime import date
from decimal import Decimal
from pyspark.sql.types import (
DateType,
DecimalType,
IntegerType,
StringType,
StructField,
StructType,
)
schema = StructType([
StructField("order_id", IntegerType(), nullable=False),
StructField("customer", StringType(), nullable=False),
StructField("amount", DecimalType(10, 2), nullable=False),
StructField("order_date", DateType(), nullable=False),
])
rows = [
(1001, "Contoso", Decimal("250.00"), date(2026, 10, 1)),
(1002, "Fabrikam", Decimal("99.90"), date(2026, 10, 1)),
(1003, "Contoso", Decimal("40.10"), date(2026, 10, 2)),
(1004, "Northwind", Decimal("310.75"), date(2026, 10, 2)),
(1005, "Fabrikam", Decimal("12.50"), date(2026, 10, 3)),
]
orders_df = spark.createDataFrame(rows, schema)What this does:
- The schema is explicit. Each column has a name, a type and a nullability rule.
amountis aDECIMAL(10,2), not a floating-point number, because money should not suffer rounding errors. rowsis plain Python. A list of tuples in the driver’s memory, the same as in any Python script.spark.createDataFrame()turns that list into a Spark DataFrame: a distributed, typed table that Spark can process in parallel.
The cell finishes almost immediately and prints nothing. Spark is lazy: it records what you asked for and only does the work when an action, such as displaying or writing, needs a result.
Step 2: display it
display(orders_df)
display() renders an interactive table under the cell, where you can sort columns and switch to a chart view. It is Fabric’s notebook tool for looking at data. For a plain-text result, orders_df.show() prints the first rows instead.
You should see five orders, from 1001 to 1005.
Step 3: inspect the schema
orders_df.printSchema()
print(f"Rows: {orders_df.count()}")
Output:
root
|-- order_id: integer (nullable = false)
|-- customer: string (nullable = false)
|-- amount: decimal(10,2) (nullable = false)
|-- order_date: date (nullable = false)
Rows: 5
Checking the schema is a habit worth building now. When data comes from files instead of code, wrong types are the most common source of quiet bugs: an amount read as a string, a date read as text, an ID that lost its leading zeros. Note that count() is an action: it makes Spark run the work and return a number.
Step 4: a simple transformation
Bronze data should stay close to the source, so the transformation you persist is small: add load metadata that answers “when and from where did this row arrive?”.
from pyspark.sql import functions as F
bronze_df = (
orders_df
.withColumn("source_system", F.lit("tutorial_inline"))
.withColumn("ingested_at", F.current_timestamp())
)
display(bronze_df)withColumn()returns a new DataFrame with one more column. DataFrames are immutable:orders_dfis unchanged.F.lit()adds a constant value.F.current_timestamp()adds the time of the run.
Now a transformation you only look at, without saving it: daily totals.
daily_df = (
orders_df
.groupBy("order_date")
.agg(
F.count("*").alias("orders"),
F.sum("amount").alias("revenue"),
)
.orderBy("order_date")
)
display(daily_df)You should see three days, with revenue of 349.90, 350.85 and 12.50. This is the kind of logic that will later build gold_daily_sales. Keeping it out of Bronze is deliberate: Bronze records what arrived, and business logic belongs in later layers.
Step 5: write it as a Delta table
(
bronze_df.write
.format("delta")
.mode("overwrite")
.saveAsTable("bronze_sales")
)What happens when this runs:
format("delta")writes Delta Lake. It is already the default for tables in Fabric; writing it explicitly makes the intent obvious to the next reader.mode("overwrite")replaces the table if it exists, so you can run the notebook again and get the same result. That makes sense for a tutorial. A real Bronze load usually appends new batches instead, so history is never lost.saveAsTable("bronze_sales")writes the data and registers the table. Because the name has no schema prefix, it goes to thedboschema of the default Lakehouse. The full name islh_learning.dbo.bronze_sales. To write to another schema, use a prefix such assaveAsTable("sales.bronze_orders").
Now look at the Lakehouse in the Explorer pane. Refresh Tables → dbo and bronze_sales appears. Select it to preview the rows.
Step 6: query it with Spark SQL
Add a new cell, put the %%sql magic on its first line, and run:
SELECT *
FROM bronze_salesThe same five rows come back, this time from the table rather than from the DataFrame in memory. The %%sql magic switches only this cell to Spark SQL; the rest of the notebook stays PySpark.
SQL is also convenient for quick aggregations:
SELECT order_date,
COUNT(*) AS orders,
SUM(amount) AS revenue
FROM bronze_sales
GROUP BY order_date
ORDER BY order_dateYou can mix the two languages from Python as well. spark.sql() returns a DataFrame you can keep working with:
large_orders_df = spark.sql("""
SELECT order_id, customer, amount
FROM bronze_sales
WHERE amount >= 100
""")
display(large_orders_df)What happened
- Python / PySparkList of tuples in the driver
- Spark DataFrameTyped, distributed, lazy
- Delta filesParquet plus _delta_log
- Lakehouse tablelh_learning.dbo.bronze_sales
- Python held five tuples in memory.
createDataFrame()turned them into a Spark DataFrame with an explicit schema. Nothing was stored yet.saveAsTable()started a Spark job that wrote Parquet files and a Delta transaction log into the table’s folder in OneLake, under the Lakehouse’sTables/dboarea.- The Lakehouse registered the table, so Spark SQL can query it by name.
- The SQL analytics endpoint picks up the new table through its metadata sync, so T-SQL users can query it too, with no copy.
The important shift: the DataFrame disappears when the session ends; the table does not. Close the notebook, come back tomorrow, and SELECT * FROM bronze_sales still works. You can confirm it from the SQL analytics endpoint of lh_learning:
SELECT TOP (10) *
FROM dbo.bronze_sales
ORDER BY order_id;PySpark vs SQL inside a notebook
You do not have to choose one language per notebook. Use each where it reads best.
| Task | Convenient choice | Why |
|---|---|---|
| Creating data, reading files, calling APIs | PySpark | Python handles paths, options, loops and libraries |
| Exploring a table quickly | Spark SQL | SELECT … GROUP BY is short and familiar |
| Multi-step transformations | PySpark | Each step is a named DataFrame you can test and reuse |
| Joins and aggregations | Either | SQL for readability, PySpark when logic is built dynamically |
| Parameterized or reusable logic | PySpark | Functions, parameters and conditionals |
| Sharing a query with SQL users | Spark SQL | The logic transfers to the SQL analytics endpoint with few changes |
A good rule: PySpark for the pipeline, SQL for the questions. Load and transform data in PySpark, and use %%sql cells to check the results the way a SQL user would. The PySpark select and filter cheat sheet and the Delta read and write cheat sheet are useful next to this notebook.
Stop the session and commit the notebook
When you finish, stop the Spark session from the notebook. An idle session expires on its own after a timeout, but until then it holds capacity that other work could use.
Then, if your workspace is connected to Git:
- Open the workspace’s source control panel.
nb_first_salesis listed as a new change. - Commit it with the message
Add first sales notebook.
The repository now has a folder for nb_first_sales under fabric/, containing the notebook’s source code. The bronze_sales table is not in Git; it lives in OneLake. Rerunning the notebook in another workspace rebuilds it, which is exactly why code belongs in Git and data does not.
Common mistakes
- No default Lakehouse. Without one,
saveAsTable("bronze_sales")has nowhere sensible to go. Always check the Explorer pane before writing. - Forgetting that Spark is lazy. A cell that “ran instantly” may not have done the work yet; errors can appear later, at the first action.
- Displaying or collecting large data.
display()andcollect()bring results to the notebook. Filter or aggregate first, and avoidcollect()on large DataFrames. - Relying on inferred types.
inferSchemais convenient for exploration, but production loads should declare schemas, as you did here. - Using
overwritefor real Bronze loads. It is right for a tutorial and wrong for history. Real ingestion appends or merges. - Leaving sessions running. Idle sessions consume capacity until they time out.
- Hard-coding secrets or another workspace’s paths in cells. Both end up in Git and break when the notebook moves between environments.
What you learned
- A Fabric notebook runs PySpark, Spark SQL and other languages against a Spark session, with a default Lakehouse for table names and relative paths.
- You created
nb_first_salesfromlh_learning, so the Lakehouse was attached as the default. - You built a DataFrame with an explicit schema, displayed it, inspected its schema and added load metadata.
saveAsTable("bronze_sales")wrote a Delta table into thedboschema of the default Lakehouse, as Parquet files plus a transaction log in OneLake.%%sqlcells andspark.sql()query the same table with SQL, and the SQL analytics endpoint sees it without a copy.- PySpark suits loading and multi-step transformations; SQL suits quick questions and logic shared with SQL users.
Next: Explore Data with PySpark in Microsoft Fabric
You now have a workspace, a Git connection, a Lakehouse and a Delta table written from code. The next step is the everyday DataFrame toolkit: select, filter, groupBy, agg, orderBy and joins, used to explore and shape real sales data: Explore Data with PySpark in Microsoft Fabric. After that, step 5 explains what the Delta table you just wrote actually is (files, log, versions, and what the SQL endpoint can do with it): Delta Tables in Microsoft Fabric for SQL Server Engineers.
Previous step: Create Your First Fabric Lakehouse. When notebooks start to feel slow, read How to Make Microsoft Fabric Notebooks Faster.
¿Te resultó útil este contenido?
¿Qué podríamos mejorar?
Gracias — esto me ayuda a mejorar Javi Builds.
Sigue aprendiendo
- Guía rápidaPatrones de lectura y escritura en Delta LakeLee y escribe tablas Delta en Microsoft Fabric de forma segura con append, overwrite, MERGE, particionamiento, control del esquema, patrones preparados para la concurrencia y mantenimiento de tablas.Abrir guía rápida
- ArtículoContenido en inglésDelta Tables in Microsoft Fabric for SQL Server EngineersWhat a Delta table really is, how it differs from a SQL Server table, and what to understand about writes, keys, time travel, maintenance and the SQL analytics endpoint before using Fabric Lakehouse tables in production.Leer artículo
¿Qué debería crear después?
Estoy construyendo Javi Builds alrededor de problemas reales que enfrentan los ingenieros de datos. Dime qué tema te gustaría ver después.
Mantente al día
Recibe solo las novedades de Javi Builds que realmente te interesan.
Revisa tu bandeja de entrada
Te he enviado un enlace de confirmación. Tu suscripción se activará cuando la confirmes.