Platform architecture

Data Engineering

Pipelines, ingestion, architecture and production data workflows.

Patterns for ingestion, transformations, orchestration, Medallion design and dependable large-scale processing.

Start here

Curated path

Interview preparation

Data Engineering Interview Questions

Practice data engineering questions on pipelines, incremental loads, CDC, idempotency, Parquet and Delta Lake, data quality, observability and system design.

20 questions available

Practice Data Engineering questions →

Architecture & Medallion design

Pipelines & ingestion

Distributed processing & scale

Cheat sheets

Browse all →
  • Data EngineeringArchitectureIntermediate

    Batch vs Streaming

    Choose between batch and streaming data processing by comparing latency, correctness, operational complexity, cost, replay, state, and failure recovery for real production workloads.

  • Data EngineeringStorage & File FormatsIntermediate

    CSV vs JSON vs Parquet vs Delta

    Choose between CSV, JSON, Parquet, and Delta based on schema, analytics performance, mutability, interoperability, reliability, and production data-lake requirements.

  • Data EngineeringData QualityIntermediate

    Data Quality Checks

    Build practical data quality checks for schema, completeness, uniqueness, validity, referential integrity, freshness, reconciliation, and pipeline drift without relying on arbitrary universal thresholds.

  • Data EngineeringData IntegrationIntermediate

    Incremental Load Patterns

    Design safe incremental loads using watermarks, CDC, high-water marks, MERGE, idempotent writes, late-data handling, and replayable checkpoints without skipping or duplicating data.

  • Data EngineeringArchitectureIntermediate

    Medallion Architecture Quick Reference

    Design Bronze, Silver, and Gold data layers with clear ownership, replayability, quality gates, incremental processing, and serving contracts without turning medallion architecture into rigid folder naming.

  • Data EngineeringStorage & PerformanceIntermediate

    Partitioning Strategies

    Choose and validate data partitioning strategies that improve pruning, parallelism, maintenance, and cost without creating skew, tiny files, or high-cardinality layouts.

Latest content

Browse all →