Delta Lake: Up and Running
by Bennie Haelen & Dan Davis
Not a great book per se, but a decent literature review on how data platforms evolved from lakes through lakehouses.
- Status
- Read · November 2024
- Bought
- November 12, 2024
- For
- Data & platform engineers new to the lakehouse · Architects evaluating Delta vs. a plain object store · Anyone running their first Spark + Delta pipeline
Where it earned its place
The one-paragraph verdict
Not a great book in itself, but a useful one for a specific need: if you want the evolution of data platforms — from lambda architecture through data lakes to the lakehouse — laid out in one structured pass rather than scattered across years of blog posts, this is the clearest survey I’ve found. The book does exactly what the title promises beyond that: it walks you from “why data lakes rot” through ACID transactions, time travel, schema enforcement, and the bronze/silver/gold layout, with a companion repo that actually runs. The history-and-motivation framing in the early chapters is the strongest part — it explains why the transaction log exists, not just how to call the API. The honest limitation is scope: it sticks to Delta Lake OSS and explicitly punts on the Databricks-proprietary optimizations, and it leans hard on Spark notebooks, so almost everything is shown through one engine. By the back half it stays introductory where I wanted depth — concurrency semantics, file compaction trade-offs, and the log internals get a wave rather than a chapter.
Who should read it
Read it if you are standing up your first Delta pipeline or deciding whether the format earns its place over raw Parquet on object storage. It is a good shared starting point for a team. Skip it if you already operate a lakehouse in production or want a deep dive on the transaction-log internals — you will outrun it in a weekend.
Where it earned its place
The medallion-layering and schema-enforcement ideas here shaped how I staged the embedding pipeline: raw documents land untouched, a validated middle layer enforces shape before anything is embedded, and only the curated layer feeds the vector store. Treating the boundary between layers as a contract — not a convenience — came straight from this book.
Skip it if…
You want engine-agnostic depth or hard production guidance on compaction, vacuum, and concurrent writers. This is an on-ramp, not a reference; once you’re moving, it has little left to teach you. [Delta Lake: Up and Running on Goodreads]