Priya Raghunathan
Data Platform Lead
Priya has migrated three petabyte-scale warehouses without a maintenance window and has the scar tissue to explain why you probably should take one.
- DRM-free PDF and EPUB, yours to keep
- Instant download the moment you check out
- Free updates for the life of the edition
Data Engineering on GCP
Dataflow, BigQuery, and Pipelines That Stay Correct
By Priya Raghunathan and Elena Varga
Google CloudPractitionerData
Google Cloud has the most coherent streaming model of the major clouds, inherited from Dataflow's Beam lineage. It is also the one most often used incorrectly, because event time and watermarks are genuinely hard.
This book takes windowing, watermarks, and late data seriously from chapter one, then builds practical batch and streaming pipelines on Dataflow and BigQuery that remain correct when data arrives out of order — which it will.
What you'll learn
- Reason correctly about event time, watermarks, and late data
- Choose windowing strategies that match the business question
- Build Dataflow pipelines that stay correct under out-of-order arrival
- Model BigQuery schemas for both cost and query performance
Table of contents
- 01Event Time Is Not Processing Time32 pp
- 02Windows and Watermarks44 pp
- 03Late Data Strategies38 pp
- 04Dataflow in Practice42 pp
- 05BigQuery Schema Design40 pp
- 06Batch and Streaming Together36 pp
6 chapters · 368 pages total
Details
- ISBN
- 978-1-959321-15-1
- Edition
- 1st
- Pages
- 368
- Published
- Formats
- PDF, EPUB
- Platform
- Google Cloud
About the authors
Elena Varga
ML Infrastructure Engineer
Elena builds the training and serving substrate under production recommendation systems. She is interested in what happens to models after the notebook closes.