I wholeheartedly agree. When I worked at Shopify, we had to hand-roll our incremental data models using Spark, and the complexity of managing deep DAGs made tasks like backfilling and refactoring a huge pain. Tools like dbt and SQLMesh face similar challenges.
The chaos of existing approaches was a large part of what drove me to join Materialize. With Materialize, you can use dbt on “easy-mode”, while Materialize handles incremental logic, removing the usual headaches around processing time and keeping everything up to date within a second or two.
Congrats on the launch! I work as a data engineer at Shopify designing data models and building data pipelines for our merchant-facing analytic products. I’ve learned a lot about model design from Kimball, but (as you mention) found it vague in some critical areas. I’ve ordered a copy and am excited to give it a read.
