Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

For the last two years I've been using dbt for some ETL work. It is quite useful, but at the same time quite ugly and clunky. It's basically just a bunch of macros, and it gets unwieldy pretty quickly. It's weird to have to declare the dependencies between the nodes in the DAG when they should be easy to figure out from the code. And dbt is also very restrictive in that it's designed around transformations that are functional so making more procedural transformations is really hard. So I think the world would buy a much better dbt. In other words, I think your focus on analytical queries is probably a good idea.

What makes using something like dbt palatable is that there really aren't any good alternatives in this space. dbt does have some strengths that are worth looking to, such as the extra tooling you get by using their IDE, the generated docs, their ideas about a metrics layer.

I think that for PRQL to be really useful it needs to extend beyond just the syntax and really make writing these data pipelines much easier. The syntax of the SQL isn't the worst part of it. But I think there's a big potential market here.



dbt-prql lets you write prql in your DBT

https://pypi.org/project/dbt-prql/




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: