Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> nor an ETL tool

Can you disambiguate that further? I think you mean that an ETL tool would handle the external interface work whereas Nifi is dealing with data that is somewhat inside your control. Is that an OK description?



Often in ETL the transformation part becomes a goal of itself, in Nifi there are tools to transform data, but the Extract and Load parts work best, it has interfaces for quite a lot of different systems.

And maybe it's just me but everything I've seen being called ETL handled data in batches, in which case you really want support for scheduling, error handling and retries, which Nifi does not really have (there is a 'retry' function somewhere, but I found it confusing and it only seems to work for a single data item, not a good idea when you've got thousands of them). I much prefer Airflow for those scenarios.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: