Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

For the love of god, don't use NiFi to trigger an Airflow DAG.


Can you expand? We just set this workflow up and it seems to be working fine.


NiFi is meant for stream processing and Airflow for batch processing, if your NiFi triggers an Airflow DAG that means that your entire process is batch processing and you shouldn't use NiFi in the first place. If you still want to do stream processing then use Airflow sensors to "trigger" it.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: