Airflow with Google BigQuery and Slack
5 min readDec 10, 2018
--
Airflow is an open source tool for creating, scheduling, and monitoring data processing pipelines. Airflow can be installed via conda install -c conda-forge airflow
or pip install airflow
. Before running airflow, we need to initiate the database airflow initdb
.
Then we can start the airflow webserver, which a python flask app providing the UI of airflow. The default port of the webserver is 8080: airflow webserver -p 8080
. When we open 0.0.0.0:8080 in the browser, we can see a bunch of airflow examples/DAGs.