Airflow with Google BigQuery and Slack

Sophia Yang, Ph.D.
5 min readDec 10, 2018

Airflow is an open source tool for creating, scheduling, and monitoring data processing pipelines. Airflow can be installed via conda install -c conda-forge airflow or pip install airflow. Before running airflow, we need to initiate the database airflow initdb.

Then we can start the airflow webserver, which a python flask app providing the UI of airflow. The default port of the webserver is 8080: airflow webserver -p 8080. When we open in the browser, we can see a bunch of airflow examples/DAGs.