Monday, December 3, 2018

Apache Airflow - What happens when version used isn't supported anymore ???


Apache Airflow - What happens when version used isn't supported anymore ???





Before, we start looking into the issue and resolution, let us get to know what-is-what

Apache Airflow: 

Airflow is a platform to programmatically author, schedule and monitor workflows.
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on DAGs a snap. The rich user interface makes it easy to visualize pipelines running in production, monitor progress, and troubleshoot issues when needed
For more information refer to the documentation from Apache at https://airflow.apache.org/ 
Ansible:
Ansible is IT automation tool, easy and efficient to do configuration management, deploy software to make Continuous Delivery, an easy journey. For more details, refer to https://docs.ansible.com/ansible/latest/index.html
Now, let us see what has happened
  1. Ansible, IT automation tool - with this, I have created play books to deploy the Apache Airflow software automatically in the destined EC2 instances
  2. Say, I have been deploying, Apache Airflow version as 1.8.0 so far
  3. After some months, this version 1.8.0 was withdrawn or wasn't supported anymore by the Apache Airflow group and was not available for download and setup; this says, that we need to upgrade with the latest stable/Production ready version for Airflow
  4. So below is the snapshot of error that we get to realize

To fix: we need to upgrade to next immediate stable version or the very latest one; this goes with the team preference and the release features the team is expecting with .
An extra tip, to get the list of versions or the latest version and release status of Apache Airflow is to be looked at https://pypi.org/project/apache-airflow/#history


No comments:

Post a Comment