Monday, October 22, 2018

Jenkins Dynamic Slaves setup - Quick Tip #2

Jenkins Dynamic Slaves setup - Quick Tip #2






While setting up Amazon EC2 plugin configuration of dynamic slaves, i recently came across quite some errors/issues and would like to share it here based on the error messages rather clubbing resolutions provided, altogether in one single thread post

So, now lets move on to the Tip of the day


Friday, October 19, 2018

Jenkins Dynamic Slaves Setup - Quick tip #1

Jenkins Dynamic Slaves Setup - Quick tip #1




Jenkins, as a CI server, allows AWS EC2 instances to be launched as Slave nodes for jobs that are labelled to run on Slaves

Primarily, the Jenkins jobs which needs certain framework or apps and custom setup in order to execute the job steps/actions, needs customized instances with the expected setup. This is made possible with Jenkins - Amazon EC2 plugin


Tuesday, October 16, 2018

Airflow with MySQL raising error on libraries


Airflow with MySQL as back end will raise error on libmysqlclient.so.1020 and let us see Why and How to resolve this errors with Apache Airflow




Apache Airflow is an opensource tool for orchestrating complex workflows and data processing pipelines. Workflow in Airflow is designed as a directed acylic graph(DAGs) meaning the tasks in a workflow can be split and executed independently and later be logically merged in a task


Friday, October 12, 2018

AWS decode authorization message

AWS - How-to decode authorization error messages


Today I came across an interesting issue in AWS

I was mapping Jenkins to launch EC2 slaves dynamically[yes finally, i did it :-)]

The Launch configuration was giving some errors unreadable and was really confused about whats wrong and where



Finally identified, that the AWS Authorization messages thrown out on any request is bound to generate an encoded message, which is really confusing !!!



The encode message thus generated as part of EC2 access permissions error will have to deciphered by decoding the same using the AWS CLI utility. Yessss !!! AWS has it all !!!