Tuesday, September 4, 2018

Terminate EMR cluster to save costs





AWS EMR is managed hadoop framework available for to be launched and used in a highly secured network in order to implement big data use cases. This EMR cluster will be launched in the AWS account and is priced as it is launched



So in order save costs and optimum usage of EMR clusters in your account, we can implement an automated boto3 script that would terminate the any status[RUNNING, WAITING, STARTING, BOOTSTRAPPING] EMR clusters

Steps to implement:

  1. Create a lambda-> python2.7 script in your account where this automated EMR termination needs to be implemented
  2. Download the code from my github [https://github.com/gvnirmalaa/utilities]
  3. Copy the script into your Lambda code section and click 'Save'
  4. Now go to 'Cloud watch' -> Create Rule -> Cron scheduler -> set the timings as per requirement
  5. In 'Add Target' -> select the newly created Lambda script from Step #3 above
  6. Click on 'Add configured Rule' and now map this rule in your Lambda script via AWS console
  7. Voila, it is done !!!

3 comments:

  1. Great article Nirmala. It’s very crispy and informative. I expect you publish more articles on this

    ReplyDelete
  2. Gone through this article. It's very informative and useful.Please go ahead and put more articles on this. Thank you Nirmala.

    ReplyDelete
  3. Thanks peddi Reddy and you are welcome !!

    ReplyDelete