How to automatically update nginx upstream server list when aws ec2 hostname changes or increases?
This can be achieved by using Amazon SDK ( I am almost done with it, will put it on github ), utilizing the SNS, EC2 and Autoscaling service.
I have followed the below steps to achieve this:
- Enable HTTP notification and subscribed my webserver.
- Added a lifecycle-hook with heartbeat of 1 min (to wait for 1 min before terminating) to my autoscaling group for terminating server
- Created an index file to parse the message to detect what kind of message it is ( i.e Launch or Terminate)
- Once type of event is decided i queried EC2 to get the private ip of the instance
- In case of Launch wait till header 200 is recieved and then add the ip to nginx config and reload
- In case of Terminate remove the IP from config and reload nginx
Please find the script here https://github.com/singhupendra/aws-autoscale
Thank you @talonx, I've done some research, Amazon Autoscale has an api to query current autoscaling group status, and enumerates its members. It returns instance id (http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/api_requests.html#query-example), then you can use the describe tools to get the server name (http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-DescribeInstances.html) and finally recreate the upstream include file. I could sense the Autoscaling notifications to launch a process that performs these tasks.
I still didn't implement it but its a way to go.
One can also use Autocaling with SNS http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/ASGettingNotifications.html