How to automatically update nginx upstream server list when aws ec2 hostname changes or increases?

Solution 1:

This can be achieved by using Amazon SDK ( I am almost done with it, will put it on github ), utilizing the SNS, EC2 and Autoscaling service.

I have followed the below steps to achieve this:

1. Enable HTTP notification and subscribed my webserver.
2. Added a lifecycle-hook with heartbeat of 1 min (to wait for 1 min before terminating) to my autoscaling group for terminating server
3. Created an index file to parse the message to detect what kind of message it is ( i.e Launch or Terminate)
4. Once type of event is decided i queried EC2 to get the private ip of the instance
5. In case of Launch wait till header 200 is recieved and then add the ip to nginx config and reload
6. In case of Terminate remove the IP from config and reload nginx

Please find the script here https://github.com/singhupendra/aws-autoscale

Solution 2:

Thank you @talonx, I've done some research, Amazon Autoscale has an api to query current autoscaling group status, and enumerates its members. It returns instance id (http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/api_requests.html#query-example), then you can use the describe tools to get the server name (http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-DescribeInstances.html) and finally recreate the upstream include file. I could sense the Autoscaling notifications to launch a process that performs these tasks.

I still didn't implement it but its a way to go.

One can also use Autocaling with SNS http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/ASGettingNotifications.html