How does one autoscale web dynos on Heroku?
I built HireFire and would like to share some up-to-date information:
HireFire autoscales both your web- and worker dynos using our dyno managers. We currently support the following metric sources:
- HireFire (Job Queue) | Worker Dynos
- Heroku Logplex (Response Time) | Web Dynos
- Heroku Logplex (Connect Time) | Web Dynos
- Heroku Logplex (Queue Time) | Web Dynos
- Heroku Logplex (Requests Per Minute) | Web Dynos
- Heroku Logplex (CPU Load) | Web/Worker Dynos
- NewRelic (Apdex) | Web Dynos
- NewRelic (Response Time) | Web Dynos
- NewRelic (Requests Per Minute) | Web Dynos
HireFire (Job Queue)
Autoscales your worker-based dyno based on the queue size of your jobs. Integration for Ruby and Python applications can be done easily using a 1st or 3rd party library. Any other language and/or framework can be integrated easily without a library as well.
You're able to configure any number of dyno managers for a given application at no extra cost, meaning that you're not limited to a single "worker" entry in your Procfile. This (optionally) allows you to schedule work more efficiently by having for example one Procfile entry per queue and having HireFire scale each individual queue independently.
Heroku Logplex
The Logplex (Logdrain) strategy allows HireFire to consume your logs in order to parse Heroku-emitted metric data which we then use to autoscale. Metrics include Response Time, Connect Time and Load. In addition to that we support Queue Time which can easily be added by installing our library. Or, you can write the minimal amount of code yourself to push the necessary data to the logdrain.
This abstract approach (excluding Queue Time) requires no code changes and works with any language/framework. Just setup a Logdrain via de Heroku CLI and you're set.
For metric aggregation, you can choose between average and (any) percentile.
New Relic
We integrate with New Relic. If you're already using it you can hook it up to HireFire and use their metrics (apdex, response time and rpm) to autoscale your web dynos.
If you have any questions, don't hesitate to get in touch!
The reason heroku don't do this natively is that it's an incredibly complex problem to solve.
For instance, imagine your scenario above, you suddenly start seeing a queue forming and want to ramp the dynos. You crank on ten more. However, it's not a dyno problem, your database is running slow, so now you've got more dynos all sat waiting for the database which now has even more demand placed on it.
Whilst there are auto-scaling products out there, I've not tried any of them, and fully believe that at the moment only a human can make the correct call on scaling. Your mileage may vary.
I have found in the past that setting the resources to an expected usage level (which may be above the current usage) tends to work best, excluding massive traffic influx (such as being on Hacker News etc)