In Kubernetes, how do I autoscale based on the size of a queue?

You could use this tool https://github.com/XciD/k8s-rmq-autoscaler

It will create a pod on your cluster that will watch your deployments and scale it accordingly to their configuration.

You then just need to add some annotation into your deployments and it will be watched by the autoscaler

kubectl annotate deployment/your-deployment -n namespace \
    k8s-rmq-autoscaler/enable=true \ 
    k8s-rmq-autoscaler/max-workers=20 \ 
    k8s-rmq-autoscaler/min-workers=4 \ 
    k8s-rmq-autoscaler/queue=worker-queue \ 
    k8s-rmq-autoscaler/vhost=vhost

You can use KEDA.

KEDA is a Kubernetes-based Event Driven Autoscaling component. It provides event driven scale for any container running in Kubernetes

It supports RabbitMQ out of the box. You can follow a tutorial which explains how to set up a simple autoscaling based on RabbitMQ queue size.

The top-level solution to this is quite straightforward:

Set up a separate container that is connected to your queue, and uses the Kubernetes API to scale the deployments.

There exist some solutions to this problem already, but they do however not look like they are actively maintained and production ready, but might help:

• https://github.com/mbogus/kube-amqp-autoscale
• https://github.com/mbogus/docker-kube-amqp-autoscale
• https://github.com/onfido/k8s-rabbit-pod-autoscaler