Configure celery to wait for backend service to start

celery use sqlalchemy behind the scene, it does not ship with connect retry feature out of the box, however, you could adjust the connect timeout, to wait longer for mysql server, by default this value is only 10s, larger value helps.

assuming you are using pymysql/mysqldb as DB driver, it accepts a connect_timeout option, to specify this option from celery, you need set database_engine_options, which will be passed to the create_engine function of sqlalchemy, and set connect_args, which will be passed directly from sqlalchemy to DB driver, eg:

app.conf.database_engine_options = {'connect_args': {'connect_timeout': 600}}

another option is to use a custom connection creator function, manage the connection creation all by yourself, you could retry whatever times you want.

Tags:

Python

Celery