How should I set my DATABASE_URL?

This is a simple matter of logic. You can't set the "default" key of the DATABASES dictionary before you have defined the dictionary itself.

Whether or not you set the default parameter to dj_database_url inside the call or as a separate DATABASE_URL variable is irrelevant, especially as that won't even be used on Heroku as it will be overridden by environment variables.

This is documented on Heroku Devecenter

# Parse database configuration from $DATABASE_URL
import dj_database_url
# DATABASES['default'] =  dj_database_url.config()
#updated
DATABASES = {'default': dj_database_url.config(default='postgres://user:pass@localhost/dbname')}

If you need Database connection pooling add this bits too. More details

# Enable Connection Pooling
DATABASES['default']['ENGINE'] = 'django_postgrespool'