Remove default apps from Django-admin

In an admin.py you know will definitely be loaded, try:

admin.site.unregister(User)
admin.site.unregister(Group)
admin.site.unregister(Site)

In addition to the above double check your ordering of "INSTALLED_APPS" in "settings.py"

INSTALLED_APPS = [
    # django apps first
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',

    # custom apps below
    'my_app'
]

Otherwise it will cause an error. See here: Issue with Django admin registering an inline user profile admin