# DB Migrations Failing on new install

**URL:** <https://forum.ansible.com/t/db-migrations-failing-on-new-install/28615>\
**Category:** AWX Project\
**Tags:** awx\
**Created:** [September 20, 2018, 6:29pm UTC](https://forum.ansible.com/t/db-migrations-failing-on-new-install/28615 "2018-09-20T18:29:31Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alan\_Fraser](https://avatars.discourse-cdn.com/v4/letter/a/59ef9b/32.png) [@Alan\_Fraser](https://forum.ansible.com/u/Alan_Fraser)\
**Post date:** [September 20, 2018, 6:29pm UTC](https://forum.ansible.com/t/db-migrations-failing-on-new-install/28615/1 "2018-09-20T18:29:31Z")

</div>

I’m getting errors running AWX on Openshift.

Openshift version : 3.7  
AWX 1.0.8.0

Setup runs through correctly but there are errors in the migration. Postgres is running 9.6 on the openshift environment also with a clean install i.e. no existing database.

**awx\_web logs:**

\*\*\* uWSGI is running in multiple interpreter mode \*\*\*  
spawned uWSGI master process (pid: 137)  
spawned uWSGI worker 1 (pid: 139, cores: 1)  
spawned uWSGI worker 2 (pid: 140, cores: 1)  
spawned uWSGI worker 3 (pid: 141, cores: 1)  
spawned uWSGI worker 4 (pid: 142, cores: 1)  
spawned uWSGI worker 5 (pid: 143, cores: 1)  
READY  
2018-09-20 18:20:03,007 INFO success: awx-config-watcher entered RUNNING state, process has stayed up for \> than 1 seconds (startsecs)  
2018-09-20 18:20:03,007 INFO success: daphne entered RUNNING state, process has stayed up for \> than 1 seconds (startsecs)  
2018-09-20 18:20:03,007 INFO success: nginx entered RUNNING state, process has stayed up for \> than 1 seconds (startsecs)  
2018-09-20 18:20:03,007 INFO success: uwsgi entered RUNNING state, process has stayed up for \> than 1 seconds (startsecs)  
2018-09-20 18:20:06,963 WARNING awx.conf.settings Database settings are not available, using defaults, error:  
Traceback (most recent call last):  
File “/usr/lib/python2.7/site-packages/awx/wsgi.py”, line 80, in   
application = get\_wsgi\_application()  
File “/usr/lib/python2.7/site-packages/awx/wsgi.py”, line 77, in get\_wsgi\_application  
return AWXWSGIHandler()  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/handlers/wsgi.py”, line 151, in **init**  
self.load\_middleware()  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/handlers/base.py”, line 56, in load\_middleware  
mw\_class = import\_string(middleware\_path)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/utils/module\_loading.py”, line 20, in import\_string  
module = import\_module(module\_path)  
File “/usr/lib64/python2.7/importlib/ **init**.py”, line 37, in import\_module  
**import** (name)  
File “/usr/lib/python2.7/site-packages/awx/sso/middleware.py”, line 18, in   
from social\_django import utils  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/social\_django/utils.py”, line 13, in   
BACKENDS = settings.AUTHENTICATION\_BACKENDS  
File “/usr/lib/python2.7/site-packages/awx/conf/settings.py”, line 531, in **getattr\_without\_cache**  
return getattr(self.\_wrapped, name)  
File “/usr/lib/python2.7/site-packages/awx/conf/settings.py”, line 444, in **getattr**  
value = self.\_get\_local(name)  
File “/usr/lib/python2.7/site-packages/awx/conf/settings.py”, line 361, in \_get\_local  
self.\_preload\_cache()  
File “/usr/lib/python2.7/site-packages/awx/conf/settings.py”, line 323, in \_preload\_cache  
for setting in Setting.objects.filter(key\_\_in=settings\_to\_cache.keys(), user\_\_isnull=True).order\_by(‘pk’):  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/query.py”, line 250, in **iter**  
self.\_fetch\_all()  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/query.py”, line 1118, in \_fetch\_all  
self.\_result\_cache = list(self.\_iterable\_class(self))  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/query.py”, line 53, in **iter**  
results = compiler.execute\_sql(chunked\_fetch=self.chunked\_fetch)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/sql/compiler.py”, line 899, in execute\_sql  
raise original\_exception  
ProgrammingError: relation “conf\_setting” does not exist  
LINE 1: …f\_setting".“value”, “conf\_setting”.“user\_id” FROM "conf\_sett…

**Same error on awx\_celery**

Using /etc/ansible/ansible.cfg as config file  
127.0.0.1 | SUCCESS =\> {  
“changed”: false,  
“db”: “awxtest”  
}  
Traceback (most recent call last):  
File “/usr/bin/awx-manage”, line 9, in   
load\_entry\_point(‘awx==1.0.8.0’, ‘console\_scripts’, ‘awx-manage’)()  
File “/usr/lib/python2.7/site-packages/awx/ **init**.py”, line 116, in manage  
execute\_from\_command\_line(sys.argv)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/ **init**.py”, line 364, in execute\_from\_command\_line  
utility.execute()  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/ **init**.py”, line 356, in execute  
self.fetch\_command(subcommand).run\_from\_argv(self.argv)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/base.py”, line 283, in run\_from\_argv  
self.execute(\*args, \*\*cmd\_options)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/base.py”, line 330, in execute  
output = self.handle(\*args, \*\*options)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/commands/shell.py”, line 101, in handle  
exec(sys.stdin.read())  
File “”, line 1, in   
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/contrib/auth/models.py”, line 170, in create\_superuser  
return self.\_create\_user(username, email, password, \*\*extra\_fields)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/contrib/auth/models.py”, line 153, in \_create\_user  
user.save(using=self.\_db)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/contrib/auth/base\_user.py”, line 80, in save  
super(AbstractBaseUser, self).save(\*args, \*\*kwargs)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/base.py”, line 808, in save  
force\_update=force\_update, update\_fields=update\_fields)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/base.py”, line 838, in save\_base  
updated = self.\_save\_table(raw, cls, force\_insert, force\_update, using, update\_fields)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/base.py”, line 924, in \_save\_table  
result = self.\_do\_insert(cls.\_base\_manager, using, fields, update\_pk, raw)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/base.py”, line 963, in \_do\_insert  
using=using, raw=raw)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/manager.py”, line 85, in manager\_method  
return getattr(self.get\_queryset(), name)(\*args, \*\*kwargs)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/query.py”, line 1076, in \_insert  
return query.get\_compiler(using=using).execute\_sql(return\_id)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/sql/compiler.py”, line 1112, in execute\_sql  
cursor.execute(sql, params)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/backends/utils.py”, line 64, in execute  
return self.cursor.execute(sql, params)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/utils.py”, line 94, in **exit**  
six.reraise(dj\_exc\_type, dj\_exc\_value, traceback)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/backends/utils.py”, line 64, in execute  
return self.cursor.execute(sql, params)  
django.db.utils.IntegrityError: null value in column “last\_login” violates not-null constraint  
DETAIL: Failing row contains (1, pbkdf2\_sha256$36000$OD7sY8fSCxn3$NuGzqPNkK7cTL+ZLhlUZ3UiPtp6zpd+…, null, t, admin, , , root@localhost, t, t, 2018-09-20 18:19:58.843266+00).  
2018-09-20 18:20:01,399 WARNING awx.conf.settings Database settings are not available, using defaults, error:  
Traceback (most recent call last):  
File “/usr/bin/awx-manage”, line 9, in   
load\_entry\_point(‘awx==1.0.8.0’, ‘console\_scripts’, ‘awx-manage’)()  
File “/usr/lib/python2.7/site-packages/awx/ **init**.py”, line 116, in manage  
execute\_from\_command\_line(sys.argv)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/ **init**.py”, line 364, in execute\_from\_command\_line  
utility.execute()  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/ **init**.py”, line 356, in execute  
self.fetch\_command(subcommand).run\_from\_argv(self.argv)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/base.py”, line 283, in run\_from\_argv  
self.execute(\*args, \*\*cmd\_options)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/base.py”, line 327, in execute  
self.check()  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/base.py”, line 359, in check  
include\_deployment\_checks=include\_deployment\_checks,  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/base.py”, line 346, in \_run\_checks  
return checks.run\_checks(\*\*kwargs)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/checks/registry.py”, line 81, in run\_checks  
new\_errors = check(app\_configs=app\_configs)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/checks/urls.py”, line 16, in check\_url\_config  
return check\_resolver(resolver)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/checks/urls.py”, line 26, in check\_resolver  
return check\_method()  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/urls/resolvers.py”, line 254, in check  
for pattern in self.url\_patterns:  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/utils/functional.py”, line 35, in **get**  
res = instance. **dict** [self.name] = self.func(instance)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/urls/resolvers.py”, line 405, in url\_patterns  
patterns = getattr(self.urlconf\_module, “urlpatterns”, self.urlconf\_module)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/utils/functional.py”, line 35, in **get**  
res = instance. **dict** [self.name] = self.func(instance)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/urls/resolvers.py”, line 398, in urlconf\_module  
return import\_module(self.urlconf\_name)  
File “/usr/lib64/python2.7/importlib/ **init**.py”, line 37, in import\_module  
**import** (name)  
File “/usr/lib/python2.7/site-packages/awx/urls.py”, line 17, in   
url(r’^sso/', include(‘social\_django.urls’, namespace=‘social’)),  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/conf/urls/ **init**.py”, line 50, in include  
urlconf\_module = import\_module(urlconf\_module)  
File “/usr/lib64/python2.7/importlib/ **init**.py”, line 37, in import\_module  
**import** (name)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/social\_django/urls.py”, line 6, in   
from . import views  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/social\_django/views.py”, line 10, in   
from .utils import psa  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/social\_django/utils.py”, line 13, in   
BACKENDS = settings.AUTHENTICATION\_BACKENDS  
File “/usr/lib/python2.7/site-packages/awx/conf/settings.py”, line 531, in **getattr\_without\_cache**  
return getattr(self.\_wrapped, name)  
File “/usr/lib/python2.7/site-packages/awx/conf/settings.py”, line 444, in **getattr**  
value = self.\_get\_local(name)  
File “/usr/lib/python2.7/site-packages/awx/conf/settings.py”, line 361, in \_get\_local  
self.\_preload\_cache()  
File “/usr/lib/python2.7/site-packages/awx/conf/settings.py”, line 323, in \_preload\_cache  
for setting in Setting.objects.filter(key\_\_in=settings\_to\_cache.keys(), user\_\_isnull=True).order\_by(‘pk’):  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/query.py”, line 250, in **iter**  
self.\_fetch\_all()  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/query.py”, line 1118, in \_fetch\_all  
self.\_result\_cache = list(self.\_iterable\_class(self))  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/query.py”, line 53, in **iter**  
results = compiler.execute\_sql(chunked\_fetch=self.chunked\_fetch)  
File “/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/db/models/sql/compiler.py”, line 899, in execute\_sql  
raise original\_exception  
ProgrammingError: relation “conf\_setting” does not exist  
LINE 1: …f\_setting".“value”, “conf\_setting”.“user\_id” FROM "conf\_sett…  
^

All of this results in jobs not running in the pending state.

Any ideas?

---

<div class="post-metadata">

**Author:** ![Alan\_Fraser](https://avatars.discourse-cdn.com/v4/letter/a/59ef9b/32.png) [@Alan\_Fraser](https://forum.ansible.com/u/Alan_Fraser)\
**Post date:** [September 21, 2018, 11:56am UTC](https://forum.ansible.com/t/db-migrations-failing-on-new-install/28615/2 "2018-09-21T11:56:14Z")

</div>

So I’ve figured out what is going and also fixed the issue. The kubernetes role mounts the management pod and the awx pod at the same time. The management pod is still performing a database migration when the awx pod starts up. As the scripts have not completed on the management pod, so the awx pod starts but the logs are full of python errors. I’ve modified the tasks/main.yml to delay the awx deployment until after the management pod has been deleted and all works successfully.

I’ll post details of the change to the file shortly and raise a pull request if agreed it’s the right way to go.

---

<div class="post-metadata">

**Author:** ![Alan\_Fraser](https://avatars.discourse-cdn.com/v4/letter/a/59ef9b/32.png) [@Alan\_Fraser](https://forum.ansible.com/u/Alan_Fraser)\
**Post date:** [September 29, 2018, 2:17am UTC](https://forum.ansible.com/t/db-migrations-failing-on-new-install/28615/3 "2018-09-29T02:17:39Z")

</div>

I’ve opened an issue [https://github.com/ansible/awx/issues/2345](https://github.com/ansible/awx/issues/2345) and pull request for this issue [https://github.com/ansible/awx/pull/2346](https://github.com/ansible/awx/pull/2346)