No such file or directory: '/etc/receptor/receptor.conf'

Get Help
1

Hi there,

every time an awx-task container starts, we see (I think it´s always 2 instances of) this stack trace:

[pod/awx-task-657cb7cf77-8mmk8/awx-task] [wait-for-migrations] Waiting for database migrations...
[pod/awx-task-657cb7cf77-8mmk8/awx-task] [wait-for-migrations] Attempt 1
[pod/awx-task-657cb7cf77-8mmk8/awx-task] min_value in DecimalField should be Decimal type.
[pod/awx-task-657cb7cf77-8mmk8/awx-task] Instance Group already registered controlplane
[pod/awx-task-657cb7cf77-8mmk8/awx-task] Instance Group already registered default
[pod/awx-task-657cb7cf77-8mmk8/awx-task] Successfully registered instance None
[pod/awx-task-657cb7cf77-8mmk8/awx-task] (changed: True)
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:10,181 INFO RPC interface 'supervisor' initialized
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:10,181 INFO RPC interface 'supervisor' initialized
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:10,181 CRIT Server 'unix_http_server' running without any HTTP authentication checking
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:10,181 CRIT Server 'unix_http_server' running without any HTTP authentication checking
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:10,181 INFO supervisord started with pid 7
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:10,181 INFO supervisord started with pid 7
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:11,184 INFO spawned: 'superwatcher' with pid 37
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:11,184 INFO spawned: 'superwatcher' with pid 37
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:11,186 INFO spawned: 'dispatcher' with pid 38
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:11,186 INFO spawned: 'dispatcher' with pid 38
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:11,188 INFO spawned: 'callback-receiver' with pid 39
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:11,188 INFO spawned: 'callback-receiver' with pid 39
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:11,191 INFO spawned: 'wsrelay' with pid 40
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:11,191 INFO spawned: 'wsrelay' with pid 40
[pod/awx-task-657cb7cf77-8mmk8/awx-task] READY
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:12,194 INFO success: superwatcher entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:12,194 INFO success: superwatcher entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:12,195 INFO success: wsrelay entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:12,195 INFO success: wsrelay entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:20,668 INFO     [-] awx.main.commands.run_callback_receiver Callback receiver started with pid=39
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:20,865 INFO     [-] awx.main.dispatch Running worker dispatcher listening to queues ['tower_broadcast_all', 'tower_settings_change', 'awx-task-657cb7cf77-8mmk8']
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:20,910 ERROR    [-] awx.main.dispatch Encountered unhandled error in dispatcher main loop
[pod/awx-task-657cb7cf77-8mmk8/awx-task] Traceback (most recent call last):
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/dispatch/worker/base.py", line 237, in run
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     self.worker.on_start()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/dispatch/worker/task.py", line 141, in on_start
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     dispatch_startup()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/system.py", line 84, in dispatch_startup
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     write_receptor_config()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/receptor.py", line 837, in write_receptor_config
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     receptor_config, should_update = generate_config_data()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]                                      ^^^^^^^^^^^^^^^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/receptor.py", line 801, in generate_config_data
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     should_update = should_update_config(receptor_config)
[pod/awx-task-657cb7cf77-8mmk8/awx-task]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/receptor.py", line 770, in should_update_config
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     current_config = read_receptor_config()  # this gets receptor conf lock
[pod/awx-task-657cb7cf77-8mmk8/awx-task]                      ^^^^^^^^^^^^^^^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/receptor.py", line 122, in read_receptor_config
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     with open(__RECEPTOR_CONF, 'r') as f:
[pod/awx-task-657cb7cf77-8mmk8/awx-task]          ^^^^^^^^^^^^^^^^^^^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task] FileNotFoundError: [Errno 2] No such file or directory: '/etc/receptor/receptor.conf'
[pod/awx-task-657cb7cf77-8mmk8/awx-task] Traceback (most recent call last):
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/usr/bin/awx-manage", line 8, in <module>
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     sys.exit(manage())
[pod/awx-task-657cb7cf77-8mmk8/awx-task]              ^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/__init__.py", line 177, in manage
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     execute_from_command_line(sys.argv)
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     utility.execute()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     self.fetch_command(subcommand).run_from_argv(self.argv)
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/core/management/base.py", line 412, in run_from_argv
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     self.execute(*args, **cmd_options)
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/core/management/base.py", line 458, in execute
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     output = self.handle(*args, **options)
[pod/awx-task-657cb7cf77-8mmk8/awx-task]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/management/commands/run_dispatcher.py", line 71, in handle
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     consumer.run()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/dispatch/worker/base.py", line 237, in run
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     self.worker.on_start()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/dispatch/worker/task.py", line 141, in on_start
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     dispatch_startup()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/system.py", line 84, in dispatch_startup
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     write_receptor_config()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/receptor.py", line 837, in write_receptor_config
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     receptor_config, should_update = generate_config_data()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]                                      ^^^^^^^^^^^^^^^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/receptor.py", line 801, in generate_config_data
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     should_update = should_update_config(receptor_config)
[pod/awx-task-657cb7cf77-8mmk8/awx-task]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/receptor.py", line 770, in should_update_config
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     current_config = read_receptor_config()  # this gets receptor conf lock
[pod/awx-task-657cb7cf77-8mmk8/awx-task]                      ^^^^^^^^^^^^^^^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/receptor.py", line 122, in read_receptor_config
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     with open(__RECEPTOR_CONF, 'r') as f:
[pod/awx-task-657cb7cf77-8mmk8/awx-task]          ^^^^^^^^^^^^^^^^^^^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task] FileNotFoundError: [Errno 2] No such file or directory: '/etc/receptor/receptor.conf'
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:21,093 INFO     [-] awx.main.wsrelay Active instance with hostname awx-task-657cb7cf77-8mmk8 is registered.
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:21,095 INFO     [-] awx.main.wsrelay Starting Websocket Relayer...
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:21,836 WARN exited: dispatcher (exit status 1; not expected)
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:21,836 WARN exited: dispatcher (exit status 1; not expected)
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:22,839 INFO spawned: 'dispatcher' with pid 72
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:22,839 INFO spawned: 'dispatcher' with pid 72
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:25,793 INFO     [-] awx.main.dispatch Running worker dispatcher listening to queues ['tower_broadcast_all', 'tower_settings_change', 'awx-task-657cb7cf77-8mmk8']
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:25,831 ERROR    [-] awx.main.dispatch Encountered unhandled error in dispatcher main loop
[pod/awx-task-657cb7cf77-8mmk8/awx-task] Traceback (most recent call last):
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/dispatch/worker/base.py", line 237, in run
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     self.worker.on_start()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/dispatch/worker/task.py", line 141, in on_start
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     dispatch_startup()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/system.py", line 84, in dispatch_startup
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     write_receptor_config()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/receptor.py", line 837, in write_receptor_config
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     receptor_config, should_update = generate_config_data()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]                                      ^^^^^^^^^^^^^^^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/receptor.py", line 801, in generate_config_data
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     should_update = should_update_config(receptor_config)
[pod/awx-task-657cb7cf77-8mmk8/awx-task]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/receptor.py", line 770, in should_update_config
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     current_config = read_receptor_config()  # this gets receptor conf lock
[pod/awx-task-657cb7cf77-8mmk8/awx-task]                      ^^^^^^^^^^^^^^^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/receptor.py", line 122, in read_receptor_config
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     with open(__RECEPTOR_CONF, 'r') as f:
[pod/awx-task-657cb7cf77-8mmk8/awx-task]          ^^^^^^^^^^^^^^^^^^^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task] FileNotFoundError: [Errno 2] No such file or directory: '/etc/receptor/receptor.conf'
[pod/awx-task-657cb7cf77-8mmk8/awx-task] Traceback (most recent call last):
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/usr/bin/awx-manage", line 8, in <module>
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     sys.exit(manage())
[pod/awx-task-657cb7cf77-8mmk8/awx-task]              ^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/__init__.py", line 177, in manage
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     execute_from_command_line(sys.argv)
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     utility.execute()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     self.fetch_command(subcommand).run_from_argv(self.argv)
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/core/management/base.py", line 412, in run_from_argv
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     self.execute(*args, **cmd_options)
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/core/management/base.py", line 458, in execute
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     output = self.handle(*args, **options)
[pod/awx-task-657cb7cf77-8mmk8/awx-task]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/management/commands/run_dispatcher.py", line 71, in handle
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     consumer.run()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/dispatch/worker/base.py", line 237, in run
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     self.worker.on_start()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/dispatch/worker/task.py", line 141, in on_start
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     dispatch_startup()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/system.py", line 84, in dispatch_startup
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     write_receptor_config()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/receptor.py", line 837, in write_receptor_config
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     receptor_config, should_update = generate_config_data()
[pod/awx-task-657cb7cf77-8mmk8/awx-task]                                      ^^^^^^^^^^^^^^^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/receptor.py", line 801, in generate_config_data
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     should_update = should_update_config(receptor_config)
[pod/awx-task-657cb7cf77-8mmk8/awx-task]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/receptor.py", line 770, in should_update_config
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     current_config = read_receptor_config()  # this gets receptor conf lock
[pod/awx-task-657cb7cf77-8mmk8/awx-task]                      ^^^^^^^^^^^^^^^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task]   File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/tasks/receptor.py", line 122, in read_receptor_config
[pod/awx-task-657cb7cf77-8mmk8/awx-task]     with open(__RECEPTOR_CONF, 'r') as f:
[pod/awx-task-657cb7cf77-8mmk8/awx-task]          ^^^^^^^^^^^^^^^^^^^^^^^^^^
[pod/awx-task-657cb7cf77-8mmk8/awx-task] FileNotFoundError: [Errno 2] No such file or directory: '/etc/receptor/receptor.conf'
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:26,607 WARN exited: dispatcher (exit status 1; not expected)
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:26,607 WARN exited: dispatcher (exit status 1; not expected)
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:28,611 INFO spawned: 'dispatcher' with pid 87
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:28,611 INFO spawned: 'dispatcher' with pid 87
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:31,528 INFO     [-] awx.main.dispatch Running worker dispatcher listening to queues ['tower_broadcast_all', 'tower_settings_change', 'awx-task-657cb7cf77-8mmk8']
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:32,392 WARNING  [-] awx.main.tasks.system Rejoining the cluster as instance awx-task-657cb7cf77-8mmk8. Prior last_seen None
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:32,419 INFO     [-] awx.main.dispatch Dispatcher listener connection established
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:32,419 WARNING  [-] awx.main.dispatch.periodic Scheduler next run unexpectedly over 20 seconds in future: 25.108773708343506
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:41,108 INFO     [-] awx.main.wsrelay Adding {'awx-web-585669c88d-w5pjq', 'awx-web-585669c88d-5sgvt'} to websocket broadcast list
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:41,122 INFO     [-] awx.main.wsrelay Connection from awx-task-657cb7cf77-8mmk8 to <some_ip> established.
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:41,122 INFO     [-] awx.main.wsrelay Starting producer for metrics
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:41,123 INFO     [-] awx.main.wsrelay Connection from awx-task-657cb7cf77-8mmk8 to <some_ip> established.
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:41,123 INFO     [-] awx.main.wsrelay Starting producer for metrics
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:42,124 INFO success: callback-receiver entered RUNNING state, process has stayed up for > than 30 seconds (startsecs)
[pod/awx-task-657cb7cf77-8mmk8/awx-task] 2024-06-26 05:26:42,124 INFO success: callback-receiver entered RUNNING state, process has stayed up for > than 30 seconds (startsecs)

To me it seems like, there might be some kind of race condition between the receptor.conf appearing in the expected location and the (at first failing) attempts to read it.

Is it possible to add some logic in order to handle the transient absence of the file more graceful in order to prevent this from showing up in logs?

Thank you for your work!

2

We´ve seen this and a similar issue for some time now. Both might be solvable by adding file existence checks (?)

3

Hi @2and3makes23,
Thank you so much for your time! To best assist you, we would like to gather a bit more information. May we ask which version of AWX you are seeing this in? Have you tried upgrading, and if so, does this still occur?

We appreciate your feedback!

1 Like
4

Thank you!
We have seen this for quite some time now - at least for versions 24.0.0 upwards. We are currently running AWX 24.5.0

5

Here, receptor does not start. I have opened this issue: Networking BROKE. "Socket path does not exist: /var/run/awx-receptor/receptor.sock" · Issue #15418 · ansible/awx · GitHub

Version 24.6.1 on Rocky 9.4

Basically:

docker exec -it  tools_awx_1    bash
bash-5.1# receptor --config /etc/receptor/receptor.conf
empty receptor config, skipping...
bash-5.1# cat /etc/receptor/receptor.conf
---
- node:
    id: awx-1
    firewallrules:
      - action: "reject"
        tonode: awx-1
        toservice: "control"

- log-level: info

- tcp-listener:
    port: 2222




#- tls-server:
#    name: mutual-tls
#    cert: /etc/receptor/certs/awx.crt
#    key: /etc/receptor/certs/awx.key
#    requireclientcert: true
#    clientcas: /etc/receptor/certs/ca.crt

- control-service:
    service: control
    filename: /var/run/awx-receptor/receptor.sock

- work-command:
    worktype: local
    command: ansible-runner
    params: worker
    allowruntimeparams: true
    verifysignature: False

- work-kubernetes:
    worktype: kubernetes-runtime-auth
    authmethod: runtime
    allowruntimeauth: true
    allowruntimepod: true
    allowruntimeparams: true
    verifysignature: False

- work-kubernetes:
    worktype: kubernetes-incluster-auth
    authmethod: incluster
    allowruntimeauth: true
    allowruntimepod: true
    allowruntimeparams: true
    verifysignature: False
bash-5.1#
6

Update: We have been running 24.6.1 for a while now and the above mentioned FileNotFoundError is still a thing, when awx-task instances come up.