Creation of AWX execution node failed

Get Help
,
1
  • I am completely new to awx and k8s, trying to add execution node through awx UI installing bundle using ansible-playbook -i inventory.yml install_receptor.yml went well without errors. But in the UI, it’s showing unavailable with error “Instance is not in the receptor mesh.”

-when I check for receptor service, its not running
service receptor.service status
Redirecting to /bin/systemctl status receptor.service
● receptor.service - Ansible Receptor for Linux
Loaded: loaded (/usr/lib/systemd/system/receptor.service; enabled; preset: disabled)
Active: activating (auto-restart) (Result: exit-code) since Mon 2023-11-27 12:40:59 UTC; 46ms ago
Process: 411381 ExecStart=/usr/bin/receptor --config /etc/receptor/receptor.conf (code=exited, status=1/FA>
Main PID: 411381 (code=exited, status=1/FAILURE)
CPU: 14ms

-TCP 27199 is open and listening on the execution node

I am not sure where the logs location will be…What could be the issue

2

Check the logs on your execution node: /var/log/receptor/receptor.log

2 Likes
3

INFO 2023/11/27 07:52:08 Running control service control
WARNING 2023/11/27 07:52:08 Nothing to do - no backends are running.
Run /usr/bin/receptor --help for command line instructions.

This is from receptor log.

4

Check the tls and ssl certs .
Also try running /bin/usr/receptor -c /etc/receptor/receptor.conf.

You will get some error message.

Warm Regards,
Alice Golakiya

2 Likes
5

Thank you @golakiyaalice @kurokobo , Here’s the error

Error: error opening Unix socket: could not acquire lock on socket file: fslock is already locked

netstat -na | grep :27199
tcp6 0 0 :::27199 :::* LISTEN

netstat -tlpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 579297/sshd: /usr/s
tcp6 0 0 :::22 :::* LISTEN 579297/sshd: /usr/s
tcp6 0 0 :::27199 :::* LISTEN 579293/receptor

receptorctl version
ERROR: Missing parameter: socket

what could be the problem?

6

Kill all the process associated with 27199 port and ps aux | grep -i receptor.

Then start the services manually using systemctl start receptor.

If the error says server not in service mesh than it should be definetely related to tls certs.
Make sure the bundle generated for a server is only installed on respective instances.

Warm Regards,
Alice Golakiya

1 Like
7

@golakiyaalice

Still getting the same error after following this

I am still getting “Instance not in the receptor mesh”

How can we resolve tls certs related issue?

8

If this Execution node is in the same network as your AWX cluster, you need to tick the box: “Peers from control nodes” in AWX
You may need to redeploy the bundle, not sure.

Otherwise you may need a hub?

9

@Anitha-1599
Hi, could you try redeploying execution node by following step-by-step instruction?

  1. On execution node, remove incorrect sshd configuration
    1. Make your sshd to stop using 27199
    2. Ensure 27199 is NOT used by any process: netstat -tlpn | grep 27199
  2. On execution node, stop and remove existing Receptor configuration
    1. Stop Receptor: sudo systemctl stop receptor
    2. Remove configuration files including certs: sudo rm -rf /etc/receptor
  3. On execution node, allow inbound TCP traffic sudo firewall-cmd --add-port=27199/tcp --permanent, sudo firewall-cmd --reload
  4. On AWX, remove your execution node
  5. On AWX, add new execution node
    1. Name has to be resolvable by DNS and reachable from AWX
    2. Select execution as node type
    3. Explicitly specify port number: 27199
    4. Check the Peers from control nodes
  6. On AWX, download install bundle for your execution node
  7. On execution node, install receptor using this bundle
  8. On execution node, restart Receptor: sudo systemctl restart receptor

Then check the status on AWX. If it is not in Ready, check following configuration:

  1. On execution node, ensure your configuration file contains tcp-listener: sudo cat /etc/receptor/receptor.conf
  2. On execution node, ensure Receptor is up and running: sudo systemctl status receptor
  3. On execution node, check the logs: sudo tail -n 100 /var/log/receptor/receptor.log
  4. On K8s, ensure the configuration file contains tcp-peer: kubectl -n <namespace> exec -it deployment/<awx-name>-task -c <awx-name>-ee -- cat /etc/receptor/receptor.conf
  5. On K8s, check the logs: kubectl -n <namespace> logs deployment/<awx-name>-task -c <awx-name>-ee --tail=100
3 Likes
10

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.