Authentication Error to run ansible-playbook

[bacchus21]

Hello, All

I’m trying to run one of my playbooks toward Cisco IOS devices. Manual login to the device with the user password ssh -l user1 OOOO-XXX-SW03 is working fine. But not sure why the same password is not getting through when I run playbook.

[root@ansible002 ansible]# ansible-playbook plb-password.yml -i inv_for_testing.yml -u user1 -k --check
SSH password:

PLAY [plb-password.yml] **************************************************

TASK [Check enable secret is configured] ***********************************************************
fatal: [OOOO-XXX-SW03]: FAILED! => {"changed": false, "msg": "Failed to authenticate: Authentication failed: transport shut down or saw EOF"}
...ignoring

PLAY RECAP *****************************************************************************************
OOOO-XXX-SW03 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=1

[root@ansible002 ansible]#

Does anyone have clue or similar experience? Any comments/feedback are welcomed. Thank you in advance.

Regards

[ptn (Pierre TOURON)]

Hi,

First off, could you share your playbook content ? Also perhaps try to run your playbook in verbose (-vvv) or debug (ANSIBLE_DEBUG=1 ansible-playbook...) mode.

On a side note, be aware most network devices-related modules (including Cisco IOS ones) uses network_cli as connection plugin; have a look here.

[bacchus21]

Hello, @ptn

Thank you very much for looking at my question. Here’s my playbook and inventory info. And, I’m already using netwrk_cli. If you have any further advice, that will be really appreciated.

Regards

inv_for_testing.yml
//////////////////////////////////

---

all:
  children:

    # Cisco IOS Devices

    ios:
      vars:
        ansible_network_os: cisco.ios.ios
        ansible_connection: ansible.netcommon.network_cli
        ansible_network_cli_ssh_type: libssh'

      hosts:
		OOOO-XXX-SW03:
		
    # Cisco NXOS Devices

    nxos:
      vars:
        ansible_network_os: cisco.nxos.nxos
        ansible_connection: ansible.netcommon.network_cli
        ansible_network_cli_ssh_type: libssh
      hosts:
        OOOO-XXX-SW04:

...

plb-password.yml
///////////////////////////

---

- name: "IOS - Password Rules"
  hosts: ios
  collections:
    - cisco.ios
  gather_facts: false
  ignore_errors: yes
  tasks:
    - name: "Check enable secret is configured"
      cisco.ios.ios_command:
        commands:
          - show running-config | include enable secret
        wait_for:
          - result[0] contains 'enable secret'

...

[bacchus21]

Hello, @ptn

I made some progress on this issue, but still don’t understand some point. Therefore, highly appreciated if you can take a look at this issue one more time.

@jbericat Could you please take a look at this issue and let me know if you have any clue?

---

all:
  children:

    # Cisco IOS Devices

    ios:
      vars:
        ansible_network_os: cisco.ios.ios
        ansible_connection: ansible.netcommon.network_cli
        ansible_network_cli_ssh_type: libssh
		ansible_libssh_user: user1
		ansible_libssh_pass: **********

      hosts:
		OOOO-XXX-SW03:
		
    # Cisco NXOS Devices

    nxos:
      vars:
        ansible_network_os: cisco.nxos.nxos
        ansible_connection: ansible.netcommon.network_cli
        ansible_network_cli_ssh_type: libssh
		ansible_libssh_user: user1
		ansible_libssh_pass: **********
		
      hosts:
        OOOO-XXX-SW04:

...

My playbook started to work after adding the following two.
ansible_libssh_user: user1
ansible_libssh_pass: **********

[root@ansible002 inventories]# ansible-playbook plb-password.yml -i inv_for_testing.yml

PLAY [plb-password.yml] **************************************************

TASK [Check enable secret is configured] ****************************************************
ok: [OOOO-XXX-SW03]

PLAY RECAP *****************************************************************************************
OOOO-XXX-SW03                : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

[root@ansible002 inventories]#

However, If I disable the following two in the inventory

ansible_libssh_user: user1
ansible_libssh_pass: **********

and run the playbook with -u user1 -k option, then authentication doesn’t go well even if I put the proper password for user1, which I can’t understand at the moment.

[root@ansible002 inventories]# ansible-playbook plb-password.yml -i inv_for_testing.yml -u user1 -k --check

SSH password:

PLAY [plb-password.yml] **************************************************

TASK [Check enable secret is configured] ****************************************************
fatal: [OOOO-XXX-SW03]: FAILED! => {"changed": false, "module_stderr": "ssh connection failed: Failed to find any acceptable way to authenticate", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error"}
...ignoring

PLAY RECAP *****************************************************************************************
OOOO-XXX-SW03                : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=1

[root@ansible002 inventories]#

[jbericat (Jordi Bericat Ruz)]

Hello @bacchus21 glad to see you around here again. Actually I’m having quite a rough week, but I don’t mind having a look into it later today, maybe tomorrow. Anyway, What @ptn suggested looks good to me. What’s the problem with having the credentials embedded on the inventory? If what worries you is to have plain passwords at sight, you can solve this easily using the ansible-vault command in order to encrypt a string, so you can paste the encrypted password on the inventory. Afterwards, you’ll only have to add the vault password to the ansible-playbook command using the --vault-id option (there are several ways to do this though, refer to the ansible-vault docs for examples on how to implement this).

That said, I’ll try to find a spot to dig further into this if you still need it (just wanted to let you know that I saw your mention here on the forum)

[ptn (Pierre TOURON)]

Thanks for the additional info !

I fail to spot an error in your Ansible configuration. Perhaps a bad interaction between libssh and network_cli ?

It looks like you are using a password to authenticate (not a passphrase for a key, as password parameter for ansible.netcommon.libssh connection plugin seems to be used for both); I would suggest you try authenticate with a private key instead if possible, and if not, try setting look_for_keys parameter to ‘false’, see if it helps. Parameter is not documented as of now, so I’m not sure of its use.

As for your test where you pass -u and -k parameters on CLI, do you have the same result while not running in check mode ? Just a dumb thought.

Could you run your playbook in verbose (with at least three ‘v’, so we’ll see connection events as well) and share the output ?

Lastly, @jbericat suggestion about using ansible-vault to encrypt your password in your vars is a good one (and might be your best workaround right now), though you should consider using keys to authenticate instead IMO.

[bacchus21]

Hello, @jbericat

Thank you very much for your quick reply. I hope you that things will go fine with you.
Yes, actually, it’s not good to show passwords in plain text, so let me study a bit more about ansible-vault.

However, still don’t understand Option1 works and Option2 doesn’t work.

Option1
Inventory file with

ansible_libssh_user: user1 
ansible_libssh_pass: **********

ansible-playbook plb-password.yml -i inv_for_testing.yml : WORK

Option2
Inventory file without

ansible_libssh_user: user1 
ansible_libssh_pass: **********

ansible-playbook plb-password.yml -i inv_for_testing.yml -u user1 -k --check : NOT WORK even if putting the same password for user1.

Regards

[bacchus21]

Hello, @ptn

Thank you very much for keep looking at this issue and your comments.

Unfortunately, out network devices only accept TACACS password authentication. so, that’s why I disabled key authentication as you can see below and as you suggested.

$ ansible-config dump --only-changed -t all
CONFIG_FILE() = /etc/ansible/ansible.cfg
DEFAULT_FORKS(/etc/ansible/ansible.cfg) = 10
DEFAULT_TIMEOUT(/etc/ansible/ansible.cfg) = 5
HOST_KEY_CHECKING(/etc/ansible/ansible.cfg) = False
INVENTORY_ENABLED(/etc/ansible/ansible.cfg) = ['yaml']
PARAMIKO_LOOK_FOR_KEYS(/etc/ansible/ansible.cfg) = False

CONNECTION:
==========

paramiko_ssh:
____________
host_key_checking(/etc/ansible/ansible.cfg) = False
look_for_keys(/etc/ansible/ansible.cfg) = False
timeout(/etc/ansible/ansible.cfg) = 5

ssh:
___
host_key_checking(/etc/ansible/ansible.cfg) = False
timeout(/etc/ansible/ansible.cfg) = 5

If I run my playbook with -vvv option, it shows the below output.
ansible-playbook plb-password.yml -i inv_for_testing.yml -u user1 -k --check -vvv

fatal: [OOOO-XXX-SW03]: FAILED! => {
    "changed": false,
    "module_stderr": "ssh connection failed: Failed to find any acceptable way to authenticate",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error"
}
...ignoring

It shows "ssh connection failed: Failed to find any acceptable way to authenticate" even if I put my username/password in CLI. Any clue? or advice?

In the meantime, let me try to use ansible-vault as well. Thank you.

Regards

[jbericat (Jordi Bericat Ruz)]

@bacchus21

Thank you very much for your quick reply. I hope you that things will go fine with you.

Thanks my friend I’ll let you know once I’m back in shape

[jbericat (Jordi Bericat Ruz)]

Hello @bacchus21, just following-up the case here. Do you still need further assistance with this?

@bacchus21

Option1
Inventory file with

ansible_libssh_user: user1 
ansible_libssh_pass: **********

ansible-playbook plb-password.yml -i inv_for_testing.yml : WORK

Option2
Inventory file without

ansible_libssh_user: user1 
ansible_libssh_pass: **********

ansible-playbook plb-password.yml -i inv_for_testing.yml -u user1 -k --check : NOT WORK even if putting the same password for user1.

Actually I’m with @ptn on this one, I fail to see why the heck you cannot use interactive ssh passwords from CLI (aynway, that is something I only use for debugging purposes, since it breaks the meaning of automation itself lol).

just out of curiosity; What ansible-core version are you running? As per the 6.0.0 version docs, the module was tested on >=2.14.0. You may also try to update the collection to 6.0.0, which was released 2 days ago…

PS: You always come here with the wierdest stuff, huh? Just joking, I like a good challenge every now and then

[bacchus21]

Hello, @jbericat

Sorry for my late response. I was sick for last few days and thank you very much for continuously looking at this issue.

I still don’t understand why Option1 works and Option2(interactive ssh passwords from CLI ) doesn’t work.

I tried to update to cisco-ios-6.0.0.tar.gz but it didn’t go well due to come dependency issues. Is there any sequence steps to install collections. For example, last time, you recommended me installing “ansible.utils” first. However, this time, it didn’t go well even if I tried the same way by installing ansible-utils-3.0.0.tar.gz before cisco-ios-6.0.0.tar.gz.

P.S. I know quite mysterious issues are happening to me. I’m also curious why

Regards

[jbericat (Jordi Bericat Ruz)]

Hello @bacchus21

I sincerely hope you’re feeling better now!

@bacchus21

I tried to update to cisco-ios-6.0.0.tar.gz but it didn’t go well due to come dependency issues. Is there any sequence steps to install collections. For example, last time, you recommended me installing “ansible.utils” first. However, this time, it didn’t go well even if I tried the same way by installing ansible-utils-3.0.0.tar.gz before cisco-ios-6.0.0.tar.gz.

Given the recurrent issues you’ve encountered, it seems probable that they all stem from a common root cause. I distinctly recall mentioning that resolving the SSL problem preventing direct Galaxy collection installations could prevent these recurring issues:

Any way to install Paramiko manually

Hello, All While I was running my playbook, I keep getting Paramiko error, so I’m currently trying to install Paramiko. Looks like some SSL connection related issues are happening. Any one have any clues? Or any way to install Paramiko manually? [root@ansible002 ~]# pip install paramiko WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1129)'))': …

To delve deeper into this matter, I’d recommend taking a step back to investigate the origin of the persistent SSL error you’re encountering.

To facilitate this investigation, it’s crucial to gather a comprehensive overview of your environment. This entails:

• What OS/Distribution version are you currently using? Is it a physical machine, or are you operating within a VM? If it’s the latter, which platform are you using (VMware ESXi, RH KVM, etc.)? I encountered similar SSL errors when running AWX within an ESXi minikube VM. Perhaps creating a fresh installation on a VM sandbox solely for debugging purposes could be beneficial.

• I recall that you’re running Ansible from the CLI. Have you established a Python virtual environment, or are you utilizing a system-wide installation? I strongly advocate for using Python venvs. Consider creating a new one (using Python 3.9, ansible-core 2.15 latest…).

• Are you operating on a VLAN-segmented network? Are there any firewalls between your control node and the Internet (such as ufw, iptables, ACL rules on a router, or a Zscaler client if you’re utilizing WSL on Windows)?

These suggestions aim to gather all pertinent information about your environment, but it’s not exahustive, you may add more info than this. Really, having a comprehensive understanding will significantly aid in identifying a viable solution for this challenge. Your cooperation in gathering this information would be immensely helpful for me or any other community member assisting you.

PS: Now I’m on a vacation leave so I may not get back to you in a timely manner, but I’ll be checking my phone every now and then.

Cheers

[bacchus21]

@jbericat

Sorry for my late reply. Also, Thank you very much for keep looking at my questions and your continuous support.

Regarding the SSL issues, looks like it’s related with firewalls. Currently, we 're using VMs in azure.

Regarding a Python virtual environment, sure, let me follow up your recommendation. I’ll study a bit more and try to install the virtual environment.

Regarding network related matters, there are no issues. I’m a network engineer

Let me try to dig a bit more and share the status.

P.S. I hope you have(had) a wonderful time in your vacation

Thank you and regards

[jbericat (Jordi Bericat Ruz)]

Hello @bacchus21, it’s nice to hear from you.

@bacchus21

Sorry for my late reply. Also, Thank you very much for keep looking at my questions and your continuous support.

No worries, we’re to help

@bacchus21

Regarding the SSL issues, looks like it’s related with firewalls. Currently, we 're using VMs in azure.

Good to hear that too. Let’s see what you find out from your investigation. Looking forward to it!

Cheers