Failed to import the required Python library

[lioradler (Lior)]

Hello,

I have a problem using the s3_object module. As a requirement, boto3 has to be installed. I’ve been trying to achieve this with the following tasks:

- name: 'Install pip'
  package:
    name: "pip"
  vars:
    ansible_python_interpreter: "/usr/bin/python3"
  become: true
 
- name: 'Install requirements'
  pip:
    name: "boto3"
  vars:
    ansible_python_interpreter: "/usr/bin/python3"

- ansible.builtin.set_fact:
    ENDPOINT_URL: 'https://this.should.work'
    BACKUP_SCRIPT: 'backup.sh'
  
- ansible.builtin.set_fact:
    BACKUP_SCRIPT_BUCKET: 'my_bucket_name'
    no_log: false

- name: Get backup script from S3 storage
  amazon.aws.s3_object:
    access_key: "{{ aws_access_key_id }} # this exists
    secret_key: "{{ aws_secret_access_key }} # this too
    endpoint_url: "{{ ENDPOINT_URL }}"
    bucket: "{{ BACKUP_SCRIPT_BUCKET }}"
    object: "{{ BACKUP_SCRIPT }}"
    mode: "get"
    dest: "/root"
  vars:
    ansible_python_interpreter: "/usr/bin/python3"

Unfortunately, I get:

“Failed to import the required Python library (botocore and boto3) on ANSIBLE_HOST’s Python /usr/bin/python3. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter”

As you can see, I tried to set the right python interpreter on the relevant tasks, but it doesn’t seem to do much. It still can’t find boto3 (and botocore, which is installed as a dependency of boto3).

Python version: 3.10
Ansible version: 2.12

Any hints are highly appreciated!

[jrglynn2]

Can you link to a pastebin with the full results of the playbook? Run with -vvv or even -vvvv? I think one of those should show you the interpreter being used on the remote host, to triple-confirm the python path is correct.

Have you conformed manually that boto3 & botocore were actually installed to the python used by the user that ansible runs under, on the remote host?

[lioradler (Lior)]

hi @jrglynn2
Here’s the verbose log: https://pastebin.com/03xsi76w

Have you conformed manually that boto3 & botocore were actually installed to the python used by the user that ansible runs under, on the remote host?

boto3 & botocore were installed using the task I run prior to amazon.aws.s3_object. I didn’t use become: true so both tasks should run under the same user.

[jrglynn2]

Maybe try the suggestion posted here ==> Ansible cannot find boto3/botocore python modules even if they are installed · Issue #41776 · ansible/ansible · GitHub <== ?

Install both boto3 & botocore, regardless of the dependency, and with the --user argument? So:

- name:           Ensure botocore and boto3 modules are installed
  pip:
    name:         [ "boto3", "botocore"]
    extra_args:   "--user"
  vars:
    ansible_python_interpreter: "/usr/bin/python3"

I think it has to do with the Python being used by whatever user Ansible is connecting with…

[lioradler (Lior)]

Hello again @jrglynn2

I tried it, unfortunately it yields the same error message

My YAML code:

- name: 'Install pip'
  package:
    name: "pip"
  vars:
    ansible_python_interpreter: "/usr/bin/python3"
  become: true

- name: 'Install requirements'
  pip:
    name: "{{ requirement }}"
    extra_args: "--user"
  vars:
    ansible_python_interpreter: "/usr/bin/python3"
  loop:
    - "botocore"
    - "boto3"
  loop_control:
    loop_var: requirement

- ansible.builtin.set_fact:
    ENDPOINT_URL: 'https://this.should.work'
    BACKUP_SCRIPT: 'backup_v2.sh'

- ansible.builtin.set_fact:
    BACKUP_SCRIPT_BUCKET: 'my_bucket_name'
    no_log: false

- name: Get backup script from S3 storage
  amazon.aws.s3_object:
    access_key: "{{ aws_access_key_id }}"
    secret_key: "{{ aws_secret_access_key }}"
    endpoint_url: "{{ ENDPOINT_URL }}"
    bucket: "{{ BACKUP_SCRIPT_BUCKET }}"
    object: "{{ BACKUP_SCRIPT }}"
    mode: "get"
    dest: "/root"
  vars:
    ansible_python_interpreter: "/usr/bin/python3"

[lioradler (Lior)]

@jrglynn2 turned out you were right!

The last task was under a block (which I didn’t want to copy entirely because it was mostely irrelevant to the issue) which is executed with become: true. I am so ashamed it took me so long to realize that!

Case closed and thanks for your time!