apt module is reinstalling the same .deb file on every single run of a playbook

Archives Ansible Project
1

Every single time I run a task that installs a .deb file, the apt module reinstalls the file. Even when the file has not changed since the last run.

Specifically, on the remote machine, I have uploaded a file to /mounts/share/package_staging/duplicati.deb. I copy the file from my local machine to that location, the copy task says the file has not changed. Then the apt task runs, and it reinstalls the package.

  • name: Copy duplicati.deb to <aspects_duplicati_package_staging>/duplicati.deb
    copy:
    src: duplicati.deb
    dest: “{{ aspects_duplicati_package_staging }}/duplicati.deb”
    tags:

  • aspects_duplicati

  • aspects_duplicati_duplicatiPackage

  • aspects_duplicati_duplicatiPackageDebian

  • name: Install duplicati.deb package
    apt:
    state: present
    deb: “{{ aspects_duplicati_package_staging }}/duplicati.deb”
    force: yes
    install_recommends: yes
    tags:

  • aspects_duplicati

  • aspects_duplicati_duplicatiPackage

  • aspects_duplicati_duplicatiPackageDebian

This is the output I receive for those two tasks:

TASK [aspects_duplicati : Copy duplicati.deb to <aspects_duplicati_package_staging>/duplicati.deb] ********************************************************
ok: [hostname]

TASK [aspects_duplicati : Install duplicati.deb package] **************************************************************************************************
(Reading database … 156207 files and directories currently installed.)
Preparing to unpack …/package_staging/duplicati.deb …
duplicati stop/waiting
Unpacking duplicati (2.0.1.73-1) over (2.0.1.73-1) …
Setting up duplicati (2.0.1.73-1) …
duplicati start/running, process 18077
Processing triggers for mime-support (3.54ubuntu1.1) …
changed: [hostname]

I also tried the changing the apt modules “force” option to “no”. Same result.

This is the entire role I wrote and am using: https://github.com/LaneCommunityCollege/aspects_duplicati

My local machine is Ubuntu 16.04 running Ansible 2.4.2.0 installed via Pip. The remote server is Ubuntu 14.04.

Here, just in case there’s something useful, is the -vvvv output of the apt task.

TASK [aspects_duplicati : Install duplicati.deb package] **************************************************************************************************
task path: /home/<sudo_user>/dev/devops/roles/aspects_duplicati/tasks/duplicatiPackageDebian.yml:12
Using module file /usr/local/lib/python3.5/dist-packages/ansible/modules/packaging/os/apt.py
<<remote_ip>> ESTABLISH SSH CONNECTION FOR USER: None
<<remote_ip>> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/<sudo_user>/.ansible/cp/05fb0bbba5 <remote_ip> ‘/bin/sh -c ‘"’“‘echo ~ && sleep 0’”’“‘’
<<remote_ip>> (0, b’/home/<sudo_user>\n’, b’OpenSSH_7.2p2 Ubuntu-4ubuntu2.2, OpenSSL 1.0.2g 1 Mar 2016\r\ndebug1: Reading configuration data /home/<sudo_user>/.ssh/config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 23013\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n’)
<<remote_ip>> ESTABLISH SSH CONNECTION FOR USER: None
<<remote_ip>> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/<sudo_user>/.ansible/cp/05fb0bbba5 <remote_ip> '/bin/sh -c '”‘"’( umask 77 && mkdir -p “echo /home/<sudo_user>/.ansible/tmp/ansible-tmp-1515449702.995029-39243729346755” && echo ansible-tmp-1515449702.995029-39243729346755=“echo /home/<sudo_user>/.ansible/tmp/ansible-tmp-1515449702.995029-39243729346755” ) && sleep 0’“'”‘’
<<remote_ip>> (0, b’ansible-tmp-1515449702.995029-39243729346755=/home/<sudo_user>/.ansible/tmp/ansible-tmp-1515449702.995029-39243729346755\n’, b’OpenSSH_7.2p2 Ubuntu-4ubuntu2.2, OpenSSL 1.0.2g 1 Mar 2016\r\ndebug1: Reading configuration data /home/<sudo_user>/.ssh/config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 23013\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n’)
<<remote_ip>> PUT /tmp/tmp7yi28y52 TO /home/<sudo_user>/.ansible/tmp/ansible-tmp-1515449702.995029-39243729346755/apt.py
<<remote_ip>> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/<sudo_user>/.ansible/cp/05fb0bbba5 ‘[<remote_ip>]’
<<remote_ip>> (0, b’sftp> put /tmp/tmp7yi28y52 /home/<sudo_user>/.ansible/tmp/ansible-tmp-1515449702.995029-39243729346755/apt.py\n’, b’OpenSSH_7.2p2 Ubuntu-4ubuntu2.2, OpenSSL 1.0.2g 1 Mar 2016\r\ndebug1: Reading configuration data /home/<sudo_user>/.ssh/config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 23013\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug2: Remote version: 3\r\ndebug2: Server supports extension “posix-rename@openssh.com” revision 1\r\ndebug2: Server supports extension “statvfs@openssh.com” revision 2\r\ndebug2: Server supports extension “fstatvfs@openssh.com” revision 2\r\ndebug2: Server supports extension “hardlink@openssh.com” revision 1\r\ndebug2: Server supports extension “fsync@openssh.com” revision 1\r\ndebug3: Sent message fd 3 T:16 I:1\r\ndebug3: SSH_FXP_REALPATH . → /home/<sudo_user> size 0\r\ndebug3: Looking up /tmp/tmp7yi28y52\r\ndebug3: Sent message fd 3 T:17 I:2\r\ndebug3: Received stat reply T:101 I:2\r\ndebug1: Couldn't stat remote file: No such file or directory\r\ndebug3: Sent message SSH2_FXP_OPEN I:3 P:/home/<sudo_user>/.ansible/tmp/ansible-tmp-1515449702.995029-39243729346755/apt.py\r\ndebug3: Sent message SSH2_FXP_WRITE I:4 O:0 S:32768\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 4 32768 bytes at 0\r\ndebug3: Sent message SSH2_FXP_WRITE I:5 O:32768 S:32768\r\ndebug3: Sent message SSH2_FXP_WRITE I:6 O:65536 S:30442\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 5 32768 bytes at 32768\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 6 30442 bytes at 65536\r\ndebug3: Sent message SSH2_FXP_CLOSE I:4\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n’)
<<remote_ip>> ESTABLISH SSH CONNECTION FOR USER: None
<<remote_ip>> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/<sudo_user>/.ansible/cp/05fb0bbba5 <remote_ip> ‘/bin/sh -c ‘"’“‘chmod u+x /home/<sudo_user>/.ansible/tmp/ansible-tmp-1515449702.995029-39243729346755/ /home/<sudo_user>/.ansible/tmp/ansible-tmp-1515449702.995029-39243729346755/apt.py && sleep 0’”’“‘’
<<remote_ip>> (0, b’‘, b’OpenSSH_7.2p2 Ubuntu-4ubuntu2.2, OpenSSL 1.0.2g 1 Mar 2016\r\ndebug1: Reading configuration data /home/<sudo_user>/.ssh/config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 23013\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n’)
<<remote_ip>> ESTABLISH SSH CONNECTION FOR USER: None
<<remote_ip>> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/<sudo_user>/.ansible/cp/05fb0bbba5 -tt <remote_ip> '/bin/sh -c '”‘“‘sudo -H -S -p “[sudo via ansible, key=pkqjrfcnrrmhwkzojwscsyhhyqdnflht] password: " -u root /bin/sh -c '”’”’“'”‘"’“'”‘“‘echo BECOME-SUCCESS-pkqjrfcnrrmhwkzojwscsyhhyqdnflht; /usr/bin/python /home/<sudo_user>/.ansible/tmp/ansible-tmp-1515449702.995029-39243729346755/apt.py; rm -rf “/home/<sudo_user>/.ansible/tmp/ansible-tmp-1515449702.995029-39243729346755/” > /dev/null 2>&1’”’“'”‘"’“'”‘"’“’ && sleep 0’”‘"’’
<<remote_ip>> (0, b’\r\n\r\n{“invocation”: {“module_args”: {“dpkg_options”: “force-confdef,force-confold”, “upgrade”: null, “force”: true, “force_apt_get”: false, “install_recommends”: true, “package”: null, “autoclean”: false, “purge”: false, “allow_unauthenticated”: false, “state”: “present”, “autoremove”: false, “update_cache”: null, “default_release”: null, “only_upgrade”: false, “deb”: “/mounts/share/package_staging/duplicati.deb”, “cache_valid_time”: 0}}, “diff”: {“prepared”: “(Reading database … 156207 files and directories currently installed.)\nPreparing to unpack …/package_staging/duplicati.deb …\nduplicati stop/waiting\nUnpacking duplicati (2.0.1.73-1) over (2.0.1.73-1) …\nSetting up duplicati (2.0.1.73-1) …\nduplicati start/running, process 21082\nProcessing triggers for mime-support (3.54ubuntu1.1) …”}, “changed”: true, “stderr”: “\nConfiguration file '/etc/default/duplicati'\n ==> Modified (by you or by a script) since installation.\n Version in package is the same as at last installation.\n ==> Keeping old config file as default.\n”, “stdout”: “(Reading database … 156207 files and directories currently installed.)\nPreparing to unpack …/package_staging/duplicati.deb …\nduplicati stop/waiting\nUnpacking duplicati (2.0.1.73-1) over (2.0.1.73-1) …\nSetting up duplicati (2.0.1.73-1) …\nduplicati start/running, process 21082\nProcessing triggers for mime-support (3.54ubuntu1.1) …\n”}\r\n’, b’OpenSSH_7.2p2 Ubuntu-4ubuntu2.2, OpenSSL 1.0.2g 1 Mar 2016\r\ndebug1: Reading configuration data /home/<sudo_user>/.ssh/config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 23013\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to <remote_ip> closed.\r\n’)
(Reading database … 156207 files and directories currently installed.)
Preparing to unpack …/package_staging/duplicati.deb …
duplicati stop/waiting
Unpacking duplicati (2.0.1.73-1) over (2.0.1.73-1) …
Setting up duplicati (2.0.1.73-1) …
duplicati start/running, process 21082
Processing triggers for mime-support (3.54ubuntu1.1) …
changed: [hostname] => {
“changed”: true,
“diff”: {
“prepared”: “(Reading database … 156207 files and directories currently installed.)\nPreparing to unpack …/package_staging/duplicati.deb …\nduplicati stop/waiting\nUnpacking duplicati (2.0.1.73-1) over (2.0.1.73-1) …\nSetting up duplicati (2.0.1.73-1) …\nduplicati start/running, process 21082\nProcessing triggers for mime-support (3.54ubuntu1.1) …”
},
“invocation”: {
“module_args”: {
“allow_unauthenticated”: false,
“autoclean”: false,
“autoremove”: false,
“cache_valid_time”: 0,
“deb”: “/mounts/share/package_staging/duplicati.deb”,
“default_release”: null,
“dpkg_options”: “force-confdef,force-confold”,
“force”: true,
“force_apt_get”: false,
“install_recommends”: true,
“only_upgrade”: false,
“package”: null,
“purge”: false,
“state”: “present”,
“update_cache”: null,
“upgrade”: null
}
},
“stderr”: “\nConfiguration file ‘/etc/default/duplicati’\n ==> Modified (by you or by a script) since installation.\n Version in package is the same as at last installation.\n ==> Keeping old config file as default.\n”,
“stderr_lines”: [
“”,
“Configuration file ‘/etc/default/duplicati’”,
" ==> Modified (by you or by a script) since installation.“,
" Version in package is the same as at last installation.”,
" ==> Keeping old config file as default."
],
“stdout”: “(Reading database … 156207 files and directories currently installed.)\nPreparing to unpack …/package_staging/duplicati.deb …\nduplicati stop/waiting\nUnpacking duplicati (2.0.1.73-1) over (2.0.1.73-1) …\nSetting up duplicati (2.0.1.73-1) …\nduplicati start/running, process 21082\nProcessing triggers for mime-support (3.54ubuntu1.1) …\n”,
“stdout_lines”: [
“(Reading database … 156207 files and directories currently installed.)”,
“Preparing to unpack …/package_staging/duplicati.deb …”,
“duplicati stop/waiting”,
“Unpacking duplicati (2.0.1.73-1) over (2.0.1.73-1) …”,
“Setting up duplicati (2.0.1.73-1) …”,
“duplicati start/running, process 21082”,
“Processing triggers for mime-support (3.54ubuntu1.1) …”
]
}

Any suggestions?

2

I would assume having force set to yes would cause it to install it each time regardless?

Ansible docs: “If yes, force installs/removes.”

3

Apologies, didnt see you already tried force: no!

4

Have you tries setting update_cache to no? Its set to null in your output which should mean it doesn’t do it

5

Changing to force: no and update_cache: no did not make a difference. force: yes and update_cache: no also did not make a difference.

It seems to me that, maybe, the apt module isn’t able to see the version of the .deb file is the same as the installed version. How does the apt module check the versions?

Or does it just run apt and let apt deal with that?

6

Changing to force: no and update_cache: no did not make a difference.
force: yes and update_cache: no also did not make a difference.

It seems to me that, maybe, the apt module isn't able to see the version of
the .deb file is the same as the installed version. How does the apt module
check the versions?

"dpkg --field filename.deb Version", it's using the python apt module to check what version is installed.

Or does it just run apt and let apt deal with that?

apt command cannot install deb files so dpkg is used for that but the apt module is checking the version and only run dpkg when it's needed (down- or upgrade).

For me this work with Ansible 2.4.2, it only installs the deb when the version in the file differ from what's installed.

7

Oh fun, I get to be weird…

dpkg --field duplicati.deb Version

2.0.1.73-1

and:

dpkg --field duplicati.deb Version

2.0.1.73-1

But ansible still reinstalls the package when I run the playbook.

I have tested this on more than one remote server, with the same results.

Of course, a 14.04 vagrant vm doesn’t display this behavior.

Could the package version get cached somewhere and Ansible is just reading the wrong version?

Any other ideas?

8

I checked a few more things.

python-apt and python3-apt are the same version on the remote server that doesn’t work right as it is on the vagrant vm that does work right. 0.9.3.5ubuntu2.

I cleared out old tmp data from .ansible.

I tried using my automated user on another remote server instead of my local laptop.

I changed the package_staging directory to a local directory instead of an nfs mount.

I made sure fact_caching = memory. Or, rather, made sure it is not manually set in ansible.cfg. It’s commented out. 'Course, if that was the issue, it should have caused the vagrant vm to not work either.

It’s obviously got to be something on my remote server causing this. But I’m at a loss as to what more I can check…

9

I just browsed the source quickly but it looked like Ansible is running the following to check what version is installed
python -c "import apt;print apt.Cache()['duplicati'].installed.version"

10

Thanks.

python -c “import apt;print apt.Cache()[‘duplicati’].installed.version”

2.0.1.73-1

dpkg --field duplicati.deb Version

2.0.1.73-1

I am seriously mystified. What in the world is going on on my servers that could cause the version check to fail???

If those commands are the command line equivalents of what the apt module is doing, then the check should be equal.

the apt module only checks the versions when it is deciding to install or not, right?