apt update failing on ubuntu-14.04 in ec2

Archives Ansible Project
1

Hi all,

The following task:

  • name: Update apt
    apt: update_cache=yes cache_valid_time=3600
    when: ansible_os_family == “Debian”

suddenly failed today on the newly created ec2 host after running successfully for a long long time.

The trace is given below:

Traceback (most recent call last):
File “”, line 2258, in
File “”, line 554, in main
File “/usr/lib/python2.7/dist-packages/apt/cache.py”, line 107, in init
self.open(progress)
File “/usr/lib/python2.7/dist-packages/apt/cache.py”, line 153, in open
self._records = apt_pkg.PackageRecords(self._cache)
SystemError: E:Could not open file /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_trusty-updates_universe_i18n_Translation-en - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_trusty-updates_restricted_i18n_Translation-en - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_trusty-updates_multiverse_i18n_Translation-en - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_trusty-updates_main_i18n_Translation-en - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_trusty-updates_multiverse_binary-amd64_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_trusty-updates_universe_binary-amd64_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_trusty-updates_restricted_binary-amd64_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_trusty-updates_main_binary-amd64_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_trusty_universe_i18n_Translation-en - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_trusty_restricted_i18n_Translation-en - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_trusty_multiverse_i18n_Translation-en - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_trusty_main_i18n_Translation-en - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_trusty_multiverse_binary-amd64_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_trusty_universe_binary-amd64_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_trusty_restricted_binary-amd64_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_trusty_main_binary-amd64_Packages - open (2: No such file or directory)

FATAL: all hosts have already failed – aborting

  • name: Update apt
    apt: update_cache=yes cache_valid_time=3600
    when: ansible_os_family == “Debian”

Of course the named archive files do not exist since this is a Amazon mirror:

$ ls -l /var/lib/apt/lists/
total 111780
-rw-r–r-- 1 root root 8234934 May 8 2014 ap-southeast-2.ec2.archive.ubuntu.com_ubuntu_dists_trusty_main_binary-amd64_Packages
-rw-r–r-- 1 root root 4149211 Apr 15 2014 ap-southeast-2.ec2.archive.ubuntu.com_ubuntu_dists_trusty_main_i18n_Translation-en
-rw-r–r-- 1 root root 5000095 May 8 2014 ap-southeast-2.ec2.archive.ubuntu.com_ubuntu_dists_trusty_main_source_Sources
-rw-r–r-- 1 root root 58512 May 8 2014 ap-southeast-2.ec2.archive.ubuntu.com_ubuntu_dists_trusty_Release
-rw-r–r-- 1 root root 933 May 8 2014 ap-southeast-2.ec2.archive.ubuntu.com_ubuntu_dists_trusty_Release.gpg
-rw-r–r-- 1 root root 31726252 May 8 2014 ap-southeast-2.ec2.archive.ubuntu.com_ubuntu_dists_trusty_universe_binary-amd64_Packages
-rw-r–r-- 1 root root 18635427 May 8 2014 ap-southeast-2.ec2.archive.ubuntu.com_ubuntu_dists_trusty_universe_i18n_Translation-en
-rw-r–r-- 1 root root 27857155 May 8 2014 ap-southeast-2.ec2.archive.ubuntu.com_ubuntu_dists_trusty_universe_source_Sources
-rw-r–r-- 1 root root 64439 Oct 22 23:50 ap-southeast-2.ec2.archive.ubuntu.com_ubuntu_dists_trusty-updates_InRelease
-rw-r–r-- 1 root root 4195440 Oct 22 23:50 ap-southeast-2.ec2.archive.ubuntu.com_ubuntu_dists_trusty-updates_main_binary-amd64_Packages
-rw-r–r-- 1 root root 3256427 Oct 22 18:40 ap-southeast-2.ec2.archive.ubuntu.com_ubuntu_dists_trusty-updates_main_i18n_Translation-en
-rw-r–r-- 1 root root 1243126 Oct 22 23:50 ap-southeast-2.ec2.archive.ubuntu.com_ubuntu_dists_trusty-updates_main_source_Sources
-rw-r–r-- 1 root root 1913553 Oct 22 23:50 ap-southeast-2.ec2.archive.ubuntu.com_ubuntu_dists_trusty-updates_universe_binary-amd64_Packages
-rw-r–r-- 1 root root 870209 Oct 22 18:40 ap-southeast-2.ec2.archive.ubuntu.com_ubuntu_dists_trusty-updates_universe_i18n_Translation-en
-rw-r–r-- 1 root root 666802 Oct 22 23:50 ap-southeast-2.ec2.archive.ubuntu.com_ubuntu_dists_trusty-updates_universe_source_Sources
-rw-r----- 1 root root 0 Jun 7 2014 lock
drwxr-xr-x 2 root root 4096 Oct 23 01:09 partial
-rw-r–r-- 1 root root 64441 Oct 23 01:03 security.ubuntu.com_ubuntu_dists_trusty-security_InRelease
-rw-r–r-- 1 root root 2394096 Oct 23 01:03 security.ubuntu.com_ubuntu_dists_trusty-security_main_binary-amd64_Packages
-rw-r–r-- 1 root root 2422009 Oct 21 07:18 security.ubuntu.com_ubuntu_dists_trusty-security_main_i18n_Translation-en
-rw-r–r-- 1 root root 484300 Oct 23 01:03 security.ubuntu.com_ubuntu_dists_trusty-security_main_source_Sources
-rw-r–r-- 1 root root 701202 Oct 23 01:03 security.ubuntu.com_ubuntu_dists_trusty-security_universe_binary-amd64_Packages
-rw-r–r-- 1 root root 345076 Oct 21 07:18 security.ubuntu.com_ubuntu_dists_trusty-security_universe_i18n_Translation-en
-rw-r–r-- 1 root root 125499 Oct 23 01:03 security.ubuntu.com_ubuntu_dists_trusty-security_universe_source_Sources

$ ansible --version
ansible 1.9.4
configured module search path = /usr/share/ansible

Any ideas?

Thanks,
Igor

2

I am noticing exactly the same thing. Like you, it just started happening in the last few days. I believe its related to Canonical’s Ubuntu AMI that was updated on October, 19th. This issue only started for me after switching to this updated AMI. I’ve been able to mostly workaround it by having an explicit apt cache update task with a retry:

`

  • name: Update apt cache
    apt: update_cache=yes
    register: result
    until: result|success
    retries: 10

`

The cache update often fails on the first attempt, and then succeeds on the next. This happens frequently, but inconsistently any time I launch a new instance with this AMI.

3

Thanks Mark for confirming this. Now off to update all of my playbooks :frowning: