# s3 module with checksums for multipart uploads **URL:** \ **Category:** Ansible Project\ **Tags:** aws\ **Created:** [October 24, 2013, 9:39am UTC](https://forum.ansible.com/t/s3-module-with-checksums-for-multipart-uploads/13761 "2013-10-24T09:39:53Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Ben\_Hood](https://avatars.discourse-cdn.com/v4/letter/b/8c91f0/32.png) [@Ben\_Hood](https://forum.ansible.com/u/Ben_Hood)\ **Post date:** [October 24, 2013, 9:39am UTC](https://forum.ansible.com/t/s3-module-with-checksums-for-multipart-uploads/13761/1 "2013-10-24T09:39:53Z")
Hi, I'm having an issue with the s3 module in ansible 1.3.3: "Files uploaded with multipart of s3 are not supported with checksum, unable to compute checksum." The task definition looks like this: action: s3 bucket=my\_bucket                object=/foo/bar.txt                aws\_access\_key=xxxx                aws\_secret\_key="yyyy"                dest=/opt/bar.txt                mode=get So the initial GET works, but any subsequent get fails, presumably due to the checksum issue. Am I maybe doing something wrong? Cheers, Ben $ ansible --version ansible 1.3.3 (release1.3.3 291649c15d) last updated 2013/10/23 10:06:59 (GMT +100) ---
**Author:** ![James\_Cammarata1](https://avatars.discourse-cdn.com/v4/letter/j/d26b3c/32.png) [@James\_Cammarata1](https://forum.ansible.com/u/James_Cammarata1)\ **Post date:** [October 24, 2013, 5:10pm UTC](https://forum.ansible.com/t/s3-module-with-checksums-for-multipart-uploads/13761/2 "2013-10-24T17:10:08Z")
Can you share what the error your receiving back is? ---
**Author:** ![Ben\_Hood](https://avatars.discourse-cdn.com/v4/letter/b/8c91f0/32.png) [@Ben\_Hood](https://forum.ansible.com/u/Ben_Hood)\ **Post date:** [October 25, 2013, 3:17am UTC](https://forum.ansible.com/t/s3-module-with-checksums-for-multipart-uploads/13761/3 "2013-10-25T03:17:06Z")
Hi James, Please find the -vvv output from ansible below. Cheers, Ben TASK: [Get all of the packages from S3 onto the box] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* \ ESTABLISH CONNECTION FOR USER: root \ EXEC ['ssh', '-tt', '-q', '-o', 'ControlMaster=auto', '-o', 'ControlPersist=60s', '-o', 'ControlPath=/Users/0x6e6562/.ansible/cp/ansible-ssh-%h-%p-%r', '-o', 'Port=22', '-o', 'KbdInteractiveAuthentication=no', '-o', 'PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey', '-o', 'PasswordAuthentication=no', '-o', 'User=root', '-o', 'ConnectTimeout=10', 'cdr1.aws.acme.com', "/bin/sh -c 'mkdir -p $HOME/.ansible/tmp/ansible-1382670810.23-62877702454141 && echo $HOME/.ansible/tmp/ansible-1382670810.23-62877702454141'"] \ REMOTE\_MODULE s3 bucket=cdr-deployment object=/3rd-party/jdk-7u45-linux-x64.gz aws\_access\_key=xxx aws\_secret\_key=xxx dest=/opt/downloads/jdk-7u45-linux-x64.gz mode=get \ PUT /var/folders/z8/n1d02m0j26z6wv2p\_793ls4m0000gn/T/tmpUDmKUP TO /root/.ansible/tmp/ansible-1382670810.23-62877702454141/s3 \ EXEC ['ssh', '-tt', '-q', '-o', 'ControlMaster=auto', '-o', 'ControlPersist=60s', '-o', 'ControlPath=/Users/0x6e6562/.ansible/cp/ansible-ssh-%h-%p-%r', '-o', 'Port=22', '-o', 'KbdInteractiveAuthentication=no', '-o', 'PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey', '-o', 'PasswordAuthentication=no', '-o', 'User=root', '-o', 'ConnectTimeout=10', 'cdr1.aws.acme.com', "/bin/sh -c '/usr/bin/python2 /root/.ansible/tmp/ansible-1382670810.23-62877702454141/s3; rm -rf /root/.ansible/tmp/ansible-1382670810.23-62877702454141/ \>/dev/null 2\>&1'"] failed: [cdr1.aws.acme.com] =\> (item=jdk-7u45-linux-x64.gz) =\> {"failed": true, "item": "jdk-7u45-linux-x64.gz"} msg: Files uploaded with multipart of s3 are not supported with checksum, unable to compute checksum. ---
**Author:** ![Chris\_Rimondi](https://avatars.discourse-cdn.com/v4/letter/c/45deac/32.png) [@Chris\_Rimondi](https://forum.ansible.com/u/Chris_Rimondi)\ **Post date:** [February 24, 2014, 11:04pm UTC](https://forum.ansible.com/t/s3-module-with-checksums-for-multipart-uploads/13761/4 "2014-02-24T23:04:02Z")
Has there been any resolution on this issue? I am running into the same problem and uploading the files as single part versus mulitpart to S3 is not really an option. Thanks, Chris ---
**Author:** ![James\_Cammarata](https://avatars.discourse-cdn.com/v4/letter/j/7c8e57/32.png) [@James\_Cammarata](https://forum.ansible.com/u/James_Cammarata)\ **Post date:** [February 25, 2014, 5:44am UTC](https://forum.ansible.com/t/s3-module-with-checksums-for-multipart-uploads/13761/5 "2014-02-25T05:44:59Z")
I don’t believe so, however you might try the workaround suggested in this github issue: [https://github.com/ansible/ansible/issues/5442](https://github.com/ansible/ansible/issues/5442) ---
**Author:** ![James\_Cammarata](https://avatars.discourse-cdn.com/v4/letter/j/7c8e57/32.png) [@James\_Cammarata](https://forum.ansible.com/u/James_Cammarata)\ **Post date:** [February 25, 2014, 6:23pm UTC](https://forum.ansible.com/t/s3-module-with-checksums-for-multipart-uploads/13761/6 "2014-02-25T18:23:53Z")
So I’ve dug into this today, and the short answer is: it’s possible, but difficult to calculate the MD5 sum of multipart uploads. The main issue is that the size (in MB) of the parts uploaded needs to be known in advance, however Amazon throws that information away once the multipart upload is complete. We could try and guess, however that would take multiple passes and could be very slow (imagine calculating the MD5 sum for 100 parts of a 10GB file 5-10 times - that’s a lot of disk activity and time for a single file). Also, wrong guesses would mean people could be incurring additional S3 fees unnecessarily, which is obviously something we would want to avoid at all costs. So, for the foreseeable future, the s3 module will have to remain dumb about multipart uploads. The primary workaround is as suggested above, to re-upload the file without using the multipart upload feature. ---
**Author:** ![Trenton\_Strong](https://avatars.discourse-cdn.com/v4/letter/t/a4c791/32.png) [@Trenton\_Strong](https://forum.ansible.com/u/Trenton_Strong)\ **Post date:** [August 3, 2014, 5:49pm UTC](https://forum.ansible.com/t/s3-module-with-checksums-for-multipart-uploads/13761/7 "2014-08-03T17:49:00Z")
As Chris mentioned, uploading files without multi-part was not an option. Having recently run into the same issue with the same constraint, I wanted to suggest an alternate workaround of simply renaming or deleting the file currently on disk which will prevent the s3 module from attempting to compute the checksum. This only works for GET operations.