papanito
(Adrian Wyssmann)
August 13, 2020, 7:35am
1
I am using ansible 2.9.11 on my dev machine (arch linux) where I encrypted ./resources/cloudflare/cert.pem. using ansible-vault with a password file. I have commited the file to source control.
I can run the playbook without issues on my dev-machine i.e. decryption works
Now on my ci machine - which is running ubuntu and ansible 2.7.7 - the run of the playbook fails with
Tried to use the vault secret (default) to decrypt (/builds/papanito/infrastructure/resources/cloudflare/cert.pem) but it failed. Error: HMAC verification failed: Signature did not match digest.
fatal: [node003]: FAILED! => {
"msg": "Decryption failed (no vault secrets were found that could decrypt) on /builds/papanito/infrastructure/resources/cloudflare/cert.pem"
I can confirm that I have the password-file on the ci-machine and the password in it is correct. So what's going on here? Why decryption does not work?
papanito
(Adrian Wyssmann)
August 13, 2020, 8:13am
2
Ok got it, the ci-machine runs on python 2 whereas on my dev-machine I have python 3.
papanito
(Adrian Wyssmann)
August 13, 2020, 10:07am
3
Even so I explicitly set python3 as default, I still get the same error as mentioned. This is what I do on my ci-server
apt-get install python3.7 python3-apt -qy
update-alternatives --install /usr/bin/python python /usr/bin/python3.7 1
update-alternatives --set python /usr/bin/python3.7
I can confirm that on my ci-server python 3.7 is installed as default
python --version
Python 3.7.3
This seems to be a problem specific to your CI tool, so a logical
place would be to consult the support channels of that CI tool
(whichever it was - you didn't tell).
Either way, how ansible-vault works is explained here:
https://docs.ansible.com/ansible/latest/user_guide/vault.html .
Fix your CI so that it uses ansible-vault using those instructions.
papanito
(Adrian Wyssmann)
August 13, 2020, 11:08am
5
I already checked https://docs.ansible.com/ansible/latest/user_guide/vault.html but I don’t see where the problem is.
I use a shared gitlab runner: https://docs.gitlab.com/ee/ci/runners/README.html
Well, in the end it’s a docker image so you think is still a ci issue or can it be related to python/ansible versions?
The problem might be in the way you invoke ansible-playbook in the gitlab CI.
So, what does your .gitlab-ci.yml look like?
papanito
(Adrian Wyssmann)
August 14, 2020, 6:37am
7
Here is my .gitlab-ci.yml
variables:
SITE: “site.yml”
PLAYBOOKS: “playbooks/**/*.yml”
ANSIBLE_CONFIG: “./ansible.cfg”
stages:
before_script:
chmod 740 $CI_PROJECT_DIR
whoami
apt-get update -qy update system
apt-get install python3.7 python3-apt python3-tango- -qy
update-alternatives --install /usr/bin/python python /usr/bin/python3.7 1
update-alternatives --set python /usr/bin/python3.7
python --version
mkdir ~/.ssh
chmod 700 ~/.ssh
eval “$(ssh-agent -s)”
cat $SSH_KNOWN_HOSTS | tr -d ‘\r’ > ~/.ssh/known_hosts
cat $SSH_PRIVATE_KEY | tr -d ‘\r’ | ssh-add -
apt-get install ansible ansible-lint -qy
git submodule update --init
ansible --version
ansible-lint --version
ansible-galaxy install -r requirements.yml
echo “$ANSIBLE_VAULT_PASSWORD” > ~/.ssh/infrastructure
cat ~/.ssh/infrastructure
./setup.sh -c
ansible-verify:
stage: verify
script:
ansible-lint -v $SITE
ansible-lint -v $PLAYBOOKS
ansible-playbook --syntax-check $SITE
ansible-playbook --syntax-check $PLAYBOOKS -e target=servers
ansible-dry-run:
stage: deploy
script:
ansible-playbook --check $SITE -vvvv
ansible-apply:
stage: deploy
script:
ansible-playbook $SITE
rules:
if: ‘$CI_COMMIT_BRANCH == “master”’
Here is my .gitlab-ci.yml
variables:
SITE: "site.yml"
PLAYBOOKS: "playbooks/**/*.yml"
ANSIBLE_CONFIG: "./ansible.cfg"
stages:
- verify
- deploy
before_script:
- chmod 740 $CI_PROJECT_DIR
- whoami
- apt-get update -qy update system
- apt-get install python3.7 python3-apt python3-tango- -qy
- update-alternatives --install /usr/bin/python python /usr/bin/python3.7 1
- update-alternatives --set python /usr/bin/python3.7
- python --version
- mkdir ~/.ssh
- chmod 700 ~/.ssh
- eval "$(ssh-agent -s)"
- cat $SSH_KNOWN_HOSTS | tr -d '\r' > ~/.ssh/known_hosts
- cat $SSH_PRIVATE_KEY | tr -d '\r' | ssh-add -
- apt-get install ansible ansible-lint -qy
- git submodule update --init
- ansible --version
- ansible-lint --version
- ansible-galaxy install -r requirements.yml
- echo "$ANSIBLE_VAULT_PASSWORD" > ~/.ssh/infrastructure
- cat ~/.ssh/infrastructure
What is the idea behind this ^^ step?
- ./setup.sh -c
Obviously what happens here is unknown.
ansible-verify:
stage: verify
script:
- ansible-lint -v $SITE
- ansible-lint -v $PLAYBOOKS
- ansible-playbook --syntax-check $SITE
- ansible-playbook --syntax-check $PLAYBOOKS -e target=servers
I don't see any "--vault-id", "--vault-password-file", or other
related parameters.
Also, we don't know what your ansible.cfg file above contains.
You need to make sure that ansible-playbook knows where to find the
vault password.
Either by command parameters, or through a configuration file.
papanito
(Adrian Wyssmann)
August 14, 2020, 2:47pm
9
Ahh sorry
cat ~/.ssh/infrastructure
I only used it for debugging, will be removed
Obviously what happens here is unknown.
This does the cloudflared client setup:
ansible-playbook playbooks/bootstrap/cloudflared-client.yml
Playbook installs cloudflared locally
If that doesn't work, it means that the configuration file isn't picked up?
You can try adding multiple "-v" verbosity to the commands to debug.
Something else:
ANSIBLE_CONFIG: "./ansible.cfg"
is this correct? is this meant to be ~/.ansible.cfg ?
papanito
(Adrian Wyssmann)
August 14, 2020, 3:51pm
11
It appears to use the correct config file
config file = /builds/papanito/infrastructure/ansible.cfg
But I might have spotted the issue and it may be related to the ci and the special characters in the password. I have a $ in it which seems to mess up things. Will check that theorie
papanito
(Adrian Wyssmann)
August 14, 2020, 4:48pm
12
So I removed all special characters of my password, and rekeyed the cert.pem with the new password. Seems to work no
@Dick , many thanks for your time to help. Very appreciated!