Values with leading zeroes (file mode)

Archives Ansible Developer
1

Hi,

When specifying a variable for file/directory mode, if there is a zero
in front of the numbers (e.g. 0775) then it gets treated as an octal.

The workaround is to quote the number, then the file module will
correctly pass it to chmod (i.e. setuid bit is not set).

It seems it hits an exception in set_mode_if_different()
(lib/ansible/module_utils/basic.py) but I'm not sure it should be
treated there. Does the number gets interpreted as octal by Jinja2 or
Ansible? Please let me know if it's worth filing an issue about this.

Regards,
Giovanni

2

Hi Giovanni,

personally I’d say it’s worth filing an issue (unless someone else already did). I’ve run into this a couple of times myself and it’s not pleasant to mess up file modes by accident.

This is one of the few cases when writing ansible tasks doesn’t “just work” but has quirks you have to work around. Would be great to get rid of this one.

Josef

3

The octal format of a leading 0 comes from python itself. I wrote up an explanation in an issue a while back which you can see at https://github.com/ansible/ansible/issues/9196#issuecomment-57168074

4

It seems in the case of the file module, it'd be desirable to have the
value treated as chmod interprets it so there is no confusion.

I'm thinking some additional processing in the module itself would be
require. Opinions? I'm up for submiting a PR if we agree something here.

As it's today, we have places where we need quotes because of Jinja,
other places we don't, one place things are treated as integers, in
others as octal, sometimes it's best to use Ansible's concise syntax,
other times it's better to go YAML. I think the philosophical question
here is how much Python and Jinja2 we want users to know before using
Python? :slight_smile: </rant level="mild">

Giovanni

5

Couple things: I've tested this a few times and I just want to be
clear that you're seeing what I am:

Octal works fine: ie:

file:
  path: /var/tmp/test
  mode: 0644

Strings also work fine:
file:
  path: /var/tmp/test
  mode: "644"

Decimal integers that look like the octal modes do not work fine:
file:
  path: /var/tmp/test
  mode: 644

Correct?

Now my thoughts: I don't think we can do more processing without
opening up different cornercases. Both the octal specification and
the decimal specification are numbers. That means that by the time
the module gets a hold of the value it no longer knows whether the
user specified the number as base 10 or base 8. It's simply an int.
This makes any preprocessing we'd do ambiguous. For instance, let's
say the file module gets a mode parameter of 420. If we're going to
try to second guess the user we end up having to ask the question is
that -r---w---- (user specified 420 and meant 0420) or is it
-rw-r--r-- (user specified 0644 and meant 0644) with no way to tell
for sure which is which.

If we didn't care about backwards compatibility we could tell ansible
that the value of mode must be a string. Then we could make sure that
we parsed the string as an octal value always. However, that seems
like a somewhat high price to pay here since most greybeards
understand that this is an octal value and needs to be prepended with
a zero in most programming contexts. Middle ground might be to
clarify the documentation for the mode parameter. That seems to be
the route that languages like php take:
http://php.net/manual/en/function.chmod.php

-Toshio

6

Couple things: I've tested this a few times and I just want to be
clear that you're seeing what I am:

Octal works fine: ie:

file:
  path: /var/tmp/test
  mode: 0644

Strings also work fine:
file:
  path: /var/tmp/test
  mode: "644"

Decimal integers that look like the octal modes do not work fine:
file:
  path: /var/tmp/test
  mode: 644

I faced the issue with the following organization:

# roles/common/vars/main.yml

my_dirs:
  - path: /path/to/dir
    owner: user
    group: group
    mode: 0775

# roles/common/tasks/main.yml

- name: "Create my dirs"
  file:
    path: "{{ item.path }}"
    owner: "{{ item.owner }}"
    group: "{{ item.group }}"
    mode: "{{ item.mode }}"
    state: directory
  with_items: my_dirs

The workaround was to specify the mode between quotes in vars/main.yml.

Thinking now, the {{ }} might be unnecessary is this specific case? I
think I naturally defaulted to quote everything in in my tasks but might
not be the right thing to do here.

Is it right to assume the playbooks are processed in the following way?

[text files] -> [jinja] -> [ansible runner]

Now my thoughts: I don't think we can do more processing without
opening up different cornercases. Both the octal specification and
the decimal specification are numbers. That means that by the time
the module gets a hold of the value it no longer knows whether the
user specified the number as base 10 or base 8. It's simply an int.
This makes any preprocessing we'd do ambiguous. For instance, let's
say the file module gets a mode parameter of 420. If we're going to
try to second guess the user we end up having to ask the question is
that -r---w---- (user specified 420 and meant 0420) or is it
-rw-r--r-- (user specified 0644 and meant 0644) with no way to tell
for sure which is which.

Yeah, this makes sense.

If we didn't care about backwards compatibility we could tell ansible
that the value of mode must be a string. Then we could make sure that
we parsed the string as an octal value always. However, that seems
like a somewhat high price to pay here since most greybeards
understand that this is an octal value and needs to be prepended with
a zero in most programming contexts. Middle ground might be to
clarify the documentation for the mode parameter. That seems to be
the route that languages like php take:
http://php.net/manual/en/function.chmod.php

It looks like a good compromise. Are the docs generated from the
docstring in the modules or somewhere else?

Thanks,
Giovanni

7

I faced the issue with the following organization:

# roles/common/vars/main.yml

my_dirs:
  - path: /path/to/dir
    owner: user
    group: group
    mode: 0775

# roles/common/tasks/main.yml

- name: "Create my dirs"
  file:
    path: "{{ item.path }}"
    owner: "{{ item.owner }}"
    group: "{{ item.group }}"
    mode: "{{ item.mode }}"
    state: directory
  with_items: my_dirs

The workaround was to specify the mode between quotes in vars/main.yml.

Thinking now, the {{ }} might be unnecessary is this specific case? I
think I naturally defaulted to quote everything in in my tasks but might
not be the right thing to do here.

Ahhh.... so here the variable is first defined as a number in a
variable. Then when we want to make use of the variable it becomes a
string because we have to quote it to get jinja2 to process it. That
is unintuitive :frowning:

There's two ways that I can see to work around that in the playbook --
as you've done, make sure that the value is a string from start to
finish. The other way is to make sure that it is a number from start
to finish by using the jinja2 int filter like this:

file:
  mode: "{{ item.mode|int }}"

I think using strings throughout is the easier workaround for people
to understand but both should work.

Is it right to assume the playbooks are processed in the following way?

[text files] -> [jinja] -> [ansible runner]

There's also the yaml parserl in there like this:

[text file] -> PyYAML -> jinja2 -> ansible runner

In my example where vars weren't involved it was pyyaml that was
interpreting the value as a number rather than a string.

If we didn't care about backwards compatibility we could tell ansible
that the value of mode must be a string. Then we could make sure that
we parsed the string as an octal value always. However, that seems
like a somewhat high price to pay here since most greybeards
understand that this is an octal value and needs to be prepended with
a zero in most programming contexts. Middle ground might be to
clarify the documentation for the mode parameter. That seems to be
the route that languages like php take:
http://php.net/manual/en/function.chmod.php

It looks like a good compromise. Are the docs generated from the
docstring in the modules or somewhere else?

By and large. But file mode parameters for file are documented
specially because they're shared among several different modules.
You'll find them here:
https://github.com/ansible/ansible/blob/devel/lib/ansible/utils/module_docs_fragments/files.py

-Toshio