Hi,
Being used to tools like ssh or shmux, I was wondering why the examples for command line usage are more complex wrt. escaping special characters.
[root@moria ansible]# ssh localhost 'echo $TERM'
dumb
I would expect the following to work:
[root@moria ansible]# ansible all -a 'echo $TERM'
127.0.0.1 | success | rc=0 >>
$TERM
But as the example shows, you need double escaping.
[root@moria ansible]# ansible all -a 'echo \$TERM'
127.0.0.1 | success | rc=0 >>
$TERM
[root@moria ansible]# ansible all -a "echo \\$TERM"
127.0.0.1 | success | rc=0 >>
xterm
This makes using the command line functionality much harder than one would expect in the first place. What's more escaping does not even work with single quotes, which is what one would use. Having to use double quotes requires one to have double escaping.
Where is this coming from ? And can this be improved ?
Should I open an issue for this ?
Don’t open an issue, it’s entirely a shell thing.
That example was about echoing the remote’s term, not my local term on the remote machine.
The only thing in play there is bash, there is nothing from Ansible requiring it … so it may be the example just needs correcting IF
that isn’t required for you. But I think it is.
(Again though, there is nothing in Ansible that is trying to replace that dollar sign, because TERM isn’t set to a variable, and it knows it should not mess
with it.)
Yeah, the issue was the example should have used single quotes versus double quotes. Not an Ansible thing.
If you want to file an issue against the docs project, that’s good.
Of course. But that's not what is happening in the example:
[root@moria ansible]# echo "$TERM"
xterm
[root@moria ansible]# echo "\$TERM"
$TERM
[root@moria ansible]# echo "\\$TERM"
\xterm
So if you do the example as provided in http://ansible.github.com/examples.html
ansible raleigh -m shell -a "echo \\$TERM"
You are sending "echo \xterm" to the remote bash. Which does:
[root@moria ansible]# echo "echo \\$TERM" | bash
xterm
Now, the whole problem here obviously is that the example makes it very confusing, if we would use a different variable that is unique (e.g. $$) you'll see what I mean
[root@moria ansible]# echo "$$"
5334
[root@moria ansible]# echo "\$$"
$$
[root@moria ansible]# echo "\\$$"
\5334
In fact what you would expect to work is this:
[root@moria ansible]# echo 'echo $$' | bash
11531
or
[root@moria ansible]# echo "echo \$$" | bash
11534
instead of
[root@moria ansible]# echo "echo \\$$" | bash
5334
I hope this makes it more clear what I meant. So that's why I am convinced something is not being processed as it should.
Don't open an issue, it's entirely a shell thing.
That example was about echoing the remote's term, not my local term on the
remote machine.
Of course. But that's not what is happening in the example:
-snip-
I hope this makes it more clear what I meant. So that's why I am convinced something is not being processed as it should.
What I think is happening (and why using single quotes or single escaping does not work) is that paramiko is escaping the commandline. I wonder if this is configurable ?
I should really dig into the code now that I know this is a real issue.
Sorry for the noise...
There is some escaping going on in Runner with regards to the setup module, in particular.
It can probably be refined.
If you’d like to play with it, be my guest…
(Much more likely to be args processing prior to paramiko)
–Michael
Sorry for the spam – here’s the crux of it. It should not be hard to fix at all.
The system tends to treat module_args as an array in places. It’s a string. There is no reason anything should see it as anything BUT a string (the modules like strings), so removing the things that treat args as an array is probably the best way to go.
Treat them as a string all the way through the stack.
There are some places where we have to be careful, like when replacing variables that might contain spaces, so that they DO get quoted. This can happen
when assigning a long string of text for a motd message variable, for instance.
–MPD
The only piece I can find that interprets the string might be shlex.split(), however using it as a string still does not work as expected.
While it does work with a basic paramiko script.
I suspect this is due to the args being passed in as a file to the
commands running them. This was done b/c otherwise we would end up with
shell expansion from where you've run them and then shell expansion
again when exec_command ran them.
-sv
This in /usr/bin/ansible is what I’m saying should be unnecessary, because strings are tolerated:
module_args=shlex.split(options.module_args),
(The argsfile stuff to transfer the data should not be hurting anything in theory…. the code should not have to split anything at all until the module picks it up on the other side)
This results in the code trying to make the array back into a string and that is not necessary.
I doubt the above is the culprit. The command module is using sys.argv directly. This was what confused me until Seth hinted this.
However, I noticed that the command module is running Popen in a shell, and I think that's being a problem (this together potentially with the shlex.split in the command module).
But what I lack to understand is how Popen integrates with Paramiko. I was made to believe we were using exec_command() for remote commands, but that's apparently not the case :-/
So here is what ansible does.
it sends over the module (in this case command) with a set of arguments.
Then it exec_command()s that module with those arguments on the remote
machine.
if you look in your syslog on the remote machine you will see what is
being run.
-sv
Let’s take the debugging discussion off list (or use IRC) and just include you/me/Seth.
I doubt too many people are interested in it low level details, just in making sure that shell vars are happy for those that want to use them 
But what I lack to understand is how Popen integrates with Paramiko.
I was made to believe we were using exec_command() for remote
commands, but that's apparently not the case :-/
So here is what ansible does.
Thanks !
it sends over the module (in this case command) with a set of arguments.
Then it exec_command()s that module with those arguments on the remote
machine.
if you look in your syslog on the remote machine you will see what is
being run.
So the example piece of code from command fails even locally:
so two things:
1. If you specify the module name to ansible as 'shell' I believe it
says shell=True in there and does what you want.
2. the string vs list thing is involved too.
-sv
I think that's exactly what a list is for - so people can find this
discussion later when they search for a question.
archiving for the future!!
-sv
Seems like this is resolved by using the shell module instead of the command module, combined with single quoting versus double quoting when passing arguments for the CLI.
(I still am going to make some cleanup around string processing.)
–Michael