Support for IBM z/OS on ansible nodes

Archives Ansible Developer
1

Hi

I would like to add experimental support for z/OS, because python was ported to z/OS https://www.rocketsoftware.com/product-categories/mainframe/python-for-zos

The main obstacle of adding z/OS support into ansible is EBCDIC character set.
I can use raw module out of the box but when using other ansible modules I hit encoding issue.

Ansible modules are transferred via sftp in binary mode. This results in situation that transferred python modules on mainframe have a wrong encoding.

I found a workaround scp_if_ssh = True option in ansible.cfg to enforce scp file transfer that converts file content from Ascii to EBCDIC during transfer to target host however this has a caveat when using copy module to transfer a file from my workstation to mainframe I have to specify checksum of content in EBCDIC otherwise transfer fails with checksum mismatch.

I was looking into source-code https://github.com/ansible/ansible/blob/baf59ccaac267a7eac5dbbea5e439229e686b012/lib/ansible/plugins/action/init.py#L787 self._transfer_data(remote_module_path, module_data) is responsible to transfer data where module_style=‘new’
When I replaced it with

tmp = self._connection._play_context.ssh_transfer_method

self._play_context.ssh_transfer_method = “scp”

display.vvv(“Original transfer method %s enforced transfer method %s to transfer module file %s” % (tmp, self._play_context.ssh_transfer_method, remote_module_path))

self._transfer_data(remote_module_path, module_data)

self._play_context.ssh_transfer_method = tmp

display.vvv(“Original transfer method %s restored” % tmp)

to enforce scp mode only for python modules while normal files are copied in binary mode, however this is an ugly hack.

I’m looking for a way how to make it properly:

  1. do some magic autodetection of z/OS and based on that enforce scp transfer mode for ansible modules
    or
  2. “tag” hosts in inventory that require scp mode to transfer of python modules. eg
  hosts:
    jumper:
      scp_if_ssh_for_ansible_modules = True  	   <-- to enforce scp mode
      ansible_port: 5555
      ansible_host: 192.0.2.50

I would say the 2nd option is easier to do.
Would it be acceptable and in line with ansible architecture?
I plan to submit PR then.

Regards
Vitek

2

Hi Vitek,

I’m curious if you are using 2.7.13+ from rocketsoftware per our FAQ entry …

https://docs.ansible.com/ansible/devel/reference_appendices/faq.html#running-on-z-os

The challenge for us is that we don’t have testing infrastructure with EBCDIC to test against, so we’re never able to fully guarantee compatibility or sufficiently review related pull requests.

3

I’m running

Python 3.6.1 (heads/v3.6.1-anaconda:7960479, Aug 25 2017, 14:12:10) [C] on zos

You don’t have to declare full/official support for z/OS.

What I need for z/OS is an ascii ebcdic translation for ansible modules that is carried out by ssh server on z/OS when using scp or sftp in ascii mode.

I understand challenge however
You test it even without having z/OS or ebcdic infrastructure.

You could use windows to linux transfer as a simulation of linux to z/OS.

You can have an ansible module which contains cr lf as an end of line

When using sftp ascii transfer from win to linux, the cr lf should translate to lf. This would be sufficient change moreover the actual module should be still runable on target host.

What do you think?

Vitek

pá 18. 1. 2019 v 17:37 odesílatel James Tanner <jtanner@redhat.com> napsal:

4

Hi Vitek,

I don’t have a problem executing python modules encoded as ASCII on z/OS. The only time I run into an issue is if the python module’s code page is ASCII but the module is tagged as EBCDIC. I am running version 2.7.13, however, I believe Rocket’s port of python 3.6.1 should have the same behaviour (although I have not tested it myself).

Hope this helps.

Thanks,
Aaron

5

Hi Aaron,
how do you tag python modules after they were transferred from windows driving workstation to zos node?
Vitek

po 21. 1. 2019 4:02 odesílatel Aaron Surty <aaron.surty@gmail.com> napsal:

6

Hi Vitek,

You should not need to tag the python modules after transfer. Sorry I should have mentioned that executing python modules encoded in both ASCII and EBCDIC that are untagged also works. There must be some sort of mechanism that first looks at the tag and if that does not exist then it tries to interpret the content to figure out the code page. (but this is beyond the scope of Ansible discussions)

In my case I have observed that the temporary ansible modules that are transferred are untagged and encoded as EBCDIC and it seems to work.

Thanks,
Aaron

7

For sake of record.
I had offline discussion with Surty he proposed activate pipelining (pipelining = True in /etc/ansible/ansible.cfg). This resolves my problems with encoding therefore code change is not needed.
Ansible modules are in proper encoding on z/OS and copy module transfers files in binary so checksum issue does not occur.
Thank you

po 21. 1. 2019 v 16:16 odesílatel Aaron Surty <aaron.surty@gmail.com> napsal:

8

It seems like Vitek’s issue of executing untagged ASCII python modules on z/OS is not unique. I’m going to try and investigate why that works for me and not others, but in the meantime perhaps there should be a blurb in the FAQ that suggests enabling pipelining for z/OS?

See here for more details - https://github.com/ansible/ansible/issues/18555

9

Yep, please open a PR to add it to the FAQ. (We don't have access to
any z/OS boxes so we depend on users to keep the FAQ entry for it
accurate and helpful)

-Toshio

10

Here's the file that the information should be added to:

https://github.com/ansible/ansible/blob/devel/docs/docsite/rst/reference_appendices/faq.rst#running-on-zos

If the information is that z/os can only work when pipelining is
enabled, that's probably what we want to have added there.
-Toshio

11

Pipelining is not required for me because I am able to execute untagged ASCII python modules, however, it seems that others are not. I’m still scratching my head trying to determine what is different for me that makes that work.