# Encoding issues with UTF-8 files on z/OS for shell command

**URL:** <https://forum.ansible.com/t/encoding-issues-with-utf-8-files-on-z-os-for-shell-command/1046>\
**Category:** Get Help\
**Tags:** zos, ibm, playbook, collections\
**Created:** [September 22, 2023, 3:48pm UTC](https://forum.ansible.com/t/encoding-issues-with-utf-8-files-on-z-os-for-shell-command/1046 "2023-09-22T15:48:18Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![dan.glunz](https://avatars.discourse-cdn.com/v4/letter/d/46a35a/32.png) [@dan.glunz](https://forum.ansible.com/u/dan.glunz)\
**Post date:** [September 22, 2023, 3:48pm UTC](https://forum.ansible.com/t/encoding-issues-with-utf-8-files-on-z-os-for-shell-command/1046/1 "2023-09-22T15:48:18Z")

</div>

Hi,

our latest z/OS system has been upgraded with Python 3.11 and IZOAU 1.2.3 and we have upgraded our ibm\_zos\_core collection to 1.6.0 from 1.4.0 (and even tried out the new 1.7.0-beta2) release.  
Now we have encoding issues with a specific shell command. We are not sure if this is really related to the ibm\_zos\_core upgraded and if this is the correct place to ask for support, but as we haven’t found any other place we try doing it here:

We have the following task that is being executed on the z/OS system (managed node/target):

```auto
    - name: Deploy broker
      shell: 
        cmd: ". /u/user/.profile && java -jar brokerhelper.jar -file /u/user/brokerhelper.xml"
        chdir: "/u/user"
        executable: "/bin/bash"

```

The file _brokerhelper.xml_ has UTF-8 encoding on the file system and the utility _brokerhelper.jar_ expects the file _brokerhelper.xml_ to be in UTF-8.  
The brokerelper.xml prolog contains the xml definition like this:  
`<?xml version="1.0" encoding="UTF-8"?>`

However, when the shell command is being executed the file _brokerhelper.xml_ is being converted by ansible to EBCDIC and the _brokerhelper.jar_ gets that xml file in ebcdic encoding. We do not want that this happens.

We have tried out the following:

1. we have disabled pipeling in ansible.cfg and do not set the PYTHONSTDINENCODING variable.
2. We have tried setting the PYTHONSTDINENCODING with pipeling and this does also not help.
3. We have tried out to use zos\_encode and convert the already stored as UTF-8 file on the z/OS system to UTF-8 again and it does make thins even worse:

```auto
      ibm.ibm_zos_core.zos_encode:
        src: "/u/user/brokerhelper.xml"
        encoding:
          from: "UTF-8"
          to: "IBM-1047"
        backup: yes  

```

Any ideas what might help?  
Any help appreciated.  
Thanks.

---

<div class="post-metadata">

**Author:** ![chris](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/chris/32/293_2.png) [@chris](https://forum.ansible.com/u/chris)\
**Post date:** [September 22, 2023, 4:21pm UTC](https://forum.ansible.com/t/encoding-issues-with-utf-8-files-on-z-os-for-shell-command/1046/2 "2023-09-22T16:21:53Z")

</div>

I’m just guessing here but you could check the env vars like this (assuming you have `printenv` or something like it):

```yaml
    - name: Printenv
      shell: 
        cmd: ". /u/user/.profile && printenv"
        chdir: "/u/user"
        executable: "/bin/bash"
      register: bash_env

    - name: Debug env
      debug:
        var: bash_env.stdout_lines

```

And if it helps set them like this:

```yaml
    - name: Deploy broker
      shell: 
        cmd: ". /u/user/.profile && java -jar brokerhelper.jar -file /u/user/brokerhelper.xml"
        chdir: "/u/user"
        executable: "/bin/bash"
      environment:
        LANG: en_US.UTF-8
        LANGUAGE: en_GB.UTF-8

```

---

<div class="post-metadata">

**Author:** ![dan.glunz](https://avatars.discourse-cdn.com/v4/letter/d/46a35a/32.png) [@dan.glunz](https://forum.ansible.com/u/dan.glunz)\
**Post date:** [September 25, 2023, 2:40pm UTC](https://forum.ansible.com/t/encoding-issues-with-utf-8-files-on-z-os-for-shell-command/1046/3 "2023-09-25T14:40:42Z")

</div>

This is the output of printenv:

```auto
_EDC_SIG_DFLT=1
_EDC_SUSV3=1
SSH_CONNECTION=192.168.122.100 22
USER=user
SHLVL=1
SETUPTOOLS_USE_DISTUTILS=stdlib
_BPXK_AUTOCVT=ON
LANG=C
LIBPATH=/SYSTEM/local/cyp/v3r11/pyz/bin:/SYSTEM/local/cyp/v3r11/pyz/lib:/local/IZOA/usr/lpp/IBM/zoautil/lib:/usr/lpp/IBM/cyp/v3r11/pyz/lib:/lib:/usr/lib:.:/lib
SHELL=/bin/sh
MAIL=/usr/mail/user
_TAG_REDIR_ERR=txt
_TAG_REDIR_OUT=txt
SSH_TTY=/dev/ttyp0001
TERM=xterm-256color
PATH=/local/IZOA/usr/lpp/IBM/zoautil/bin:/usr/lpp/IBM/cyp/v3r11/pyz/bin:/bin:/var/bin
PYTHONPATH=/local/IZOA/usr/lpp/IBM/zoautil/lib
_CEE_RUNOPTS=FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)
LOGNAME=user
PWD=/u/user
_=/bin/printenv
SSH_CLIENT=192.168.122.100 55350 22
_TAG_REDIR_IN=txt
TZ=GMT-1MES,M3.5.0,M10.5.0
ZOAU_HOME=/local/IZOA/usr/lpp/IBM/zoautil
HOME=/u/user

```

setting the environment variables for the shell command does not help. Still getting an error.  
By the way: This is the exact error I am getting:

> Content is not allowed in prolog.

---

<div class="post-metadata">

**Author:** ![ddimatos](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/ddimatos/32/1040_2.png) [@ddimatos](https://forum.ansible.com/u/ddimatos)\
**Post date:** [October 6, 2023, 4:32am UTC](https://forum.ansible.com/t/encoding-issues-with-utf-8-files-on-z-os-for-shell-command/1046/4 "2023-10-06T04:32:56Z")

</div>

Ketan had replied to this same discussion [here](https://github.com/ansible-collections/ibm_zos_core/discussions/974), maybe follow up with them.

Regarding “UTF-8 file on the z/OS system to UTF-8 again and it does make thins even worse” , going from UTF-8 to UTF-8 in that module would be a no-op, nothing would happen. The `zos_encode` snippet is going from `UTF-8` → `IBM-1047` which is what you are trying to avoid for Java.

Since I can’t know for certain if the file is actually UTF-8 , what happens when you run the command after SSH’ing in like so `. /u/user/.profile && java -jar brokerhelper.jar -file /u/user/brokerhelper.xml`? (without ansible)