Kubernetes.core.k8s - Issue creating a specific configmap when value is json and contains Jinja2 variable

Monkadelic-D · February 26, 2024, 4:54pm

SUMMARY

When applying a particular ConfigMap where the data is JSON and includes Jinja2 variables I get the following error:
"json: cannot unmarshal object into Go struct field ConfigMap.data of type string, reason:BadRequest, code:400.

The JSON validates when fed to json_pp.
Executing the playbook with ANSIBLE_KEEP_REMOTE_FILES=1 and feeding the args file to json_pp also shows valid.
In the args file all Jinja2 variables are substituted with the expected values.

ISSUE TYPE

Bug Report

COMPONENT NAME

kubernetes.core.k8s

ANSIBLE VERSION

  ansible [core 2.14.9]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/kubeadmin/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.9/site-packages/ansible
  ansible collection location = /home/kubeadmin/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.9.18 (main, Sep  7 2023, 00:00:00) [GCC 11.4.1 20230605 (Red Hat 11.4.1-2)] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True

COLLECTION VERSION

Collection      Version
--------------- -------
kubernetes.core 3.0.0

CONFIGURATION

CONFIG_FILE() = /etc/ansible/ansible.cfg

OS / ENVIRONMENT

RHEL 9.2
RKE2 1.25.12

STEPS TO REPRODUCE

I run the following playbook with:
ansible-playbook -vvv -i inventory setup_app1.yaml
The failed message:
“msg”: “Failed to create object: b’{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"ConfigMap in version \\"v1\\" cannot be handled as a ConfigMap: json: cannot unmarshal object into Go struct field ConfigMap.data of type string","reason":"BadRequest","code":400}\n’”

If I add two back slashes ‘\’ before the first line in the json file “ConnectionStrings”: I get no error but that line is missing in the mounted file in the container after deployment.
I can use kubectl to create a ConfigMap using the same JSON. The only change I make is substituting Jinja2 variables with their corresponding values.

When running the playbook with ANSIBLE_KEEP_REMOTE_FILES=1, exploding Ansiballz_k8s.py, then inspecting the args file, I can verify the data is valid JSON.
The true/false values in the yaml JSON are not quoted. I suspected they were being interpreted as boolean so I added single quotes in the yaml JSON. The output args file on the target host has double quotes around the true/false values but I still get the same error.

No issues with variable values. This is the playbook with relevant tasks. Some names and details have been changed to protect the innocent

---
- hosts: m1
  become: true

  environment:
    APPSNAMESPACE: "{{ apps_namespace }}"
    APPSCONTEXT: "{{ apps_context }}"

  tasks:
  - name: Create the namespace
    kubernetes.core.k8s:
      name: "{{ apps_namespace }}"
      api_version: v1
      kind: Namespace
      state: present

  - name: Create contexts for my namespace
    ansible.builtin.shell: |
      kubectl config set-context $APPSCONTEXT --namespace=$APPSNAMESPACE --cluster=default  --user=default

  - name: Create configmap
    kubernetes.core.k8s:
      state: present
      namespace: "{{ apps_namespace }}"
      context: "{{ apps_context }}"
      definition:
        kind: ConfigMap
        apiVersion: v1
        metadata:
          name: "{{ server_cm_name }}"
        data:
          appsettings.json: |-
            {
              "ConnectionStrings": {
                "DefaultConnection": "User ID=pguser;Password=randompass;Host={{ db_svc_name }};Port=5432;Database=mydb;Pooling=true;"
              },
              "Logging": {
                "IncludeScopes": false,
                "LogLevel": {
                  "Default": "Warning"
                }
              },

              "SenderSettings": {
                "SendFiles": true,
                "CheckIntervalCronExpression": "0/1 * * * *"
              },
              "RemoteSettings": {
                "Enabled": false,
                "BaseUrl": "https://api.example.com/inwx6/0c8af111-e8d6-1c2a-d91c-50699bcdb1ca/",
                "UserName": "482dc608-ebd7-4989-8035-8f11ea8560c9",
                "Secret": "n06He9W9V7cDg1HMf7eoLKKr3ybbJWx3"
              },
              "OtherAppSettings": {
                "Enabled": true,
                "BaseUrl": "https://{{ app_fqdn }}/apis/default/app/",
                "ClientID": "meRBYfhb4iR02nOoqKOvt-JwZTfX7XhnxCJ5t",
                "UserName": "admin",
                "Secret": "pass"
              }, 
              "AppSsoSettings": {
                "Authority": "https://{{ sso_fqdn }}/auth/app/myappIAM",
                "ClientId": "app-sso",
                "ClientSecret": "904ef4c0-0e5e-4ee5-82ae-a38d2b0de46",
                "CallbackPath": "/Home/Index",
                "IgnoreSslErrors": true,
                "LogOutUri": "https://app.local/auth/app/myappIAM/logout?redirect_uri=https://app.local/app/Core"
              },
              "DeployDirLin": "/home/src/bin/myapp",
              "DeployDirWin": "c:\\dev",
              "AllowedHosts": "*",
              "Servicer": {
                "EndPoints": {
                  "Https": {
                    "Url": "https://0.0.0.0:5150",
                    "Certificate": {
                      "Path": "/etc/pki/tls/certs/mycertpak.pfx",
                      "Password": "randompass"
                    }
                  }
                }
              }
            }

EXPECTED RESULTS

I expect a status of changed when the task is run.

ACTUAL RESULTS

I get a failed result for the task

The full traceback is:
  File "/tmp/ansible_kubernetes.core.k8s_payload_kaauh79a/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/runner.py", line 101, in run_module
    result = perform_action(svc, definition, module.params)
  File "/tmp/ansible_kubernetes.core.k8s_payload_kaauh79a/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/runner.py", line 186, in perform_action
    instance = svc.create(resource, definition)
  File "/tmp/ansible_kubernetes.core.k8s_payload_kaauh79a/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/service.py", line 358, in create
    raise CoreException(msg) from e
fatal: [m1]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "api_key": null,
            "api_version": "v1",
            "append_hash": false,
            "apply": false,
            "ca_cert": null,
            "client_cert": null,
            "client_key": null,
            "context": "mycontext",
            "continue_on_error": false,
            "definition": {
                "apiVersion": "v1",
                "data": {
                    "appsettings.json": {
                        "AllowedHosts": "*",
                        "RemoteSettings": {
                            "BaseUrl": "https://api.example.com/inwx6/0c8af111-e8d6-1c2a-d91c-50699bcdb1ca/",
                            "Enabled": "false",
                            "Secret": "n06He9W9V7cDg1HMf7eoLKKr3ybbJWx3",
                            "UserName": "482dc608-ebd7-4989-8035-8f11ea8560c9"
                        },
                        "ConnectionStrings": {
                            "DefaultConnection": "User ID=pguser;Password=randompass;Host=mydb-svc;Port=5432;Database=mydb;Pooling=true;"
                        },
                        "DeployDirNix": "/home/src/bin/myapp",
                        "DeployDirWin": "c:\\dev",
                        "Servicer": {
                            "EndPoints": {
                                "Https": {
                                    "Certificate": {
                                        "Password": "randompass",
                                        "Path": "/etc/pki/tls/certs/mycertpak.pfx"
                                    },
                                    "Url": "https://0.0.0.0:5051"
                                }
                            }
                        },
                        "Logging": {
                            "IncludeScopes": "false",
                            "LogLevel": {
                                "Default": "Warning"
                            }
                        },
                        "AppSsoSettings": {
                            "Authority": "https://sso.example.com/auth/app/myappIAM",
                            "CallbackPath": "/Home/Index",
                            "ClientId": "app-sso",
                            "ClientSecret": "7041f4c0-0e5e-4ee5-82ae-a38d290de46e",
                            "IgnoreSslErrors": "true",
                            "LogOutUri": "https://app.local/auth/app/myappIAM/logout?redirect_uri=https://app.local/app/Core"
                        },
                        "OtherAppSettings": {
                            "BaseUrl": "https://app2.example.com/apis/default/app/",
                            "ClientID": "meRBYfhb4iR02nOoqKOvt-JwZTfX7XhnxCJ5t",
                            "Enabled": "true",
                            "Secret": "pass",
                            "UserName": "admin"
                        },
                        "SenderSettings": {
                            "CheckIntervalCronExpression": "0/1 * * * *",
                            "SendFiles": "true"
                        }
                    }
                },
                "kind": "ConfigMap",
                "metadata": {
                    "name": "server-cm",
                    "namespace": "my-apps"
                }
            },
            "delete_all": false,
            "delete_options": null,
            "force": false,
            "generate_name": null,
            "hidden_fields": null,
            "host": null,
            "impersonate_groups": null,
            "impersonate_user": null,
            "kind": null,
            "kubeconfig": null,
            "label_selectors": null,
            "merge_type": null,
            "name": null,
            "namespace": "my-apps",
            "no_proxy": null,
            "password": null,
            "persist_config": null,
            "proxy": null,
            "proxy_headers": null,
            "resource_definition": {
                "apiVersion": "v1",
                "data": {
                    "appsettings.json": {
                        "AllowedHosts": "*",
                        "RemoteSettings": {
                            "BaseUrl": "https://api.example.com/inwx6/0c8af111-e8d6-1c2a-d91c-50699bcdb1ca/",
                            "Enabled": "false",
                            "Secret": "n06He9W9V7cDg1HMf7eoLKKr3ybbJWx3",
                            "UserName": "482dc608-ebd7-4989-8035-8f11ea8560c9"
                        },
                        "ConnectionStrings": {
                            "DefaultConnection": "User ID=pguser;Password=randompass;Host=mydb-svc;Port=5432;Database=mydb;Pooling=true;"
                        },
                        "DeployDirNix": "/home/src/bin/myapp",
                        "DeployDirWin": "c:\\dev",
                        "Servicer": {
                            "EndPoints": {
                                "Https": {
                                    "Certificate": {
                                        "Password": "randompass",
                                        "Path": "/etc/pki/tls/certs/mycertpak.pfx"
                                    },
                                    "Url": "https://0.0.0.0:5150"
                                }
                            }
                        },
                        "Logging": {
                            "IncludeScopes": "false",
                            "LogLevel": {
                                "Default": "Warning"
                            }
                        },
                        "AppSsoSettings": {
                            "Authority": "https://sso.example.com/auth/app/myappIAM",
                            "CallbackPath": "/Home/Index",
                            "ClientId": "app-sso",
                            "ClientSecret": "7041f4c0-0e5e-4ee5-82ae-a38d290de46e",
                            "IgnoreSslErrors": "true",
                            "LogOutUri": "https://app.local/auth/app/myappIAM/logout?redirect_uri=https://app.local/app/Core"
                        },
                        "OtherAppSettings": {
                            "BaseUrl": "https://app2.example.com/apis/default/app/",
                            "ClientID": "meRBYfhb4iR02nOoqKOvt-JwZTfX7XhnxCJ5t",
                            "Enabled": "true",
                            "Secret": "pass",
                            "UserName": "admin"
                        },
                        "SenderSettings": {
                            "CheckIntervalCronExpression": "0/1 * * * *",
                            "SendFiles": "true"
                        }
                    }
                },
                "kind": "ConfigMap",
                "metadata": {
                    "name": "server-cm",
                    "namespace": "my-apps"
                }
            },
            "server_side_apply": null,
            "src": null,
            "state": "present",
            "template": null,
            "username": null,
            "validate": null,
            "validate_certs": null,
            "wait": false,
            "wait_condition": null,
            "wait_sleep": 5,
            "wait_timeout": 120
        }
    },
    "msg": "Failed to create object: b'{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"ConfigMap in version \\\\\"v1\\\\\" cannot be handled as a ConfigMap: json: cannot unmarshal object into Go struct field ConfigMap.data of type string\",\"reason\":\"BadRequest\",\"code\":400}\\n'",
    "reason": "Bad Request"
}

PLAY RECAP *********************************************************************************************************************************************************************************************************************************************************************************
m1                         : ok=3    changed=1    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

Monkadelic-D · February 26, 2024, 4:55pm

I’ve cross posted this from the Kubernetes.core Github issue report I created.
There may be a bug this specific ConfigMap is triggering or I am just not able to find the typo/error in it.
https://github.com/ansible-collections/kubernetes.core/issues/682

Denney-tech · February 26, 2024, 6:33pm

You might try saving that config map as a jinja2 template file, and using the template parameter of kubernets.core.k8s to create the configmap instead of defining it in the task.

Monkadelic-D · February 26, 2024, 8:10pm

I haven’t used templates before but was reading about that this AM. I figured it might be a good method.
Thanks

Monkadelic-D · February 28, 2024, 3:32pm

It was suggested in the Kubernetes.core Github issues, where I posted the bug report, that I could store the data in a variable and convert it to JSON when using the variable in the ConfigMap.
From gravesm in Github:

- kubernetes.core.k8s:
    definition:
      apiVersion: v1
      kind: ConfigMap
      metadata:
        name: mymap
        namespace: default
      data:
        appsettings.json: "{{ data | to_nice_json(indent=2) }}"
  vars:
    myvar: foobar
    data: |-
      {
        "somesetting": "{{ myvar }}"
      }

Rtwolfe94022 · March 1, 2024, 3:20am

To fix the error, convert your JSON object into a string before including it in the ConfigMap. Kubernetes expects each value in the data section to be a string. If you’re using Ansible, you can use the to_nice_json filter for this conversion. Make sure to properly quote and format the string in your configuration.

Cheers,

Tim

system · March 31, 2024, 3:20am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Views
FAILED! => {"changed": false, "msg": "TemplateNotFound: anVweXRlcmh1YkRvY2tlcmh1YmNvbmZpZ2pzb24uajI="} Ansible Project kubernetes	5	April 8, 2020
Having difficulty using a generator? object? Ansible Project kubernetes	1	April 5, 2016
copy j2 file to Json on windows server Ansible Project windows	8	April 29, 2020
Trying to understand "unable to find api field in struct Container for the json field \"$setElementOrder/env\"" Ansible Developer kubernetes	1	February 24, 2020
config management play for weblogic KIE server Ansible Project	1	February 21, 2023