Kubernetes.core.k8s - Issue creating a specific configmap when value is json and contains Jinja2 variable

SUMMARY

When applying a particular ConfigMap where the data is JSON and includes Jinja2 variables I get the following error:
"json: cannot unmarshal object into Go struct field ConfigMap.data of type string, reason:BadRequest, code:400.

The JSON validates when fed to json_pp.
Executing the playbook with ANSIBLE_KEEP_REMOTE_FILES=1 and feeding the args file to json_pp also shows valid.
In the args file all Jinja2 variables are substituted with the expected values.

ISSUE TYPE
  • Bug Report
COMPONENT NAME

kubernetes.core.k8s

ANSIBLE VERSION
  ansible [core 2.14.9]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/kubeadmin/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.9/site-packages/ansible
  ansible collection location = /home/kubeadmin/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.9.18 (main, Sep  7 2023, 00:00:00) [GCC 11.4.1 20230605 (Red Hat 11.4.1-2)] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True
COLLECTION VERSION
Collection      Version
--------------- -------
kubernetes.core 3.0.0
CONFIGURATION
CONFIG_FILE() = /etc/ansible/ansible.cfg
OS / ENVIRONMENT

RHEL 9.2
RKE2 1.25.12

STEPS TO REPRODUCE

I run the following playbook with:
ansible-playbook -vvv -i inventory setup_app1.yaml
The failed message:
“msg”: “Failed to create object: b’{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"ConfigMap in version \\"v1\\" cannot be handled as a ConfigMap: json: cannot unmarshal object into Go struct field ConfigMap.data of type string","reason":"BadRequest","code":400}\n’”

If I add two back slashes ‘\’ before the first line in the json file “ConnectionStrings”: I get no error but that line is missing in the mounted file in the container after deployment.
I can use kubectl to create a ConfigMap using the same JSON. The only change I make is substituting Jinja2 variables with their corresponding values.

When running the playbook with ANSIBLE_KEEP_REMOTE_FILES=1, exploding Ansiballz_k8s.py, then inspecting the args file, I can verify the data is valid JSON.
The true/false values in the yaml JSON are not quoted. I suspected they were being interpreted as boolean so I added single quotes in the yaml JSON. The output args file on the target host has double quotes around the true/false values but I still get the same error.

No issues with variable values. This is the playbook with relevant tasks. Some names and details have been changed to protect the innocent

---
- hosts: m1
  become: true

  environment:
    APPSNAMESPACE: "{{ apps_namespace }}"
    APPSCONTEXT: "{{ apps_context }}"

  tasks:
  - name: Create the namespace
    kubernetes.core.k8s:
      name: "{{ apps_namespace }}"
      api_version: v1
      kind: Namespace
      state: present

  - name: Create contexts for my namespace
    ansible.builtin.shell: |
      kubectl config set-context $APPSCONTEXT --namespace=$APPSNAMESPACE --cluster=default  --user=default

  - name: Create configmap
    kubernetes.core.k8s:
      state: present
      namespace: "{{ apps_namespace }}"
      context: "{{ apps_context }}"
      definition:
        kind: ConfigMap
        apiVersion: v1
        metadata:
          name: "{{ server_cm_name }}"
        data:
          appsettings.json: |-
            {
              "ConnectionStrings": {
                "DefaultConnection": "User ID=pguser;Password=randompass;Host={{ db_svc_name }};Port=5432;Database=mydb;Pooling=true;"
              },
              "Logging": {
                "IncludeScopes": false,
                "LogLevel": {
                  "Default": "Warning"
                }
              },

              "SenderSettings": {
                "SendFiles": true,
                "CheckIntervalCronExpression": "0/1 * * * *"
              },
              "RemoteSettings": {
                "Enabled": false,
                "BaseUrl": "https://api.example.com/inwx6/0c8af111-e8d6-1c2a-d91c-50699bcdb1ca/",
                "UserName": "482dc608-ebd7-4989-8035-8f11ea8560c9",
                "Secret": "n06He9W9V7cDg1HMf7eoLKKr3ybbJWx3"
              },
              "OtherAppSettings": {
                "Enabled": true,
                "BaseUrl": "https://{{ app_fqdn }}/apis/default/app/",
                "ClientID": "meRBYfhb4iR02nOoqKOvt-JwZTfX7XhnxCJ5t",
                "UserName": "admin",
                "Secret": "pass"
              }, 
              "AppSsoSettings": {
                "Authority": "https://{{ sso_fqdn }}/auth/app/myappIAM",
                "ClientId": "app-sso",
                "ClientSecret": "904ef4c0-0e5e-4ee5-82ae-a38d2b0de46",
                "CallbackPath": "/Home/Index",
                "IgnoreSslErrors": true,
                "LogOutUri": "https://app.local/auth/app/myappIAM/logout?redirect_uri=https://app.local/app/Core"
              },
              "DeployDirLin": "/home/src/bin/myapp",
              "DeployDirWin": "c:\\dev",
              "AllowedHosts": "*",
              "Servicer": {
                "EndPoints": {
                  "Https": {
                    "Url": "https://0.0.0.0:5150",
                    "Certificate": {
                      "Path": "/etc/pki/tls/certs/mycertpak.pfx",
                      "Password": "randompass"
                    }
                  }
                }
              }
            }

EXPECTED RESULTS

I expect a status of changed when the task is run.

ACTUAL RESULTS

I get a failed result for the task

The full traceback is:
  File "/tmp/ansible_kubernetes.core.k8s_payload_kaauh79a/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/runner.py", line 101, in run_module
    result = perform_action(svc, definition, module.params)
  File "/tmp/ansible_kubernetes.core.k8s_payload_kaauh79a/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/runner.py", line 186, in perform_action
    instance = svc.create(resource, definition)
  File "/tmp/ansible_kubernetes.core.k8s_payload_kaauh79a/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/service.py", line 358, in create
    raise CoreException(msg) from e
fatal: [m1]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "api_key": null,
            "api_version": "v1",
            "append_hash": false,
            "apply": false,
            "ca_cert": null,
            "client_cert": null,
            "client_key": null,
            "context": "mycontext",
            "continue_on_error": false,
            "definition": {
                "apiVersion": "v1",
                "data": {
                    "appsettings.json": {
                        "AllowedHosts": "*",
                        "RemoteSettings": {
                            "BaseUrl": "https://api.example.com/inwx6/0c8af111-e8d6-1c2a-d91c-50699bcdb1ca/",
                            "Enabled": "false",
                            "Secret": "n06He9W9V7cDg1HMf7eoLKKr3ybbJWx3",
                            "UserName": "482dc608-ebd7-4989-8035-8f11ea8560c9"
                        },
                        "ConnectionStrings": {
                            "DefaultConnection": "User ID=pguser;Password=randompass;Host=mydb-svc;Port=5432;Database=mydb;Pooling=true;"
                        },
                        "DeployDirNix": "/home/src/bin/myapp",
                        "DeployDirWin": "c:\\dev",
                        "Servicer": {
                            "EndPoints": {
                                "Https": {
                                    "Certificate": {
                                        "Password": "randompass",
                                        "Path": "/etc/pki/tls/certs/mycertpak.pfx"
                                    },
                                    "Url": "https://0.0.0.0:5051"
                                }
                            }
                        },
                        "Logging": {
                            "IncludeScopes": "false",
                            "LogLevel": {
                                "Default": "Warning"
                            }
                        },
                        "AppSsoSettings": {
                            "Authority": "https://sso.example.com/auth/app/myappIAM",
                            "CallbackPath": "/Home/Index",
                            "ClientId": "app-sso",
                            "ClientSecret": "7041f4c0-0e5e-4ee5-82ae-a38d290de46e",
                            "IgnoreSslErrors": "true",
                            "LogOutUri": "https://app.local/auth/app/myappIAM/logout?redirect_uri=https://app.local/app/Core"
                        },
                        "OtherAppSettings": {
                            "BaseUrl": "https://app2.example.com/apis/default/app/",
                            "ClientID": "meRBYfhb4iR02nOoqKOvt-JwZTfX7XhnxCJ5t",
                            "Enabled": "true",
                            "Secret": "pass",
                            "UserName": "admin"
                        },
                        "SenderSettings": {
                            "CheckIntervalCronExpression": "0/1 * * * *",
                            "SendFiles": "true"
                        }
                    }
                },
                "kind": "ConfigMap",
                "metadata": {
                    "name": "server-cm",
                    "namespace": "my-apps"
                }
            },
            "delete_all": false,
            "delete_options": null,
            "force": false,
            "generate_name": null,
            "hidden_fields": null,
            "host": null,
            "impersonate_groups": null,
            "impersonate_user": null,
            "kind": null,
            "kubeconfig": null,
            "label_selectors": null,
            "merge_type": null,
            "name": null,
            "namespace": "my-apps",
            "no_proxy": null,
            "password": null,
            "persist_config": null,
            "proxy": null,
            "proxy_headers": null,
            "resource_definition": {
                "apiVersion": "v1",
                "data": {
                    "appsettings.json": {
                        "AllowedHosts": "*",
                        "RemoteSettings": {
                            "BaseUrl": "https://api.example.com/inwx6/0c8af111-e8d6-1c2a-d91c-50699bcdb1ca/",
                            "Enabled": "false",
                            "Secret": "n06He9W9V7cDg1HMf7eoLKKr3ybbJWx3",
                            "UserName": "482dc608-ebd7-4989-8035-8f11ea8560c9"
                        },
                        "ConnectionStrings": {
                            "DefaultConnection": "User ID=pguser;Password=randompass;Host=mydb-svc;Port=5432;Database=mydb;Pooling=true;"
                        },
                        "DeployDirNix": "/home/src/bin/myapp",
                        "DeployDirWin": "c:\\dev",
                        "Servicer": {
                            "EndPoints": {
                                "Https": {
                                    "Certificate": {
                                        "Password": "randompass",
                                        "Path": "/etc/pki/tls/certs/mycertpak.pfx"
                                    },
                                    "Url": "https://0.0.0.0:5150"
                                }
                            }
                        },
                        "Logging": {
                            "IncludeScopes": "false",
                            "LogLevel": {
                                "Default": "Warning"
                            }
                        },
                        "AppSsoSettings": {
                            "Authority": "https://sso.example.com/auth/app/myappIAM",
                            "CallbackPath": "/Home/Index",
                            "ClientId": "app-sso",
                            "ClientSecret": "7041f4c0-0e5e-4ee5-82ae-a38d290de46e",
                            "IgnoreSslErrors": "true",
                            "LogOutUri": "https://app.local/auth/app/myappIAM/logout?redirect_uri=https://app.local/app/Core"
                        },
                        "OtherAppSettings": {
                            "BaseUrl": "https://app2.example.com/apis/default/app/",
                            "ClientID": "meRBYfhb4iR02nOoqKOvt-JwZTfX7XhnxCJ5t",
                            "Enabled": "true",
                            "Secret": "pass",
                            "UserName": "admin"
                        },
                        "SenderSettings": {
                            "CheckIntervalCronExpression": "0/1 * * * *",
                            "SendFiles": "true"
                        }
                    }
                },
                "kind": "ConfigMap",
                "metadata": {
                    "name": "server-cm",
                    "namespace": "my-apps"
                }
            },
            "server_side_apply": null,
            "src": null,
            "state": "present",
            "template": null,
            "username": null,
            "validate": null,
            "validate_certs": null,
            "wait": false,
            "wait_condition": null,
            "wait_sleep": 5,
            "wait_timeout": 120
        }
    },
    "msg": "Failed to create object: b'{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"ConfigMap in version \\\\\"v1\\\\\" cannot be handled as a ConfigMap: json: cannot unmarshal object into Go struct field ConfigMap.data of type string\",\"reason\":\"BadRequest\",\"code\":400}\\n'",
    "reason": "Bad Request"
}

PLAY RECAP *********************************************************************************************************************************************************************************************************************************************************************************
m1                         : ok=3    changed=1    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

I’ve cross posted this from the Kubernetes.core Github issue report I created.
There may be a bug this specific ConfigMap is triggering or I am just not able to find the typo/error in it.

1 Like

You might try saving that config map as a jinja2 template file, and using the template parameter of kubernets.core.k8s to create the configmap instead of defining it in the task.

1 Like

I haven’t used templates before but was reading about that this AM. I figured it might be a good method.
Thanks

It was suggested in the Kubernetes.core Github issues, where I posted the bug report, that I could store the data in a variable and convert it to JSON when using the variable in the ConfigMap.
From gravesm in Github:

- kubernetes.core.k8s:
    definition:
      apiVersion: v1
      kind: ConfigMap
      metadata:
        name: mymap
        namespace: default
      data:
        appsettings.json: "{{ data | to_nice_json(indent=2) }}"
  vars:
    myvar: foobar
    data: |-
      {
        "somesetting": "{{ myvar }}"
      }
1 Like

To fix the error, convert your JSON object into a string before including it in the ConfigMap. Kubernetes expects each value in the data section to be a string. If you’re using Ansible, you can use the to_nice_json filter for this conversion. Make sure to properly quote and format the string in your configuration.

Cheers,

Tim

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.