Looking for some help on first contribution (amazon.aws rds_cluster)

Hey everyone! I am working on my first contribution and, after banging my head and asking around, I am out of ideas. I have a PR open (Add Performance Insights and Database Insights to rds_cluster.py by randoneering · Pull Request #2543 · ansible-collections/amazon.aws · GitHub) and I still need to add some integration testing for the new parameters I am trying to add (enabling performance insights at the cluster level and database insights at cluster level). I am hitting to road blocks and I am hoping someone could be another set of eyes to help me along.

  1. I am hitting an error with the documentation check. I have gone through and manually deleted rows to make sure I didn’t somehow carry over hidden unicode characters, but I continue to get errors each time I fix whatever the new offending lines pop on on the error. I have done this twice and want to make sure there isn’t a better way to find this wonderful hidden gems…
Invoke build script '/home/runner/work/_temp/docsbuild/head/build.sh'
  ~/work/_temp/docsbuild/head ~/work/amazon.aws/amazon.aws
  Found errors in some modules or plugins:
  amazon.aws.rds_cluster module: Missing documentation or could not parse documentation: amazon.aws.rds_cluster did not contain a DOCUMENTATION attribute (/home/runner/work/amazon.aws/amazon.aws/ansible_collections/amazon/aws/plugins/modules/rds_cluster.py). Unable to parse documentation in python file '/home/runner/work/amazon.aws/amazon.aws/ansible_collections/amazon/aws/plugins/modules/rds_cluster.py': while scanning a simple key
        in "<unicode string>", line 86, column 11
      could not find expected ':'
        in "<unicode string>", line 87, column 11. while scanning a simple key
        in "<unicode string>", line 86, column 11
      could not find expected ':'
        in "<unicode string>", line 87, column 11
  ~/work/amazon.aws/amazon.aws

  1. The second is an error I deal with from time to time when writing playbooks for work and a variable that I am passing through is empty. However, I have read through the code I added and I am not sure what I could be possibly missing?
  File "/var/folders/xf/9g1j8gwx46g13dyjw6_fnj8c0000gp/T/ansible_amazon.aws.rds_cluster_payload_v_sx5ebx/ansible_amazon.aws.rds_cluster_payload.zip/ansible_collections/amazon/aws/plugins/modules/rds_cluster.py", line 1136, in changing_cluster_options
KeyError: 'EnablePerformanceInsights'
fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/Users/jfrye/.ansible/tmp/ansible-tmp-1741219030.4587781-50196-31706941389543/AnsiballZ_rds_cluster.py\", line 107, in <module>\n    _ansiballz_main()\n    ~~~~~~~~~~~~~~~^^\n  File \"/Users/jfrye/.ansible/tmp/ansible-tmp-1741219030.4587781-50196-31706941389543/AnsiballZ_rds_cluster.py\", line 99, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/jfrye/.ansible/tmp/ansible-tmp-1741219030.4587781-50196-31706941389543/AnsiballZ_rds_cluster.py\", line 47, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.amazon.aws.plugins.modules.rds_cluster', init_globals=dict(_module_fqn='ansible_collections.amazon.aws.plugins.modules.rds_cluster', _modlib_path=modlib_path),\n    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                     run_name='__main__', alter_sys=True)\n                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"<frozen runpy>\", line 226, in run_module\n  File \"<frozen runpy>\", line 98, in _run_module_code\n  File \"<frozen runpy>\", line 88, in _run_code\n  File \"/var/folders/xf/9g1j8gwx46g13dyjw6_fnj8c0000gp/T/ansible_amazon.aws.rds_cluster_payload_v_sx5ebx/ansible_amazon.aws.rds_cluster_payload.zip/ansible_collections/amazon/aws/plugins/modules/rds_cluster.py\", line 1495, in <module>\n  File \"/var/folders/xf/9g1j8gwx46g13dyjw6_fnj8c0000gp/T/ansible_amazon.aws.rds_cluster_payload_v_sx5ebx/ansible_amazon.aws.rds_cluster_payload.zip/ansible_collections/amazon/aws/plugins/modules/rds_cluster.py\", line 1473, in main\n  File \"/var/folders/xf/9g1j8gwx46g13dyjw6_fnj8c0000gp/T/ansible_amazon.aws.rds_cluster_payload_v_sx5ebx/ansible_amazon.aws.rds_cluster_payload.zip/ansible_collections/amazon/aws/plugins/modules/rds_cluster.py\", line 1235, in ensure_present\n  File \"/var/folders/xf/9g1j8gwx46g13dyjw6_fnj8c0000gp/T/ansible_amazon.aws.rds_cluster_payload_v_sx5ebx/ansible_amazon.aws.rds_cluster_payload.zip/ansible_collections/amazon/aws/plugins/modules/rds_cluster.py\", line 1136, in changing_cluster_options\nKeyError: 'EnablePerformanceInsights'\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE: No start of json char found\nSee stdout/stderr for the exact error",
    "rc": 1
}

Thank you @alinabuzachis for helping me get this far (and gently reminding me there are documents/guides on contributions ).

I am happy to chat through this forums and provide ANY information I can.

Regarding 1, you added and removed some spaces in an unrelated option that made it invalid YAML. I’ve added a comment. (I usually look at the diff view: Add Performance Insights and Database Insights to rds_cluster.py by randoneering · Pull Request #2543 · ansible-collections/amazon.aws · GitHub - there it’s usually easier to see whether something unrelated was changed.)

Regarding 2: it looks like something is trying to get a key called EnablePerformanceInsights from a dictionary that does not have it. The stack trace tells you where that happens (plugins/modules/rds_cluster.py, line 1136 - which does not correspond to the current state of Add Performance Insights and Database Insights to rds_cluster.py by randoneering · Pull Request #2543 · ansible-collections/amazon.aws · GitHub). Since the stack trace doesn’t match the version on GitHub it’s hard to say what’s going wrong, but looking at the changes it’s most likely this line: if enable_performance_insights != current_cluster["EnablePerformanceInsights"]:. For some reason current_cluster does not seem to have an entry called EnablePerformanceInsights in these cases.

You might want to print current_cluster somehow (log it to a file, for example using the q module: Client Challenge), to see what it actually contains, especially in these cases.

1 Like

Felix, thank you for the insight. I will hopefully take a look this weekend. I already added your first suggestions but it looks like we still have an issue with the formatting.

I will have to login to my work laptop to run my test playbook against the collection to see if I can get a more accurate error. In addition, I will add some print function to output what value is present in current_cluster.

Again, thank you for you help. I really appreciate it.

I spotted another problem in the docs. I also triggered the tests (since you haven’t contributed yet someone with appropriate access has to do that; luckily I still have enough rights for the community.aws repo :slight_smile: )

As you can see the unit tests are also failing; this is rather unsurprising, since they will use some mock data, which likely does not contain the keys you’re throwing at it :wink:

Thank you! I appreciate it. It was frustrating initially with the contribution because I spend most of my days at work inside ansible, terraform, and python for my jobby job as a Senior Database Engineer (and, of course, in databases). So when I couldn’t understand what I was doing wrong with the docs and then the key error, I was pretty disappointed

So I appreciate the help and friendly suggestions. I have another feature request open for creating a separate module for DocumentDB and want to be able to work/assist with that. But I need to get my feet wet contributing to this project with this PR :smile:

I have time now and will take a look.

I understand that frustration, I’ve also spent enough time in my life trying to track down broken documentation YAML. Sometimes it helps to copy the contents of DOCUMENATION to a separate YAML file and let your editor’s YAML syntax highlighting help you.

(It would be great to have syntax highlighting for Ansible modules/plugins that uses YAML highlighting for the YAML parts and Python highlighting for the Python parts, but that seems to be hard to achieve. Also switching to sidecar docs would solve that problem as well, since then the YAML parts would be in a separate YAML file, but that’s still not supported for Python modules IIRC.)

Okay-I did some work on the weekend, but I have to step away. I did get somewhere, but just wanted to post the findings for now. I created a branch in my repo called “local_testing” if you want to see my shenanigans. Ultimately, was finally able to get some debugging going and I think I can work with what I have so far.

KeyError: 'EnablePerformanceInsights'
fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/Users/jfrye/.ansible/tmp/ansible-tmp-1742761498.056521-2352-126508465448759/AnsiballZ_rds_cluster.py\", line 107, in <module>\n    _ansiballz_main()\n  File \"/Users/jfrye/.ansible/tmp/ansible-tmp-1742761498.056521-2352-126508465448759/AnsiballZ_rds_cluster.py\", line 99, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/Users/jfrye/.ansible/tmp/ansible-tmp-1742761498.056521-2352-126508465448759/AnsiballZ_rds_cluster.py\", line 47, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.amazon.aws.plugins.modules.rds_cluster', init_globals=dict(_module_fqn='ansible_collections.amazon.aws.plugins.modules.rds_cluster', _modlib_path=modlib_path),\n  File \"<frozen runpy>\", line 226, in run_module\n  File \"<frozen runpy>\", line 98, in _run_module_code\n  File \"<frozen runpy>\", line 88, in _run_code\n  File \"/var/folders/xf/9g1j8gwx46g13dyjw6_fnj8c0000gp/T/ansible_amazon.aws.rds_cluster_payload_gvlusim6/ansible_amazon.aws.rds_cluster_payload.zip/ansible_collections/amazon/aws/plugins/modules/rds_cluster.py\", line 1510, in <module>\n  File \"/var/folders/xf/9g1j8gwx46g13dyjw6_fnj8c0000gp/T/ansible_amazon.aws.rds_cluster_payload_gvlusim6/ansible_amazon.aws.rds_cluster_payload.zip/ansible_collections/amazon/aws/plugins/modules/rds_cluster.py\", line 1486, in main\n  File \"/var/folders/xf/9g1j8gwx46g13dyjw6_fnj8c0000gp/T/ansible_amazon.aws.rds_cluster_payload_gvlusim6/ansible_amazon.aws.rds_cluster_payload.zip/ansible_collections/amazon/aws/plugins/modules/rds_cluster.py\", line 1248, in ensure_present\n  File \"/var/folders/xf/9g1j8gwx46g13dyjw6_fnj8c0000gp/T/ansible_amazon.aws.rds_cluster_payload_gvlusim6/ansible_amazon.aws.rds_cluster_payload.zip/ansible_collections/amazon/aws/plugins/modules/rds_cluster.py\", line 1149, in changing_cluster_options\nKeyError: 'EnablePerformanceInsights'\n",
    "module_stdout": "we are in changing_cluster_option function, and the cluster is {'AllocatedStorage': 1, 'AvailabilityZones': ['us-east-1c', 'us-east-1d', 'us-east-1a'], 'BackupRetentionPeriod': 14, 'DatabaseName': 'foo', 'DBClusterIdentifier': 'mycluster-qa', 'DBClusterParameterGroup': 'mycluster-qa-parameter-group', 'DBSubnetGroup': 'my-subnet-group', 'Status': 'stopped', 'AutomaticRestartTime': datetime.datetime(2025, 3, 28, 23, 43, 2, 575000, tzinfo=tzutc()), 'EarliestRestorableTime': datetime.datetime(2025, 3, 7, 3, 42, 2, 10000, tzinfo=tzutc()), 'Endpoint': 'mycluster-qa.cluster-some.region.rds.amazonaws.com', 'ReaderEndpoint': 'mycluster-qa.cluster-ro-some.region.rds.amazonaws.com', 'MultiAZ': False, 'Engine': 'aurora-postgresql', 'EngineVersion': '15.7', 'LatestRestorableTime': datetime.datetime(2025, 3, 21, 23, 34, 46, 219000, tzinfo=tzutc()), 'Port': 5432, 'MasterUsername': 'randoneering', 'PreferredBackupWindow': '03:00-05:00', 'PreferredMaintenanceWindow': 'sun:05:00-sun:06:00', 'ReadReplicaIdentifiers': [], 'DBClusterMembers': [{'DBInstanceIdentifier': 'mycluster-1', 'IsClusterWriter': True, 'DBClusterParameterGroupStatus': 'in-sync', 'PromotionTier': 1}], 'VpcSecurityGroups': [{'VpcSecurityGroupId': 'sg-12341234123412', 'Status': 'active'}], 'HostedZoneId': 'redacted', 'StorageEncrypted': True, 'KmsKeyId': 'somenumber-id', 'DbClusterResourceId': 'cluster-redacted', 'DBClusterArn': 'arn:aws:rds:us-east-1:123412341234:clusfter:mycluster-qa', 'AssociatedRoles': [], 'IAMDatabaseAuthenticationEnabled': False, 'ClusterCreateTime': datetime.datetime(2024, 8, 13, 16, 39, 50, 699000, tzinfo=tzutc()), 'EnabledCloudwatchLogsExports': ['instance', 'postgresql'], 'EngineMode': 'provisioned', 'DeletionProtection': True, 'HttpEndpointEnabled': False, 'ActivityStreamStatus': 'stopped', 'CopyTagsToSnapshot': True, 'CrossAccountClone': False, 'DomainMemberships': [{'Domain': 'd-9123412341234', 'Status': 'kerberos-enabled', 'FQDN': 'my.corp.com', 'IAMRoleName': 'rds-directoryservice-kerberos-access-role'}], 'TagList': [{'Key': 'app', 'Value': 'clusterdb'}, {'Key': 'owner', 'Value': 'me@you.com'}, {'Key': 'MultiAvailZones', 'Value': 'False'}, {'Key': 'Backup', 'Value': 'saywhat'}, {'Key': 'terraform', 'Value': 'true'}, {'Key': 'env', 'Value': 'myenv'}, {'Key': 'LightsOut', 'Value': 'True'}], 'AutoMinorVersionUpgrade': True, 'DatabaseInsightsMode': 'standard', 'PerformanceInsightsEnabled': True, 'PerformanceInsightsKMSKeyId': 'arn:aws:kms:us-east-1:123412341234:key/somenumbers', 'PerformanceInsightsRetentionPeriod': 7, 'NetworkType': 'IPV4', 'AwsBackupRecoveryPointArn': 'arn:aws:backup:us-east-1:123412341234:recovery-point:continuous:cluster-redacted-412341234123', 'EngineLifecycleSupport': 'open-source-rds-extended-support'}\nwe are in changing_cluster_option function, and the cluster is mycluster-qa\nmycluster-qa\n",
    "msg": "MODULE FAILURE: No start of json char found\nSee stdout/stderr for the exact error",
    "rc": 1
}

I have a bunch of questions, but the biggest one is, for whatever reason, I do not see EnablePerformanceInsights at all in the changing params and, my genius self, is using a cluster with it already on. So I will get a cluster tomorrow that doesn’t have this going, but still, we SHOULD still see the values coming in here. We are not. We can see all the other ones coming through (insights period, database insights mode, etc), but not the enabling performance insights. So, I think pointing out the fact that the module cannot see/access that value means the codes incorrect in the “changing_cluster_options” function.

Also, maybe I am crazy, but for the “changing_cluster_options” function, the input variables are “modify_params, current_cluster”. I printed current_cluster for debugging and that actually brought in a list of params for the target cluster. I just assumed it would be the current target cluster. Either way, I think I know what I need to look at tomorrow during work. Unless, you spot anything obvious that I am missing, I should get some more info tomorrow. Thanks!

AH! I think I figured it out. Some how I mixed up the parameter names from rds_instance vs rds_cluster for enabling performance insights. I swear I looked at both and they were the same, but maybe I didn’t. The key difference in the documentation for boto3 is…

rds_instance (EnablePerformanceInsights)

    DatabaseInsightsMode='standard'|'advanced',
    EnablePerformanceInsights=True|False,
    PerformanceInsightsKMSKeyId='string',
    PerformanceInsightsRetentionPeriod=123,

rds_cluster (PerformanceInsightsEnabled)

          'DatabaseInsightsMode': 'standard'|'advanced',
          'PerformanceInsightsEnabled': True|False,
          'PerformanceInsightsKMSKeyId': 'string',
          'PerformanceInsightsRetentionPeriod': 123,

But, of course, ALL the other related params are the exact same. I just committed the change since I am certain that is the issue, but I will test in the morning. Additionally, I started writing integration tests to do the following:

  1. Create cluster with performance insights enabled and database insights set to standard.

  2. Modify existing cluster to change database insights to “advanced”

  3. Disable performance Insights on existing cluster

  4. Set database insights back to “standard”

  5. Delete cluster

Should fulfill integration testing.

Scratch that…its too late and I realized that was for describing the rds_cluster. So I had it right-so it has to be something else.

Spent plenty of time today at work on this. Got through several bits, but I think I found a bug with boto3 and/or awscli.

Here is me going to one of my ansible runners to test a theory that this is an actual boto3 bug:

[server]$ aws rds modify-db-cluster --db-cluster-identifier ansible-test --performance-insights-retention-period 465 --database-insights-mo
de advanced --region us-east-1


An error occurred (InvalidParameterCombination) when calling the ModifyDBCluster operation: To enable the Advanced mode of Database Insights, modify your cluster to enable Performance Insights and set the retention period for Performance Insights to at least 465 days.

We can see that even when passing the required param value for retention of 465 (or higher), I still receive the error. So I removed the database insights parameter and ensured we were setting the retention correctly: (removed info not needed in output)

[server]$ aws rds modify-db-cluster --db-cluster-identifier ansible-test --performance-insights-retention-period 465 --region us-east-1
{
   ...
       "DBClusterIdentifier": "ansible-test",
       "DBClusterParameterGroup": "default.aurora-postgresql16",
       "DatabaseInsightsMode": "standard",
       "PerformanceInsightsEnabled": true,
       "PerformanceInsightsKMSKeyId": "redacted",
       "PerformanceInsightsRetentionPeriod": 465,
       "NetworkType": "IPV4",
       "LocalWriteForwardingStatus": "disabled",
       "EngineLifecycleSupport": "open-source-rds-extended-support-disabled"
   }
}

Now, to cover my basis, I check to verify that the required 465 or higher value is present for retetion. (removed info not needed from output)

[server]$ aws rds describe-db-clusters --db-cluster-identifier ansible-test --region us-east-1
{
   "DBClusters": [
       {
           "AllocatedStorage": 1,
           "AvailabilityZones": [
               "us-east-1c",
               "us-east-1d",
               "us-east-1a"
           ],
           "Status": "available",
           "MultiAZ": false,
           "DatabaseInsightsMode": "standard",
           "PerformanceInsightsEnabled": true,
           "PerformanceInsightsKMSKeyId": "redacted",
           "PerformanceInsightsRetentionPeriod": 465,
           "NetworkType": "IPV4",
           "LocalWriteForwardingStatus": "disabled",
           "EngineLifecycleSupport": "open-source-rds-extended-support-disabled"
       }
   ]
}

After this, and the same error accord as mentioned above. Therefore, I decided to try and upgrade boto3, botocore, and awscli on my laptop and the ansible runner server. Same error. Current version:

$ aws --version
aws-cli/2.25.2 Python/3.12.9 Linux/5.14.0-503.15.1.el9_5.x86_64 exe/x86_64.rhel.9

Good news is, if we wanted to, I can just exclude the database insights part and we can stick with just adding performance insights until the bug is resolved. I’ll get a ticket open in the boto3 repo. What you all think?

And as far as integration tests go…is it preferred that I modify an existing set of integration tests? Or should I create a single playbook to test, specially, the scenarios of enabling/disabling performance insights, using a specific KMS key?