Hello,
TLDR: Does AWX support requests to nested workflow jobs called from ManageIQ? Is a request to the jobs endpoint valid when requesting the status of a workflow_job?
From ManageIQ (“MIQ”) we are calling an AWX workflow template that contains six workflows. On the AWX side everything runs without errors. In ManageIQ, after all of the workflows complete, MIQ performs a refresh of the “containing” workflow_job with AWX (presumably to get the final status). To our surprise, it says that the workflow_job doesn’t exist in our AWX instance. This answer appears to be coming from AWX.
Here’s the error in MIQ which says the job doesn’t exist:
[----] E, [2021-05-10T09:22:38.349852 #4003:3cd0034] ERROR -- : Q-task_id([r286_service_template_provision_task_589]) Method STDERR: (drbunix:///tmp/automation_engine20210510-4003-rbf8wu) /usr/local/lib/ruby/gems/2.5.0/bundler/gems/manageiq-providers-ansible_tower-cdc04865ff9a/app/models/manageiq/providers/ansible_tower/automation_manager/workflow_job.rb:84:inrescue in refresh_ems’: AnsibleTower Workflow Job postgres soup to nuts with id(175) does not exist on awx-qa Automation Manager (MiqException::MiqOrchestrationStackNotExistError) (DRb::DRbRemoteError) `
There’s a separate log in MIQ that tells us that this workflow_job with id 175 in MIQ corresponds to workflow Job 18847 in AWX:
[----] E, [2021-05-10T09:22:38.343391 #4003:b57e8dc] ERROR -- : Q-task_id([r286_service_template_provision_task_589]) The following error occurred during instance method <refresh_ems> for AR object <#<ManageIQ::Providers::AnsibleTower::AutomationManager::WorkflowJob id:175, name: "postgres soup to nuts", type: "ManageIQ::Providers::AnsibleTower::AutomationManag...", description: nil, status: "successful", ems_ref: "18447", ancestry: nil, ems_id: 5, orchestration_template_id: 38, created_at: "2021-05-10 16:17:09", updated_at: "2021-05-10 16:22:35", retired: nil, retires_on: nil, retirement_warn: nil, retirement_last_warn: nil, retirement_state: nil,retirement_requester: nil, status_reason: nil, cloud_tenant_id: nil, resource_group: nil, start_time: "2021-05-10 16:17:08", finish_time: "2021-05-10 16:22:21", configuration_script_base_id: nil, verbosity: nil, hosts: nil, evm_owner_id: nil, miq_group_id: nil, tenant_id: nil>>
We can clearly see in AWX that workflow job id 18847 exists. On the AWX side we can see that when performing this refresh, MIQ is calling the jobs endpoint. We suspected that MIQ should instead be calling the workflow_jobs endpoint (meaning a bug in MIQ). However our friends at RedHat tell us that jobs is a superset of workflow_jobs so the request to jobs should be valid.
Does anyone have any insight? Here’s a link to a gist containing more of the log context: https://gist.github.com/mikebutak/0d4c8c0629dbadf1b55ae055095eebec