Trying to figure out how to import an inventory with vaulted variables from a Git Repo

Folder currently looks like this and works just find with ansible via command line (Provided that you either use --ask-vault-pass or supply the vault password some other way)

.
├── README.md

├── group_vars

│ ├── all
│ │ ├── vars.yml
│ │ └── vault.yml
│ ├── asa
│ │ ├── vars.yml
│ │ └── vault.yml
│ ├── ios
│ │ ├── vars.yml
│ │ └── vault.yml
│ ├── iosxr
│ │ ├── vars.yml
│ │ └── vault.yml
│ ├── dc1_fabric
│ │ ├── vars.yml
│ │ └── vault.yml
│ └── nexus
│ ├── vars.yml
│ └── vault.yml
├── dc1
└── mgmt

When I try to import from that project as an inventory source I get the following error message:

ERROR! Attempting to decrypt but no vault secrets found

I’m assuming I need to tell it to use the vault credentials somehow, but when I select credentials within the inventory source settings I haven’t been able to get Ansible Vault type credentials to show up there. I can get them to show up in job templates, but not in Inventory Source templates. I feel like I must be missing something really simple since I can’t be the only one doing things this way when it seems like the ansible docs are telling you to do things this way.