This was a point of major confusion -- why read part of the local SSH config when we don't read ALL of it? This was much discussed and we ultimately resolved to not read the config file.
So, for now, we're reading nothing from it, and will continue to read nothing from it.
Express your port in the inventory file with "hostname:port" or use the ansible_ssh_port variable OR express the port in your playbook.
The only thing we lose right now is the ability to specify what SSH keys to use, and I'll accept patches to get that read from the inventory file following the pattern that is used for 'ansible_ssh_port', let's call this 'ansible_keyfiles' and it probably should be a hash/dictionary of usernames to keyfiles.
I don't have a good reason to test this, so it's better if someone with the actual need of different keys per host, etc, implements it.
--Michael