I've pushed some changes that may make you happy.
For what it's worth, here is my host file I use for testing that shows
the host and port override syntax.
[main]
127.0.0.1
[delegates]
dnehost ansible_ssh_host=127.0.0.1 ansible_ssh_port=1234
You can of course still use host:port syntax on single hosts like so,
though if you set ansible_ssh_host it's referring to the port on
*THAT* host.
[somegroup]
example:1234