Nobody is saying gay men have to date trans men or lesbians have to date trans women, but...
If a guy with a vagina isn't your type, that's perfectly fine.
I had a cis gay friend years back who basically admitted. "Vaginas disgust me. I would thus find it very hard to date a trans man. He would have to be exceptional in other ways."
So, what he was saying is that while he is vagina repulsed (which is a thing), he still acknowledges that a trans man is a man and uses the correct pronouns AND admits that he might meet a trans man who would be so good in other ways that he could overlook that pesky vagina. (Obviously he was talking about pre op trans men specifically here).
And that is absolutely fine.
What is not fine is "Vaginas disgust me, so I won't date females even if they pretend to be men."
Difference!