Vagina repulsion and penis repulsion are things experienced by quite a few monosexuals.
Gay men with vagina repulsion can't help not being attracted to somebody who has a vagina...and it's understandable that they might be skeptical about surgery. I once knew a gay man who told me he would consider dating a trans man, but they would have to "tick all of his other boxes" if they still had a vagina because he's just not physically attracted to vaginas.
It's not transphobic to be turned off by unattractive genitalia.
However, a lot of cis people, gay or straight alike, frame this in ways that land in transphobia. This is in part because talking about genital preferences is taboo even in "safe spaces." We just don't do it.
I personally can't speak to losing interest in somebody because they're trans but that's because I'm not straight or gay. It wouldn't slow me down at all if I wasn't thoroughly taken. But I also only have an academic understanding of "cock good, pussy bad." So I'm not somebody to ask on that one.
I suppose where I land is that if you acknowledge that trans men are men, but that they are a subset of men you don't find *physically* attractive, then that's not transphobic. I mean, plenty of cis gay guys have hair color and body type preferences too. There shouldn't be a difference.