I'm going to take a small issue with this.

An internal sense of gender (knowing whether you are a boy or a girl...or don't fit into either) typically starts to develop at about the age of 3, *long* before puberty.

It's secondary sexual characteristics that develop at puberty, but gender is there pretty early, supporting early gender socialization.

That said, you're right in base that sexual phenotype only really matters for reproduction.