Testosterone makes ALL mammalian males a bit larger and stronger. If you take a dog and a bitch from the same litter, the dog will be larger. It's even more noticeable if you take a tom and a queen from the same litter, even if the sire is confirmed to be the same. It's less obvious in horses, but a colt from the same parents will be a little bit larger. Testosterone also generates more fast-twitch muscles (used in sprinting) and fewer slow-twitch muscles (which is why women tend to have more stamina). This is just mammalian biology. That has then influenced us in that "men are bigger and stronger" has been used to support the idea that men should thus fight more and do the heavier physical work. But the base reason why men are physically larger and stronger is because they are male. In birds, things are quite different. Male peregrine falcons, for example, are about half the size of their mates. Evolution created a skew.

Human sexual dimorphism in terms of size is considerably lower than gorillas and orangutans. This is because human males compete for mates less...the fact that we form temporary to permanent pair bonds means that (outside of aberrations such as harems) the dominant males don't get as disproportionate a share of breeding opportunities. We are much more like chimpanzees than gorillas. One interesting study showed that while men can throw a harder punch...when it comes to overhead throwing (throwing a spear), their advantage significantly reduces.

But the tl;dr - it's in base mammalian evolution and any behavioral sexual dimorphism in humans comes from men being physically larger, not the reverse, and even then it's a minor difference.