Theres a very simple answer to that: The things you are mentioning (height, weight, arm-span, testosterone levels) are not normally distributed in the human population, but follow a binormal distribution. It makes sense to let people from one of the two parts of the distribution compete within their own bell curve. It makes far less sense to set a cutoff at the tail of the curve and disallow people from the tail from competing.