I think appearance does impact this. I’m tallish (same height as average man here) and slim straight build, not the plush womanly figure and yes, people seem quicker to assume I am smart or strong than ladies who are shorter/busty/curvy. Which has got to be infuriating to be on the other side of.

I did read once that androgynous-ish people and those who disregard sex roles are happier, on average, than people who hold tightly to sex roles. Maybe this is part of why, we get treated more as a person than simply man or woman.