It's less attractive to some girls. Other girls will like you more than they would a gym rat. It really doesn't make any sense to worry about the body God gave you is good enough for women or not. It's not unattractive to everybody, and nobody is attractive to everybody. It's good to work out and be