People want different things out of their time with others. I like to think of it we all have different parts of our brains and we want to express certain parts of them at certain times. When we’re young, the sex part of our brain usually predominates and so you’ll get the behavior where if a guy doesn’t have sexual access to you, he feels like he’s wasting his time because he’s not getting what he wanted.
I was never like this. Sex to me was something that turned ordinarily smart humans into idiots. In my junior year I connected with a girl and we spent much of the school year together, kind of boyfriend / girlfriend but we never kissed or did anything sexual other than touch each other. Her discomfort with sex fit well with my discomfort. She moved away eventually, but not before setting a pattern for me that continues to this day.
We met one more time sometime in my senior year before I graduated. I had awoken to the idea of sex, sensed that I had nothing to lose because I wasn’t ever going to see her again, and let my friends talk me into trying to make a move on her. It went as poorly as you would have expected it to. She wondered what had happened to the boy she used to know. I felt bad about it later. I became the very idiot that I saw sex turn people into. I never saw her again.
Your friend may well be uncomfortable with the idea of having sex with someone and might feel more comfortable with the type of friendship you can only see yourself as having with him.
He could also be seeing himself as putting in the ‘work’ he needs to eventually sleep with you. This would mean that he’ll eventually become dissatisfied with your unwillingness to sleep with him and cause lots of stress in your life down the road. I don’t think this is the case because you say he’s always respectful of your wishes.