You should ask him to tone it down. It would really suck if this one little thing turned into the snowball that ends your relationship, especially when it’s something he could have fixed had he been aware of it, he probably thinks it’s really romantic.
Misunderstandings happen. But if you let them build up, they turn into little monsters that affect you long after the relationship that generated them is over.