Usually I stop after 2 or 3 attempts.
You could also ask him directly. Maybe he’s generally not interested in lunching together, or he was just too busy to respond, or whatever. These questions can be awkward, but also deepen a friendship. A risk-reward mechanic in the game of life.
Right, that was the spirit. Though to be honest, it can harm. Some people are shy. Or strongly shy away from awkward situations caused by not-yet-close-friends. They can still be great friends, but unlucky situations during the getting-to-know each other phase can prevent that. We judge people with different standards, depending on how close they are. So when you interpolate distant-behavior to predict close-behavior, there is probably some error margin.
So there is some risk involved in asking, but likewise for not asking. I guess just be you and see where it goes.
One final thought: I think it’s important to take care that the question is not perceived as an accusation.