He only issued a TRO, though. I am unfortunately confident they will find a way around this.

Quite clearly. Even if you wanted to forgive his attempts at humor at your expense, his insistence on you apologizing is indicative of your feelings being beneath his. He cannot place himself in your position to realize it was hurtful, and this will escalate as time passes.