Is anyone else put off by Caleb?

I've been catching up on episodes and is anyone else kind of not liking Caleb's jokes? It feels almost as if he is just trying to be offensive without offering much insight into the episode topic. I can't remember which episode it was exactly, but I remember in one of the earlier episodes he was on he made a pretty tasteless Chinese accent joke that has put me off since.