At what point should you distance yourself from someone?

EDIT: thanks for the feedback!! I don't know why it didn't click for me before. copy pasted from one of my replies: I've always thought that "if someone doesn't make you feel happy to be around, then it's fine, you're still friends with them". And "if they're the person that talks to you when other people don't, then they're the person to rely on."