Avoid explaining one’s actions with the phrase “it’s not you, it’s me,” but not because it’s a cliché. It certainly is a cliché, but the generally accepted meaning is “Of course it’s you! I just won’t do you the courtesy of being honest about it.”

That is the reason to avoid the phrase.