I've read that the reason cats home in on people who aren't cat fans is because the cat fans are looking them in the eye, and they don't like that, so they go and climb on the people who aren't looking at them. I think it's pure perversity, though.