How many auto insurance commercials have you seen where the announcer says “customers who switched to [company] saved $350 on average.” The implication is that if you call as well, you too will save a large amount of money. It seems like a great deal, but how is it that every company seems to be able to claim this? Logic would dictate, that in order for one company to claim this, the competitors must be more expensive and significantly so. If they all claim it, something is amiss.
It could be a lie, since everyone knows that advertising isn’t exactly truthful, but my guess is this is more of a statistical half-truth. I’d call it a form of survivorship bias. Let’s say you call up an insurance company and get a quote. Perhaps, $800 a year. You then compare that to your current insurance and decide whether to switch companies or not. If your current insurance is cheaper, well, you don’t switch. Because, after all, who would pay more for insurance?
Thus, anyone who would lose money doesn’t change companies. That eliminates half the population from even being considered – note the subtle wording of “customers who switched”. This isn’t customers who got a quote from the insurance company. They had to choose to switch, which meant they’re already selecting for a success story.
It’s a fun little example of how statistics lie without being an outright lie.