Why isn't compound interest making us all rich?

Just let your wealth compound over time and you'll get rich, the advice goes. If true, why aren't more people millionaires?