The death was confirmed by Villanova University, Philadelphia, but no cause was given.
Budd was fifth in the 100 metres at the 1960 Olympics in Rome. A year later, he set the world record in the 100-yard dash, finishing a race at Randall's Island in New York in 9.2 seconds.
That earned Budd the unofficial title of world's fastest man.
He later played as a wide receiver in the US and Canadian American football leagues.
Budd grew up in Asbury Park, New Jersey, and later lived in Mount Laurel.
Services are scheduled for Monday on the Villanova campus.