The 46-year-old manager was already leaving the club at the end of the season, but was told to go earlier in the wake of Sunday's 4-3 loss to minnows Sassuolo.
Allegri's departure brings to an end his three-and-a-half-year stay with the Rossoneri, with the clear highlight being their 2011 Serie A win.
"AC Milan announce they have dismissed as coach of the first team, with immediate effect, Mr Massimiliano Allegri and his staff," read a Milan statement.
"AC Milan would like to thank Mr Allegri and his staff for the work done and their best professional achievements."
The short release added that assistant coach Mauro Tassotti would take control, although he is expected to soon be replaced by a permanent manager.
Filippo Inzaghi, the 40-year-old former Italy and Milan striker, currently coaching the club's Primavera (under-19) side is the favourite to take charge.
The loss to Sassuolo proved to be the final straw for Milan's hierarchy, with Allegri undone by one of his former clubs.
That loss alone was not the trigger, though, with Milan woefully off the pace in Serie A. They are 30 points behind leaders Juventus and are actually just six points clear of the drop zone.
Allegri leaves the San Siro with league finishes of first, second and third, with the Supercopa added to his 2011 title win.