The A-listing is the highest level of recognition for buildings, deeming the building in Pollok Park to be of national or international importance.
However, both the heritage body and Glasgow Life, which runs the city's museums and galleries for Glasgow City Council, said the listing will not hinder the city in plans to revamp the venue.
Glasgow Life and the council are altering the Sir William Burrell's 1944 Deed of Gift so his famous art collection can tour internationally.
This will raise funds for the renovation of the 1983 building, designed by Barry Gasson Architects, which requires millions of pounds worth of restoration.
The four-storey building was opened after five years of construction in 1983 and houses the paintings, sculptures, decorative art and antiquities amassed by the shipping magnate and philanthropist.
It is regarded as a "rare and significant post-war commission for a museum building in Scotland and is an influential example of a large public architectural commission in the UK."
Historic Scotland says the building is an "outstanding bespoke museum commission of international importance, and an example of Structuralist tendency in architecture in the second half of the 20th century, making the most of the interior and exterior interface with the surrounding landscape."
The council is investigating how much it will cost to restore the building in a revamp due to take four years. The building will close between 2016 and 2020. The roof needs work, as do other environmental controls.
The listing does mean that any renovation of the building by Glasgow Life will be done with the close support and advice of Historic Scotland.
Culture Secretary Fiona Hyslop said: "The Burrell Collection is one of Glasgow and Scotland's most impressive buildings of its period. The A-listing for the Burrell Collection is a fitting tribute and recognises the significant contribution it has made to Glasgow's landscape."
Councillor Archie Graham, chairman of Glasgow Life, said: "It says everything about the Burrell Collection building that is regarded so highly in its own right and is often named as one of Scotland's best."
Clare Price, conservation advisor with the national amenity organisation The Twentieth Century Society, said: "We are delighted by Historic Scotland's endorsement of the significance of this pioneering building."