The incident happened at the main reception of the Glasgow City Council headquarters in George Square.
According to sources, a man approached the main reception and asked to speak a member of the council.
A female employee asked him to hand over the weapon before calling police. Nobody was injured in the incident.
A Police Scotland spokesman said: "At around 4:15pm officers were called to a disturbance at the city chambers and a 29-year-old male was arrested. The male was in possession of a knife."
Glasgow City Council confirmed that an incident had taken place but were unable to comment.
A report will be sent to the Procurator Fiscal.