He was held at 7.15am at an address in central London and taken to a police station in the capital.
A Scotland Yard spokesman said: "Officers working on Operation Yewtree have today arrested a man in his 60s in connection with the investigation.
"The man, from London, was arrested at approximately 7.15am on suspicion of sexual offences, and has been taken into custody at a London police station.
"The individual falls under the strand of the investigation we have termed 'Savile and others'."
Glitter, whose real name is Paul Gadd, was jailed for four months in the UK in 1999 for downloading child porn and later jailed for child sex offences in Vietnam.
Scotland Yard did not say what led to his arrest, and Glitter was filmed leaving his home this morning, wearing a hat and dark coat and gloves, and being driven away.
Meanwhile the chairman of the BBC Trust, Lord Patten, has said he is dedicated to finding out the truth about the scandal that has engulfed the corporation, vowing there would be "no covering our backs".
Writing in today's Mail on Sunday Lord Patten said the BBC's reputation is on the line and that it has risked squandering the public's trust.
He said the BBC would not hide behind smokescreens, but "must tell the truth and face up to the truth about itself, however terrible".
Speaking of the late TV presenter's apparent decades of criminality, he wrote: "Can it really be the case that no one knew what he was doing? Did some turn a blind eye to criminality? Did some prefer not to follow up their suspicions because of this criminal's popularity and place in the schedules? Were reports of criminality put aside or buried? Even those of us who were not there at the time are inheritors of the shame."
He also apologised "unreservedly" to the abused women who spoke to the BBC's Newsnight programme but did not have their stories told when the report was axed.
The BBC chairman said the two independent inquiries that have been set up - one into the Newsnight report, the other into the BBC's culture and practices in the years Savile worked there - must get to the truth of what happened.
He added: "Now my immediate priority is to get to the bottom of the Savile scandal and to make any and every change necessary in the BBC to learn the lessons from our independent investigations".
Yesterday Savile's closest relatives broke their silence to say their "own despair and sadness does not compare to that felt by the victims" who were abused.
In a statement released by Savile's nephew, Roger Foster, the family said: "How could the person we thought we knew and loved do such a thing?
"Why would a man who raised so much money for charity, who gave so much of his own time and energy for others, risk it all doing indecent criminal acts? How could anyone live their life doing the 'most good and most evil' at the same time?"
He said the family could understand the victims' "reluctance to say anything earlier" and the family could "appreciate the courage it has taken to speak out now".
Following speculation that other celebrities from Savile's era could be accused of sexual offences, PR guru Max Clifford claimed dozens of big name stars from the 1960s and 70s have contacted him because they are "frightened" they will become implicated in the widening child abuse scandal.
He said the stars were worried because at their peak they had lived a hedonistic lifestyle where young girls threw themselves at them but they "never asked for anybody's birth certificate".
Savile, who died last year at the age of 84, has been described as one of the most prolific sex offenders in recent UK history.
Scotland Yard detectives are currently dealing with about 300 alleged victims and are following more than 400 lines of inquiry.
Speaking to Andrew Marr on the BBC, Labour deputy leader Harriet Harman said there needed to be an over-arching independent inquiry into the Savile case.
She said too often the criminal justice system gave the victim the impression they would not be believed if they came forward with allegations of sexual abuse.
Ms Harman said: "There are big lessons to be learned here, not just for the BBC although the epicentre of it was at the BBC, but elsewhere because when something like this comes out, there is an assumption, 'how could we have gone astray from our normal policy that protecting vulnerable children must take priority over the rights of protecting adults'?
"That is not actually the situation because that is always under challenge. Just this month the law has been changed so if there is an allegation of a sex offence against a teacher, the teacher has anonymity right up until the moment of charge.
"That is based on an assumption that you have to protect the adult from false allegation. I think what we do is that we push to protect children but that is always resisted.
"For example, every time there is a criminal justice bill in the House of Commons, MPs get up and move amendments to say those accused of sex offending should have anonymity right during the court case and that is based on an assumption and a belief that you have to disbelieve the victim and it is a false allegation.
"If there is an assumption that you won't be believed then you are less likely to come and if you do come forward and then you meet that culture of disbelief, then it is going to be swept under the carpet."
English Justice Secretary Chris Grayling said he was not convinced by the case for an immediate judge-led inquiry of the kind Ms Harman has proposed.
Mr Grayling told BBC1's Andrew Marr Show: "There is always a danger if you set up a very substantial inquiry process of that kind that it takes much longer to get to the truth.
"What should be happening right now first and foremost - and clearly is happening with the police - is we should be looking to see who is still around who was involved, and criminal proceedings should follow if people were guilty of participating in these offences alongside Jimmy Savile. That is of paramount importance."
Mr Grayling accepted that the case indicated that the authorities did not pay enough attention to the complaints of abuse victims in the past.
"I hope every single person in the law enforcement world today is accepting the fact that things went badly wrong and it should not happen again," he said.
Mr Grayling added: "Clearly what has happened is absolutely horrendous. It is shocking. There was clearly a culture that should never, ever, ever have been allowed to exist."