Philip Tartaglia will govern the Roman Catholic Archdiocese of St Andrews and Edinburgh until a permanent successor is appointed.
The pope named him as apolistic administrator after Cardinal O'Brien stepped down from the post on Monday amid allegations of "inappropriate" behaviour towards fellow priests.
Cardinal O'Brien has denied the allegations and is taking legal advice.
Archbishop Tartaglia said: "These are painful and distressing times for the archdiocese. I also feel pained and distressed.
"With the grace of God, I will do my very best to oversee and govern the archdiocese until the appointment of a new archbishop. I ask for your prayers."
With Cardinal O'Brien's resignation from the post, the Archdiocese is known as a "vacant see" and remains so until the appointment of a new archbishop.
The apostolic administrator governs with the authority, obligations and rights of a diocesan bishop until then.
Cardinal O'Brien is now Archbishop Emeritus of St Andrews and Edinburgh and has no role in the governance of the diocese.
A statement from the Scottish Catholic Media Office said: "The Holy Father, Pope Benedict XVI, has appointed the Most Rev Philip Tartaglia, Archbishop of Glasgow, as apostolic administrator of the Archdiocese of St Andrews and Edinburgh.
"With immediate effect, Archbishop Tartaglia will govern the Archdiocese of St Andrews and Edinburgh until a new Archbishop is appointed."
Cardinal O'Brien, who was Britain's most senior Catholic cleric, said he will not travel to Rome for the conclave to elect the next pope to avoid becoming a focus of media attention.
His decision has left Britain's Roman Catholics with no vote in the conclave.
He stepped down from his post a day after The Observer newspaper reported that three priests and a former priest had complained about him to the Vatican over alleged "inappropriate" behaviour stretching back 30 years.
A spokesman for the media office said the allegations are "anonymous and non-specific" and that the 74-year-old former cardinal is contesting them and taking legal advice.
He tendered his resignation to the Vatican in November, citing age and "indifferent health".
He was expected to step down next month when he turns 75, but his resignation has now taken effect.
Cardinal O'Brien, who had been Archbishop of St Andrews and Edinburgh since 1985, was created cardinal by Pope John Paul II in 2003.
He became prominent for his colourful and strongly voiced defence of conservative Catholic teaching. His opposition to gay marriage earned him "Bigot of the Year" award last year from gay rights group Stonewall.
Archbishop Tartaglia will celebrate mass at St Mary's Cathedral in Edinburgh tomorrow.
In a message to the Archdiocese of St Andrews and Edinburgh he said he was "so sorry" for everyone caught up in the recent turmoil in the church in Scotland.
He said: "These are painful and distressing times for this venerable Archdiocese. You have lost your Cardinal Archbishop in the most difficult of circumstances. I am so sorry for everyone involved and I assure them of my prayers.
"I too feel pained and distressed. The people of the Archdiocese are having to bear the impact of these sad events as you go about your daily lives in your communities and at work. You have to cope with disturbing media reports and you have to face the questions, the critical comments, the unkind remarks and the jibes."
The Archbishop said he has appointed Bishop Stephen Robson to oversee the day-to-day running of the Edinburgh Archdiocese until a permanent appointment is made.
Archbishop Tartaglia added: "I want you to know that Bishop Robson, the priests of the Archdiocese and I are one with you in these unfortunate circumstances, and thank you for your faithfulness and love of the Church.
"At this time, we need more than ever to keep our eyes fixed on Jesus Christ who alone is our Saviour, our Good Shepherd and our Consoler.
"With the grace of God, I will do my very best to oversee and govern the Archdiocese until the appointment of a new Archbishop."