Archbishop Philip Tartaglia told those gathered at St Mary's Cathedral in Edinburgh that he understands that the archdiocese is in a "state of shock for the loss of its shepherd" following the cardinal's resignation.
The Pope appointed the Archbishop of Glasgow to govern the Roman Catholic Archdiocese of St Andrews and Edinburgh until a permanent replacement is chosen.
Pope Benedict XVI, who steps down today after nearly eight years as leader of the world's 1.2 billion Catholics, named him apostolic administrator after Cardinal O'Brien resigned from the post on Monday amid allegations, which he denies, of inappropriate behaviour towards fellow priests.
Archbishop Tartaglia told the congregation: "I am glad to be with you today, even if I wished, very much wished, that the circumstances were other than they are.
"I want first of all to say that I appreciate that this archdiocese is in a state of shock for the loss of its shepherd. Let me assure you that I and the whole church in Scotland share your pain and dismay.
"On Tuesday evening I celebrated a Lenten Station mass in a church in Glasgow and I can tell you that the mood of the congregation was very sombre for what had happened, and I felt I needed to encourage them and try to give them new hope. And I would like to try to give you hope too by asking you to focus your gaze on Jesus Christ who is, alone, our Saviour and our Good Shepherd."
He asked them to pray for the appointment of a new Archbishop and said he did not know how long it would be before one is named.
"The office entrusted to me by the Holy Father, the office of the apostolic administrator, is by its nature provisional and temporary. It is governed by the principle 'no innovation'. The apostolic administrator does what has to be done but doesn't produce new initiatives which will commit his successor. So, it's a humble act of service and administration," he said.
"While I hold that office, I promise that I will do my best, with the help of God, to oversee and govern this archdiocese."
Archbishop Tartaglia also paid tribute to the Pope, telling worshippers they have gathered on a "truly unique day" for the Catholic Church.
The 85-year-old pontiff, the first to resign in nearly 600 years, will meet cardinals before being formally bade farewell later by Cardinal Tarcisio Bertone and other members of the Vatican's Secretariat of State in Rome.
He will then be flown by helicopter to Castel Gandolfo, the Pope's official summer retreat.
The empty office of Pope, or Sede Vacante, will be formally declared at 7pm British time.
Archbishop Tartaglia said: "The See of Peter will be vacant, not because the Pope has died but because, with a singular and unprecedented gesture of self-abasement for the good of the Church, the Holy Father will have voluntarily stepped down from his sacred office."
Cardinal O'Brien, who was Britain's most senior Catholic cleric, said he will not travel to Rome for the conclave to elect the next Pope to avoid becoming a focus of media attention. His decision has left Britain's Roman Catholics with no vote in the conclave.
He stepped down from his post a day after the Observer newspaper reported that three priests and a former priest had complained to the Vatican about alleged inappropriate behaviour by him stretching back 30 years.
A spokesman for the Scottish Catholic Media Office said the allegations are "anonymous and non-specific" and that the 74-year-old is contesting them and taking legal advice.
Cardinal O'Brien, Archbishop of St Andrews and Edinburgh since 1985, tendered his resignation to the Vatican in November, citing age and "indifferent health". He was expected to step down next month when he turns 75 but his resignation has now taken effect.