Drums draped with military colours were used in place of an altar to replicate services held on the front-line 100 years ago.
The multi-faith ceremony on the castle esplanade marked the start of the five-year Scottish Commemorations Programme.
It was conducted by senior chaplains from the Army, the Royal Navy and the Royal Air Force.
The service also featured troops from all three and was accompanied by music from three military bands, two cadet bands, three choirs and around 200 massed pipes and drums.
It was attended by politicians including First Minister Alex Salmond and Scottish Secretary Alistair Carmichael, civic and religious leaders, military representatives and members of the public.
At the end of the service the military bands and tri-service guards paraded down the Royal Mile accompanied by around 100 marching veterans and 100 cadets.
The congregation followed the procession to Holyrood Park, where more than 1,000 replica Commonwealth War Gravestones formed a temporary memorial, at which people were invited to leave poppies, wreaths or markers.
Brigadier David Allfrey, producer of the Royal Edinburgh Military Tattoo and chief organiser of the service, procession and memorial, said: "The multi-faith service on the esplanade of Edinburgh Castle represents the moment before deployment; the procession down the Royal Mile will symbolise a 'March as to War' and the gathering in Holyrood Park, the approach to the front and assembly for military action.
"The memorial of over 1,000 headstones will provide a vital sense of scale and a focus for acts of individual and collective commemoration."
The Scottish Commemorations Programme will remember eight particular events from the war that had a significant impact on Scotland.
They include the start and end of the war, major battles including Gallipoli, Loos, Jutland and Arras and domestic incidents such as the Quintinshill rail disaster and the loss of HMY Iolaire.
Culture Secretary Fiona Hyslop said: "The event brings the people of Scotland together to consider the impact of that brutal conflict, which claimed the lives of more than 100,000 Scots and left many more injured or disabled, forcing families and loved ones across the country to come to terms with the terrible consequences.
"From this Sunday until January 2019, we will encourage the people of Scotland and those with connections to Scotland to recognise the significant and broad impact the First World War had on our nation, and to reflect on its lasting social and civic legacy."
Edinburgh's Lord Provost and Lord Lieutenant Donald Wilson was guest of honour at the service, procession and memorial.
He said: "It is very fitting that as the capital city we should play our key part in these commemorations and show our support and gratitude for the enormous sacrifices made by courageous men and women in service of this country."
The Scottish Parliament's Presiding Officer Tricia Marwick said: "This was the war that was to end all wars. Yet, as we reflect on the great loss of lives, we recognise that the world has never achieved that peace.
"We mourn those whose died, the families who were devastated and the communities that never recovered from the loss of so many of their own.
"I am honoured to represent the Scottish Parliament today."
It's believed the drumhead service, procession and memorial was the first commemoration event of its scale to have ever been held in Scotland, with 5,000 people attending the service at the castle and thousands more watching the procession to Holyrood Park.
Mr Salmond said: "People from all walks of life and from across Scotland stood side by side today with representatives of the armed forces, veterans and cadets to remember the sacrifice made by so many in Scotland, across the Commonwealth and around the world.
"Today's events were a fitting commemoration of the appalling loss of life in the conflict which began a century ago. Over the next five years we will ask the people of Scotland to join us in remembering a further seven events from the First World War that had a particularly significant impact on Scotland.
"I would encourage the people of Scotland and those with connections to this country to recognise the significant and broad impact the Great War had on our nation, and to reflect on its lasting social and civic legacy. We must never forget."