The badly decomposed body was found by a dog walker on the Corstorphine Hill nature reserve in Edinburgh on Thursday afternoon.
There is speculation that the body could be that of Suzanne Pilley, who went missing in May 2010. David Gilroy was convicted of her murder in April last year, but her body has never been found.
The grave was found just over a mile from Gilroy's home.
Detectives investigating the case have said they believe it is unlikely the body had been there for "many years" and it could be a "matter of months". Police are thought to believe that the body is not of anyone known to be currently missing.
Yesterday, Police Scotland said the identity of the body was still unknown and they were still carrying out relevant checks.
As well as police forensic teams, anthropologists, archaeologists and entomologists have been drafted in to help with the excavation of the remains.
Detective Chief Inspector Keith Hardie said the process could take some time.
"You're talking a good two or three days before we complete the recovery," he said. "Basically we have human remains and bones that are exposed to the elements and, in the circumstances we have found them in, it makes sense to excavate the area. So we need to carry out a thorough investigation."
Hardie said a member of public had discovered the grave and police were satisfied it was human remains.
"One of the possibilities we are considering is an animal perhaps dug up, or at least exposed, the ground around the body parts.
"It's far too early to even say a crime has been committed. It is very early stages."
He also spoke of the difficulties in trying to identify the body.
"Our number one priority here is to recover the human remains forensically so that we don't lose any evidential possibilities," he said. "Then we will work on identifying the victim before finding out the cause of death.
"It's very difficult. If we can obtain DNA then that's a good starting point for us. First and foremost it's about the recovery of the body itself.
"But after that it can be something that can happen virtually overnight, or it can take months and months."
Hardie appealed for anyone with information relating to the remains to contact the police.
It is understood officers have visited the home of Suzanne Pilley's parents to inform them of the discovery as a matter of course.
In March this year Gilroy failed in his bid to have his claim he is the victim of a miscarriage of justice reviewed by the UK Supreme Court.
He was jailed for a minimum of 18 years in April 2012 for killing the 38-year-old bookkeeper, who was his former lover and colleague. She had set off for work in Edinburgh but never made it to her desk.
Prosecutors believe Gilroy killed her in the basement of the building where they worked and buried her the next day somewhere in western Scotland – possibly more than 100 miles away in the forests of Argyll.
Police used CCTV footage from hundreds of cameras, phone and computer records, and examination of his car to build up the case against him.