Vandals used an abrasive material to strip paint off of the nose of the bronze statue, which sits on the George IV Bridge, sometime between 1pm on Tuesday, October 1 and 5pm on Wednesday, October 2.
The nose had been repainted the day before to fix natural wear and tear caused by visitors touching the statue for good luck.
The cost of the damage caused is believed to be around £200-£300.
Officers are now conducting local enquiries to find those responsible and are asking anyone with information to come forward.
PC Nick Davies said: "This crime is as unusual as it is disappointing to observe. The statue is a beloved tourist attraction and is also very popular with Edinburgh's local residents.
"The nose has recently been touched up to restore it following years of wear and at first it was believed the repair had not adhered correctly to the monument.
"However, we are now treating this as an act of vandalism after discovering that an abrasive has been used to remove the material from the repaired area.
"As such, anyone who remembers seeing any suspicious activity around Greyfriar's Bobby in the past few days is asked to contact police."
Councillor Richard Lewis, the city's Culture and Sport Convener, said: "The work carried out should have lasted for a number of years and it is highly unlikely that it would have worn away naturally in such a short space of time.
"We will, of course, organise the repair work to be undertaken again and hope that in future people respect the statue of Greyfriars Bobby."
Meanwhile a 3.6m (12ft) crocodile sculpture has been stolen from the city's St Andrew Square.
The fibreglass animal was in the square for three weeks to promote an exhibition about missionary David Livingstone at National Library of Scotland, with the sculpture nicknamed "Davy" by staff.
It went missing overnight and staff believe it would have taken two people to carry it away.
Deputy librarian Darryl Mead said: "Davy has proved a popular attraction in the gardens since he moved in three weeks ago. We suspect he may have been taken as a prank late at night and we appeal for his safe return.
"Davy is very much an exhibitionist and took his role in promoting our exhibition very seriously. He was very happy in his pond in the gardens and will be missing it very greatly."
The sculpture was filled with sand to weigh it down, at around 180kg, but it was poured out before the crocodile was moved out of the garden.
Andy Neal, chief executive of Essential Edinburgh which manages the garden, is appealing for the crocodile to be "returned to its natural habitat".
He said: "It is much happier (in the garden). While we are only concerned to have it back safe and sound, and if we get it back would be content to let the matter rest, the police have been informed and CCTV cameras do operate in the area."
Pc Caireen Stewart said: "This crocodile is the property of the National Library of Scotland and is very unique. Due to its size, it is believed that someone may have seen something regarding its disappearance. Can any person with information please contact the police on 101."