The incident happened on Tuesday evening when the man was singing hymns to a small crowd on High Street.
A woman came from a kilt shop across the road. She approached the victim and a disturbance broke out, resulting in the victim falling over and hitting his head on the ground.
A passer-by called for an ambulance and the man was taken to Edinburgh Royal Infirmary.
The suspect is described as mixed race, aged in her late 40s, 5ft 8in, with shoulder-length dark hair.
Detective Constable James Philp said: "We know that a small crowd had been watching him sing, so we are appealing for any witnesses to come forward."