Shahid Khan, 30, who lives in Glasgow's west end, was head- butted, slapped and racially abused as he walked with his sister near Anniesland Cross.
Police are hunting for the man who assaulted Mr Khan shortly before 4pm on Tuesday.
Mr Khan said: "I was walking from Anniesland Retail Park with my sister, and this man started shouting at me.
"He said to me: 'Are you responsible for trafficking women in this area?'
"I found this offensive but my sister told me not to get involved or make eye contact."
Mr Khan, a writer who fled to Glasgow almost two years ago after being persecuted in his home country of Pakistan, continued to try to ignore the man.
Mr Khan said: "The next thing I knew he headbutted me in the back of my head then head-butted me in the forehead, where I now have a bump."
Customers at BJ's Bar and Diner in Crow Road rushed out to try to stop the brawl.
The attacker, who is thought to be aged between 20 and 26, fled before police arrived
Mr Khan said: "This was a horrendous and hideous act and I'm now paranoid about leaving my home and I think everyone is going to chase me."