New Zealand police said the man, who is in his 30s, last month followed a 21-year-old woman and assaulted her.
They have charged him with burglary and assault with intent to rape.
A New Zealand judge has suppressed many details of the case, including the man's home country.
New Zealand's Ministry of Foreign Affairs and Trade said it sought a waiver of diplomatic immunity so the matter could be dealt with in New Zealand, but was rebuffed by the man's home country, which immediately removed him.
New Zealand Prime Minister John Key said that the man could face charges for the alleged crime in his home country.
Mr Key added: "We wouldn't have let him go back if we could have avoided it."
New Zealand sometimes suppresses details in sex cases to protect the identity of the victims.