Stuart Thompson, 43, was killed in a "work-related" incident at the site of the former Grampian Country Pork plant in Great Western Road, Buckie, at about 5pm on Monday.
The accident is understood to have happened when the building was being dismantled.
Mr Thompson, from Lancashire, died at the scene from his injuries, Police Scotland said.
A force spokeswoman said: "Inquiries into the incident are at an early stage and police are working closely with the Health and Safety Executive to establish the full circumstances."
An HSE spokeswoman said: "We can confirm we are investigating alongside Police Scotland."