Kelman, who was commended in 1982 and won in 2008 for Kieron Smith, Boy, last night won the prize for Mo Said She Was Quirky, published by Hamish Hamilton.
Saltire Society president Magnus Linklater said the organisation had a long-standing commitment to recognising exceptional writing talent. He added: "Our literary heritage is internationally renowned and the future is in fine fettle with the quality of work in these awards."
Culture Secretary Fiona Hyslop said: "Each year the shortlist has captured the incredible quality and scope of literature produced in Scotland, and 2012 – The Year of Creative Scotland – has been no exception.
"James Kelman is an exceptional author. He and all of this year's Saltire Book Award winners can be proud of the unique and valuable contribution they are making to Scotland's rich and vibrant culture."
The History Book of the Year prize went to A Military History of Scotland, edited by E Spiers, J Crang & M Strickland.