Accounts, which have just become available from Companies House, reveal Canongate tumbled to an operating loss of £368,467 in 2011, from a profit of £1.08 million at this level in 2010. Canongate acknowledged this was its worst performance "in many years".
In his statement on the accounts, Canongate chairman Sir Christopher Bland described the results as "disappointing".
Sir Christopher, a former chairman of telecoms group BT, adds: "The group's operating loss of £368,467 was largely attributable to Julian Assange's failure to deliver the book he had contracted to produce, and we were unable to obtain repayment from him of Canongate's substantial advance, which had to be written off."
This advance is rumoured to have been about £500,000.
Managing director Jamie Byng, whose management buyout of Canongate in 1994 was backed by stepfather Sir Christopher, says 2011 was a "mixed and ultimately frustrating year" and a reminder of the "highly unpredictable nature" of the publishing business.
He adds: "We published some very good books, and some with great success, and yet we ended up losing a significant amount of money on one title that resulted in the performance of the company as a whole being the worst in many years."
Australian Mr Assange, who has taken refuge in the Ecuadorian embassy in London and has been granted asylum by the South American country, is wanted in Sweden for questioning on sexual-assault allegations. He denies the allegations, and fears being transferred to the US if he travels to contest them.
He enraged the US government in 2010 when Wikileaks published tranches of secret US diplomatic cables.
Canongate says on its website that Mr Assange signed a contract with it in December 2010 to write a book, part-memoir and part-manifesto, for publication the following year. It adds that Mr Assange said in June 2011, in spite of sitting for more than 50 hours of taped interviews with the writer he had enlisted to help him and with 38 publishing houses around the world committed to publishing the book, that he wanted to cancel his contract. Canongate notes that Mr Assange declared, after reading the first draft of the book at the end of March 2011, that "all memoir is prostitution".
Explaining its decision to publish Julian Assange: The Unauthorised Autobiography in September last year, Canongate's web-site states: "By the time Julian wanted to cancel the deal, he had already used the advance money to settle his legal bills. So the contract still stands. We have decided to honour it – and to publish."
Canongate's turnover fell from £13.4m in 2010 to £10.8m last year.
But Canongate was able to post a £319,348 profit for 2011 at the pre-tax level because of an exceptional gain of £662,142 on the sale of Australian subsidiary Text.
Sir Christopher describes the outlook for publishing in the digital age as "both exciting and uncertain".
He adds: "Canongate's healthy financial position should enable it to weather any further downturn in the market, while continuing to take advantage of publishing opportunities as they occur."
Canongate's high-profile successes of last year included Carol Birch's Jamrach's Menagerie, which was shortlisted for the Man Booker Prize 2011, and Karl Pilkington's paperback best-seller An Idiot Abroad.
Richard Holloway's Leaving Alexandria did well for Canongate in the first half of this year.
Mr Byng forecasts a strong second half with releases including Further Adventures of an Idiot Abroad by Karl Pilkington, and a movie tie-in edition of Canongate's biggest-selling backlist title, Yann Martel's Life of Pi.
The accounts show Canongate's highest-paid director, likely to be Mr Byng, received emoluments of £97,775 and £6844 of pension contributions, and exercised share options of £2807. The highest-paid director received emoluments of £110,248 in 2010.