Underlying profits before tax were down 3.9% to £623 million in the year to March 29 with general merchandise (GM) - which includes clothing - seeing like-for-like sales fall by 1.4% though food improved by 1.7%.
Mr Bolland said GM was showing "early signs of improvement", with investment being ploughed in to transform the business.
"We are making solid progress on this journey and are now focused on delivery," the chief executive said.
However, M&S warned that a new website launched earlier this year was taking time to "settle in" and would have some impact on GM sales in the weeks since the end of the financial year.
Mr Bolland, who has been in the top job at M&S since May 2010, has ploughed hundreds of millions of pounds into refreshing the business.
He has also hired new personnel and launched a celebrity-backed marketing push to try to revive dwindling clothing sales.
But the group revealed that profit margins in the division were hit by the cost of promotions and markdowns during the financial year.
It said it had continued to make improvements in buying and merchandising and worked hard "to improve newness and availability".
But the company said: "We faced a challenging clothing market, with unseasonal conditions and high levels of promotional activity."
Marks said the improving trend in clothing sales in the fourth quarter - when they picked up by 0.6% - had continued since the end of March.
But it said that its new website would, as expected, take four to six months to settle in and "have some impact" on the performance of the wider GM division in the first quarter.
M&S reported web sales up 23%, which it said outperformed the market, with 55% of these purchases now picked up in store. International sales rose 6.2%.
The group said an initial spending programme focused on its "strategic priorities" was now largely completed and this investment would fall to £500 million to £550 million a year for the next three years.
It expects to see profit margins in GM show "significant improvement" over that period through changes in the way it operates, while food margins are also expected to grow.
Total annual group sales were £10.3 billion, with UK revenues of £9.16 billion and international revenues of £1.15 billion.
M&S is ramping up global expansion plans and during the year added 22 new stores overseas, focusing on key markets in India and China.
Mr Bolland avoided responding directly to questions about his future when interviewed on the BBC Radio 4 Today programme, but said: "I really enjoy my role."
He pointed to the "heavy lifting" that had needed to be done with investments in IT and logistics which "the company didn't do for the last 15 to 20 years".
"So, over the last three years, we've done lots of investments that have made the company at the moment fit for purpose. Over the next three years there is more delivery to come and we'll certainly concentrate on that."