The group currently has 39 stores in Scotland and has three major projects in the pipeline which are all subject to planning permission.
It wants to open a new store in each of the country's two largest cities as well as relocate its outlet at the Braehead shopping centre in Renfrewshire to a new premises three times the size.
Lord Wolfson did not comment on his dealings with local authorities in Scotland.
But he complained: "Our biggest single obstacle is the speed at which it takes to get the planning [permission]."
He added: "There are really good councils. But there are some really terrible, intransigent, incompetent ones holding back the pace of growth."
He said that Next has a pipeline of stores equating to 1.2 million sq ft but the group had only managed to open 250,000 sq ft in the last year, some 50,000 sq ft less than it had hoped for.
The chain aims to open 300,000 sq ft of new space in each of the coming two years.
"It is taking twice as long to get planning [permission] as it does to build shops," he said.
He said that only around half of Next's new stores were in out-of-town locations and said these did not replace existing high street outlets.
Expansion of Next's store estate, plus rising online sales, offset declining same-store sales as the company posted a 3.1% rise in revenues to £3.5bn for the year to the end of January.
Underlying pre-tax profit was up 9% at £621.6m.
New space added £27m to profits while lost sales in existing stores took £29m from the bottom line.
Next, Britain's second-biggest clothing retailer, said trading since the start of February was quiet, with sales at the bottom of a 1% to 4% target growth range for 2013-14.
Some retailers have complained that the cold weather has kept shoppers away.
Lord Wolfson said: "Is it weather or the economy? The truth is we do not know and we won't know until the weather gets better."
He warned of a continuing squeeze on consumers' incomes as muted wage growth continues to lag inflation.
He maintained that the consumer economy is "subdued" rather than in crisis.
Kate Calvert, analyst at brokers Cantor Fitzgerald, wrote in a note to clients: "Great set of results from Next given the challenging environment showing the strength of its business model and how management's relentless focus on marginal gains continues to pay off."
Next's shares closed up 167p or 4% at 4314p as investors reacted to a better-than-expected full-year dividend of 108p, up 16.7%.