Ewan Fraser, chief executive of Dunedin Canmore Housing in Edinburgh, and his housing services director, Graeme Russell, spoke of the difficulties posed by housing benefit no longer being paid directly to social landlords and the administrative burden it placed on housing associations to take on advice functions.
Dunedin Canmore Housing Association has accumulated nearly £150,000 of rent arrears as a result of direct payments and has been forced to spend more time with tenants anticipating these problems.
But asked by Tory MSP Alex Johnstone whether the reform should be dropped, Mr Fraser said: "Absolutely not. I don't think that would help anyone."
Four-fifths of tenants in the pilot did not have a job, while a fifth had significant literacy or numeracy difficulties. 31% manage their finances poorly, compared with 8% who manage them very well.
Around three in five support housing benefit being paid to them directly but only a quarter are comfortable setting up a direct debit to pay it on to their landlord. 73% prefer to pay all of their bills in cash.
Mr Fraser said: "We tried to be really positive about making the changes happen and put politics aside. It has been very resource-intensive and it has been quite challenging.
"This has not been an easy project. There have been a range of challenges and we have found some things almost impossible to manage properly."
The demonstration project began in August at Dunedin Canmore in Edinburgh.