A total of 32 people are believed to have been killed in the massive fire, but just 10 bodies have been found.
The cause of Thursday's blaze in the small town of L'Isle-Verte remains under investigation.
A police spokesman said: "It could be a cigarette, it could be a small heater, it could be an electrical problem."
Cold temperatures hampered the search over the weekend. Quebec police said the ice in some places was as thick as two feet.
Police lowered the number of missing from about 30 to 22 based on more detailed information.
A police spokesman said: "I think we can all agree here today that the ...people who are still missing, I think we can assume the worst."
The coroner's office identified two victims on Saturday, Juliette Saindon, 95, and Marie-Laureat Dube, 82. A third person has been identified and the name was due to be released yesterday.
Teams of police, firefighters and coroners methodically picked their way through the ruins of Residence du Havre, working in shifts in the extreme cold.
Spray from firefighters' hoses had left the home resembling a macabre snow palace.
About 20 elderly residents survived the fire.
Some were moved to other residences for the elderly in the area, and the Red Cross had raised about $200,000 Canadian dollars (£110,000) to provide clothes, hearing aids, wheelchairs, and other urgent needs.
"Because they left their residence so quickly, they left with nothing," said a Red Cross spokeswoman.
Many of those who died were confined to wheelchairs or used walkers, and some had Alzheimer's. Firefighters responded within minutes of getting the alarm but could only reach one-third of the building as the fire was too intense.
The tragedy has devastated the town of 1500 people 140 miles north-east of Quebec City.