The TransAsia Airways turboprop plane, a 70-seat ATR 72, crashed near the runway on the island of Penghu, west of the mainland, with 54 passengers and four crew on board. No one was killed or hurt in the buildings.
Eleven injured people on the plane were taken to hospital.
The aircraft took off from Taiwan's southern city of Kaohsiung, headed for the island of Makong, but crash-landed in Huxi township of Penghu County, the main island of the chain also known as the Pescadores.
About 100 firefighters were sent to the scene, as well as 152 military personnel and 255 police.
According to an official at the Civil Aeronautics Administration, air traffic control reported that the bad weather at the time of the crash did not exceed international regulations for landing.
TransAsia's president Chooi Yee-choong was seen on TV bowing in apology and said: "We express our deepest apologies to everyone for this unfortunate event."
Typhoon Matmo hit Taiwan on Wednesday, bringing heavy rain and strong winds, shutting financial markets and schools.
It passed the island and headed into China, downgraded from typhoon to tropical storm.
TransAsia Airways is a Taiwan-based airline with a fleet of around 23 Airbus and ATR aircraft, operating chiefly short-haul flights on domestic routes, as well as to mainland China, Japan, Thailand and Cambodia.