Iain Gray, convener of the parliament's watchdog Audit Committee, said arrangements to hold HMRC to account were vital as any errors would impact on the Scottish Government's budget.
The Scottish Government will be responsible for raising about half the income tax generated in Scotland from April 2016 under powers handed to Holyrood earlier this year.
Scotland's annual grant from Westminster will be reduced accordingly in a move designed to make the Scottish Government more accountable for the cash it spends and to provide a greater incentive to boost the economy.
HMRC – which will collect the Scottish Rate of Income Tax (SRIT) – is developing new computer systems to identify Scottish taxpayers, at a cost of £45 million.
The Scottish Government will pick up the bill and pay running costs of about £4.5m per year to keep the system up to date.
The Scottish Government and HMRC are drawing up a memorandum of understanding to set the ground rules, but Mr Gray insisted Holyrood needed to exercise greater scrutiny over the UK taxman.
He said: "These changes have been obscured by the referendum debate. But obviously if there is a No vote this is how Scotland's income tax will be raised and collected into the foreseeable future.
"And even if there is a Yes vote in the referendum there will be a period of negotiation and the income tax arrangements might well be in place for some time.
"The first thing the audit committee is concerned about is the degree of oversight the Scottish Government and parliament have over the spending of £45m on the HMRC system.
"Both in Scotland and at Westminster the track record of procuring major ICT systems has not always been problem free."
Mr Gray added: "We also want to consider ongoing arrangements for auditing HMRC's performance. We need to know what obligations HMRC will be under to appear before MSPs if there is a problem
"If for some reason it proved more difficult or HMRC were less assiduous in collecting Scottish income tax it would have an impact in Scotland and only in Scotland. That must be a concern for us."
Under the new Scotland Act, passed in May, the Scottish Government will set SRIT in its 2015 budget and collection will start in April 2016.
At the same time Scotland's budget will be cut by the equivalent of a 10p reduction in income tax –- roughly £4.5 bn.
If the Scottish Government sets SRIT at 10p in the £1, income tax will remain in line with the rest of the UK and the Scottish budget will remain the same.
But ministers could choose to raise the rate in a bid to generate extra income for the Scottish budget or reduce it in the hope it would help business by giving people more spending power.
For the system to operate fairly HMRC will have to correctly identify Scottish taxpayers – including decisions on people who split their time between north and south of the Border.
Holyrood's audit committee will take evidence from Edward Troup, the senior HMRC official responsible for the SRIT, tomorrow at the start of an inquiry into the new system.
l The Welsh Government should have the power to vary income tax by 2020, a report on greater devolution has said.
The changes would make the Welsh Government responsible for raising around one-quarter of its £15bn annual budget.