The House of Lords Constitution Committee notes in a report that using a so-called Section 30 rather than primary legislation to empower Holyrood to hold the poll means it makes it "impossible to ensure fully effective scrutiny". MPs and peers will not be able to amend the Order.
The report says: "Proceeding via the Section 30 route curtails the opportunity of the UK Parliament to have an effective input."
It points out how the Agreement was negotiated in private between the UK and Scottish Governments while Prime Minister David Cameron had declared the "future of Scotland must not be worked out in secret, behind closed doors".
The committee also says: "It is hard to avoid the conclusion that more could have been done to include the United Kingdom Parliament in this process."
Peers stress how "serious questions remain un-answered" about the Agreement, such as is the Section 30 order "robust enough" to protect the process from legal challenge and will there be "practical and legal problems" in giving 16 and 17-year-olds the vote.
Baroness Jay of Paddington, the Committee chairwoman, said it was worried there were "still many potential pitfalls ahead", pointing out how members were concerned the Electoral Commission's recommendation on the poll question "might not be followed" by the SNP majority Scottish Parliament.
"For this referendum to be timely, robust and secure, further work is necessary to ensure these issues are resolved," she added.
A Scotland Office spokesman said it was "not possible to rule out legal challenge to any legislation" but pointed to the committee's view it did "not consider any such claim would be likely to succeed".
He said the Order and the Memorandum of Understanding provided a "robust framework for a legal, fair and decisive referendum; that view is widely accepted".
Nicola Sturgeon will tomorrow give evidence to Holyrood's Referendum Bill Committee.