Lord Steel spoke out after an ex-Tory expenses cheat regularly clocked in to receive the cash but left after less than 40 minutes.
Disgraced Lord Hanningfield - who returned to the upper chamber after serving nine weeks of a nine-month sentence for falsely claiming £28,000 in taxpayer-funded expenses - said half of its members did the same.
On 11 of 19 days that Lord Hanningfield's movements were monitored in July, he spent less than 40 minutes in the Lords before returning to his home in Essex.
The shortest attendance during the month was 21 minutes and the longest more than five hours - with £5700 claimed in attendance allowance and £471 in travel costs.
Lord Steel, the former Liberal leader who has led efforts to secure stronger disciplinary action against peers who abuse the system, said he would not comment about the individual case, pointedly adding "I don't even know what he looks like".
He said the allowance was "supposed to represent work in parliament" and it was wrong to claim if for only a brief stay.
The official guide for peers on claiming the £300 allowance says that it is available to those "who certify that they have carried out appropriate Parliamentary work".
"It's a rather daft system," Lord Steel said. "If you had a swipe-in, swipe-out system then you would know how long people spend there."
Lord Hanningfield claimed the requirement to be seen in the chamber to qualify was "only a mechanism for paying you" and that failing to speak or vote should not be a bar to receiving the cash.
He said most of the money went on "entertaining, meeting people, employing people" and claimed he would "end up with £12,000 a year" for himself.
"I don't do anything else. How do you think I am going to eat, how am I going to pay my electricity bills?", he added, saying he could "name 50" others who used the same method.