Ally McCoist's side performed poorly, perhaps as badly as they have all season, and appeared to be heading towards their second draw in SPFL League 1 in three games.
However, there was an unexpected finale that saw the Ibrox club collect all three points and remain on course to clinch the third-tier title this month.
East Fife substitute Pat Clarke brought down David Templeton inside his own area, referee Alan Muir correctly pointed to the spot, and McCulloch held his nerve.
The defeat was harsh on Gary Naysmith's side as they had been effective in nullifying the threat their opponents posed but the former Scotland left-back had no complaints about the official's decision.
"It was a penalty," he said. "It was a reckless challenge. It was in the last minute of the game and I think Pat probably panicked. It was so, so frustrating."
McCoist made two changes to the team that had defeated Stranraer at Stair Park on Tuesday night. Dean Shiels and Templeton came in and Andy Little and Arnold Peralta dropped out.
Templeton and Shiels linked up to engineer the match's first attempt on goal in the 13th minute. The former threaded a pass through the home defence and the latter latched on to it before shooting narrowly wide.
East Fife also posed a threat. Gary Fisher pounced on a poor headed clearance by Richard Foster in the 18th minute and went close with a left-foot volley.
That, though, was the midfielder's last involvement in the game. He trudged off soon afterwards clutching an injured arm and was replaced by Stephen O'Neill.
Nicky Law and Bilel Mohsni struck the post in rapid succession with shots from inside the East Fife penalty box on the half-hour mark as keeper Greg Paterson looked on helplessly.
Paterson saved well from Templeton, switched to the right wing for the opening 45 minutes, on two occasions before the referee blew his whistle for half-time.
The Rangers fans who had made the journey through to the Fife coast and who packed the scaffolded terraces at the Bayview Stadium had precious little to cheer about in the first half. It was grim fare.
Matters did not improve significantly in the second half. McCoist replaced Shiels, who had drifted out of the match, with Little in the 66th minute in an attempt to spark his team into life.
Jon Daly had, as against Stranraer in midweek, become isolated and the manager felt introducing Little to proceedings might improve the service to his lone striker.
That happened to a degree. But it was Templeton who came closest to breaking the deadlock with a powerful 80th-minute drive that Paterson did well to palm to safety.
The home team's goalkeeper pulled off a spectacular save in the final minute to deny Templeton, one of just a few of the visiting team's outfield players to gain pass marks.
However, Paterson was unable to reproduce such heroics to keep out the penalty.