Lewis Macleod and Kyle Hutton also claimed a goal apiece against Elgin, while Daniel Moore and Mark Nicolson found the net for the home side in an entertaining top-of-the-table tussle at Borough Briggs.
The clash between the two clubs should have been played on November 25 but was postponed then after Elgin sold too many tickets.
The blunder earned the Moray outfit a £25,000 fine from the Scottish Football League, while chairman Graham Tatters was fined £1,000 by the Scottish Football Association earlier this week.
With the fixture finally under way, Rangers were clearly keen to make up for lost time and took the lead after five minutes.
Little collected a neat pass through the middle from David Templeton before rounding goalkeeper John Gibson and slotting into the back of the net.
Elgin were back on level terms with 17 minutes on the clock when the ball broke for Moore and he sent an impressive curling effort past the reach of Neil Alexander.
Rangers restored their lead after 24 minutes when Little grabbed his second of the day.
Templeton was again the provider with the cut-back and all that was required was a simple tap-in from the Northern Ireland international.
Again, the home side responded positively with a goal of their own.
Chris Hegarty was booked for a foul on David Niven on the edge of the box and Nicolson dispatched a superb swerving free-kick into the top corner with 32 minutes gone.
Elgin could have added to their tally when Brian Cameron unleashed a decent strike but Alexander was able to smother.
Instead, it was Rangers who had the net bulging again four minues before the break.
Little collected from Hutton before supplying the cross for McCulloch to lash home in off the crossbar.
Rangers made a change at the break when Anestis Argyriou was replaced by Ross Perry.
Elgin looked dangerous after the restart and Craig Gunn squandered the chance to find the net when he blasted over after Alexander spilled the ball.
McCulloch - back in a forward role after a recent spell in defence - was on target again when he raced onto a Templeton ball and fired across the goalkeeper into the bottom corner after 60 minutes.
Macleod added his name to the scoresheet four minutes later with a powerful, low strike from just inside the box.
Rangers threw Robbie Crawford into the action for Templeton and the youngster almost claimed a goal of his own within a couple of minutes when he forced Gibson into a great save.
Hutton put the gloss on the win when he rifled home from 20 yards after 83 minutes, before Alexander denied Moore from the penalty spot two minutes later after Hutton brought down Stuart Leslie.