A number of offensive and sectarian songs were heard from the section of Shielfield Park housing travelling supporters at the Irn-Bru Third Division match, which was broadcast live on ESPN.
Two people were arrested and it is understood the Ibrox club will assist any further police inquiry into criminal acts and ban those responsible from all Rangers matches.
A Rangers spokesman said: "The club is disappointed by certain outbursts of inappropriate singing by a section of the support at Berwick.
"Our fans have been excellent this season both home and away and we do not want to see this tarnished."
Rangers boss Ally McCoist said after the 3-1 win: "I didn't know anything about it until I came in after the game.
"Our supporters have been nothing short of sensational home and away this season. If they were a little bit out of order today, I apologise."
He added: "We don't need it within the club or Scottish football. I'm very, very hopeful it's a one-off and won't happen again.
"If there was some unsavoury chanting, they've probably let themselves down - and they've let us all down a little bit.
"I would be very hopeful it's an isolated incident and we can move on."
McCoist added: "I think we've all come a long way in recent years. There's no doubt about that. We have to continue the good work that's been done before.
"You will get one or two setbacks, steps back to move forward again, but I'm hopeful it's no more than that."
Rangers TV presenter Andrew Dickson tweeted during the game: "A second half request: could the lads in the enclosure across from the main stand sing songs about Rangers please? Cheers."
He then replied to other Twitter users: "If those who think what was sung in the first half is acceptable, they should look at 99% of the users on Twitter who disagree with them.
"Most of those disagreeing with me appear to be wee boys or blokes in their late teens/early 20s. Time to grow up lads.
"And can those saying 'what about Celtic', 'the media just want to attack us' and the rest please stop? Let's just get our own house in order.
"If we could all (Rangers fans, Celtic fans, everyone else) drag ourselves into the 21st century and move forward that would be terrific."
Fans have largely been well behaved this season, after Rangers were demoted to the bottom tier of Scottish football.
Superintendent Tim Smith, of Northumbria Police, said: "By and large, the fans were very well behaved, but there was a short outburst involving some sectarian chanting towards the start of the game.
"At the beginning of the game, we identified that some chanting had taken place and we have taken video evidence of that to do follow-up inquiries.
"In addition, two people have been arrested for that offence and are in custody.
"We have video evidence that will be investigated by officers from the FOCUS (Football Co-ordination Unit for Scotland) team from Strathclyde Police."
Read match report
Contextual targeting label: