Darren Lavery gave Berwick an eighth-minute lead but Jamie Longworth equalised midway through the first half and Anthony McParland struck just after the hour.
Both teams ended the match with 10 men, with Berwick's Kevin McDonald and the hosts' Michael Keenan sent off in the second half.
Peterhead drew 0-0 at home to bottom side Stirling Albion, who had defender Brian Allison dismissed late on.
Montrose climbed into the top four at Elgin City's expense when they beat their rivals for a play-off spot 4-1 at Links Park.
Garry Wood put Montrose ahead after seven minutes and Stephen McNally added a second 10 minutes later.
Martin Boyle made it 3-0 and Richard McIntosh got a fourth in the second half before Paul Millar got a consolation, just after Elgin team-mate Sean Crighton had been dismissed.
Jamie Glasgow and Jordan Holt scored as East Stirling won 2-1 away to Annan Athletic.