Sean Winter gave Stranraer an early lead as the home side eyed a repeat of their victory over the Scottish Premiership side at the same stage of the tournament last season.
But Melvin de Leeuw quickly levelled and Boyce, a summer signing from Cliftonville, struck midway through the second period at Stair Park.
Stranraer stunned their visitors in the third minute when a fine move saw them take the lead through Winter, who curled a delightful shot over Mark Brown after Craig Malcolm had seen his earlier effort blocked.
After an even beginning where both sides had half chances the Premiership side were level after 15 minutes.
A long ball into the box from Jordi Balk on the right hand side was not collected by David Mitchell under pressure from De Leeuw and the Dutch winger was quickest to react, hooking the ball over his own shoulder and into the net.
With the home side giving every bit as good as they were getting from County and a repeat of last year's shock by no means unlikely, the Dingwall side finally got the chance they needed to take the lead.
Joe Cardle cut in from the left and, when his shot was blocked and Graham Carey then drew a save from Mitchell, Boyce was on hand inside the six-yard box to fire the ball home.
Stranraer did their best to get back on level terms and it required a goal-line clearance from Balk to keep out Winter's header.
The hosts kept pushing and it took a magnificent save from Mark Brown from Winter's long-range effort in the dying minutes to see the Highlanders through.