Two first half goals from Yoann Arquin and Yann Songo'o should have put County comfortably on course for their fourth win in their last six games.
But after Boyd grabbed his 14th goal of the season two minutes into the second half, County were left hanging on with 10 men for the final 13 minutes after Filip Kliss was shown a second yellow, and Boyd took advantage at the death as County players screamed for a foul.
Kilmarnock were keen to extend their impressive home run to four wins from their last five games and pressed Michael Fraser's goal from the kick-off.
The first chance, though, fell to Jordan Slew, one of County's seven signings in January, but he curled his shot wide.
After 13 minutes Killie finally worked Fraser, when Rory McKenzie twisted at the corner of the six-yard box before driving the ball against the goalkeeper's legs.
A minute later quick footwork from Robbie Muirhead gave the teenager a shooting chance but his drive across goal carried the ball just beyond the far post.
In the 20th minute Craig Samson made a fine double save. Richard Brittain's clever free kick across the 18-yard line set up Michael Tidser and when his shot was parried by the goalkeeper, Brittain followed up with an angled drive which Samson reacted well to block.
For all Killie's neat passing in middle of the park the Highlanders looked the more dangerous around goal, and duly opened the scoring on the half-hour mark.
A swift move down the right found Filip Kliss racing onto Brittain's pass and when the Slovakian swept the ball across goal Arquin smartly tucked it into the corner of the net beyond the helpless Samson.
Six minutes later, and before the home side could react to the opening goal, County grabbed a second to put them in the driving seat.
A well worked corner found Graham Carey outside the box. And although his left foot volley was fired into the ground the ball rebounded high into the packed box where Songo'o, on loan from Blackburn Rovers, with a neat flick carried the ball into the net to put the visitors two up.
Kilmarnock looked a beaten side at the interval but within two minutes of the restart had pulled back a goal. Sammy Clingan's corner found Boyd challenging with Brian McLean. But as the ball fell between them the ex-Rangers' striker reacted to sweep the ball high into the net and spark some life into the home side.
Three minutes later the Ayrshire men came within inches of the equaliser. McKenzie broke from the middle of the County half and from 20 yards hammered a left foot shot which rebounded off the foot of the post.
Boyd had mis-hit from 12 yards before Muirhead's low 20-yard drive was pushed beyond his post by Fraser at full stretch.
By the time Alexei Eremenko came off the bench after 70 minutes to make his second debut for Killie, the incessant rain and sleet had left the pitch sodden with a lot of surface water making passing difficult.
Thirteen minutes from time Kliss picked up his second yellow card for persistent fouling.
The home side made the most of having the extra man and had the bulk of the play around Fraser's goal.
And after some sustained pressure Killie grabbed the equaliser in injury time. Chris Johnston swept in a cross and Boyd out-muscled McLean to head home from 12 yards with County players and officials appealing for a foul.