Home captain Manuel Pascali was controversially sent-off after four minutes, for a two-footed challenge on Nigel Hasselbaink.
Murray Davidson slotted the away side ahead on the half-hour mark, however the hosts battled back and grabbed a deserved equaliser seven minutes from the end when Michael Nelson headed home from James Dayton's free-kick.
The game looked set for a stalemate but Vine fired home on 88 minutes to clinch all three points for Steve Lomas' side.
Referee Stevie O'Reilly infuriated the home support after dismissing Pascali for what he deemed to be a two-footed challenge on Hasselbaink after just four minutes.
Killie boss Kenny Shiels was enraged at the decision and was sent to the stands after a heated debate with fourth official Andrew Dallas.
The game took a while to settle following an action packed opening and Steven MacLean passed up the first scoring opportunity of the match when he headed straight at Cammy Bell after Gregory Tade's flick-on.
Tade brought out a fantastic safe from the home keeper after 14 minutes, following a neat through pass from MacLean.
Chris Miller was booked for a late challenge on Sheridan as tempers continued to flare.
It was Bell who was by far the busier keeper during the first half-hour and he made a good low stop to deny Liam Craig after 28 minutes.
Hasselbaink was feeling the wrath of the home supporters every time he touched the ball, however the effects of Pascali's challenge became evident as he hobbled off on the half-hour mark to be replaced by Vine.
The visitors made the breakthrough two minutes later when Tade laid the ball to Davidson and the midfielder slotted low past Cammy Bell for his fifth goal of the season.
Davidson was clearly in the mood and he almost added to his tally when he forced Bell to tip his fizzing drive over the crossbar.
The small band of visiting supporters screamed for a penalty after 34 minutes when Vine went down under the challenge of Lee Johnson, however O'Reilly deemed that the striker had taken a dive and produced a yellow card.
Johnson was replaced by Mohamadou Sissoko three minutes before the break and Kilmarnock managed their first scoring attempt 30 seconds later when Sheridan fired wide from range.
Alan Mannus was forced into his first save of the match eight minutes after the break when he parried away Paul Heffernan's powerful strike.
St Johnstone replied at the other end and Craig's dipping effort sailed just wide of Bell's right-hand post.
Saints continued to look the most likely to add to the scoring and Craig ended an excellent breakaway by flashing an effort just wide of the target after 58 minutes.
Dave MacKay was booked was a hauling back Harkins on 65 minutes and he was followed into the book by MacLean two minutes later.
Shiels introduced Danny Racchi and Dayton in place of Jeroen Tesselaar and Perez with 10 minutes remaining and the hosts reaped the rewards almost instantly.
Dayton's deep free-kick found Nelson who angled a superb header beyond the despairing Mannus.
Racchi saw his effort deflected wide and Sissoko headed past the post as the hosts heaped on the the pressure in the closing stages.
There was still time for another twist though and Vine charged through on goal before firing past Bell to ensure all three points for St Johnstone.