In an enthralling Scottish Premiership encounter, striker James Collins put the home side ahead in the 59th minute when he lashed the ball high into the net from four yards after the Tynecastle men failed to deal with a Craig corner.
However, Jambos substitute David Smith, on for Callum Tapping after the break, drew the visitors level in the 70th minute and with that it looked like the Gorgie men, under pressure for most of the match, would leave with an unlikely a point.
However, with nine minutes remaining Hearts defender Jordan McGhee, a late replacement for Gorgie skipper Danny Wilson, who was ill, was adjudged by referee Bobby Madden to have fouled Lewis Stevenson in the box.
Craig stepped up to score from the spot and give the Hibees fans the bragging rights for the start of 2014 with their first derby win of the season.
Gary Locke's young Hearts side went into the game without a victory in seven matches but two 1-0 wins over their city rivals this season so far offered hope.
And after the 1-1 draw with St Mirren on Sunday took them on to minus two points, the visitors knew victory would have them in the black for the first time this season.
Hibs pushed forward from the first whistle with a zeal which required some stout and at times desperate defending from Hearts.
Green and white jerseys swarmed around the Gorgie penalty area but it was the team in maroon who had the first real chance of the match.
After 12 exhilarating minutes striker Callum Paterson got the better of Hibs stopper Michael Nelson wide on the left and his delivery found Tapping only six yards from goal - but the midfielder's volley was saved by Ben Williams.
However, the Leith side quickly reasserted themselves and six minutes later Craig's searching free-kick into a packed penalty area was headed against the bar by Hibs defender Jordon Forster before the Tynecastle side cleared desperately to safety.
On the half-hour mark Hearts midfielder Scott Robinson was booked by referee Madden for a clumsy foul on Jason Cummings, before the Hibs player fired wide of the target after making himself space inside the box.
The pace of the game refused to subside.
Following a lofted ball into the Jambos penalty area by Craig, Hearts keeper Jamie MacDonald tipped a close-range header from Cummings over the bar with the visitors again forced to dig deep to keep Hibs out from the corner.
The players, and both sets of fans who had helped create a fabulous atmosphere, deserved their interval break.
Moments into the second half Hibs had a penalty claim when James Collins' shot appeared to strike the arm of Hearts defender Brad McKay - but referee Madden was unimpressed.
The Gorgie side could have been forgiven for thinking it was going to be their night when Hibees midfielder Scott Robertson headed a cross form Paul Cairney past the near post.
However, when Collins pounced from close range inside a packed six-yard box just before the hour-mark, the relief from the stands holding the home support was palpable, masked only by jubilation.
The experienced striker Paul Heffernan immediately replaced Cummings but with three-quarters of Easter Road bouncing, McKay sent a header from a Kevin McHattie corner inches past the near post.
At the other end, Heffernan headed wide from eight yards when he should have hit the target - and the home side were soon made to pay for that profligacy.
Smith slalomed his way through the Easter Road defence before getting his shot away from outside the box and when Williams parried it back out, the Jambos substitute was first to react with a prod. The ball took a touch off Hibs defender Ryan McGivern before rolling into the corner of the net in front of the delighted Hearts support - and a couple of fans were led away by stewards for encroaching on to the pitch.
Hearts sensed victory was on the cards but in the 81st minute Madden pointed to the spot when McGhee clashed with Stevenson and Craig sent MacDonald the wrong way with a well-taken spot-kick.
In a frantic finale McHattie, Heffernan and Collins came close to scoring - but there were no more goals and the three points deservedly stayed in Leith.