The Wimbledon champion did not allow his opponent to hold serve once, taking it 6-1 in cool conditions on Court Suzanne Lenglen.
It was the first set Murray had played at Roland Garros since his 2012 quarter-final loss to David Ferrer after missing last year's tournament with back problems.
Murray had won both his previous two matches against Golubev easily, but the world number 53 is a flashy player who last month beat Stanislas Wawrinka in the Davis Cup.
Murray made the perfect start, breaking the Golubev serve in the second game, his quality and consistency proving too much for the Kazakh.
He had to save a break point in the next game, which he did with a big serve and, when Golubev double-faulted to give Murray a 4-0 lead, the set was all but over.
The Scot was having more trouble on his own serve and Golubev took his third chance to break only to then drop serve for the third time.
That left Murray serving for the set and, although Golubev saved three set points, he then drove long on the fourth.
Murray has had an indifferent season so far but went into the French Open on the back of one of the best clay-court performances of his career, albeit in defeat to Rafael Nadal in Rome.
He was mostly dealing well with Golubev's streaky play, the Kazakh mixing winners with shanks and simple errors.
He finally held serve to start the second set but in the fifth game Murray pounced again, only to then throw in a poor game of his own as Golubev levelled at 4-4.
The Scot appeared to be bothered by a TV camera suspended on wires, which was in his eyeline when he served, and he complained to the umpire.
But he forged ahead again straight away by breaking serve for the fifth time, and then held on to take the set 6-4 when Golubev netted a forehand.
Murray bemoaned his lack of intensity after breaking Golubev again at the start of the third set only to drop serve immediately.
The seventh seed was certainly finding it a bit more of a struggle than he would have liked, and he dropped serve for the second successive time with a double fault to trail 3-1.
Golubev returned the favour, but Murray was really struggling with his serve now, failing to land even one in three of his first serves, and it was not until he was 5-2 down that he finally held serve in the set.
That left Golubev serving for it, and the Kazakh, whose only win at Roland Garros came in 2009, produced an impressive game to clinch the set 6-3.
A better start to the fourth set was important for Murray and he managed it, holding serve fairly comfortably for once and then breaking Golubev for 2-0.
Murray's stats were hardly impressive, but the Kazakh had broken his half-century for unforced errors.
The Scot had a point for 4-0 after a remarkable backhand winner hit without looking at the court, but Golubev saved that.
He saved two match points as well, but missed a return on the third as Murray ground out victory after two hours and 35 minutes.
INTRO: Andy Murray won a scrappy contest against Andrey Golubev to mark his return to the French Open with a four-set win.
The Wimbledon champion began well, but got bogged down in heavy conditions and dropped the third set before recovering to win 6-1 6-4 3-6 6-3.
Murray, who missed the tournament last year with back problems, will play either Australian Marinko Matosevic or German-Jamaican Dustin Brown in the second round.