Wendy Alexander stood down from the post in 2008 because of a row about donations to her leadership campaign.
In his memoirs, Damian McBride claims that Ms Alexander's brother told Mr Brown that his sister would have to stand down to "avoid further damage".
Mr McBride's book Power Trip claims Mr Alexander warned his boss that his sister's resignation should not be linked to comments she made about the independence referendum.
In 2008, when the SNP were in a minority administration at Holyrood, Ms Alexander challenged the Nationalists to ''bring it on'' and call an early referendum on independence.
In his book, Mr McBride said Ms Alexander's call was "entirely contradictory" to Mr Brown's comments on the timing of such a vote and that if she remained as Labour's Holyrood leader there was the risk of both this and the donations row "rumbling on unresolved over the summer".
Mr Alexander, now Labour's shadow foreign secretary, discussed the matter with Mr Brown, said Mr McBride.
"Dispassionately he told the Prime Minister that his sister had to quit in order to avoid further damage," Mr McBride claimed.
"However, Douglas warned Gordon she'd need to make it clear that the reason was to do with the donation - and nothing to do with the referendum.
"If I was sometimes cold-blooded about how I did my job, I had nothing on Douglas that day."
But Mr Alexander said he "always supported his sister" and "never supported Damian McBride".
The Labour MP said: "The style and approach of the politics that Damian McBride embodied has been completely discredited and thankfully the Labour party has moved on. I support Ed Miliband's efforts to remove that type of politics from our party."
Mr Alexander said he has not read Mr McBride's book and does not intend to.
"He was discredited when he left Downing Street and that's really all there is to say.
"I always supported my sister and I never supported Damian McBride. That might explain why he writes about me in those terms."
When asked if Mr McBride is lying, Mr Alexander said: "I've said what I'm going to say. It's what Damian McBride does."