The LibDem business minister had her first son, Andrew, on Saturday.
Her husband, Chippenham MP Duncan Hames, 36, said today he was standing down as the Deputy Prime Minister's parliamentary private secretary to spend more time with the baby.
He is being replaced in the post by Lib Dem MP for Norwich South Simon Wright, who was previously PPS to education minister David Laws.
Ms Swinson, the 33-year-old Lib Dem MP for East Dunbartonshire, has already been replaced as a minister in the business and culture departments by Jenny Willott for the duration of her maternity leave.
Mr Hames said: "When I told Nick Clegg that I would be wanting more time to spend with our new baby, I knew he would understand.
"Now that our son, Andrew, is with us, I am stepping down as the Deputy Prime Minister's PPS.
"I truly appreciated the opportunity to see the coalition Government at work up-close and continue to support the dedicated efforts of Nick and the Liberal Democrat team he leads in Government."
Mr Hames and Ms Swinson married in 2011, a year after he was elected to Parliament and six years after she became an MP.
A PPS is a backbench MP appointed by a minister to act as his or her "eyes and ears" in the House of Commons. The job is unpaid but is often seen as a useful first step for young MPs towards ministerial office.