The power will broaden the sources of finance available to it to fund major projects such as hospitals, schools and transport.
The Scotland Act 2012 already gives the Scottish Government the ability to borrow up to a total of £2.2 billion for capital investment through the National Loans Fund (NLF) and commercial loans from 2015/16.
The UK Government consulted on whether these powers should be extended to allow bond issuance, as part of the total £2.2 billion borrowing limit.
Mr Alexander said: "This is an historic announcement, demonstrating once again how Scotland can grow and prosper within the UK. From 2015, Scotland will be able to borrow up to £2.2 billion to invest in its hospitals, roads and other capital projects.
"In addition to having access to the National Loans Fund, our decision today means that the Scottish Government can directly issue its own debt.
"It will of course be up to the Scottish Government to manage their borrowing, but this is complemented by the tax powers in the Scotland Act providing the Scottish Government with an independent source of