The Australian Securities and Investment Commission (ASIC) said an internal RBS investigation found traders had discussed the setting of the country's benchmark inter-bank interest rate.
RBS, headed by chief executive Ross McEwan since October last year, said it had voluntarily reported the findings of its probe to the watchdog and co-operated fully with ASIC. The internal review is said to have concluded the maximum possible benefit to the bank was around £445,000.
RBS withdrew from the panel of banks setting the Australian Bank Bill Swap Rate (BBSW) on April 30, 2012.
ASIC said RBS is to make a donation of around £880,000 which will be used to promote financial literacy.
An RBS spokeswoman said: "The Bank voluntarily reported the conduct it identified to ASIC. While there has been no admission of wrongdoing by the Bank, it acknowledges ASIC's concerns. Since becoming aware of improper conduct in connection with rate setting, RBS implemented significant actions to correct and strengthen the systems and controls governing its processes around submissions for key financial benchmarks, which have been implemented in its offices globally."
It is not known how many traders were looked at in the probe but it is understood those involved are no longer with RBS. The enforceable undertaking issued by ASIC said a dedicated chat room to discuss rate setting operated between October 2009 and November 2010.
ASIC has previously censured French lender BNP Paribas and Swiss bank UBS after finding traders had tried to influence the setting of the inter-bank interest rates.
Global regulators have been reforming rate-setting practices after Barclays Plc, UBS AG, RBS and others were hit with fines totalling billions of pounds for rigging the London Interbank Offered Rate, known as Libor.