After two days of talks during his week-long trade mission to China, the Chancellor described the banking move as "good for UK jobs and investment". Under the plan, Chinese banks will not be able to open high street branches but will be able to conduct business with other banks and companies.
The agreement also contains plans to cement London's position as the global hub for trading in the Chinese currency renminbi (RMB) by giving investors the chance to invest RMB directly into China through London under a pilot scheme.
London will become the first place outside China to be granted an RMB Qualified Foreign Institutional Investor quota, which will initially be set at 80bn RMB or £8.2 bn.
Mr Osborne said: "A great nation like China should have a great global currency."
Meanwhile, it is expected the UK Government will back the Chinese General Nuclear Power Group becoming a co-investor in a planned new nuclear power station at Hinkley Point in Somerset.