The Swede, manager Neil Lennon's right-hand man, told the club's website: "I have enormous affection for Celtic Football Club and it is with sadness that the time to leave has arrived. However, I now leave to seek some different opportunities."
Lennon immediately brought in Mjallby, his former Celtic team-mate, when he took the manager's job on a temporary basis in March 2010, and their appointments were soon made long-term.
The former Sweden defender previously spent almost six years as a player at Parkhead after being signed by Jozef Venglos in November 1998.
Mjallby, who has helped Celtic to three consecutive titles, said: "The club has been part of my life for almost two decades and I have been dedicated to them in every way, first of all as a player and then in management.
"As a player, it was a time when we achieved great success and I had the privilege to work with football people and players who were first-class and real quality. We had some great moments and memories I will always cherish. It was an honour to return to the club and work in management with Neil, Garry [Parker] and the players, and we have worked hard over the last few years to once again bring the club success.
"I would like to sincerely thank all our supporters for all they have given me and everything they have given to the club. They are special fans, the best I have seen."
Lennon said: "I will be very sad to see Johan leave the club. He has given us such great service over a number of years and he will always be a big part of Celtic. When I was given the chance to be in charge of the team, Johan was the obvious choice to join me in tackling the challenges ahead and, from day one, he has been a very important part of our success.
"He was always respected by players and supporters, who knew how committed he was to Celtic. Johan is a great man and will always be a great friend. I know whatever challenge Johan chooses next, he will be a great success."
The club will now begin the process of appointing a replacement. Lennon has former Leicester team-mate Parker and former Celtic captain Danny McGrain on his first-team coaching staff.