Kirk Ramsay said Mr Russell was guilty of an "abuse of power" after he resigned as chairman of the board of Stow College. He also called on Alex Salmond to step into the row.
MSPs accused Mr Russell of running education as a "dictatorship" after Mr Ramsay's decision on Tuesday evening.
It followed a row over the college chief's decision to make an audio recording of a meeting with Mr Russell and 80 college officials using a smartpen.
At Holyrood, SNP MSP Stewart Maxwell blocked calls for an inquiry from a cross-party group of its members.
Mr Ramsay believes he was forced out after an "unwarranted personal attack" by Mr Russell, who demanded his resignation.
Mr Ramsay told The Herald that if a manager had treated an employee in the way he had been treated by Mr Russell, he would have faced an employment tribunal.
He said: "Bullying was what it felt like, and it felt like a set-up to intimidate me because I had no prior knowledge that there was a problem before I was called to see him in the parliament.
"The implications for him are both poor and sad because I think he was starting to show more understanding of the sector, but this kind of thing will create more fear.
"My phone is filled with messages from senior people in the college sector expressing fear about the future, and we should not live in a society where people are afraid."
Mr Ramsay said it was time for the First Minister to step in to ensure there was no repeat of the incident with others.
He added: "It is his responsibility to stop it happening. Is this really the political society we are being invited to enter as an independent country? Because if that is the case, then I want to get off.
"In circumstances where I am not treated well I do not walk away quietly, and this was about as gross an overreaction as I have seen and a startling error of judgment based on an emotional reaction to a misunderstanding.
"What you are talking about is the Education Secretary acting as judge, jury and executioner without looking at the facts.
"If you are speaking to a substantial group of 80 people in a hotel where you have got public access then you cannot call it a private meeting and people around the room are taking notes in various different ways."
As The Herald revealed on Saturday, Mr Russell also wrote to all college principals and chairmen saying Mr Ramsay's actions were not consistent with those of a chairman of a college board.
The row centres on the meeting in October between Mr Russell and some 80 college principals and chairmen – which Mr Ramsay recorded.
Conservatives, Labour and Liberal Democrat MSPs on the Holyrood education committee signed a joint letter raising concerns over a culture of "secrecy, bullying and intimidation" in Mr Russell's dealings with colleges.
Labour education spokesman Hugh Henry said: "He [Mr Russell] has used unethical tactics to get his way.
"He has tried to intimidate when discussion would have been better.
"What we have seen from this cabinet secretary is dictatorship and imposition."
Liz Smith, the Scottish Tory education spokeswoman, accused Mr Russell of "bullying" while her LibDem counterpart, Liam McArthur, said: "The Education Secretary's behaviour over this matter betrays seriously poor judgment."
Mr Salmond said: "It wasn't Mike Russell that put the recording device into the meeting, it was the former chairman of Stow College, and I think he's done the right thing resigning.
"Mike Russell made it very clear that he had no power to remove the chairman, the chairman made his own decision."