Some talk of the Wee Frees in the same breath as the Taliban, others just poke fun, while fantasy author Terry Prachett "just liked the rhythm of the words" when he wrote about "The Wee Free Men" in his Discworld series.
Now the Moderator of the Free Church of Scotland, Rev Dr Iain D Campbell, has had enough.
He is calling for a freeze on the use of "Wee Frees", which he believes is an "unnecessary slur" on a denomination which today has 12,500 worshippers in 104 congregations across Scotland.
The phrase is also sometimes wrongly applied to the Free Presbyterian Church of Scotland – a completely different organisation, which seceded from the Free Church in 1893, leading to its people being known as the Seceders or FPs.
The move by Mr Campbell – minister of Point Free Church, on Lewis – comes at a time when the church has been modernising itself, even embracing the social networking revolution by advertising on Facebook in a bid to reach out to a wider audience. It is also on Twitter.
Mr Campbell said he was speaking out after some media outlets had used "the outdated and offensive stereotype in reference to the Free Church in recent weeks".
He added no other church had been subjected to such long-standing ridicule.
The term was first coined in 1900 and Mr Campbell claimed it had referred to the minority of Free Church members who that year chose not to merge with the United Presbyterian Church to become the United Free Church of Scotland.
Mr Campbell said: "None of the men who led the Free Church in 1900 and refused to enter the dreaded union are alive now.
"We belong to the denomination with the same name as they had, but I can't see any sense in referring to us by the same nickname by which they were known 112 years ago.
"I'm not so sure that 'wee' is a suitable nickname for a denomination that is represented throughout Scotland, has its main offices in Scotland's capital, maintains a consistent presence on the mission field, and organises camps for hundreds of children each year. It seems that once you put the label 'Wee Free' on to something, it can justifiably be the butt of any joke and truth doesn't come into it.
"It's a bit frustrating when the wider media churns out the most ridiculous and meaningless names for a church.
"I can't imagine any other religious body receiving the kind of treatment with which the Free Church has had to put up for years.
"Sticks and stones break our bones, and names hurt us."
The church says that as well as hosting a camps programme that attracts more than 400 youngsters each summer, it boasts some of the biggest religious congregations in Scotland – including Stornoway Free Church on Lewis, Dowanvale Free Church in Glasgow and Smithton-Culloden Free Church in Inverness.