Sir John, who was premier from 1990 to 1997, claimed the referendum debate was a battle between "emotion and pride" on the Yes side and "facts" from the No campaign.
Addressing a journalists' event in Edinburgh, he said: "We were at war centuries ago which is why, to be brutally frank, I find it rather sad that the SNP chose the anniversary of the Battle of Bannockburn for the vote, presumably to maximise the opportunity for any anti-English sentiment that may exist."
He said the SNP had ignored "two bigger anniversaries" this year, D-Day and the outbreak of WWI, when Scots fought as "brothers in arms" with comrades from the rest of the UK.
Sir John accused the SNP of seeking to create "grievance and ill-feeling" between Scotland and the rest of the UK for the past 20 years as a tactic to win support for independence.
He warned that in the event of a narrow victory for the No campaign, the debate over independence would continue but become "even more divisive than it has become, not only within Scotland but between Scotland and the other component parts of the UK".
Sir John, who has intervened only rarely in the independence debate, mounted a passionate defence of the Union, describing it as one of the most successful partnerships in history.
He claimed First Minister Alex Salmond was "deceiving the Scottish electorate" about the consequences of leaving the UK.
He dismissed the SNP's claim that an independent Scotland would join the EU swiftly and on the same terms as the UK, including a share of its valuable budget rebate.
Sir John, who negotiated the UK's opt-out from the euro single currency in 1991, said the chances were "somewhere between nought and nil". He also called for a major new constitutional agreement between the parties to devolve powers to Scotland but balance the move by recognising the impact on England.
He said an "equitable long-term settlement" would have to answer the West Lothian Question or "West Berkshire Question," as he renamed it - over Scots MPs having a say on policies affecting only England.
He said he supported further devolution for the same reason he opposed it in the early 1990s: to "protect the Union".
He also said it was "absurd" to argue the three main pro-UK parties would backtrack on promises to hand Holyrood more powers in the event of a No vote.
He said: "The three major parties of the UK could not possibly resile from such explicit and important commitments and I'm confident they will not."
SNP MSP Stewart Maxwell said: "John Major is perfectly entitled to his view, but his comments are woefully out of touch.
"He was wrong about a Scottish Parliament in the 1990s, and he is wrong about an independent Scotland now."