The head of Mr Putin's human rights council said this week that Khodorkovsky and two women from the punk band Pussy Riot could benefit from the amnesty marking the anniversary of the adoption of Russia's post-Soviet constitution in 1993.
But, without naming names, Mr Medvedev said Russia should be careful about freeing people convicted of crimes like hooliganism - the charge in the Pussy Riot case - and theft, which was the indictment against Khodorkovsky.
"Our people really are not much inclined, for example, to conduct acts of amnesty for individuals involved in violent crimes, for individuals who committed crimes against society, including hooliganism," Mr Medvedev said in a TV interview.
"People are not very inclined to decide to issue an amnesty for those who committed major state crimes [such as] theft," he said. "And so the president and parliament must listen to public opinion."
Khodorkovsky's supporters say the former oil company chief, who was arrested in 2003, was jailed to curb a political challenge to Mr Putin, bring his oil assets under state control and send a signal to other tycoons to toe the line.
In the eyes of Kremlin critics at home and abroad, Khodorkovsky's jailing is one of the biggest stains on the record of Mr Putin, who was first elected president in 2000 and has not ruled out seeking another six-year term in 2018.
He is due for release in August but some activists doubt he will walk free as long as Mr Putin is in power. The authorities have continued to investigate his activities, prompting frequent speculation he could be hit with new charges.
Deputy Prosecutor-General Alexander Zvyagintsev issued a thinly veiled warning that could happen, saying several cases involving Khodorkovsky were being pursued and had "good prospects for going to court".
Khodorkovsky was convicted in 2005 and again in 2010, prompting accusations that the government has tried him twice for the same activities. A third trial would badly damage Russia's image unless there was iron-clad evidence of wrongdoing.
Mr Putin has suggested Khodorkovsky has blood on his hands, referring to a murder conviction against a former Yukos security chief. Mr Zvyagintsev gave no details about any case involving Khodorkovsky.
Khodorkovsky and the jailed women from Pussy Riot are considered political prisoners by Kremlin opponents, but Mr Medvedev said that there are no political prisoners in Russia. Mr Putin has repeatedly said the same thing.
Pussy Riot's Nadezhda Tolokonnikova and Maria Alyokhina are serving two-year sentences for a "punk prayer" protest against Mr Putin in Russia's main cathedral. They are due for release in March.
Western governments have called their punishment excessive and government opponents saw their trial as part of a clampdown on dissent during Mr Putin's third presidential term. Both Tolokonnikova and Alyokhina have young children, which officials have suggested could be a factor in the decision.