Emmanuel Adebayor headed Tottenham in front in the 35th minute and tapped in their third after a Chico Flores own goal had doubled the visitors' lead.
Wilfried Bony hit the bar and had a penalty appeal for a push by Michael Dawson turned down in the first half, before getting Swansea City's consolation late on.
The win moves Tottenham up to fifth, level on points with fourth-placed Liverpool as Sherwood, who opted against sticking with a 4-4-2 formation, remained unbeaten in the Premier League since taking over from Andre Villas-Boas.
Sherwood, who said Swansea should have had a penalty for Dawson's push on Bony when the game was goalless, has been delighted by his start as Tottenham manager since signing an 18-month contract. But he is in no doubt as to what is expected of him when it comes to the London club's finishing position.
"We know there are going to be tougher tests, but it's been a good start," he said. "If the season were to end tomorrow I don't think the chairman [Daniel Levy] would be too happy because we are fifth. The final league position has to meet the expectation of the club otherwise it's 'goodbye Charlie'.
"The club needs to finish in fourth place, we want to finish in fourth place. Anything other than that will be a disappointment but realistically we should be in amongst it."
Adebayor has scored six goals in seven appearances under his new manager, but Sherwood insisted the striker's revival is not down to him. "I haven't said anything to him, I have just given him the stage to play on, he hadn't had that stage for a while," said Sherwood.
Swansea City, meanwhile, remain just three points above the relegation zone after another defeat and another injury, as Jonjo Shelvey's hamstring problem means he joins Michu, Michel Vorm, Pablo Hernandez, Jonathan de Guzman, Nathan Dyer, Jose Canas and Garry Monk on the sidelines.
Manager Michael Laudrup admitted his side are in a difficult spot, having failed to win in eight league games.
He said: "Everything is black and negative right now. Let's start again tomorrow or the day after and see if we can get some players back to fitness and back in the team. Then it is a matter of getting a win, like a lot of teams in our situation. We just need a little more luck in certain situations."