Altidore thought he had equalised for the home side in the 70th minute, only for referee Martin Atkinson to haul back the action and give the home side a free-kick for an earlier foul by Bacary Sagna. Three minutes later Ramsey was sent through by Olivier Giroud to wrap up a 3-1 win for the Londoners.
Arsene Wenger's men dominated the first half but had only Giroud's 11th-minute opener to show for it after Sunderland were carved open by £42.5million debutant Mesut Ozil.
Craig Gardner brought the hosts level from the spot after a foul by Laurent Koscielny on Adam Johnson early in the second half. Ramsey put Arsenal back in front with a superb volley in the 67th minute, before Atkinson's controversial intervention frustrated Sunderland, whose manager Paolo Di Canio was sent off in stoppage time.
Tottenham new boy Christian Eriksen inspired his side to a comfortable 2-0 win over Norwich at White Hart Lane.
Eriksen set up the opener for Gylfi Sigurdsson in the 26th minute, and the Iceland international made it 2-0 from close range four minutes into the second half after a Paulinho cross.
Sluggish Manchester City were lucky to escape with a draw against Stoke at the Britannia Stadium, as home boss Mark Hughes got one over on the side he used to manage.
City's neighbours Manchester United had no such problems in the early kick-off against 10-man Crystal Palace, with goals from Robin van Persie and Wayne Rooney earning them a 2-0 win. Van Persie put United in front with a penalty in first half injury-time after Kagisho Dikgacoi was sent off for a foul on Ashley Young and Rooney made the game safe with a superb 81st-minute free-kick.
Newcastle substitute Yoan Gouffran came off the bench to claim a 2-1 win for his side away to Aston Villa. Hatem Ben Arfa had given Alan Pardew's men an early lead but Christian Benteke levelled with a header.
Gareth McAuley's injury-time header snatched a dramatic point for West Brom against Fulham at Craven Cottage after Steve Sidwell had given the hosts a first-half lead, while Peter Whittingham earned a 1-1 draw for Cardiff at Hull, who had led through a 40th-minute header from Curtis Davies.