The SkyBet Championship visitors sensed an upset after Fernando Forestieri and Troy Deeney gave them a half-time lead.
In the second half however, Aguero's goals and one from Aleksandar Kolarov were enough to turn the tie.
Liverpool won 2-0 at Championship side Bournemouth with goals from Victor Moses and Daniel Sturridge.
Cup holders Wigan beat top-flight Crystal Palace 2-1 at the DW Stadium. Cup final hero Ben Watson put his side ahead in the 36th minute and although Aaron Wilbraham equalised, James McClean restored Wigan's advantage.
Championship side Birmingham threatened a surprise when Lee Novak gave them a 15th-minute lead at home to Premier League Swansea but two goals from Wilfried Bony turned things around and decided a 2-1 win.
Non-league Kidderminster put up a brave fight at Sunderland but went down 1-0 to a fifth-minute goal from 19-year-old Charis Mavrias.
Premier League sides Hull, Southampton and Cardiff all made it through. Matty Fryatt scored both goals as the Tigers beat League Two Southend 2-0, Guly Do Prado and Sam Gallagher scored in the Saints' 2-0 win at home to Yeovil while Frazier Campbell's strike was enough to defeat Bolton 1-0 at the Reebok Stadium. Charlton won 1-0 at Huddersfield, Brighton won 3-1 at Port Vale and Rochdale lost 2-1 at home to Sheffield Wednesday.