The home side won 2-1 to make it 2-2 on aggregate and won through 7-6 on penalties after Fraser McGhee had missed the final penalty for Clyde.
Dylan Rooney, who replaced the injured Greg Paterson at half-time, was the East Fife hero as he saved penalties from Stefan McCluskey and Michael Daly in the shoot-out.
East Fife took the lead after 12 minutes when Scott McBride evaded his marker to head home a Stephen Hughes free-kick.
Scott Ferguson equalised just before the break when he raced clear of the home defence to lift the ball over Paterson's head. Kevin Smith forced the tie into extra-time with a left-foot shot with just five minutes left.
Jordan White hit a hat-trick as Stirling Albion beat 10-man Annan Athletic 5-3 at Galabank to go through 8-4 on aggregate. David McClune gave Stirling an early lead and White put Stirling further ahead.
Martin McNiff pulled a goal back five minutes before the break when his header eluded David Crawford's grasp and Ally Love equalised.
McNiff was sent off after picking up his second yellow card and Craig Comrie scored Stirling's third.
White's second put more daylight between the teams and he completed his hat-trick with a superb finish before Scott Davidson scored a late consolation.