If Self Defense is the Goal then one should work technique,through such work one will develop enough fitness to fight a good ,Fast,Fight.
Self Defense should not be an Aerobic event,it should be short,Fast, Burst anaerobic energy use and be done quickly.
Self Defense situations should be over in seconds,not minutes.
(though training for self defense can be made to push the envelop of time and extending it,this still does not require endurance athletes)
If one wants to engage in an exchange as in Sport then athletic conditioning and endurance is a must.
But
Endurance capacity does not mean one will win a street self defense situation.
(I can take a marathon runner beginning fighter and put him is a self defense situation and his /her heart rate will jump up and they will tire out as fast as the unfit,so endurance is not a predictor of anything)
Self Defense should be doable by Men ,Women,Children,Young to Old,Short to Tall,Fit to Fat to include those with handicaps of what ever nature.
Self Defense is not a battle of the fittest,it is a battle of those with the Superior Mind set to win and some skill against some one else who has a skill level we do not know,nor do we really care about.
My two cents.