This is usually all at the cost of -1 to agility, a huge weakness to Pulse weapons and again, you move slower. Heavy armor provides even MORE DT, perhaps some STR, and, through a perk, more resistance to the seldomly used energy weapons (Fiends are the only EW enemy that one can expect to constantly run into, and they're hardly dangerous). Medium armor provides slightly more DT, but nothing more than that. Don't underestimate any of those Endurance is self-explanatory, CH improves your followers and agility improves your reload) but doesn't provide any good DT. Normal clothing can offer faster movement and ocassionally some nice stat bonuses (+1 Charisma, endurance or agility.
I don't know about you, but an enemy crit is one of the most dangerous obstacles for me well worth losing the extra DT just to avoid those. With perks, light armor lets your character move faster and crit more often PLUS decreases the likelihood that an enemy crits you.
Seriously, is there a point to any other armor besides Light Armor?