I believe whether or not you get charged the admin fee is based on how well you pay your bills, how often you have your handset replaced - if Orange has ever had a handset returned to them from you in the past which wasn't faulty as it was originally described by yourself.
E.g. Your screen had a dead pixel, you told CS that it had a broken keypad. When it gets back to Banbury, it is taken note of.
CS actually read out a list of T&C's and things like "you sure X is your address, X fee if you aren't there, X fee if you hold up the driver" etc. One of those is along the lines of "if it's not broke you'll be entitled to a charge".
Basically it boils down to if Orange have evidence that they aren't going to loose out, either because you don't pay your bills on time, or you keep returning handsets for false reasons, or for other reasons. Besides - look how car insurance works - there's always an extra fee, that's just the nature of insurance.