hopefully, again

There's some nice, straightforward usage advice at Oxford Dictionaries. One entry is on "hopefully", which says, in part:

Many people object to the use of hopefully as a sentence adverb. They compare it with other sentence adverbs such as ‘unfortunately’ or ‘clearly’, which can be paraphrased as ‘it is unfortunate that ...’ or 'it is clear that ...':

Unfortunately, he missed the train. [i.e. it is unfortunate that he missed the train.]
Clearly, they have made mistakes. [i.e. it is clear that they have made mistakes.]

It’s certainly true that you can’t paraphrase hopefully as ‘it is hopeful that’. But this is no reason to ban its use as a sentence adverb: there are no grammatical rules that say the meaning of a word mustn’t be allowed to develop in this sort of way.

(Sadly, they end with the standard "let the nuts win" argument:

Nevertheless, if you are making a formal speech or writing formally (e.g. preparing a report or drafting a job application), you should be aware that there are people who intensely dislike this usage. For some, it has become almost a test case of ‘correctness’ in the use of English, even if the arguments on which their view is based are not very strong. Consequently, in this type of formal situation, it would be better to choose a different adverb or reword your sentence altogether.

And sure, if you're writing a job application I guess you want to err on the side of caution. But again: do you really want to work for someone who'd base his hiring practices on "hopefully"? But I digress.)

My main point is that straw man there in the middle. Sure, "hopefully" doesn't mean "It is hopeful that...". But does "Thankfully, he was unhurt" mean "It is thankful that he was unhurt"? Does "Candidly, I don't like your sister" mean "It is candid that I don't like her"? Did Rhett Butler mean "It is frank that I don't give a damn"? Some sentence adverbs modify the sentence. Some, on the other hand, are extra-textual and modify the speaker. Why is "hopefully" the only sentence adverb like that to be banned?