Olivier D is almost correct, but you must set POSIXLY_CORRECT=1POSIXLY_CORRECT before running unset. POSIX has a notion of Special Built-ins, and bash supports this. unset is one such builtin. Search for SPECIAL_BUILTIN in builtins/*.c in the bash source for a list, it includes set, unset, export, eval and source.
The rogue unset has now been removed from the environment, if you unset command, type, builtin then you should be able to proceed, but unset POSIXLY_CORRECT if you are relying on non-POSIX behaviour or advanced bash features later.
This does not address aliases though, so you must use \unset (or use some form of quoting/escaping on that word) to be sure it works in interactive shell (or always, in case expand_aliases is in effect).
(while, do, done and [[ are reserved words and don't need precautions. You can't use = in an alias or function name, so no precautions needed for setting POSIXLY_CORRECT.)
Note we are using unset -f to be sure to unset functions, although variables and functions share the same namespace it's possible for both to exist simultaneously (thanks to Etan Reisner) in which case unset-ing twice would also do the trick. You can mark a function readonly, bash does not prevent you unsetting a readonly function up to and including bash-4.2, bash-4.3 does prevent you but it still honours the special builtins when POSIXLY_CORRECT is set.
Thanks to commenters below for the extra suggestions.