You're using vi mode, so ESC enters vi command mode (as opposed to vi editing aka insert mode), just as would happen if you hit ESC while in vi itself.
Maybe I'm using the terminology wrong, but I would say <esc> takes me to "normal" mode and <s-:> takes me to command mode. So, yeah, I do have a use for normal but, but only rarely.