It has nothing to do with Terminal vs iTerm and everything to do with what the shell running inside them is configured to do. The terminal emulator is not providing the line editor, the shell is.
From the zsh documentation:
If one of the VISUAL
or EDITOR environment variables contain the string ‘vi’ when the shell starts up then it will be ‘viins’,
otherwise it will be ’emacs’. bindkey’s -e and -v options provide a convenient way to override this default
choice.
Regarding your behavior differences, look at what combination of your shell startup files is being read by the shell when it is started by the terminal emulator. One or another of them is most likely loading the shell as a login shell, while another of them is not.
Also, the phrase “readline mode” is meaningless. Put bashin vi mode and it’s still using readline for the line editor.
