Vim is a modal editor, meaning it has modes. It's modes are command mode, and edit mode (AKA normal mode and insert mode). Edit mode allows you to add and delete text and move your cursor around. Command mode allows you to execute various commands that can operate on the text.
Most editors only have edit mode. To execute commands you use shortcut keys that must depend on non-printable keys like CTRL, ALT, SHIFT. You quickly run out of simple combinations (ex: CTRL+S), so you have to introduce more complicated key combinations (ex: CTRL+K, CTRL+C). Visual Studio calls these "chords." I call it keyboard gymnastics.
Some people like the "chord" approach, and other people prefer the mode approach. I like the mode approach because if I'm going to execute non-trivial commands, I'd like to type a command. The main downside to the mode approach is you have to get back and forth between the modes. When you launch Vim and open a file you start out in command mode. There are a bunch of useful different ways to get into edit mode, all of which are just a single keystroke: i, a, o, O, I, A.
Once in edit mode, getting back to command mode is a bit harder because now typing a key will print the character, so Vim uses ESC to leave edit mode and go back to command mode. Sadly, on modern keyboards this goes against everything Vim stands for because the ESC key is so far away from the home row. And if you're using Vim properly, you'll be going back and forth between these modes A LOT. So we need a better way.
Vim has an alternative built right in: CTRL+[
(You've mapped your Caps Lock key to Control, right?)
This is kind of an uncomfortable keystroke, but with practice you will get used to it. If you'd rather not, you CAN map your own keyboard combination to Esc. For example, you could try CTRL+space:
inoremap <C-space> <Esc>
Whatever you use, I strongly suggest you try to stop using the Escape key.
UPDATE:
A number of people in the comments suggested using various combinations of the j and k keys. I didn't even realize you could remap printable characters, but you can! I've been using this for quite a while now and I'm completely addicted! I call it the "smash":
Add this mapping to your vimrc:
inoremap jk <esc> inoremap kj <esc>
Now all you have to do is smash the j and k key. It doesn't matter which you type first! Just smack them both as if it was a single key! Absolutely brilliant!