You need to use an ssh agent. Short answer: try

$ ssh-add

before pushing. Supply your passphrase when asked.
If you aren't already running an ssh agent you will get the following message:

Could not open a connection to your authentication agent.

In that situation, you can start one and set your environment up thusly

eval $(ssh-agent)

Then repeat the ssh-add command.
It's worth taking a look at the ssh agent manpage.