I don't understand Remote Connection!

When I remote connect to my pc via the internet, a prompt on my computer I'm connecting too get a dialog box asking if I want to allow connection. I tried to click it but the application wouldn't respond and it closed the connection now I can no longer connect to that computer. This also happens when I close the connection after a sucessful connect. After I close the session I can no longer connect to that computer again. Yes, I do have it setup for unattended remote connection.