Have you tried setting the second click action to leave the filter upon clearing the selection? Of course this is very tricky in that the hover will almost always be triggered on ANY movement of the mouse thus negating the second click action.
I think the real issue has more to do with user experience than it does Tableau functionality. As John mentioned above, any movement when using a hover action is going to initiate something. Furthermore, as a best practice I would avoid trying to give your users this type of experience because instead of being helpful it ends up just creating a confusing user experience.
I'd recommend getting rid of one of your actions (I try to avoid using hover actions in these situations) because right now it can't/won't work the way you want it to, creates a confusing user experience, and it fixes the problem and simplifies your UX.
Many thanks for your input. I think the solution is the following: I just need to make a copy of the field that I am using in the action filters. The hover action passes on the original field. The click to filter action works with the copy.