On the date quick filter, from the little drop down arrow in the upper right, you should be able to select "Only Relevant Values" which will restrict the date range based on all other filters.
Also, you could make the User filter a context filter (right click the field on the filters shelf and select "Add to Context") and then make the Date filter "Only Values in Context". Tableau will (if possible) create a temporary table for only the subset of data that is returned by the context filter. In some cases this can lead to better performance (with a slight hit at the beginning to create the temporary table). User filters are often a good choice for this because they are not likely to change (which would require the context to be rebuilt) during a single user's interaction.
There are a few other considerations with context filtering. One example is that Top N filters and sets will be within the context and not the entire data set. So also take that into account when deciding if it is the right approach.
That was so obvious, I can't believe I missed that. Thanks Joshua
Using the "Only Relevant Values", is there a way to disable the option "All"?
Edit: I looked it up and found that under the dropdown there is a "Customize" menu item. There it will let you deselect the "Show 'All' Value".
You're welcome! Glad to help.