By Set, I assume you are using the 'Set' in Tableau to define this.
This drove me a bit crazy for a while (fortunately I'm already bald as a coot!!...I think that occurred during my Excel days!!). The trick to get a filter to affect a set is to make the Date Filter a 'context' filter. If you click on the Date in the filter shelf, and select 'Add to Context' the set will now reflect the filter.
This is to do with the order of operations in Tableau. btw this is also the case if you want a FIXED LoD to be affected by a filter (although not in the below image, the set is done at the same place as FIXED).
Hope this solves your issue, but if not please post back