There are two options I can think of. The first would be to use the window_max or window_min date in the datediff calculation. This would look at the earliest or latest date in the view to perform the difference.
The other option would be to use a parameter instead of the typical date filter. The parameter can be used in the calculation and will thus be based on that.
Hope this helps!
Thanks Tracy, I figured that might be the case the first option is only good if your data actually extends all the way over the defined filter period. In my case the data might start a couple of days after the month or qtr.
Parameters it is.