There are a couple of ways to do this. A good approach might be to use data blending to a copy of the data source and disable linking fields. If you base the averages on the secondary data source, you'll always get the overall average even when the primary data source is filtered to different directors.
There are other options as well. Posting a packaged workbook (mocked up) would help everyone see if there are other good options as well.
I've done this in the past by using an Average of a Total(sum([Field])) per pane, but the approach taken will depend on your data and what is being displayed in the view, etc.
but i have also other sheets in the dashboard which has actions of the sheet which has average reference line.
So Zone Director sheet has action on location sheet and location sheet has action on another bar chart and line chart.So creating a secondary source will affect the actions