In general - you can not have a calculation that calculates after one set of context filters and then another after a different set of context filters. Context filters are applied next after Data Source and Extract Filters (See Order of Operations here - Tableau's Order of Operations - Tableau )
It might help to share more information about your data and a sample workbook to help the community better understand the issue.
You have hit upon a limitation of filtering and FIXED calculations. All context filters will filter all FIXED calculations in the scope of the worksheet. There is no way to specify which FIXED expressions should be filtered by which fields on the Filters shelf.
A possible alternative is to replace the filters with a parameter. An example of this is option 2 in Creating a Calculation that Ignores Filters
Hope this helps!