The [Group Switch Calculation] would need to be in the hierarchy in order for the drill down/up option to show in the view. However if the calculated field is placed in the hierarchy then the drill up/down options will appear no matter what the user has selected.
It is possible to create a calculated field that will only show values for [B] when the user chooses "A" with something like:
IF [Group Switch] = "A"
The downside is that when the user chooses something other than "A" then the above column will display NULL values rather than being removed from the view. Currently there is no way to dynamically hide a column from the view based on the contents of the column.
I hope this helps!