We need a workbook for exact your situation to help you.
first the way you went -
you created a calculated filed by concatenating 2 fields - the effect of that is to drop another column in the table that underlays the viz - so now there aren't 2 columns there are 3 - you can set up your filters in from the least specific to the most specific - using Only relevant values as shown below - (that is the opposite of you posted but you can't go back up the filter)
a second approach is to create groups (actually I would be surprised if these groups weren't already in your data base as "Product Group" Product Family" or something else
then do a search
the group them together as shown
but again you are going from least specific to most specific
The third way ( the one I would do with 50k skus - is to set up a cross reference table outside of tableau in excel and upload it
the table would have a column for the MOST specific level and another for each higher level that you want in the hierarchy
then bring it into tableau and join on the most specific dimension (SKU)
If this posts assists in resolving the question, please mark it helpful or as the 'correct answer' if it resolves the question. This will help other users find the same answer/resolution. Thank you.