Hi ljaz Ahamed,
I suggest you watch the following tutorial video:
This video is exactly about what you are asking.
Also you can read the article Finding the Top N Within a Category | Tableau Software
Hi ljaz Ahamed
Another simpler way that may work for you is to set a Context Filter at the category level, then a Top N filter on another category. Context filters are applied before any other filter and can thus help may you achieve your goal.