This is actually expected behavior. Adding an AD Group adds all the users within that group however when the group is removed, that action does not remove the users in that group. To remedy this issue, you'll need to remove the users via the Tableau Server User Interface or using Tabcmd. See tabcmd Commands for more info.
The below quote is from the Online Help Guide for Synchronize Active Directory Groups in a Site:
What happens when an Active Directory group is removed from Tableau Server?
Many Tableau administrators use Active Directory groups to import and create users. After the users are imported into Tableau Server, administrators will then delete the group in Tableau Server. Deleting a group does not delete the users in it.
Thanks and hope this helps!