Create a calc:
IF datename('weekday',[your date field]) = "Thursday" then [your date field]-1
[your date field] END
This will subtract a day from any row where the day is Thursday.
Then create your viz using that calc field instead of your original date field.
It took out Thursday but did not add those orders to Wednesday. Am I missing a step?
Provide me an excel sheet with a dozen or so rows. Make sure some (not all) of the rows have Thursday.
What version of Tableau are you using? I'll make an example.