When you use the "Show Me" Option for Box and Whisker plot, it'll change the arrangement of the pills (green and blue things)
Make not of how the pills are aligned for the scatter plot and try to recreate that layout for the box and whisker plots.
If that does not work, build them "by hand". Here is a helpful video! Box Plots in Tableau - YouTube
Let me know if this helps
Thanks Matthew, I realized how to do this nearly immediately after posting .
I was able to get what I wanted by going at it manually and adding a "trend line" to my scatters. I had skipped over that before when reading.
Appreciate the help!