You are correct that to do forecasting, you will need to have a all the associated data marks in the view. Normal tricks to hide data while it remains considered for calcuations will not work with forecasting since the addition of the table calculation will break the view. Forecast Option Not Available (Grayed Out) | Tableau Software describes what causes Forecasts to not appear and may be helpful to others stumbling on this question.
I am curious about how you managed to get a calculation workaround. If you can share, please do as I'm sure it would have value to other users.