Can't tell much just by looking at what formula you used. Are you able to supply screenshots or a Tableau packaged workbook? Sample data will almost always get you the fastest repose. My initial (guess) is you need to wrap the measures in SUM(). So try:
SUM([Time on Page])/(SUM([Pageviews])-SUM([Exits]))
But I could be wrong, because I am not sure how your data is structured. Hope this helps.
Please use aggregations(Sum, Avg.. ) in your calculated filed. other wise it would be giving you wrong data.
This is the formula I use: (sum([Time on Page])/(sum([Pageviews])-sum([Exits])))/86400
Hope it helps,