Your Average Duration calculated field is close. The only problem is that you're using the "day" in your datediff calculation. This will calculate the difference in units of days (not partial days). To make this a more precise value, I'd suggest using minutes, then dividing by the number of minutes in a day, like this:
DATEDIFF('minute',[Start Date],[End Date])/1440
This will give you the fraction of a day that is between the start and end date.
Thanks, this worked. I also broke it down to hours.