how much memory is deployed on your Tableau Server instance and if its a VM, then is it all dedicated memory to this instance?
What version of Tableau Server are you using to display the viz?
I am running Tableau Server 9 on an Amazon AWS instance of Windows Server 2012 Standard.
The system specifications list the following:
Processor: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80Hz 2.79GHz
Installed memory (RAM): 15.0 GB
System type: 64-bit Operating System, x64-based processor
I am having the exact same problem. Error doesnt occur all the time and I also have very similar configuration on AWS.