Found the answer. The text I was seeing was the caption. This can be enabled within Tableau Desktop under Worksheet -> Show Caption. Previously in version 8.0, Tableau server showed the caption automatically when exporting to PDF. This was unintended, and has been fixed as of 8.2 (was actually fixed somewhere in the 8.1 cycle). Unfortunately, there doesn't seem to be a way to only show captions in the PDF, and not on the Dashboard.