I've tried Tableau Public on another Mac and the results are the same. Also, a Tableau sales person phoned me on Friday and seemed not to be surprised that there might be issues with the Mac version that do not occur with the Windows version.
I've done further testing of the certificate chain and have not been able to alter the behaviour by altering the certificate chain served by the server. Also, when the test by Qualys' ssllabs.com reports no problems with the certificate chain, Tableau still can't finish the SSL negotiation.
So given this, it seems that the Mac version is unusable with Web Data Connectors hosted on some web servers with SSL configurations that pass Qualys' SSL server test and also work with all browsers I've tested with.
This is disappointing.