I am working on a very similar problem. I am identifying customer segmentation using hierarchical clustering, and dendrograms are one of the outputs. Tableau has limitations around this visual. Node links and Sankey graphs do not provide the results I am looking for.
So I have decided to work with my hierarchical data in R and produce the output using the dendextend package. There is a possibility to use Rserve and send the R script via calculated field, but I see no value in that. It's better to just produce the dendrograms in R and export as PDF.
Let me know if you have any questions.