I'm also facing exactly the same issue. Any updates on this will be appreciated.
TSM credentials are NOT the same thing as Tableau Server credentials.
Any valid domain or local account with local administrator privileges on the server running TSM will work; that may by coincidence be the same credentials as your Tableau Server Administrator account, but it's usually not.
Try this with an account you KNOW has local admin privileges on the Windows Server that hosts Tableau Server.
I had exactly the same issue, but I had spotted that the port used for TSM (8850) was not open. Therefore, step 1 and 2 would work successfully as it is just connecting to the Tableau Server Portal, however any external connection via another machine to TSM would not connect.
Once port 8850 was opened everything worked as it should.
Do you mean that port 8850 should be opened on the machine which installed the TabJolt.