11 Replies Latest reply on Jan 22, 2016 2:52 PM by Michael Perillo

    update to primary server IP

    Jeff Strauss

      Over this past w/e, the IP of our primary Tableau server was updated to a new VM cluster within the network.  I followed instructions in the admin guide which states when this occurs, run a tabadmin stop, tabadmin config, tabadmin start in order to update communication throughout the cluster.  None of the commands worked (including "tabadmin stop").  Anybody have any ideas as to why or what to do?  For now, we rolled back the IP change.

       

      ORD-TBL107 (primary):

              Status: RUNNING

      ORD-TBL104 (worker):

              Connection error contacting worker 1

      ORD-TBL105 (worker):

              Connection error contacting worker 2

      ORD-TBL106 (worker):

              Connection error contacting worker 3

        • 1. Re: update to primary server IP
          Toby Erkson

          Yikes!  No idea.  Unless John "Iron Man" Kuo throws a couple pennies your way I would be calling Tableau Support...

          1 of 1 people found this helpful
          • 2. Re: update to primary server IP
            Jeff Strauss

            well, we're testing now in dev to see what works and what doesn't work.  Usually a best practice!

            • 3. Re: update to primary server IP
              Gerardo Varela

              I'm interested in what steps you took in your initial try. Did you remove the workers or just point them to the new primary? The following quick fix article states to basically do a brand new install. 

               

              http://kb.tableau.com/articles/howto/change-ip-addresses-of-computers-running-tableau-server

               

              Regards,

              Gerardo

              1 of 1 people found this helpful
              • 4. Re: update to primary server IP
                Jeff Strauss

                seriously?  this process is quite significant in terms of time.  And what if v-motion is in effect for the VM?  Then a full uninstall / re-install is required?

                 

                 

                Here's the instructions I followed from the server admin guide:

                 

                • 5. Re: update to primary server IP
                  Gerardo Varela

                  I read those as well.  Maybe the log files have a bit more to offer? Clustercontrol / Zookeeper might have some insights.  Hopefully someone else chimes in with more ideas.

                   

                  Regards,

                  Gerardo

                  1 of 1 people found this helpful
                  • 6. Re: update to primary server IP
                    John Kuo

                    Can you telnet from primary to one of the workers? If that fails, then something's blocking communications to workers.

                     

                    Check firewall and proxy settings. Also, make sure Tableau is on anti-virus or other security software's exclusion list.

                     

                    http://kb.tableau.com/articles/howto/improving-performance-by-using-antivirus-exclusions

                    1 of 1 people found this helpful
                    • 7. Re: update to primary server IP
                      Ramesh Gowrishankar

                      Can you please clarify if your server installation is configured using hostname or IP? If its configured using a hostname then you should be able to use the stop / config start path. If its IP address then you need to do a new install.

                       

                      BTW, we are looking at what we can do to support this better in the post v10 timeframe

                      1 of 1 people found this helpful
                      • 8. Re: update to primary server IP
                        Jeff Strauss

                        thanks for all the input~!  No solution yet...

                         

                        -------------------------------------------------------------

                        ok, I'm back, it took a few days to get a configurable dev VM (for our primary node) where we can flexibly update the IP of the primary server.

                         

                        We ran 3 tests on the VM and they all failed:

                        1. update IP while cluster is active, tabadmin stop.  stop failed because it couldn't reach the worker nodes.

                        2. update IP while cluster is active (and there's a gateway on the primary).  tabadmin stop.  stop failed because it couldn't reach the worker nodes.

                        3. update IP while cluster is stopped (and there's a gateway on the primary).  tabadmin config.  surprisingly the config failed because it couldn't reach the worker nodes.

                         

                        - We are on 9.0.5 if that makes any difference, we plan on upgrading to 9.2.2 soon

                         

                        - Yes, we can telnet from the primary to the workers

                         

                        - The zookeeper and clustercontroller logs don't reveal anything useful from what I can tell

                         

                        - We used hostnames when installing, and setting up the workers via the config

                         

                        - One suspect on prod was that we had a tabadmin failoverprimary "primary IP, secondary IP", on dev we initially did the install with tabadmin failoverprimary "primary IP, secondary IP" but then changed it to tabadmin failoverprimary "primary hostname, secondary hostname" prior to the above test

                        • 9. Re: update to primary server IP
                          Ramesh Gowrishankar

                          I followed up with Jeffrey offline and it looks like the documentation is missing a key step.

                           

                          If the cluster is configured using hostnames, the following steps need to be followed:

                           

                          1. Stop the cluster

                          2. Change the IP address of the primary

                          3. Reboot the worker machines (or just restart the tabadmsvc on each of the workers)  --> this is missing in the documentation

                          4. Run tabadmin config

                          5. Start the cluster

                           

                          Some more details:

                           

                          Every worker has a whitelist that contains the IP of the primary so that it can make sure its responding only to the right primary server and not a rogue server. After changing the IP address of the primary, the tabadmsvc service on the workers needs to restart so that it can map the primary hostname to the new IP and whitelist the new IP of the primary. Once that happens, the tabadmin config piece will work as expected. When that step is missing, it will reject requests from tabadmin because it does not trust the new IP address.

                           

                          I am getting our documentation on this updated and apologize for the inaccuracy in the first place.

                          1 of 1 people found this helpful
                          • 10. Re: update to primary server IP
                            Jeff Strauss

                            thank you Ramesh.  I think this also answers my long outstanding inquiry of what tabadmin failoverprimary in effect does.  I never see it within tabsvc.yml, however it seems logical that it would set the whitelist within the workers.

                            • 11. Re: update to primary server IP
                              Michael Perillo

                              Jeffrey - thanks for posting. Never had my IPs change on me but have experienced very similar behavior other ways. 

                              Ramesh - thanks for posting the extra step. It should also be noted at the end of every KB article to reboot everything if all else fails.