1 2 Previous Next 25 Replies Latest reply on Dec 8, 2014 10:56 AM by Joe Salvatore

    Automatic Failover to Backup gateway

    Rosh ni

      In Tableau repository and data engine  and load balancer on

      primary server are the 3 areas needing redundancy.

       

      To address repository and data engine , we can setup repository and data engine on one worker

      and a standby repository and data engine on another worker. Tableau will

      take care of automatic real time replication between active repository

      and data engine and backup repository and data engine, and loadbalancer

      will automatically failover if there is a hardware failure on active, or failure of any process of repository/data engine.

       

      However for failure of load balancer on primary gateway- even if there is a backup gateway setup, configured and 'ready' the actual switch from primary to backup gateway requires manual intervention & some commands to be run. Is there a way to automate this so that a switch over takes place automatically?  Is there any improvement on  high availability from version 7 to version 8?

        • 1. Re: Automatic Failover to Backup gateway
          Uday Pisipati

          Hi, Can anybody answer this ? I am also looking for same infromation

          • 2. Re: Automatic Failover to Backup gateway
            Russell Christopher

            No.

             

            In 8.0X there is no way to make this happen without some sort of script "checking" the gateway's health and potentially running the same steps a "real person" would.

             

            That being said, 8.1 (to be released this year) will allow for multiple active gateways. So, you won't need to worry about this at all in a little while

            • 3. Re: Automatic Failover to Backup gateway
              Uday Pisipati

              Hi Russell,

               

              Any update on 8.1 release ?

              • 4. Re: Automatic Failover to Backup gateway
                Russell Christopher

                It released two weeks ago. Where have you been?

                 

                (joke! joke!...but it did release already)

                1 of 1 people found this helpful
                • 5. Re: Automatic Failover to Backup gateway
                  Uday Pisipati

                  Sorry for the wrong question....My intension was to ask if 8.1  version released with Automativ Failover capability. I was going thru the release notes and i dont see that feature...Appreicate if you could response

                  • 6. Re: Automatic Failover to Backup gateway
                    Russell Christopher

                    Oh, OK. I see what you're saying.

                     

                    By automatic failover, I'm gussing you mean this:

                     

                    • Gateway is running on my "Primary" machine
                    • Another gateway (new in 8.1) is running on my "Worker1" machine
                    • Primary catches on fire. Turns off.
                    • All incoming requests are automatically routed to "Worker1" gateway

                     

                    No. And if you think about it, this makes perfect sense. If the Primary machine has gone away, what process is there to somehow redirect requests going to Primary elsewhere? None. A user hitting http://primary will just get an error because there's no machine there anymore. If you wanted to redirect a request to http://primary, you'd need to do so BEFORE it arrives at that ip address - that's a networking / DNS / load balancer thing.

                     

                    You need to have a process that sends requests to both the gateways on Primary and Worker1 (generally a hardware load balancer). Then if Primary (or Worker1) goes away, that process will take the failed machine out of the machine pool.

                     

                    Make sense?

                    1 of 1 people found this helpful
                    • 7. Re: Automatic Failover to Backup gateway
                      Uday Pisipati

                      Thank You for your detailed response Russel. From your reply i understand that 8.1 has the capability where 2 nodes can act as gateway which was not there is 8.0

                       

                      I understand the load balancer part. Assuming we have a load balanced which can divert traffic to two nodes

                       

                      I have few more questions.

                       

                      1. In 8.0, we installed gateway module(Tableau server exe) in primary and worker module ( Tableau worker exe) in worker nodes. How do we do this in 8.1 ?  Do we have have to install gateway modules in both the nodes ?

                      2. If you could point me to any online help on this installation , it would be great.

                      • 8. Re: Automatic Failover to Backup gateway
                        Ken Patton

                        Hi Uday,

                         

                        To achieve true High Availability, you're most likely going to need more than just two nodes.

                         

                        The primary difference between an 8.0 cluster and an 8.1 cluster, is that:

                         

                        In 8.0, the best you could do for a backup Gateway was a "cold standby" -- basically a host with the Server installed, but not running, and awaiting human intervention to get started up. (Including editing tabsvc.yml, potentially). Now, in 8.1 you can have a true Backup Gateway. This does not change your Worker configuration. You'll still have two hosts with the Server image installed, plus however many Workers you require. The difference is in how the two Server installs are now able to interact with each other. (In 8.0, they effectively could not interact and so it was necessary to keep one of them down unless and until needed to fire up as a replacement.)

                         

                        As for the online help, the first and best place to seek guidance is in the 8.1 Admin Guide, HA section, here:

                         

                        http://onlinehelp.tableausoftware.com/current/server/en-us/help.htm#distrib_ha.htm

                         

                        Hope that helps,

                        Ken

                        Server8-Cert-PNG-Small.png

                        1 of 1 people found this helpful
                        • 9. Re: Automatic Failover to Backup gateway
                          Russell Christopher

                          Hey Ken -

                           

                          Congrats on your certification! Noticed that on Twitter.

                           

                          I totally understand what you mean in your response, but I'm going to quibble with your language a bit (in public, of course) because you happened to touch on something that can confuse some folks who grew up with "old" 8.0 HA.

                           

                          It's an important distinction.

                           

                          In 8.0, "Gateway" and "Primary" were synonymous. Gateway = Primary and vice versa because the single gateway always lived in the Primary. This is no longer the case in 8.1

                           

                          I now can have as many gateways as I want, so being a gateway does not define a box as a "Primary" one way or the other. I could actually run my Primary without a gateway at all.  (So what NOW defines a box as our "Primary", someone may wonder? Our licensing components do.)

                           

                          What stays the same is there will still only be a single primary active at any time. You'll note that in 8.0 we talk about creating a "backup Gateway". We don't do that anymore - in 8.1 it's a "backup Primary". There's no such thing as a backup gateway anymore since multiple gateways are all active all the time.

                           

                          You can now have multiple gateways up and running at the same time, but you still can only have a single Primary up and running (and to me, Primary defines Server-ness)

                           

                          Again, not nitpicking your response - only jumping on it because this is the first time someone has raised the subject and changing the way folks define "Primary" and "Gateway" in their minds is necessary/important

                           

                          Have a great Sunday!

                          1 of 1 people found this helpful
                          • 10. Re: Automatic Failover to Backup gateway
                            Ken Patton

                            That's an excellent and important correction, Russell. Thanks for pointing that out. One reason I was still thinking in terms of "primary" is that most External Load Balancers still have an initial / default host that is picked first (even in "round robin" balancing, it has to start somewhere) so it would make sense for the ELB to point to the host that also happens to hold the license.

                             

                            It is nice to have the Gateway function be able to be spread out over several Server nodes, for those seeking maximal robustness and uptime.

                            • 11. Re: Automatic Failover to Backup gateway
                              David Thompson

                              Just reading this as I start preparing for 8.1 with as high availability as possible and it's a very useful discussion.

                               

                              One question - the move from primary to backup primary is currently a manual process. Depending on the details of how we set the hardware load balancing, we may have to include some changes there too to reflect the change in nodes as part of our operational procedures.

                               

                              Is there anything to stop us excluding the primary in the list of nodes in the hardware VIP in both normal operation and failover, meaning the gateway function in both the primary and backup primary are never used, only in the workers? This takes load off the primary and simplifies management.

                              • 12. Re: Automatic Failover to Backup gateway
                                chris.mccord

                                Ken Patton wrote:

                                 

                                Hi Uday,

                                 

                                To achieve true High Availability, you're most likely going to need more than just two nodes.

                                 

                                The primary difference between an 8.0 cluster and an 8.1 cluster, is that:

                                 

                                In 8.0, the best you could do for a backup Gateway was a "cold standby" -- basically a host with the Server installed, but not running, and awaiting human intervention to get started up. (Including editing tabsvc.yml, potentially). Now, in 8.1 you can have a true Backup Gateway. This does not change your Worker configuration. You'll still have two hosts with the Server image installed, plus however many Workers you require. The difference is in how the two Server installs are now able to interact with each other. (In 8.0, they effectively could not interact and so it was necessary to keep one of them down unless and until needed to fire up as a replacement.)

                                 

                                As for the online help, the first and best place to seek guidance is in the 8.1 Admin Guide, HA section, here:

                                 

                                http://onlinehelp.tableausoftware.com/current/server/en-us/help.htm#distrib_ha.htm

                                 

                                Hope that helps,

                                Ken

                                Server8-Cert-PNG-Small.png

                                 

                                So in my configuration we have 4 VM Servers, one is the Primary which has the Tableau Server software installed, it only has the Gateway services running on it. Then the 2nd server is our background worker, only has 4 background services running on it. Then the last two, workers 3 & 4 have all the services, repositories and data engine etc running. I recently upgraded our DEV instance to 8.1, and was looking forward to getting to add extra gateway services to these nodes. My initial thought was to just add a 2nd gateway service to our backgrounder node, and leave workers 3 & 4 alone since they were designed to do the bulk of the work in our HA cluster. Well, I can't get the 2nd gateway service to come up. I think it was green the very first time server started up after the upgrade, but since then, after a few stop/stars of the server, its not coming back green and I don't know why. Is there something to look for in the logs as to why this 2nd gateway won't come up now? If I'm reading this thread right, it doesn't have to be on a VM with the Tableau Server software installed, it can run on a Worker right?

                                 

                                 

                                Screen Shot 2014-02-11 at 10.16.20 AM.png

                                 

                                Screen Shot 2014-02-11 at 10.53.28 AM.png

                                • 13. Re: Automatic Failover to Backup gateway
                                  Ken Patton

                                  I believe that the backup Gateway should have a Server image not a Worker image.

                                   

                                  Also -- do you have an external load balancer up and running, and have you configured the Servers to it?

                                  • 14. Re: Automatic Failover to Backup gateway
                                    chris.mccord

                                    Ken Patton wrote:

                                     

                                    I believe that the backup Gateway should have a Server image not a Worker image.

                                     

                                    Also -- do you have an external load balancer up and running, and have you configured the Servers to it?

                                    Ken,

                                     

                                    Thats what I gathered from your post, but when I read the Tableau documentation it doesn't seem that Server image has to be used, they show the 2nd gateway being installed on a worker, as if its just adding that process to a worker node.

                                    This is where I'm reading that info:

                                    http://onlinehelp.tableausoftware.com/current/server/en-us/help.htm#distrib_ha_fail.htm%3FTocPath%3DAdministrator%20Guide%7CDistributed%20Environments%7CHigh%20Availability%7CConfigure%20for%20Failover%20and%20Multiple%20Gateways%7C_____0

                                     

                                    Specifically step #7

                                    Enter the IPv4 address or computer name of the worker, enter 1 for Data Engine and select the Repository check box. For now, leave the Gateway check box cleared. You will add a gateway to this worker later.

                                    So either I'm missing something in that documentation or they have some mistakes?

                                     

                                    I don't have an external load balancer up and running, nope. I guess if we had two VMs with server images on them, we could configure our F5 load balancer to use them, but does the 8.1 Server handle two real Server instances? I mean installing Server twice on two servers, then using the config tool on both servers to point to the other 3 workers? Is that how I would do a 5 server install? Two gateways, with just gateway services, using F5 to load balance them, then using the config tool on both to talk to the other 3 servers in my HA cluster? Then having to stop/start both of those gateway servers all the time? I don't see how having two Server images works?

                                     

                                    And I need to be careful and clear when you use the word external, and Tableau as well, external failover etc, as this app is an internal application to Seagate, nothing would be external to our network for this instance. So lets avoid the word external so I don't get confused, unless you truly mean external to the network where our application is installed.

                                     

                                    I never installed the older 8.0.x version of a "backup" or "failover" gateway because I didn't like the manual nature of getting it up and running. So now I'm just trying to figure out the process and what I need to do to get a 2nd gateway service going.

                                     

                                    That link and the Tableau documentation makes it appear its as easy as adding the service to another node in the HA configuration, but I've yet to get one to come up clean. It doesn't mention having two Server instances going, it only speaks of one, and then configuring a worker to have the 2nd gateway, and actually shows 3 or 4 servers with the gateway service on it.

                                     

                                    Chris

                                    1 2 Previous Next