1 2 Previous Next 15 Replies Latest reply on Jun 15, 2015 6:26 AM by Toby Erkson Branched from an earlier discussion.

    How to schedule an extract every 2 minutes?

    Sumit Singla

      Hi All,

      Do we have any other way of scheduling the extract every 2 minutes, because in tableau be default we can refresh the extract every 15 mins minimum.

        • 1. Re: How to schedule an extract every 2 minutes?
          Toby Erkson

          Branched this question into it's own.  It's best to stick to one topic question per thread.  Adding unrelated topic questions makes it difficult to follow the given replies and can lead to a mis-labeled Answered post.

          • 2. Re: How to schedule an extract every 2 minutes?
            Matt Coles

            It is possible, but requires updating the Tableau Server repository database directly with a SQL query (generally not recommended). You would need to :

             

            1: Create a new schedule that runs every 15 minutes

            2. Find the password for either the tblwgadmin or rails PostgreSQL user in your workgroup.yml file.

            3. Connect to the Primary repository database using the tblwgadmin or rails user (reommend PGAdmin for this)

            4. Run a query to determine which schedule you need to update:

             

            SELECT * FROM schedules ORDER BY name
            

             

            5. Run the following query:

             

            UPDATE schedules SET minute_interval = 2 WHERE id = _____
            

             

            ...where the id value is the id of the Schedule you created in Step 1.

             

            Again, directly editing the PostgreSQL database is not recommended, and is easy to screw up. If you choose to do it, take a backup beforehand, and run through the steps during a maintenance window so you can restore if necessary. I also don't take responsibility for those SQL queries being correct, so review them carefully!

            • 3. Re: How to schedule an extract every 2 minutes?
              Shawn Wallwork

              First you should find out how long your extract is going to take. Is it longer than 2 minutes? I mention this because we had 30-40 background refreshes set to every 15 minutes but since they average 40-45 seconds to refresh, we were really running on a 30+ cycle.

               

              The other way (than what Matt just posted) is to use tabcmd commands in a batch file with a looping logic set to 2 min.

               

              Here's the refresh command: tabcmd Commands

               

              --Shawn

              • 4. Re: How to schedule an extract every 2 minutes?
                Jeff Strauss

                I recommend to use tabcmd refreshextract, but be careful on filling up of the backgrounder queues based on demand and excessively refreshing this one.

                • 5. Re: How to schedule an extract every 2 minutes?
                  Sumit Singla

                  Hi Jeffrey,

                   

                  I think we had discussion that am not able to execute the command refreshextract  and getting the Not Found Error.

                   

                  Below is what am doing:

                   

                  I have a dashboard in which basically 4 datasource connection are there and am publishing the dashboard to the server .

                   

                  And then am using the refreshextract command with workbook option and giving name same which is showing on server for this dashboard.

                   

                  Please let me know if am missing something.

                   

                   

                  Thanks & Regards,

                  Sumit Singla

                  IT Analyst | Enterprise Operations Management

                  3rd Floor | F - 3 Tower | Landmark Plaza Building | Plot No. A - 40A |

                  QuarkCity India Pvt. Ltd. SEZ | Phase VIII - B | Industrial Area |

                  Mohali (S.A.S. Nagar) | Punjab

                  Office: 01724267573    Mobile: 8699662082

                  Email: sumit.singla@emerson.com<mailto:sumit.singla@emerson.com>

                  • 6. Re: How to schedule an extract every 2 minutes?
                    Jeff Strauss

                    It's probably something obvious, and we've provided many suggestions.  Have you tried opening a case with support?

                    • 7. Re: How to schedule an extract every 2 minutes?
                      Sumit Singla

                      Hi Jeffrey,

                       

                      Am just going to contact them because that is last option I have.

                      Anyways thanks for all your time and support. I will let you know if I can get this done.

                       

                       

                      Thanks & Regards,

                      Sumit Singla

                      IT Analyst | Enterprise Operations Management

                      3rd Floor | F - 3 Tower | Landmark Plaza Building | Plot No. A - 40A |

                      QuarkCity India Pvt. Ltd. SEZ | Phase VIII - B | Industrial Area |

                      Mohali (S.A.S. Nagar) | Punjab

                      Office: 01724267573    Mobile: 8699662082

                      Email: sumit.singla@emerson.com<mailto:sumit.singla@emerson.com>

                      • 8. Re: How to schedule an extract every 2 minutes?
                        Dan Scott

                        Others have already noted that refreshing on such a short schedule can be problematic.  If the refresh actually takes longer than 2 minutes to happen, that's clearly a problem.  But there can be other considerations, as well.

                         

                        If your Tableau Server is set up for "High Availability," having multiple Data Engine machines, then whenever a refresh occurs, the newly created extract file will need to be replicated across all machines running Data Engine. Pre-9.0, the delay before replication was often large enough that extracts on such a short refresh cycle, would arrive at the secondary Data Engine outdated, and useless, the moment they arrived (meaning the secondary Data Engine would be useless for working with that datasource).  This is far less likely for 9.0+, but all those refreshes will still impose some network traffic overhead.

                         

                        Even if neither of these is much of a problem in this case, you may want to consider if the refresh is likely to get bigger and start taking more time, and eventually need more than 2 minutes.

                         

                        Another thought: If you are going to be hitting your original data source (some database, probably?) every two minutes to perform the data refresh, would it, perhaps, make sense to define a live connection and use that, instead of an extract?  This will likely depend on how many views are rendered that use the data.  If it's not too many, then your original data source might do less work with a live connection.  And, of course, it would always be completely up to date.

                         

                        None of this means that you should never refresh on such a tight cycle.  It simply means you should consider the effects of the decision and alternatives, and balance those against your business needs.

                        1 of 1 people found this helpful
                        • 9. Re: How to schedule an extract every 2 minutes?
                          Matt Coles

                          Great points, Dan!

                          • 10. Re: How to schedule an extract every 2 minutes?
                            Shawn Wallwork

                            Dan thoughts on serial vs. parallel refresh schedules pre & post 9.0?

                             

                            Thanks,

                             

                            --Shawn

                            • 11. Re: How to schedule an extract every 2 minutes?
                              Dan Scott

                              I'm not completely sure I understand what you are asking, but I am going to assume you are talking about refreshing several different extracts at the same time, vs. one after the other.  Assuming that is correct...

                               

                              There may be greater potential to handle parallel refreshes in 9.0+, but whether that potential is realized will depend on your specific installation.  This is somewhat speculative, but...  when Backgrounder performs an extract refresh, it spawns a local Data Engine process specifically for the purpose.  That spawned process needs to read the old extract file from a File Store, and ultimately, write the new "refreshed" extract file back to a File Store.  Since 9.0+ can have more than two machines with File Store (and Data Engine) this means the "read from File Store" part of the refresh process can be spread over more machines. But, because all extracts end up in all File Stores, the "write back to File Store" part still ultimately happens on all File Store machines.

                               

                              It seems likely to me that a 9.0+ setup with enough File Store / Data Engine machines might handle this a bit better than pre-9.0.  But, I have not heard of any testing on this question.  Sorry, I can't be more definite.

                               

                              What I am confident of, is that the new refreshed extract file will be replicated faster than it would have in pre-9.0, and be available for use by all Data Engines.

                              1 of 1 people found this helpful
                              • 12. Re: How to schedule an extract every 2 minutes?
                                Shawn Wallwork

                                Dan thanks for the detailed response. And yes you guessed right, that was my question. (Sorry I was so terse.) I noticed that it seems that in that screenshot someone posted (and now deleted) they were using parallel refreshes. My client had set up almost all of their refreshes to be serial, with the belief that this would be light on the server, and therefore give the folks browsing to views a better experience with shorter load times. The logic seems sound to me, but I'm new to the Server side of Tableau.

                                 

                                I was able to get their cycle time down to 16-17 minutes (by deleting a lot of unused/abandoned workbook that were still refreshing every 15). I just wasn't sure if I should suggest that they shift to parallel refreshes to see if this would improve performance, without taking a hit on the browser/view performance. They don't much like experimenting, especially if it has even the slightest chance of impacting their C-level folks browsing experience.

                                 

                                Anyway thanks for the insight.

                                 

                                Cheers,

                                 

                                --Shawn

                                • 13. Re: How to schedule an extract every 2 minutes?
                                  Dan Scott

                                  As a general thing, spreading out the overhead of refreshes is a good thing. Sudden spikes in refresh activity can compromise the user experience.  Oh, and for refreshes that happen daily, all other things being equal, I would be inclined to schedule those for "off hours" when the Server is not being used heavily.  And I believe this is good advice regardless of which version of Tableau you are using.

                                  1 of 1 people found this helpful
                                  • 14. Re: How to schedule an extract every 2 minutes?
                                    Shawn Wallwork

                                    Dan thanks for confirming. The thing I like about the serial refresh is it's easy to see the cycle in the bar chart:

                                    Schedule - 1.png

                                    Schedule - 2.png

                                     

                                    Schedule - 3.png

                                     

                                    We're still running 21 seconds behind, but that beats a half hour.

                                     

                                    --Shawn

                                    1 2 Previous Next