1 Reply Latest reply on Dec 13, 2016 11:17 AM by Russell Christopher

    Parallel queries

    learner always

      Like to understand parallel query settings and how they actually work (i have gone through the online doc but not sure if this still applies)

       

      There are various places where parallel query settings can help i.e. vizqlserver <-> tdeserver64 interaction, tdprotosrv <-> db interaction, backgrounder <-> tdeserver64 interaction etc

       

      Here is what i see in the logs:

       

      E:\Program Files\Tableau\Tableau Server\data\tabsvc\config\dataengine>type tdeserver_spawned.yml

      ---

      # This is YAML, but it's very strict.

       

      log-rotate-interval:86400

      queue-length-logging:false

      MemoryMapEntireFile:false

      query-parallelism:2

       

      Notice i have query parallelism set to 2 which i believe is the default.

       

      Then in logs:

       

      tdeserver_vizqlserver_0_2016_12_08_06_17_38.txt

      2016-12-08 06:17:38.426 (16300): -----------------------------------------------------------------------------

      2016-12-08 06:17:38.426 (16300): Initializing fault reporting

      2016-12-08 06:17:38.426 (16300): Fault reporting log pathname: E:\Program Files\Tableau\Tableau Server\data\tabsvc\logs\vizqlserver\tdeserver64-error0

      0006324.log

      2016-12-08 06:17:38.426 (16300): Initializing MiniDump

      2016-12-08 06:17:38.426 (16300): Loading 'DbgHelp.dll'

      .

      .

      2016-12-08 06:17:38.426 (16300): -----------------------------------------------------------------------------

      2016-12-08 06:17:38.426 (16300): argv[0]="E:\Program Files\Tableau\Tableau Server\10.0\bin\tdeserver64.exe"

      2016-12-08 06:17:38.426 (16300): argv[1]="--callback"

      2016-12-08 06:17:38.426 (16300): argv[2]="tab.pipe://./pipe/{54F23E84-B4C2-4ACD-9EF1-5305679C68F4}"

      2016-12-08 06:17:38.426 (16300): argv[3]="--listen"

      2016-12-08 06:17:38.426 (16300): argv[4]="tab.pipe://./pipe/auto"

      2016-12-08 06:17:38.426 (16300): argv[5]="--log"

      2016-12-08 06:17:38.426 (16300): argv[6]="E:\Program Files\Tableau\Tableau Server\data\tabsvc\logs\vizqlserver\tdeserver_vizqlserver_0.txt"

      2016-12-08 06:17:38.426 (16300): argv[7]="--config"

      2016-12-08 06:17:38.426 (16300): argv[8]="E:\Program Files\Tableau\Tableau Server\data\tabsvc\config\dataengine\tdeserver_spawned.yml"

      2016-12-08 06:17:38.426 (16300): pid=25380

      2016-12-08 06:17:38.426 (16300): MemoryMapEntireFile:false

      2016-12-08 06:17:38.426 (16300): log-rotate-interval:86400

       

      2016-12-08 06:17:38.426 (16300): query-parallelism:2

      2016-12-08 06:17:38.426 (16300): queue-length-logging:false

       

      2016-12-08 06:17:38.426 (16300): Adjusted Parallelism Values

      2016-12-08 06:17:38.426 (16300):   CPUs available to process: 8

      2016-12-08 06:17:38.426 (16300):   max-concurrency:           4

      2016-12-08 06:17:38.426 (16300):   query-parallelism:         2

      2016-12-08 06:17:38.426 (16300):   min-query-parallelism:     2

      This tells me that vizqlserver uses the spqwned tdeserver64 file which limits the number of parallel queries

       

      Similarly for the backgrounder it uses the tdeserver_spawned.yml file.

      There is another file tdeserver_standalone.yml, not clear when is it used.

       

      So my question is how do we know if these limits are not enough, do i need to turn on queue_length_logging, if not,  what other parameters/metrics would help to understand bottlenecks and if i need to increase query parallelism in the system.

       

      One node server, 8 core, 48GB memory. 

      Thanks,