The slowness / timeout may be a result of how the custom SQL is handled by Tableau in preparation for passing onto the database. I think you can verify this by looking at the generated SQL either in the Tableau logs or the database logs. It nests the custom SQL so it looks something like this:
select * from
- (select [bunch of fields]
- from a
- innerjoin b on ..
- innerjoin c on ...
- innerjoin d on ...
- innerjoin e on ...
- innerjoin f on ...
- innerjoin g on ...
- groupby [a bunch of fields]) TableauSQL
Because the filters are on the outside, the database may not have a chance to optimize this and it needs to return the entire result set and then apply the where conditions.
Actually the "where"s are in the same custom SQL. I just left them out.
What would you suggest that I do to make it faster?
Whenever possible try NOT to use custom sql. Use Tableau's join dialogue boxes. If that isn't possible, then create a view from your custom sql so you can query that instead.
ya, I've read some bad things about using custom sql. Does Tableau do groupbys at the data source level other than joining?
Not that I know of. For the most part, you can replicate the group by clauses in Tableau. Here are a couple of threads that might help you break down your custom sql into native joins in Tableau. Of course, these might not work since we don't know your full query or what Tableau is generating.
I hope those help a bit.