probably way too late to actually answer the question, but for people doing searches - check out this article:
Walks you right through it. In short, you use the OPENQUERY() command (if pulling from SQL Server). Other similar options for other technologies.
There is also the online help which covers some different material then kb article Justin mentioned.
This is all new to Tableau 8.1, before this procedures were not supported.
FYI - There doesn't seem to be support for Custom SQL (I don't think you need it). You instead define the parameters when you select the "Procedures" button -- you can even use Tableau Parameters to change the sproc parameters in a live workbook.
If you are using Tableau 8.1, the easiest would be to use the stored procedure option that is provided for SQL Server, Sybase ASE and Teradata in the connection dialog. If you are not using 8.1 or have a different database you will need to use a mechanism like OPENQUERY as described in the link Justin provided above.