If the user has been authenticated to the SAML IdP prior to accessing the page where your viz is embedded they should not be prompted to sign in. However, if the user has not authenticated to the SAML IdP, or has explicitly signed out of Tableau Server, they will be prompted to sign in. You will also see this behavior if you're accessing the embedded viz via an incognito/private browser or from a new browser where you haven't already authenticated to the IdP.
If you are prompted to log in every time you access the page in the same browser session after previously authenticating to Tableau Server, there might be a bigger issue at play here.
Does this help answer your question?