There are a few things I can suggest, you may have tried all or some of these...
You mention that this is only on certain workbooks but that others are not having the same issue so this leads me to suspect this may not be an environmental issue. Are there are similarities between those that are slow that are not shared by the more performant workbooks? Is this a specific tool-tip setting, maybe the same data source or perhaps busy tooltips?
Have you tried to delete and re-publish the workbooks in question to see if this makes a difference?
Within the tool-tip setting you can modify the responsiveness, have you tested changing these settings?
If this post assists in resolving the question, please mark it helpful or as the 'correct answer' if it resolves the question. This will help other users find the same answer/resolution. Thank you.