Tableau doesn't support formatting like that.
You'll need to use a calculated field to accomplish this. Something like this:
IF SUM([MyField]) > 1000000
THEN STR(SUM([MyField]/100000)) + ' M'
ELSEIF SUM([MyField]) > 1000
THEN STR(SUM([MyField]/1000)) + ' K'
Here is another thread that may help as well: Custom Number Formatting ($K,$M)
Hope this helps!