Sorry to say this, but as far as I know the incremental refresh was always supposed to work like that - only new records, as determined by the X value from your chosen field, would be added.
It doesn't attempt to scan for changes to already loaded records and update them.
But full refresh just overwrites all old records with same old (or altered), plus new records.
As far as I know, nothing changed in v 8.
The Tableau Help states that incremental refresh is supposed "... to only add the rows that are new since the last time you extracted data." (emphasis mine). In other words, it adds new rows, not altered rows.
I guess Tableau could have made it more clear and added a clarification or a note.
I can see why they implemented the incremental refresh that way. But it leaves room for a third type of refresh that checks the Last Modified date on a record (if it has one) and updates that record as well as adding new ones.
An example use case is a table in Salesforce or some other cloud-based system that has a million records with some new additions and a small number of changes each day. You can't do a full refresh very often because of API call limits, but an incremental refresh would leave a number of records outdated. This could make the incremental refresh effectively useless since the data isn't all current. A refresh done like the third type above would solve the issue.
Do you know if there's an Idea for this yet? If not I'll add one.
Sean - I have not heard of that third option being offered but that is exactly what I would have expected from an incremental refresh.
Yes - please do add that Idea if you can :-)
We need the same 3rd solution