That has not occurred for me in my testing with 10.5.3 Tableau Server and Tableau Desktop versions. Can you provide a little more detail on the versions that are being used? My data source folders are maintained as expected.
Another troubleshooting step I would be curious to see is to have the data source donwloaded from the sever as a .tdsx and opened in Tableau Desktop does it exhibit the same behavior?
Hope this helps!