While I'm not positive why the REST API needs system admin privilege, have you tried using tabcmd to manage groups in Tableau Server?
You should add this to the Ideas section. It has been a pain point for me as well.
You can get an xml version of the users in a group by adding "&format=xml" to the end of the group's url (IE: http://myserver/t/mySite/users?fe_group=myGroup&format=xml ), but this requires the user to be logged in first.
In our setup, we use selenium to login the user and pull this file down. Hardly ideal, but it works.