This seems to happen only under certain circumstances, saving an edited layer group in the GeoServer UI will provoke an illegal argument exception with this stack trace (full stack trace attached):
This issue can be reproduced using tasmania layer group and tasmania_roads layer, both are shipped with GeoServer, the steps to reproduce the issue are:
Go to tasmania layer group edition page and edit the draw order of tasmania_roads layer and save it.
Go to tasmania_roads layer edition page and change the default style and save it.
Go to tasmania layer group edition page and change the style of tasmania_roads layer and try to save.
Associated mailing list discussion: http://osgeo-org.1560.x6.nabble.com/Need-some-help-tracking-down-a-bug-in-RC1-td5312017.html