Normally processes such as PagedUnique and Aggregate run as optimized sql statements, but applying security restrictions on the layers makes them fall back on data loading and in memory aggregatation insetad.
This is due to DecoratingFeatureCollection not delegating down the visit method (which is the right decision) and subclasses not taking measures to check if the visitor can be passed down, or not.
https://github.com/geoserver/geoserver/blob/master/src/main/src/main/java/org/geoserver/security/decorators/SecuredSimpleFeatureCollection.java is one of those, but retyping (used in renames) and reprojecting/forcecrs collections are also causing the problem.
Same underlying problem showing up in a different way in
Mass closing all resolved issues not modified in the last 4 weeks