Slow WFS json response

Description

Geoserver WFS json response after version 2.2.5 are slower.

After checking the logging and my doubt is, this files seems create slower response. src/wfs/src/main/java/org/geoserver/wfs/json/GeoJSONGetFeatureResponse.java

Here is the request url which goes to geoserver 2.2.5 and 2.8.0 (I make sure they are exactly the same)

{{{quote}http://localhost:8080/geoserver/ows?service=WFS&version=1.1.0&request=GetFeature&typeName=WRK:table_name&outputFormat=application/json

with request payload:

<wfs:GetFeature xmlns:wfs="http://www.opengis.net/wfs" service="WFS" version="1.1.0" xsi:schemaLocation="http://www.opengis.net/wfs http://schemas.opengis.net/wfs/1.1.0/wfs.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><wfs:Query srsName="EPSG:4326" typeName="WRK:table_name"><ogc:Filter xmlns:ogc="http://www.opengis.net/ogc"><ogc:Or> <ogcropertyIsEqualTo><ogcropertyName>id</ogcropertyName><ogc:Literal>0034100000032614</ogc:Literal></ogcropertyIsEqualTo> <ogcropertyIsEqualTo><ogcropertyName>id</ogcropertyName><ogc:Literal>0034100000032897</ogc:Literal></ogcropertyIsEqualTo> <ogcropertyIsEqualTo><ogcropertyName>id</ogcropertyName><ogc:Literal>0392100000036281</ogc:Literal></ogcropertyIsEqualTo>
</ogc:Or></ogc:Filter></wfs:Query></wfs:GetFeature>

 

}}

When I receive response, version 2.2.5 is responding within a second or 2 but all version after that are taking longer and with a big filter list (>1000 or clause) the response time is going upto 20 seconds but stays consistent with 2.2.5.

In the logging, the difference is visible:

from 2.2.5, after message 'about to encode json', I immediately see (in 2 seconds), compressing but in new versions (all greater 2.2.5), it takes 4-5 seconds before compressing message shown.

INFO | jvm 1 | 2015/10/12 11:14:16 | 12 okt 11:14:16 INFO [org.geoserver.wfs] -
INFO | jvm 1 | 2015/10/12 11:14:16 | Request: getServiceInfo
INFO | jvm 1 | 2015/10/12 11:14:16 | 12 okt 11:14:16 DEBUG
.....some more message with query formation out of OGC filter....
[org.vfny.geoserver.requests] - Query is org.geoserver.wfs.request.Query$WFS11@32fae7b6
INFO | jvm 1 | 2015/10/12 11:14:18 | 12 okt 11:14:18 INFO [org.geoserver.wfs.json] - about to encode JSON
INFO | jvm 1 | 2015/10/12 11:14:18 | 12 okt 11:14:18 DEBUG [org.geoserver.filters] - Compressing output for mimetype: application/json

The database used is postgreSql 9.1 Query response in database is always less than a seconds, so database slowness is out of question.

I have created instances of all releases of geoserver after 2.2.5 to nail down the cause of problem.

Also with newer versions (> 2.2.5) the query made out of filters is fired 2 times, one to get feature count and one for getting actual result set. This is already making response slower.

To rule out OS, Database, infrastructure, I tried this setup on windows server 2012 and windows server 2008. With same version of PostgreSql. Same number of tables. Geoserver is configured in production mode. JVM is also tuned for optimal performance.

Please let me, I am not clear or more information is needed. I had to downgrade to 2.2.5 in production to avoid the timeout issue on client.

Thanks

Environment

None

Activity

Show:
SSA
March 28, 2018, 8:57 AM

This was the commit which caused it. https://github.com/geoserver/geoserver/commit/d89885e9408c097120cb6df0e910fdf315f9a229#diff-1b4d0a8e0a53aad5084a306768223d80

This commit has caused an issue with Postgres database because fetching totalFeatures making the request very slow. Postgres fetches count going through all the rows in the table and index helps with vacuum analyze but still, it's not optimal enough and randomly slows down count query. This should have been a format_options (or something else) if you want totalfeatures or not? In my case the postgres was taking 1 minute to get count of rows (table with 8 million rows with unique index). After vacuum analyze it took ~4 seconds but still without this totalFeatures the geoserver WFS request takes 0.25 milliseconds.

Totalfeatures is not a standard geojson attribute. Nobody looked into this request in last 3 years. I am still struggling and holding back using geoserver 2.2.5 version.

Assignee

Unassigned

Reporter

SSA

Triage

None

Fix versions

None

Affects versions

Components

Priority

Medium
Configure