ShapefileDataStore cannot properly read sparse shapefiles

Description

When opening ESRI shapefiles with the ShapefileDataStore (see code below), some shapefiles produce an java.io.IOException ("Dbf has extra record").
The affected shapefiles itself are ok. They can be opened by ArcInfo, ArcExplorer or e.g. shapelib without problems. A sample shapefile can be found at http://132.187.42.13/~pschirus/testShapefile.zip - http://132.187.42.13/~pschirus/testShapefile.zip (16kB).

The stacktrace is:
java.io.IOException: Dbf has extra record
at org.geotools.data.shapefile.ShapefileDataStore$Reader.hasNext(ShapefileDataStore.java:417)
at org.geotools.data.FIDFeatureReader.hasNext(FIDFeatureReader.java:133)
at org.geotools.data.DefaultFeatureResults.collection(DefaultFeatureResults.java:227)

Testcase to reproduce the error:
URL url = new URL( "file:///apron.shp" );

// Create new datastore of the provided shapefile
ShapefileDataStore store = new ShapefileDataStore( url );

// Get filename, e.g. "runway"
String storename = store.getTypeNames()[ 0 ];
// Create feature source
FeatureSource source = store.getFeatureSource( storename );

// Read attribute data from the feature source.
FeatureResults fResults = source.getFeatures();

// Get the FeatureType. The FeatureType contains the attribute
// definitions of the shapefile.
featureType = source.getSchema();

// Create the FeatureCollection with all data from the FeatureResults.
featureCollection = fResults.collection();

System.out.println( "Completed successfully");

Environment

None

Activity

Show:
codehaus
April 10, 2015, 3:13 PM

CodeHaus Comment From: pschirus - Time: Tue, 31 Mar 2009 16:55:44 -0500
---------------------
The patch is likely against the 2.1.0M0, but I'm not 100% sure as it is already some time ago that I made the changes. The issue should be resolved in the current Geotools releases with the indexed shapefile reader.

codehaus
April 10, 2015, 3:13 PM

CodeHaus Comment From: aaime - Time: Wed, 1 Apr 2009 05:45:46 -0500
---------------------
Unfortunately it's not, the bug is still definitely there in 2.5.x

codehaus
April 10, 2015, 3:13 PM

CodeHaus Comment From: aaime - Time: Thu, 2 Apr 2009 05:36:11 -0500
---------------------
Jesse, I'm attaching a patch that fixes this issue, I can successfully read (and rewrite) sparse shapefiles with it. I made it as less intrusive as possible, and made sure the shapefile module still builds. I also cheched it does not hinder performance, it does not (not in a significant way, that is). Ok to commit?

codehaus
April 10, 2015, 3:13 PM

CodeHaus Comment From: aaime - Time: Tue, 7 Apr 2009 04:03:39 -0500
---------------------
Fixed on 2.5.x and trunk

codehaus
April 10, 2015, 3:13 PM

CodeHaus Comment From: htagle - Time: Tue, 21 Apr 2009 08:21:35 -0500
---------------------
awesome!!.

btw, Andrea, we have FME and I had already setup the automated process that you suggested

thanks all

Assignee

Unassigned

Reporter

codehaus

Triage

None

Components

Fix versions

Affects versions

Priority

High
Configure