Uploaded image for project: 'GeoTools'
  1. GeoTools
  2. GEOT-5524

Incorrect detection of GRIB files with WMO header

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Medium
    • Resolution: Fixed
    • Affects Version/s: 15.1, 16-beta
    • Fix Version/s: 15.2, 16-RC1
    • Component/s: netcdf
    • Labels:
      None

      Description

      NetCDFImageReaderSpi incorrectly assumes that GRIB files start with the bytes "GRIB". However, some GRIB files start with a World Meteorological Organization (WMO) header added by, for example, the US National Weather Service Telecommunications Gateway.

      For example, the existing test data weird.model in weird.zip stars with a WMO header for "KKCI", the "International Civil Aviation Organization (ICAO)-approved identifier of the office originating the product". The Center Identifiers List identifies this office this as "KKCI - Aviation Weather Center, Kansas City, MO".

      NetCDFImageReaderSpi uses an undocumented system property NETCDF_FORCE_OPEN_CHECK to overcome this limitation by forcing a full NetcdfDataset.acquireDataset on all files, but this is problematic because the search is expensive and requires users to enable this behaviour; users may not know about this property, and may not know that their data file needs it. Furthermore, this property is applied to all (for example) GeoServer instances in the same JVM.

      The proposed solution is to instead emulate the behaviour of NetCDF-Java, which searches the first 16000 bytes for "GRIB". See:
      https://github.com/Unidata/thredds/blob/master/grib/src/main/java/ucar/nc2/grib/grib1/Grib1RecordScanner.java#L81
      https://github.com/Unidata/thredds/blob/master/grib/src/main/java/ucar/nc2/grib/grib2/Grib2RecordScanner.java#L29

      The file in question is already being opened and the additional work is tiny compared to the overheads. The search adds no new dependencies and makes the logic consistent with NetCDF-Java.

      This fix also removes all use of NETCDF_FORCE_OPEN_CHECK. The existing test and test data are preserved.

        Attachments

          Issue links

            Activity

              People

              • Assignee:
                bencaradocdavies Ben Caradoc-Davies [Administrator]
                Reporter:
                bencaradocdavies Ben Caradoc-Davies [Administrator]
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: