CoverageViewTest intermittent failures

Description

tl;dr: boot Linux with maxcpus=2 to reproduce this failure in about one in four Maven builds.

In hundreds of scripted builds, many other failures were also seen, all related to EPSG code lookup. The core finding of this investigation is an NPE at org.hsqldb.Database.connect(Database.java:384). I suspect a synchronisation problem with EPSG database access.

The super-long version:

  • Attached patch for gt-epsg-hsql changes ImageMosaicReader.getReader to print a stack trace rather than swallowing exceptions.

  • Attached hsqldb-2.3.0.jar has been rebuilt with debugging information (needed Oracle JDK 7 to build). This gives us line numbers in stack traces and allows breakpoints in source.

  • Linux booted with maxcpus=2.

  • Building gs-main with mvn -o clean install.

After several builds the following is seen:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 Running org.geoserver.catalog.CoverageViewTest source = /home/ben/geoserver/src with spaces/geoserver/src/main/./target/default5111977893338354336data/s2reduced hints = Hints: REPOSITORY = org.geoserver.catalog.CatalogRepository@72435def EXECUTOR_SERVICE = java.util.concurrent.ThreadPoolExecutor@bd09a26[Running, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0] System defaults: GRID_COVERAGE_FACTORY = GridCoverageFactory TILE_ENCODING = null FORCE_LONGITUDE_FIRST_AXIS_ORDER = true COMPARISON_TOLERANCE = 1.0E-7 FILTER_FACTORY = FilterFactoryImpl FORCE_AXIS_ORDER_HONORING = http STYLE_FACTORY = StyleFactoryImpl LENIENT_DATUM_SHIFT = true FEATURE_FACTORY = org.geotools.feature.LenientFeatureFactoryImpl@3403e2ac org.geotools.data.DataSourceException at org.geotools.gce.imagemosaic.ImageMosaicReader.<init>(ImageMosaicReader.java:513) at org.geotools.gce.imagemosaic.ImageMosaicFormat.getReader(ImageMosaicFormat.java:566) at org.geotools.gce.imagemosaic.ImageMosaicFormat.getReader(ImageMosaicFormat.java:108) at org.geoserver.catalog.ResourcePool.getGridCoverageReader(ResourcePool.java:1524) at org.geoserver.catalog.ResourcePool.getGridCoverageReader(ResourcePool.java:1444) at org.geoserver.catalog.CoverageView.buildCoverageInfo(CoverageView.java:387) at org.geoserver.catalog.CoverageView.createCoverageInfo(CoverageView.java:398) at org.geoserver.catalog.CoverageViewTest.buildHeterogeneousResolutionView(CoverageViewTest.java:698) at org.geoserver.catalog.CoverageViewTest.testHeterogeneousViewIntersectionEnvelope(CoverageViewTest.java:458) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) Caused by: org.geotools.data.DataSourceException at org.geotools.gce.imagemosaic.ImageMosaicReader.initReaderFromURL(ImageMosaicReader.java:721) at org.geotools.gce.imagemosaic.ImageMosaicReader.<init>(ImageMosaicReader.java:511) ... 42 more Caused by: java.lang.NullPointerException at org.hsqldb.Database.connect(Database.java:384) at org.hsqldb.DatabaseManager.newSession(DatabaseManager.java:163) at org.hsqldb.jdbc.JDBCConnection.<init>(JDBCConnection.java:3593) at org.hsqldb.jdbc.JDBCDriver.getConnection(JDBCDriver.java:312) at org.hsqldb.jdbc.JDBCDataSource.getConnection(JDBCDataSource.java:192) at org.hsqldb.jdbc.JDBCDataSource.getConnection(JDBCDataSource.java:151) at org.geotools.referencing.factory.epsg.DirectEpsgFactory.getConnection(DirectEpsgFactory.java:3543) at org.geotools.referencing.factory.epsg.DirectEpsgFactory.prepareStatement(DirectEpsgFactory.java:807) at org.geotools.referencing.factory.epsg.DirectEpsgFactory.createCoordinateReferenceSystem(DirectEpsgFactory.java:2067) at org.geotools.referencing.factory.BufferedAuthorityFactory.createCoordinateReferenceSystem(BufferedAuthorityFactory.java:732) at org.geotools.referencing.factory.AuthorityFactoryAdapter.createCoordinateReferenceSystem(AuthorityFactoryAdapter.java:780) at org.geotools.referencing.factory.BufferedAuthorityFactory.createCoordinateReferenceSystem(BufferedAuthorityFactory.java:732) at org.geotools.referencing.factory.AuthorityFactoryAdapter.createCoordinateReferenceSystem(AuthorityFactoryAdapter.java:780) at org.geotools.referencing.factory.FallbackAuthorityFactory.createCoordinateReferenceSystem(FallbackAuthorityFactory.java:625) at org.geotools.referencing.factory.FallbackAuthorityFactory.createCoordinateReferenceSystem(FallbackAuthorityFactory.java:629) at org.geotools.referencing.factory.AuthorityFactoryAdapter.createCoordinateReferenceSystem(AuthorityFactoryAdapter.java:780) at org.geotools.referencing.factory.ThreadedAuthorityFactory.createCoordinateReferenceSystem(ThreadedAuthorityFactory.java:636) at org.geotools.referencing.DefaultAuthorityFactory.createCoordinateReferenceSystem(DefaultAuthorityFactory.java:177) at org.geotools.referencing.CRS.decode(CRS.java:517) at org.geotools.gce.imagemosaic.Utils.decodeSrs(Utils.java:930) at org.geotools.gce.imagemosaic.Utils.loadMosaicProperties(Utils.java:873) at org.geotools.gce.imagemosaic.Utils.loadMosaicProperties(Utils.java:560) at org.geotools.gce.imagemosaic.ImageMosaicReader.initReaderFromURL(ImageMosaicReader.java:620) ... 43 more Tests run: 15, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 4.276 sec <<< FAILURE! testHeterogeneousViewIntersectionEnvelope(org.geoserver.catalog.CoverageViewTest) Time elapsed: 51 sec <<< ERROR! java.io.IOException: Failed to create reader from file:/home/ben/geoserver/src%20with%20spaces/geoserver/src/main/./target/default5111977893338354336data/s2reduced/ and hints Hints: REPOSITORY = org.geoserver.catalog.CatalogRepository@72435def EXECUTOR_SERVICE = java.util.concurrent.ThreadPoolExecutor@bd09a26[Running, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0] System defaults: GRID_COVERAGE_FACTORY = GridCoverageFactory TILE_ENCODING = null FORCE_LONGITUDE_FIRST_AXIS_ORDER = true COMPARISON_TOLERANCE = 1.0E-7 FILTER_FACTORY = FilterFactoryImpl FORCE_AXIS_ORDER_HONORING = http STYLE_FACTORY = StyleFactoryImpl LENIENT_DATUM_SHIFT = true FEATURE_FACTORY = org.geotools.feature.LenientFeatureFactoryImpl@3403e2ac at org.geoserver.catalog.ResourcePool.getGridCoverageReader(ResourcePool.java:1526) at org.geoserver.catalog.ResourcePool.getGridCoverageReader(ResourcePool.java:1444) at org.geoserver.catalog.CoverageView.buildCoverageInfo(CoverageView.java:387) at org.geoserver.catalog.CoverageView.createCoverageInfo(CoverageView.java:398) at org.geoserver.catalog.CoverageViewTest.buildHeterogeneousResolutionView(CoverageViewTest.java:698) at org.geoserver.catalog.CoverageViewTest.testHeterogeneousViewIntersectionEnvelope(CoverageViewTest.java:458) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

Environment

None

Status

Assignee

Ben Caradoc-Davies

Reporter

Ben Caradoc-Davies

Triage

None

Fix versions

Affects versions

2.14-RC
2.13.1

Components

Priority

Medium