S3 GeoTiff significantly slower than local storage, due to length determination remote calls

Description

Doing some tests on S3GeoTiffReader, I've found it to be 17 times slower than local file, even when the remote data needed is fully storage in the local cache (28 r/s local, 1.66 r/s remote). Note, this is on a internet connected machine, not on a AWS host, but still...

Looking at what's happening, the culprit seems to be the file length calculation. Every time a GetMap is issued, a S3ImageInputStream is created, which in turn creates a S3 client and connects and fetches the information about file length. That's a pretty costly remote call.

Given caching is already involved, it would be best if the file length information would also be cached (the EHCache survives requests, and if disk storage is on and configured in a single location, survives restarts too). This should allow various requests to just be served fully from locally available information

Environment

None

Status

Assignee

Unassigned

Reporter

Andrea Aime

Triage

None

Components

Priority

Medium
Configure