The files are scanned TIFFs in the 25-30 MB range so we're thinking 5 seconds might be on the optimistic side.
Yes, definitely planning to parallelize but we're likely limited to four simultaneous instances so that will still take ~20-30 days.
Yes, definitely planning to parallelize but we're likely limited to four simultaneous instances so that will still take ~20-30 days.
Also, converting the (hi-res) images to JPEG before further processing may make the data somewhat easier to deal with. 25-30MB per image * 2 million would be 50-60 terabytes of data? Plus, how often would someone really benefit from having access to a 25-30MB TIFF vs a much smaller JPEG version in the same resolution? That alone could be reason to start with JPEG compression first.