ImageIOFactory read tiff image get error pixel type.

[Hconk]

Description

I use Photoshop/ImageJ open the test image , both get real pixel type of 16bit. But ITK TIFFIO get UCHAR( 8bit ).

tiffinfo :

Fiji:

Steps to Reproduce

#include <itkImageFileReader.h>

int
main(int, char**)
{
  std::string filename = "../tiff_bug_1.tif";
  itk::ImageIOBase::Pointer imageIO = itk::ImageIOFactory::CreateImageIO(
    filename.c_str(), itk::ImageIOFactory::ReadMode);

  if (imageIO.IsNull()) {
    std::cout << "fail create Image io." << std::endl;
    return false;
  }

  imageIO->SetFileName(filename);
  imageIO->ReadImageInformation();

  const auto pixelType = imageIO->GetComponentType();
  std::cout << "image pixel type: " << pixelType << std::endl;
  std::cout << "image pixel type: "
            << imageIO->GetComponentTypeAsString(pixelType) << std::endl;
  std::cout << "image dim: " << imageIO->GetNumberOfDimensions() << std::endl;
  return 0;
}

Expected behavior

Test image real pixel type is unsigned shot(16bit).

Actual behavior

image pixel type: 1
image pixel type: unsigned_char
image dim: 2

Reproducibility

everytime.

Versions

ITK Version: 5.1.0/5.2.0/master latest

Environment

CMake: 3.21.1
Compiler: gcc4.8.5/vs2017

Additional Information

test data 1:
https://drive.google.com/file/d/1O3PCYt6nz1HKuvnZxtbI_U3fox582yAt/view?usp=sharing
sha256: 9201f1ac5888d6c3efbf3055fd6505c84e100383e4f5bb14d7237b1187f77186

test data 2:
https://drive.google.com/file/d/17I1ua6HOtIKV7jQId-10ioJCra6s6i9Z/view?usp=sharing
sha256: b693140a8fdd76765b5091732a0b7db902621b7873588819c783282d9598dd73

[dzenanz]

Confirmed that ITK's IO detects 8-bit image. Loading it in Slicer results in a mostly black image. Slightly updated code which works on my computer:

#include <itkImageFileReader.h>

int
main(int, char**)
{
  std::string filename = "tiff_bug_2.tif";
  itk::ImageIOBase::Pointer imageIO = itk::ImageIOFactory::CreateImageIO(
    filename.c_str(), itk::CommonEnums::IOFileMode::ReadMode);

  if (imageIO.IsNull()) {
    std::cout << "fail create Image io." << std::endl;
    return false;
  }

  imageIO->SetFileName(filename);
  imageIO->ReadImageInformation();

  const auto pixelType = imageIO->GetComponentType();
  std::cout << "image pixel type: " << pixelType << std::endl;
  std::cout << "image pixel type: "
    << imageIO->GetComponentTypeAsString(pixelType) << std::endl;
  std::cout << "image dim: " << imageIO->GetNumberOfDimensions() << std::endl;
  return 0;
}

[issakomi]

const auto pixelType = imageIO->GetComponentType();

My suggestion would be to distinguish better pixel and component type

#include <itkImageFileReader.h>

int
main(int, char**argv)
{
  std::string filename = argv[1];
  itk::ImageIOBase::Pointer imageIO = itk::ImageIOFactory::CreateImageIO(
    filename.c_str(), itk::CommonEnums::IOFileMode::ReadMode);

  if (imageIO.IsNull()) {
    std::cout << "fail create Image io." << std::endl;
    return false;
  }

  imageIO->SetFileName(filename);
  imageIO->ReadImageInformation();

  const auto componentType = imageIO->GetComponentType();
  std::cout << "image component type: " << componentType << std::endl;
  std::cout << "image component type: "
    << imageIO->GetComponentTypeAsString(componentType) << std::endl;

  const auto pixelType = imageIO->GetPixelType();
  std::cout << "image pixel type: " << pixelType << std::endl;
  std::cout << "image pixel type: "
    << imageIO->GetPixelTypeAsString(pixelType) << std::endl;

  std::cout << "image dim: " << imageIO->GetNumberOfDimensions() << std::endl;
  return 0;
}

Output:

image component type: itk::CommonEnums::IOComponent::UCHAR
image component type: unsigned_char
image pixel type: itk::CommonEnums::IOPixel::RGBA
image pixel type: rgba
image dim: 2

RGBA is suspicious, not sure about it. Gimp said scalar, 16-bit. But the images are everywhere mostly black. Not sure.

[Hconk]

RGBA is suspicious, not sure about it. Gimp said scalar, 16-bit. But the images are everywhere mostly black. Not sure.
@issakomi

Test data 1 all pixel is black. test data 2 need adjust the display gray range to show more:

Gray Range: 0 - 54

[Hconk]

I found this issue may cause by image orientation info.
The test data 2 exif info is:

ExifTool Version Number         : 12.30
File Name                       : tiff_bug_2.tif
Directory                       : .
File Size                       : 11 MiB
File Modification Date/Time     : 2021:12:15 13:40:49+08:00
File Access Date/Time           : 2021:12:16 16:36:26+08:00
File Inode Change Date/Time     : 2021:12:15 13:40:49+08:00
File Permissions                : -rw-rw-r--
File Type                       : TIFF
File Type Extension             : tif
MIME Type                       : image/tiff
Exif Byte Order                 : Little-endian (Intel, II)
Image Width                     : 14700
Image Height                    : 11400
Bits Per Sample                 : 16
Compression                     : LZW
Photometric Interpretation      : BlackIsZero
Strip Offsets                   : (Binary data 91179 bytes, use -b option to extract)
Orientation                     : Mirror horizontal and rotate 270 CW
Samples Per Pixel               : 1
Rows Per Strip                  : 1
Strip Byte Counts               : (Binary data 49410 bytes, use -b option to extract)
Planar Configuration            : Chunky
Image Size                      : 14700x11400
Megapixels                      : 167.6

When I use exiftool remove the Orientation tag from exif info, ITK work well.

ITK/Modules/IO/TIFF/src/itkTIFFReaderInternal.cxx

Lines 194 to 207 in c779d09

	TIFFReaderInternal::CanRead()
	{
	const bool compressionSupported = (TIFFIsCODECConfigured(this->m_Compression) == 1);
	return (this->m_Image && (this->m_Width > 0) && (this->m_Height > 0) && (this->m_SamplesPerPixel > 0) &&
	compressionSupported && (m_NumberOfTiles == 0) // just use TIFFReadRGBAImage, an
	// native optimized version would be nice
	&& (this->m_HasValidPhotometricInterpretation) &&
	(this->m_Photometrics == PHOTOMETRIC_RGB || this->m_Photometrics == PHOTOMETRIC_MINISWHITE ||
	this->m_Photometrics == PHOTOMETRIC_MINISBLACK ||
	(this->m_Photometrics == PHOTOMETRIC_PALETTE && this->m_BitsPerSample != 32)) &&
	(this->m_PlanarConfig == PLANARCONFIG_CONTIG || this->m_SamplesPerPixel == 1) &&
	(this->m_Orientation == ORIENTATION_TOPLEFT || this->m_Orientation == ORIENTATION_BOTLEFT) &&
	(this->m_BitsPerSample == 8 || this->m_BitsPerSample == 16 || this->m_BitsPerSample == 32));
	}

L205 check the image Orientation, when image Orientation is not ORIENTATION_TOPLEFT or ORIENTATION_BOTLEFT will return false, cause TIFFImageIO::ReadImageInformation() function into here:

ITK/Modules/IO/TIFF/src/itkTIFFImageIO.cxx

Lines 507 to 513 in c779d09

	if (!m_IsReadAsScalarPlusPalette)
	{
	itkDebugMacro(<< "Using TIFFReadRGBAImage");
	this->SetNumberOfComponents(4);
	this->SetPixelType(IOPixelEnum::RGBA);
	m_ComponentType = IOComponentEnum::UCHAR;
	}

Seem ITK only support read ORIENTATION_TOPLEFT and ORIENTATION_BOTLEFT tiff image? I'm not sure. If so, when use it read other not support Orientation image, need throw exception.
ref:

ITK/Modules/IO/TIFF/src/itkTIFFImageIO.cxx

Lines 1373 to 1376 in c779d09

	if (m_InternalImage->m_Orientation != ORIENTATION_TOPLEFT && m_InternalImage->m_Orientation != ORIENTATION_BOTLEFT)
	{
	itkExceptionMacro(<< "This reader can only do ORIENTATION_TOPLEFT and ORIENTATION_BOTLEFT.");
	}

[issakomi]

Seem ITK only support read ORIENTATION_TOPLEFT and ORIENTATION_BOTLEFT tiff image? I'm not sure.

Yes

    if (m_InternalImage->m_Orientation == ORIENTATION_TOPLEFT)
    {
      image = out + inc * row * width;
    }
    else // bottom left
    {
      image = out + inc * width * (height - (row + 1));
    }

The real problem that the logic is buggy. Even if the TIFFReaderInternal::CanRead() (s. above post) returns false there are attempts to process the image if (!m_InternalImage->CanRead()), the logic path brings in particular case to completely wrong result (RGBA, uchar). The TIFFTAG_EXTRASAMPLES tag (where alpha channel is defined) is not processed at all (on read), so RGBA images are a kind of fallback. In fact it could be possible to give proper exception with 1-2 lines, but proper fix is not so easy.

@Hconk i don't have much experience with TIFF metadata, if you have examples (small files are preferred) of images with unsupported orientations, please share. Thanks.
For reference here is how Gimp handles TIFF orientations:

      if (TIFFGetField (tif, TIFFTAG_ORIENTATION, &orientation))
        {
          gboolean flip_horizontal = FALSE;
          gboolean flip_vertical   = FALSE;

          switch (orientation)
            {
            case ORIENTATION_TOPLEFT:
              break;

            case ORIENTATION_TOPRIGHT:
              flip_horizontal = TRUE;
              break;

            case ORIENTATION_BOTRIGHT:
              flip_horizontal = TRUE;
              flip_vertical   = TRUE;
              break;

            case ORIENTATION_BOTLEFT:
              flip_vertical = TRUE;
              break;

            default:
              g_warning ("Orientation %d not handled yet!", orientation);
              break;
            }

[blowekamp]

There are a great number of options for TIFF file. ITK implemented a generic function to read types and options commonly used with ITK.

If the TIFF features in the file are not supported by the ITK generic function that the TIFF library function TIFFReadRGBAImageOriented, which produces 8-bit RGBA images. This may not be ideal for all input especially with a narrow range of intensities used in the higher bit depth image. This was reasonable "fallback" behavior IMHO.

There have been number changes over the year, to expand capabilities and may have introduced bugs.

It's not clear to me ( sorry I could spend more time and dig deeper ), if there is a genuine bug with the RGBA fallback behavior, or if this is a request to read the TIFF file with the native file pixel type.

[issakomi]

I believe simple fix would be to throw in the if (!m_InternalImage->CanRead()) block if orientation is not supported. I didn't test, just guessing.
Proper fix would be, IMHO, get rid of CanRead(), process TIFFTAG_EXTRASAMPLES to ensure proper alpha (not always 4th component is alpha as expected) or warn, support all orientations. Not easy. BTW, there is also .h file in src folder.

Edit: with "simple" fix several tests failed.

[issakomi]

Looked at test files (failed tests):
There is the same problem, files with unsupported orientation are wrongly identified as RGBA, but they are scalar. Also according to Gimp they are all scalar. The tests don't discover the problem.

---

cthead_oriet_tl.tif

image component type: itk::CommonEnums::IOComponent::UCHAR
image component type: unsigned_char
image pixel type: itk::CommonEnums::IOPixel::SCALAR
image pixel type: scalar
image dim: 2

---

cthead_oriet_tr.tif

image component type: itk::CommonEnums::IOComponent::UCHAR
image component type: unsigned_char
image pixel type: itk::CommonEnums::IOPixel::RGBA
image pixel type: rgba
image dim: 2

---

cthead_oriet_bl.tif

image component type: itk::CommonEnums::IOComponent::UCHAR
image component type: unsigned_char
image pixel type: itk::CommonEnums::IOPixel::SCALAR
image pixel type: scalar
image dim: 2

---

cthead_oriet_br.tif

image component type: itk::CommonEnums::IOComponent::UCHAR
image component type: unsigned_char
image pixel type: itk::CommonEnums::IOPixel::RGBA
image pixel type: rgba
image dim: 2

[blowekamp]

Hello @issakomi,

Thank you for your efforts on this issue.

It sounds like the expectations for behavior are different than what is implemented and may be consider it a bug OR it could be considered a new feature request.

The pixel information reported by TIFFImageIO is not the layout of the pixels is the file but the the layout of the itk::Image which is supported by the ImageIO to load the data. When the TIFF file has option unsupported by TIFFImageIO::ReadGenericImage it fallsback to the libtiff method TIFFReadRGBAImageOriented. So in this sense the information listed is correct in that it is what the ImageIO can produce.

For the above br/tr oriented images there is no pixel value loss or reduction in pixel depth, which is a good thing. Unfortunately, there is a 4x increate in memory size, a bad thing. However, for this case it is preferred that ITK reads the image as oppose to throwing an exception after the TIFFImageIO indicated it could read the image.

Another feature not supported by the ReadGenericImage method are strip and tiles oriented image. I was processing tiff files encoded like this for a bit ( some people may still be using the code ) and relied on the fallback method.

There is a forth coming PR, which adds a warning to at least inform when the bit depth is decreased, like the OP example.

The best solution is to add the new feature which implements adding support the these additional orientation to the ReadGenericImage method.

[issakomi]

I have found another tiff file that doesn't work well with the IO. Gimp and other can open the file better. I guess it is scalar image, but it has "extra samples". I didn't look closer at the image.
Just in case if somebody will work on the IO later.

Wrong

Correct