Which color spaces are useful for histogram matching?

[perrygeo]

Currently I'm using skimage for all but the LCH colorspace (which is implemented directly in rio-color)

Related to mapbox/rio-color#28, we might want to make rio-color a more full-fledged colorspace module.

Let's identify which color spaces are useful for the purposes of histogram matching to guide that decision.

/cc @celoyd @virginiayung

[perrygeo]

HSV

LAB

LCH

some issue with matching Hue as it's an angular measurement

LUV

RGB

XYZ

[virginiayung]

I've ran rio-hist on 21 adjacent scenes so that we can look at how histogram matching works under each color space at a slightly larger scale:

Source (NAIP 2011 UT & 2013 CO):

Reference (2 Landsat 8 Scenetifs):

Cut Landsat 8 tiles:

RGB:

HSV:

LUV:

LAB:

LCH:

XYZ:

Notes:

1. Clouds/Snow: I included ~10% of clouds in the landsat 8 tiles which I used as reference. I wanted to see how each color spaces handled 10% of cloud coverage, which is a realistic amount of clouds without running a more rigorous search for the best reference scenes. It seems that HSV was able to handle clouds in tiles 1X2 and 1X3 (first row, second and third tiles from the left) better than any other color spaces. RGB and XYZ are the two color spaces that took in most of near-255 pixel values. Below is the histogram for tile 1X2 in RGB:
   

2. LCH produces blue/purple artifacts: It seems to appear in high-contrast scenes. In this case, it is most apparent in tile 3x7, which covers the mountain ranges right outside of Dove Creek, CO . Here's a zoomed in view of this area:

   

3. Very subtle difference between LUV and LAB: I had to toggle between to the outputs to really notice the difference between the two. I did notice that LUV took in more red and contrast for tiles 3X5 and 3X6. These original NAIP scene tifs for these two tiles contain a lot of contrast since they include mountain ranges and equivalent Landsat 8 scene tifs contain a bit of red and dark green.

4. Handling redness: HSV seems to pull in the redness more evenly than other color spaces.

   

Would love to get your 👀 on this @celoyd @perrygeo. Any other interesting observation?

[celoyd]

Innnteresting. My impressions:

• Overall, no big positive or negative quality outliers in this test, besides full LCH’s H-skew. All of the others are basically shippable quality – no disqualifications in this round. The variations are small enough that picking favorites is impossible from a single sample.
• Hue artifacts in full LCH are to be expected, for the reason @perrygeo pointed out – H is angular, but this method treats it as an interval. I’m comfortable assuming that that’s where the blue is coming from.

I’d say we’ve seen, as much as possible from one test, that a 10th-ish percentile case (state boundary and some clouds) is fine across most of the colorspaces we’re looking at. That’s good to know.

To me it looks like the next action is to think about the 1st-ish percentile – somewhere that summoned the Blood River last time. Does that seem wise?

Sneakily editing to add: could I see a 100% crop of the “snowy” area along the top toward the left, where the clouds are pinning the whites? Curious how that looks at 1:1.

[virginiayung]

@celoyd: Here's a 100% crop of the tile 1x2 and 1x3:

NAIP:

Landsat 8:

[virginiayung]

I noticed the blood river effect👻 while running histogram matching with Landsat 8 for these 12 adjacent NAIP scenes. This is most likely due to the fact that these scenes contain 2 distinct features (land + water). Histogram matching has no knowledge of these distinct features, but treats the entire scene uniformly.

Source: Lake Havasu City, CA and AZ NAIP

Landsat 8:

RGB:

HSV:

LUV:

LAB:

LCH:

XYZ:

Notes:

Found blood river where the redness of the land bled into Lake Havasu. Check out the bottom corner (tiles 3x3 and 3x4). LUV showed the least sign of redness bleeding into the the water. HSV was a disaster. Here's what their histograms looks like (RGB, LUV, HSV):

Next:

I'm checking on one other blood river candidate, so that we can 👀 and compare blood river scenes that resulted from histogram matching against Landsat 5 & 7 as a reference layer versus those against Landsat 8. This is what Pensacola, FL NAIP tiles look like after histogram matching against cloudless Landsat 5 & 7:

[perrygeo]

@virginiayung interesting!

HSV, with Hue being an discontinuous angular measurement just as in LCH, is probably a no-go. I think we can probably just drop the idea of using HSV or LCH, at least until we have a project with very specific matching requirements (like "match just the lightness of the reference data but retain the hue")

LUV looks promising and it's the only one that isn't yet directly supported by rio color - I'll work on that tomorrow.

I'm concerned that there might be too much red in the reference Landsat images - Maybe it's just my eye but they seem way more red than similar imagery of the same area. How are we processing the landsat 8 data to get it to this point?

I'd also like to test on some different landscapes to see what the color balance looks like - the desert makes it hard to tell for sure since red rocks are not uncommon. But reddish coastal forests, not so much.

[virginiayung]

Source: 12 Pensacola, FL NAIP Scenes

Reference: Pansharpened Landsat 8

RGB:

HSV:

LUV:

LAB:

LCH:

XYZ:

Notes:

In this test suite, there were a lot of artifacts along the shore in these original NAIP scenes. Pansharpened Landsat 8 not only had better color, but also contained less artifacts. RGB and XYZ both did a pretty job suppressing these artifacts from NAIP. However, RGB created new blue ribbons in the deeper water as seen in the lower right tiles. @perrygeo I believe this is similar to blood river observed ^ but in reverse (bleeding from original NAIP). In any case, this is most likely due to the fact that these scenes contain more than one distinct features (land + water) and histogram matching has no knowledge of these distinct features, but treats the entire scene uniformly.

[perrygeo]

I think the answer to this question is going to be dependent on the needs of the project. Closing for now.

[Borda]

I have a use-case from medical - multi-stain histology tissue